Deejay (00:15.31) Elliot, you've been posting on the CTO craft Slack, which we're both members of about your journey using Gen.ai in the development process. And you set some quite ambitious goals to maybe be doing very little development of code, like programmers tapping on keyboards by the end of 2026. Can you tell us about how that's been going, how it started and yeah, the progress that you've seen. Elliott Beaty (00:35.475) Yeah, I'd be happy to. The AI scene in general is something that I'm, to be honest, fairly new to, right? We've had AI coding assistance for quite a while now. I have been fairly reluctant to embrace it until the end of last year, really end of 2024. I got to start with it. And I started experimenting with it and seeing what the capabilities were and I very quickly realized that we were at an inflection point in the industry where. Affordability was intersecting with a viability, right? So for a while, you know, we use visual studio for much of our development and the, the AI was really just like auto completing your if statements. And, it is. Ask that point where. It's just boiler code or it only does scaffolding it is well into able to think and operate on tasks like at a very high level, the hierarchical tasks. Once I realized that, I realized that we were not in an arms race. Right? So if I don't get my team to this point of being able to use AI for everything, someone else will, my competitors will. And that's when I really, we pivoted hard as a team and really started leaning in on the AI coatings tools. started with cursor and Winstraf, like many people do. We did some trials between the two. Once I kind of had a feel for the trajectory around or the course of two months, I would say I could see, I mean, I could see week over week improvements in these toolings and you know, Basically, I set the goal, like you said, to not have my engineering team be writing their own code by the end of 2026. I think we have to get there. Deejay (02:37.966) How did that go down? That's like quite an audacious goal as goals go. How did the team feel about that? How did the developers respond to that? Elliott Beaty (02:49.395) So, you know, the, business stakeholders, my owner of a company or president, they're all like, yeah, that sounds amazing because it's, you know, sounds free. the team we had, you know, my engineering team, had varying degrees of, you know, excitement and terror. So, you know, the early adopters, the ones that are really aggressive with their, you know, trying new things out, they were all for it. we had some people that, you know, were largely ambivalent to it because they were like I was back in mid early 2024. And then we had some people that were a little nervous that it might replace them. I also with that, I told my team I had no intentions of replacing anyone. This is a productivity tool and not a replacement. And we're fortunate to be in a growth stage. So we're going to. Use AI to offset hiring needs as long as we can. And it's just going to slow that process down. And we're not going the other direction where we're, we're laying engineers off. We're already very lean. and we're, we have like everyone on my team is fantastic. We don't have any fat to cut. they're all high level senior engineers for the most part. Deejay (04:07.474) And is it one big team? How many people are on the team? Or are you split across different kind of value streams? Or is it all one team working on the same product? Elliott Beaty (04:15.475) So we're separated by skill sets right now. It's the first time I've done it this way, but we have a front end team, we have a backend team, we have a mobile team that does Flutter development. they handle both Android and iOS. And then we have a QA team of QA engineers as well. We have about 10 engineers, a couple of managers, and then a team of four on the QA side. One of them is test automation engineer. Elliott Beaty (04:46.203) It works well, would say, especially for, you know, we unintentionally set ourselves up to be structured well for AI development, right? Where we can have one team focusing on making sure front end code quality is really good. The packet team is like specialized in.net code and.net core. And so they can, they have that deep expertise that you need because now you're, you're replacing the more mundane, these are stuff that like a junior or mid-level engineer can do. And so you need that expertise. Deejay (05:17.748) That's really interesting, like thinking about what it means for team structure and things like that, instead of going cross-functional, kind of having more tech aligned teams. So you've got that expertise and it will be interesting to see that kind of pendulum swing from one side to the other in the industry as AI tooling takes more place. Elliott Beaty (05:38.803) Yeah. And you know, where we're, we're, really benefiting from it a lot is we have always thought about our teams as like delivering a service for the other team. Right. So my backend team that are building APIs, we have a microservices architecture. They they're responsible for providing good documentation through like swagger. And so what I'm realizing is the most important thing for all of this AI implementation is really good documentation that AI TrueLink can hook up to. So by already providing like the swagger and having a really good swagger definition file for each API. Now our AI agents are far more capable of hooking up to it, figuring out what to use when, and don't need as much babysitting for that. And so we've actually gone taking a step back and had our engineers take AI agents to another level in documenting. All the API endpoints, right? So now we have agents going through all of our classes and figuring out what properties should be used for what in a response or a payload to an HTTP payload. So we're adding more documentation and really making that very robust because it just helps the AI agents even more. And what used to be a traditionally like a job that most engineers really dread is not dreadful anymore, right? Because you just Deejay (07:02.158) You Elliott Beaty (07:06.245) set Claude or windsurf on it, tell them what to do and then review it afterwards. So it's no reason here. It's painful anymore either. Deejay (07:14.264) Did you find that the I'm going to go nerdy and like the engineer in me is wondering, did you find that explicit documentation that would typically be read by a human like the swagger docs? that was the AI able to interpret that better than like self documenting code? Like if functions and things were very expressively named, was it as as able to interpret those or did it need the explicit kind of human readable documentation. Elliott Beaty (07:45.171) It does interpret those. we know that partly because we have networking on code that doesn't have any of that documentation. It does a pretty admirable job where it has access to the code base. But also it is doing some of this documentation for us. So it is gaining context based on your method names or your variable names. It's looking at the names of DTO classes. You know, in our ORM, it's interrogating that and building its own context for these either properties or methods in the Swagger documentation based on that. And it's doing an admirable job. Deejay (08:30.046) I suppose you can imagine that the the documentation when you when you set the LLM off to read these functions, read these methods, write some documentation about it, then when it needs to reason about that and reasoning in quotes, I'm not going to get into that rabbit hole. Once it's it's kind of like snapshotting a level of understanding. So instead of having to read the code and understand it all in one go, and then make some decisions with it, the documentation is almost like a kind of Elliott Beaty (08:44.285) Yeah. Deejay (08:56.33) snapshot or crystallization of what an LLM understood about that function previously. Elliott Beaty (09:01.427) Yes, yeah, we're using it, you know, our QA teams using it, and I use it once in a while, just to figure out what some code is doing. For example, we're using, we use a platform called Agora for live meetings with our mentors right at work where it's kind of like you can schedule meeting within the app. It's Uber of financial mentors kind of set up and I didn't know what I, you we had a bunch of projects set up in the Agora. platform and I didn't know which ones we were using. And so I could just go in there and say like, Hey, what AP what endpoints and what, you know, app IDs are we using with Agora? That's the only context I gave it, set it on the right repository. And it went and found that for me. And they gave me a beautiful, printout of, you know, with bullet points of all the things I needed to know. And then some stuff that was extra, but useful information. And it was really good at it. It was really impressive. And that was something I would have had to like ask an engineer to step away context, which go find out and then that nobody, nobody enjoys going and digging through a repository for that kind of information. And then when you factor in like, have a secret storage for variables and so you can't always get the actual variable values. It can go do all of that, which is crazy, right? It can actually hook up to the Azure DevOps APIs and go pull values for you if you want. The non-encrypted ones, of course. And it's not only that, but you have complexities with local file, variable files, you know, overriding project level ones, which are overriding, you know, global ones, know, in your environment. So they can navigate all that far more accurately than a human can. Deejay (10:39.01) Yeah, just thinking of engineering projects I've been on in the distant past when I was still competent enough to write code of trying to do that diff in your head between the multiple layers of different properties that apply in different situations. You, I think, was it last week you posted an update in Slack that was a bit of a landmark in terms of progress of how far you've got with some of the code generation. Elliott Beaty (11:03.973) Yeah. I'll tell you what it feels like we're having one of those landmarks every week now. So that was, I mean, the ideal state for like every CTO or VP of engineering is to be able to do these little mundane tasks as like an afterthought. that was the first, last week was the first time where I had a Jira ticket that one of my QA resources created. It was a bug. was valid bug. actually a pretty complex one where we had a race condition with payment confirmations coming in and not, not do doping them properly. And, I, all I did, and this was, this was, I just managed to get Claude code to work with the Jira hosted and, MCP server. And I literally just said, fixed the issue in ticket number one, two, three, and it went and pulled. all the context from the ticket. I had to point it at the right code base. went and, stepped through all the code. It found a race condition between a web hook and a traditional, just, HTTP controller, you know, API that we. Let's talk to with our front end. So it was two completely different parts of the code base that don't have any like logical relationship within the code, right? So it's determined all this based on context. from the documentation and the variable names and the method names and identified it, fixed it, got it to build, wrote unit tests. And the only thing it stumbled on that it needed any babysitting was the one of the unit tests failed when we got to our, you know, we have a CI CD pipeline and then where we rerun all the unit tests and it didn't work in that hosted environment because of how it handled the variable. And I had to copy the error message out of our build pipeline. give it to, in this case it was Claude said, Hey, we got, I literally typed, Hey, when we, when I ran this in our CI CD pipeline, this error message was produced and it went and found the error and the unit test fixed it. I push it up, you know, right now we have it so it can't actually merge itself. and. Resulted on its own. took, it would not only did we avoid all like that, the handoff and the planning and, Elliott Beaty (13:30.355) you know, figuring out who can work on it. it took probably a two day task down to like 30 minutes. Just while I was doing other stuff. It's crazy. Deejay (13:40.044) Yeah. And the, the, solving that problem that it's like apparently disconnected that, you know, it, it, it's not obvious that, you know, if there's sort of a code reference and there's, you know, you're using.net, you? So it's kind of like, well, I mean, in the back end, so it will be strongly typed. could compile. could figure that out, but if it's, you know, separated, then that's pretty impressive. And one of the other things you mentioned there was, the absence of all of the overhead, not just actually doing the programming task, but putting it in the backlog, figuring out who's going to do it, estimating it, all of those sorts of things. It's, know, a nonlinear kind of time saving. Elliott Beaty (14:20.113) Yeah, and you know, especially for us, we run so lean that, you know, if I don't, if I have, have three backend engineers that are, you know, pedal to the metal every day. And if I have something like a payment issue come through that, you know, we consider that like, you know, a tier zero issue, right? If anything that touches revenue, we drop stuff for to work on it. I didn't have to do that, right? I didn't have to pull one of my, you know, very valuable resources off of what they were currently working on to solve this. I was able to just throw AI at it and fix it, which is super valuable to me because it gives me a lot more options when we have a fire drill. It was 48-hour turnaround time by the time we got it through QA and released it. Deejay (15:00.536) And so, sorry. Yeah, that's impressive. Are you using AI systemically in your development process or is it on a case by case basis, somebody opens up a Claude code and goes, right, this problem I'm going to try solving with AI or are there things that they're waiting for tickets to appear and then automatically reacting to them? Elliott Beaty (15:27.613) So they are not waiting for tickets to appear yet. know, that use case I just described isn't always the case. Sometimes it just fails still, right? And it doesn't solve it. I had it work on a feature. Similarly, I was like, I don't have anyone to throw on this, but we really need this because when other teams working on it, was basically email verification for updating your email. It was all broken from a holdover system we inherited and it, it, did it. It built a pretty good product, but it didn't really fit within our current architecture and the way we were doing email verification on signup. didn't, missed some reusability stuff. And when I had one of my senior engineers look at it, we were kind of realized like, this is good. We can work with it, but we do need to tell it and give it some fundamental changes. And so we can't trust that enough yet for it to just like a ticket comes in, it operates on it and You know, starts working on it our inflow of tickets is nonstop. And so I couldn't also don't have the resources for the code reviews at the tail end. don't have the QA resources to keep up if Claude was constantly just banging out tickets for us. So right now where we're at is I'm trying to encourage adoption across the whole team. And I have one or two guys that are really focusing heavy on. setting up tooling for everyone else, setting up the documentation for everyone else. My goal is that by the end of this year, we'll start sharing that across all the teams. We're going to hit that. We're going to, we're very close to it already, but I, there's so much noise out there right now in the dev space for AI tooling that I'm trying to figure out what the best tools are before I just have everyone do it. So like everyone on my team, all the engineers that are writing code have access to Windsurf right now. they have cursor in some cases, like win search pattern. And there's no pressure to use it yet. There's going to be pressure to start using it by the end of this year. And I'm socializing this idea because a lot of this is also just creating space for them to feel comfortable about it, right? And learn it. And so that's on me. What we're going to do in August is every... Elliott Beaty (17:47.843) My team doesn't know this yet, so here they're going to find out. Every Thursday is going to be AI Thursday for the month of August where they don't work on code or projects and they work on learning something about AI. They can try and solve a problem or like a feature or whatever with AI, but the intent is to create space for them to actually like start figuring out what works on their teams, what works for them, how to hook up to our documentation because we have like tons of markdown files that are being created to. provide context in the right places for FOD. And then Friday mornings, we're going to get back together and talk through it, right? And say like, hey, what did you learn? What did you do? And start doing some knowledge sharing. We're just going to do it for a month. And then we'll see if that's enough, see if people are comfortable, see where our gaps are. So no, right now I have one engineer who's basically running a bot swarm. He has tons of Claude agents that are just running, we have them in containers and they're, you know, we can scale them out. the biggest problem right now is getting them to talk to each other successfully. and then he's also like my utility player when we have like a performance issue. So he's performance tuning one and API, what Claude's building a whole new feature for them. and I'm just trying to keep him as kind of a floater where he, he's still contributing a lot, but he's working on a pie in the sky feature that we've wanted for a long time that we just can't fit on our roadmap. Right. So it's a brand new ground up. internal portal for support that he's building. it's pretty wild, to be honest with you, what he's done so far. He's done a couple of months worth of work in the last week, building a fully baked admin portal with authentication, authorization, talking to our APIs, modifying client records. It's pretty crazy. But he's leading the charge, knowing that we're going to hook up to his, whatever his output is eventually on across the whole team. Deejay (19:43.704) Cool. I'm going to have to remember to come back to that later because the whole kind of swarm of Claude code instances are really interesting. Find out more about how that works specifically. You mentioned some of your team are using Windsurf, some are using Cursor. We've mentioned Claude code as well. It seems like a really great way of experimenting with something that's new. There's a, you know, there's so much noise, as you mentioned, that this is such a fast rate of change. We don't want to necessarily insist that everybody start using this tool across the board. That would be premature. So it sounds like a quite sensible way that you're adapting to this change. Earlier on, you mentioned that over the year, the Elliott Beaty (20:16.796) Thanks. Deejay (20:29.932) results were improving. Was that because the tools are improving with no input from you or is it about you knowing how to use them better and prompt them better or is it a combination of the two? Elliott Beaty (20:41.427) It's a combination of that and us investing the time to build resources for the tooling. So part of it is just, you do the same task with the same tool a week apart and you're already getting better results. I remember one day I was using, this was with a Lenssurf. I was trying to have it build a, basically an API, right? Or talk to an API. That's what it was. It was talking to, I think it was open AI actually, we were integrating with their APIs and it was failing. with some network request error and it failed and it couldn't do it. The next week I came back and looked at it and it, Windsurf actually stepped out of the project, went to my operating system, started using Wget to make these, to hand roll its own HTTP requests and, or sorry, curl, hand roll its own HTTP requests to troubleshoot. the payloads and the responses manually and then step back into like the code base essentially, and implemented the you know, updated its models to get that right. That was just one week apart. We saw windsurf do that. That's when I was like, my god, this like we're on we're on the hockey stick curve right now. Right? This is going up. So partly that Partly learning curve. So as an engineer, there's a very steep learning curve. And we know we have found that you need to give it very good step-by-step directions and have it revisit those regularly throughout your coding. And then also having resources that it can lean on. every one of our microservices, have an MD file for, markdown file. And with Claude code specifically, whatever directory it's working in, it will first look for a Claude.md file for context. You know, in our project structures, we have like the solution, then multiple projects within it. And then maybe a repository folder and a service folder. And each one of those now is getting a Claude and Markdown file that gives context to what's in that folder. Right. And so now, you know, when it steps into like a, we have a CQRS system that we predominantly use and, you know, it, when it steps into the command handler folder, it, it has a folder that tells it like, Hey, we. Elliott Beaty (22:55.111) Here at fruition, we, we use command handlers for executing requests and we do our entity framework interactions with it directly within the command handler. We don't use services, even though you may see some services that's that's whole over legacy code. no longer use. and it has that context. And so we, there's less of this like back and forth, like, don't build it that way. Do all this over, but with a different architecture. That's a lot of what we were doing early on and now we don't do nearly as much of that. Deejay (23:24.14) It's fascinating to kind of realize that the Gen AI advancements mean that basically we're just going to write better documentation. Like if you were a new engineer coming into one of your projects, that's all stuff that maybe the other engineers might have told you, or you might have picked up by osmosis. But now it's like explicitly written down. This is our architectural style. If you see this, don't pay any attention to it. It's kind of great that all of that implicit knowledge in the head is becoming knowledge in the world that is expressed. Elliott Beaty (23:46.515) No, and I've, I'll be honest, I've traditionally been fairly reluctant to invest in documentation because like every software I've ever worked on is evolving and a living document, right? And unless you're in a really large company where you can afford to have resources dedicated to it, it's Deejay (23:53.927) And that's not one of the outcomes I would have guessed. Elliott Beaty (24:16.551) from a business value, it's like an 80-20 rule where 20 % of the effort gets you 80 % of the value with documentation, right? And so I haven't been one to force engineers to write good documentation and now all of a sudden it's crucial, right? It's absolutely important, but we also have the tooling to make it really easy and does a far better job than even a documentation specialist can do sometimes because it has both the technical acumen and the written skills. Deejay (24:50.626) The swarm of bots. So I've used Claude code. I've played with it on the command line. How does that work having multiple instances of it each running in containers and how do those collaborate? Elliott Beaty (25:03.475) We're still figuring that out for sure. And a lot of stuff hasn't worked. So we do have containers that spin up, have it all work. My goal for that project was to basically be able to have hosted service out there that we could scale up as we needed more, like, you know, out in AWS. And, you know, side benefit of that was going to be not having to buy expensive hardware for my engineers, right? All of a sudden you could just have, you know, a deep Mac book and connect through SSH and run cloud. That was like my pie in the sky idea. It hasn't worked out that way. A lot of, we're running into two problems. One is the communication between the agents. So they, we don't want to invest too much time in building some sort of architecture where you have like a and an architect and a, you know, a QA analyst, because I know for a fact that somebody is going to build that for us eventually, right? And Claude is. Claude has come up with that recently in a beta, and it doesn't work. So we haven't been able to implement it yet. But we're just reluctant to spend too much time on something we know Claude is going to deliver, or someone else will deliver. So part of it is the communication between them. And when we did run our own tests, we found that one bot took over. Right. we had like one test we ran, had like five agents trying to talk to each other with different roles. And eventually the architect was just like, this isn't working. I'm going to do it all. And the architect agents just built everything. So that's, we're just not there yet. With the other pieces, we need to, we need a good way to share documentation. And that's the nut we're trying to crack right now so that they're all working on the same context at the same time, or at least in, you know, harmonious ways. And so my original thought for that was actually basically like a rag model where we have like a, you know, a shared Postgres instance that they could go to and get context from. And what we're actually finding is that Markdown files are a easier way to one for us as humans to see, we know what the context is. And two, the it's like all the Elliott Beaty (27:25.127) the agents work with it pretty natively as it seems to be coming to the standard, which is kind of wild, right? Cause the industry went from like XML to JSON, and now we're going to markdown for a lot of this kind of information you feed, you know, bots. And so what we have, what we just started doing, one of my engineers built a docu source, basically like internal website, which is all markdown driven. And we're going to host that. And then that has very nice folder structures that are easy for an agent to navigate for different topics, architecture, best practices. Eventually, we'll have a central one for all the different microservices we have so that you can get links to the swagger documentations and it's all easily discovered by an agent. So that's the bit that we're still figuring out, but we have a pretty good traction with is that sharing the knowledge between the agents. Deejay (28:21.752) cool. The gave the example earlier of a bug being, you know, almost entirely fixed by by core code. I think it was all by itself. The types of problem that you think this is going to be good for solving. Have you seen any evidence yet that it will be useful for defining system boundaries? The kind of impression I'm getting talking to people and looking at things is that given an API or an interface or some existing code kind of filling in the gaps is where the state of the art is right now. It's quite happy doing that, but maybe defining an API, defining, you know, the responsibilities of microservice, that kind of stuff is still very much Like you need to have an architectural knowledge of the whole system and the overall goals. Is that fair? Elliott Beaty (29:09.203) Yes, I would say that's, that's fair and accurate. Right now AI is largely monkey see monkey do. Right. It looks at what you've done in your code base and does a pretty good job of imitating it. Right. That's at the end of the day, that's what AI is. It's pattern matching. It's not at the point where it can solve a complex architectural problem. not even close. would say. However, for stuff where there's just kind of like an industry best practice already. It is solving it, right? So, if we go back to that example where we had the email verification, it, it decided what endpoints and APIs to build and it defined the architecture itself. it left a code comment saying like, Hey, you need to plum this bit into your event bus. Cause we didn't have context for the, you know, how our microservices talk to each other, which is event driven. And, it, knew enough to say, leave some comments saying like, we need to send, you know, a notification on the event bus to trigger this email. so it, you know, it, did a pretty admirable job and it was certainly as good as like a mid-level engineer could have done. but now if you want to start talking about like asynchronous document loading, that's driven off of a queue and some of those more advanced, subjects that aren't your traditional, you know, crud application problems to solve. It's not there. No, I won't do that yet, or it'll do it. And, and you're just going to roll the dice. Maybe it'll be. good, maybe it'll be a terrible implementation. Deejay (30:40.706) That's actually touches on an interesting issue or question, or I think it's an interesting question, because I'm going to ask it. The the bug that got solved was kind of it was indirect, there wasn't a direct code link with architectural styles. You mentioned kind of event driven systems there and you know, being connected up via an event bus. Is it harder in your experience for a gen AI to figure out what's going on in white decoupled systems? Or is it smart enough to figure that out? Is it better to have things more tightly coupled so that the AI can figure out what's going on? Elliott Beaty (31:19.667) You don't need to know. Is it better? Yeah, especially if you're in a strongly typed language where you can, know, it's classes are easily discoverable and, you know, interface implementations are easily found. But we're not like that. We're very microservices driven. have a lot of libraries that are not part of the same code base, and it can do it pretty well. But what you have to look out for is it it's going to think it can do it no matter what. So if you don't provide it like the repository with the, you know, the other, the other chunk of code that's, you know, not part of its context, currently, it's going to assume some other erroneous make some other erroneous assumption that this is how it works, right? It, there's still this very optimistic, I can do anything. If you ask me to, you know, AI agent mentality, mentality, quote unquote. And so that's where that's where you run into problems. And we see that quite a bit, right? Like we give it a task where we don't give it all the information it needs to solve the task, and it still finds a way to solve it, or at least it thinks it's solved it. And that that may mean it goes and creates like 10 APIs, if you're not careful, right? Like it goes and builds a whole system because it doesn't think it exists. So that's, in fact, that's been one of our biggest challenges is it building more than we asked for. Right. So I told you, I have this one engineer that's working on rebuilding our internal admin, we call it our admin portal where we as fruition employees can modify customer data or troubleshoot stuff. And it built things that like profiles for the admins, we don't care about that, right? I don't need a, you know, a profile picture for one of my 30 employees here at fruition. All this little ancillary stuff that is traditionally part of maybe that system or it sees examples of that on the internet or whatever, I don't know where it got the ideas from, it wasn't implemented. We didn't want any of that. Now we're actually spending more effort telling it to remove that stuff. Or we give it requirements. One example is we gave it a requirement to be able to have late-to-leak credit operations for a client organization of ours. One way we sell it is through HR departments, essentially. Elliott Beaty (33:47.661) And it built all sorts of features. You know, it built a wizard and all we really needed was to be able to enter the organization name, contact information, and we were done, but then it, built, you know, three more steps in the wizard that we didn't ask for. So again, we, weren't good enough with our requirements and we, we were just, we just let it run wild and I built more than they asked for. Good problem to have. Right. Some of the stuff we're like, yeah, that's actually pretty good. We'll keep that. Some of the some of the stuff you can just get sucked on the rabbit hole if you're not careful. We just throw it away. Deejay (34:23.15) I love the idea that we might need to fine tune these coding LLMs to be more insecure, more nervous and less gung-ho. Elliott Beaty (34:29.555) Yeah, I haven't seen any like, raw, wild hallucinations, you know, that that's what a lot of people talk about. That's one of the buzzwords here on LinkedIn, right? I haven't seen any of that, to be honest with you. It when it does go kind of wild and outside of what you were expecting it to do. It's because it's trying to solve the problem and the only way it can figure out how and it's not always bad. Sometimes it's actually pretty good and it works. But a lot of times it's just extra noise. Deejay (35:04.002) the, where do you see this going? If the current trajectory holds and it continues getting better at the rates that it has done and your team become more comfortable and continue to achieve success with this, where do you think that, where, if the team are not going to be writing all the code themselves, what's the kind of end state that you see or not the end state, but the, sort of the next big milestone, how do you see their roles evolving and the way that the team operates evolving? Elliott Beaty (35:35.111) That's a good question. Part of our challenges, we're trying to do all this while we're growing as a company. And what I'm actually seeing with my engineers is that they we have less, they have less time than more, right? This isn't creating space for them to work less hours in the day. It's, it's creating more channels that they can work on simultaneously. And so what happens is this development process is quicker, but all the other housekeeping stuff you do to actually write good production quality code that you can take live like code reviews and architectural meetings and QA and back and forth with QA and talking to your product team, that stuff is stacking up, right? We're doing a lot more of that now because the coding bit goes faster. And so, you know, there's, there's definitely going to be a need and probably a, you know, scarcity of senior level engineers in the short term, you know, the next decade, I would, I would guess, because we are able to replace right now, junior and mid level engineers. And so the value of a senior engineer is going up, because they have no longer is their job also to produce code, it's really to ensure quality. So, you know, on my team, we'll be hiring only senior engineers, probably moving forward. And their job is going to be largely understanding systems, understanding the nuances of like, how our microservices, microservices interact with each other. And then supporting the agents, right? So using building tooling to help with documentation and the markdown files or solving the problems that the AI can't, right? Making sure that the agents aren't building solutions to problems we don't have. We've seen that quite a few times where like, it goes and builds all sorts of features for like logging and troubleshooting and graphing error rates and quite literally hooking into providers that we have never used before. Elliott Beaty (37:57.703) right, installing the new get packages and wiring up. Cause it's a good idea. And it's like, actually that was a pretty good idea, but we don't need that. so, you know, controlling the bounds of the project. but it's going to be a lot less coding for sure. and what I've been struggling with personally, and also seeing on my teams now manifest a little bit is this multitasking overload, right? Where people are like context switching is the norm because They're working on something themselves and they have two bots agents going and writing code and the agent gets stuck on something or they have to ask you a question or needs approval to run a command on your system and you have to switch back. Right. And so for me personally, I struggle with that a lot. My, my mental fatigue is really strong at the end of the day now compared to, you know, what I wasn't doing as much. think it's because of that context switching. but that's where I see it going now. What I don't know is we're going to have this. drop off in people entering the engineering workforce, right, because AI is replacing them. And this, that we're gonna have a gap because the senior folks will move on, they'll retire, you know, I don't know what, but they won't be in the workforce anymore. There'll be this gap where we know we need people moving into the senior roles, but we don't have them to do that, right? Because the kids coming out of college didn't have to learn these important skills like architecture and, you know, strong entity relationship design for your database structures and these fundamentals that we used, know, my generation spent many classes and courses covering at university. I don't know that that's gonna be necessary. And so I don't know how we're gonna fill this gap, you know, a decade from now. Deejay (39:41.95) I think that's a really good point. it's, you know, maybe the alarm bell needs ringing for academia and education, because I can entirely see that problem in architecture for physical buildings. is certainly in the UK, there is no such thing as a junior architect. Legally, you have to have done a bachelor's degree and then a master's degree, which is another couple of years after your first university degree, to be able to call yourself an architect. I wonder if in the future, programmers, know, the kind of active coding will be more like, I don't know, bricklaying or something like that. And the Yeah, and like the entry level to, to the software industry will be more knowledge about things like CQRS, event sourcing, CRDTs, you know, all the all this kind of bigger picture stuff that, you know, certainly was Elliott Beaty (40:18.223) area of an apprenticeship. Deejay (40:37.087) new to me when I entered the industry. like, great, I can just about program in Java, I can get a job. And all the other stuff was like arcane knowledge that was passed down to me by principal engineers. Elliott Beaty (40:46.771) Yeah, I have the same generation where man if you had a pulse and a keyboard, you could get a job as an engineer here in the US right when we were coming out of college. And it made a lot of space for a lot of mediocre people or people that weren't really interested in tech but they'd like the salaries that came with it and they could make it they could carve out a good living. That that stuff is those those people those engineers are as good as replaced right now today. And it's going to put an emphasis on the really high value ones, right? That top 1 % that are really intelligent are staying up to date with new technologies and new design patterns. And yeah, we're going to fill that gap. It's going to be an issue. I can tell you right now, I wouldn't hire anyone that didn't have AI tooling experience. If I sat down with an engineer for a new position and they're like, don't use any tooling, would probably be the end of interview. that'd be so important. Moving forward. And I think that's what universities are going to have to focus on is, you know, teaching their students how to use these tools, teaching them the stuff that AI can't do natively right now, or, you know, effectively right now. And it's going to probably be a lot more theoretical stuff, right? Less less about, you know, the nuances of a particular language. I'm all of a sudden a Python developer. I never used to be a Python developer, but right. Like, yeah, I was really good at it. And they build stuff really quickly. So I'm learning it by osmosis. Yeah, I don't think you're gonna have to know how to troubleshoot your own code in a decade. Probably two years. Deejay (42:20.574) Absolutely. Like Python, I was playing around with cursor a couple of months ago. And it's like, I've never really done Python, just about read it, but you know, not write it. And I I got to write a for loop I've written, you know, God knows how many for loops and God knows how many languages and I was like, just just do it for me. And I was like, Yay, I don't need to don't need to know all the tedious ins and outs and the idiosyncratic differences between languages anymore. So that was thoroughly pleasant. The Elliott Beaty (42:40.061) Yeah. Deejay (42:49.762) The changes that you've made, have they started to have any impact on the rest of the business? Do the like product folks and the people that are inputting requirements into your teams, are they aware of any of this yet? Elliott Beaty (43:01.243) They have. Yes, they feel it. Our output as of this month is about 250 % greater than it was in October of last year. So we're, measuring that by number of tickets going through our QA team and passing and story points really. However, that number is skewed a little bit because in that time our Deejay (43:14.509) well. Elliott Beaty (43:29.747) tasks that our features have been wildly more complex as well. Though it's not apples to apples, we've bitten off some really huge challenging features that are very, very intricate and nuanced in that time as well. the productivity gains are greater than that and we're just getting started. So the product team is definitely struggling to keep up because of this new phone velocity. Who's feeling it the most? Deejay (43:54.999) You Elliott Beaty (43:58.417) I would say though, is our QA team. The tooling for QA resources is not there. Even our automation engineer isn't getting the gains out of AI tooling that are comparable to what the engineers are seeing. QA inherently, our testers, they have a skill set that AI doesn't have right now, which is that critical thinking bit. There's still a lot that AI can do and we're trying to figure out how we can help them. but right now the answer is to hire more on that team because the tooling is not there. We've done deep dives into tooling and a lot of it on the QA side at least is around, like building auto test automations. already have a guy that specializes in that. That's amazing at it and does a better job than AI can. what we've seen when we've tested some of that tooling is that it's really meant for people who don't haven't invested in a quality QA team. And just need like a product manager to write some test cases and try and get some automation out there. Right. It's not the super intricate stuff like we have where it's, you know, they, creates a new account, does a bunch of tests after it logs in and then deletes the account and cleans up after itself. Right. Like that's AI is not able to do that kind of critical thinking yet. at least we haven't seen it and the tooling is not there. So, yeah, we're, we're bottlenecking at our QA team. Our product team is just. barely keeping up now. and it's getting, it's getting worse, right? Like as we invest more in the AI tooling, the velocity is going up and it is creating challenges. Deejay (45:32.878) That's fascinating here. it's kind of, you know, theory of constraints, you speed up one part of the value stream, and then all of a sudden, you see where the bottlenecks are in other parts. Is the rest of the organization kind of aware of this? And do they have good ideas of what they can do about it? Or is it at the moment, sort of not entirely conscious that, okay, we might have to make some changes elsewhere in our process? Elliott Beaty (46:00.965) I it's a spectrum for sure. know, the QA team is absolutely trying hard to, they see the problem, they're trying to find a way to solve it. The product team, similar. You know, they were already really quite efficient and are also constrained by some of the tooling available, but things like Figma and Linear are getting their own MCP servers and they're getting some tooling there. So that's getting better, but. still lagging behind. again, much of what they do is the creative half of the brain, right? Which is where AI struggles. And so it's just not as good as code, which is fairly, you know, black and white. The marketing team, we use it heavily. So we have a growth team that handles marketing and sales and they're integrating everything they can. There's actually a lot, think marketers are just inherently very process and automation driven. And so there's actually, there's already quite a bit of stuff there that's helping them. And a lot of content, which Claude's very good at. So yes, we're growing. I have the luxury of having an engineer that I can just throw at it like every day. Right. And, it's pretty easy to justify spending that time on it because we are getting huge ROI on that investment. These other areas, it's harder to do because the ROI just isn't as strong. Deejay (47:28.014) And thinking about the larger organization and what you're mentioning about marketing, was the drive for Gen.ai in the development process, was that something that you took upon yourself? Was there a kind of board level edict coming from on high? Like we need to be an AI first enterprise. Was it just inside dev? Is it everywhere? Or is it kind of happening organically from the bottom up? Elliott Beaty (47:50.973) So everyone's been doing a little bit of it just on their own, Because even just having a subscription to chat GPT for 20 bucks a month is super valuable, even in your just day to day, right? So everyone's kind of, not everyone, most people are already kind of adopting it organically that way, but it was my decision to drive us towards fully AI. And then I went and dangled the shiny object, right? Like we did it and they were like, holy shit. look at what we just did. And then we showed that to other people. And now, now everyone's a believer. So and you know, I mentioned, so we have a really strong relationship with Merit Financial, which is a huge billion dollar investment firm. And they're seeing some of this output too. And you know, now everyone's leaning in, right? Like, like I told you that the financial industry is inherently kind of slow to adapt. And that's not the case right now. I think everyone is seeing that this is the future. We just don't know what it looks like. So, but yeah, I realized that we this was an inflection point for my team. And it was either sink or swim, we were going to figure this out because it is the future, there's no denying it. Or we were going to get beat. So we just did it. Not really a choice. Deejay (49:08.718) It's surprising to me the number of developers that I know who are still somewhat AI skeptic or, know, kind of it's happening somewhere else. I don't need to get too involved in it. I'm just going to carry on doing what I do because it's not going to replace me. And I'm not saying that it necessarily will, especially for mid to senior people, but it's definitely going to have an impact and you should probably be preparing for that. Elliott Beaty (49:33.381) It is and it'll, I think it may not replace senior engineers right now, but you're going to, you're going to get squeezed out by an engineer that does embrace AI if you're not right. And what I have found is if you have to have a champion within your dev ranks that can, that can build and dangle the shiny objects. Right. And that's where I've really gained a lot of traction getting people to buy in is we'll have this engineer go and he just built something overnight. Right. Like This admin portal would have been months of building, he'd built it in a week. And it's amazing and it solves a huge business need because the old one was just terrible. And that's where we started to get Believer's and after he does a demo and then another engineer, she says, hey, can I get a cloud license, please? That was like, they're feeling the pressure because their peers are becoming so productive. So, and like I said, it is up to me to some extent to give them space. to learn this on the clock, right? I don't expect anyone to go home and do work or stuff for work outside of their working day. And so I need to provide space. And I haven't done that for everyone yet. I need to. Deejay (50:40.428) Yeah, the the upcoming as yet surprise the team AI Thursday sounds like a great way of doing that. And I really like the idea of even as a senior developer, you're not going to get replaced by AI, you're to get replaced by a senior developer who is embracing AI. I think that's a really good way of putting it. Elliott Beaty (50:58.599) Yeah, exactly. know, another thing I've noticed, you know, I've been in leadership now for a number of years. And oftentimes, things like paired programming, or code camps, or hackathons are like, we want to do them because we want to learn new stuff, or we want to, like, experiment with technologies. But it's really hard to sell the benefits of that to other stakeholders in their organization, right? Your your leadership or explaining to the product team why you're taking a week off to do a hackathon, because the ROI hasn't traditionally been there, right? Like you build games or whatever during a hackathon. With AI, we see an immediate return on investment. Just spending a day to get one engineer back up to speed is going to probably double or triple their output that same week, right? Once they get a taste of it and start using it. And so it's very, very, very easy to sell this to the business users. Additionally, like with like cloths expensive is 200 bucks a month for one engineer, right? That's dirt cheap, because I'm essentially tripling his output, right? So with a really high value resource, especially for you, have US engineers where you're paying them a couple hundred thousand bucks a year. The 200 bucks a month is a drop in the bucket, but even with, you know, offshore resources that are making anywhere from like six to $10,000 a month, $200. a month is so justifiable. so easy to invest in that tool and you get an immediate return on it. Windsurf is like 60 bucks a month or 70 bucks a month, I forget. Deejay (52:32.801) Yeah, that's Yeah, and given the kind of productivity gains you're saying, I did hear that right 250 % in terms of team velocity improvement there or thereabouts given the changing complexity. Nice. Elliott Beaty (52:45.489) Yes. Yep. Just in story points, making its way through our QA team. And that was over the course of the last, guess, six, several months. Deejay (52:57.418) Awesome. We are getting close to an hour. And I should probably let you get on with your day and my ears beginning to hurt from these, you know, studio headphones that I'm wearing, which of course the people listening to the podcast won't be able to see but trust me, they are big fat, over ear headphones. Elliott Beaty (53:06.066) Ha ha. Deejay (53:24.482) What kind of advice would you give to people to help them adopt this new technology in the most straightforward way with the greatest chance of success? Elliott Beaty (53:37.563) I would say the, part of the part of this is also pitching it to your business, right? And making everyone, all the other stakeholders feel good about you dabbling with this and tools like windsurf cursor, they're cheap. They have an ID, right? So you are actually seeing the files change while the AI agent's doing it. It's a very easy way to step into it and it's low cost. So if you give everyone a license or even just a handful of engineers, a license to windsurf, it's, it's pretty affordable. You need to what I have found is you need even with your own engineers, you need to slow feed this right and let because it's so deep, you can't just jump right into the middle of ocean, you got to start at the shoreline and start swimming out slowly. And so you want to start small, have one engineer, do it, figure it, figure out then bring another engineer and then figure out what tooling works for you. And then you start investing in the more expensive tooling. and dangling shiny objects. then, you know, I think where I made a mistake early on was this just over-communicating this big audacious goal I had of basically replacing people at keyboards coding, right? And instead, I've had much more success in like, hey, Alan, go check this tool out, see what you can do with it, see what it does for us, and then report back. And that's been far easier. to gain support and adoption for. Deejay (55:07.15) Oh, right. Well, Elliot, it's been an absolute pleasure speaking to you. It's really good to hear from someone who is actually doing this stuff and rolling it out in their organization and seeing great results from it instead of just crowing on LinkedIn about how, you know, I was going to change the world. You wouldn't catch me doing that. Of course not. So yeah, it's been an absolute pleasure. And maybe we in a six months or something we can catch up and it would be great to hear about the progress that you've made. Elliott Beaty (55:33.058) That'd be great. Yeah, I'm taking notes along the way, I'd love that. Deejay (55:37.358)