Name: Beyond Watermelon Metrics: The Actual Costs of Agentic AI and Paths to Productivity
Uploaded: 2026-10-14T17:01:32.473Z
Duration: 45 min 8 s
Description: Beyond Watermelon Metrics: The Actual Costs of Agentic AI and Paths to Productivity

Transcript for "Beyond Watermelon Metrics: The Actual Costs of Agentic AI and Paths to Productivity": Welcome to blueprint plus plus, a series where we cover topics at the intersection of business and technology. My guest today, Mary Williams, is the former CTO of Pleo, Monzo, Mu, and Marks and Spencer's, technology advisor and mentor at many other companies, and also the chair of the lead dev conference series. So you're very welcome, Mary. Thank you for having me. Thanks for joining us. As far as I understand, your last, full time gig was at Pleo where your team achieved some very impressive results, 400% throughput over their twenty twenty bay... Two base and also having cycle time. So can you tell me a bit about the changes that produced those results? Sure. So we, changed a a number of things, which from an experimental point of view wasn't ideal because it was quite hard to know exactly which change caused, which results. But we pursued architectural change. So we went went to an API first architectural strategy. We did a ton of upskilling, a lot of it with a company called SkillAway, who do sort of in person deep coaching, with with engineers. We did have some, of that improvement related to to AI, but it was, as you said, 2022 to 2025. So a little before the major improvements that we've seen since November. And, and a whole bunch of it was changes to ways of working as well. So just getting different definitions of done, getting product and design and data and, engineering to work really closely together. We've seen that all of those things have a huge impact as well on the effectiveness of agents. So can you tell us a bit more about, like, what you did on? So you went to API first on the architecture. Were there any other major changes there and maybe things which would now make, the architecture and the code bases more agent ready? Yes. Absolutely. So, I mean, the other thing was a was a very, classic move from monoliths to microservices. And I think that that combination of strong architectural plat... Patterns and intent and sort of an architectural north star combined with good golden paths and sensible defaults is exactly what's needed for for agents. Because I think if you don't set out what your architectural vision is and you don't have go... Golden paths or or good, sensible defaults, you can end up with a sort of mosaic of individually sensible technical decisions made by made by the agents and made by engineers, but that don't add up to any patterns that make it very hard to understand that, on a specific part of the product or or or at at a code level why something was done the way that it's written. 100%. And, actually, sometimes, monorepos can be easier for agents to navigate because it's just one code base. It can get more difficult, actually, when you split into all of these different services. You talked about upscaling, which is huge topic overall. It's not just AI specific, but, what what were the most impactful, you know, tactics that you used at Cleo that you think contributed the most to some of these results? I think, having training that people actually enjoyed, one of which sounds so so simple, but, but I think we have so much death by PowerPoint webinar training in our industry that, that is so... Not even just unengaging but disengaging. I think that was the the major difference was taking this sort of very different approach to, to training and development. Having people have prebook time in their calendar when they were definitely going to be, doing training was was super helpful. And we didn't limit it to technical skills. So we we also had, like, tech lead training, which was a sort of fourteen week, set of sessions teaching some soft skills, some of the business understanding, commercial skills, all of those things as well. How do you think some of those skills translate to managing agents and agentic development? Because it's kind of a halfway house between developers and total control over the computer, you could say. Yes. Absolutely. And I I I actually think it's a little sad that some of the things that are improvements for agents, we're now willing to do, but they would have been improvements for humans if we've done them before.... Representation. It's all classical development productivity stuff, really. Yeah. Yeah. So, I mean, I I think, I think the two big surprises of this sort of agentic era have been that people are now willing to invest in documentation in a way that they never would have done before, like, was always on the to do list and never in the reality list. And then the second is I think a lot of people assumed we would become more focused on product engineering, in this AI era. And, actually, what's proving out is the platform engineering is is probably more important now than it was when all of your engineers were were humans. Definitely. And so I think we're seeing this sort of, need to invest in standard sensible defaults, golden paths, whatever whatever you want to call it, that that platform engineering approach is much more important than people anticipated. And so I think we're seeing that in the reality in the market as well. But there are teams that are recently started and they're agentic first from the very beginning, and they often are going very fast and don't know that they are building a code base without patterns. Because they're all individually sensible decisions, but they're not adding up to anything that is a recognizable code base. I think there are people with long standing code bases who invested a ton in platform engineering and in making, you know, some of that boring maintenance work, etcetera, easier to to do with fewer people. And they are flying now with AI. And then there's a bunch in the middle who probably are not AI first. They're not at the beginning. They're not greenfield, but they also haven't invested in platform engineering. They don't have good, kind of pipelines and, and standards. And they are going very fast but very hazardly. So you you see them, going very quickly but then having loads of production issues, or you see them going very fast but then struggling to maintain things. And and I think a lot of the problem that's always been there in tech is that we talk all the time about building, but all the value is delivered when operating. And and so I think that we spend all of our energy talking about building the software in the first place and really deprioritize or misunderstand that the the value is delivered when it's running. It's actually surprising about platform engineering because we have a lot of conversations with, like, AI enablement leads and AI competence leads and whatever. And you... Maybe developer experience, but you rarely see anyone from platform engineering even though, like you say, it's the obvious place in which to sort of implement a lot of this stuff. Right? Absolutely. And there's some folks like, the Spotify folks have been really, generous with their sharing, and I think that's very positive. They've, they've written a bunch on that engineering blog about about their approach, and it's all platform engineering. It's, you know, it's how do we make it... Their first use of AI was to do dependency management in a in a much better, much more holistic, and, large scale way. And and I... And, you know, they're one of the organizations that I think is seeing the upper end of the productivity improvement, which is about... You know, there was a great talk at lead dev in the summer, from DX, and they found that from real data in their platform, they they, you know, they're seeing about 70% improvement being the top end of of the improvement. The average is 10 to 20%. That's very far from the 10 x people are expecting. Right? 1.7 is it's not what what people are thinking is the ceiling of of improvement. But, yeah, you know, when you talk to the Spotify folks, they think they've seen that sort of 70% improvement. They've shared a lot about what they're doing and why they're doing it, and it... It's super evident how much platform engineering is needed. Well, they have their own internal development platform that they have, open source, backstage, I think. Yeah. So they're pretty advanced in this stuff for sure. The two results that you got appeal, like, one is throughput and one is cycle time. Right? And often we see throughput increases and then cycle time, like, falls off a cliff. So, like, was was there... Did you see that sequence as well or was it different? No. We managed to find improvements in both, but I think partly because we were not... Just the timeline meant it was not very AI driven in in the beginning. And so I think that when you see that happen, it's often because AI has, made the the coding part of the process massively faster, but not the rest of the process faster. And we had the, the benefit, I suppose, of going in. And the first thing we did was change ways of working, and that improved the collaboration between product and design and engineering and data. And then that was the thing that affected cycle time the most. And then a lot of the rest of the improvement was about not losing the improvements in cycle time that had come from this close in that way of working and and better approach to, you know, my partner in crime, the chief product officer, did, you know, did some great work with with, with her team to make sure that we were clear on what success looked like at the beginning, which is such a... Sounds like such a basic thing, but it but it really affects teams if they don't have that from the outset. Right? A 100%. What we often see actually is that because of the rising cost of AI and so on usage, people then start looking at, oh, but, actually, how am I even measuring my engineering productivity or efficiency or whoever you wanna you wanna call it? And it's only because of the AI stuff that they start looking at some of some of those things. So what are the most common mistakes that, like, you see in organizations when it comes to trying to measure or... I don't know. Get some kind of handle on, like, how how well is our engineering organization functioning? Yeah. I, I think the biggest anti pattern is what I call watermelon metrics, where they're sort of green on the outside and red on the inside. That's nice. And I think it's very easy to accidentally invest in the metrics that are easy to measure rather than most predictive of success. I think that we're we're in a interesting space. I think I think that there's a lot of places that are measuring lines of code which is, you know, such a historical anti pattern that it's almost funny that people are back at the back using it. Right? It's it's... I I... It's a joke when somebody says that, and then sometimes they turn out to be serious, and it really worries me. And then I think I think the other big mistake people are making is code was never the bottleneck. Code was an investment, and it was something that needed to get done. But the code now being, you know, for better or for worse, very rapidly delivered highlights all of the other ways that we weren't streamlined. So if you, you know, if you still have some sort of change approval board or you still have some manual part of your process of getting something into production. If you don't good enough testing to trust that something is ready for production, all of those things are really awful when you then start accelerating the coding aspect with it. And what people tend to forget as well is that, code is cast. It has to be maintained. I think you talked about, debt... Technical debt reduction as well as one of the measures that you took at Plio. So how does that fit into the AI era? I mean, there's two perspectives. Right? I think there are people who believe this is just a new level of abstraction, and we aren't gonna have to care about the code level anymore. And there are folks who, in in my experience, tend to be very focused on that first build and not on the ongoing maintenance and ongoing, improvements of things. They're the same folks who predicted the SaaS pocalypse. Right? They they were like, well, we're just gonna... Everybody, it'll be so easy to build a tool. You'll just build your own tool, and you'll never bother with any of these SaaS products again. And if that's a mindset that's very indicative of somebody who's never run a production, product, who's never seen the actual value delivered by it being live and it being used. And and so I think that there's, you know, may... Maybe this perspective though does turn out to be right in the same way that it was with us moving from, you know, assembly to higher level languages. Maybe this is just a new level of abstraction. We don't need to care about the code anymore, and and it'll all be fine. My belief at the moment, and there's some pretty decent research to support this, is that understanding what's going on in the code is still very, very important to producing something that is production ready. And by production ready, I mean, scalable, testable, usable, accessible, all of all of these other things. And a lot of that work is the stuff that doesn't show up in your initial spec or in your initial build. Right? It's the it's the things that show up a a bit later on. Exactly. It's harder to see in your immediate metrics. You'll see them further down the road. So we've talked a bit about the product, you know, how to improve productivity even aside from, adding AI to the mix. The other side of that is cost, which all of our customers are concerned about right now. We are as well at JetBrains because we have thousands of developers using a lot of AI. So what do you think that engineering leaders need to most when it comes to, like, what they're spending? Because, again, this is something they've never really had to do before. I think we're we're in this very weird spot with, with AI. We're not sure what it's gonna cost. We're not sure what it should cost. And then it's really difficult to make sensible, trade offs, because because you're, you're unsure about it. I equally think there's a whole bunch of people and and engineers who have been invited or in... Or incentivized to token max to use as much as possible. And so there's a bunch of engineers... Companies have encouraged that. Yeah. Yeah. Some... Loads of companies have encouraged that. And and just proving you using it was so important for a while that I'm not shocked that there are engineers who will put the most basic things on the most, advanced model. And that chows through your, your token allowance very fast. I hear stories of companies where, you know, devs will run out of of, tokens and down tools. And, you know, it's it's back to when we used to wait while things compiled, I suppose. Right? The the build system on the c I d... CD system, like, ran... Or if it went down, then you couldn't do anything. I remember those days. We also hear... Like, I talked to a lot of engineering leaders as well who know that this, increase in cost is coming, and are worried about it also. So what do you think that engineering leaders need to understand the most about managing cost? Because they're going to get probably a budget. We see that happening as well. Like teams or projects getting a budget, and they have to figure out how to use it. Yeah. So I think that we're we're gonna see a load of optimization in terms of, like, are you using the right model for the right thing? Are you using open source models for for stuff that's pretty solved? Do you really need Fable on this particular challenge? I I I think there's a load of like, pick the right model. It's one of the reasons that Stripe buying OpenRoutes was really, really interesting to me. As I think that that sort of, load balancing the right query with the right model, pushing back on this default use the most complex thing, is quite is quite interesting. And I think it's gonna become very much the default infrastructure people need. I think we, with cloud providers, have decided that being on multiple cloud providers isn't super sensible. We don't want, you know, we don't want that complexity. Most places don't want the complexity of running on Amazon and AWS... Sorry, AWS and GCP. I think that the opposite is true with AI. AI. I think we all are gonna want to always have the options of the Frontier Labs and a bunch of, of open source models, and maybe you end up with some stuff that you don't want to ask a big American company, given some of the socio economic and, geopolitical challenges that we're that we're seeing and that we're very aware of, at at at the moment. We also see people looking at that diversity for cost reasons, like, not not now, not just for the performance reasons as in you want, you know, the best performing model or whatever. Yeah. You talked about platform engineering earlier and, probably building all that stuff into whatever platform you're building or using is is the best because then developers don't need to think about it. Right? Yeah. And I think making it so that there's a there's a couple of CTOs who I, who I coach where, you know, there's things like telling everybody to actively choose a lower model, or do you have some sort of auto choice and what does that do to your cost? What does that do to to things as as they as they get going? There is just a very real risk that we'll pay all of this money, and we will just throw more shit at the wall to see what sticks. And so I I also think in this era, we're in a position where judgment and actually restraint matter a lot more than they used to. Because I think, as a user, do I want the interface where somebody has just added every thought they've had in the last year, and whacked it in and hoped that it'll work? And probably not measured that it worked particularly effectively because product analytics is not as, I mean, it should be solved, but it's not in many in many organizations. And and so I I do think that we're going to be in this world where we're spending a lot. We're not really able to tie that directly to business results, And it's almost like it's more the the cost of doing work has just gone up. And and I think that there are organizations where that growth is sort of absorbed by other things. And then there are some organizations with, you know, there are... In an ideal world profitable, and they are easing into profit by spending that money. And so they are very much more incentivized to to say, are we actually getting the extra capacity? Is... And is that extra capacity being used worthwhile? Or are we just throwing more stuff at the wall and still none of it sticks? But but... Yeah. I I I also think it's it's led to this very interesting, thing where just measuring results is bit... Is much more important and and directly correlating features to improve commercial results is much more important. I think it's also uncovering a reality for a lot of particularly SaaS providers that the bar is constantly raising in terms of, user expectations. And so just to stay still, you have to keep improving. And and that's not something that many CFOs are ready for. It's not something that, you know, we've we've been the last few years, CTOs have gotten better at articulating why tech debt's important and why we have to invest in it. But, you know, usability and accessibility debt is a thing too, and just keeping up with what the latest design choices in the iOS and Android, operating systems is. You know, there's just a bunch of continuing to develop to feel fresh and to feel modern that doesn't tie very directly to commercial results. It maybe helps you reduce your churn, but it's very hard to articulate that that's definitely happening. And and I think it takes some, sense of conviction on the on the part of products and technology leaders to to say it's not just maintenance that has to be... We don't need to just keep it secure. We also need to keep it usable. We also need to keep it, sort of findable until such point as we get to agents being the primary interface. And I think, you know, maybe maybe then if I'm interacting with you primarily through my agent, primarily through your MCP server, maybe then it doesn't matter that you've got 12,000 options. If my agent can sift through them all and decide what the right thing to do is, then maybe that's okay. But I think while we still have user interfaces, that that sense of restraint and and judgment is gonna be very important still. Well, these two words, restraint and discipline, aren't things we've heard a lot of in the first few years of the AI Gold Rush. But I think we are actually coming back. Strangely enough, now that people are trying to scale up, agent usage in particular, You're seeing, a lot more return to old fashioned engineering discipline, which still works. So as you mentioned, just basics like doing documentation, which developers never wanted to do, but that they can generate now, so it's easier. What do you see, what practices do you see from, like, old school engineering best practices becoming more important as you're trying to scale up agent usage? I think TDD is very important again because I think that we've seen agents and and, LLMs write tests that pass rather than tests that test. And so I think I think that you actually end up using more of your human time on testing and on success criteria than than you might have done previously. But I I personally think there's a lot to be said for, for that sort of TDD approach. I think there's a lot to be said for the tidy first approach in refactoring, you know, a lot of getting to grips with a new area of the code. It's important to to maybe do a bit of tidying up or a bit of refactoring or bit of improving. And that's, you know, on the one hand, very helpful. On the other hand, maybe it then results in these huge diffs and massive PRs and and and everything else. So I I think it's it's a little, double edged. I think code review is the really interesting one, because we used to do code review partly as a quality aid, but mostly as a knowledge, management thing. Like, the main reason to do code review wasn't I don't trust my fellow engineer. It was at least two people should understand how this thing freaking works. And and I think that we're, we're hitting breaking points with code review. And it's very few, if any engineers got into engineering to review code written by a robot. What... Reviewing code is never the fun bit. Engineers don't like reading any code that's not their own, so it's not surprising. Yeah. And so we've gone from, your job is to understand the problem and solve it creatively with code to your job is to understand the problem well enough to tell a robot how to solve it, and then to review this robot solutions of other people. I'm not sure it's as exciting or interesting a job as it used to be. And I think there are people who are still trying to keep that architectural integrity and that standard and that, you know, that quality of output at the code level. Not just that the product is good, but that the code is good because they still believe humans will have to read that code at two in the morning to figure out what the hell went wrong. And then I think that there's a there's a set of folks, and maybe they're right, who are thinking we'll just have really great observability and really great bots that can solve incidents, that can grab all the right information, pull it all into one place, and present a human with a choice to make that doesn't require them to be in the code at depth. And I I think that's interesting. And, you know, and there are some companies that I really believe in. Like, incident IO, I've got an AI SRE that... Apologies. They they call it something else. It might be called investigations. But but it's essentially an AI, SRE, and it's gonna be very interesting whether that proves that the new level of abstraction is the behavior of the of the products rather than the the code... The underlying code. But there's definitely some evidence that the people who've got the strongest computer science fundamentals and deeply understand how systems are put together and how they work, they are doing the best at coding with AI. And so, you know, one of the things I think is very fascinating and very interesting in this area is how do you build up the judgment and the perspective that people used to build by doing this work manually? How do you build that up when you're not doing the work directly with your hands anymore? How do you, judge whether the output is good enough when you don't look under the under the hood and see how the engine is running? I've actually talked a lot, in your speaking engagements and whatever about how tech... The tech business needs to learn from other areas of knowledge. So what are the ones that you think are most important in... For the agentic era? Because, also, you have inadvertently... Every developer is becoming a team lead of a batch of agents, which is not the job that they signed up for. That's one example. I think, as you rightly point out, there's a bunch of it that's analogous to traditional technical leadership. So articulate what the architecture should look like, what patterns we're using and not using, what... When do we buy versus build, when do we use open source, all all of those kind of things. I think you're right though it's earlier in career than people are used to. There's a bunch of teaching skills that are necessary in a very different way than they used to be. A lot of, mentoring and coaching used to happen at the code level during pairing, and that was a sign of a great team was if they were, you know, both willing, able, and likely to be spending time pairing with each other every week. That, for me, was always a sign of health of a team. You know? Is somebody sitting, helping you ask agents for things very good pairing? I I don't I don't know. Everybody still got at such a stage of having their own personal way of using AI as well. I'm... I might learn something interesting watching somebody else using their AI, but it's probably not gonna teach me directly how to get the best out of my personal setup. And so I do think there's this sort of, like, loss of a load of other, of mechanisms that that we used to depend on. Critical thinking is really important. Systems thinking is really important. Having the ability to reason about, what users will and won't enjoy, what they do and don't want, the ways in which things are going to scale. And I think knowing the system well enough without knowing the code in detail is quite challenging. I think the places that are really suffering right now are places that didn't have great observability, didn't have great testing, and didn't have great ways of working to stay familiar with the the product code itself. And that's a known problem now called, you know, cognitive test where people no longer really totally understand the system that they're working with, and there's all kinds of, knock on effects from that which are quite negative. You mentioned systems thinking as well, and not just actually for, junior engineers, this is relevant, but because one of the things that AI AI does is allow you to, do tasks in areas with which you're not really that familiar. May... Maybe it's a language that's not your main language or it's a repository or it's a domain that you don't normally work in. So do you think that systems thinking becomes actually more critical for every engineer, not just for people who are learning? Yes. Absolutely. And I and I think not having that systems perspective or... And and some some of it is about having the perspective and some of it's about reasoning with that perspective and some of it is about communicating that perspective. Because I I think that, again, in the past, we used to tend to bring things down to a micro concrete level. We used to be like, let me show you this specific thing in the code that's not scaling properly because of, you know, because of this reason. And we're now having to almost operate Jaeger it. We're like robot hands. And we're at a distance from the actual reality, and we're having to reason about it without necessarily having access. And on the one hand, that feels like a really new thing. On the other hand, I spent my first ten years of my career at Procter and Gamble in what was much more of a corporate IT kind of environment than, engineering environment. And a lot of that experience was very similar. You you know, you had engineers in India and you would write specs and try to articulate what you needed and then somebody else would actually go into the code and try and figure out what to make it do. And I remember it being a huge personal advantage when I was in those roles that I could go into the code and check what was really going on. And so, I, you know, I I think people forget that we've had experiences of, having the code at arm's length. And it made things slower and less performance and often less of a good product. And and I I worry that we're sleepwalking in some of the same situation, with, with some of what we're doing now. And like you said, like defining intent, defining requirements is becoming even more important, not less important. Yeah. In fact, some of the testing, vendors are now actually moving into requirements, definition and, acceptance criteria because of how critical that is to actually getting the results you want at the end, including with testing. Like, one of your specialties I know has been scaling engineering teams. So what do you think is different now if you were trying to scale a team in, like, 2026, assuming they have access to lots of agents as well? Like, what would change, and what would stay the same? I think what would change versus what was ideal and what would change versus what was reality are quite different. Right? So I think it was always true that it would have been great if we'd written down what our architecture was and what our architectural decision records said and, documented things as we went. I think it's just more likely now that we will actually do it, which I think is sad. It's it's sad to do it for the robots, so we wouldn't do it for our fellow humans. Right? But I do think that's that's important. I I think that everything that was about being familiar with the code falls away a little bit, and everything that's about being familiar with the system stays. So how do you know whether the system is healthy is probably more important than how do you know whether an error has been thrown at this particular block of code. I... You know, a lot of what I would invest in from a scaling point of view was onboarding, was getting people to the point where they could be independently productive within their first six weeks rather than it taking three months to be independently productive. It helps a lot for things to be written down, and people spending some of the time that they save by having AI write the code to write docs is is really helpful. But I also think that we need to be more explicit about, well, we need to be more explicit about everything. Problem. Right? So you... In an ideal world, you want to be able to point an agent to your systems and go, here's how you figure out whether things are working. Here's how you figure out what the work is that needs to be done. Here's how you, you know, you could you could... If everything was properly captured, then your ability to to guide your agents is is fantastic. And I think that includes a load of the business context. And reality is that a lot of engineers only really understood the bits of the business logic that they had implemented and not as much about the overall business. And so I think, getting people to be better aware, to better understand, like, how does this company make money? How do we survive? What... When is growth the most important thing? When is profit the most important thing? What... You know, am I aligned with the incentives of this business? I think is increasingly important. Something that I really admired that Jonas, Templestein, the the founding CTO at at Monzo had done was he had all the engineers do retail banking qualifications. They all they all did, like, a retail banking, and n q or whatever. I forget what the sort of, they're an a level equivalent, but I can't know what they what they call that a sort of more job focused. And I think it made a huge difference that that you had all these engineers who were very good engineers, who came in and didn't really know the domain, didn't understand banking. But they were given at least some grounding in how banks worked, and what the, you know, what the traditional model was and what we would try not to do and what we were gonna have to, you know, accept doing because that's what the regulation required and and and those kind of things. And so I I think that with onboarding now, you're really aiming to help people understand the context that they're in, the commercials, the business logic, and the systems. Probably in that order where we used to start with, you know, understand this little bit of a system, then broaden out, then broaden out, then, you know, maybe by the time you're senior, you'll have seen enough of it to really understand this business if you if you stay for a long time. And I I think the combination of people moving roles more frequently, people, using agents more, means that we need to have that knowledge captured much more explicitly and much more shareably. So understanding the domain and the business that you're working in is becoming more important, not less important. It it... It's a fascinating industrial sabotage risk as well. Like, if you if you do now need to have your entire business model and way of working and, understanding of your customers, like, if all of that is now documented, it's not just in people's heads. It's documented in a way that agents can use it. Bluntly, that's documented in a way that someone can steal it as well. And I think that's probably the immensely valuable previously intangible information that meant that, you know, you would you would try to hire somebody from that company in order to get some of that knowledge or you would... Whatever. And I think now there's a there's a real risk that if somebody steals the right bit of context, then, they can they can achieve a lot. One of the things I think is really fascinating with Gen z is there's apparently a whole bunch of folks who have, absolutely adopted AI, but they don't wanna use the company accounts. They wanna use their personal accounts because they're like, my advantage is what I understand and what it... What my agents get, and what they can do. I don't want the company to own that because then they can just get rid of me and keep the agent. And I think that's a, like, survival instincts fantastic. Like 10 out of 10 kids are alright. Right? But but from a company perspective, it... It's it's tough. Right? Like, it... You know, you can 100% empathize with somebody wanting to do that, but but from a company point of view, you really don't want that happening. You don't want all these silos and people competing on how best to use stuff. You want them to be sharing with each other, and everybody use the best version of, of the skill or the rule or the, whatever other artifact it is. Yeah. And that's a huge risk to the business. Multiple risks to the business there about the industrial espionage and also people not cooperating and sharing their knowledge. Can I ask you for, like, your parting piece of advice to engineering leaders in 2026? I I think it's pretty much the opposite of what I was saying for the last twenty five years where I was very resolute that engineering managers should not code. And engineering leaders had so much work to do on catching up on leadership and on management and on people development That if they were trying to keep coding, they were hanging on to the past and not able to fully develop the skills they needed for their new role. And I think that the thing that's changed is if you've never built anything with AI, if you don't code, if you aren't up to date with what a developer's experience looks like these days, then I think that you are getting more and more out of touch day by day. And and I think that understanding how... What the real lived experiences of engineers is very important. And we're at an inflection point now where the way it was for you is not the way it is for people now. And so you can't rely on when you used to be an engineer, when you used to be an architect, architect, when you used to be a, whether it was a, you know, a staff engineer or a senior engineer or or whatever you were, because it's changed so much. And and I think that we've had a couple of inflection points like that along the way in the last thirty, forty years. But mostly, they were evolution, not revolution. This is a real revolution. And so I think that, managers and leaders need to get back to coding again. I don't think that they should be on the critical path. I don't think you should be doing stuff that means that you prioritize coding over people, because I think that was always the risk and that was always the problem. But I I think that we have to get back to being hands on, in order to understand the challenges that our that our people are facing. That's a great point on which to finish. So thanks a million for joining us, Marie, and thanks to everybody who joined us for Blueprint plus plus. Thanks so much for having me.