Transcript for "How to Build AI You Can Trust, With Adyen's Dhruv Ghulati": Hi. Welcome to Blueprint plus plus, a webinar series in intersection of business and technology. And our guest today is Dhruv Galati, a product leader at Adyen. Hi, Dhruv. Nice to see you here. So, let's start with a little background. Could you tell a bit about yourself, how you got here, and what you're working on right now? Sure. Yeah. So my background, I actually started off my career in finance and banking. My first job was at Merrill Lynch on the trading floor, so I had I had nothing to do with technology. But this was around 02/2014. I got very interested in in tech and got into AI back then. Yeah. And then my career has spanned kind of being an early stage founder. I was in London, and I was working on a company that was detecting misinformation, disinformation online. And then after exiting my first company, I decided to tune in and hone in some of the skill sets that I wanted to gain, which is around AI product management. Because I think I felt that to be a great technical leader, and and be a great entrepreneur, actually, you have to understand product. Then I moved to Amsterdam, where I was leading out, the Uber AI team on product, in Amsterdam. So I was the first site lead here. So worked in a lot of interesting AI problems across mobility, Uber Eats, delivery, forecasting problems, and we can talk talk a bit about those. Mhmm. And currently, I'm a principal PM at Adyen, so I'm I'm leading out some of their their goals of becoming and and building out some of their their AI products, for payments. Mhmm. Yeah. And you've been telling about the, your first start up, basically, FactMatter. So how do you feel nowadays, about the fact that you've started using, machine learning methods and PhD folks who are, in your company working on fact checking. But nowadays, machine learning capabilities are more kind of a contributing into the problem. I'd say, like, a lot of my my my career has been about, I guess, like, my company was detecting information that you just don't want to have on your website or on your ad network, and it was kinda detecting bad things. Right? Same thing with, when I was at Onfido, I was detecting counterfeits and forged documents. So, now even at at at Adyen, I'm working on chargeback fraud detection. So I've always been detecting kind of fraud, bad things, and I think that to understand and be able to do that effectively, you have to understand how that information is generated. You have to go very deep into fraud patterns, you know, generative AI for disinformation creation. What is the motivation of the attacker? What's the motivation of the creator? What techniques are they likely to deploy? Mhmm. If you don't understand that, you can't build detection systems for that. So, you know, a lot of this is is is actually a cat and mouse game, and I think we're gonna still have this through the Internet. Right? There's always people who there there's this wave of of creation of things, and then there's this wave of regulation and moderation and and making sure that things are are safe. Talking about, Adian, actually. So you said check back, fraud detection, and it seems like coming through different companies, you've been getting in more and more governed, companies. So Mhmm. Adehan, for example, with the financial corporation. And probably chargeback, detection is a lot more governed question and anti fraud detection than, for example, what was done in a feeder or before that in a fact matter. So how do you feel about that whole governance procedures that are being applied? And at the end, how did it change from the other companies, and how does it affect the AI capabilities that you are able to use in the company? Yeah. Interesting. Yeah. I think I think, I think your question's a lot a lot about sort of what are the different ranges of security that you need to have towards towards AI models. Yeah. The fact is, I think in the last kind of probably, and it's accelerated particularly in Europe, right, we've had the AI Act that's been coming out, and I think that's meant to go live. I don't know the exact date, but I think it might actually be live now. Yes. But a lot of the wave has been to for AI regulators, and and and governments to be sort of trying to catch up, right, because AI has moved so quickly that, you know, you don't wanna be in a regulatory situation where effectively the regulation is is is slowing things down for no reason or the actual regulation makes zero sense. I think one of the the key things that you typically had with AI is, like, what what data is it trained on? Mhmm. So, you know, does it contain personal information? Is it trained on information that's kind of, like, a person identifiable? Yeah. Does it have biases? For sure. Does it have biases in the training data? But then there's the decision elements. Right? So if you have, AI models that are taking real decisions where they actually affect people's lives, that's where AI regulation has has has focused very heavily. Right? So whether it's, for example, getting a loan application denied or accepted Mhmm. We're gonna go through a lot of that now because you're gonna have agents you know, as you have a world where the the the drive is to automate as many things as possible and manual processes, you've got to remember, they were manual for a reason. You had a human in a loop in it for a reason to do that loan application or to do the mortgage application. And then I think now regulators are sort of looking at, what you know, is a human in the loop because you actually need the human expertise, or actually could a could a machine be doing this in a more fair way? I think that's the question that regulators are are asking. I think another part of your question was around, you know, how have I seen the different ranges of how AI company or or tech companies basically open up their data. Yep. I think, you have on the one hand of the spectrum, you have very locked down closed user groups and user permissions where certain parts of the code base are just not accessible to other people. You have, you know, limitations on on downloading code or downloading files to local files. That's another thing that I've seen. You've had just access permissions denied. You've had so trying to ring fence access to data versus access to code versus access to certain types of tools, whether it's your Kafka pipelines or whether it's your Production databases. Production databases. I've seen and then on the other hand of the spectrum, when you have an early stage company, you have situations where just everyone has access to everything. You don't have time to have that annoying thing where you're trying to access some information. You don't have access. That is a that's a pain, right, if you're trying to move move fast. I think that's basically the thing that companies are going through where they put all these access controls in place for a reason. And now we're gonna have to question, like, why are those what is the biggest risk that we can face by giving people access to that, right, whether it's a product manager, whether it's a salesperson. Is the risk that they're gonna, for example, interpret the data incorrectly, and then that's gonna lead to some some outcomes that are incorrect, when they you know, from a data access perspective? Is it that you're you're scared of someone basically accessing the code, deleting some code? I think you can create I think agents actually have a chance to create these permission layers automatically. Right? Mhmm. Whereas in the past, it's been really manual to make You have to have a security expert. Yeah. You had a security expert, but you also had a lot lots of meetings internally of, like, who gets access to this, who gets access to that, different levels of permissioning I've seen. I think it has a benefit. Right? But I think, well, like with any security problem, like, you only know, when things go wrong. Right? So so it's kind of one of those things that, like, okay. Do we keep all this architecture going of these, like, really intense user access permissions? What are the risks? Or is actually is that slowing down our progress? Do you see the, do you see the parallel with, basically, an application of agents? So as far as far as I see that, when I have a small start up, you have indeed access to everything. Mhmm. In a lot of cases, founder is the only one who is able to fix production right now. And the same is actually happening with the companies who are trying to become more of AI companies. They do see actually influx of, like, tens of thousands of new developers, agents, which actually have the very similar problems with permissions, with trying to access the production database, or trying to drop something. Do you see the parallel here? Big time. I think I think one of the biggest things that that AI native companies are starting to do is is is, for example, just create MCPs, create create access to all these different tools that they were using. Mhmm. And it's magic when you can actually do that. You know, you know, I know that, you know, at Adjourn, for example, we have an internal team that's built this this this system for YouTrack, which is our kind of task tracking system. Those are that to you. Right. Yeah. So so that, basically, you know, like, I was doing this yesterday. You know, PM could basically start editing tickets, you know, on their terminal and and and adjusting them, changing the statuses, doing them in bulk. I mean, that would have taken so long, you know, pointing and clicking at at certain things. So I think just that magical unlock of of unlocking access to your tools, I think every single, SaaS company has to have an MCP strategy right now. Mhmm. And because the the the magic of of opening that up for, like, a new user base is is is huge. Right? Like, people really want to see. Okay. Does this that's how I think. Right? Like, does this tool have an FCP or not? Is it integrated? Is it open? In in a way, we're in the state we're in the same kind of world of APIs. Right? Like, k. Does this is this is this promoting an open ecosystem? Mhmm. And I think that's where, companies can really start, like, actually, like, the the the unlock you could get from creating toolkits and skills for your tools, MCPs, you know, and access, creating little use cases and builders for interacting with your data information. Mhmm. That's gonna be a real unlock, I think, for for for talking about about unlock. So, you've been before in Uber, and, there is a very well known nowadays, Michelangelo platform, that was created in Uber. And that is, as far as I understand, unlocking a lot of AI capabilities inside of that. Could you tell a bit more about? Yeah. I think I think Uber was is a very advanced company when it comes to and I've I've definitely now seen different ranges of people on the on the stack. Uber built Michelangelo where effectively the idea was they had they had a marketplace real time algorithm where people you know, drivers and and and riders would be matched in real time. Mhmm. It's incredibly hard to do that. You know, you're you're running models at scale, low latency, you know, in real time with with constant pinging, different times, different time zones. You have to keep the systems up at all times, and you have to have redundancy. You have to check around observability at all times. This this machine learning infrastructure, they had to build systems to make that that easy and possible. Mhmm. And that was the birth of Michelangelo. But since then, what that enabled was that, you know, as as an organization as the organization grew, you had all sorts of other teams saying, hey. We now need to do marketplace matching of the drivers, but how do we do real time pricing? Mhmm. You know, how do we do real time incentives that we give people? How do we, you know, run other models that we might wanna have that do, you know, real time prediction of of where our driver should go to predict demand and supply. So the use case has started to grow. And instead of all of these different internal model builders starting to have to compute features for their models again and again Mhmm. Or try to reuse, you know, all the different systems they had to build in pipelines, whether it's, you know, MLflow, Airflow, different DAGs that they had to build out. Why not reuse these components? And that was that was the benefit of Michelangelo. And I see that, you know, the role of an MLOps person, you know, it's very hard to that was if you go on LinkedIn now, it's actually very nowadays, obviously, it's it's a big role, but I think a lot of large corporates don't sort of don't understand, like, that is an actual role. It's not a data engineer. It's an ML ops person. And that that goes into the language of features, feature stores, you know, model pipelines, observability, model evaluation metrics, retraining systems, AutoML. It isn't just, you know, data catalogs, data pipelines. It's a it's a different system. Yeah. It's a whole system. And I think that role, a lot of companies I'm seeing so, okay. Let's let's hire some scientists to to fix you know, build this model for us. But the model builders can't build a model if there isn't the architecture to run models effectively or or run experiments quickly or test them both offline and online, to instrument your your test datasets. Like, how do you have versioning on your test datasets? All of these things, like, are, sort of linked to data, but they're they're a different language. Mhmm. And so I think that's really, really important for a big business to really build the infrastructure so that anyone can go in and say, hey. I have a machine learning problem. I'm sitting in the sales department. I'm sitting in the marketing department. I wanna build a model. Where do I start? And and build this internal experience that's really easy for them. And how to create that model so it would be production level ready. Exactly. Yeah. Talking about, like, product culture and the fact that you are being amplified a lot with the AI agents, you still have a lot of, like, standard product metrics and standard product approaches. Yeah. It doesn't probably matter whether you are, like, helping developers to understand what they should be doing and what product they are creating or agents. Mhmm. So how do you approach that, especially in the era of AI and the agents? Yeah. So if you have traditional ML, right, like, take let's take kind of, you know, fraud detection, a a great example. Right? You you have precision and you have recall. Right? And then you have a curve, RCAC curve, where you can kind of tune those things in different ways. Precision is effectively in the case of fraud, you know, like, how good it how when you do flag something as a fraud, is it correct or not? Mhmm. Recall is of the things that you should be catching, what percentage did you catch? You know, you could also have a system that flags everything, and so your recall is a 100%. You caught everything. But then when you caught it, you're gonna have a lot of false positives. Right? You're gonna be so these these are just, like, basic frameworks that, like, I think people have to just understand, like, what are the core ML model metrics. Mhmm. But, honestly, like, you can deduce by logic. Right? Like, it's it's so it's depending on the problem that that you have. Right? Predictions have the same thing. Classification has the same thing. In the world of AI, where you're building, let's say, a chat assistant, or you're building, you know, to chat GPT Mhmm. Your evals are a little bit different, right, because you have other types of things you have to to understand, like, okay. Was this conversation a good conversation? Mhmm. How many multi turn conversations did you have versus one shot conversations where you just answered it one time? How many tie like, what was the average length of a conversation? In terms of, you know, eve we had this phrase of evals, right, and and LLMs of, like, how do you measure that that response was a good response versus a bad response? Because it's a subjective thing. Right? For some people, that would have been a good response. Some people, that wouldn't have been a good response. So there's this element of how do you kind of try to take a very subjective thing, which is a conversation, and try to put quantitative metrics on it? Yeah. Do you see the same problem actually with just a training of, large language models with RHLF and so on? Yeah. So so as in RHLF used as an eval metric? And, mainly, I think, talk about the cases, like, for example, in the case of OpenAI or Entropic, they've been tuning their models with GPT five, five dot one, five dot two, trying to be less friendly, more friendly, so on. It's also kind of a product work. But for a large language model, we're trying to understand which personality should it have. I mean, look. A lot of those model the way that people are are doing this is, like like I said, this is subjective thing. Right? So people forget that there's there's obviously people who are figuring out the model. Mhmm. Does the model work well? And there's a lot of benchmarks that are built for AI agents. Right? That that like, open source benchmarks for their how good are they on these specific hard coded tasks. Right? Mhmm. So you can try to close the domain and say, okay. We think that, most of our feedback that we got was that the conversation wasn't very friendly. So if we make the conversations more in a friendlier tone, people will be be happier. The challenge is that you have these internal model metrics, but then you have your online actual outcome is that is ChatChi Boutique growing. Yeah. That's what management cares about. Right? Is is it and and and that may not be at all because of the fact that the conversations are friendlier. Yeah. It may be that that actually the the cohort that you're going for, the the next growth segment, if you're not to get to the next billion users, maybe they don't want friendly conversations. Maybe they want more targeted, rationalized conversations that achieve certain tasks. Or maybe it's actually because the user experience is not very good, you know, for ChattyPT. Or maybe, actually, the the users that you're targeting, they need to access spreadsheets very well, and, you know, the tool calling for that isn't very good on on ChattyPT, but it's very good for Claude. So the the the the the the complexity and the vastness of the space of metrics that you can have is so, so big. I think most ML AI product building has to split. Find the ML metrics, the one or two core ML metrics Mhmm. That really drive, like, it this is a good ML product, but then the sort of online metrics, you know, which really Aligned to them. Aligned. And and and ideally and this is where people get it very wrong is, like, connecting that together. So if I really make if if I was to make the, the responses perfect for every single conversation, it answered every single question perfectly, Would that lead to growth of chat g p t? But, what do you see else, like, in a such a governed company? What are the main methods and things that you should be thinking about governing the AI inside the organization? Yeah. So AI inside the organization, people have to think about access controls. You need to have a data catalog. Right? What data is being spawned? What data already exists? What jobs are running in the background, what sub metadata sets are being created. You need to have a good handle of that. You need to be able to log it and understand what is actually in that data. Mhmm. Once you've logged that data, you have to understand what are the potential use cases that that data could be used for. Mhmm. I mentioned before, you know, if you had data on previous loan applications and if they were rejected or rejected, and you also have features that tell you what race that person is is in or what country they're from, You can maybe you're not doing it now, but you can possibly understand that that could be used, you know, as you're building a model that that learns from those features in an incorrect way. Yeah. So putting safeguards typically, what people don't have is to put safeguards on, like, testing of the model, so you have kind of red teaming where the model's basically triple checking. Are they making decisions and learning from those features too much? Right? Mhmm. Or or if you only had those isolated, those features, and you just had that as the feature that that the model was learning from, would it take would be the decision boundaries that were created from that? Mhmm. The the challenge with this is that actually, factually, it is correct that, let's say, one race versus another race has been accepted from a loan application. But then there was probably something wrong in your own system. Yeah. Like, there is something wrong with society. In society in society, but also maybe the way that you had been doing your own loan applications as a as a business. So it's not that you are that the system is wrong. It's sort of starting to unlock, changes in the overall, actually, online system itself. There was basically something about your system. So Exactly. Yeah. I've actually heard a lot of, feedback about some trained systems in the big corporations or even trained system in some domains where people are saying that, yeah, folks. It's, like, it's not correct. It should not be working that way. But, yeah, in in the end, you find out that it's actually working the way the model is predicting. The problem is not in the model. The problem is the system. System. Yeah. And then then you can say, well, look. Let's even remove that feature. Mhmm. It's not gonna learn from that feature. But then then you would then still still train the system. It would work it's probably slightly worse for your task. And so you've taken a business decision to not learn from that feature. Mhmm. You've effectively got worse at automated decisioning, yeah, for loans because you've decided to ignore that feature. But then you can if you did a correlation and actually, like, actually, it's still like, even that system that you train that didn't use that feature is still correlated on if you had isolated that feature or not. So I think I think it's basically what I'm what I'm trying to say is that, by looking at governance and looking at what is actually driving a decision for an ML model Mhmm. You can really improve that whole not just the you you obviously, like, not not just automate the system itself, but you'll start to actually rearchitect the way that you're doing loans or the way that you're doing fraud detection, for example. Yeah. We we've I can give you an example. You know, we've we've chargeback fraud detection. You might have, for example, if you were to dump all of the features that you had available to do this chargeback fraud detection, you might find that actually some features are really, really predictive of that, but they make no explainable sense whatsoever. They just work. And so there's this decision for companies to be like, well, do I just build the most highly precise and highly high high recall system, or do I actually put sacrifice some performance on this wider test set to at least make the system more explainable? Yeah. To to to make it more interpretable. Yeah. That is actually a very important question because interpretability also helps you to improve the system. Yeah. Yeah. Exactly. Talking about investments. So you were saying that, if you're, like, a big company, middle company, and you have a flat budget right now for EI, what where would you invest money? So Uber, for example, they've invested in Michelangelo, like, ten years ago at least. And nowadays, they are able to deliver more and more machine learning models, on any level, basically, both the developers and product managers, whoever. Yeah. So what would be your advice to the CTO? My advice to CTO would be, like, you cannot predict all the use cases for machine learning in your organization. You've got thousands of people who who have tasks to automate or or jobs to be done that they take on in their day to day. So instead of trying to map out all those AI use cases, which is what I see a lot of people do. It's like, where where can we get value from AI? Let's, like, write it in a document. And that will change in the next quarter. That will change in the next quarter. Build a sandbox environment. Invest for two quarters in a sandbox environment that enables anyone to build a machine learning model. Mhmm. Open up your data so that you have features that people can access, whether it's of data streams that are coming through of what the business is doing, whether it's sales calls, whether it's, in the case of payments, payments going through transactions, you know, information that's anonymized about shoppers who are doing those transact. Open that up to people. Mhmm. Make it easy for them to train models using some templates, simple XGBoost, logistic regression, so simple models, and just see what use cases come out. Obviously, open up LLMs as well. Right? Open up access to the documents and the files that you have and and see what use cases people want to do and create an open channel where those use cases and the issues that come up are fed through to you so you can patch them and fix them. And you will learn so much about what people want to use AI for in your organization. That was a great and nice discussion. Yeah. Happy to talk to you. Thank you. So maybe you have, I don't know, any other advices to the folks who are listening to that? Yeah. I would say just, yeah, it's it's the age of building. It's the age of creation. I think that's a really big opportunity for not just people in the workplace, but also people humanity in general. Mhmm. And it's just, like, great opportunity to to recreate that that magic of of of creativity. So I think even if you are in a job or you are an entrepreneur, it doesn't really matter. Try to bring that and weave that into your day in the way that you you you you you work. So it's basically be creative, have an agency, and be hands on. Yeah. Yeah. Okay. Great. Thank you. Thanks for joining. And if you want to know more about how AI may fit into your organization, check the link below.