Kavak's Playbook for Rebuilding a Company Around AI

I'm investing more I'm investing more in tokens than in knowledge workers. We in tokens than in knowledge workers. We in tokens than in knowledge workers. We could build superhuman agents. This could build superhuman agents. This could build superhuman agents. This means that by every dimension that means that by every dimension that means that by every dimension that matters, our [music] agents would matters, our [music] agents would matters, our [music] agents would outperform the best human if we had ever outperform the best human if we had ever outperform the best human if we had ever hired. hired. hired. >> ambitious companies listening to this >> ambitious companies listening to this >> ambitious companies listening to this will decide to follow suit, which is you will decide to follow suit, which is you will decide to follow suit, which is you decided to build an agent per customer. decided to build an agent per customer. decided to build an agent per customer. >> Yes. Every day between 100 and 200,000 >> Yes. Every day between 100 and 200,000 >> Yes. Every day between 100 and 200,000 [music] [music] [music] agents get instantiated specifically for agents get instantiated specifically for agents get instantiated specifically for this customer with its own virtual this customer with its own virtual this customer with its own virtual machine. machine. machine. >> There's a lot of people worried about >> There's a lot of people worried about >> There's a lot of people worried about how the organizations of the future are how the organizations of the future are how the organizations of the future are going to look like and the role that going to look like and the role that going to look like and the role that humans are going to play. humans are going to play. humans are going to play. >> If you haven't faced fear before, you >> If you haven't faced fear before, you >> If you haven't faced fear before, you haven't felt it, then you haven't tried haven't felt it, then you haven't tried haven't felt it, then you haven't tried AI. We launched a program [music] AI. We launched a program [music] AI. We launched a program [music] inside Kavak that's called the Jedi inside Kavak that's called the Jedi inside Kavak that's called the Jedi Academy. From the CEO to like AI Academy. From the CEO to like AI Academy. From the CEO to like AI engineers to mechanics, we train engineers to mechanics, we train engineers to mechanics, we train everyone and after [music] 6 weeks, they everyone and after [music] 6 weeks, they everyone and after [music] 6 weeks, they launch state-of-the-art agents to launch state-of-the-art agents to launch state-of-the-art agents to production. production. production. >> What advice do you have to future >> What advice do you have to future >> What advice do you have to future founders or first-time founders that founders or first-time founders that founders or first-time founders that might be listening? might be listening? might be listening? >> What works right now is >> Welcome back to the ACC podcast. Today >> Welcome back to the ACC podcast. Today we have Ali Massa, the head of AI at we have Ali Massa, the head of AI at we have Ali Massa, the head of AI at Kavak. We're going to discuss today the Kavak. We're going to discuss today the Kavak. We're going to discuss today the transformation that Ali led within Kavak transformation that Ali led within Kavak transformation that Ali led within Kavak to turn it into an AI-native company. to turn it into an AI-native company. to turn it into an AI-native company. Thank you, Ali, for being with us today. Thank you, Ali, for being with us today. Thank you, Ali, for being with us today. >> Thanks for having me. >> Thanks for having me. >> Thanks for having me. >> Before starting at Kavak, >> Before starting at Kavak, >> Before starting at Kavak, you were running a company called Opy you were running a company called Opy you were running a company called Opy Analytics. Analytics. Analytics. >> That's right. >> That's right. >> That's right. >> And you were very much into AI before >> And you were very much into AI before >> And you were very much into AI before ChatGPT. You want to tell us a little ChatGPT. You want to tell us a little ChatGPT. You want to tell us a little bit about that journey? bit about that journey? bit about that journey? >> Yes, yes, of course. Well, we called it >> Yes, yes, of course. Well, we called it >> Yes, yes, of course. Well, we called it machine learning back then. It was a machine learning back then. It was a machine learning back then. It was a different family of of algorithms and different family of of algorithms and different family of of algorithms and and we founded a company and we founded a company and we founded a company with this very like with this very like with this very like like ambitious vision there that that like ambitious vision there that that like ambitious vision there that that new machine learning models would be so new machine learning models would be so new machine learning models would be so powerful that they could solve any powerful that they could solve any powerful that they could solve any complex problem. This was complex problem. This was complex problem. This was pre-transformers, right? This was like pre-transformers, right? This was like pre-transformers, right? This was like 2013. 2013. 2013. So, we started building the company that So, we started building the company that So, we started building the company that way and way and way and I think we were like I think we were like I think we were like 10 years ahead of time. But, we built a 10 years ahead of time. But, we built a 10 years ahead of time. But, we built a great company. We served like 14 500 great company. We served like 14 500 great company. We served like 14 500 companies companies companies uh around like risk algorithms, uh around like risk algorithms, uh around like risk algorithms, logistics, forecasting, marketing, um logistics, forecasting, marketing, um logistics, forecasting, marketing, um but like really the the the power of but like really the the the power of but like really the the the power of what transformers and then like the what transformers and then like the what transformers and then like the ChatGPT moment ChatGPT moment ChatGPT moment uh when it arrived make things like very uh when it arrived make things like very uh when it arrived make things like very clearly that that we could now build a clearly that that we could now build a clearly that that we could now build a whole new whole new whole new uh uh uh company and and company and and company and and and way of of of building companies. and way of of of building companies. and way of of of building companies. Um and we joined Kavak through and Um and we joined Kavak through and Um and we joined Kavak through and Carlos to build it. Carlos to build it. Carlos to build it. >> Amazing. All right, so we're going to >> Amazing. All right, so we're going to >> Amazing. All right, so we're going to spend the bulk of this podcast talking spend the bulk of this podcast talking spend the bulk of this podcast talking about exactly how you've identified about exactly how you've identified about exactly how you've identified Kavak, but maybe just to start, what Kavak, but maybe just to start, what Kavak, but maybe just to start, what does Kavak do and what is your role does Kavak do and what is your role does Kavak do and what is your role there? there? there? >> Kavak uh started out as a used case as a >> Kavak uh started out as a used case as a >> Kavak uh started out as a used case as a used car marketplace. So, we buy cars, used car marketplace. So, we buy cars, used car marketplace. So, we buy cars, we refurbish them, and then we sell them we refurbish them, and then we sell them we refurbish them, and then we sell them and finance them. But to do that, we and finance them. But to do that, we and finance them. But to do that, we also had to build a fintech and a also had to build a fintech and a also had to build a fintech and a logistics company and a Carfax and like logistics company and a Carfax and like logistics company and a Carfax and like basically all the infrastructure for basically all the infrastructure for basically all the infrastructure for this to work didn't exist in in LatAm. this to work didn't exist in in LatAm. this to work didn't exist in in LatAm. So, we had to build everything So, we had to build everything So, we had to build everything vertically so we could so serve our vertically so we could so serve our vertically so we could so serve our customers the right way. customers the right way. customers the right way. >> I want to sort of start with the framing >> I want to sort of start with the framing >> I want to sort of start with the framing of what the architecture looks like. So, of what the architecture looks like. So, of what the architecture looks like. So, a consumer comes in and says, "I want to a consumer comes in and says, "I want to a consumer comes in and says, "I want to sell my car." Like how many agents do sell my car." Like how many agents do sell my car." Like how many agents do they touch? Like what's the harness look they touch? Like what's the harness look they touch? Like what's the harness look like? Like ground us in how you design like? Like ground us in how you design like? Like ground us in how you design this. this. this. >> Right. So, so so we bet the company in >> Right. So, so so we bet the company in >> Right. So, so so we bet the company in transforming transforming transforming to a company run by agents. The to a company run by agents. The to a company run by agents. The questions we ask ourselves is, "How questions we ask ourselves is, "How questions we ask ourselves is, "How would we build Kavak in 2035 with Fable would we build Kavak in 2035 with Fable would we build Kavak in 2035 with Fable 10 or or or GPT 10 level intelligence?" 10 or or or GPT 10 level intelligence?" 10 or or or GPT 10 level intelligence?" And actually that company looks very And actually that company looks very And actually that company looks very different that than what we had built or different that than what we had built or different that than what we had built or what we had back then. what we had back then. what we had back then. So, when a customer comes in right now, So, when a customer comes in right now, So, when a customer comes in right now, um agent will get spawned specifically um agent will get spawned specifically um agent will get spawned specifically for this customer with its own virtual for this customer with its own virtual for this customer with its own virtual machine. It'll remember years of machine. It'll remember years of machine. It'll remember years of interaction of this customers with with interaction of this customers with with interaction of this customers with with Cabal, what they visited in the web page Cabal, what they visited in the web page Cabal, what they visited in the web page or call they had 2 years ago. or call they had 2 years ago. or call they had 2 years ago. Remember everything like in its memory, Remember everything like in its memory, Remember everything like in its memory, come up with a strategy and set a come up with a strategy and set a come up with a strategy and set a long-term goal to maximize the lifetime long-term goal to maximize the lifetime long-term goal to maximize the lifetime value of this customer and do whatever value of this customer and do whatever value of this customer and do whatever it takes to make the customer happy and it takes to make the customer happy and it takes to make the customer happy and convert them into like all our different convert them into like all our different convert them into like all our different products products products like across time. And like across time. And like across time. And this is a completely new and this is a completely new and this is a completely new and groundbreaking architecture at scale, I groundbreaking architecture at scale, I groundbreaking architecture at scale, I think, because think, because think, because like people are still building like people are still building like people are still building multi-agent system with with experts and multi-agent system with with experts and multi-agent system with with experts and and we realized too bad and we realized too bad and we realized too bad that long-running agents with hard that long-running agents with hard that long-running agents with hard goals, not just workflows, goals, not just workflows, goals, not just workflows, could could maximize our our customers could could maximize our our customers could could maximize our our customers satisfaction and obviously their their satisfaction and obviously their their satisfaction and obviously their their lifetime value. lifetime value. lifetime value. >> Awesome. Okay, so we're going to jump to >> Awesome. Okay, so we're going to jump to >> Awesome. Okay, so we're going to jump to the nuances that but maybe versus many the nuances that but maybe versus many the nuances that but maybe versus many companies that say, "Hey, we want to be companies that say, "Hey, we want to be companies that say, "Hey, we want to be agentic." And they try some workflows. agentic." And they try some workflows. agentic." And they try some workflows. >> Yes. >> Yes. >> Yes. >> You guys took the just rip like we had >> You guys took the just rip like we had >> You guys took the just rip like we had to make this work. You had to downsize to make this work. You had to downsize to make this work. You had to downsize dramatically. It didn't work for a year. dramatically. It didn't work for a year. dramatically. It didn't work for a year. >> Right. >> Right. >> Right. >> So, do you want to talk through >> So, do you want to talk through >> So, do you want to talk through obviously you had to obviously you had to obviously you had to tune a lot of things to make that work. tune a lot of things to make that work. tune a lot of things to make that work. Like describe the harness at that time Like describe the harness at that time Like describe the harness at that time and like what models you were using and and like what models you were using and and like what models you were using and sort of specifically. Yeah. sort of specifically. Yeah. sort of specifically. Yeah. >> So so there there were like three main >> So so there there were like three main >> So so there there were like three main decisions that that we had to make. The decisions that that we had to make. The decisions that that we had to make. The first and this is what where I think first and this is what where I think first and this is what where I think many companies are stuck right now is many companies are stuck right now is many companies are stuck right now is the first instinct is, "Okay, let's the first instinct is, "Okay, let's the first instinct is, "Okay, let's adopt AI." And you you basically leave adopt AI." And you you basically leave adopt AI." And you you basically leave your structure as it is and just give your structure as it is and just give your structure as it is and just give ChatGPT or Claude to to your team and ChatGPT or Claude to to your team and ChatGPT or Claude to to your team and then then then there's no efficiencies. Your customers there's no efficiencies. Your customers there's no efficiencies. Your customers have the same problems and and nothing have the same problems and and nothing have the same problems and and nothing happens, right? And so so you need to happens, right? And so so you need to happens, right? And so so you need to redesign your whole company redesign your whole company redesign your whole company around the agents and around the future around the agents and around the future around the agents and around the future capabilities. And this means really like capabilities. And this means really like capabilities. And this means really like rebuilding most of your APIs, rebuilding rebuilding most of your APIs, rebuilding rebuilding most of your APIs, rebuilding your system so the your system so the your system so the agents can use them to to perform. agents can use them to to perform. agents can use them to to perform. Then, you need to start generating the Then, you need to start generating the Then, you need to start generating the data and the feedback loops to fine-tune data and the feedback loops to fine-tune data and the feedback loops to fine-tune these agents. The only way to really these agents. The only way to really these agents. The only way to really make them work is if you teach them. make them work is if you teach them. make them work is if you teach them. Like, how do you teach them? You you put Like, how do you teach them? You you put Like, how do you teach them? You you put them out in the open, you you put them them out in the open, you you put them them out in the open, you you put them in front of customers, you get that in front of customers, you get that in front of customers, you get that data, you get those evals, and then you data, you get those evals, and then you data, you get those evals, and then you train your your train your your train your your your agents. And this is the second bet your agents. And this is the second bet your agents. And this is the second bet that we made that that we could build that we made that that we could build that we made that that we could build superhuman agents. This means that by superhuman agents. This means that by superhuman agents. This means that by every dimension that matters, like every dimension that matters, like every dimension that matters, like conversion, conversion, conversion, lifetime value, uh customer experience, lifetime value, uh customer experience, lifetime value, uh customer experience, our agents would outperform the best our agents would outperform the best our agents would outperform the best human we had we had ever hired. And we human we had we had ever hired. And we human we had we had ever hired. And we put them in front of the hardest put them in front of the hardest put them in front of the hardest problems. problems. problems. And and finally, you you start to change And and finally, you you start to change And and finally, you you start to change how you measure the success of the how you measure the success of the how you measure the success of the company. Cazoo was uh transactional company. Cazoo was uh transactional company. Cazoo was uh transactional company. We used to measure how many company. We used to measure how many company. We used to measure how many cars we bought, how many cars we sold, cars we bought, how many cars we sold, cars we bought, how many cars we sold, how many brakes we we needed to to break how many brakes we we needed to to break how many brakes we we needed to to break pads we needed to buy. pads we needed to buy. pads we needed to buy. Uh and we moved to a relational company Uh and we moved to a relational company Uh and we moved to a relational company where now I have 10 million customers in where now I have 10 million customers in where now I have 10 million customers in my database, and I have agents assigned my database, and I have agents assigned my database, and I have agents assigned to most of them to most of them to most of them with the task of maximizing their with the task of maximizing their with the task of maximizing their lifetime value. Now, lifetime value. Now, lifetime value. Now, we're selling cars and and and personal we're selling cars and and and personal we're selling cars and and and personal loans and very high-ticket items. So, loans and very high-ticket items. So, loans and very high-ticket items. So, just activating 1% of this customer just activating 1% of this customer just activating 1% of this customer base, it's like hundreds of millions of base, it's like hundreds of millions of base, it's like hundreds of millions of of dollars uh if we do it the right way. of dollars uh if we do it the right way. of dollars uh if we do it the right way. So, so it's a bet that made sense So, so it's a bet that made sense So, so it's a bet that made sense for us because of our industry, because for us because of our industry, because for us because of our industry, because of the ticket, and because at the end of of the ticket, and because at the end of of the ticket, and because at the end of the day, the day, the day, customers need to build trust with with customers need to build trust with with customers need to build trust with with a company because they're buying a used a company because they're buying a used a company because they're buying a used car. car. car. And the way to build trust is to to know And the way to build trust is to to know And the way to build trust is to to know them and to and to plan and and nurture them and to and to plan and and nurture them and to and to plan and and nurture a long-term relationship. a long-term relationship. a long-term relationship. >> I like I just wanted to double-click on >> I like I just wanted to double-click on >> I like I just wanted to double-click on something, you know, evals over agent something, you know, evals over agent something, you know, evals over agent demos. Um you probably get pitched a lot demos. Um you probably get pitched a lot demos. Um you probably get pitched a lot of, you know agents and you know of, you know agents and you know of, you know agents and you know it's never been easier to build things it's never been easier to build things it's never been easier to build things like before but um one of the questions like before but um one of the questions like before but um one of the questions is like how do you how do you guys go is like how do you how do you guys go is like how do you how do you guys go about evaluating this because not about evaluating this because not about evaluating this because not everybody tests them across 90% of the everybody tests them across 90% of the everybody tests them across 90% of the customer interactions to see if they're customer interactions to see if they're customer interactions to see if they're really working and you know you guys I really working and you know you guys I really working and you know you guys I believe it's a about 98% of the believe it's a about 98% of the believe it's a about 98% of the interactions or something like interactions or something like interactions or something like >> Yes. >> Yes. >> Yes. >> that are now handled by agents. >> that are now handled by agents. >> that are now handled by agents. >> Yes, totally. So so so to give you a >> Yes, totally. So so so to give you a >> Yes, totally. So so so to give you a sense of the scale sense of the scale sense of the scale um 90 like 96% of all interactions uh um 90 like 96% of all interactions uh um 90 like 96% of all interactions uh are handled by agents. are handled by agents. are handled by agents. So so no humans there. So so no humans there. So so no humans there. Um like 95% of all transactions are Um like 95% of all transactions are Um like 95% of all transactions are completely handled by by agents. completely handled by by agents. completely handled by by agents. Obviously you meet a human when you pick Obviously you meet a human when you pick Obviously you meet a human when you pick up your car like there's someone up your car like there's someone up your car like there's someone physically there to give you the keys physically there to give you the keys physically there to give you the keys but the rest of the of the of the but the rest of the of the of the but the rest of the of the of the experience of the journey is handled by experience of the journey is handled by experience of the journey is handled by by an agent. by an agent. by an agent. Every day Every day Every day between 100,000 between 100,000 between 100,000 agents get instantiated in a day. They agents get instantiated in a day. They agents get instantiated in a day. They wake up. They work sometimes for 3 wake up. They work sometimes for 3 wake up. They work sometimes for 3 minutes, sometimes for 8 hours, minutes, sometimes for 8 hours, minutes, sometimes for 8 hours, sometimes for 3 days and they like set sometimes for 3 days and they like set sometimes for 3 days and they like set an alarm clock for their next task and an alarm clock for their next task and an alarm clock for their next task and they go back to sleep. So the scale of they go back to sleep. So the scale of they go back to sleep. So the scale of this this this is just is just amazing and and it's is just is just amazing and and it's is just is just amazing and and it's working. Now, how do you get this to working. Now, how do you get this to working. Now, how do you get this to work at scale? work at scale? work at scale? And the answer you mentioned it is is And the answer you mentioned it is is And the answer you mentioned it is is Evals. Like Evals. Like Evals. Like I like to move extremely fast I like to move extremely fast I like to move extremely fast but in order to move fast you need to but in order to move fast you need to but in order to move fast you need to have brakes, right? Imagine a car have brakes, right? Imagine a car have brakes, right? Imagine a car uh uh uh you'll hit on the gas just if you have you'll hit on the gas just if you have you'll hit on the gas just if you have the right brakes. the right brakes. the right brakes. And AI is super powerful and I've seen And AI is super powerful and I've seen And AI is super powerful and I've seen many companies get this wrong because many companies get this wrong because many companies get this wrong because they try to go slow they try to go slow they try to go slow because they they don't have the right because they they don't have the right because they they don't have the right brakes. So so I thought about it the brakes. So so I thought about it the brakes. So so I thought about it the other way around like how fast can we other way around like how fast can we other way around like how fast can we go? go? go? Well, it depends on the quality of our Well, it depends on the quality of our Well, it depends on the quality of our Evals. So a good rule of thumb here is Evals. So a good rule of thumb here is Evals. So a good rule of thumb here is we spend we spend we spend about the same amount of time about the same amount of time about the same amount of time engineer time engineer time engineer time tokens and and and money on building the tokens and and and money on building the tokens and and and money on building the evals, the building the agents. And this evals, the building the agents. And this evals, the building the agents. And this is how you get better and better and is how you get better and better and is how you get better and better and better. Not not letting evals as an better. Not not letting evals as an better. Not not letting evals as an afterthought. So, what do we measure? afterthought. So, what do we measure? afterthought. So, what do we measure? First and foremost, like the the the First and foremost, like the the the First and foremost, like the the the resource for the business. Like, if my resource for the business. Like, if my resource for the business. Like, if my customer is happy, customer is happy, customer is happy, they'll buy a car, they'll they'll get they'll buy a car, they'll they'll get they'll buy a car, they'll they'll get their loan approved, uh they'll sell a their loan approved, uh they'll sell a their loan approved, uh they'll sell a car to us. And and that's the like first car to us. And and that's the like first car to us. And and that's the like first check. Like, did it convert? check. Like, did it convert? check. Like, did it convert? And that's where most things break. And that's where most things break. And that's where most things break. Like, I I see companies Like, I I see companies Like, I I see companies like measuring number of calls or like measuring number of calls or like measuring number of calls or minutes during the call or or some like minutes during the call or or some like minutes during the call or or some like superficial KPIs that give you some superficial KPIs that give you some superficial KPIs that give you some information, but that doesn't really information, but that doesn't really information, but that doesn't really work. Like, the important thing is work. Like, the important thing is work. Like, the important thing is did this customer convert? Is it did this customer convert? Is it did this customer convert? Is it bringing value to the customer? And is bringing value to the customer? And is bringing value to the customer? And is the customer happy to reengage with us the customer happy to reengage with us the customer happy to reengage with us after a while? And once you get those after a while? And once you get those after a while? And once you get those evals connected, then it's just evals connected, then it's just evals connected, then it's just optimizing the right agent and the tech optimizing the right agent and the tech optimizing the right agent and the tech architecture and giving the agent skills architecture and giving the agent skills architecture and giving the agent skills to to scale this and and cater to to to scale this and and cater to to to scale this and and cater to millions of customers. millions of customers. millions of customers. >> It's really really amazing. And you >> It's really really amazing. And you >> It's really really amazing. And you know, related to this is like, okay, so know, related to this is like, okay, so know, related to this is like, okay, so you create the right evals, you know you create the right evals, you know you create the right evals, you know it's working. it's working. it's working. You know, some people some companies You know, some people some companies You know, some people some companies still feel a little bit risk averse in still feel a little bit risk averse in still feel a little bit risk averse in putting them in front of of of the putting them in front of of of the putting them in front of of of the customers and being able to perform customers and being able to perform customers and being able to perform >> Mhm. >> Mhm. >> Mhm. >> the highest leverage task, which in your >> the highest leverage task, which in your >> the highest leverage task, which in your case would be selling. Do your agents case would be selling. Do your agents case would be selling. Do your agents really sell to customers? really sell to customers? really sell to customers? >> Yes. So, so we never built customer >> Yes. So, so we never built customer >> Yes. So, so we never built customer support or customer service agents. We support or customer service agents. We support or customer service agents. We we we built like sales agents. It's we we built like sales agents. It's we we built like sales agents. It's extremely hard to sell a car in in Latin extremely hard to sell a car in in Latin extremely hard to sell a car in in Latin America. So, America. So, America. So, imagine someone wanting to buy a car, imagine someone wanting to buy a car, imagine someone wanting to buy a car, they can choose like among like 20,000 they can choose like among like 20,000 they can choose like among like 20,000 SKUs. Then they need to pick like SKUs. Then they need to pick like SKUs. Then they need to pick like financing and go through the financing financing and go through the financing financing and go through the financing process, insurance, and and coverage. process, insurance, and and coverage. process, insurance, and and coverage. And then they're probably trading in And then they're probably trading in And then they're probably trading in their car. So, so we need to quote that their car. So, so we need to quote that their car. So, so we need to quote that car. So, car. So, car. So, it's a process that if someone does it it's a process that if someone does it it's a process that if someone does it or or the way Kavak did it back in in in or or the way Kavak did it back in in in or or the way Kavak did it back in in in 2020 2021, was you need to be extremely 2020 2021, was you need to be extremely 2020 2021, was you need to be extremely good at 15 different things and have 15 good at 15 different things and have 15 good at 15 different things and have 15 different experts in 15 different teams. different experts in 15 different teams. different experts in 15 different teams. And usually the person would go and And usually the person would go and And usually the person would go and speak with the expert in financing, the speak with the expert in financing, the speak with the expert in financing, the expert in car car advisory, the expert expert in car car advisory, the expert expert in car car advisory, the expert in buying, the expert in insurance, and in buying, the expert in insurance, and in buying, the expert in insurance, and they'll build a package and buy a car. they'll build a package and buy a car. they'll build a package and buy a car. That's extremely hard to do. That's extremely hard to do. That's extremely hard to do. But, like But, like But, like the first thing we did was, okay, can we the first thing we did was, okay, can we the first thing we did was, okay, can we get an agent to be better than the get an agent to be better than the get an agent to be better than the expert in each of these things? And then expert in each of these things? And then expert in each of these things? And then put it together and have like a mega put it together and have like a mega put it together and have like a mega expert that's an expert in insurance, expert that's an expert in insurance, expert that's an expert in insurance, financing, etc. And that's who we put in financing, etc. And that's who we put in financing, etc. And that's who we put in front of the customer. So, the front of the customer. So, the front of the customer. So, the experience for the customer is amazing. experience for the customer is amazing. experience for the customer is amazing. We tripled uh NPS and customer We tripled uh NPS and customer We tripled uh NPS and customer satisfaction scored by putting the the satisfaction scored by putting the the satisfaction scored by putting the the agent in front of the of the customer. agent in front of the of the customer. agent in front of the of the customer. And it first it converted like 50% more And it first it converted like 50% more And it first it converted like 50% more than our human team, and now it's than our human team, and now it's than our human team, and now it's converting over that like converting over that like converting over that like 2.1 2.1 2.1 uh x uh x uh x more. So, it's a completely different more. So, it's a completely different more. So, it's a completely different company. company. company. >> agents are better sellers. >> agents are better sellers. >> agents are better sellers. >> Totally better. And and >> Totally better. And and >> Totally better. And and and you get this, right? Because they're and you get this, right? Because they're and you get this, right? Because they're experts and they're infinitely patient experts and they're infinitely patient experts and they're infinitely patient and they know all your history and they and they know all your history and they and they know all your history and they they they can plan for the long term and they they can plan for the long term and they they can plan for the long term and they never get tired. So, they never get tired. So, they never get tired. So, and if they make a mistake, they learn and if they make a mistake, they learn and if they make a mistake, they learn it and the next day, not just them, but it and the next day, not just them, but it and the next day, not just them, but the other 200,000 agents will have the other 200,000 agents will have the other 200,000 agents will have learned from that mistake. So, that's learned from that mistake. So, that's learned from that mistake. So, that's the feedback loop that we engaged and the feedback loop that we engaged and the feedback loop that we engaged and and that's showing in the growth and and that's showing in the growth and and that's showing in the growth and results and satisfaction of our results and satisfaction of our results and satisfaction of our customers. customers. customers. >> Yeah. >> Yeah. >> Yeah. One of the well, one of two of the very One of the well, one of two of the very One of the well, one of two of the very cool things I think about Kavak is I cool things I think about Kavak is I cool things I think about Kavak is I think I think the world has gotten think I think the world has gotten think I think the world has gotten comfortable with AI can do customer comfortable with AI can do customer comfortable with AI can do customer service. It's still very hard to do service. It's still very hard to do service. It's still very hard to do well. But, you know, as Gabe said, well. But, you know, as Gabe said, well. But, you know, as Gabe said, there's still a view that well, there's still a view that well, there's still a view that well, customers aren't going to want to buy customers aren't going to want to buy customers aren't going to want to buy expensive things from AI. And you are expensive things from AI. And you are expensive things from AI. And you are proving them wrong. proving them wrong. proving them wrong. >> Yes. >> Yes. >> Yes. >> The next layer on that is well, you're >> The next layer on that is well, you're >> The next layer on that is well, you're not actually going to be able to do not actually going to be able to do not actually going to be able to do regulated financial services end-to-end regulated financial services end-to-end regulated financial services end-to-end with AI. with AI. with AI. >> But if you walk through what you're >> But if you walk through what you're >> But if you walk through what you're doing, you are underwriting a thin or no doing, you are underwriting a thin or no doing, you are underwriting a thin or no file customer. file customer. file customer. >> Yes. >> Yes. >> Yes. >> Pricing them correctly. >> Pricing them correctly. >> Pricing them correctly. >> Yes. >> Yes. >> Yes. >> Doing servicing. So so maybe talk >> Doing servicing. So so maybe talk >> Doing servicing. So so maybe talk through how did you how did you write through how did you how did you write through how did you how did you write the evals to get comfortable with that? the evals to get comfortable with that? the evals to get comfortable with that? And then versus I don't know, going to a And then versus I don't know, going to a And then versus I don't know, going to a bank branch or or even a fintech. Sort bank branch or or even a fintech. Sort bank branch or or even a fintech. Sort of how how is that experience? of how how is that experience? of how how is that experience? >> Yes. >> Yes. >> Yes. >> That much better. >> That much better. >> That much better. >> the first financial product that that we >> the first financial product that that we >> the first financial product that that we launched was a car loan. And usually in launched was a car loan. And usually in launched was a car loan. And usually in in Mexico and in in some emerging in Mexico and in in some emerging in Mexico and in in some emerging markets, it'll get like markets, it'll get like markets, it'll get like two months or or or or more to get a car two months or or or or more to get a car two months or or or or more to get a car loan approved. loan approved. loan approved. >> Mhm. >> Mhm. >> Mhm. >> Um we usually approve it in under three >> Um we usually approve it in under three >> Um we usually approve it in under three minutes. minutes. minutes. Uh Uh Uh which is like pretty cool because we which is like pretty cool because we which is like pretty cool because we have all this data around the customer have all this data around the customer have all this data around the customer and the car. and the car. and the car. And if the customer can't pay for the And if the customer can't pay for the And if the customer can't pay for the car anymore, car anymore, car anymore, they'll just return it to us and we can they'll just return it to us and we can they'll just return it to us and we can give them a cheaper car. And and then give them a cheaper car. And and then give them a cheaper car. And and then they're pay an uh uh they're pay an uh uh they're pay an uh uh a smaller amount each month and they a smaller amount each month and they a smaller amount each month and they like get out of the water, which is like get out of the water, which is like get out of the water, which is amazing about the the vertical amazing about the the vertical amazing about the the vertical integration of of the business. integration of of the business. integration of of the business. But then like But then like But then like when we started launching other when we started launching other when we started launching other financial products, we we realized that financial products, we we realized that financial products, we we realized that this is a very important decision for this is a very important decision for this is a very important decision for the customer, right? Like like they the customer, right? Like like they the customer, right? Like like they usually take three to four months to usually take three to four months to usually take three to four months to make their make up their mind and buying make their make up their mind and buying make their make up their mind and buying a car and and and and getting a loan or a car and and and and getting a loan or a car and and and and getting a loan or getting a personal loan, like a large getting a personal loan, like a large getting a personal loan, like a large personal loan that that we also personal loan that that we also personal loan that that we also um do. So um do. So um do. So if you get to know your customer if you get to know your customer if you get to know your customer throughout this process and make the the throughout this process and make the the throughout this process and make the the the process easy for them, then just the process easy for them, then just the process easy for them, then just your conversion and retention metrics your conversion and retention metrics your conversion and retention metrics start going through the roof. It's not start going through the roof. It's not start going through the roof. It's not just the transaction, it's understanding just the transaction, it's understanding just the transaction, it's understanding each customer personally each customer personally each customer personally and get them to to convert when they're and get them to to convert when they're and get them to to convert when they're ready ready ready with a very deep personalization of the with a very deep personalization of the with a very deep personalization of the interest rate, the interest rate, the interest rate, the the risk, the max amount of the loan the risk, the max amount of the loan the risk, the max amount of the loan in a way that makes sense for the in a way that makes sense for the in a way that makes sense for the portfolio as a whole, obviously, but portfolio as a whole, obviously, but portfolio as a whole, obviously, but that's optimized to the risk level and that's optimized to the risk level and that's optimized to the risk level and and and and probably the other offers that the probably the other offers that the probably the other offers that the customer is getting. customer is getting. customer is getting. >> And then maybe give us just to be >> And then maybe give us just to be >> And then maybe give us just to be you know, Evals are always a very hot you know, Evals are always a very hot you know, Evals are always a very hot topic. You kind of led with that. What topic. You kind of led with that. What topic. You kind of led with that. What is like what is an example of maybe a a is like what is an example of maybe a a is like what is an example of maybe a a hard to design area for Evals or one hard to design area for Evals or one hard to design area for Evals or one where you had to spend extra amount of where you had to spend extra amount of where you had to spend extra amount of time? What's just given the fact that time? What's just given the fact that time? What's just given the fact that like there's real money PII at risk. like there's real money PII at risk. like there's real money PII at risk. Yeah. Yeah. Yeah. >> So >> So >> So when we decided to to to to redesign the when we decided to to to to redesign the when we decided to to to to redesign the company around the AI company around the AI company around the AI you ask you ask you ask the question, okay, is is AI going to be the question, okay, is is AI going to be the question, okay, is is AI going to be able able able to do this job, like even the CEO job or to do this job, like even the CEO job or to do this job, like even the CEO job or or jobs where the leadership is? or jobs where the leadership is? or jobs where the leadership is? And the answer honestly is probably yes, And the answer honestly is probably yes, And the answer honestly is probably yes, like in 2035 with a rate of improvement, like in 2035 with a rate of improvement, like in 2035 with a rate of improvement, it will be able to do so. We said, okay, it will be able to do so. We said, okay, it will be able to do so. We said, okay, let's let's try it now. let's let's try it now. let's let's try it now. Let's try and build an AI CEO. Let's try and build an AI CEO. Let's try and build an AI CEO. So So So we carved out a city in Mexico, it's we carved out a city in Mexico, it's we carved out a city in Mexico, it's it's Cuernavaca, and we put like an it's Cuernavaca, and we put like an it's Cuernavaca, and we put like an agent in one of our harnesses agent in one of our harnesses agent in one of our harnesses as a CEO and it starts learning and it as a CEO and it starts learning and it as a CEO and it starts learning and it starts making decisions and evaluating starts making decisions and evaluating starts making decisions and evaluating on those decisions. And it's only been on those decisions. And it's only been on those decisions. And it's only been running for for 6 weeks now. running for for 6 weeks now. running for for 6 weeks now. The goal of the first month was to The goal of the first month was to The goal of the first month was to double the the the profits of of double the the the profits of of double the the the profits of of Cuernavaca. It didn't reach it, but it Cuernavaca. It didn't reach it, but it Cuernavaca. It didn't reach it, but it was 1.5x, was 1.5x, was 1.5x, like 50% more profits. Just like like 50% more profits. Just like like 50% more profits. Just like managing the the city, which is it's managing the the city, which is it's managing the the city, which is it's crazy, right? It's it's amazing and and crazy, right? It's it's amazing and and crazy, right? It's it's amazing and and it's it's a CEO, like people were it's it's a CEO, like people were it's it's a CEO, like people were like that was the last job AI was like that was the last job AI was like that was the last job AI was supposed to stick, and no, it isn't, supposed to stick, and no, it isn't, supposed to stick, and no, it isn't, really. And really. And really. And how did this happen? And and how did this happen? And and how did this happen? And and it's like it's like it's like uh very smart person, like like Fields uh very smart person, like like Fields uh very smart person, like like Fields Medal level smart, like going into every Medal level smart, like going into every Medal level smart, like going into every single number, every single customer, single number, every single customer, single number, every single customer, making the perfect forecast, and going making the perfect forecast, and going making the perfect forecast, and going to micromanage every single things that to micromanage every single things that to micromanage every single things that needs to be executed every day needs to be executed every day needs to be executed every day to reach a plan. So, he'll literally to reach a plan. So, he'll literally to reach a plan. So, he'll literally send messages to all the physical send messages to all the physical send messages to all the physical workers in in Cuernavaca with their workers in in Cuernavaca with their workers in in Cuernavaca with their plans for the day and ask them to send plans for the day and ask them to send plans for the day and ask them to send voice notes back voice notes back voice notes back to to know their their progress. to to know their their progress. to to know their their progress. So, customer satisfaction grew, So, customer satisfaction grew, So, customer satisfaction grew, uh we got a better inventory, we rotated uh we got a better inventory, we rotated uh we got a better inventory, we rotated better, better financing penetration, better, better financing penetration, better, better financing penetration, like every KPI started to to improve. like every KPI started to to improve. like every KPI started to to improve. Um so, it's super cool, it's super Um so, it's super cool, it's super Um so, it's super cool, it's super exciting. Now, what are the jobs where where we think what are the jobs where where we think uh uh uh we're still like training and hiring we're still like training and hiring we're still like training and hiring humans. humans. humans. Those are related to the physical world. Those are related to the physical world. Those are related to the physical world. So, when we talk about mechanics, Kavak So, when we talk about mechanics, Kavak So, when we talk about mechanics, Kavak has around I think in Mexico around 800 has around I think in Mexico around 800 has around I think in Mexico around 800 uh mechanics. There's lots of dexterity uh mechanics. There's lots of dexterity uh mechanics. There's lots of dexterity and and senses that's super hard to and and senses that's super hard to and and senses that's super hard to substitute. So, there we also build this substitute. So, there we also build this substitute. So, there we also build this agents with exact same harness that's agents with exact same harness that's agents with exact same harness that's scaling. scaling. scaling. And the the mechanics have the psychic. And the the mechanics have the psychic. And the the mechanics have the psychic. Um I was telling you guys earlier, it's Um I was telling you guys earlier, it's Um I was telling you guys earlier, it's like like like the the movie Ratatouille, like the the the the movie Ratatouille, like the the the the movie Ratatouille, like the the mouse that's actually a chef mouse that's actually a chef mouse that's actually a chef collaborating with a with a human. collaborating with a with a human. collaborating with a with a human. It's kind of like that. So, it's a It's kind of like that. So, it's a It's kind of like that. So, it's a psychic, we call it El Mike, and it psychic, we call it El Mike, and it psychic, we call it El Mike, and it tells them how to inspect a car and tells them how to inspect a car and tells them how to inspect a car and gives them tips and and shows them the gives them tips and and shows them the gives them tips and and shows them the way to to do it, and way to to do it, and way to to do it, and the quality of inspections, again, went the quality of inspections, again, went the quality of inspections, again, went through the roof. We're inspecting through the roof. We're inspecting through the roof. We're inspecting faster, we're repairing faster, it's faster, we're repairing faster, it's faster, we're repairing faster, it's cheaper, but most importantly, we're cheaper, but most importantly, we're cheaper, but most importantly, we're delivering higher quality cars. delivering higher quality cars. delivering higher quality cars. Um Um Um warranties came down around like 20 26% warranties came down around like 20 26% warranties came down around like 20 26% since we since we launched and customer since we since we launched and customer since we since we launched and customer satisfaction again satisfaction again satisfaction again went up. So, went up. So, went up. So, it's about this. Like how would you it's about this. Like how would you it's about this. Like how would you design your organization from scratch design your organization from scratch design your organization from scratch with with with with with with with abundant super intelligence that's with abundant super intelligence that's with abundant super intelligence that's that's cheap and just go build it. that's cheap and just go build it. that's cheap and just go build it. >> Now, like this this is a good segue to >> Now, like this this is a good segue to >> Now, like this this is a good segue to key topic right now in Silicon Valley key topic right now in Silicon Valley key topic right now in Silicon Valley where you know, there's a lot of people where you know, there's a lot of people where you know, there's a lot of people worried about how the organizations of worried about how the organizations of worried about how the organizations of the future are going to look like and the future are going to look like and the future are going to look like and and the the role that humans are going and the the role that humans are going and the the role that humans are going to play. to play. to play. >> Yes. And I think you touched a little >> Yes. And I think you touched a little >> Yes. And I think you touched a little bit on that. So, we'd love to hear yeah, bit on that. So, we'd love to hear yeah, bit on that. So, we'd love to hear yeah, like how you guys are thinking about like how you guys are thinking about like how you guys are thinking about that and yes. that and yes. that and yes. >> And >> And >> And the organizations. the organizations. the organizations. >> Yeah. >> Yeah. >> Yeah. Totally. So, um Totally. So, um Totally. So, um we we took that question very seriously we we took that question very seriously we we took that question very seriously 3 years ago and the truth is that 3 years ago and the truth is that 3 years ago and the truth is that everyone's everyone's everyone's job will change. job will change. job will change. So, and and what we were doing a couple of years ago what we were doing a couple of years ago what we were doing a couple of years ago will probably be be be performed better will probably be be be performed better will probably be be be performed better by an AI agent. Right? So, what does by an AI agent. Right? So, what does by an AI agent. Right? So, what does this mean? this mean? this mean? We need to train everyone. So, so we We need to train everyone. So, so we We need to train everyone. So, so we launched a program inside Kavak that's launched a program inside Kavak that's launched a program inside Kavak that's called the Jedi Academy where anyone called the Jedi Academy where anyone called the Jedi Academy where anyone from Kavak like from the CEO from Kavak like from the CEO from Kavak like from the CEO >> We love this. >> We love this. >> We love this. >> to Yeah, and it tells them like from the >> to Yeah, and it tells them like from the >> to Yeah, and it tells them like from the CEO to like AI engineers to mechanics CEO to like AI engineers to mechanics CEO to like AI engineers to mechanics like go into the academy. It's super like go into the academy. It's super like go into the academy. It's super hard. Like I I I've hard. Like I I I've hard. Like I I I've I led I led I led I can you I can you I can you >> You got that myself. >> You got that myself. >> You got that myself. >> You designed the program. >> You designed the program. >> You designed the program. >> I designed the program. >> I designed the program. >> I designed the program. >> And but constantly >> And but constantly >> And but constantly >> Constantly because you >> Constantly because you >> Constantly because you you need to be upgrading the the program you need to be upgrading the the program you need to be upgrading the the program because everything's changing so fast. because everything's changing so fast. because everything's changing so fast. And there there's And there there's And there there's like you can't send these people like like you can't send these people like like you can't send these people like outside to Stanford to to learn this outside to Stanford to to learn this outside to Stanford to to learn this because like it's new stuff, right? So, because like it's new stuff, right? So, because like it's new stuff, right? So, we train everyone and after 6 weeks we train everyone and after 6 weeks we train everyone and after 6 weeks they launch state-of-the-art they launch state-of-the-art they launch state-of-the-art um agents AI agents to production. um agents AI agents to production. um agents AI agents to production. And it's And it's And it's mechanics and and finance guys and mechanics and and finance guys and mechanics and and finance guys and engineers, like everyone can do it. engineers, like everyone can do it. engineers, like everyone can do it. And what this generated is maybe this And what this generated is maybe this And what this generated is maybe this person won't become an AI engineer, some person won't become an AI engineer, some person won't become an AI engineer, some of them have, of them have, of them have, but they they know how to collaborate but they they know how to collaborate but they they know how to collaborate with this new technology, right? So, the with this new technology, right? So, the with this new technology, right? So, the way we looked about it was guys, there's way we looked about it was guys, there's way we looked about it was guys, there's no way back. Like this is the way Kavak no way back. Like this is the way Kavak no way back. Like this is the way Kavak is going. This is the way the company is going. This is the way the company is going. This is the way the company will look like. These are the changes will look like. These are the changes will look like. These are the changes for the engineering team, the finance for the engineering team, the finance for the engineering team, the finance team, the product team. Like this is team, the product team. Like this is team, the product team. Like this is what's going to change. what's going to change. what's going to change. You have the choice to like train and You have the choice to like train and You have the choice to like train and and get the skills to perform in this and get the skills to perform in this and get the skills to perform in this new reality in this new world, um or new reality in this new world, um or new reality in this new world, um or maybe leave Kavak if this is not for maybe leave Kavak if this is not for maybe leave Kavak if this is not for you, but this is the way we're going. you, but this is the way we're going. you, but this is the way we're going. And And And like we're great. Like like we we we like we're great. Like like we we we like we're great. Like like we we we strengthened the culture, and we were strengthened the culture, and we were strengthened the culture, and we were super excited. Um super excited. Um super excited. Um people really know how to build these people really know how to build these people really know how to build these agentic systems. And then, if you look agentic systems. And then, if you look agentic systems. And then, if you look at Kavak now, any process is really a at Kavak now, any process is really a at Kavak now, any process is really a collaboration of agents and humans, and collaboration of agents and humans, and collaboration of agents and humans, and sometimes like agents are the bosses or sometimes like agents are the bosses or sometimes like agents are the bosses or of of humans, and sometimes humans are of of humans, and sometimes humans are of of humans, and sometimes humans are designing the agents, but designing the agents, but designing the agents, but like we we we managed to really build like we we we managed to really build like we we we managed to really build this and and change this. this and and change this. this and and change this. Um Um Um And it's through this And it's through this And it's through this idea that we need to be learning every idea that we need to be learning every idea that we need to be learning every day, and things will continue to change. day, and things will continue to change. day, and things will continue to change. And the only way to to continue being And the only way to to continue being And the only way to to continue being relevant is to relevant is to relevant is to upgrade your skills uh every month or upgrade your skills uh every month or upgrade your skills uh every month or every couple months. every couple months. every couple months. >> But you do have or did have, you know, >> But you do have or did have, you know, >> But you do have or did have, you know, thousands of people. Now agents do most thousands of people. Now agents do most thousands of people. Now agents do most things. things. things. >> Yes. >> Yes. >> Yes. >> So, like what is the org structure of >> So, like what is the org structure of >> So, like what is the org structure of Kavak? Like does the middle management Kavak? Like does the middle management Kavak? Like does the middle management concept even exist anymore? Like what concept even exist anymore? Like what concept even exist anymore? Like what does your org look like? does your org look like? does your org look like? >> Right. So, >> Right. So, >> Right. So, the way it looks like now is very flat the way it looks like now is very flat the way it looks like now is very flat teams, very senior teams, super teams, very senior teams, super teams, very senior teams, super empowered. empowered. empowered. If you look at a team, you'll have If you look at a team, you'll have If you look at a team, you'll have engineering, AI, like operations, like engineering, AI, like operations, like engineering, AI, like operations, like everything. And they're either everything. And they're either everything. And they're either building the agents, building the agents, building the agents, working for the agents, working for the agents, working for the agents, or being in the physical world in front or being in the physical world in front or being in the physical world in front of the customer. Like most of our of the customer. Like most of our of the customer. Like most of our organization looks like that. So, so organization looks like that. So, so organization looks like that. So, so it's really built it's really built it's really built around around around um um um a around the idea of how organizations a around the idea of how organizations a around the idea of how organizations will look like in the future and the will look like in the future and the will look like in the future and the round AI and really harnessing this this round AI and really harnessing this this round AI and really harnessing this this new technology. new technology. new technology. Obviously, this required lots of Obviously, this required lots of Obviously, this required lots of retraining because in 2023 retraining because in 2023 retraining because in 2023 or 2022, no one was building agents, no or 2022, no one was building agents, no or 2022, no one was building agents, no one was helping agents or or taking one was helping agents or or taking one was helping agents or or taking orders from from agents. And the way you orders from from agents. And the way you orders from from agents. And the way you cater to the physical world or the cater to the physical world or the cater to the physical world or the customers was customers was customers was in a different way than if an agent's in a different way than if an agent's in a different way than if an agent's telling you what to do or helping you telling you what to do or helping you telling you what to do or helping you make your job uh better. make your job uh better. make your job uh better. >> Yeah. >> Yeah. >> Yeah. >> Um so, it's a completely different >> Um so, it's a completely different >> Um so, it's a completely different structure than than we had just 2 years structure than than we had just 2 years structure than than we had just 2 years ago. ago. ago. >> Yeah. Explain um we talked about this >> Yeah. Explain um we talked about this >> Yeah. Explain um we talked about this before what working for the agents look before what working for the agents look before what working for the agents look like. Like I think the way you described like. Like I think the way you described like. Like I think the way you described it was the agentic system and then it was the agentic system and then it was the agentic system and then sometimes when it fails, it's like, "Oh, sometimes when it fails, it's like, "Oh, sometimes when it fails, it's like, "Oh, let's kick out to kind of a human let's kick out to kind of a human let's kick out to kind of a human queue." But then that's loss. And so, queue." But then that's loss. And so, queue." But then that's loss. And so, how have you brought that together? how have you brought that together? how have you brought that together? >> So, so >> So, so >> So, so like the the way we see human in the like the the way we see human in the like the the way we see human in the loops and and and most of these agentic loops and and and most of these agentic loops and and and most of these agentic systems in production right now, like systems in production right now, like systems in production right now, like large-scale agentic systems, usually if large-scale agentic systems, usually if large-scale agentic systems, usually if an agent hits a wall or can't perform an agent hits a wall or can't perform an agent hits a wall or can't perform anymore, anymore, anymore, it'll it'll it'll like send uh like send uh like send uh this case or this customer to a tier two this case or this customer to a tier two this case or this customer to a tier two support and forget about it. support and forget about it. support and forget about it. That doesn't really work because you That doesn't really work because you That doesn't really work because you don't close the loops, so you don't don't close the loops, so you don't don't close the loops, so you don't generate the data to train the agent to generate the data to train the agent to generate the data to train the agent to do this better. do this better. do this better. What works right now is we have an agent What works right now is we have an agent What works right now is we have an agent that's obsessed with each of the that's obsessed with each of the that's obsessed with each of the customers, like millions of this. customers, like millions of this. customers, like millions of this. They have access to every single API, They have access to every single API, They have access to every single API, every single skill, like and we have every single skill, like and we have every single skill, like and we have agents building those humans building agents building those humans building agents building those humans building those skills for them. those skills for them. those skills for them. And then if an agent hits a wall or And then if an agent hits a wall or And then if an agent hits a wall or cancels something, it'll call this API cancels something, it'll call this API cancels something, it'll call this API saying I need help. saying I need help. saying I need help. And on the other side, it's not an agent And on the other side, it's not an agent And on the other side, it's not an agent or software, it's a human or software, it's a human or software, it's a human helping them out. But if you map this helping them out. But if you map this helping them out. But if you map this out in an org chart, it's really human out in an org chart, it's really human out in an org chart, it's really human teams that have an agent I'm getting teams that have an agent I'm getting teams that have an agent I'm getting better results. It's super clear, like better results. It's super clear, like better results. It's super clear, like it makes sense. it makes sense. it makes sense. >> That's actually a perfect segue. And I >> That's actually a perfect segue. And I >> That's actually a perfect segue. And I know you get lots of of leaders at know you get lots of of leaders at know you get lots of of leaders at larger institutions inbounding to you. larger institutions inbounding to you. larger institutions inbounding to you. So so maybe this will save you many So so maybe this will save you many So so maybe this will save you many phone calls. But I think phone calls. But I think phone calls. But I think rationally many leaders of companies rationally many leaders of companies rationally many leaders of companies intuitively understand this. It is very intuitively understand this. It is very intuitively understand this. It is very hard still to deploy AI through the hard still to deploy AI through the hard still to deploy AI through the organization. Like the models are good organization. Like the models are good organization. Like the models are good enough. You know that. It's a it's an enough. You know that. It's a it's an enough. You know that. It's a it's an org problem. It's a psychology problem. org problem. It's a psychology problem. org problem. It's a psychology problem. Like what advice do you have or what Like what advice do you have or what Like what advice do you have or what have you seen? have you seen? have you seen? >> I think it's two things. >> I think it's two things. >> I think it's two things. Uh the first is Uh the first is Uh the first is it has to be top-down it has to be top-down it has to be top-down because of this. Like because of this. Like because of this. Like if you just get adoption, it won't go if you just get adoption, it won't go if you just get adoption, it won't go anywhere because it's it's hard to anywhere because it's it's hard to anywhere because it's it's hard to generate this taste or a strategy for generate this taste or a strategy for generate this taste or a strategy for people to to bottom-up decide what to people to to bottom-up decide what to people to to bottom-up decide what to build or what not and and come up with build or what not and and come up with build or what not and and come up with something that works for the company. So something that works for the company. So something that works for the company. So the transformation has to be top-down the transformation has to be top-down the transformation has to be top-down and leaders need to adopt and leaders and leaders need to adopt and leaders and leaders need to adopt and leaders have to have a very clear have to have a very clear have to have a very clear plan on what to build. I've seen so many plan on what to build. I've seen so many plan on what to build. I've seen so many companies that's just like, oh, like companies that's just like, oh, like companies that's just like, oh, like we're doing a hackathon. People are we're doing a hackathon. People are we're doing a hackathon. People are coming up with use cases. We're coming up with use cases. We're coming up with use cases. We're sponsoring some of these use cases. That sponsoring some of these use cases. That sponsoring some of these use cases. That doesn't work. It's doesn't work. It's doesn't work. It's like like like be very clear on what the company will be very clear on what the company will be very clear on what the company will look like in three or five years. look like in three or five years. look like in three or five years. And then start building that. And be And then start building that. And be And then start building that. And be like very vertical in in guiding your like very vertical in in guiding your like very vertical in in guiding your troops towards that. Like an army troops towards that. Like an army troops towards that. Like an army doesn't really work if everyone comes up doesn't really work if everyone comes up doesn't really work if everyone comes up with ideas on the strategy and tactics with ideas on the strategy and tactics with ideas on the strategy and tactics and goes to the battlefield and and goes to the battlefield and and goes to the battlefield and like does whatever they want. Like you like does whatever they want. Like you like does whatever they want. Like you need a very clear strategy. need a very clear strategy. need a very clear strategy. And that's what we need now. It's a like And that's what we need now. It's a like And that's what we need now. It's a like transformation stage. transformation stage. transformation stage. The second one is you need to measure The second one is you need to measure The second one is you need to measure what what really matters. And it's what what really matters. And it's what what really matters. And it's evals, but it's also the right evals. evals, but it's also the right evals. evals, but it's also the right evals. So, So, So, I see a lot of companies spending now I see a lot of companies spending now I see a lot of companies spending now huge amounts and they say, "Okay, I got huge amounts and they say, "Okay, I got huge amounts and they say, "Okay, I got adoption. I'm just spending like adoption. I'm just spending like adoption. I'm just spending like hundreds of millions of dollars in in hundreds of millions of dollars in in hundreds of millions of dollars in in tokens now. What about that? Like, tokens now. What about that? Like, tokens now. What about that? Like, there's quality in the tokens." So, I there's quality in the tokens." So, I there's quality in the tokens." So, I have a framework here that's also have a framework here that's also have a framework here that's also useful. Like, level tier three tokens, useful. Like, level tier three tokens, useful. Like, level tier three tokens, the most valuable, the most valuable, the most valuable, are these agents where you can are these agents where you can are these agents where you can get the ROI of each specific token. And get the ROI of each specific token. And get the ROI of each specific token. And I can do that now. That's great news for I can do that now. That's great news for I can do that now. That's great news for me. me. me. Because I'm growing and because I know Because I'm growing and because I know Because I'm growing and because I know the ROI of each token, because it goes the ROI of each token, because it goes the ROI of each token, because it goes to agents that are to agents that are to agents that are performing the job of the organization, performing the job of the organization, performing the job of the organization, right? These are the best tokens. right? These are the best tokens. right? These are the best tokens. Tier two tokens are Tier two tokens are Tier two tokens are things that you can measure indirectly. things that you can measure indirectly. things that you can measure indirectly. Do I see devs in the in the code base Do I see devs in the in the code base Do I see devs in the in the code base and I can evaluate the value of these and I can evaluate the value of these and I can evaluate the value of these tokens at least indirectly and then push tokens at least indirectly and then push tokens at least indirectly and then push those to production. those to production. those to production. Tier one, when most companies are, is Tier one, when most companies are, is Tier one, when most companies are, is people are just just saying, "Click code people are just just saying, "Click code people are just just saying, "Click code or ChatGPT or code work or whatever. or ChatGPT or code work or whatever. or ChatGPT or code work or whatever. What happened with those?" I have no What happened with those?" I have no What happened with those?" I have no idea. idea. idea. So, it's not just about adoption. It's So, it's not just about adoption. It's So, it's not just about adoption. It's really about having a very clear vision really about having a very clear vision really about having a very clear vision and then measuring that each token you and then measuring that each token you and then measuring that each token you spend is bringing you those benefits and spend is bringing you those benefits and spend is bringing you those benefits and just iterate, iterate, iterate from from just iterate, iterate, iterate from from just iterate, iterate, iterate from from there. there. there. >> And then one of you we touched on this a >> And then one of you we touched on this a >> And then one of you we touched on this a little bit, but I think it's worth a little bit, but I think it's worth a little bit, but I think it's worth a dive as dive as dive as maybe the the most ambitious companies maybe the the most ambitious companies maybe the the most ambitious companies listening to this will decide to follow listening to this will decide to follow listening to this will decide to follow suit, which is you decided to build an suit, which is you decided to build an suit, which is you decided to build an agent per customer agent per customer agent per customer >> Yes. >> Yes. >> Yes. >> versus per task. And then discovered >> versus per task. And then discovered >> versus per task. And then discovered along the way that each one of those along the way that each one of those along the way that each one of those agents needs its own kind of micro agents needs its own kind of micro agents needs its own kind of micro virtual machine. So, maybe kind of walk virtual machine. So, maybe kind of walk virtual machine. So, maybe kind of walk us through us through us through >> Yes. >> Yes. >> Yes. >> those decisions that architecture. >> those decisions that architecture. >> those decisions that architecture. >> And and >> And and >> And and I think the the I think the the I think the the we're seeing these results now, but it we're seeing these results now, but it we're seeing these results now, but it was a really risky bet because was a really risky bet because was a really risky bet because people usually go from workflows. people usually go from workflows. people usually go from workflows. >> Mhm. >> Mhm. >> Mhm. >> And like, if I could advise everyone, >> And like, if I could advise everyone, >> And like, if I could advise everyone, don't build agent workflows to graphs or don't build agent workflows to graphs or don't build agent workflows to graphs or or functions or or objectives. And we or functions or or objectives. And we or functions or or objectives. And we built that that these are multi-agent built that that these are multi-agent built that that these are multi-agent systems that can perform systems that can perform systems that can perform a whole function for a complex goals. a whole function for a complex goals. a whole function for a complex goals. Like the ones I told you that to sell a Like the ones I told you that to sell a Like the ones I told you that to sell a car you need to do financing, car you need to do financing, car you need to do financing, purchasing, like recommendations, etc. purchasing, like recommendations, etc. purchasing, like recommendations, etc. And we had thousands, like tens of And we had thousands, like tens of And we had thousands, like tens of thousands of these agents working at thousands of these agents working at thousands of these agents working at scale running the business back in scale running the business back in scale running the business back in December. December. December. But then But then But then Opus 4.5 came out. Opus 4.5 came out. Opus 4.5 came out. And I realized like this isn't the right And I realized like this isn't the right And I realized like this isn't the right paradigm anymore. paradigm anymore. paradigm anymore. Like the the intelligence now doesn't Like the the intelligence now doesn't Like the the intelligence now doesn't need like the graph and the multi-agent need like the graph and the multi-agent need like the graph and the multi-agent lattice work and harness because it will lattice work and harness because it will lattice work and harness because it will constrain constrain constrain this level of intelligence. this level of intelligence. this level of intelligence. So we decided to like So we decided to like So we decided to like destroy everything we had been building destroy everything we had been building destroy everything we had been building for for two years that was working, that for for two years that was working, that for for two years that was working, that brought us to profitability, that brought us to profitability, that brought us to profitability, that brought us amazing growth. brought us amazing growth. brought us amazing growth. And start over with a harness that we And start over with a harness that we And start over with a harness that we thought would be robust and scalable and thought would be robust and scalable and thought would be robust and scalable and and and and leverage leverage leverage recursive self-improvement. And or new recursive self-improvement. And or new recursive self-improvement. And or new models, more intelligent models coming models, more intelligent models coming models, more intelligent models coming out every month. So the way this looks out every month. So the way this looks out every month. So the way this looks like it's it's a virtual machine like it's it's a virtual machine like it's it's a virtual machine with an agent with access to memory and with an agent with access to memory and with an agent with access to memory and evals and a CLI where they can access evals and a CLI where they can access evals and a CLI where they can access every tool and every API in my company every tool and every API in my company every tool and every API in my company and the long-term goal. and the long-term goal. and the long-term goal. And I instantiate hundreds of thousands And I instantiate hundreds of thousands And I instantiate hundreds of thousands of these each day with long-term goals. of these each day with long-term goals. of these each day with long-term goals. Like maximizing the the lifetime value. Like maximizing the the lifetime value. Like maximizing the the lifetime value. >> Yeah. The self-improving organization. >> Yeah. The self-improving organization. >> Yeah. The self-improving organization. >> Exactly. The self-improving >> Exactly. The self-improving >> Exactly. The self-improving organization. And I think people are organization. And I think people are organization. And I think people are super obsessed with with RSI now super obsessed with with RSI now super obsessed with with RSI now and this will improve the the and this will improve the the and this will improve the the the models, but if you look at it this the models, but if you look at it this the models, but if you look at it this way, way, way, economic value economic value economic value in humanity for the past 4,000 years in humanity for the past 4,000 years in humanity for the past 4,000 years has been delivered by organizations, not has been delivered by organizations, not has been delivered by organizations, not by individuals. So, what you want by individuals. So, what you want by individuals. So, what you want to self-improve and to engage in that to self-improve and to engage in that to self-improve and to engage in that loop is the organization that can loop is the organization that can loop is the organization that can deliver more economic value. Right? So, deliver more economic value. Right? So, deliver more economic value. Right? So, that's the loop that I think companies that's the loop that I think companies that's the loop that I think companies will start to to focus on because if you will start to to focus on because if you will start to to focus on because if you get that loop working and it's an get that loop working and it's an get that loop working and it's an organization that is really organization that is really organization that is really self-improving self-improving self-improving and harnessing the the newer models and and harnessing the the newer models and and harnessing the the newer models and the better intelligence that we're the better intelligence that we're the better intelligence that we're getting every couple of days now, getting every couple of days now, getting every couple of days now, uh uh uh then like you hit the exponential not then like you hit the exponential not then like you hit the exponential not just in intelligence, just in intelligence, just in intelligence, but in the value uh that you can but in the value uh that you can but in the value uh that you can generate as a company. So, so that's generate as a company. So, so that's generate as a company. So, so that's really exciting. That's what we're really exciting. That's what we're really exciting. That's what we're working on. working on. working on. >> You mentioned that because of all the >> You mentioned that because of all the >> You mentioned that because of all the challenges on adopting AI, you saw the challenges on adopting AI, you saw the challenges on adopting AI, you saw the biggest opportunity on net new companies biggest opportunity on net new companies biggest opportunity on net new companies being formed being formed being formed working on this new way and then working on this new way and then working on this new way and then disrupting markets. Like you want to disrupting markets. Like you want to disrupting markets. Like you want to talk a little bit about that? talk a little bit about that? talk a little bit about that? >> Yes, um >> Yes, um >> Yes, um there's this concept in economics about there's this concept in economics about there's this concept in economics about creative destruction from from from creative destruction from from from creative destruction from from from Joseph Joseph Joseph Schumpeter. Schumpeter. Schumpeter. And what it says is that the the the way And what it says is that the the the way And what it says is that the the the way innovation hits the economy isn't by innovation hits the economy isn't by innovation hits the economy isn't by companies companies companies adopting the the the new technology, adopting the the the new technology, adopting the the the new technology, but by companies remaining the way they but by companies remaining the way they but by companies remaining the way they they were and incumbents with the new they were and incumbents with the new they were and incumbents with the new technology technology technology destroying destroying destroying the old companies. So, this destroys the old companies. So, this destroys the old companies. So, this destroys value in the short term in the economy, value in the short term in the economy, value in the short term in the economy, but in the long term it's better for but in the long term it's better for but in the long term it's better for everyone because of these new everyone because of these new everyone because of these new more efficient, more effective uh more efficient, more effective uh more efficient, more effective uh companies will provide better products companies will provide better products companies will provide better products and services for the for the economy as and services for the for the economy as and services for the for the economy as a whole. a whole. a whole. And this has happened in the past like And this has happened in the past like And this has happened in the past like industrial revolutions and this has industrial revolutions and this has industrial revolutions and this has always happened. And always happened. And always happened. And this is a great opportunity for this is a great opportunity for this is a great opportunity for entrepreneurs and and people today entrepreneurs and and people today entrepreneurs and and people today because because because it's hard to adopt it's hard to adopt it's hard to adopt AI deeply. AI deeply. AI deeply. Uh it's really hard for a CEO today, Uh it's really hard for a CEO today, Uh it's really hard for a CEO today, especially of a large company or public especially of a large company or public especially of a large company or public company, to go and say, "Hey, like I'm company, to go and say, "Hey, like I'm company, to go and say, "Hey, like I'm betting everything on AI. betting everything on AI. betting everything on AI. Uh the company has to look this way. Uh the company has to look this way. Uh the company has to look this way. I'll I'll I'll like destroy and rebuild everything I've like destroy and rebuild everything I've like destroy and rebuild everything I've been building for the past 40 years been building for the past 40 years been building for the past 40 years to become an AI native company. Like how to become an AI native company. Like how to become an AI native company. Like how many CEOs will will will do that in a many CEOs will will will do that in a many CEOs will will will do that in a company at scale? So, while they they company at scale? So, while they they company at scale? So, while they they adopt adopt adopt new companies can be formed that are new companies can be formed that are new companies can be formed that are built around the the the strengths of of built around the the the strengths of of built around the the the strengths of of AI AI AI and and take over and and and and bring and and take over and and and and bring and and take over and and and and bring new products and services to to to the new products and services to to to the new products and services to to to the masses. And and this has happened masses. And and this has happened masses. And and this has happened before. Like this happened with before. Like this happened with before. Like this happened with electricity. This is the story I always electricity. This is the story I always electricity. This is the story I always tell tell my team. tell tell my team. tell tell my team. The the the technologies for Ford's The the the technologies for Ford's The the the technologies for Ford's production line were developed in 1879 production line were developed in 1879 production line were developed in 1879 and 1881. Edison started commercializing and 1881. Edison started commercializing and 1881. Edison started commercializing electricity in in New York and then electricity in in New York and then electricity in in New York and then London and he invented a dynamo that was London and he invented a dynamo that was London and he invented a dynamo that was extremely efficient. So, you could have extremely efficient. So, you could have extremely efficient. So, you could have built built built Ford's factory Ford's factory Ford's factory 40 years before Ford. The technology was 40 years before Ford. The technology was 40 years before Ford. The technology was there. Everything was there. But the way there. Everything was there. But the way there. Everything was there. But the way people adopted electricity and and people adopted electricity and and people adopted electricity and and Ford's dynamo was okay. I'm going to Ford's dynamo was okay. I'm going to Ford's dynamo was okay. I'm going to leave my factory like four floors, leave my factory like four floors, leave my factory like four floors, shafts, and belts. I just change my coal shafts, and belts. I just change my coal shafts, and belts. I just change my coal engine for an electric engine. And this engine for an electric engine. And this engine for an electric engine. And this will bring you benefits. Yes, but like will bring you benefits. Yes, but like will bring you benefits. Yes, but like 6% efficiency. 6% efficiency. 6% efficiency. What needed to be done was like to What needed to be done was like to What needed to be done was like to destroy that factory, destroy that factory, destroy that factory, build it in a flat surface, not in the build it in a flat surface, not in the build it in a flat surface, not in the center of New York, but in Connecticut center of New York, but in Connecticut center of New York, but in Connecticut or New Jersey. or New Jersey. or New Jersey. And redesign your coal factory around And redesign your coal factory around And redesign your coal factory around small dynamos and and electricity. And small dynamos and and electricity. And small dynamos and and electricity. And then you get like the 3x uh improvement then you get like the 3x uh improvement then you get like the 3x uh improvement in productivity that that like powered in productivity that that like powered in productivity that that like powered the US during the 20th century. And the the US during the 20th century. And the the US during the 20th century. And the same happened again with a computer. And same happened again with a computer. And same happened again with a computer. And the same is happening again today. the same is happening again today. the same is happening again today. People want to adopt it, but they're not People want to adopt it, but they're not People want to adopt it, but they're not willing to redesign the whole company. willing to redesign the whole company. willing to redesign the whole company. And they just adopt it superficially. And they just adopt it superficially. And they just adopt it superficially. And in the end that'll give you a 6% or And in the end that'll give you a 6% or And in the end that'll give you a 6% or a 10% improvement, not a 10x a 10% improvement, not a 10x a 10% improvement, not a 10x improvement. And it's like the improvement. And it's like the improvement. And it's like the innovator's dilemma innovator's dilemma innovator's dilemma at an industrial at an industrial at an industrial uh scale again. uh scale again. uh scale again. >> I think you've just made an amazing case >> I think you've just made an amazing case >> I think you've just made an amazing case for any future founders out there that for any future founders out there that for any future founders out there that it's time to build. it's time to build. it's time to build. And and maybe a great a great place to And and maybe a great a great place to And and maybe a great a great place to to end this, you know, you've built and to end this, you know, you've built and to end this, you know, you've built and scaled your own company, you've now scaled your own company, you've now scaled your own company, you've now turned Kavak fully agentic. Like, what turned Kavak fully agentic. Like, what turned Kavak fully agentic. Like, what advice do you have to future founders or advice do you have to future founders or advice do you have to future founders or first-time founders that might be first-time founders that might be first-time founders that might be listening? listening? listening? >> So, this is the most exciting time >> So, this is the most exciting time >> So, this is the most exciting time in human history. I I believe that. Like in human history. I I believe that. Like in human history. I I believe that. Like like we're living in the most exciting like we're living in the most exciting like we're living in the most exciting time in human history. And it's the most time in human history. And it's the most time in human history. And it's the most exciting time to to be a founder because exciting time to to be a founder because exciting time to to be a founder because it's the first time that anyone it's the first time that anyone it's the first time that anyone has access has access has access to the most powerful tools and to the most powerful tools and to the most powerful tools and intelligence in the world. intelligence in the world. intelligence in the world. Like for almost for free or for $20 a Like for almost for free or for $20 a Like for almost for free or for $20 a month. So, so literally there month. So, so literally there month. So, so literally there the the democratization of the tools for the the democratization of the tools for the the democratization of the tools for people to build people to build people to build has never been uh this way in human has never been uh this way in human has never been uh this way in human history. And there's so much history. And there's so much history. And there's so much problems to be solved and a new reality problems to be solved and a new reality problems to be solved and a new reality to be built around this this new to be built around this this new to be built around this this new paradigm. So, paradigm. So, paradigm. So, say like just go for it, but go for it say like just go for it, but go for it say like just go for it, but go for it deep. Like imagine deep. Like imagine deep. Like imagine what the future around AI will look what the future around AI will look what the future around AI will look like. like. like. It's just a uh it's not even an It's just a uh it's not even an It's just a uh it's not even an exponential. Just just map exponential. Just just map exponential. Just just map a trend that's linear. If things keeps a trend that's linear. If things keeps a trend that's linear. If things keeps getting like AI keeps getting better at getting like AI keeps getting better at getting like AI keeps getting better at a linear scale, a linear scale, a linear scale, and just build for that. And and you'll and just build for that. And and you'll and just build for that. And and you'll come up with with wonderful ideas that come up with with wonderful ideas that come up with with wonderful ideas that will like bring a lot of value to the will like bring a lot of value to the will like bring a lot of value to the world. world. world. >> Amazing. And I thank you for joining us. >> Amazing. And I thank you for joining us. >> Amazing. And I thank you for joining us. >> Thank you. Thanks for having me.