The Two Harvard Dropouts Who raised $800M to take on NVIDIA

We know inference is going to be the We know inference is going to be the biggest market in the world. Whoever biggest market in the world. Whoever biggest market in the world. Whoever produces the most tokens is going to be produces the most tokens is going to be produces the most tokens is going to be the most valuable company in the world. the most valuable company in the world. the most valuable company in the world. >> We we had people quit. >> We we had people quit. >> We we had people quit. >> Yeah. >> Yeah. >> Yeah. >> That people literally were like, "This >> That people literally were like, "This >> That people literally were like, "This problem is unsolvable." And uh best of problem is unsolvable." And uh best of problem is unsolvable." And uh best of luck, guys. luck, guys. luck, guys. >> You kind of have to be sick in the head >> You kind of have to be sick in the head >> You kind of have to be sick in the head to join our company. You're going to to join our company. You're going to to join our company. You're going to convince your family to move to San Jose convince your family to move to San Jose convince your family to move to San Jose for the semiconductor company run by two for the semiconductor company run by two for the semiconductor company run by two what 24 year olds now going against the what 24 year olds now going against the what 24 year olds now going against the biggest companies in the world with a biggest companies in the world with a biggest companies in the world with a design that they're saying is not going design that they're saying is not going design that they're saying is not going to be like 10% better, but it's going to to be like 10% better, but it's going to to be like 10% better, but it's going to be 10 10x better. we need $100 million be 10 10x better. we need $100 million be 10 10x better. we need $100 million to do this. If we do this, we think this to do this. If we do this, we think this to do this. If we do this, we think this could be one of the most important could be one of the most important could be one of the most important companies of all time. companies of all time. companies of all time. >> You're sitting in that moment, you >> You're sitting in that moment, you >> You're sitting in that moment, you think, holy crap, we can't afford this. think, holy crap, we can't afford this. think, holy crap, we can't afford this. And I began looking at like, huh, how And I began looking at like, huh, how And I began looking at like, huh, how hard is it to go back to Harvard? All right, gentlemen. It's been three All right, gentlemen. It's been three years or so, Gavin, since you and I last years or so, Gavin, since you and I last years or so, Gavin, since you and I last did this, which is nuts. And at the did this, which is nuts. And at the did this, which is nuts. And at the time, I was just wildly intrigued by time, I was just wildly intrigued by time, I was just wildly intrigued by your story and what you were going to your story and what you were going to your story and what you were going to build. I didn't know a lot about chips build. I didn't know a lot about chips build. I didn't know a lot about chips at the time. I was considering investing at the time. I was considering investing at the time. I was considering investing in the company. And so, I was calling in the company. And so, I was calling in the company. And so, I was calling everyone I could conceive of that could everyone I could conceive of that could everyone I could conceive of that could give me an opinion or something. And at give me an opinion or something. And at give me an opinion or something. And at the time, basically, the consensus was the time, basically, the consensus was the time, basically, the consensus was these kinds of companies are not built these kinds of companies are not built these kinds of companies are not built by young people. that the semis world uh by young people. that the semis world uh by young people. that the semis world uh the best companies are founded by 40 the best companies are founded by 40 the best companies are founded by 40 50-year-old people that have had a whole 50-year-old people that have had a whole 50-year-old people that have had a whole career's worth of experience, have career's worth of experience, have career's worth of experience, have learned all the problems, have done have learned all the problems, have done have learned all the problems, have done have shipped multiple chips. Two 21-year-olds shipped multiple chips. Two 21-year-olds shipped multiple chips. Two 21-year-olds are like not going to do this. Like it's are like not going to do this. Like it's are like not going to do this. Like it's just not going to work. And it was it just not going to work. And it was it just not going to work. And it was it was indicative of a of a theme which was was indicative of a of a theme which was was indicative of a of a theme which was nobody believes in us. That's obviously nobody believes in us. That's obviously nobody believes in us. That's obviously changed a lot now. You know, just walk changed a lot now. You know, just walk changed a lot now. You know, just walk the halls and talk to the people that the halls and talk to the people that the halls and talk to the people that have chosen to come work here. But in have chosen to come work here. But in have chosen to come work here. But in the early days, it felt like this was the early days, it felt like this was the early days, it felt like this was this was something that you had to face this was something that you had to face this was something that you had to face down. What was that like facing that down. What was that like facing that down. What was that like facing that down where like a set of incumbents and down where like a set of incumbents and down where like a set of incumbents and an industry worth of people and an industry worth of people and an industry worth of people and investors and everyone else sort of investors and everyone else sort of investors and everyone else sort of didn't believe in you and what did that didn't believe in you and what did that didn't believe in you and what did that anneal in you to build the company the anneal in you to build the company the anneal in you to build the company the way that that you are like what was the way that that you are like what was the way that that you are like what was the impact of that? impact of that? impact of that? >> I think there's a certain level of >> I think there's a certain level of >> I think there's a certain level of naivity required to think that you could naivity required to think that you could naivity required to think that you could build a chip better than every other AI build a chip better than every other AI build a chip better than every other AI chip ever built and build a company to chip ever built and build a company to chip ever built and build a company to do it way faster than ever has been do it way faster than ever has been do it way faster than ever has been done. Um and we have the naivity. done. Um and we have the naivity. done. Um and we have the naivity. There's many times where we would say There's many times where we would say There's many times where we would say like why isn't this possible and you like why isn't this possible and you like why isn't this possible and you know really push on it and it turns out know really push on it and it turns out know really push on it and it turns out that like everybody's answers are that like everybody's answers are that like everybody's answers are extremely siloed to a set of constraints extremely siloed to a set of constraints extremely siloed to a set of constraints that aren't true anymore and the reality that aren't true anymore and the reality that aren't true anymore and the reality is the entire semiconductors and data is the entire semiconductors and data is the entire semiconductors and data center industry is built on buffer and center industry is built on buffer and center industry is built on buffer and what I mean by that is every part of the what I mean by that is every part of the what I mean by that is every part of the stack from the EDA tools to the power stack from the EDA tools to the power stack from the EDA tools to the power modules to the circuit boards to the modules to the circuit boards to the modules to the circuit boards to the chip design and standard cells chip design and standard cells chip design and standard cells everything is built to be general everything is built to be general everything is built to be general purpose for everything not just in the purpose for everything not just in the purpose for everything not just in the data center but IO IoT on the edge and data center but IO IoT on the edge and data center but IO IoT on the edge and so forth. Uh, and when you have a so forth. Uh, and when you have a so forth. Uh, and when you have a specific use case use case you're really specific use case use case you're really specific use case use case you're really trying to design for, you can change the trying to design for, you can change the trying to design for, you can change the constraints a lot. And I'll give you constraints a lot. And I'll give you constraints a lot. And I'll give you just a very simple example that we're just a very simple example that we're just a very simple example that we're not the only one who does, which is one not the only one who does, which is one not the only one who does, which is one of the things you care a lot about is of the things you care a lot about is of the things you care a lot about is the clock speed of your chip. It's the clock speed of your chip. It's the clock speed of your chip. It's proportional to the throughput of your proportional to the throughput of your proportional to the throughput of your system. You know, when you are doing system. You know, when you are doing system. You know, when you are doing signoff uh for different timing, uh you signoff uh for different timing, uh you signoff uh for different timing, uh you you know what clock speed you're you know what clock speed you're you know what clock speed you're actually going to be able to run on uh actually going to be able to run on uh actually going to be able to run on uh when you tape out your chip. Uh there's when you tape out your chip. Uh there's when you tape out your chip. Uh there's this concept called corners, which is, this concept called corners, which is, this concept called corners, which is, you know, what temperatures are you you know, what temperatures are you you know, what temperatures are you going to be able to run at this clock going to be able to run at this clock going to be able to run at this clock speed. You know, the default speed. You know, the default speed. You know, the default configurations for a lot of these EDA configurations for a lot of these EDA configurations for a lot of these EDA tools assume that you're going to be tools assume that you're going to be tools assume that you're going to be running your chips in freezing running your chips in freezing running your chips in freezing temperatures. Now, I don't know about temperatures. Now, I don't know about temperatures. Now, I don't know about you, but I've never seen an AI data you, but I've never seen an AI data you, but I've never seen an AI data center with ice in it. So, you know, we center with ice in it. So, you know, we center with ice in it. So, you know, we can feel pretty confident that our chips can feel pretty confident that our chips can feel pretty confident that our chips don't need to run at full speed at 0 don't need to run at full speed at 0 don't need to run at full speed at 0 degrees CC. In fact, like they're never degrees CC. In fact, like they're never degrees CC. In fact, like they're never really going to be running below 80° C really going to be running below 80° C really going to be running below 80° C anyway. And just by knowing that that's anyway. And just by knowing that that's anyway. And just by knowing that that's a constraint that doesn't matter, we can a constraint that doesn't matter, we can a constraint that doesn't matter, we can make a ton of changes throughout the make a ton of changes throughout the make a ton of changes throughout the entire system. That's like a very simple entire system. That's like a very simple entire system. That's like a very simple one, but there's many more that you get one, but there's many more that you get one, but there's many more that you get 20% here, 50% there, 2x here, and these 20% here, 50% there, 2x here, and these 20% here, 50% there, 2x here, and these compound to a system that can be compound to a system that can be compound to a system that can be radically better for inference. radically better for inference. radically better for inference. >> Well, I think you found two kinds of >> Well, I think you found two kinds of >> Well, I think you found two kinds of people. There are some folks who went people. There are some folks who went people. There are some folks who went purely on heristics. Okay, young purely on heristics. Okay, young purely on heristics. Okay, young founders. They claim they can go beat founders. They claim they can go beat founders. They claim they can go beat the biggest company in the world out the biggest company in the world out the biggest company in the world out performance. It cannot happen. And there performance. It cannot happen. And there performance. It cannot happen. And there is nothing you could go say to me that is nothing you could go say to me that is nothing you could go say to me that would make me change my mind. But would make me change my mind. But would make me change my mind. But there's also people out there who are of there's also people out there who are of there's also people out there who are of course skeptical that are willing to go course skeptical that are willing to go course skeptical that are willing to go ahead and say, "I'll spend the time. ahead and say, "I'll spend the time. ahead and say, "I'll spend the time. I'll do the work." And is actually I'll do the work." And is actually I'll do the work." And is actually possible. Like for example, one of our possible. Like for example, one of our possible. Like for example, one of our earliest earliest uh supporters was this earliest earliest uh supporters was this earliest earliest uh supporters was this guy uh Mark Ross. And Mark was a very guy uh Mark Ross. And Mark was a very guy uh Mark Ross. And Mark was a very prestigious semiconductor expert. Used prestigious semiconductor expert. Used prestigious semiconductor expert. Used to be CTO at Cypress Semi that sold for to be CTO at Cypress Semi that sold for to be CTO at Cypress Semi that sold for $9 billion. And when we met him, we were $9 billion. And when we met him, we were $9 billion. And when we met him, we were just a couple of guys in a dorm room. just a couple of guys in a dorm room. just a couple of guys in a dorm room. And we came to him and say, "Hey, we And we came to him and say, "Hey, we And we came to him and say, "Hey, we want to go build hardware for inference. want to go build hardware for inference. want to go build hardware for inference. We think we can be much faster than We think we can be much faster than We think we can be much faster than Nvidia." And Mark's like, "No, you Nvidia." And Mark's like, "No, you Nvidia." And Mark's like, "No, you can't. It will not work." can't. It will not work." can't. It will not work." But he want to go convince me if you But he want to go convince me if you But he want to go convince me if you write a white paper. You should go ahead write a white paper. You should go ahead write a white paper. You should go ahead and build a functional simulation and and build a functional simulation and and build a functional simulation and show me. And so after a lot of very long show me. And so after a lot of very long show me. And so after a lot of very long nights, went back to Mark and said, nights, went back to Mark and said, nights, went back to Mark and said, "Hey, here's a simulation. "Hey, here's a simulation. "Hey, here's a simulation. What do you think?" And he was like, What do you think?" And he was like, What do you think?" And he was like, "Huh, this works." But to go do a "Huh, this works." But to go do a "Huh, this works." But to go do a company like this, you'll need a large company like this, you'll need a large company like this, you'll need a large amount of capital. You need at least $3 amount of capital. You need at least $3 amount of capital. You need at least $3 million even to get get started. End up million even to get get started. End up million even to get get started. End up went ahead and raised five, raised a lot went ahead and raised five, raised a lot went ahead and raised five, raised a lot more after that. And then he was again more after that. And then he was again more after that. And then he was again surprised but got more involved and then surprised but got more involved and then surprised but got more involved and then he became an adviser and a halftime he became an adviser and a halftime he became an adviser and a halftime adviser and eventually a full-time CTO adviser and eventually a full-time CTO adviser and eventually a full-time CTO as he saw more and more of the as he saw more and more of the as he saw more and more of the development progress. And I think in development progress. And I think in development progress. And I think in general this has filtered really general this has filtered really general this has filtered really heavily. folks who want to go ahead and heavily. folks who want to go ahead and heavily. folks who want to go ahead and be right regardless, folks who will want be right regardless, folks who will want be right regardless, folks who will want to be very truth seeeking and say, to be very truth seeeking and say, to be very truth seeeking and say, "Sure, I'm skeptical, but I will go "Sure, I'm skeptical, but I will go "Sure, I'm skeptical, but I will go ahead and work through the numbers ahead and work through the numbers ahead and work through the numbers myself. And if I can go figure out why myself. And if I can go figure out why myself. And if I can go figure out why this is possible, well, let's go build this is possible, well, let's go build this is possible, well, let's go build it." it." it." >> The specifics that you've made bets on, >> The specifics that you've made bets on, >> The specifics that you've made bets on, the way that you've built this system the way that you've built this system the way that you've built this system are immensely interesting to me. And are immensely interesting to me. And are immensely interesting to me. And because so many people are trying to do because so many people are trying to do because so many people are trying to do this now, build new chips that will do a this now, build new chips that will do a this now, build new chips that will do a better job of serving inference at better job of serving inference at better job of serving inference at massive scale, the world is interested massive scale, the world is interested massive scale, the world is interested in the research approaches, the in the research approaches, the in the research approaches, the different architecture approaches that different architecture approaches that different architecture approaches that people are taking to building a new AI people are taking to building a new AI people are taking to building a new AI chip. And I'd love you to just start by chip. And I'd love you to just start by chip. And I'd love you to just start by describing what this thing is, what it describing what this thing is, what it describing what this thing is, what it does, but maybe more interestingly and does, but maybe more interestingly and does, but maybe more interestingly and more importantly, the process that you more importantly, the process that you more importantly, the process that you went through to decide what bets to went through to decide what bets to went through to decide what bets to take, what technologies to invent, and take, what technologies to invent, and take, what technologies to invent, and compare and contrast those with what compare and contrast those with what compare and contrast those with what you've seen the rest of the marketplace you've seen the rest of the marketplace you've seen the rest of the marketplace try to do. try to do. try to do. >> Yeah, I think that you go start with the >> Yeah, I think that you go start with the >> Yeah, I think that you go start with the product. We're not just building a chip. product. We're not just building a chip. product. We're not just building a chip. We're building a full inference We're building a full inference We're building a full inference solution. And that means a rack. That solution. And that means a rack. That solution. And that means a rack. That means the chip. That means the power means the chip. That means the power means the chip. That means the power delivery into the chip. That means the delivery into the chip. That means the delivery into the chip. That means the board in which it sits. That means the board in which it sits. That means the board in which it sits. That means the interconnect by which the chips talk to interconnect by which the chips talk to interconnect by which the chips talk to each other. That means the production each other. That means the production each other. That means the production for this mass volume of racks. Really for this mass volume of racks. Really for this mass volume of racks. Really the production is the product. We think the production is the product. We think the production is the product. We think about how we get our advantage. There about how we get our advantage. There about how we get our advantage. There are two key parts of running inference. are two key parts of running inference. are two key parts of running inference. There's prefill and there's decode. Now There's prefill and there's decode. Now There's prefill and there's decode. Now we have two key tags for both of these we have two key tags for both of these we have two key tags for both of these things. Prefill is reading in a huge things. Prefill is reading in a huge things. Prefill is reading in a huge volume of text and decode is then using volume of text and decode is then using volume of text and decode is then using that data to generate output tokens. that data to generate output tokens. that data to generate output tokens. When you go out and run prefill, yeah, When you go out and run prefill, yeah, When you go out and run prefill, yeah, your key job is not to go predict your key job is not to go predict your key job is not to go predict tokens. You already know the text. Your tokens. You already know the text. Your tokens. You already know the text. Your job is to go ahead and get the model's job is to go ahead and get the model's job is to go ahead and get the model's memory. memory. memory. >> Yeah. >> Yeah. >> Yeah. >> What we call is KV cache into the right >> What we call is KV cache into the right >> What we call is KV cache into the right state. state. state. >> Then you can go ahead and run decode >> Then you can go ahead and run decode >> Then you can go ahead and run decode with that same KV cache. So we will with that same KV cache. So we will with that same KV cache. So we will often go do is we call PD disagregation. often go do is we call PD disagregation. often go do is we call PD disagregation. Prefilled decode disagrees. You'll then transfer those model You'll then transfer those model memories, those KDV caches over to the memories, those KDV caches over to the memories, those KDV caches over to the decode cluster and then go ahead and uh decode cluster and then go ahead and uh decode cluster and then go ahead and uh use that cluster to go generate the next use that cluster to go generate the next use that cluster to go generate the next tokens. tokens. tokens. >> So it's sort of like loading the gun and >> So it's sort of like loading the gun and >> So it's sort of like loading the gun and then firing it. Like if I think about it then firing it. Like if I think about it then firing it. Like if I think about it in super simple terms. in super simple terms. in super simple terms. >> Yeah, you got it. It's getting the model >> Yeah, you got it. It's getting the model >> Yeah, you got it. It's getting the model to remember the right things and then to remember the right things and then to remember the right things and then using those things to go do tasks. using those things to go do tasks. using those things to go do tasks. >> Generally people think about this market >> Generally people think about this market >> Generally people think about this market a bit lazily or they say are you prefill a bit lazily or they say are you prefill a bit lazily or they say are you prefill chip? Are you decode chip? If you're chip? Are you decode chip? If you're chip? Are you decode chip? If you're decode chip, are you an HPM chip? Are decode chip, are you an HPM chip? Are decode chip, are you an HPM chip? Are you an SRAMM chip? Are you 3D RAM chip? you an SRAMM chip? Are you 3D RAM chip? you an SRAMM chip? Are you 3D RAM chip? Uh are you using optics using copper? Uh are you using optics using copper? Uh are you using optics using copper? When we started this, we just wanted to When we started this, we just wanted to When we started this, we just wanted to understand why extremely smart people understand why extremely smart people understand why extremely smart people were working on these different were working on these different were working on these different directions. We seriously looked at directions. We seriously looked at directions. We seriously looked at architectures like having bunch of DDR architectures like having bunch of DDR architectures like having bunch of DDR memory and like a shared memory pool and memory and like a shared memory pool and memory and like a shared memory pool and looking at advanced packaging to looking at advanced packaging to looking at advanced packaging to basically break out of the shoreline. We basically break out of the shoreline. We basically break out of the shoreline. We looked at things like, you know, is looked at things like, you know, is looked at things like, you know, is there ways to put uh memory dies on top there ways to put uh memory dies on top there ways to put uh memory dies on top of compute dies. In doing so, we of compute dies. In doing so, we of compute dies. In doing so, we realized that there's no free lunch. You realized that there's no free lunch. You realized that there's no free lunch. You know, everything has a trade-off, right? know, everything has a trade-off, right? know, everything has a trade-off, right? You know, 3D RAM, you have a thermal You know, 3D RAM, you have a thermal You know, 3D RAM, you have a thermal issue, you have a supply chain issue, issue, you have a supply chain issue, issue, you have a supply chain issue, you have to figure out hybrid bonding, you have to figure out hybrid bonding, you have to figure out hybrid bonding, you have to figure out the flops. So, you have to figure out the flops. So, you have to figure out the flops. So, now you're a decode chip. So we went now you're a decode chip. So we went now you're a decode chip. So we went through everything uh both on the through everything uh both on the through everything uh both on the prefill and the decode side. In doing prefill and the decode side. In doing prefill and the decode side. In doing so, we realized there's a few design so, we realized there's a few design so, we realized there's a few design spaces that nobody had seriously tried spaces that nobody had seriously tried spaces that nobody had seriously tried to explore because they were never done to explore because they were never done to explore because they were never done in AI chips. And we asked ourselves, in AI chips. And we asked ourselves, in AI chips. And we asked ourselves, what are the actual metrics that are what are the actual metrics that are what are the actual metrics that are going to matter the most? On the prefill going to matter the most? On the prefill going to matter the most? On the prefill side, the thing that matters is flops side, the thing that matters is flops side, the thing that matters is flops and flops density. And people talk about and flops density. And people talk about and flops density. And people talk about flops often as a headline number, but in flops often as a headline number, but in flops often as a headline number, but in reality, you should care about the flops reality, you should care about the flops reality, you should care about the flops you're getting when you're running real you're getting when you're running real you're getting when you're running real workloads. there's this concept called workloads. there's this concept called workloads. there's this concept called MFU or model flops utilization which is MFU or model flops utilization which is MFU or model flops utilization which is you know for every peak flop advertised you know for every peak flop advertised you know for every peak flop advertised how many cents on the dollar are you how many cents on the dollar are you how many cents on the dollar are you actually getting and on GPUs you often actually getting and on GPUs you often actually getting and on GPUs you often get somewhere between 20 and 50% get somewhere between 20 and 50% get somewhere between 20 and 50% depending on the workload uh and depending on the workload uh and depending on the workload uh and actually you can provably not run at actually you can provably not run at actually you can provably not run at 100% because you have a thermal issue 100% because you have a thermal issue 100% because you have a thermal issue where as you increase the flop where as you increase the flop where as you increase the flop utilization you have more transistors utilization you have more transistors utilization you have more transistors going on and off you draw more power and going on and off you draw more power and going on and off you draw more power and the chip will self-regulate and actually the chip will self-regulate and actually the chip will self-regulate and actually lower its clock speed to make sure it lower its clock speed to make sure it lower its clock speed to make sure it doesn't overheat as we looked at doesn't overheat as we looked at doesn't overheat as we looked at inference And we said if we want way inference And we said if we want way inference And we said if we want way more flops because we want to run at way more flops because we want to run at way more flops because we want to run at way higher throughputs, we fundamentally higher throughputs, we fundamentally higher throughputs, we fundamentally need to solve the thermal problem before need to solve the thermal problem before need to solve the thermal problem before we even think about adding flops to the we even think about adding flops to the we even think about adding flops to the chip. If I just add more flops to a GPU chip. If I just add more flops to a GPU chip. If I just add more flops to a GPU today or another AI chip, I'm not today or another AI chip, I'm not today or another AI chip, I'm not actually going to get more performance actually going to get more performance actually going to get more performance because it's just going to thermal because it's just going to thermal because it's just going to thermal throttle. Fundamentally, the essence of throttle. Fundamentally, the essence of throttle. Fundamentally, the essence of that is this concept of dinard scaling, that is this concept of dinard scaling, that is this concept of dinard scaling, which is voltage is quadratically which is voltage is quadratically which is voltage is quadratically proportional to power. So if I 2x my proportional to power. So if I 2x my proportional to power. So if I 2x my voltage, my power goes up by 4x. If I voltage, my power goes up by 4x. If I voltage, my power goes up by 4x. If I cut my voltage in half, my power goes cut my voltage in half, my power goes cut my voltage in half, my power goes down by a quarter. So we asked down by a quarter. So we asked down by a quarter. So we asked ourselves, how could we run voltages ourselves, how could we run voltages ourselves, how could we run voltages lower than GPUs? And we talked to a lot lower than GPUs? And we talked to a lot lower than GPUs? And we talked to a lot of people about this. We flew out to to of people about this. We flew out to to of people about this. We flew out to to Silicon Valley after dropping out and Silicon Valley after dropping out and Silicon Valley after dropping out and basically asked, you know, dozens of basically asked, you know, dozens of basically asked, you know, dozens of people in semiconductors at all these people in semiconductors at all these people in semiconductors at all these different chip companies how they did different chip companies how they did different chip companies how they did it. And the answer we got was like, you it. And the answer we got was like, you it. And the answer we got was like, you can't you can't run at voltages lower can't you can't run at voltages lower can't you can't run at voltages lower than GPUs. And this was very than GPUs. And this was very than GPUs. And this was very dissatisfying because there was many dissatisfying because there was many dissatisfying because there was many different industries of chips that run different industries of chips that run different industries of chips that run at voltages lower than GPUs. I mean, at voltages lower than GPUs. I mean, at voltages lower than GPUs. I mean, Bitcoin miners run at under a quarter of Bitcoin miners run at under a quarter of Bitcoin miners run at under a quarter of the voltage of GPUs. So this is the voltage of GPUs. So this is the voltage of GPUs. So this is obviously physically possible. The obviously physically possible. The obviously physically possible. The question is, are there issues with GPU question is, are there issues with GPU question is, are there issues with GPU architectures that make it unable to run architectures that make it unable to run architectures that make it unable to run at these voltages? And when we looked at at these voltages? And when we looked at at these voltages? And when we looked at the problem for a long time, we were the problem for a long time, we were the problem for a long time, we were able to create a new mechanism of able to create a new mechanism of able to create a new mechanism of running at much lower voltages, a new running at much lower voltages, a new running at much lower voltages, a new type of power delivery that we call low type of power delivery that we call low type of power delivery that we call low voltage inference. And we think all AI voltage inference. And we think all AI voltage inference. And we think all AI chips in the future are going to be low chips in the future are going to be low chips in the future are going to be low voltage chips. They're going to have to voltage chips. They're going to have to voltage chips. They're going to have to cram way more flops in the same silicon cram way more flops in the same silicon cram way more flops in the same silicon area and without thermal throttling run area and without thermal throttling run area and without thermal throttling run at way lower voltages. That's pretty at way lower voltages. That's pretty at way lower voltages. That's pretty simple. For decode, it is all a memory simple. For decode, it is all a memory simple. For decode, it is all a memory game. game. game. more memory bound, you can load the more memory bound, you can load the more memory bound, you can load the model faster, load the KV cache faster, model faster, load the KV cache faster, model faster, load the KV cache faster, and uh serve more tokens per second per and uh serve more tokens per second per and uh serve more tokens per second per user. We think people ask the wrong user. We think people ask the wrong user. We think people ask the wrong question here. People often ask how much question here. People often ask how much question here. People often ask how much memory bandwidth is on your chip. You memory bandwidth is on your chip. You memory bandwidth is on your chip. You should be asking how much memory should be asking how much memory should be asking how much memory bandwidth is on your full scaleup bandwidth is on your full scaleup bandwidth is on your full scaleup cluster. What we are able to do is add cluster. What we are able to do is add cluster. What we are able to do is add way, way more bandwidth and a much lower way, way more bandwidth and a much lower way, way more bandwidth and a much lower latency from chip to chip to our latency from chip to chip to our latency from chip to chip to our interconnects. It allows us to be able interconnects. It allows us to be able interconnects. It allows us to be able to go serve models at this much higher to go serve models at this much higher to go serve models at this much higher speed because you can go use the SRAMM speed because you can go use the SRAMM speed because you can go use the SRAMM and the HPM from the full scaleup and the HPM from the full scaleup and the HPM from the full scaleup cluster as a single pool. And that's our cluster as a single pool. And that's our cluster as a single pool. And that's our second key technical bet. What we call second key technical bet. What we call second key technical bet. What we call cluster scale memory. And on GPUs today, cluster scale memory. And on GPUs today, cluster scale memory. And on GPUs today, the cluster memory bandwidth is often the cluster memory bandwidth is often the cluster memory bandwidth is often very badly utilized because the time to very badly utilized because the time to very badly utilized because the time to go hop from one GPU to another is go hop from one GPU to another is go hop from one GPU to another is extremely long. For example, on extremely long. For example, on extremely long. For example, on blackwell chips, it can be about 4,000 blackwell chips, it can be about 4,000 blackwell chips, it can be about 4,000 nonds to go point to point. And that nonds to go point to point. And that nonds to go point to point. And that means that if you go ahead and go to an means that if you go ahead and go to an means that if you go ahead and go to an 8xtp setup, you will get way way less 8xtp setup, you will get way way less 8xtp setup, you will get way way less than an 8x improvement in your tokens than an 8x improvement in your tokens than an 8x improvement in your tokens per second per user. per second per user. per second per user. And what we did is built our own totally And what we did is built our own totally And what we did is built our own totally custom interconnect stack. We took custom interconnect stack. We took custom interconnect stack. We took everything above the second layer of everything above the second layer of everything above the second layer of Ethernet, built it full custom, and we Ethernet, built it full custom, and we Ethernet, built it full custom, and we can go out and do far better latencies can go out and do far better latencies can go out and do far better latencies and bandwidths this way too. We can go and bandwidths this way too. We can go and bandwidths this way too. We can go ahead and cut this by more than a factor ahead and cut this by more than a factor ahead and cut this by more than a factor of 5x. And that allows us to then use of 5x. And that allows us to then use of 5x. And that allows us to then use the memory of other chips much more the memory of other chips much more the memory of other chips much more effectively. As you scale the world effectively. As you scale the world effectively. As you scale the world size, your uh time per token goes down size, your uh time per token goes down size, your uh time per token goes down proportionally. It's not that surprising proportionally. It's not that surprising proportionally. It's not that surprising given all these architectures were built given all these architectures were built given all these architectures were built before chat GPT. So if we're trying to before chat GPT. So if we're trying to before chat GPT. So if we're trying to build a chip for the modern workloads, build a chip for the modern workloads, build a chip for the modern workloads, it's going to look very different. The it's going to look very different. The it's going to look very different. The way we, you know, organize our flops, way we, you know, organize our flops, way we, you know, organize our flops, the way we do our voltage domains, the the way we do our voltage domains, the the way we do our voltage domains, the way we do our power planes are going to way we do our power planes are going to way we do our power planes are going to look super different. The way we do the look super different. The way we do the look super different. The way we do the packaging is going to look super packaging is going to look super packaging is going to look super different. The way we do the board different. The way we do the board different. The way we do the board design is going to look different. Uh design is going to look different. Uh design is going to look different. Uh and then on the decode side, the way we and then on the decode side, the way we and then on the decode side, the way we connect everything is going to look very connect everything is going to look very connect everything is going to look very different. We're now bringing forward different. We're now bringing forward different. We're now bringing forward our first generation of this low voltage our first generation of this low voltage our first generation of this low voltage inference technology, which is running inference technology, which is running inference technology, which is running at under half the voltage of any other at under half the voltage of any other at under half the voltage of any other AI chip. AI chip. AI chip. >> RAMP is the only platform built to make >> RAMP is the only platform built to make >> RAMP is the only platform built to make your finance team leaner, faster, and your finance team leaner, faster, and your finance team leaner, faster, and better, saving businesses 5% annually on better, saving businesses 5% annually on better, saving businesses 5% annually on average, so you can stay focused on average, so you can stay focused on average, so you can stay focused on growth. Ramp customers grew revenue 3.2 growth. Ramp customers grew revenue 3.2 growth. Ramp customers grew revenue 3.2 times faster than the average American times faster than the average American times faster than the average American business. Visa, Perscell, Cursor, business. Visa, Perscell, Cursor, business. Visa, Perscell, Cursor, Stripe, Notion, 11lab, Shopify, and Stripe, Notion, 11lab, Shopify, and Stripe, Notion, 11lab, Shopify, and 70,000 other businesses all run on Ramp. 70,000 other businesses all run on Ramp. 70,000 other businesses all run on Ramp. Mine does, too, and so should yours. Mine does, too, and so should yours. Mine does, too, and so should yours. Learn more at ramp.com/invest. OpenAI, Cursor, Anthropic, Perplexity, OpenAI, Cursor, Anthropic, Perplexity, and Verscell all have something in and Verscell all have something in and Verscell all have something in common. They all use WorkOS. And here's common. They all use WorkOS. And here's common. They all use WorkOS. And here's why. To achieve enterprise adoption at why. To achieve enterprise adoption at why. To achieve enterprise adoption at scale, you have to deliver on core scale, you have to deliver on core scale, you have to deliver on core capabilities like SSO, skim, arbback, capabilities like SSO, skim, arbback, capabilities like SSO, skim, arbback, and audit logs. That's where work OS and audit logs. That's where work OS and audit logs. That's where work OS comes in. Instead of spending months comes in. Instead of spending months comes in. Instead of spending months building these missionritical building these missionritical building these missionritical capabilities yourself, you can just use capabilities yourself, you can just use capabilities yourself, you can just use work OS APIs to gain all of them on day work OS APIs to gain all of them on day work OS APIs to gain all of them on day zero. That's why so many of the top AI zero. That's why so many of the top AI zero. That's why so many of the top AI teams you hear about already run on work teams you hear about already run on work teams you hear about already run on work OS. Work OS is the fastest way to become OS. Work OS is the fastest way to become OS. Work OS is the fastest way to become enterprise ready and stay focused on enterprise ready and stay focused on enterprise ready and stay focused on what matters most, your product. Visit what matters most, your product. Visit what matters most, your product. Visit works.com to get started. works.com to get started. works.com to get started. Felix by Rogo is a personal finance Felix by Rogo is a personal finance Felix by Rogo is a personal finance agent that turns a single prompt into agent that turns a single prompt into agent that turns a single prompt into finished client ready work using your finished client ready work using your finished client ready work using your firm's own templates, context, and firm's own templates, context, and firm's own templates, context, and standards. Send Felix an email like, standards. Send Felix an email like, standards. Send Felix an email like, "Take these comments and turn them for "Take these comments and turn them for "Take these comments and turn them for me." Or, "Udate my tracker with the me." Or, "Udate my tracker with the me." Or, "Udate my tracker with the context of these emails." Or, "Run the context of these emails." Or, "Run the context of these emails." Or, "Run the ability to pay math on this buyer." And ability to pay math on this buyer." And ability to pay math on this buyer." And Felix sends back finished PowerPoint Felix sends back finished PowerPoint Felix sends back finished PowerPoint decks, Excel models, and sourced decks, Excel models, and sourced decks, Excel models, and sourced research. Felix works the way your team research. Felix works the way your team research. Felix works the way your team already does, delivering work quickly already does, delivering work quickly already does, delivering work quickly and accurately around the clock. Learn and accurately around the clock. Learn and accurately around the clock. Learn more at rogo.ai/felix. If you zoom all the way out, um, why is If you zoom all the way out, um, why is this so important? like what why is the this so important? like what why is the this so important? like what why is the delivery of much higher throughput, much delivery of much higher throughput, much delivery of much higher throughput, much lower cost per um token better better lower cost per um token better better lower cost per um token better better tokens per watt like all of these tokens per watt like all of these tokens per watt like all of these metrics that the the universe is going metrics that the the universe is going metrics that the the universe is going to start talking about more and more. to start talking about more and more. to start talking about more and more. Everyone knows everyone's the the supply Everyone knows everyone's the the supply Everyone knows everyone's the the supply side of the equation is is a big problem side of the equation is is a big problem side of the equation is is a big problem right now. Why is this in a bigger right now. Why is this in a bigger right now. Why is this in a bigger picture looking out a decade like the picture looking out a decade like the picture looking out a decade like the bottleneck in the technology world? bottleneck in the technology world? bottleneck in the technology world? Well, I think it comes down to Well, I think it comes down to Well, I think it comes down to productivity where we are at this productivity where we are at this productivity where we are at this extremely interesting moment in the extremely interesting moment in the extremely interesting moment in the history of civilization where there's history of civilization where there's history of civilization where there's real artificial intelligence, not like real artificial intelligence, not like real artificial intelligence, not like sci-fi stuff, but like these models can sci-fi stuff, but like these models can sci-fi stuff, but like these models can solve problems that most humans can't. solve problems that most humans can't. solve problems that most humans can't. And like it's going to create new And like it's going to create new And like it's going to create new scientific discoveries. It's going to scientific discoveries. It's going to scientific discoveries. It's going to create instant access to medical care, create instant access to medical care, create instant access to medical care, instant access to education, and now instant access to education, and now instant access to education, and now it's just about how many people can use it's just about how many people can use it's just about how many people can use this at the same time, how many products this at the same time, how many products this at the same time, how many products can serve this at the same time. Um, and can serve this at the same time. Um, and can serve this at the same time. Um, and also the speed of of doing different also the speed of of doing different also the speed of of doing different tasks. So you know when you think about tasks. So you know when you think about tasks. So you know when you think about wall clock time if we can take an agent wall clock time if we can take an agent wall clock time if we can take an agent that you know can run at a certain model that you know can run at a certain model that you know can run at a certain model quality and could take a year to solve a quality and could take a year to solve a quality and could take a year to solve a certain task using inference time certain task using inference time certain task using inference time compute if you have way faster decode compute if you have way faster decode compute if you have way faster decode speed you can compress that into a speed you can compress that into a speed you can compress that into a month. So the amount of scientific month. So the amount of scientific month. So the amount of scientific innovation and the amount of actual innovation and the amount of actual innovation and the amount of actual proliferation of technology will happen proliferation of technology will happen proliferation of technology will happen much faster. And then the second part is much faster. And then the second part is much faster. And then the second part is concurrency where today you just like concurrency where today you just like concurrency where today you just like it's just not possible for a billion it's just not possible for a billion it's just not possible for a billion people to use these models concurrently. people to use these models concurrently. people to use these models concurrently. Ultimately, some people are going to get Ultimately, some people are going to get Ultimately, some people are going to get downgraded. Some people's models are downgraded. Some people's models are downgraded. Some people's models are going to be slower. Some people just going to be slower. Some people just going to be slower. Some people just won't be able to access the hardware. A won't be able to access the hardware. A won't be able to access the hardware. A few years from now, there's going to be few years from now, there's going to be few years from now, there's going to be giant models serving billions of users. giant models serving billions of users. giant models serving billions of users. We're very much in the early innings of We're very much in the early innings of We're very much in the early innings of AI today where the paid plans, there's AI today where the paid plans, there's AI today where the paid plans, there's only a few million users in the world only a few million users in the world only a few million users in the world using paid plans of AI models. So, we're using paid plans of AI models. So, we're using paid plans of AI models. So, we're at 1 1,000th of the global population at 1 1,000th of the global population at 1 1,000th of the global population actually using this stuff. So, if you actually using this stuff. So, if you actually using this stuff. So, if you want to serve a giant scale, uh, a lot want to serve a giant scale, uh, a lot want to serve a giant scale, uh, a lot of things change and one of them is the of things change and one of them is the of things change and one of them is the number of chips that communicate number of chips that communicate number of chips that communicate together where, you know, people usually together where, you know, people usually together where, you know, people usually think about this in the context of think about this in the context of think about this in the context of training. you know, you have these giant training. you know, you have these giant training. you know, you have these giant training clusters. You have Colossus training clusters. You have Colossus training clusters. You have Colossus with over 100,000 GPUs that are allorked with over 100,000 GPUs that are allorked with over 100,000 GPUs that are allorked together. And you know, the inference together. And you know, the inference together. And you know, the inference side, you know, today people usually side, you know, today people usually side, you know, today people usually think about it as an 8 chip cluster or think about it as an 8 chip cluster or think about it as an 8 chip cluster or maybe just, you know, NVL72 uh as the maybe just, you know, NVL72 uh as the maybe just, you know, NVL72 uh as the scaleup domain, but very quickly this is scaleup domain, but very quickly this is scaleup domain, but very quickly this is going to become thousands of chips and going to become thousands of chips and going to become thousands of chips and tens of thousands of chips. And the way tens of thousands of chips. And the way tens of thousands of chips. And the way to get the most performance there, the to get the most performance there, the to get the most performance there, the time between sending data from one chip time between sending data from one chip time between sending data from one chip to another, that primitive matters way to another, that primitive matters way to another, that primitive matters way more than is getting credit right now. more than is getting credit right now. more than is getting credit right now. So when we think about optimizing memory So when we think about optimizing memory So when we think about optimizing memory bandwidth for the system, you have to bandwidth for the system, you have to bandwidth for the system, you have to think about how fast these chips can think about how fast these chips can think about how fast these chips can communicate together because if they can communicate together because if they can communicate together because if they can only communicate really quickly with only communicate really quickly with only communicate really quickly with themselves and very slowly with other themselves and very slowly with other themselves and very slowly with other chips, you're not going to actually be chips, you're not going to actually be chips, you're not going to actually be able to serve giant models at 10,000 able to serve giant models at 10,000 able to serve giant models at 10,000 20,000 tokens per second. So we need 20,000 tokens per second. So we need 20,000 tokens per second. So we need multiple orders of magnitude of multiple orders of magnitude of multiple orders of magnitude of infrastructure built out throughout the infrastructure built out throughout the infrastructure built out throughout the entire stack from the power, you know, entire stack from the power, you know, entire stack from the power, you know, from from the wafer to the watt, you from from the wafer to the watt, you from from the wafer to the watt, you know, from the the transistor to the know, from the the transistor to the know, from the the transistor to the token, uh, to actually bring this stuff token, uh, to actually bring this stuff token, uh, to actually bring this stuff to the world. And I think that you look to the world. And I think that you look to the world. And I think that you look at most other goods like the iPhone for at most other goods like the iPhone for at most other goods like the iPhone for example, they've gotten into this example, they've gotten into this example, they've gotten into this economies of scale where as a result economies of scale where as a result economies of scale where as a result more money does not really buy a better more money does not really buy a better more money does not really buy a better iPhone that if you're a billionaire or iPhone that if you're a billionaire or iPhone that if you're a billionaire or if you're just the average American, you if you're just the average American, you if you're just the average American, you buy the same phone. And tokens aren't buy the same phone. And tokens aren't buy the same phone. And tokens aren't like that yet. We are still in the very like that yet. We are still in the very like that yet. We are still in the very early days where a general purpose early days where a general purpose early days where a general purpose system, relatively small one, is kind of system, relatively small one, is kind of system, relatively small one, is kind of handcrafting these tokens uh like they handcrafting these tokens uh like they handcrafting these tokens uh like they made screws back in like the made screws back in like the made screws back in like the Renaissance. And uh I wanted to live in Renaissance. And uh I wanted to live in Renaissance. And uh I wanted to live in a world where you have the same a world where you have the same a world where you have the same economies of scale for token making that economies of scale for token making that economies of scale for token making that you do for making say iPhones or cars or you do for making say iPhones or cars or you do for making say iPhones or cars or anything else. anything else. anything else. >> I think that is one of the huge unlocks >> I think that is one of the huge unlocks >> I think that is one of the huge unlocks that allows a huge group of people to go that allows a huge group of people to go that allows a huge group of people to go use the best quality models. I think use the best quality models. I think use the best quality models. I think that economies of scale have all made that economies of scale have all made that economies of scale have all made capitalism very I don't know fair. I capitalism very I don't know fair. I capitalism very I don't know fair. I think that allows you to go ahead and think that allows you to go ahead and think that allows you to go ahead and have the same product in many many have the same product in many many have the same product in many many different hands and you're able to go different hands and you're able to go different hands and you're able to go then serve way more users on a single then serve way more users on a single then serve way more users on a single scale up cluster allows you to get scale up cluster allows you to get scale up cluster allows you to get closer to that point for token serving closer to that point for token serving closer to that point for token serving too. too. too. >> Yeah. And also just certain products >> Yeah. And also just certain products >> Yeah. And also just certain products aren't usable if they're slow. aren't usable if they're slow. aren't usable if they're slow. >> Yeah. >> Yeah. >> Yeah. >> So if you want to serve coding models uh >> So if you want to serve coding models uh >> So if you want to serve coding models uh you know and you want people to actually you know and you want people to actually you know and you want people to actually use them like there's a certain number use them like there's a certain number use them like there's a certain number of tokens per second you need to hit. So of tokens per second you need to hit. So of tokens per second you need to hit. So the question is while maintaining that the question is while maintaining that the question is while maintaining that per token speed uh how many users can I per token speed uh how many users can I per token speed uh how many users can I serve at the same time? and you can serve at the same time? and you can serve at the same time? and you can basically decide I'm going to shut off a basically decide I'm going to shut off a basically decide I'm going to shut off a bunch of the world from using this stuff bunch of the world from using this stuff bunch of the world from using this stuff or everyone's going to get a worse or everyone's going to get a worse or everyone's going to get a worse experience. So fundamentally you need to experience. So fundamentally you need to experience. So fundamentally you need to find ways to push out the curve and find ways to push out the curve and find ways to push out the curve and that's why there's such a pressure for that's why there's such a pressure for that's why there's such a pressure for new hardware. I'd like to take some time new hardware. I'd like to take some time new hardware. I'd like to take some time to step back and hear both of your to step back and hear both of your to step back and hear both of your stories for how you came to this idea stories for how you came to this idea stories for how you came to this idea and this company and then kind of walk and this company and then kind of walk and this company and then kind of walk through what it's been like to build it through what it's been like to build it through what it's been like to build it because I think in so doing we'll because I think in so doing we'll because I think in so doing we'll understand the system that you've built understand the system that you've built understand the system that you've built for for the company itself that will for for the company itself that will for for the company itself that will then be able to power subsequent then be able to power subsequent then be able to power subsequent generations of of products like this one generations of of products like this one generations of of products like this one uh for this crazy inference future that uh for this crazy inference future that uh for this crazy inference future that that we're staring down. Rob, maybe that we're staring down. Rob, maybe that we're staring down. Rob, maybe starting with you just take it how starting with you just take it how starting with you just take it how however far back you want, but what what however far back you want, but what what however far back you want, but what what what I'm curious about and your personal what I'm curious about and your personal what I'm curious about and your personal story was actually the very first thing story was actually the very first thing story was actually the very first thing I ever heard from either one of you was I ever heard from either one of you was I ever heard from either one of you was your personal story uh many years ago your personal story uh many years ago your personal story uh many years ago now, which really kind of blew me away. now, which really kind of blew me away. now, which really kind of blew me away. I'm most interested in your motivation I'm most interested in your motivation I'm most interested in your motivation ultimately for being here doing this ultimately for being here doing this ultimately for being here doing this thing. thing. thing. >> It starts back actually in high school >> It starts back actually in high school >> It starts back actually in high school for me. I've been very unlucky and lucky for me. I've been very unlucky and lucky for me. I've been very unlucky and lucky at different points in life. Uh this was at different points in life. Uh this was at different points in life. Uh this was one of the tougher times. You know, at one of the tougher times. You know, at one of the tougher times. You know, at the end of my sophomore year of high the end of my sophomore year of high the end of my sophomore year of high school, I got injured at a martial arts school, I got injured at a martial arts school, I got injured at a martial arts tournament. the next day it couldn't tournament. the next day it couldn't tournament. the next day it couldn't walk for some reason. Uh and they walk for some reason. Uh and they walk for some reason. Uh and they thought it was something wrong with like thought it was something wrong with like thought it was something wrong with like my SI joint or something. I went through my SI joint or something. I went through my SI joint or something. I went through physical therapy. I did different types physical therapy. I did different types physical therapy. I did different types of scans. They couldn't figure it out. of scans. They couldn't figure it out. of scans. They couldn't figure it out. Uh and eventually they found this big Uh and eventually they found this big Uh and eventually they found this big bump on my back uh in an MRI and you bump on my back uh in an MRI and you bump on my back uh in an MRI and you know told me it was a tumor. Stage four know told me it was a tumor. Stage four know told me it was a tumor. Stage four bone cancer was told I had under 30% bone cancer was told I had under 30% bone cancer was told I had under 30% chance of survival. Uh it was like a chance of survival. Uh it was like a chance of survival. Uh it was like a 2-year crazy chemotherapy surgery 2-year crazy chemotherapy surgery 2-year crazy chemotherapy surgery learning to walk again experience. Uh learning to walk again experience. Uh learning to walk again experience. Uh and when you go through something like and when you go through something like and when you go through something like that, it you know really uh changes the that, it you know really uh changes the that, it you know really uh changes the Overton window of human experience and Overton window of human experience and Overton window of human experience and makes you appreciate like what actually makes you appreciate like what actually makes you appreciate like what actually matters. Uh and you also ask yourself matters. Uh and you also ask yourself matters. Uh and you also ask yourself like what are you going to do if you like what are you going to do if you like what are you going to do if you have the chance to live? Like if you have the chance to live? Like if you have the chance to live? Like if you actually want to get through something actually want to get through something actually want to get through something like that, you need to be like hoping like that, you need to be like hoping like that, you need to be like hoping for something. Um and I always knew I for something. Um and I always knew I for something. Um and I always knew I wanted to do something very impactful if wanted to do something very impactful if wanted to do something very impactful if I had the chance to to to get through I had the chance to to to get through I had the chance to to to get through it. Um it took me a couple years to it. Um it took me a couple years to it. Um it took me a couple years to figure out what that was going to be. figure out what that was going to be. figure out what that was going to be. And at the same time as I got to college And at the same time as I got to college And at the same time as I got to college and met a bunch of other people building and met a bunch of other people building and met a bunch of other people building cool tech uh I got extremely excited by cool tech uh I got extremely excited by cool tech uh I got extremely excited by AI models especially once GPT3 came out AI models especially once GPT3 came out AI models especially once GPT3 came out and I was like wow this is the first and I was like wow this is the first and I was like wow this is the first model that can kind of speak English and model that can kind of speak English and model that can kind of speak English and these things are going to get really these things are going to get really these things are going to get really smart. What happened was when GPT4 came smart. What happened was when GPT4 came smart. What happened was when GPT4 came out uh there was GPT4V which was the out uh there was GPT4V which was the out uh there was GPT4V which was the first model with image uploading. So I first model with image uploading. So I first model with image uploading. So I went through my camera roll and I found went through my camera roll and I found went through my camera roll and I found a picture of my back with this bump on a picture of my back with this bump on a picture of my back with this bump on it before I was diagnosed and I said you it before I was diagnosed and I said you it before I was diagnosed and I said you know hey chat GPT you know pretend know hey chat GPT you know pretend know hey chat GPT you know pretend you're an expert doctor. A patient comes you're an expert doctor. A patient comes you're an expert doctor. A patient comes in uh and they says they have this, you in uh and they says they have this, you in uh and they says they have this, you know, bump on their back. What could it know, bump on their back. What could it know, bump on their back. What could it be? be? be? >> And it immediately says like this could >> And it immediately says like this could >> And it immediately says like this could be a tumor. You should get an MRI be a tumor. You should get an MRI be a tumor. You should get an MRI immediately. Like go to the doctor. And immediately. Like go to the doctor. And immediately. Like go to the doctor. And I just kind of sat there still. It was I just kind of sat there still. It was I just kind of sat there still. It was like like like >> this took six months. >> this took six months. >> this took six months. >> Yeah, that took me 6 months. And you >> Yeah, that took me 6 months. And you >> Yeah, that took me 6 months. And you know, yesterday this feature wasn't know, yesterday this feature wasn't know, yesterday this feature wasn't there. Today it's here. I go to show my there. Today it's here. I go to show my there. Today it's here. I go to show my parents uh and I got this like parents uh and I got this like parents uh and I got this like notification being like, you know, notification being like, you know, notification being like, you know, you're all out of image credits today. you're all out of image credits today. you're all out of image credits today. Like you need you need to get a pro Like you need you need to get a pro Like you need you need to get a pro plan. And I was like, holy crap. like plan. And I was like, holy crap. like plan. And I was like, holy crap. like this is going to change everything and this is going to change everything and this is going to change everything and you know like we clearly don't have the you know like we clearly don't have the you know like we clearly don't have the infrastructure to serve it and there's infrastructure to serve it and there's infrastructure to serve it and there's very few things you can work on that can very few things you can work on that can very few things you can work on that can actually bring this technology at scale actually bring this technology at scale actually bring this technology at scale to the world faster. I mean there's like to the world faster. I mean there's like to the world faster. I mean there's like plenty of people that are super smart plenty of people that are super smart plenty of people that are super smart working on models. Uh you know the the working on models. Uh you know the the working on models. Uh you know the the fabs seem maybe unreachable to work on fabs seem maybe unreachable to work on fabs seem maybe unreachable to work on but it seemed like the hardware was all but it seemed like the hardware was all but it seemed like the hardware was all designed before chat GPT. So like every designed before chat GPT. So like every designed before chat GPT. So like every GPU, every TPU, every AI chip that was GPU, every TPU, every AI chip that was GPU, every TPU, every AI chip that was serving these models were just like serving these models were just like serving these models were just like fundamentally built before this and are fundamentally built before this and are fundamentally built before this and are retrofit to serve these modern models. retrofit to serve these modern models. retrofit to serve these modern models. There's going to be an entire new wave There's going to be an entire new wave There's going to be an entire new wave of architectures that came out and like of architectures that came out and like of architectures that came out and like what a more exciting thing to work on what a more exciting thing to work on what a more exciting thing to work on than bringing this to everybody. Very than bringing this to everybody. Very than bringing this to everybody. Very different angle at the same time. I I different angle at the same time. I I different angle at the same time. I I was running a startup incubator called was running a startup incubator called was running a startup incubator called ProR uh which has incubated a bunch of ProR uh which has incubated a bunch of ProR uh which has incubated a bunch of different companies. Some of the different companies. Some of the different companies. Some of the earliest ones being cursor and anyphere earliest ones being cursor and anyphere earliest ones being cursor and anyphere which merged and mer and etched went which merged and mer and etched went which merged and mer and etched went through it a handful of others. Yeah. At through it a handful of others. Yeah. At through it a handful of others. Yeah. At the time as these models were getting the time as these models were getting the time as these models were getting smarter, it was yeah 2022 I was smarter, it was yeah 2022 I was smarter, it was yeah 2022 I was realizing all of these companies are realizing all of these companies are realizing all of these companies are spending all the money they raise on spending all the money they raise on spending all the money they raise on compute and I had had this realization compute and I had had this realization compute and I had had this realization as I was working on some of my own stuff as I was working on some of my own stuff as I was working on some of my own stuff that like oh my god all the products I that like oh my god all the products I that like oh my god all the products I want to build are going to cost tens of want to build are going to cost tens of want to build are going to cost tens of millions of dollars a year in inference. millions of dollars a year in inference. millions of dollars a year in inference. Hey, this is not going to be tenable. Hey, this is not going to be tenable. Hey, this is not going to be tenable. Like the cost structure of every Like the cost structure of every Like the cost structure of every software company, the COGS is not going software company, the COGS is not going software company, the COGS is not going to be like zero anymore for an to be like zero anymore for an to be like zero anymore for an incremental user. It's going to be like incremental user. It's going to be like incremental user. It's going to be like quite high and it's going to be a quite high and it's going to be a quite high and it's going to be a function of inference and then the opex function of inference and then the opex function of inference and then the opex of every business is going to also be of every business is going to also be of every business is going to also be inference as people use more and more inference as people use more and more inference as people use more and more coding agents. So fundamentally it seems coding agents. So fundamentally it seems coding agents. So fundamentally it seems like inference is going to be really like inference is going to be really like inference is going to be really important and it feels like we are on a important and it feels like we are on a important and it feels like we are on a like decade march for inference to like decade march for inference to like decade march for inference to become you know the biggest market in become you know the biggest market in become you know the biggest market in the world. Uh so when you think about the world. Uh so when you think about the world. Uh so when you think about that 10 years from now there's going to that 10 years from now there's going to that 10 years from now there's going to be these giant projects where everything be these giant projects where everything be these giant projects where everything in that data center fundamentally hasn't in that data center fundamentally hasn't in that data center fundamentally hasn't been designed today like we should go been designed today like we should go been designed today like we should go pick something and work on it. And you pick something and work on it. And you pick something and work on it. And you know that's that's kind of how it got know that's that's kind of how it got know that's that's kind of how it got started. started. started. >> Gavin I I'm really excited for you to go >> Gavin I I'm really excited for you to go >> Gavin I I'm really excited for you to go back about as far early in high school back about as far early in high school back about as far early in high school maybe even earlier and tell um the the maybe even earlier and tell um the the maybe even earlier and tell um the the your favorite hash marks on the timeline your favorite hash marks on the timeline your favorite hash marks on the timeline that ultimately led to your ambition to that ultimately led to your ambition to that ultimately led to your ambition to to drop out of Harvard and and start to drop out of Harvard and and start to drop out of Harvard and and start this company. My first job ever was at a this company. My first job ever was at a this company. My first job ever was at a company called Exnord where I did a company called Exnord where I did a company called Exnord where I did a colonel's development. I was 17. colonel's development. I was 17. colonel's development. I was 17. >> Yeah. >> Yeah. >> Yeah. >> And a 17-year-old can't sign illegally >> And a 17-year-old can't sign illegally >> And a 17-year-old can't sign illegally binding contracts. So rather than go binding contracts. So rather than go binding contracts. So rather than go ahead and do a traditional CIA, they ahead and do a traditional CIA, they ahead and do a traditional CIA, they went ahead and sat me down and said, went ahead and sat me down and said, went ahead and sat me down and said, Gavin, don't share this information. And Gavin, don't share this information. And Gavin, don't share this information. And uh Extern was one of the only companies uh Extern was one of the only companies uh Extern was one of the only companies that saw, hey, maybe this is a good that saw, hey, maybe this is a good that saw, hey, maybe this is a good trade. And uh got to work building trade. And uh got to work building trade. And uh got to work building kernels. And I've done this at another kernels. And I've done this at another kernels. And I've done this at another number of other companies since. Extern number of other companies since. Extern number of other companies since. Extern got bought by Apple for 200 million. Did got bought by Apple for 200 million. Did got bought by Apple for 200 million. Did the same thing at Octo got bought by the same thing at Octo got bought by the same thing at Octo got bought by Nvidia for hundreds of millions of Nvidia for hundreds of millions of Nvidia for hundreds of millions of dollars. But when you do this sort of dollars. But when you do this sort of dollars. But when you do this sort of colonel's work, what you realize is that colonel's work, what you realize is that colonel's work, what you realize is that the math is relatively easy. the math is relatively easy. the math is relatively easy. >> Okay. >> Okay. >> Okay. >> But to get high-speed decode, the thing >> But to get high-speed decode, the thing >> But to get high-speed decode, the thing that matters is data movement. Almost that matters is data movement. Almost that matters is data movement. Almost all the work that you do is optimizing all the work that you do is optimizing all the work that you do is optimizing how do you move data around a single how do you move data around a single how do you move data around a single chip or across multiple chips. And chip or across multiple chips. And chip or across multiple chips. And that's why we went ahead and built this that's why we went ahead and built this that's why we went ahead and built this cluster scale memory tech. we bring that cluster scale memory tech. we bring that cluster scale memory tech. we bring that interconnect time way way lower. You can interconnect time way way lower. You can interconnect time way way lower. You can go do way more movement and as a result go do way more movement and as a result go do way more movement and as a result get a much faster time to generate each get a much faster time to generate each get a much faster time to generate each subsequent token and build these crazy subsequent token and build these crazy subsequent token and build these crazy things Rob's talking about for doing a things Rob's talking about for doing a things Rob's talking about for doing a year's worth of work in a month or more year's worth of work in a month or more year's worth of work in a month or more than that in the future. than that in the future. than that in the future. >> Can you talk about the competitive drive >> Can you talk about the competitive drive >> Can you talk about the competitive drive that's evident in some of the high that's evident in some of the high that's evident in some of the high school competitions that you school competitions that you school competitions that you participated in and won? participated in and won? participated in and won? >> Uh we did a couple. uh for example I was >> Uh we did a couple. uh for example I was >> Uh we did a couple. uh for example I was very active in the FTC robotics and I very active in the FTC robotics and I very active in the FTC robotics and I was lucky to have a very talented was lucky to have a very talented was lucky to have a very talented partner Sanford that for a long time we partner Sanford that for a long time we partner Sanford that for a long time we uh were part of a traditional school uh were part of a traditional school uh were part of a traditional school team where it was about 20 guys all team where it was about 20 guys all team where it was about 20 guys all working together as often typical in working together as often typical in working together as often typical in first tech challenge and the goal is to first tech challenge and the goal is to first tech challenge and the goal is to go out and build a robot that uh scores go out and build a robot that uh scores go out and build a robot that uh scores the most points and a bunch of other the most points and a bunch of other the most points and a bunch of other things too. uh and first they put a lot things too. uh and first they put a lot things too. uh and first they put a lot of emphasis around uh collaborating with of emphasis around uh collaborating with of emphasis around uh collaborating with other teams around trying to do really other teams around trying to do really other teams around trying to do really good documentation around trying to go good documentation around trying to go good documentation around trying to go ahead and get others inspired to go do ahead and get others inspired to go do ahead and get others inspired to go do the same thing. Sanford and I decided the same thing. Sanford and I decided the same thing. Sanford and I decided rather than go ahead and do it this way rather than go ahead and do it this way rather than go ahead and do it this way we're going to win and we did nothing we're going to win and we did nothing we're going to win and we did nothing else besides build that scored the most else besides build that scored the most else besides build that scored the most points points points >> as a twoerson team >> as a twoerson team >> as a twoerson team >> as a twoerson team >> as a twoerson team >> as a twoerson team >> rather than a 20 person team >> rather than a 20 person team >> rather than a 20 person team >> that we were much much smaller than >> that we were much much smaller than >> that we were much much smaller than almost every other team in the almost every other team in the almost every other team in the competition. We figured that if we were competition. We figured that if we were competition. We figured that if we were going to go specialize, if we were gonna going to go specialize, if we were gonna going to go specialize, if we were gonna go out and do this win the damn games go out and do this win the damn games go out and do this win the damn games really well, we wouldn't need to go really well, we wouldn't need to go really well, we wouldn't need to go ahead and advance based on the quality ahead and advance based on the quality ahead and advance based on the quality of our documentation or of our outreach, of our documentation or of our outreach, of our documentation or of our outreach, >> we were just gonna go win. >> we were just gonna go win. >> we were just gonna go win. >> And so we did. We have branched off, >> And so we did. We have branched off, >> And so we did. We have branched off, built this twoerson team, uh built a built this twoerson team, uh built a built this twoerson team, uh built a robot, and decided we were going to go robot, and decided we were going to go robot, and decided we were going to go ahead and redesign it every 3 months. ahead and redesign it every 3 months. ahead and redesign it every 3 months. And we did. And uh we actually had the And we did. And uh we actually had the And we did. And uh we actually had the world record for the highest score uh world record for the highest score uh world record for the highest score uh during this competition at one point. Uh during this competition at one point. Uh during this competition at one point. Uh we were rated by OPR third in the world we were rated by OPR third in the world we were rated by OPR third in the world uh for software development and it was a uh for software development and it was a uh for software development and it was a damn good machine. damn good machine. damn good machine. >> What from that episode can I translate >> What from that episode can I translate >> What from that episode can I translate as an analogy onto how you built Etch as an analogy onto how you built Etch as an analogy onto how you built Etch the company? the company? the company? >> We think about how you want to go ahead >> We think about how you want to go ahead >> We think about how you want to go ahead and do a full rack scale product like and do a full rack scale product like and do a full rack scale product like this. There's a couple key ideas. One of this. There's a couple key ideas. One of this. There's a couple key ideas. One of them is like velocity velocity velocity them is like velocity velocity velocity them is like velocity velocity velocity that you win by shipping. You're not that you win by shipping. You're not that you win by shipping. You're not going to go out and win by having the going to go out and win by having the going to go out and win by having the best outreach or the best best outreach or the best best outreach or the best communications. We you win by having the communications. We you win by having the communications. We you win by having the best product. best product. best product. >> Chosen to have no communications. >> Chosen to have no communications. >> Chosen to have no communications. >> Exactly like your I'm just realizing how >> Exactly like your I'm just realizing how >> Exactly like your I'm just realizing how exactly like the robotics. exactly like the robotics. exactly like the robotics. >> There are many ways you can go win in >> There are many ways you can go win in >> There are many ways you can go win in business, but we'd rather go focus on business, but we'd rather go focus on business, but we'd rather go focus on just building the best product. just building the best product. just building the best product. >> And similarly, we think we can go do it >> And similarly, we think we can go do it >> And similarly, we think we can go do it with a lot fewer folks that if you're with a lot fewer folks that if you're with a lot fewer folks that if you're willing to go ahead and just focus on willing to go ahead and just focus on willing to go ahead and just focus on product, product, product product and product, product, product product and product, product, product product and paralyze relentlessly, you don't need paralyze relentlessly, you don't need paralyze relentlessly, you don't need 20,000 people like the big companies 20,000 people like the big companies 20,000 people like the big companies have, you can do the best product in the have, you can do the best product in the have, you can do the best product in the world with far fewer people. You know, world with far fewer people. You know, world with far fewer people. You know, there's a saying of the best part is no there's a saying of the best part is no there's a saying of the best part is no part. I think for us, it's also the best part. I think for us, it's also the best part. I think for us, it's also the best vendor is no vendor. As much as vendor is no vendor. As much as vendor is no vendor. As much as possible, we want to vertically possible, we want to vertically possible, we want to vertically integrate the entire product. Both integrate the entire product. Both integrate the entire product. Both because we get more performance, but we because we get more performance, but we because we get more performance, but we can move way faster. So, you know, can move way faster. So, you know, can move way faster. So, you know, everything from the chips to the boards everything from the chips to the boards everything from the chips to the boards to the cold plates to the interconnects to the cold plates to the interconnects to the cold plates to the interconnects to even the production. We want to do to even the production. We want to do to even the production. We want to do all of it as in-house as possible. We're all of it as in-house as possible. We're all of it as in-house as possible. We're actually I think we're the only startup actually I think we're the only startup actually I think we're the only startup right now that's building its own rack right now that's building its own rack right now that's building its own rack as well as its own chips. Uh, and we did as well as its own chips. Uh, and we did as well as its own chips. Uh, and we did it all at the same time. A couple years it all at the same time. A couple years it all at the same time. A couple years ago is the last time we were public. Uh ago is the last time we were public. Uh ago is the last time we were public. Uh at that point we just started building at that point we just started building at that point we just started building our rack team and we brought over Brian our rack team and we brought over Brian our rack team and we brought over Brian Lerer uh who built all of Nvidia's HGX Lerer uh who built all of Nvidia's HGX Lerer uh who built all of Nvidia's HGX and DGX systems which is like 80% of the and DGX systems which is like 80% of the and DGX systems which is like 80% of the revenue and we said we're going to build revenue and we said we're going to build revenue and we said we're going to build the rack at the same time. We actually the rack at the same time. We actually the rack at the same time. We actually went through multiple iterations of the went through multiple iterations of the went through multiple iterations of the rack before the chips even came back. Uh rack before the chips even came back. Uh rack before the chips even came back. Uh before the chips came back we made before the chips came back we made before the chips came back we made thermal chips that had the exact same thermal chips that had the exact same thermal chips that had the exact same hot spots as we expected our chips to hot spots as we expected our chips to hot spots as we expected our chips to have. So we could build the cold plates. have. So we could build the cold plates. have. So we could build the cold plates. We could overpressurize them and blow We could overpressurize them and blow We could overpressurize them and blow them up. We haven't had a single leak them up. We haven't had a single leak them up. We haven't had a single leak since our chips came back with the cold since our chips came back with the cold since our chips came back with the cold plates cuz we already validated them. plates cuz we already validated them. plates cuz we already validated them. Yeah. We have a a factory in Taiwan. We Yeah. We have a a factory in Taiwan. We Yeah. We have a a factory in Taiwan. We have a few dozen people out there. We have a few dozen people out there. We have a few dozen people out there. We built a clone of a bunch of the test built a clone of a bunch of the test built a clone of a bunch of the test stations in our office. Uh we have a 2 stations in our office. Uh we have a 2 stations in our office. Uh we have a 2 megawatt data center on this floor. Uh megawatt data center on this floor. Uh megawatt data center on this floor. Uh and we did 24/7 development cycles. You and we did 24/7 development cycles. You and we did 24/7 development cycles. You know, people are doing day shifts and know, people are doing day shifts and know, people are doing day shifts and night shifts to actually get the night shifts to actually get the night shifts to actually get the hardware up and running as quickly as hardware up and running as quickly as hardware up and running as quickly as possible. Um you know, it's that extreme possible. Um you know, it's that extreme possible. Um you know, it's that extreme vertical integration and extreme vertical integration and extreme vertical integration and extreme parallelization of the schedule that parallelization of the schedule that parallelization of the schedule that lets you get products to market way lets you get products to market way lets you get products to market way faster. M if you think about the um faster. M if you think about the um faster. M if you think about the um building of the early team and what it building of the early team and what it building of the early team and what it required as two young guys building this required as two young guys building this required as two young guys building this company, there's lots of very talented company, there's lots of very talented company, there's lots of very talented young entrepreneurs out there maybe for young entrepreneurs out there maybe for young entrepreneurs out there maybe for the first time of this scope or the first time of this scope or the first time of this scope or magnitude in a long time all of whom magnitude in a long time all of whom magnitude in a long time all of whom probably could benefit from the lessons probably could benefit from the lessons probably could benefit from the lessons that you've learned. Getting very that you've learned. Getting very that you've learned. Getting very sophisticated, talented people to come sophisticated, talented people to come sophisticated, talented people to come join you even after careers at the you join you even after careers at the you join you even after careers at the you know the other great companies. if you know the other great companies. if you know the other great companies. if you were teaching this as a class like were teaching this as a class like were teaching this as a class like here's how to get you know elite talent here's how to get you know elite talent here's how to get you know elite talent when you're young and inexperienced and when you're young and inexperienced and when you're young and inexperienced and naive like what what would be the naive like what what would be the naive like what what would be the syllabus? syllabus? syllabus? >> We have a pretty biodal talent >> We have a pretty biodal talent >> We have a pretty biodal talent philosophy. It starts with what we call philosophy. It starts with what we call philosophy. It starts with what we call the legends which is when we're trying the legends which is when we're trying the legends which is when we're trying to solve an incredibly hard technical to solve an incredibly hard technical to solve an incredibly hard technical problem and generally do something that problem and generally do something that problem and generally do something that hasn't been done before. We need to find hasn't been done before. We need to find hasn't been done before. We need to find the very best person in the world and the very best person in the world and the very best person in the world and often the number one guy in the world often the number one guy in the world often the number one guy in the world versus the number 10 guy versus the versus the number 10 guy versus the versus the number 10 guy versus the number 100 guy. Huge difference and number 100 guy. Huge difference and number 100 guy. Huge difference and whether it's actually possible to solve whether it's actually possible to solve whether it's actually possible to solve the problem. We created this system we the problem. We created this system we the problem. We created this system we call project-based recruiting where we call project-based recruiting where we call project-based recruiting where we map out all of the hardest technical map out all of the hardest technical map out all of the hardest technical problems across all industries that problems across all industries that problems across all industries that anyone has ever had to solve. We look at anyone has ever had to solve. We look at anyone has ever had to solve. We look at temporality. So who are the people who temporality. So who are the people who temporality. So who are the people who did the zero to one? Who is in charge did the zero to one? Who is in charge did the zero to one? Who is in charge quote unquote who actually did the work? quote unquote who actually did the work? quote unquote who actually did the work? Uh we talk to as many people as Uh we talk to as many people as Uh we talk to as many people as possible. And then we just track it. And possible. And then we just track it. And possible. And then we just track it. And you'd be surprised by, you know, the you'd be surprised by, you know, the you'd be surprised by, you know, the amount of people who say yes after the amount of people who say yes after the amount of people who say yes after the first conversation is pretty low, but first conversation is pretty low, but first conversation is pretty low, but the amount of people who say yes after the amount of people who say yes after the amount of people who say yes after the 20th conversation is surprisingly the 20th conversation is surprisingly the 20th conversation is surprisingly high. You really got to keep at them. high. You really got to keep at them. high. You really got to keep at them. That like when you hear no from somebody That like when you hear no from somebody That like when you hear no from somebody who really is the best in the world, who really is the best in the world, who really is the best in the world, then that really means, hey, you should then that really means, hey, you should then that really means, hey, you should go ahead and come back when you have a go ahead and come back when you have a go ahead and come back when you have a few more milestones proven out. few more milestones proven out. few more milestones proven out. >> Yeah. >> Yeah. >> Yeah. >> And I think it's one of the most >> And I think it's one of the most >> And I think it's one of the most convincing things to see is, hey, we convincing things to see is, hey, we convincing things to see is, hey, we make bold claims. And when you go ahead make bold claims. And when you go ahead make bold claims. And when you go ahead and hit those again and again and again, and hit those again and again and again, and hit those again and again and again, that is really belief inspiring. When we that is really belief inspiring. When we that is really belief inspiring. When we decided we wanted to build a rack and decided we wanted to build a rack and decided we wanted to build a rack and not just a chip, we were looking at this not just a chip, we were looking at this not just a chip, we were looking at this and we're saying, you know, how many and we're saying, you know, how many and we're saying, you know, how many products have actually shipped at scale products have actually shipped at scale products have actually shipped at scale for a rack scale system that actually for a rack scale system that actually for a rack scale system that actually have the power density that we're trying have the power density that we're trying have the power density that we're trying to solve? And you know, we just kind of to solve? And you know, we just kind of to solve? And you know, we just kind of said, if we were going to wave a magic said, if we were going to wave a magic said, if we were going to wave a magic wand, what would the best possible wand, what would the best possible wand, what would the best possible person in the world look like? and be person in the world look like? and be person in the world look like? and be like, well, if we could find somebody like, well, if we could find somebody like, well, if we could find somebody who like started at NVIDIA and built the who like started at NVIDIA and built the who like started at NVIDIA and built the entire rack team uh through all their entire rack team uh through all their entire rack team uh through all their different generations, learned all this different generations, learned all this different generations, learned all this different stuff, but is still scrappy, different stuff, but is still scrappy, different stuff, but is still scrappy, still understands the startup culture, still understands the startup culture, still understands the startup culture, but has seen scale, like that would be but has seen scale, like that would be but has seen scale, like that would be the best possible person. So, we mapped the best possible person. So, we mapped the best possible person. So, we mapped all of the different teams that related all of the different teams that related all of the different teams that related to all of the different rack scale to all of the different rack scale to all of the different rack scale products at video and we found three products at video and we found three products at video and we found three people that we thought, you know, could people that we thought, you know, could people that we thought, you know, could fit the bill. Um, and we talked to all fit the bill. Um, and we talked to all fit the bill. Um, and we talked to all of them and two of them have just of them and two of them have just of them and two of them have just retired and one of them was planning to retired and one of them was planning to retired and one of them was planning to do one more generation for Nvidia and do one more generation for Nvidia and do one more generation for Nvidia and and then retire. Um, his name is Brian. and then retire. Um, his name is Brian. and then retire. Um, his name is Brian. Uh, and over time we convinced him to Uh, and over time we convinced him to Uh, and over time we convinced him to join. Um, Brian started the HGX and DGX join. Um, Brian started the HGX and DGX join. Um, Brian started the HGX and DGX team at NVIDIA. Uh, you know, which was, team at NVIDIA. Uh, you know, which was, team at NVIDIA. Uh, you know, which was, you know, a majority of Nvidia's you know, a majority of Nvidia's you know, a majority of Nvidia's revenue, you know, tens of billions of revenue, you know, tens of billions of revenue, you know, tens of billions of dollars a quarter. Um, and the other two dollars a quarter. Um, and the other two dollars a quarter. Um, and the other two guys ended up investing, by the way. But guys ended up investing, by the way. But guys ended up investing, by the way. But when you have somebody like that, they when you have somebody like that, they when you have somebody like that, they just know what good looks like because just know what good looks like because just know what good looks like because they they've seen it. And like there's they they've seen it. And like there's they they've seen it. And like there's so many times where we'd talk to Brian so many times where we'd talk to Brian so many times where we'd talk to Brian and he'd just point to us and be like and he'd just point to us and be like and he'd just point to us and be like that's a billion dollar like a billion that's a billion dollar like a billion that's a billion dollar like a billion dollar lesson I learned. Billion dollar dollar lesson I learned. Billion dollar dollar lesson I learned. Billion dollar lesson I learned like you know that just lesson I learned like you know that just lesson I learned like you know that just saves us cycles. And you pair someone saves us cycles. And you pair someone saves us cycles. And you pair someone like Brian with somebody like Sanford. like Brian with somebody like Sanford. like Brian with somebody like Sanford. >> Do you have a name for them? So Brian's >> Do you have a name for them? So Brian's >> Do you have a name for them? So Brian's a legend. What's what's Sanford? a legend. What's what's Sanford? a legend. What's what's Sanford? >> Yeah we say chips on shoulders put uh >> Yeah we say chips on shoulders put uh >> Yeah we say chips on shoulders put uh chips in data centers. chips in data centers. chips in data centers. >> Yeah. So you know Sanford and Gavin in >> Yeah. So you know Sanford and Gavin in >> Yeah. So you know Sanford and Gavin in in in high school were world robotics in in high school were world robotics in in high school were world robotics champions. and Stanford was finishing champions. and Stanford was finishing champions. and Stanford was finishing his senior year of college uh and we his senior year of college uh and we his senior year of college uh and we called him up a couple years ago and we called him up a couple years ago and we called him up a couple years ago and we said hey can you come you know check out said hey can you come you know check out said hey can you come you know check out what we're doing we need some help on what we're doing we need some help on what we're doing we need some help on the platform side uh and he comes for a the platform side uh and he comes for a the platform side uh and he comes for a week and we say can you build the cold week and we say can you build the cold week and we say can you build the cold plate this week and if you asked like plate this week and if you asked like plate this week and if you asked like any thermal engineer like you know any thermal engineer like you know any thermal engineer like you know anything like that they they would think anything like that they they would think anything like that they they would think you're just like totally naive right I you're just like totally naive right I you're just like totally naive right I mean these things take months to do and mean these things take months to do and mean these things take months to do and like to be clear they do but you can like to be clear they do but you can like to be clear they do but you can make real progress in a week if you if make real progress in a week if you if make real progress in a week if you if you put your mind to it and you think you put your mind to it and you think you put your mind to it and you think it's possible and like he built a it's possible and like he built a it's possible and like he built a contraption in a week that like actually contraption in a week that like actually contraption in a week that like actually derisked like a pretty key power derisked like a pretty key power derisked like a pretty key power question we had. Um, and you put those question we had. Um, and you put those question we had. Um, and you put those two together and they've done incredible two together and they've done incredible two together and they've done incredible things and like one is not one is not things and like one is not one is not things and like one is not one is not possible without the other because you possible without the other because you possible without the other because you need the extremely driven people that need the extremely driven people that need the extremely driven people that just keep asking why and don't know just keep asking why and don't know just keep asking why and don't know where the bodies are buried to like take where the bodies are buried to like take where the bodies are buried to like take tons of aggressive risks and then you tons of aggressive risks and then you tons of aggressive risks and then you need the people who've seen scale um and need the people who've seen scale um and need the people who've seen scale um and still have the startup scrappy mentality still have the startup scrappy mentality still have the startup scrappy mentality to help them along the way. It's really to help them along the way. It's really to help them along the way. It's really the It's really the legends plus the the It's really the legends plus the the It's really the legends plus the raw, you know, some naivee raw first raw, you know, some naivee raw first raw, you know, some naivee raw first principles type talent, but it's the principles type talent, but it's the principles type talent, but it's the com. It's not just that you have both in com. It's not just that you have both in com. It's not just that you have both in the company. It's that they're working the company. It's that they're working the company. It's that they're working together. together. together. >> That's right. >> That's right. >> That's right. >> If I think about that funnel, anything >> If I think about that funnel, anything >> If I think about that funnel, anything else more interesting to say about how else more interesting to say about how else more interesting to say about how much better you've gotten at recruiting much better you've gotten at recruiting much better you've gotten at recruiting and like why why those metrics keep and like why why those metrics keep and like why why those metrics keep getting better. One of the shocking getting better. One of the shocking getting better. One of the shocking things is like being such a contrarian things is like being such a contrarian things is like being such a contrarian bet kind of self- selects, bet kind of self- selects, bet kind of self- selects, >> right? that like you're the kind of >> right? that like you're the kind of >> right? that like you're the kind of person who I think is some opportunistic person who I think is some opportunistic person who I think is some opportunistic going to go join whatever the hawk going to go join whatever the hawk going to go join whatever the hawk company is rather than go ahead and do company is rather than go ahead and do company is rather than go ahead and do deep digence you will not come work here deep digence you will not come work here deep digence you will not come work here >> right >> right >> right >> and it's one of the things I worry about >> and it's one of the things I worry about >> and it's one of the things I worry about as we announce more and more of the as we announce more and more of the as we announce more and more of the product and its specs we may lose some product and its specs we may lose some product and its specs we may lose some of this if we're not very careful of this if we're not very careful of this if we're not very careful >> you kind of have to be sick in the head >> you kind of have to be sick in the head >> you kind of have to be sick in the head to join our company you think about it to join our company you think about it to join our company you think about it on paper it's like you a person who you on paper it's like you a person who you on paper it's like you a person who you know is probably a very accomplished know is probably a very accomplished know is probably a very accomplished engineer making a good amount of money engineer making a good amount of money engineer making a good amount of money it's liquid it's predictable somewhere it's liquid it's predictable somewhere it's liquid it's predictable somewhere else you're going to convince your else you're going to convince your else you're going to convince your family to move to San Jose and live in family to move to San Jose and live in family to move to San Jose and live in this apartment on this housing program this apartment on this housing program this apartment on this housing program for the semiconductor company run by two for the semiconductor company run by two for the semiconductor company run by two what 24 year olds now uh that's what 24 year olds now uh that's what 24 year olds now uh that's pre-product that is going against the pre-product that is going against the pre-product that is going against the biggest companies in the world and the biggest companies in the world and the biggest companies in the world and the most supply constrained environment ever most supply constrained environment ever most supply constrained environment ever created with a design that they're created with a design that they're created with a design that they're saying is not going to be like 10% saying is not going to be like 10% saying is not going to be like 10% better but it's going to be 10 10x better but it's going to be 10 10x better but it's going to be 10 10x better. Something must be wrong with you better. Something must be wrong with you better. Something must be wrong with you to do that. And uh you know people are to do that. And uh you know people are to do that. And uh you know people are just wired differently here that they just wired differently here that they just wired differently here that they like really want to not prove people like really want to not prove people like really want to not prove people wrong who don't believe but prove people wrong who don't believe but prove people wrong who don't believe but prove people right who do believe uh and like they right who do believe uh and like they right who do believe uh and like they just take it personally and you know just take it personally and you know just take it personally and you know that's really fun to find those people that's really fun to find those people that's really fun to find those people and like you know frankly just the and like you know frankly just the and like you know frankly just the nature of the companies you know makes nature of the companies you know makes nature of the companies you know makes it very easy to whittle out the people it very easy to whittle out the people it very easy to whittle out the people who aren't like that. One of the very who aren't like that. One of the very who aren't like that. One of the very first things you and I talked about, first things you and I talked about, first things you and I talked about, Rob, was uh I I started asking about Rob, was uh I I started asking about Rob, was uh I I started asking about Sohoo, which is the name of of the first Sohoo, which is the name of of the first Sohoo, which is the name of of the first product here. And you said we could talk product here. And you said we could talk product here. And you said we could talk about that in in great detail, but the about that in in great detail, but the about that in in great detail, but the thing you should know is that what we're thing you should know is that what we're thing you should know is that what we're really focused on is building a machine really focused on is building a machine really focused on is building a machine that can at scale produce these things that can at scale produce these things that can at scale produce these things and generations of them as efficiently and generations of them as efficiently and generations of them as efficiently at the highest possible quality level. at the highest possible quality level. at the highest possible quality level. So we want to build like the company or So we want to build like the company or So we want to build like the company or the machine that is the company is the the machine that is the company is the the machine that is the company is the thing that totally thing that totally thing that totally >> will produce this thing and then >> will produce this thing and then >> will produce this thing and then subsequent things. So I'd like to talk subsequent things. So I'd like to talk subsequent things. So I'd like to talk about a few principles or cornerstones about a few principles or cornerstones about a few principles or cornerstones of the company. One of them you've of the company. One of them you've of the company. One of them you've talked you've alluded to some of them. talked you've alluded to some of them. talked you've alluded to some of them. You've said velocity, you've said You've said velocity, you've said You've said velocity, you've said vertical integration. You know have vertical integration. You know have vertical integration. You know have become more popular topics. Um become more popular topics. Um become more popular topics. Um parallelization is something maybe that parallelization is something maybe that parallelization is something maybe that we should talk about. But I'm especially we should talk about. But I'm especially we should talk about. But I'm especially interested in your guys' willingness to interested in your guys' willingness to interested in your guys' willingness to take huge risk to go faster. Maybe tell take huge risk to go faster. Maybe tell take huge risk to go faster. Maybe tell your favorite story about um why this is your favorite story about um why this is your favorite story about um why this is the philosophy, what it's allowed you to the philosophy, what it's allowed you to the philosophy, what it's allowed you to do that maybe other companies haven't do that maybe other companies haven't do that maybe other companies haven't done. done. done. >> There's a number of stories here. Uh, >> There's a number of stories here. Uh, >> There's a number of stories here. Uh, but one of my favorites is there was a but one of my favorites is there was a but one of my favorites is there was a time where we were getting close to time where we were getting close to time where we were getting close to taping out the chip. We realized, wait a taping out the chip. We realized, wait a taping out the chip. We realized, wait a minute, one of our vendors is way, way minute, one of our vendors is way, way minute, one of our vendors is way, way behind schedule. And we had two very bad behind schedule. And we had two very bad behind schedule. And we had two very bad options. options. options. One option is to keep the current vendor One option is to keep the current vendor One option is to keep the current vendor and push our timelines out by on the and push our timelines out by on the and push our timelines out by on the order of a year. order of a year. order of a year. >> Another option was to switch vendors, >> Another option was to switch vendors, >> Another option was to switch vendors, start over, and also push timelines out start over, and also push timelines out start over, and also push timelines out by a year. And uh, neither of these was by a year. And uh, neither of these was by a year. And uh, neither of these was a good option. a good option. a good option. So we had to go look for option number So we had to go look for option number So we had to go look for option number three and what that was was we we three and what that was was we we three and what that was was we we figured out they're all in Bangalore figured out they're all in Bangalore figured out they're all in Bangalore team actually going and doing the work. team actually going and doing the work. team actually going and doing the work. We went out and shipped a dozen of our We went out and shipped a dozen of our We went out and shipped a dozen of our top engineers across the world to top engineers across the world to top engineers across the world to Bangalore for 6 months. I was there as Bangalore for 6 months. I was there as Bangalore for 6 months. I was there as well. I lived in Bangalore for four and well. I lived in Bangalore for four and well. I lived in Bangalore for four and a half months personally and every a half months personally and every a half months personally and every morning we'd go ahead and walk across morning we'd go ahead and walk across morning we'd go ahead and walk across the crazy busy Bangalore streets into the crazy busy Bangalore streets into the crazy busy Bangalore streets into the office. Uh we'd be the first ones the office. Uh we'd be the first ones the office. Uh we'd be the first ones in. We'd go out and built a wide variety in. We'd go out and built a wide variety in. We'd go out and built a wide variety of tools, both things like, hey, of tools, both things like, hey, of tools, both things like, hey, auditing a huge amount of the code that auditing a huge amount of the code that auditing a huge amount of the code that was going in, building a bunch of tools was going in, building a bunch of tools was going in, building a bunch of tools as well to make this go even faster, as well to make this go even faster, as well to make this go even faster, making sure we're making the right making sure we're making the right making sure we're making the right design decisions on the spot right design decisions on the spot right design decisions on the spot right there. No 12-hour back and forth. We there. No 12-hour back and forth. We there. No 12-hour back and forth. We could go ahead and decide immediately, could go ahead and decide immediately, could go ahead and decide immediately, and then at 1:00 a.m., we'd go walk back and then at 1:00 a.m., we'd go walk back and then at 1:00 a.m., we'd go walk back through the now empty Bangalore streets through the now empty Bangalore streets through the now empty Bangalore streets and do it all again the next day. We and do it all again the next day. We and do it all again the next day. We still had the a bunch of the team in the still had the a bunch of the team in the still had the a bunch of the team in the US. We ran these 12-h hour on each side, US. We ran these 12-h hour on each side, US. We ran these 12-h hour on each side, you know, handoffs where we had 24-hour you know, handoffs where we had 24-hour you know, handoffs where we had 24-hour development cycle where at 8 a.m. and development cycle where at 8 a.m. and development cycle where at 8 a.m. and 8:00 p.m. every day, we'd all get on the 8:00 p.m. every day, we'd all get on the 8:00 p.m. every day, we'd all get on the Zoom, we'd share all the data, we'd say, Zoom, we'd share all the data, we'd say, Zoom, we'd share all the data, we'd say, "When I wake up, like, this must be "When I wake up, like, this must be "When I wake up, like, this must be done. Like, we must get this chip out." done. Like, we must get this chip out." done. Like, we must get this chip out." Uh, and it was extremely intense. At the Uh, and it was extremely intense. At the Uh, and it was extremely intense. At the same time, we saw other chips, you know, same time, we saw other chips, you know, same time, we saw other chips, you know, at the same stage as us with that same at the same stage as us with that same at the same stage as us with that same vendor that ended up taking years that vendor that ended up taking years that vendor that ended up taking years that still aren't out today, that still still aren't out today, that still still aren't out today, that still haven't even taped out today. uh and haven't even taped out today. uh and haven't even taped out today. uh and it's that level of extreme urgency it's that level of extreme urgency it's that level of extreme urgency that's required uh to bring products to that's required uh to bring products to that's required uh to bring products to market. market. market. >> What is the key to doing this? Well, >> What is the key to doing this? Well, >> What is the key to doing this? Well, this has become a a trope because of this has become a a trope because of this has become a a trope because of Elon mostly that like his special skill Elon mostly that like his special skill Elon mostly that like his special skill and others that seek to emulate him uh and others that seek to emulate him uh and others that seek to emulate him uh would try to do this too is figure out would try to do this too is figure out would try to do this too is figure out like what the binding constraint is and like what the binding constraint is and like what the binding constraint is and just like flood the zone personally on just like flood the zone personally on just like flood the zone personally on that thing which is kind of like going that thing which is kind of like going that thing which is kind of like going to Bangalore or something. It seems like to Bangalore or something. It seems like to Bangalore or something. It seems like this is this is a central tenant of the this is this is a central tenant of the this is this is a central tenant of the business and of any business that's business and of any business that's business and of any business that's going to do this kind of vertical going to do this kind of vertical going to do this kind of vertical integration. What's the key to doing integration. What's the key to doing integration. What's the key to doing that? Well, like again, what have you that? Well, like again, what have you that? Well, like again, what have you learned about that specific act? For me, learned about that specific act? For me, learned about that specific act? For me, I think there are two key tricks to I think there are two key tricks to I think there are two key tricks to this. The first one is that you can't this. The first one is that you can't this. The first one is that you can't build a chip alone. It's got to be a build a chip alone. It's got to be a build a chip alone. It's got to be a team problem. And your most important team problem. And your most important team problem. And your most important job is to go get great people to go with job is to go get great people to go with job is to go get great people to go with you and great people to go ahead and be you and great people to go ahead and be you and great people to go ahead and be inspired and excited to go ahead and do inspired and excited to go ahead and do inspired and excited to go ahead and do crazy things like this. It is a huge ask crazy things like this. It is a huge ask crazy things like this. It is a huge ask to go say, "Hey guys, uproot your lives to go say, "Hey guys, uproot your lives to go say, "Hey guys, uproot your lives for six months or in one case 12 for six months or in one case 12 for six months or in one case 12 months." That we had sent one guy out months." That we had sent one guy out months." That we had sent one guy out well ahead. It sucks, but we're lucky to well ahead. It sucks, but we're lucky to well ahead. It sucks, but we're lucky to have team members who are in it for the have team members who are in it for the have team members who are in it for the right reasons. But I think the second right reasons. But I think the second right reasons. But I think the second big thing, too, is being able to make big thing, too, is being able to make big thing, too, is being able to make decisions very fast. That one of the decisions very fast. That one of the decisions very fast. That one of the worst items is when there's a factory or worst items is when there's a factory or worst items is when there's a factory or there's a vendor who is waiting for you there's a vendor who is waiting for you there's a vendor who is waiting for you to go ahead and make some call and is to go ahead and make some call and is to go ahead and make some call and is then just stalled. then just stalled. then just stalled. >> And this happens all the time, even for >> And this happens all the time, even for >> And this happens all the time, even for very small things. So send folks, very small things. So send folks, very small things. So send folks, delegate a big amount of responsibility delegate a big amount of responsibility delegate a big amount of responsibility to them and say make a reasonable call. to them and say make a reasonable call. to them and say make a reasonable call. It's okay if you're wrong every now and It's okay if you're wrong every now and It's okay if you're wrong every now and then, but I would much much rather be then, but I would much much rather be then, but I would much much rather be right most of the time and give an right most of the time and give an right most of the time and give an answer immediately than wait every time answer immediately than wait every time answer immediately than wait every time for the perfect response. Speed wins. for the perfect response. Speed wins. for the perfect response. Speed wins. >> What about spending money to go faster? >> What about spending money to go faster? >> What about spending money to go faster? There's this learn by doing thing which There's this learn by doing thing which There's this learn by doing thing which has become so interesting and as the has become so interesting and as the has become so interesting and as the world has gone away from software and world has gone away from software and world has gone away from software and towards more hardware again in in the towards more hardware again in in the towards more hardware again in in the world of technology that uh we've world of technology that uh we've world of technology that uh we've outsourced so much of the learn by doing outsourced so much of the learn by doing outsourced so much of the learn by doing totally by shipping stuff overseas and totally by shipping stuff overseas and totally by shipping stuff overseas and effectively just being the idea guys effectively just being the idea guys effectively just being the idea guys here in the US. Seems like that here in the US. Seems like that here in the US. Seems like that obviously is reversing and you've obviously is reversing and you've obviously is reversing and you've adopted this way of learning by do like adopted this way of learning by do like adopted this way of learning by do like you want to be in that iteration you want to be in that iteration you want to be in that iteration learning loop. learning loop. learning loop. >> Absolutely. >> Absolutely. >> Absolutely. >> And and part of that is willingness to >> And and part of that is willingness to >> And and part of that is willingness to spend and take risk with dollars. Yeah. spend and take risk with dollars. Yeah. spend and take risk with dollars. Yeah. >> Can you talk about that a little bit? >> Can you talk about that a little bit? >> Can you talk about that a little bit? there I think there's a great quote of there I think there's a great quote of there I think there's a great quote of like the biggest risk is not taking risk like the biggest risk is not taking risk like the biggest risk is not taking risk very similar here which is like every very similar here which is like every very similar here which is like every day there's over a billion dollars of day there's over a billion dollars of day there's over a billion dollars of revenue in this category and a lot of revenue in this category and a lot of revenue in this category and a lot of it's inference uh so every day we don't it's inference uh so every day we don't it's inference uh so every day we don't ship we're just leaving tons of ship we're just leaving tons of ship we're just leaving tons of opportunity on the table um so your opportunity on the table um so your opportunity on the table um so your willingness to spend money should be willingness to spend money should be willingness to spend money should be extremely high uh if you can get a very extremely high uh if you can get a very extremely high uh if you can get a very clear ROI out of it so we have this clear ROI out of it so we have this clear ROI out of it so we have this concept that we call pre-fetching which concept that we call pre-fetching which concept that we call pre-fetching which is when you're waiting for one thing to is when you're waiting for one thing to is when you're waiting for one thing to get done when you know you're going to get done when you know you're going to get done when you know you're going to do other things once you have it is do other things once you have it is do other things once you have it is there ways that you and parallelize the there ways that you and parallelize the there ways that you and parallelize the entire schedule. So for example, like we entire schedule. So for example, like we entire schedule. So for example, like we know our chip is we know our chip is know our chip is we know our chip is know our chip is we know our chip is going to come back on a certain date. We going to come back on a certain date. We going to come back on a certain date. We want it to be that everything possible want it to be that everything possible want it to be that everything possible that could be done without the chip is that could be done without the chip is that could be done without the chip is done before the chip lands. And this done before the chip lands. And this done before the chip lands. And this costs a lot of money, right? This means costs a lot of money, right? This means costs a lot of money, right? This means that like we want to build our entire that like we want to build our entire that like we want to build our entire software stack beforehand. That means we software stack beforehand. That means we software stack beforehand. That means we want to actually like we shipped racks want to actually like we shipped racks want to actually like we shipped racks to customer data centers without our to customer data centers without our to customer data centers without our chips in them with all the networking, chips in them with all the networking, chips in them with all the networking, all the CPUs, all the storage all set up all the CPUs, all the storage all set up all the CPUs, all the storage all set up so we could bring all that data center so we could bring all that data center so we could bring all that data center software up before the chips came back. software up before the chips came back. software up before the chips came back. It meant that we took over 700 FPGAs and It meant that we took over 700 FPGAs and It meant that we took over 700 FPGAs and put the entire full reticle chip on an put the entire full reticle chip on an put the entire full reticle chip on an FPGA cluster and ran a dozen different FPGA cluster and ran a dozen different FPGA cluster and ran a dozen different models with our full inference stack on models with our full inference stack on models with our full inference stack on them before the chips came back. It them before the chips came back. It them before the chips came back. It means that we built a thermal chip to means that we built a thermal chip to means that we built a thermal chip to mock the thermal profile of our chip and mock the thermal profile of our chip and mock the thermal profile of our chip and built cold plates based on that before built cold plates based on that before built cold plates based on that before the chip came back. It means we had the the chip came back. It means we had the the chip came back. It means we had the entire production line ready. It means entire production line ready. It means entire production line ready. It means we did many revs of the circuit board. we did many revs of the circuit board. we did many revs of the circuit board. It means the entire product was ready to It means the entire product was ready to It means the entire product was ready to go before the chips came back. And this go before the chips came back. And this go before the chips came back. And this is what it gives you. There was another is what it gives you. There was another is what it gives you. There was another very famous AI chip company uh that took very famous AI chip company uh that took very famous AI chip company uh that took 10 months to go from getting their 10 months to go from getting their 10 months to go from getting their silicon back to having them running silicon back to having them running silicon back to having them running inference in Iraq. And this was like inference in Iraq. And this was like inference in Iraq. And this was like publicly announced to their investors publicly announced to their investors publicly announced to their investors and was it was it was a really big deal. and was it was it was a really big deal. and was it was it was a really big deal. Um we were able to do it in 40 days. And Um we were able to do it in 40 days. And Um we were able to do it in 40 days. And it's because by the time the chip came it's because by the time the chip came it's because by the time the chip came back, everything was boring. The back, everything was boring. The back, everything was boring. The software was already written, the rack software was already written, the rack software was already written, the rack was already there, the production line was already there, the production line was already there, the production line was already set up. We were just go go was already set up. We were just go go was already set up. We were just go go go get everything together. You know, go get everything together. You know, go get everything together. You know, you don't always catch everything. You you don't always catch everything. You you don't always catch everything. You make some tweaks on the fly and then off make some tweaks on the fly and then off make some tweaks on the fly and then off you go. you go. you go. >> Although in that particular case too, >> Although in that particular case too, >> Although in that particular case too, that was a big part of it. But also like that was a big part of it. But also like that was a big part of it. But also like the shift I think made a big difference the shift I think made a big difference the shift I think made a big difference too. too. too. >> Oh, totally. >> Oh, totally. >> Oh, totally. >> That like we went out and literally had >> That like we went out and literally had >> That like we went out and literally had a day shift and a night shift. There a day shift and a night shift. There a day shift and a night shift. There were team members who would go in, come were team members who would go in, come were team members who would go in, come in at 10:00 a.m. and leave at around in at 10:00 a.m. and leave at around in at 10:00 a.m. and leave at around midnight. Yeah. midnight. Yeah. midnight. Yeah. >> Which would come in at midnight and >> Which would come in at midnight and >> Which would come in at midnight and leave at 10 a.m. running around the leave at 10 a.m. running around the leave at 10 a.m. running around the clock to get to that 40 days. clock to get to that 40 days. clock to get to that 40 days. >> Yeah. I mean, over half the company >> Yeah. I mean, over half the company >> Yeah. I mean, over half the company lives next to the office. Uh so it makes lives next to the office. Uh so it makes lives next to the office. Uh so it makes it easier to do that type of thing. it easier to do that type of thing. it easier to do that type of thing. >> You pay them to do that, right? Pay them >> You pay them to do that, right? Pay them >> You pay them to do that, right? Pay them extra. Do you still do that? extra. Do you still do that? extra. Do you still do that? >> Yeah. the the invisible hand does >> Yeah. the the invisible hand does >> Yeah. the the invisible hand does wonders. wonders. wonders. >> I mean, hey, it works for me, too. >> I mean, hey, it works for me, too. >> I mean, hey, it works for me, too. >> We're both there. >> We're both there. >> We're both there. >> I'd love to take one big step back and >> I'd love to take one big step back and >> I'd love to take one big step back and talk a bit about just the broader talk a bit about just the broader talk a bit about just the broader ecosystem here. The amount of shortages ecosystem here. The amount of shortages ecosystem here. The amount of shortages um on the supply side, the exposure of um on the supply side, the exposure of um on the supply side, the exposure of risks in the global system and the risks in the global system and the risks in the global system and the supply chain around this stuff has supply chain around this stuff has supply chain around this stuff has become like everyday Wall Street Journal become like everyday Wall Street Journal become like everyday Wall Street Journal front page news. like the stocks that front page news. like the stocks that front page news. like the stocks that people are watching and investing in and people are watching and investing in and people are watching and investing in and excited about. You know, if you think excited about. You know, if you think excited about. You know, if you think about the memory stocks, these were, you about the memory stocks, these were, you about the memory stocks, these were, you know, boring commodity like nothing know, boring commodity like nothing know, boring commodity like nothing burgers five years ago and now they're burgers five years ago and now they're burgers five years ago and now they're the center of global attention. If you the center of global attention. If you the center of global attention. If you just assess because you've been building just assess because you've been building just assess because you've been building in it, the global connected supply chain in it, the global connected supply chain in it, the global connected supply chain that's required to make stuff like this that's required to make stuff like this that's required to make stuff like this possible. Just riff on it. Like what possible. Just riff on it. Like what possible. Just riff on it. Like what scares you? What's working well? What scares you? What's working well? What scares you? What's working well? What needs to change? What do you hope you needs to change? What do you hope you needs to change? What do you hope you change by virtue of how you build this change by virtue of how you build this change by virtue of how you build this thing? like just what's your assessment thing? like just what's your assessment thing? like just what's your assessment of of this story right now? of of this story right now? of of this story right now? >> I think that one of the most undervalued >> I think that one of the most undervalued >> I think that one of the most undervalued pieces of the supply chain story is pieces of the supply chain story is pieces of the supply chain story is almost none of these things are you buy almost none of these things are you buy almost none of these things are you buy them and you don't talk to the vendor them and you don't talk to the vendor them and you don't talk to the vendor again. You have to go collaborate. That again. You have to go collaborate. That again. You have to go collaborate. That is the most important part being is the most important part being is the most important part being successful I think in ships with TSMC or successful I think in ships with TSMC or successful I think in ships with TSMC or with memory vendors. You need that with memory vendors. You need that with memory vendors. You need that partnership. I think that for TSMC in partnership. I think that for TSMC in partnership. I think that for TSMC in particular, people don't understand why particular, people don't understand why particular, people don't understand why it is so valuable. You look at the tech it is so valuable. You look at the tech it is so valuable. You look at the tech and the tech is the best in the world. and the tech is the best in the world. and the tech is the best in the world. But for me, the real value is all in the But for me, the real value is all in the But for me, the real value is all in the service. The TSMC customer service is service. The TSMC customer service is service. The TSMC customer service is way way better than I have seen at any way way better than I have seen at any way way better than I have seen at any other company in any other industry. other company in any other industry. other company in any other industry. It's the kind of thing where if you say, It's the kind of thing where if you say, It's the kind of thing where if you say, "Hey, we you can improve your yield by "Hey, we you can improve your yield by "Hey, we you can improve your yield by making this change. You can go make them making this change. You can go make them making this change. You can go make them a recommendation and then we'll go run a recommendation and then we'll go run a recommendation and then we'll go run an experiment on their own dime in our an experiment on their own dime in our an experiment on their own dime in our case to see if they could actually get case to see if they could actually get case to see if they could actually get the higher yield." And when we found the higher yield." And when we found the higher yield." And when we found that we were right and the experiment that we were right and the experiment that we were right and the experiment worked, they moved over the rest of the worked, they moved over the rest of the worked, they moved over the rest of the line. And that kind of thing just line. And that kind of thing just line. And that kind of thing just doesn't happen in most most industries. doesn't happen in most most industries. doesn't happen in most most industries. If I go to like the steel works plant, If I go to like the steel works plant, If I go to like the steel works plant, say, "Hey, I want you to change the say, "Hey, I want you to change the say, "Hey, I want you to change the composition of the steel." And they'll composition of the steel." And they'll composition of the steel." And they'll say, "Screw you. Not TSMC. It is why say, "Screw you. Not TSMC. It is why say, "Screw you. Not TSMC. It is why they are the number one and why they're they are the number one and why they're they are the number one and why they're they're going to win." they're going to win." they're going to win." >> One of the things that matters a ton is >> One of the things that matters a ton is >> One of the things that matters a ton is power availability and time to power. power availability and time to power. power availability and time to power. And the problem is uh the more power you And the problem is uh the more power you And the problem is uh the more power you want, the more shortage there is. It's want, the more shortage there is. It's want, the more shortage there is. It's actually very similar to chip clusters, actually very similar to chip clusters, actually very similar to chip clusters, which is like why is Colossus charging which is like why is Colossus charging which is like why is Colossus charging $12 an hour for black holes? It's $12 an hour for black holes? It's $12 an hour for black holes? It's because they're the only place you can because they're the only place you can because they're the only place you can buy 20,000 of them at once, right? Like buy 20,000 of them at once, right? Like buy 20,000 of them at once, right? Like why is the like 500 megawatt data center why is the like 500 megawatt data center why is the like 500 megawatt data center so hard to find? It's it's the exact so hard to find? It's it's the exact so hard to find? It's it's the exact same reason. You know, one of the things same reason. You know, one of the things same reason. You know, one of the things we need to think about is like how do we we need to think about is like how do we we need to think about is like how do we get way more juice out of each megawatt? get way more juice out of each megawatt? get way more juice out of each megawatt? Uh and like people are looking Uh and like people are looking Uh and like people are looking throughout the entire stack, whether throughout the entire stack, whether throughout the entire stack, whether it's just improving the PUE, but also it's just improving the PUE, but also it's just improving the PUE, but also entirely new hardware to get the most entirely new hardware to get the most entirely new hardware to get the most tokens per megawatt to solve this tokens per megawatt to solve this tokens per megawatt to solve this problem. Um but fundamentally like problem. Um but fundamentally like problem. Um but fundamentally like building new buildings is hard. Uh it's building new buildings is hard. Uh it's building new buildings is hard. Uh it's much easier to go from a 100 megawatt to much easier to go from a 100 megawatt to much easier to go from a 100 megawatt to a gigawatt than from a gigawatt to 10 a gigawatt than from a gigawatt to 10 a gigawatt than from a gigawatt to 10 and 10 to 100 and you know uh we are and 10 to 100 and you know uh we are and 10 to 100 and you know uh we are pushing the limits of of what's possible pushing the limits of of what's possible pushing the limits of of what's possible on these timelines. So there's a lot of on these timelines. So there's a lot of on these timelines. So there's a lot of people trying to scale in their data people trying to scale in their data people trying to scale in their data centers as much as trying to scale them centers as much as trying to scale them centers as much as trying to scale them out. out. out. >> Yeah. I mean one of the one of the >> Yeah. I mean one of the one of the >> Yeah. I mean one of the one of the interesting things about a system like interesting things about a system like interesting things about a system like this is what it replaces. Yeah. this is what it replaces. Yeah. this is what it replaces. Yeah. >> So if if I think about a rack like this >> So if if I think about a rack like this >> So if if I think about a rack like this versus I don't know a set of black wells versus I don't know a set of black wells versus I don't know a set of black wells or something or Reubens or or whatever or something or Reubens or or whatever or something or Reubens or or whatever is coming next. How should I is coming next. How should I is coming next. How should I conceptualize that? Like because not conceptualize that? Like because not conceptualize that? Like because not it's not just watts, it's also physical it's not just watts, it's also physical it's not just watts, it's also physical space to your to your point that people space to your to your point that people space to your to your point that people all of a sudden cerebras talked about all of a sudden cerebras talked about all of a sudden cerebras talked about this in their in their recent earnings this in their in their recent earnings this in their in their recent earnings call that literally this is a problem call that literally this is a problem call that literally this is a problem that there's literally no space to put that there's literally no space to put that there's literally no space to put the systems. Um yeah, how how should I the systems. Um yeah, how how should I the systems. Um yeah, how how should I conceptualize what this uh represents or conceptualize what this uh represents or conceptualize what this uh represents or replaces in terms of other units of replaces in terms of other units of replaces in terms of other units of compute? Here's how customers think compute? Here's how customers think compute? Here's how customers think about stuff uh deploying models about stuff uh deploying models about stuff uh deploying models generally which is you know when I'm generally which is you know when I'm generally which is you know when I'm building a data center or I'm building a building a data center or I'm building a building a data center or I'm building a cluster it's not like in the abstract of cluster it's not like in the abstract of cluster it's not like in the abstract of like oh I like these chips and like you like oh I like these chips and like you like oh I like these chips and like you know this is the power footprint and so know this is the power footprint and so know this is the power footprint and so forth. It's like I have a real forth. It's like I have a real forth. It's like I have a real production workload I'm trying to serve production workload I'm trying to serve production workload I'm trying to serve and you know for my product to be useful and you know for my product to be useful and you know for my product to be useful there's a certain speed I need to serve there's a certain speed I need to serve there's a certain speed I need to serve it at and for certain products it's it at and for certain products it's it at and for certain products it's really fast and certain products it's really fast and certain products it's really fast and certain products it's really slow whatever the speed is this really slow whatever the speed is this really slow whatever the speed is this is my speed the question is in a given is my speed the question is in a given is my speed the question is in a given amount of power how many users can I amount of power how many users can I amount of power how many users can I serve while guaranteeing that speed serve while guaranteeing that speed serve while guaranteeing that speed >> so another way to put it is IS iso >> so another way to put it is IS iso >> so another way to put it is IS iso what's called interactivity what is my what's called interactivity what is my what's called interactivity what is my throughput we are just finishing kind of throughput we are just finishing kind of throughput we are just finishing kind of the early innings of of the AI the early innings of of the AI the early innings of of the AI infrastructure boom where people really infrastructure boom where people really infrastructure boom where people really just cared about speed you know GPUs just cared about speed you know GPUs just cared about speed you know GPUs were not able to reach a lot of the were not able to reach a lot of the were not able to reach a lot of the speeds of other types of chips like all speeds of other types of chips like all speeds of other types of chips like all these SRAM chips uh you know thousands these SRAM chips uh you know thousands these SRAM chips uh you know thousands of tokens per second and that enabled of tokens per second and that enabled of tokens per second and that enabled tons of new use cases that got people tons of new use cases that got people tons of new use cases that got people very excited. There's an entirely new very excited. There's an entirely new very excited. There's an entirely new wave of AI chips you know us being one wave of AI chips you know us being one wave of AI chips you know us being one of them that are all going to be able to of them that are all going to be able to of them that are all going to be able to hit these speeds. The question then is hit these speeds. The question then is hit these speeds. The question then is if you're hitting these speeds, what is if you're hitting these speeds, what is if you're hitting these speeds, what is the number of users you can serve at the the number of users you can serve at the the number of users you can serve at the same time? And by proxy, if I have a 100 same time? And by proxy, if I have a 100 same time? And by proxy, if I have a 100 megawatt data center, how many, you megawatt data center, how many, you megawatt data center, how many, you know, software agents can I run at the know, software agents can I run at the know, software agents can I run at the same time? So when people are doing that same time? So when people are doing that same time? So when people are doing that evaluation, our hardware is going to evaluation, our hardware is going to evaluation, our hardware is going to generally be able to get you an order of generally be able to get you an order of generally be able to get you an order of magnitude more concurrency at a given magnitude more concurrency at a given magnitude more concurrency at a given level of interactivity. So that directly level of interactivity. So that directly level of interactivity. So that directly translates into, you know, tokens per translates into, you know, tokens per translates into, you know, tokens per watt, tokens per dollar, all the things watt, tokens per dollar, all the things watt, tokens per dollar, all the things people care about when they're actually people care about when they're actually people care about when they're actually serving, you know, these giant mixture serving, you know, these giant mixture serving, you know, these giant mixture of expert models at scale. M there's of expert models at scale. M there's of expert models at scale. M there's these uh now famous interactivity these uh now famous interactivity these uh now famous interactivity curves, right? So you can pl not many curves, right? So you can pl not many curves, right? So you can pl not many people publish them. Um but you could people publish them. Um but you could people publish them. Um but you could see a Blackwell uh curve. You could see see a Blackwell uh curve. You could see see a Blackwell uh curve. You could see an AMD curve which is a little bit worse an AMD curve which is a little bit worse an AMD curve which is a little bit worse than Blackwells and it's still an $800 than Blackwells and it's still an $800 than Blackwells and it's still an $800 billion company. So if you think about billion company. So if you think about billion company. So if you think about what then the impacts are of shifting what then the impacts are of shifting what then the impacts are of shifting that curve not just a little bit further that curve not just a little bit further that curve not just a little bit further out but but much further out. Yeah. out but but much further out. Yeah. out but but much further out. Yeah. >> What are the things that most excite you >> What are the things that most excite you >> What are the things that most excite you about what this will enable? I mean, I about what this will enable? I mean, I about what this will enable? I mean, I want to go out and solve some of the want to go out and solve some of the want to go out and solve some of the hardest problems and I want to go solve hardest problems and I want to go solve hardest problems and I want to go solve these in much less time. I mean, there these in much less time. I mean, there these in much less time. I mean, there were things growing up that I was not were things growing up that I was not were things growing up that I was not sure I'd be able to uh live to see. For sure I'd be able to uh live to see. For sure I'd be able to uh live to see. For example, the unit disc conjecture, one example, the unit disc conjecture, one example, the unit disc conjecture, one of the things we talked about in of the things we talked about in of the things we talked about in college. college. college. >> Yeah. >> Yeah. >> Yeah. >> And I was not sure I'd see that proven >> And I was not sure I'd see that proven >> And I was not sure I'd see that proven in my life. in my life. in my life. >> And this was done by an AI model and it >> And this was done by an AI model and it >> And this was done by an AI model and it was done over a long period of time. But was done over a long period of time. But was done over a long period of time. But if you're able to then run the same if you're able to then run the same if you're able to then run the same model 10 times faster, you can go shrink model 10 times faster, you can go shrink model 10 times faster, you can go shrink the time to go have these breakthroughs. the time to go have these breakthroughs. the time to go have these breakthroughs. And there's a huge number of other And there's a huge number of other And there's a huge number of other problems in math like this as well that problems in math like this as well that problems in math like this as well that I worry it will take 100 a thousand I worry it will take 100 a thousand I worry it will take 100 a thousand years to go prove a thing like this. You years to go prove a thing like this. You years to go prove a thing like this. You can either have a much smarter model or can either have a much smarter model or can either have a much smarter model or a model of the same intelligence running a model of the same intelligence running a model of the same intelligence running much faster. You can then shrink that much faster. You can then shrink that much faster. You can then shrink that and I can see it. And it's so cool and I can see it. And it's so cool and I can see it. And it's so cool seeing these breakthroughs get made. I seeing these breakthroughs get made. I seeing these breakthroughs get made. I am so so excited to see much more of am so so excited to see much more of am so so excited to see much more of this happen. I think too often people this happen. I think too often people this happen. I think too often people think about tasks and applications and think about tasks and applications and think about tasks and applications and stuff in these very short time horizons. stuff in these very short time horizons. stuff in these very short time horizons. So, you know, doing a chat and it's like So, you know, doing a chat and it's like So, you know, doing a chat and it's like 50% faster is nice, but it's not like 50% faster is nice, but it's not like 50% faster is nice, but it's not like gamechanging. gamechanging. gamechanging. As these agents go longer and longer As these agents go longer and longer As these agents go longer and longer time horizon and the models get more and time horizon and the models get more and time horizon and the models get more and more capable, you're going to see more capable, you're going to see more capable, you're going to see gigantic bodies of work that would take gigantic bodies of work that would take gigantic bodies of work that would take months of compute. And we think about months of compute. And we think about months of compute. And we think about this in wall clock time. Like if you this in wall clock time. Like if you this in wall clock time. Like if you talk to a pre-training researcher at a talk to a pre-training researcher at a talk to a pre-training researcher at a lab, they'll tell you that wall clock lab, they'll tell you that wall clock lab, they'll tell you that wall clock time often is one of the most important time often is one of the most important time often is one of the most important things that matters. And what wall clock things that matters. And what wall clock things that matters. And what wall clock time means is the time from, you know, time means is the time from, you know, time means is the time from, you know, starting your run to finishing it to starting your run to finishing it to starting your run to finishing it to actually get data back. If if you can actually get data back. If if you can actually get data back. If if you can shrink this time, you know, from a shrink this time, you know, from a shrink this time, you know, from a six-month run to, you know, a two-month six-month run to, you know, a two-month six-month run to, you know, a two-month experiment, you're going to be able to experiment, you're going to be able to experiment, you're going to be able to do many more iterations and people will do many more iterations and people will do many more iterations and people will make changes on the model architectures make changes on the model architectures make changes on the model architectures to actually improve the wall clock time. to actually improve the wall clock time. to actually improve the wall clock time. Very similar here in terms of how we Very similar here in terms of how we Very similar here in terms of how we think about the use cases, which is, you think about the use cases, which is, you think about the use cases, which is, you know, the exciting part about super low know, the exciting part about super low know, the exciting part about super low latency decode is wall clock time on latency decode is wall clock time on latency decode is wall clock time on long horizon tasks becomes much shorter. long horizon tasks becomes much shorter. long horizon tasks becomes much shorter. So a year-long compute build would now So a year-long compute build would now So a year-long compute build would now take a month and that month-long compute take a month and that month-long compute take a month and that month-long compute build will now take 3 days and that build will now take 3 days and that build will now take 3 days and that 3-day compute build will now take 7 3-day compute build will now take 7 3-day compute build will now take 7 hours and so forth and so forth. Uh so hours and so forth and so forth. Uh so hours and so forth and so forth. Uh so that's the thing that I think is really that's the thing that I think is really that's the thing that I think is really hard to internalize because the models hard to internalize because the models hard to internalize because the models are just getting capable capable enough are just getting capable capable enough are just getting capable capable enough to do this stuff. I thought it was to do this stuff. I thought it was to do this stuff. I thought it was really cool months ago when uh cursor really cool months ago when uh cursor really cool months ago when uh cursor published that they had a bunch of published that they had a bunch of published that they had a bunch of coding agents build an entire browser coding agents build an entire browser coding agents build an entire browser from scratch in a week. Totally nuts. from scratch in a week. Totally nuts. from scratch in a week. Totally nuts. And that that will soon happen in under And that that will soon happen in under And that that will soon happen in under an hour. uh and there's going to be many an hour. uh and there's going to be many an hour. uh and there's going to be many of those types of things uh that are of those types of things uh that are of those types of things uh that are going to happen with these massive going to happen with these massive going to happen with these massive parallel agents all working on a given parallel agents all working on a given parallel agents all working on a given task. task. task. >> What are the ultimate limitations of >> What are the ultimate limitations of >> What are the ultimate limitations of these systems? Is this just like a these systems? Is this just like a these systems? Is this just like a physics question like like how many physics question like like how many physics question like like how many times faster cheaper can we get times faster cheaper can we get times faster cheaper can we get theoretically? Like how do you think theoretically? Like how do you think theoretically? Like how do you think >> there's a lot there's a lot of room at >> there's a lot there's a lot of room at >> there's a lot there's a lot of room at the bottom as they say that if you think the bottom as they say that if you think the bottom as they say that if you think about chip and chip latencies on an about chip and chip latencies on an about chip and chip latencies on an NVIDIA product you're looking at 4,000 NVIDIA product you're looking at 4,000 NVIDIA product you're looking at 4,000 nconds to go from one chip to another. nconds to go from one chip to another. nconds to go from one chip to another. Mhm. Mhm. Mhm. >> We'll be able to do much better than >> We'll be able to do much better than >> We'll be able to do much better than that. that. that. >> What's the mathematical limit >> What's the mathematical limit >> What's the mathematical limit >> is speed of light. >> is speed of light. >> is speed of light. >> You can do it in just a handful like 2 >> You can do it in just a handful like 2 >> You can do it in just a handful like 2 three nconds three nconds three nconds >> and they're 4,000 >> and they're 4,000 >> and they're 4,000 >> 4,000 today. There is a lot of room at >> 4,000 today. There is a lot of room at >> 4,000 today. There is a lot of room at the bottom. I think there are things the bottom. I think there are things the bottom. I think there are things like power efficiency that sure we're like power efficiency that sure we're like power efficiency that sure we're able to go and save huge amount by able to go and save huge amount by able to go and save huge amount by bringing the voltage down by so much. bringing the voltage down by so much. bringing the voltage down by so much. But you could go lower. You could go But you could go lower. You could go But you could go lower. You could go much much lower. It's very challenging. much much lower. It's very challenging. much much lower. It's very challenging. But when I think about 20 30 years in But when I think about 20 30 years in But when I think about 20 30 years in the future then I think it's inevitable the future then I think it's inevitable the future then I think it's inevitable and also for economies of scale for and also for economies of scale for and also for economies of scale for cluster scale up for a long time eight cluster scale up for a long time eight cluster scale up for a long time eight chips was the biggest scale up domain chips was the biggest scale up domain chips was the biggest scale up domain and it even had 72 bringing it to well and it even had 72 bringing it to well and it even had 72 bringing it to well 72 but you can be way way bigger you 72 but you can be way way bigger you 72 but you can be way way bigger you look at like a fab for example you have look at like a fab for example you have look at like a fab for example you have a $40 billion single monolithic building a $40 billion single monolithic building a $40 billion single monolithic building with only a handful of lines running with only a handful of lines running with only a handful of lines running through it you could have the same kind through it you could have the same kind through it you could have the same kind of thing for some futuristic mega of thing for some futuristic mega of thing for some futuristic mega cluster $40 billion hundred billion as a cluster $40 billion hundred billion as a cluster $40 billion hundred billion as a giant mega token factory serving one or giant mega token factory serving one or giant mega token factory serving one or a handful of models for a massive number a handful of models for a massive number a handful of models for a massive number of users to get that same economies of of users to get that same economies of of users to get that same economies of scale thing. Same model, massive number scale thing. Same model, massive number scale thing. Same model, massive number of people. of people. of people. >> You you mentioned kernels engineering >> You you mentioned kernels engineering >> You you mentioned kernels engineering and that being your first job that has and that being your first job that has and that being your first job that has emerged as a thing that nobody had ever emerged as a thing that nobody had ever emerged as a thing that nobody had ever heard of in their lives to now something heard of in their lives to now something heard of in their lives to now something that you hear about all the time. the that you hear about all the time. the that you hear about all the time. the importance of it to ek more performance importance of it to ek more performance importance of it to ek more performance out of the you know the bare raw metal out of the you know the bare raw metal out of the you know the bare raw metal when will that just be something that AI when will that just be something that AI when will that just be something that AI doesn't entirely as well are humans doesn't entirely as well are humans doesn't entirely as well are humans still the best kernel engineers are they still the best kernel engineers are they still the best kernel engineers are they doing it with the assistance of AI doing it with the assistance of AI doing it with the assistance of AI systems like how far down will humans systems like how far down will humans systems like how far down will humans still be in the loop of designing these still be in the loop of designing these still be in the loop of designing these things like when will that go away things like when will that go away things like when will that go away >> today it's all very hybrid and the best >> today it's all very hybrid and the best >> today it's all very hybrid and the best kernels are still written by human AI kernels are still written by human AI kernels are still written by human AI collaborations well so any AI model is collaborations well so any AI model is collaborations well so any AI model is built up of these fundamental primitives built up of these fundamental primitives built up of these fundamental primitives like mat moles like convolutions like uh like mat moles like convolutions like uh like mat moles like convolutions like uh chipto-chip operations collectives and chipto-chip operations collectives and chipto-chip operations collectives and making these overlap and making these making these overlap and making these making these overlap and making these really fast matters enormously and it's really fast matters enormously and it's really fast matters enormously and it's the kernel designer's job to go ahead the kernel designer's job to go ahead the kernel designer's job to go ahead and figure out where can I overlap how and figure out where can I overlap how and figure out where can I overlap how do I allocate memory how do I verify do I allocate memory how do I verify do I allocate memory how do I verify that if there's some issue like a that if there's some issue like a that if there's some issue like a retransmit needed it doesn't stall the retransmit needed it doesn't stall the retransmit needed it doesn't stall the whole pipeline whole pipeline whole pipeline >> these things are very challenging but >> these things are very challenging but >> these things are very challenging but they can go make your performance be say they can go make your performance be say they can go make your performance be say 3 4% better per optimization and you can 3 4% better per optimization and you can 3 4% better per optimization and you can do so And we thought about our software do so And we thought about our software do so And we thought about our software stack. We wanted to go skate where the stack. We wanted to go skate where the stack. We wanted to go skate where the puck is going to be. And three years ago puck is going to be. And three years ago puck is going to be. And three years ago there were kind of two ways you could there were kind of two ways you could there were kind of two ways you could build software. One of them was to build software. One of them was to build software. One of them was to invest heavily in graph compilers. These invest heavily in graph compilers. These invest heavily in graph compilers. These things are not very performant but they things are not very performant but they things are not very performant but they work out of the box. They don't require work out of the box. They don't require work out of the box. They don't require a human to go come in and tweak all the a human to go come in and tweak all the a human to go come in and tweak all the kernels. But we went the opposite kernels. But we went the opposite kernels. But we went the opposite direction. We are kernels first direction. We are kernels first direction. We are kernels first programming. And that means that yeah it programming. And that means that yeah it programming. And that means that yeah it for a long time did not work out of the for a long time did not work out of the for a long time did not work out of the box. But if you were a colonel's expert, box. But if you were a colonel's expert, box. But if you were a colonel's expert, you could get incredibly incredibly high you could get incredibly incredibly high you could get incredibly incredibly high performance. And the thing about this is performance. And the thing about this is performance. And the thing about this is that now as the coding models get better that now as the coding models get better that now as the coding models get better and better, they're doing more and more and better, they're doing more and more and better, they're doing more and more of the kernel generation task. And when of the kernel generation task. And when of the kernel generation task. And when the models keep getting smarter, it'll the models keep getting smarter, it'll the models keep getting smarter, it'll eventually do all of it. They will eventually do all of it. They will eventually do all of it. They will become superhuman. So we're going to become superhuman. So we're going to become superhuman. So we're going to build for where the world is going. And build for where the world is going. And build for where the world is going. And even today, we think about our profiling even today, we think about our profiling even today, we think about our profiling tools or a debugging stack. We think tools or a debugging stack. We think tools or a debugging stack. We think about it from how will the model use about it from how will the model use about it from how will the model use these tools more than we think about how these tools more than we think about how these tools more than we think about how will humans use these tools. will humans use these tools. will humans use these tools. >> That's we sometimes run experiments >> That's we sometimes run experiments >> That's we sometimes run experiments internally and we had codeex actually internally and we had codeex actually internally and we had codeex actually get GPOSS running from scratch just get GPOSS running from scratch just get GPOSS running from scratch just based off of our docs uh completely by based off of our docs uh completely by based off of our docs uh completely by itself. itself. itself. >> Wow. >> Wow. >> Wow. >> And it did it I think overnight. Um, we >> And it did it I think overnight. Um, we >> And it did it I think overnight. Um, we think about game selection a lot and think about game selection a lot and think about game selection a lot and what we mean by that is making sure what we mean by that is making sure what we mean by that is making sure we're investing our energy in the right we're investing our energy in the right we're investing our energy in the right bets because regardless of what you bets because regardless of what you bets because regardless of what you choose to work on it will take choose to work on it will take choose to work on it will take tremendous effort and you know one of tremendous effort and you know one of tremendous effort and you know one of the things that we started with was you the things that we started with was you the things that we started with was you know the decision explicitly not to know the decision explicitly not to know the decision explicitly not to build an arbitrary graph compiler not to build an arbitrary graph compiler not to build an arbitrary graph compiler not to support arbitrary pietorch not to support arbitrary pietorch not to support arbitrary pietorch not to support arbitrary CUDA not to support support arbitrary CUDA not to support support arbitrary CUDA not to support arbitrary onyx graphs uh but instead we arbitrary onyx graphs uh but instead we arbitrary onyx graphs uh but instead we envisioned a world where there was going envisioned a world where there was going envisioned a world where there was going to be under a hundred models that to be under a hundred models that to be under a hundred models that actually mattered and they were all actually mattered and they were all actually mattered and they were all going to very similar from the going to very similar from the going to very similar from the underlying mathematical perspective and underlying mathematical perspective and underlying mathematical perspective and that we were going to build primitives that we were going to build primitives that we were going to build primitives using physics that were going to using physics that were going to using physics that were going to accelerate these as much as humanly accelerate these as much as humanly accelerate these as much as humanly possible and we were going to allow the possible and we were going to allow the possible and we were going to allow the most sophisticated customers to have most sophisticated customers to have most sophisticated customers to have direct access to the hardware and do direct access to the hardware and do direct access to the hardware and do whatever they want and that has saved us whatever they want and that has saved us whatever they want and that has saved us a tremendous amount of time not having a tremendous amount of time not having a tremendous amount of time not having to build a compiler and that has allowed to build a compiler and that has allowed to build a compiler and that has allowed us to actually get much more performance us to actually get much more performance us to actually get much more performance and funnily enough uh when we started a and funnily enough uh when we started a and funnily enough uh when we started a lot of people dismissed this idea uh and lot of people dismissed this idea uh and lot of people dismissed this idea uh and the only people that took us seriously the only people that took us seriously the only people that took us seriously were in high frequency trading uh were in high frequency trading uh were in high frequency trading uh because they all they all hate compilers because they all they all hate compilers because they all they all hate compilers too they all write their own kernels too they all write their own kernels too they all write their own kernels uh and we've had dozens of people from uh and we've had dozens of people from uh and we've had dozens of people from high frequency trading join the team high frequency trading join the team high frequency trading join the team because they saw this philosophy too. because they saw this philosophy too. because they saw this philosophy too. >> What are the limits to vertical >> What are the limits to vertical >> What are the limits to vertical integration? Like where do you how do integration? Like where do you how do integration? Like where do you how do you know where to draw the line? And I'm you know where to draw the line? And I'm you know where to draw the line? And I'm I'm starting with this question to talk I'm starting with this question to talk I'm starting with this question to talk a bit about the broader market. The a bit about the broader market. The a bit about the broader market. The circumstances of the broader market are circumstances of the broader market are circumstances of the broader market are really interesting to me where the vast really interesting to me where the vast really interesting to me where the vast majority of chips of AI chips get bought majority of chips of AI chips get bought majority of chips of AI chips get bought by a very small set of customers. Yeah. by a very small set of customers. Yeah. by a very small set of customers. Yeah. >> Many of those customers are themselves >> Many of those customers are themselves >> Many of those customers are themselves trying to design their own AI chips. trying to design their own AI chips. trying to design their own AI chips. OpenAI announced Jalapeno. It seems like OpenAI announced Jalapeno. It seems like OpenAI announced Jalapeno. It seems like this very funny circumstance where like this very funny circumstance where like this very funny circumstance where like the most valuable thing in the world all the most valuable thing in the world all the most valuable thing in the world all kind of flows through a couple chip kind of flows through a couple chip kind of flows through a couple chip makers, a couple chip buyers. They all makers, a couple chip buyers. They all makers, a couple chip buyers. They all seem to be kind of doing thinking about seem to be kind of doing thinking about seem to be kind of doing thinking about doing each other's job. And then you've doing each other's job. And then you've doing each other's job. And then you've got the circumstance where like okay got the circumstance where like okay got the circumstance where like okay then these things go in a data center then these things go in a data center then these things go in a data center and you've got neoclouds and inference and you've got neoclouds and inference and you've got neoclouds and inference providers and this other part of the providers and this other part of the providers and this other part of the stack. Um you've got model builders and stack. Um you've got model builders and stack. Um you've got model builders and providers. Like I can imagine a world providers. Like I can imagine a world providers. Like I can imagine a world where because you have the best hardware where because you have the best hardware where because you have the best hardware you design models and you build data you design models and you build data you design models and you build data centers you know like you leak outside centers you know like you leak outside centers you know like you leak outside of your current vertical. So like how do of your current vertical. So like how do of your current vertical. So like how do you think about where to draw the lines you think about where to draw the lines you think about where to draw the lines for the business? for the business? for the business? >> We have a saying that production is the >> We have a saying that production is the >> We have a saying that production is the product. product. product. >> Ultimately what matters here is we know >> Ultimately what matters here is we know >> Ultimately what matters here is we know inference is going to be the biggest inference is going to be the biggest inference is going to be the biggest market in the world. Whoever produces market in the world. Whoever produces market in the world. Whoever produces the most tokens is going to be the most the most tokens is going to be the most the most tokens is going to be the most valuable company in the world. So all valuable company in the world. So all valuable company in the world. So all the decisions we make is how do we get the decisions we make is how do we get the decisions we make is how do we get the most token capacity online as the most token capacity online as the most token capacity online as possible and part of that is building a possible and part of that is building a possible and part of that is building a really good product that's has way more really good product that's has way more really good product that's has way more throughput that can run at way better throughput that can run at way better throughput that can run at way better latencies and so forth so we can you latencies and so forth so we can you latencies and so forth so we can you know per chip we make get way more know per chip we make get way more know per chip we make get way more tokens online. Another part of it is tokens online. Another part of it is tokens online. Another part of it is like not doing parts of the stack unless like not doing parts of the stack unless like not doing parts of the stack unless we absolutely have to to get to giant we absolutely have to to get to giant we absolutely have to to get to giant scale. So there are parts that we scale. So there are parts that we scale. So there are parts that we decided to do because it was absolutely decided to do because it was absolutely decided to do because it was absolutely required to get to scale like building required to get to scale like building required to get to scale like building the rack instead of just building the the rack instead of just building the the rack instead of just building the chips. Uh like doing a CM model instead chips. Uh like doing a CM model instead chips. Uh like doing a CM model instead of doing a JDM model. But you know there of doing a JDM model. But you know there of doing a JDM model. But you know there are parts of it that are kind of noise are parts of it that are kind of noise are parts of it that are kind of noise to us right now. Like we're not going to us right now. Like we're not going to us right now. Like we're not going and building our own data centers today. and building our own data centers today. and building our own data centers today. That doesn't actually help us get more That doesn't actually help us get more That doesn't actually help us get more capacity online. You know in general our capacity online. You know in general our capacity online. You know in general our customers are actually making power and customers are actually making power and customers are actually making power and moving their clusters around to get our moving their clusters around to get our moving their clusters around to get our chips online because they're such high chips online because they're such high chips online because they're such high throughput. Um if there was a world throughput. Um if there was a world throughput. Um if there was a world where other things were a constraint we where other things were a constraint we where other things were a constraint we would totally go and integrate with would totally go and integrate with would totally go and integrate with them. But the reality is we're just them. But the reality is we're just them. But the reality is we're just purely focused on getting as many tokens purely focused on getting as many tokens purely focused on getting as many tokens online as possible. I think it just online as possible. I think it just online as possible. I think it just comes down to economies of scale. Again, comes down to economies of scale. Again, comes down to economies of scale. Again, that at certain parts of the stack there that at certain parts of the stack there that at certain parts of the stack there are huge economies of scale and others are huge economies of scale and others are huge economies of scale and others there aren't. For example, on designing there aren't. For example, on designing there aren't. For example, on designing models, huge economies of scale there. models, huge economies of scale there. models, huge economies of scale there. For chip fabrication, same story. But For chip fabrication, same story. But For chip fabrication, same story. But for example, if you think about building for example, if you think about building for example, if you think about building some small metal part inside of that some small metal part inside of that some small metal part inside of that rack, there's not that same effect. rack, there's not that same effect. rack, there's not that same effect. >> We think the natural boundaries are on >> We think the natural boundaries are on >> We think the natural boundaries are on the chip side, on the bottom, and the the chip side, on the bottom, and the the chip side, on the bottom, and the model layer at the top. and we'll fill model layer at the top. and we'll fill model layer at the top. and we'll fill the whole gap between. the whole gap between. the whole gap between. >> A few weeks ago, there was a guy who was >> A few weeks ago, there was a guy who was >> A few weeks ago, there was a guy who was running a next generation AI chip for, running a next generation AI chip for, running a next generation AI chip for, you know, one of one of the frontier you know, one of one of the frontier you know, one of one of the frontier companies. Uh, and he's trying to companies. Uh, and he's trying to companies. Uh, and he's trying to recruit one of our architects. Um, and recruit one of our architects. Um, and recruit one of our architects. Um, and this person uh actually kind of did an this person uh actually kind of did an this person uh actually kind of did an UNO reverse card and started recruiting UNO reverse card and started recruiting UNO reverse card and started recruiting the person trying to recruit our our guy the person trying to recruit our our guy the person trying to recruit our our guy and and within a week we hired him. Uh, and and within a week we hired him. Uh, and and within a week we hired him. Uh, and I was going on a walk as like we and I was going on a walk as like we and I was going on a walk as like we were kind of finalizing the offer. I was were kind of finalizing the offer. I was were kind of finalizing the offer. I was like, well, you're leading this super like, well, you're leading this super like, well, you're leading this super important project. Why are you deciding important project. Why are you deciding important project. Why are you deciding to join? And his answer was super to join? And his answer was super to join? And his answer was super interesting which was it fundamentally interesting which was it fundamentally interesting which was it fundamentally is not existential for my company for is not existential for my company for is not existential for my company for this product to win for Google with TPUs this product to win for Google with TPUs this product to win for Google with TPUs like their revenue comes from search. like their revenue comes from search. like their revenue comes from search. >> Google won't fail if TPUs fail. >> Google won't fail if TPUs fail. >> Google won't fail if TPUs fail. >> That's right. Meta won't fail if MTIA >> That's right. Meta won't fail if MTIA >> That's right. Meta won't fail if MTIA fails. Microsoft won't fail if Maya fails. Microsoft won't fail if Maya fails. Microsoft won't fail if Maya fails and OpenAI won't fail if Jalapeno fails and OpenAI won't fail if Jalapeno fails and OpenAI won't fail if Jalapeno fails. Ultimately this is our product. fails. Ultimately this is our product. fails. Ultimately this is our product. It it is like completely unsurprising It it is like completely unsurprising It it is like completely unsurprising that the best chip in the world is built that the best chip in the world is built that the best chip in the world is built by a company that only builds that chip. by a company that only builds that chip. by a company that only builds that chip. It's Nvidia, right? And like for us like It's Nvidia, right? And like for us like It's Nvidia, right? And like for us like it is completely existential for us to it is completely existential for us to it is completely existential for us to get as much to token capacity online as get as much to token capacity online as get as much to token capacity online as possible. And you know it recruits a set possible. And you know it recruits a set possible. And you know it recruits a set of talent and it recruits a support from of talent and it recruits a support from of talent and it recruits a support from suppliers and from customers that view suppliers and from customers that view suppliers and from customers that view it with the level of intensity that we it with the level of intensity that we it with the level of intensity that we do. Look at the raw flop stats. You do. Look at the raw flop stats. You do. Look at the raw flop stats. You compare any of these chips built by the compare any of these chips built by the compare any of these chips built by the labs or by the uh hyperscalers. The flop labs or by the uh hyperscalers. The flop labs or by the uh hyperscalers. The flop density for say FB8 times FB8 is lower density for say FB8 times FB8 is lower density for say FB8 times FB8 is lower than the black B300. than the black B300. than the black B300. >> Y >> Y >> Y >> and that makes sense because they don't >> and that makes sense because they don't >> and that makes sense because they don't have to go take the risk. they just have have to go take the risk. they just have have to go take the risk. they just have to go and build a similar enough product to go and build a similar enough product to go and build a similar enough product and not pay the Nvidia tax. and not pay the Nvidia tax. and not pay the Nvidia tax. >> As I think about you guys building the >> As I think about you guys building the >> As I think about you guys building the solution, you're the process of doing so solution, you're the process of doing so solution, you're the process of doing so is is solving a sequence of really hard is is solving a sequence of really hard is is solving a sequence of really hard challenges. What has been the single challenges. What has been the single challenges. What has been the single episode that was the hardest to episode that was the hardest to episode that was the hardest to overcome? overcome? overcome? >> When we were designing the chip, we >> When we were designing the chip, we >> When we were designing the chip, we built this massive massive FPGA cluster built this massive massive FPGA cluster built this massive massive FPGA cluster to go out and verify the full chip to go out and verify the full chip to go out and verify the full chip worked as is. And FPGAAS are digital worked as is. And FPGAAS are digital worked as is. And FPGAAS are digital entities. You can go test digital logic entities. You can go test digital logic entities. You can go test digital logic but not analog logic. And it turns out but not analog logic. And it turns out but not analog logic. And it turns out that when the chip came back, we began that when the chip came back, we began that when the chip came back, we began to go see issues in our attention to go see issues in our attention to go see issues in our attention adaptation of incorrect results. And we adaptation of incorrect results. And we adaptation of incorrect results. And we realized, wait a minute, there's a realized, wait a minute, there's a realized, wait a minute, there's a problem where the back pressuring logic problem where the back pressuring logic problem where the back pressuring logic across a clock domain crossing is across a clock domain crossing is across a clock domain crossing is failing. And this is going to cause the failing. And this is going to cause the failing. And this is going to cause the chip to produce wrong results. And it is chip to produce wrong results. And it is chip to produce wrong results. And it is very very hard to solve. And we realized very very hard to solve. And we realized very very hard to solve. And we realized that there was one and only one way to that there was one and only one way to that there was one and only one way to solve it as we had to go line up two solve it as we had to go line up two solve it as we had to go line up two clock signals on our chip to within 50 clock signals on our chip to within 50 clock signals on our chip to within 50 picosconds. picosconds. picosconds. That is literally 50 trillionths of a That is literally 50 trillionths of a That is literally 50 trillionths of a second. And we had to go get these second. And we had to go get these second. And we had to go get these signals aligned to this super small signals aligned to this super small signals aligned to this super small granularity and do it on every chip two granularity and do it on every chip two granularity and do it on every chip two billion times a second. billion times a second. billion times a second. >> A lot of people said this was >> A lot of people said this was >> A lot of people said this was impossible. impossible. impossible. >> We we had people quit. >> We we had people quit. >> We we had people quit. >> Yeah. That people literally were like >> Yeah. That people literally were like >> Yeah. That people literally were like this problem is unsolvable. And uh best this problem is unsolvable. And uh best this problem is unsolvable. And uh best of luck guys. Well, when you have a of luck guys. Well, when you have a of luck guys. Well, when you have a problem like that, step one is okay, problem like that, step one is okay, problem like that, step one is okay, let's assume the problem is solvable. let's assume the problem is solvable. let's assume the problem is solvable. How would it be solved? Well, first we How would it be solved? Well, first we How would it be solved? Well, first we realized what we have to be able to do realized what we have to be able to do realized what we have to be able to do is find a way to go ahead and move our is find a way to go ahead and move our is find a way to go ahead and move our clock phase by a picoscond, 10 clock phase by a picoscond, 10 clock phase by a picoscond, 10 picosconds. picosconds. picosconds. And we had an idea. What if we had these And we had an idea. What if we had these And we had an idea. What if we had these two clocks? We set them just a little two clocks? We set them just a little two clocks? We set them just a little bit apart from each other. And we bit apart from each other. And we bit apart from each other. And we figured out that hey, if we go out and figured out that hey, if we go out and figured out that hey, if we go out and figure out the phase, if we then go out figure out the phase, if we then go out figure out the phase, if we then go out and use the drifting mechanism to go and use the drifting mechanism to go and use the drifting mechanism to go ahead and wait for just the right amount ahead and wait for just the right amount ahead and wait for just the right amount of time to get that 50 picosconds always of time to get that 50 picosconds always of time to get that 50 picosconds always lined up, we can do this extremely lined up, we can do this extremely lined up, we can do this extremely reliably and then lock the phases reliably and then lock the phases reliably and then lock the phases exactly where they have to be. We can exactly where they have to be. We can exactly where they have to be. We can guarantee this never happens. People guarantee this never happens. People guarantee this never happens. People were, I think, somewhat blown away that were, I think, somewhat blown away that were, I think, somewhat blown away that this worked and it worked as well as it this worked and it worked as well as it this worked and it worked as well as it actually did. Uh, but actually did. Uh, but actually did. Uh, but we made it work. we made it work. we made it work. >> How long did that take? This is actually >> How long did that take? This is actually >> How long did that take? This is actually about two weeks that it about two weeks that it about two weeks that it >> a dark two weeks. >> a dark two weeks. >> a dark two weeks. >> It was a very scary two weeks, but it >> It was a very scary two weeks, but it >> It was a very scary two weeks, but it was the kind of thing where when that was the kind of thing where when that was the kind of thing where when that kind of thing happens, that is the most kind of thing happens, that is the most kind of thing happens, that is the most important time to go ahead and be important time to go ahead and be important time to go ahead and be investing effort. That that is the investing effort. That that is the investing effort. That that is the hardest time to go do it when you feel hardest time to go do it when you feel hardest time to go do it when you feel like things are hopeless. But like the like things are hopeless. But like the like things are hopeless. But like the sooner you solve that problem, the sooner you solve that problem, the sooner you solve that problem, the sooner you can get back to building and sooner you can get back to building and sooner you can get back to building and scaling up production to mass volumes. scaling up production to mass volumes. scaling up production to mass volumes. >> I think a lot of our story is like as >> I think a lot of our story is like as >> I think a lot of our story is like as Gavin says, assume it is possible. Like Gavin says, assume it is possible. Like Gavin says, assume it is possible. Like assume it is way possible like possible assume it is way possible like possible assume it is way possible like possible to have a chip with way more flops on to have a chip with way more flops on to have a chip with way more flops on it. Assume it is possible to have a it. Assume it is possible to have a it. Assume it is possible to have a system with way lower latency between system with way lower latency between system with way lower latency between chips. Assume it is possible to create a chips. Assume it is possible to create a chips. Assume it is possible to create a shared memory pool that can run at way shared memory pool that can run at way shared memory pool that can run at way higher bandwidth. Like how would one do higher bandwidth. Like how would one do higher bandwidth. Like how would one do it? A lot of the time when we do it? A lot of the time when we do it? A lot of the time when we do experiments, we will dozens of experiments, we will dozens of experiments, we will dozens of experiments and all of them will fail. experiments and all of them will fail. experiments and all of them will fail. But like we only need one to work. There But like we only need one to work. There But like we only need one to work. There was multiple times. I mean Gavin I think was multiple times. I mean Gavin I think was multiple times. I mean Gavin I think was uh was leading the charge uh when was uh was leading the charge uh when was uh was leading the charge uh when our and our chip bring up with I think our and our chip bring up with I think our and our chip bring up with I think 30 different board experiments and and 30 different board experiments and and 30 different board experiments and and three of them worked and all three of three of them worked and all three of three of them worked and all three of them are worth their weight in gold. them are worth their weight in gold. them are worth their weight in gold. >> This is one of the things where people >> This is one of the things where people >> This is one of the things where people come to me and say Gavin almost none of come to me and say Gavin almost none of come to me and say Gavin almost none of your experiments works and I would say I your experiments works and I would say I your experiments works and I would say I only got to get lucky once. only got to get lucky once. only got to get lucky once. >> Vanta automates security and compliance >> Vanta automates security and compliance >> Vanta automates security and compliance for over 16,000 fastmoving companies for over 16,000 fastmoving companies for over 16,000 fastmoving companies like Ramp Cursor and Harvey keeping them like Ramp Cursor and Harvey keeping them like Ramp Cursor and Harvey keeping them audit ready around the clock. It's the audit ready around the clock. It's the audit ready around the clock. It's the number one agentic trust platform and it number one agentic trust platform and it number one agentic trust platform and it now helps companies like yours watch for now helps companies like yours watch for now helps companies like yours watch for the risks that show up between audits the risks that show up between audits the risks that show up between audits across your vendors, your AI tools, and across your vendors, your AI tools, and across your vendors, your AI tools, and your whole environment. Every new tool your whole environment. Every new tool your whole environment. Every new tool your team signs up for, every vendor your team signs up for, every vendor your team signs up for, every vendor that turns on AI features is an that turns on AI features is an that turns on AI features is an opportunity for something to go wrong. opportunity for something to go wrong. opportunity for something to go wrong. And most security programs weren't built And most security programs weren't built And most security programs weren't built for AI's pace of growth. The Vant agent for AI's pace of growth. The Vant agent for AI's pace of growth. The Vant agent works like a 24/7 GRC engineer in the works like a 24/7 GRC engineer in the works like a 24/7 GRC engineer in the background, finding issues, drafting background, finding issues, drafting background, finding issues, drafting fixes for you, and cutting vendor fixes for you, and cutting vendor fixes for you, and cutting vendor assessment time by up to 50%. Whether assessment time by up to 50%. Whether assessment time by up to 50%. Whether you're a fast growing startup or a you're a fast growing startup or a you're a fast growing startup or a global enterprise, Vanta helps you earn global enterprise, Vanta helps you earn global enterprise, Vanta helps you earn and prove trust. Invest like the best and prove trust. Invest like the best and prove trust. Invest like the best listeners. Get a special offer for listeners. Get a special offer for listeners. Get a special offer for $1,000 off at vanta.com/invest. Ridgeline is the first endtoend system Ridgeline is the first endtoend system of record with embedded AI for of record with embedded AI for of record with embedded AI for investment management firms running investment management firms running investment management firms running portfolio accounting, reconciliation, portfolio accounting, reconciliation, portfolio accounting, reconciliation, reporting, trading, and compliance on reporting, trading, and compliance on reporting, trading, and compliance on one unified platform. Firms are moving one unified platform. Firms are moving one unified platform. Firms are moving off legacy technology and onto Ridgeline off legacy technology and onto Ridgeline off legacy technology and onto Ridgeline because of how far ahead Ridgeline's AI because of how far ahead Ridgeline's AI because of how far ahead Ridgeline's AI features are compared to anything else features are compared to anything else features are compared to anything else in investment management software. I've in investment management software. I've in investment management software. I've been hearing from a lot of investment been hearing from a lot of investment been hearing from a lot of investment managers about AI and they fall roughly managers about AI and they fall roughly managers about AI and they fall roughly into two camps with some unsure of where into two camps with some unsure of where into two camps with some unsure of where to even start and others convinced they to even start and others convinced they to even start and others convinced they can build their own order management can build their own order management can build their own order management system over a weekend. The reality is system over a weekend. The reality is system over a weekend. The reality is that running an investment firm will that running an investment firm will that running an investment firm will always require governance controls and a always require governance controls and a always require governance controls and a single source of truth for your data and single source of truth for your data and single source of truth for your data and no amount of AI enthusiasm changes that no amount of AI enthusiasm changes that no amount of AI enthusiasm changes that requirement. Ridgeline is built on requirement. Ridgeline is built on requirement. Ridgeline is built on exactly that foundation, which is why I exactly that foundation, which is why I exactly that foundation, which is why I believe that firms that come out ahead believe that firms that come out ahead believe that firms that come out ahead in the AI era will be the ones running in the AI era will be the ones running in the AI era will be the ones running on Ridgeline's unified platform. If on Ridgeline's unified platform. If on Ridgeline's unified platform. If you're serious about your firm's AI you're serious about your firm's AI you're serious about your firm's AI strategy, Ridgeline should be part of strategy, Ridgeline should be part of strategy, Ridgeline should be part of that conversation. You can request a that conversation. You can request a that conversation. You can request a demo at ridgeline.ai. demo at ridgeline.ai. demo at ridgeline.ai. So, one idea for um one of these stories So, one idea for um one of these stories So, one idea for um one of these stories that I'm asking about, you know, that I'm asking about, you know, that I'm asking about, you know, difficult moments in the company's difficult moments in the company's difficult moments in the company's history is around the ability to raise history is around the ability to raise history is around the ability to raise capital to fund the thing. I think when capital to fund the thing. I think when capital to fund the thing. I think when you started it, you knew you'd need you started it, you knew you'd need you started it, you knew you'd need capital, but you did not know you'd need capital, but you did not know you'd need capital, but you did not know you'd need the quantum of capital that you've the quantum of capital that you've the quantum of capital that you've ultimately um raised and and are ultimately um raised and and are ultimately um raised and and are spending to build the solution and you spending to build the solution and you spending to build the solution and you hadn't raised money before. These are hadn't raised money before. These are hadn't raised money before. These are all new things, right? And uh there was all new things, right? And uh there was all new things, right? And uh there was moments where it was really really moments where it was really really moments where it was really really difficult because I was I was there, I difficult because I was I was there, I difficult because I was I was there, I saw it. There moments where it was saw it. There moments where it was saw it. There moments where it was extremely difficult to raise the money extremely difficult to raise the money extremely difficult to raise the money that you did that without which the that you did that without which the that you did that without which the company would not exist. It would have company would not exist. It would have company would not exist. It would have died. And like many great stories, you died. And like many great stories, you died. And like many great stories, you know, there there are many near-death know, there there are many near-death know, there there are many near-death moments, but money specifically in this moments, but money specifically in this moments, but money specifically in this new world. This isn't software. You new world. This isn't software. You new world. This isn't software. You don't just need a little bit of money. don't just need a little bit of money. don't just need a little bit of money. And then maybe you could tell the story And then maybe you could tell the story And then maybe you could tell the story about um the true hardest part about about um the true hardest part about about um the true hardest part about raising money early on. Before you had raising money early on. Before you had raising money early on. Before you had something that you could show people and something that you could show people and something that you could show people and be so proud of and performance that you be so proud of and performance that you be so proud of and performance that you could show them and blow their blow could show them and blow their blow could show them and blow their blow their socks off, it was just you guys their socks off, it was just you guys their socks off, it was just you guys talking about an idea. But talk about talking about an idea. But talk about talking about an idea. But talk about the early difficulties raising money the early difficulties raising money the early difficulties raising money because it was pretty hardcore. because it was pretty hardcore. because it was pretty hardcore. >> We've had some intense moments. Um it >> We've had some intense moments. Um it >> We've had some intense moments. Um it reminds me of probably early 2024 reminds me of probably early 2024 reminds me of probably early 2024 uh before we raised our series A and we uh before we raised our series A and we uh before we raised our series A and we were at this point where we had done were at this point where we had done were at this point where we had done enough of the architecture, done enough enough of the architecture, done enough enough of the architecture, done enough of the design that we knew that the chip of the design that we knew that the chip of the design that we knew that the chip like architecture was sound. We had to like architecture was sound. We had to like architecture was sound. We had to go build it. There was a lot more to do. go build it. There was a lot more to do. go build it. There was a lot more to do. We were ready to go into what's called We were ready to go into what's called We were ready to go into what's called the physical design stage. We needed to the physical design stage. We needed to the physical design stage. We needed to sign an agreement with a physical design sign an agreement with a physical design sign an agreement with a physical design vendor which you know will cost you at vendor which you know will cost you at vendor which you know will cost you at least $40 $50 million. Um, and then we least $40 $50 million. Um, and then we least $40 $50 million. Um, and then we had this realization as the models were had this realization as the models were had this realization as the models were getting bigger and bigger and you're getting bigger and bigger and you're getting bigger and bigger and you're seeing these giant models come out that seeing these giant models come out that seeing these giant models come out that we were going to need to build the we were going to need to build the we were going to need to build the entire cluster, not just the chip, but entire cluster, not just the chip, but entire cluster, not just the chip, but we were going to need to build boards. we were going to need to build boards. we were going to need to build boards. We're going to need to build We're going to need to build We're going to need to build interconnects. We need to build cold interconnects. We need to build cold interconnects. We need to build cold plates. We're going to need to figure plates. We're going to need to figure plates. We're going to need to figure out all of the networking and out all of the networking and out all of the networking and everything. And that this was going to everything. And that this was going to everything. And that this was going to cost a lot more than the $15 million we cost a lot more than the $15 million we cost a lot more than the $15 million we had in the bank. had in the bank. had in the bank. >> And you're like, man, that was scary. >> And you're like, man, that was scary. >> And you're like, man, that was scary. >> Yeah. >> Yeah. >> Yeah. >> That like you're sitting in that moment, >> That like you're sitting in that moment, >> That like you're sitting in that moment, you think, holy crap, we can't afford you think, holy crap, we can't afford you think, holy crap, we can't afford this. Uh, and I began looking at like, this. Uh, and I began looking at like, this. Uh, and I began looking at like, huh, how hard is it to go back to huh, how hard is it to go back to huh, how hard is it to go back to Harvard? Harvard? Harvard? >> I mean, I mean, at the end of 23, we put >> I mean, I mean, at the end of 23, we put >> I mean, I mean, at the end of 23, we put together this like memo. I mean, we together this like memo. I mean, we together this like memo. I mean, we spent like a hundred hours on this cuz spent like a hundred hours on this cuz spent like a hundred hours on this cuz we're like, we're like, we're like, >> we have no idea how people are going to >> we have no idea how people are going to >> we have no idea how people are going to believe us when we ask for the amount of believe us when we ask for the amount of believe us when we ask for the amount of money we're about to ask for. Uh, and it money we're about to ask for. Uh, and it money we're about to ask for. Uh, and it was like 30 pages. It was like extremely was like 30 pages. It was like extremely was like 30 pages. It was like extremely technical and in-depth of all the technical and in-depth of all the technical and in-depth of all the different things we needed to build and different things we needed to build and different things we needed to build and like all the milestones we needed to hit like all the milestones we needed to hit like all the milestones we needed to hit and how the market was going to evolve and how the market was going to evolve and how the market was going to evolve and all the new use cases and the cost and all the new use cases and the cost and all the new use cases and the cost per token and all this modeling. And per token and all this modeling. And per token and all this modeling. And then we went and talked to investors and then we went and talked to investors and then we went and talked to investors and every major investor in the valley every major investor in the valley every major investor in the valley passed immediately. They were just like, passed immediately. They were just like, passed immediately. They were just like, "Okay, two kids that just finished "Okay, two kids that just finished "Okay, two kids that just finished Harvard, haven't taped out a chip, no Harvard, haven't taped out a chip, no Harvard, haven't taped out a chip, no test chip, you know, inference like you test chip, you know, inference like you test chip, you know, inference like you know, who knows if this is going to be a know, who knows if this is going to be a know, who knows if this is going to be a big market. Everything's going to be big market. Everything's going to be big market. Everything's going to be training. uh you know the models still training. uh you know the models still training. uh you know the models still hallucinate. This could all you know all hallucinate. This could all you know all hallucinate. This could all you know all be a bubble. You know at the time the be a bubble. You know at the time the be a bubble. You know at the time the biggest uh semiconductor fundraisers for biggest uh semiconductor fundraisers for biggest uh semiconductor fundraisers for a series A was around like 40 $50 a series A was around like 40 $50 a series A was around like 40 $50 million. We were looking at this and we million. We were looking at this and we million. We were looking at this and we were just like tallying the bill. We're were just like tallying the bill. We're were just like tallying the bill. We're like we think we're going to spend $100 like we think we're going to spend $100 like we think we're going to spend $100 million in the next 12 months. Like if million in the next 12 months. Like if million in the next 12 months. Like if we really want to do this like if we we really want to do this like if we we really want to do this like if we want to actually get to scale and like want to actually get to scale and like want to actually get to scale and like actually get the performance we're actually get the performance we're actually get the performance we're talking about like this is going to be talking about like this is going to be talking about like this is going to be extremely capital intensive. How the extremely capital intensive. How the extremely capital intensive. How the hell are we going to pull this off? I hell are we going to pull this off? I hell are we going to pull this off? I think that one of our key uh ways we got think that one of our key uh ways we got think that one of our key uh ways we got started in this process, started in this process, started in this process, >> we thought to ourselves, what is the >> we thought to ourselves, what is the >> we thought to ourselves, what is the cheapest possible way we do this? And we cheapest possible way we do this? And we cheapest possible way we do this? And we decided, well, if uh I made almost decided, well, if uh I made almost decided, well, if uh I made almost nothing. nothing. nothing. >> Yeah. >> Yeah. >> Yeah. >> And if I ate nothing but ramen, then we >> And if I ate nothing but ramen, then we >> And if I ate nothing but ramen, then we would go ahead and spend basically just would go ahead and spend basically just would go ahead and spend basically just the money for the mask of 11 tape and the money for the mask of 11 tape and the money for the mask of 11 tape and that would be that, right? that would be that, right? that would be that, right? >> And if so, we could probably do it on >> And if so, we could probably do it on >> And if so, we could probably do it on $30 million, an obsanely low number. And $30 million, an obsanely low number. And $30 million, an obsanely low number. And uh we actually went out and got uh a uh we actually went out and got uh a uh we actually went out and got uh a debt provider to be willing to go ahead debt provider to be willing to go ahead debt provider to be willing to go ahead and lend us the money we needed to go and lend us the money we needed to go and lend us the money we needed to go cross this barely barely uh ramen to a cross this barely barely uh ramen to a cross this barely barely uh ramen to a chip threshold. From there I think it chip threshold. From there I think it chip threshold. From there I think it was first to go catalyze a series of uh was first to go catalyze a series of uh was first to go catalyze a series of uh other hey maybe we can go do one more other hey maybe we can go do one more other hey maybe we can go do one more thing one more thing. One more thing. thing one more thing. One more thing. thing one more thing. One more thing. Yeah. So we're at this moment where Yeah. So we're at this moment where Yeah. So we're at this moment where we're like if we really want to build we're like if we really want to build we're like if we really want to build this company because we're not going to this company because we're not going to this company because we're not going to halfass it. Like we're not going to go halfass it. Like we're not going to go halfass it. Like we're not going to go do a test chip and spend years on it and do a test chip and spend years on it and do a test chip and spend years on it and like let the entire AI market boom like like let the entire AI market boom like like let the entire AI market boom like while we could be building the product. while we could be building the product. while we could be building the product. Like if we're going to do it, we're Like if we're going to do it, we're Like if we're going to do it, we're gonna go all the way. We're going to gonna go all the way. We're going to gonna go all the way. We're going to need to find a way to get $100 million. need to find a way to get $100 million. need to find a way to get $100 million. I mean, I remember Gavin and I were like I mean, I remember Gavin and I were like I mean, I remember Gavin and I were like sitting down in the office in Certino sitting down in the office in Certino sitting down in the office in Certino late at night just like looking at each late at night just like looking at each late at night just like looking at each other and we're like, could we cut 500k other and we're like, could we cut 500k other and we're like, could we cut 500k here? Could we cut 100K here? How long here? Could we cut 100K here? How long here? Could we cut 100K here? How long could we convince everyone not to take a could we convince everyone not to take a could we convince everyone not to take a salary? And we're like, holy the salary? And we're like, holy the salary? And we're like, holy the math is not going to close. we we we math is not going to close. we we we math is not going to close. we we we really need to solve this. And there was really need to solve this. And there was really need to solve this. And there was a period of a few weeks where you kind a period of a few weeks where you kind a period of a few weeks where you kind of just go into survival mode and you of just go into survival mode and you of just go into survival mode and you call every person that could possibly call every person that could possibly call every person that could possibly know an investor and you're like, "We know an investor and you're like, "We know an investor and you're like, "We need $100 million to do this. If we do need $100 million to do this. If we do need $100 million to do this. If we do this, we think this could be one of the this, we think this could be one of the this, we think this could be one of the most important companies of all time. Do most important companies of all time. Do most important companies of all time. Do you know somebody that wants to take an you know somebody that wants to take an you know somebody that wants to take an aggressive bet that wants to like aggressive bet that wants to like aggressive bet that wants to like believe in us? Like here's all the believe in us? Like here's all the believe in us? Like here's all the information. Like we're an open book. information. Like we're an open book. information. Like we're an open book. Here's the team. It's great people. Here's the team. It's great people. Here's the team. It's great people. We've been working super hard. we've We've been working super hard. we've We've been working super hard. we've done these things in record time, but we done these things in record time, but we done these things in record time, but we have these, you know, hundred things to have these, you know, hundred things to have these, you know, hundred things to go. Like, do you want to do this? And go. Like, do you want to do this? And go. Like, do you want to do this? And like the snowball starts and you get a like the snowball starts and you get a like the snowball starts and you get a million here and 2 million here, and million here and 2 million here, and million here and 2 million here, and you're like, "Okay, we're not going to you're like, "Okay, we're not going to you're like, "Okay, we're not going to run out of money this month." You get a run out of money this month." You get a run out of money this month." You get a $5 million check, $10 million check, and $5 million check, $10 million check, and $5 million check, $10 million check, and you're like, "Okay, maybe I can buy you're like, "Okay, maybe I can buy you're like, "Okay, maybe I can buy those FPGAs." And, you know, the those FPGAs." And, you know, the those FPGAs." And, you know, the snowball snowball happened where, you snowball snowball happened where, you snowball snowball happened where, you know, we were very lucky that we ended know, we were very lucky that we ended know, we were very lucky that we ended up putting it all together. I mean, we up putting it all together. I mean, we up putting it all together. I mean, we had a board meeting. I show you the had a board meeting. I show you the had a board meeting. I show you the spreadsheet and we look at it and it's spreadsheet and we look at it and it's spreadsheet and we look at it and it's like 103 million. It's like these are like 103 million. It's like these are like 103 million. It's like these are all like soft commits and we all look at all like soft commits and we all look at all like soft commits and we all look at each other and we say we're going to each other and we say we're going to each other and we say we're going to take it and um that was the series A and take it and um that was the series A and take it and um that was the series A and uh luckily I I think it it's been much uh luckily I I think it it's been much uh luckily I I think it it's been much easier since then and we've raised easier since then and we've raised easier since then and we've raised almost a half dozen rounds since then. almost a half dozen rounds since then. almost a half dozen rounds since then. Many of them from those investors just Many of them from those investors just Many of them from those investors just doubling and tripling down. That's doubling and tripling down. That's doubling and tripling down. That's allowed us to get to market so quickly. allowed us to get to market so quickly. allowed us to get to market so quickly. Like this rack would not be possible had Like this rack would not be possible had Like this rack would not be possible had we not have been so aggressive. we not have been so aggressive. we not have been so aggressive. >> I also think like suppliers too I think >> I also think like suppliers too I think >> I also think like suppliers too I think deserve a little bit of a commenation deserve a little bit of a commenation deserve a little bit of a commenation here. Yeah. that TSMC was willing to here. Yeah. that TSMC was willing to here. Yeah. that TSMC was willing to work with us back before we'd raised any work with us back before we'd raised any work with us back before we'd raised any of that hundred million dollars back of that hundred million dollars back of that hundred million dollars back when it was still really really scary when it was still really really scary when it was still really really scary actually went ahead and let us get uh actually went ahead and let us get uh actually went ahead and let us get uh some of their emulators on extremely some of their emulators on extremely some of their emulators on extremely favorable terms where we pay over many favorable terms where we pay over many favorable terms where we pay over many years. years. years. >> Yeah. basically a big loan and like it >> Yeah. basically a big loan and like it >> Yeah. basically a big loan and like it takes a lot of belief from your partners takes a lot of belief from your partners takes a lot of belief from your partners to go do this but at the end of that you to go do this but at the end of that you to go do this but at the end of that you come out with this very strong team and come out with this very strong team and come out with this very strong team and all the folks who back you are not in it all the folks who back you are not in it all the folks who back you are not in it just out of pure financial incentive just out of pure financial incentive just out of pure financial incentive they believe they believe they believe >> why did TSMC believe do you think >> why did TSMC believe do you think >> why did TSMC believe do you think >> this is a great story even before you >> this is a great story even before you >> this is a great story even before you joined in there was a conference as joined in there was a conference as joined in there was a conference as event and I was one of the only young event and I was one of the only young event and I was one of the only young CEOs of semiconductors I think it's kind CEOs of semiconductors I think it's kind CEOs of semiconductors I think it's kind of a novelty asked me to come in there of a novelty asked me to come in there of a novelty asked me to come in there and speak. I get to the semi event and I and speak. I get to the semi event and I and speak. I get to the semi event and I am the only uh speaker there under 40 am the only uh speaker there under 40 am the only uh speaker there under 40 and only person there under 30. I was at and only person there under 30. I was at and only person there under 30. I was at the time 22 22. So uh I go up and I the time 22 22. So uh I go up and I the time 22 22. So uh I go up and I speak. There was a speaker dinner speak. There was a speaker dinner speak. There was a speaker dinner afterwards and by pure luck happen next afterwards and by pure luck happen next afterwards and by pure luck happen next to this very senior TSMC VP. It's a very to this very senior TSMC VP. It's a very to this very senior TSMC VP. It's a very nice dinner. There's like the former CEO nice dinner. There's like the former CEO nice dinner. There's like the former CEO of ARM there. It's very bougie. of ARM there. It's very bougie. of ARM there. It's very bougie. Everyone's in a suit and I'm there with Everyone's in a suit and I'm there with Everyone's in a suit and I'm there with this VP and it turns out we both studied this VP and it turns out we both studied this VP and it turns out we both studied math in college and uh we go we get a math in college and uh we go we get a math in college and uh we go we get a little piece of paper and we begin little piece of paper and we begin little piece of paper and we begin talking in great detail about how do talking in great detail about how do talking in great detail about how do modern AI models work at the actual per modern AI models work at the actual per modern AI models work at the actual per tensor by tensor level and the guy just tensor by tensor level and the guy just tensor by tensor level and the guy just gets it and we begin talking about hey gets it and we begin talking about hey gets it and we begin talking about hey how do you run this very effectively? uh how do you run this very effectively? uh how do you run this very effectively? uh why is Louisville such a critical why is Louisville such a critical why is Louisville such a critical technology to make this work? And the technology to make this work? And the technology to make this work? And the following day, I get an email from TSC following day, I get an email from TSC following day, I get an email from TSC saying, "Gavin, I want to work with saying, "Gavin, I want to work with saying, "Gavin, I want to work with Etch, Etch, Etch, >> find a way to make it happen." >> find a way to make it happen." >> find a way to make it happen." >> Crazy. >> Crazy. >> Crazy. >> They've been a great partner ever since. >> They've been a great partner ever since. >> They've been a great partner ever since. And And And >> it's amazing to think about um some of >> it's amazing to think about um some of >> it's amazing to think about um some of the tropes and obviously like should the tropes and obviously like should the tropes and obviously like should break the fourth wall here. Like I'm a break the fourth wall here. Like I'm a break the fourth wall here. Like I'm a big Etch investor. I've been involved big Etch investor. I've been involved big Etch investor. I've been involved for a long time. Um I think the absolute for a long time. Um I think the absolute for a long time. Um I think the absolute world of you guys. So like I'm world of you guys. So like I'm world of you guys. So like I'm incredibly biased, you know, in this incredibly biased, you know, in this incredibly biased, you know, in this conversation. I'm trying to ask, you conversation. I'm trying to ask, you conversation. I'm trying to ask, you know, questions that are that are know, questions that are that are know, questions that are that are broader and interesting and could be broader and interesting and could be broader and interesting and could be objections to what you're doing and we objections to what you're doing and we objections to what you're doing and we we'll keep doing that. Um, but it's so we'll keep doing that. Um, but it's so we'll keep doing that. Um, but it's so interesting to me that like when you interesting to me that like when you interesting to me that like when you read about investing, read about investing, read about investing, everyone cites this idea of like everyone cites this idea of like everyone cites this idea of like contrarian and right as the quadrant contrarian and right as the quadrant contrarian and right as the quadrant that makes all the money and it sounds that makes all the money and it sounds that makes all the money and it sounds really nice, but contrarian means like really nice, but contrarian means like really nice, but contrarian means like everyone else thinks you're stupid. And everyone else thinks you're stupid. And everyone else thinks you're stupid. And so when you go and you you get immediate so when you go and you you get immediate so when you go and you you get immediate nos from literally everybody, uh it is a nos from literally everybody, uh it is a nos from literally everybody, uh it is a fascinating quadrant to exist in before fascinating quadrant to exist in before fascinating quadrant to exist in before you become consensus. you become consensus. you become consensus. >> I mean, what was it like for you? I'm >> I mean, what was it like for you? I'm >> I mean, what was it like for you? I'm super curious. super curious. super curious. >> Well, it's interesting. At the time, uh >> Well, it's interesting. At the time, uh >> Well, it's interesting. At the time, uh it was the it was the largest by a lot it was the it was the largest by a lot it was the it was the largest by a lot first check that I had written. Sice to first check that I had written. Sice to first check that I had written. Sice to say, I'm not a a math expert or a say, I'm not a a math expert or a say, I'm not a a math expert or a semiconductor expert or an AI expert semiconductor expert or an AI expert semiconductor expert or an AI expert really at the time. Um and so it was really at the time. Um and so it was really at the time. Um and so it was much more of a believed in the concept much more of a believed in the concept much more of a believed in the concept of this market potentially being huge. of this market potentially being huge. of this market potentially being huge. you having made very very clear bets on you having made very very clear bets on you having made very very clear bets on how the future was going to look, having how the future was going to look, having how the future was going to look, having positioned the company in order to positioned the company in order to positioned the company in order to attack those things in a in a in a attack those things in a in a in a attack those things in a in a in a hardcore way. Um, and then just the two hardcore way. Um, and then just the two hardcore way. Um, and then just the two of you and and what I felt about you was of you and and what I felt about you was of you and and what I felt about you was the majority of the reason why we made the majority of the reason why we made the majority of the reason why we made the bet when we did in 2023 or whatever the bet when we did in 2023 or whatever the bet when we did in 2023 or whatever it was. Um, but at the time it was it it was. Um, but at the time it was it it was. Um, but at the time it was it was the biggest. And I think the same was the biggest. And I think the same was the biggest. And I think the same thing you said about naivee applies to thing you said about naivee applies to thing you said about naivee applies to investing as does to maybe building a investing as does to maybe building a investing as does to maybe building a semiconductor startup which is like I semiconductor startup which is like I semiconductor startup which is like I didn't know what I didn't know. And when didn't know what I didn't know. And when didn't know what I didn't know. And when I called experts they were basically I called experts they were basically I called experts they were basically like this is stupid. They laid out in in like this is stupid. They laid out in in like this is stupid. They laid out in in very logical terms like why this wasn't very logical terms like why this wasn't very logical terms like why this wasn't going to work and why it was such a low going to work and why it was such a low going to work and why it was such a low probability bet. And I think one of the probability bet. And I think one of the probability bet. And I think one of the things I've learned from it is just like things I've learned from it is just like things I've learned from it is just like you kind of have to damn the base rate. you kind of have to damn the base rate. you kind of have to damn the base rate. Like if you invested on base rates you Like if you invested on base rates you Like if you invested on base rates you should do something other than what we should do something other than what we should do something other than what we and I do. There's always the index fund. and I do. There's always the index fund. and I do. There's always the index fund. >> Yeah, there's always an index fund. >> Yeah, there's always an index fund. >> Yeah, there's always an index fund. Exactly. So, it's actually never been Exactly. So, it's actually never been Exactly. So, it's actually never been scary for me. I think probably most of scary for me. I think probably most of scary for me. I think probably most of that is because I don't know. There's a that is because I don't know. There's a that is because I don't know. There's a lot I don't know. And if I knew more lot I don't know. And if I knew more lot I don't know. And if I knew more about what you guys have done and the about what you guys have done and the about what you guys have done and the difficulty, I probably wouldn't have difficulty, I probably wouldn't have difficulty, I probably wouldn't have done it. I don't know what that says done it. I don't know what that says done it. I don't know what that says about like maturing as an investor. Like about like maturing as an investor. Like about like maturing as an investor. Like maybe I don't want to know, you know, a maybe I don't want to know, you know, a maybe I don't want to know, you know, a lot more and and have some of that lot more and and have some of that lot more and and have some of that healthy naive. I don't know. healthy naive. I don't know. healthy naive. I don't know. >> Funny. I mean, I think uh a lot of the >> Funny. I mean, I think uh a lot of the >> Funny. I mean, I think uh a lot of the traditional semiconductor funds missed traditional semiconductor funds missed traditional semiconductor funds missed the entire AI chip like all the AI chip the entire AI chip like all the AI chip the entire AI chip like all the AI chip companies. companies. companies. uh and like all the coding experts uh and like all the coding experts uh and like all the coding experts missed all the coding companies and like missed all the coding companies and like missed all the coding companies and like I think it's very hard to realize that I think it's very hard to realize that I think it's very hard to realize that constraints have changed and like you constraints have changed and like you constraints have changed and like you know when you've looked at tape outs for know when you've looked at tape outs for know when you've looked at tape outs for 20 years and you've seen so many of them 20 years and you've seen so many of them 20 years and you've seen so many of them not work on the first try or the second not work on the first try or the second not work on the first try or the second try or the third try like you couldn't try or the third try like you couldn't try or the third try like you couldn't even run a workload you totally forget even run a workload you totally forget even run a workload you totally forget that EDA tools are way better and that that EDA tools are way better and that that EDA tools are way better and that FPGAAS like exist today in a way that FPGAAS like exist today in a way that FPGAAS like exist today in a way that they didn't exist before uh and all the they didn't exist before uh and all the they didn't exist before uh and all the types of validation you can do today um types of validation you can do today um types of validation you can do today um just wasn't possible so I think for us just wasn't possible so I think for us just wasn't possible so I think for us uh a lot of our believers either like uh a lot of our believers either like uh a lot of our believers either like they were kind of on two sides of it. they were kind of on two sides of it. they were kind of on two sides of it. They were just believers in the market They were just believers in the market They were just believers in the market and the team or they were building chips and the team or they were building chips and the team or they were building chips today and extremely technical like the today and extremely technical like the today and extremely technical like the high frequency trading firms where they high frequency trading firms where they high frequency trading firms where they would literally audit everything from would literally audit everything from would literally audit everything from the micro architecture and the RTL to the micro architecture and the RTL to the micro architecture and the RTL to like the board designs and like the like the board designs and like the like the board designs and like the schedule and the software stack and we schedule and the software stack and we schedule and the software stack and we would sit down with like 10 of their would sit down with like 10 of their would sit down with like 10 of their people who build their own chips and people who build their own chips and people who build their own chips and they're asking us such detailed they're asking us such detailed they're asking us such detailed questions that we're wondering are they questions that we're wondering are they questions that we're wondering are they going to build the chip? It was really going to build the chip? It was really going to build the chip? It was really on either of those sides. And if you on either of those sides. And if you on either of those sides. And if you were anywhere in the middle, like you were anywhere in the middle, like you were anywhere in the middle, like you just wouldn't understand it. just wouldn't understand it. just wouldn't understand it. >> In the investing world, they often talk >> In the investing world, they often talk >> In the investing world, they often talk about variant perception, something that about variant perception, something that about variant perception, something that you see or believe that others don't, you see or believe that others don't, you see or believe that others don't, right? And and and that perception right? And and and that perception right? And and and that perception creates the opportunity. creates the opportunity. creates the opportunity. >> I think I've invested, I don't know, >> I think I've invested, I don't know, >> I think I've invested, I don't know, five or so times in that. And every time five or so times in that. And every time five or so times in that. And every time when you do it, stakes are getting when you do it, stakes are getting when you do it, stakes are getting bigger and bigger. And so it it does get bigger and bigger. And so it it does get bigger and bigger. And so it it does get a little scarier and scarier. and and a little scarier and scarier. and and a little scarier and scarier. and and because you guys have been so quiet in because you guys have been so quiet in because you guys have been so quiet in the marketplace, I think it's very easy the marketplace, I think it's very easy the marketplace, I think it's very easy to dismiss you. As the stakes get bigger to dismiss you. As the stakes get bigger to dismiss you. As the stakes get bigger and bigger, those dismissals are harder and bigger, those dismissals are harder and bigger, those dismissals are harder to hear. to hear. to hear. >> And and so I I do think like betting on >> And and so I I do think like betting on >> And and so I I do think like betting on something that you see when what you something that you see when what you something that you see when what you hear from the outside world is very hear from the outside world is very hear from the outside world is very different like that that perception gap different like that that perception gap different like that that perception gap is is oppos. is is oppos. is is oppos. >> Exactly. The last thing I would say is >> Exactly. The last thing I would say is >> Exactly. The last thing I would say is the accumulated evidence of your guys's the accumulated evidence of your guys's the accumulated evidence of your guys's and your team's ability to solve and your team's ability to solve and your team's ability to solve seemingly impossible problems uh is one seemingly impossible problems uh is one seemingly impossible problems uh is one of the most interesting like things a of the most interesting like things a of the most interesting like things a company can have. It's like a binary company can have. It's like a binary company can have. It's like a binary like companies do this or they don't. like companies do this or they don't. like companies do this or they don't. >> That's the thing is a big advantage of >> That's the thing is a big advantage of >> That's the thing is a big advantage of people who have been here for a long people who have been here for a long people who have been here for a long time. You get some new joiners who are time. You get some new joiners who are time. You get some new joiners who are scared shitless. When you see a thing scared shitless. When you see a thing scared shitless. When you see a thing like this and there are oldtimers who've like this and there are oldtimers who've like this and there are oldtimers who've been here for all of two years been here for all of two years been here for all of two years >> smoking cigars in the trenches. Another >> smoking cigars in the trenches. Another >> smoking cigars in the trenches. Another one. Another one. one. Another one. one. Another one. >> Yeah, there's definitely a find a way >> Yeah, there's definitely a find a way >> Yeah, there's definitely a find a way mentality. If you're here, you're here mentality. If you're here, you're here mentality. If you're here, you're here because you assume it's possible. So, because you assume it's possible. So, because you assume it's possible. So, like we can't be saying it's impossible. like we can't be saying it's impossible. like we can't be saying it's impossible. Everything is solvable and we're just Everything is solvable and we're just Everything is solvable and we're just going to work at it until we figure it going to work at it until we figure it going to work at it until we figure it out. Um there's a f a favorite story I out. Um there's a f a favorite story I out. Um there's a f a favorite story I have this guy who's kind of a legend in have this guy who's kind of a legend in have this guy who's kind of a legend in in silicon validation uh who who joined in silicon validation uh who who joined in silicon validation uh who who joined our team and we were doing the early our team and we were doing the early our team and we were doing the early stages of what's called wafer sort. Uh, stages of what's called wafer sort. Uh, stages of what's called wafer sort. Uh, and you know when your chips are coming and you know when your chips are coming and you know when your chips are coming out of fab, they go out on these wafers out of fab, they go out on these wafers out of fab, they go out on these wafers and you have this thing called a probe and you have this thing called a probe and you have this thing called a probe card that attaches uh to the wafer card that attaches uh to the wafer card that attaches uh to the wafer before you dice it with these probe pads before you dice it with these probe pads before you dice it with these probe pads and you send these electrical signals to and you send these electrical signals to and you send these electrical signals to basically test which chips are good and basically test which chips are good and basically test which chips are good and bad. So when I, you know, slice the bad. So when I, you know, slice the bad. So when I, you know, slice the wafer into a bunch of chips, I can wafer into a bunch of chips, I can wafer into a bunch of chips, I can package it and only package the good package it and only package the good package it and only package the good ones. We go through our first wafer and, ones. We go through our first wafer and, ones. We go through our first wafer and, you know, it's like 2 3:00 a.m. because you know, it's like 2 3:00 a.m. because you know, it's like 2 3:00 a.m. because we're doing it with like TSMC over the we're doing it with like TSMC over the we're doing it with like TSMC over the phone in Taiwan. We have the screen uh phone in Taiwan. We have the screen uh phone in Taiwan. We have the screen uh with the wafer that's all gray and each with the wafer that's all gray and each with the wafer that's all gray and each chip is gray and then as you start chip is gray and then as you start chip is gray and then as you start running the patterns the the squares are running the patterns the the squares are running the patterns the the squares are supposed to turn green or red and they supposed to turn green or red and they supposed to turn green or red and they all turn red. We're like all turn red. We're like all turn red. We're like like this is really bad. Like like this is really bad. Like like this is really bad. Like everybody's like guys take a breath. He everybody's like guys take a breath. He everybody's like guys take a breath. He leans back and he's like the puzzle leans back and he's like the puzzle leans back and he's like the puzzle begins. like that. You have to have the begins. like that. You have to have the begins. like that. You have to have the attitude of like, yes, you will go out attitude of like, yes, you will go out attitude of like, yes, you will go out and you will go stare into the abyss and and you will go stare into the abyss and and you will go stare into the abyss and you will go see scary things and we'll you will go see scary things and we'll you will go see scary things and we'll solve them. solve them. solve them. >> When did you see the first green square? >> When did you see the first green square? >> When did you see the first green square? >> Within a day of that, >> Within a day of that, >> Within a day of that, but in the moment you're like, I have but in the moment you're like, I have but in the moment you're like, I have worked for years for this. I put my life worked for years for this. I put my life worked for years for this. I put my life on the line. I've asked my family to on the line. I've asked my family to on the line. I've asked my family to like, you know, stake everything on it. like, you know, stake everything on it. like, you know, stake everything on it. And then like it's red. It's extremely And then like it's red. It's extremely And then like it's red. It's extremely scary. And um there's a certain type of scary. And um there's a certain type of scary. And um there's a certain type of person who just like is addicted to that person who just like is addicted to that person who just like is addicted to that feeling uh of just feeling the fear and feeling uh of just feeling the fear and feeling uh of just feeling the fear and solving it. And um you know we are lucky solving it. And um you know we are lucky solving it. And um you know we are lucky to have a lot of those people here. to have a lot of those people here. to have a lot of those people here. >> If you think about applying all of this >> If you think about applying all of this >> If you think about applying all of this um earned knowhow from this last several um earned knowhow from this last several um earned knowhow from this last several years and now thinking ahead to Gen 2, years and now thinking ahead to Gen 2, years and now thinking ahead to Gen 2, Gen 3 and beyond. Gen 3 and beyond. Gen 3 and beyond. >> Sure. What will you be doing most >> Sure. What will you be doing most >> Sure. What will you be doing most differently as a result of everything differently as a result of everything differently as a result of everything that you've learned? Just like from a that you've learned? Just like from a that you've learned? Just like from a conceptual standpoint, like the way that conceptual standpoint, like the way that conceptual standpoint, like the way that you will attack designing and producing you will attack designing and producing you will attack designing and producing this next one based on what you learned this next one based on what you learned this next one based on what you learned doing it the first time. doing it the first time. doing it the first time. >> It took us a while to get to the >> It took us a while to get to the >> It took us a while to get to the primitives that we think are really what primitives that we think are really what primitives that we think are really what matters for scaling inference. We tried matters for scaling inference. We tried matters for scaling inference. We tried a bunch of things early on from a bunch of things early on from a bunch of things early on from compilers that would turn different compilers that would turn different compilers that would turn different models into FPGAAS to burning weights in models into FPGAAS to burning weights in models into FPGAAS to burning weights in silicon to splitting your HBM to KV silicon to splitting your HBM to KV silicon to splitting your HBM to KV cache and weights and all of these cache and weights and all of these cache and weights and all of these different things. Um, and there was a different things. Um, and there was a different things. Um, and there was a lot of cycles of learning till we got to lot of cycles of learning till we got to lot of cycles of learning till we got to the point that we realized that like the point that we realized that like the point that we realized that like fundamentally if you want to run a fundamentally if you want to run a fundamentally if you want to run a majority of the tokens in the world. You majority of the tokens in the world. You majority of the tokens in the world. You need to do three things. You need to need to do three things. You need to need to do three things. You need to build a chip with the most flops in a build a chip with the most flops in a build a chip with the most flops in a given power budget. You need to build a given power budget. You need to build a given power budget. You need to build a chip that has the lowest latency between chip that has the lowest latency between chip that has the lowest latency between other chips. So the biggest scale up other chips. So the biggest scale up other chips. So the biggest scale up domain possible. And you need to produce domain possible. And you need to produce domain possible. And you need to produce as much of it as possible. as much of it as possible. as much of it as possible. >> And I think probably in the first half >> And I think probably in the first half >> And I think probably in the first half of our journey so far, we learned the of our journey so far, we learned the of our journey so far, we learned the first two and that informed the design a first two and that informed the design a first two and that informed the design a lot. And that informs a lot about the lot. And that informs a lot about the lot. And that informs a lot about the bets we're making in the future with the bets we're making in the future with the bets we're making in the future with the low voltage inference and the cluster low voltage inference and the cluster low voltage inference and the cluster scale memory. But the production part I scale memory. But the production part I scale memory. But the production part I think in the past year has become think in the past year has become think in the past year has become extremely obvious. How much people want extremely obvious. How much people want extremely obvious. How much people want to deploy this stuff if you can have it to deploy this stuff if you can have it to deploy this stuff if you can have it available today. You know the best available today. You know the best available today. You know the best ability is availability. You know if I ability is availability. You know if I ability is availability. You know if I have a thousand chips today someone's have a thousand chips today someone's have a thousand chips today someone's going to use them. And you know we need going to use them. And you know we need going to use them. And you know we need to build a chip that's not just like way to build a chip that's not just like way to build a chip that's not just like way better than what's been built before but better than what's been built before but better than what's been built before but it needs to be available at many it needs to be available at many it needs to be available at many gigawatt scale. We need to be able to be gigawatt scale. We need to be able to be gigawatt scale. We need to be able to be building a product that is producable at building a product that is producable at building a product that is producable at gigawatts per month from the limit. As gigawatts per month from the limit. As gigawatts per month from the limit. As we think about that, um, a lot of the we think about that, um, a lot of the we think about that, um, a lot of the design decisions we're making with our design decisions we're making with our design decisions we're making with our nextgen, which which you've seen nextgen, which which you've seen nextgen, which which you've seen already, uh, is just about simplicity, already, uh, is just about simplicity, already, uh, is just about simplicity, removing tons of parts, trying to removing tons of parts, trying to removing tons of parts, trying to assemble and disassemble a thing again assemble and disassemble a thing again assemble and disassemble a thing again and again, and learning how to make it and again, and learning how to make it and again, and learning how to make it as quick and the cycle times as possible as quick and the cycle times as possible as quick and the cycle times as possible in production, making sure it's going to in production, making sure it's going to in production, making sure it's going to be reliable, making sure it's going to be reliable, making sure it's going to be reliable, making sure it's going to be serviceable, uh, and making sure it's be serviceable, uh, and making sure it's be serviceable, uh, and making sure it's going to be producable at gigantic going to be producable at gigantic going to be producable at gigantic scales. What about other um problems in scales. What about other um problems in scales. What about other um problems in the ecosystem that are outside of your the ecosystem that are outside of your the ecosystem that are outside of your control such as capacity at the leading control such as capacity at the leading control such as capacity at the leading nanometer at TSMC or availability of nanometer at TSMC or availability of nanometer at TSMC or availability of HBM4 uh memory or you know some of these HBM4 uh memory or you know some of these HBM4 uh memory or you know some of these other things where like everyone is other things where like everyone is other things where like everyone is fighting for uh scarce unit of capacity fighting for uh scarce unit of capacity fighting for uh scarce unit of capacity or whatever. How do you face up against or whatever. How do you face up against or whatever. How do you face up against those realities when you're trying to those realities when you're trying to those realities when you're trying to produce as much as humanly possible? The produce as much as humanly possible? The produce as much as humanly possible? The people deploying the most comput in the people deploying the most comput in the people deploying the most comput in the world do think about supply a bit zero world do think about supply a bit zero world do think about supply a bit zero sum which is there's only so many wafers sum which is there's only so many wafers sum which is there's only so many wafers being produced on a given nanometer node being produced on a given nanometer node being produced on a given nanometer node on a given fab right and there's only so on a given fab right and there's only so on a given fab right and there's only so much memory being produced and that's much memory being produced and that's much memory being produced and that's why actually for our firstg product we why actually for our firstg product we why actually for our firstg product we built it on a different supply chain built it on a different supply chain built it on a different supply chain than the Rubins so you know we're on than the Rubins so you know we're on than the Rubins so you know we're on 4nometer Ruben's on 3 nanometer you know 4nometer Ruben's on 3 nanometer you know 4nometer Ruben's on 3 nanometer you know we're on a different HPM than Reuben's we're on a different HPM than Reuben's we're on a different HPM than Reuben's and so forth uh so it actually is not a and so forth uh so it actually is not a and so forth uh so it actually is not a zero sum thing it's a positive something zero sum thing it's a positive something zero sum thing it's a positive something where more is more so often when we're where more is more so often when we're where more is more so often when we're talking to people deploying at scale. talking to people deploying at scale. talking to people deploying at scale. It's not a decision between a gigawatt It's not a decision between a gigawatt It's not a decision between a gigawatt of a GPU and a gigawatt of us. It's 2 of a GPU and a gigawatt of us. It's 2 of a GPU and a gigawatt of us. It's 2 gawatt. Uh and I think as much as gawatt. Uh and I think as much as gawatt. Uh and I think as much as possible thinking about supply chain possible thinking about supply chain possible thinking about supply chain early in the design decisions because if early in the design decisions because if early in the design decisions because if you have the most performant product and you have the most performant product and you have the most performant product and you can't produce it, you know, then you can't produce it, you know, then you can't produce it, you know, then you're just a podcast. That's other big you're just a podcast. That's other big you're just a podcast. That's other big thing about uh vertical integration too thing about uh vertical integration too thing about uh vertical integration too is or certain things like for the chips is or certain things like for the chips is or certain things like for the chips and for the memory, you have to go ahead and for the memory, you have to go ahead and for the memory, you have to go ahead and uh partner for most of the other and uh partner for most of the other and uh partner for most of the other stuff. Those are also very highly in stuff. Those are also very highly in stuff. Those are also very highly in demand components. And the more that you demand components. And the more that you demand components. And the more that you build yourself, uh, the more stuff you build yourself, uh, the more stuff you build yourself, uh, the more stuff you can go do on top of what the world can can go do on top of what the world can can go do on top of what the world can currently build. It is not, oh, you're currently build. It is not, oh, you're currently build. It is not, oh, you're taking availability from somebody else. taking availability from somebody else. taking availability from somebody else. You're adding way, way more. And, uh, I You're adding way, way more. And, uh, I You're adding way, way more. And, uh, I think that's how you win. think that's how you win. think that's how you win. >> One of the things I realized we haven't >> One of the things I realized we haven't >> One of the things I realized we haven't talked at all about is the models talked at all about is the models talked at all about is the models themselves, which is kind of crazy the themselves, which is kind of crazy the themselves, which is kind of crazy the things behind all of this demand. things behind all of this demand. things behind all of this demand. Anything interesting that you would say Anything interesting that you would say Anything interesting that you would say about the way that you see models about the way that you see models about the way that you see models progressing based on what we've seen so progressing based on what we've seen so progressing based on what we've seen so far? I guess I'm more interested in how far? I guess I'm more interested in how far? I guess I'm more interested in how you as thinkers about hardware think you as thinkers about hardware think you as thinkers about hardware think hardware might impact where the models hardware might impact where the models hardware might impact where the models themselves go in the future. One of the themselves go in the future. One of the themselves go in the future. One of the most important ideas that we believe in most important ideas that we believe in most important ideas that we believe in is that machines don't think like people is that machines don't think like people is that machines don't think like people think. You look at airplanes for example think. You look at airplanes for example think. You look at airplanes for example that airplanes don't fly like birds fly. that airplanes don't fly like birds fly. that airplanes don't fly like birds fly. That when you think about how mechanical That when you think about how mechanical That when you think about how mechanical devices have to work, it's often very devices have to work, it's often very devices have to work, it's often very different. And in much the same way for different. And in much the same way for different. And in much the same way for people storing data and loading memory people storing data and loading memory people storing data and loading memory is very cheap for neurons and doing math is very cheap for neurons and doing math is very cheap for neurons and doing math is relatively expensive and it is the is relatively expensive and it is the is relatively expensive and it is the exact opposite for uh chips. Generally exact opposite for uh chips. Generally exact opposite for uh chips. Generally loading data is very expensive and doing loading data is very expensive and doing loading data is very expensive and doing math is very cheap and as time goes on I math is very cheap and as time goes on I math is very cheap and as time goes on I think you will end up finding that math think you will end up finding that math think you will end up finding that math gets cheaper at a rate that is faster gets cheaper at a rate that is faster gets cheaper at a rate that is faster than uh memory gets cheaper due to this than uh memory gets cheaper due to this than uh memory gets cheaper due to this fundamental limit on any kind of DRM fundamental limit on any kind of DRM fundamental limit on any kind of DRM device. You should go ahead and think device. You should go ahead and think device. You should go ahead and think about how can I make my model use a huge about how can I make my model use a huge about how can I make my model use a huge huge amount of compute. What if I had huge amount of compute. What if I had huge amount of compute. What if I had for example many copies running at the for example many copies running at the for example many copies running at the same time? What if I activated a huge same time? What if I activated a huge same time? What if I activated a huge number of experts? What if I had number of experts? What if I had number of experts? What if I had gigantic experts that I can go ahead and gigantic experts that I can go ahead and gigantic experts that I can go ahead and run on multiple server acts at the same run on multiple server acts at the same run on multiple server acts at the same time? That is how I think you'll build time? That is how I think you'll build time? That is how I think you'll build models that are the next generation of models that are the next generation of models that are the next generation of intelligence and context too. There's intelligence and context too. There's intelligence and context too. There's been a lot of work on hey very efficient been a lot of work on hey very efficient been a lot of work on hey very efficient inference. uh what if I don't load the inference. uh what if I don't load the inference. uh what if I don't load the full context into memory and most of the full context into memory and most of the full context into memory and most of the time I think that makes a lot of sense time I think that makes a lot of sense time I think that makes a lot of sense you want to go build a super you want to go build a super you want to go build a super intelligence why can't it go look at a intelligence why can't it go look at a intelligence why can't it go look at a billion tokens of context why can't it billion tokens of context why can't it billion tokens of context why can't it spend a huge amount of compute to go spend a huge amount of compute to go spend a huge amount of compute to go ahead and read all that in a super fast ahead and read all that in a super fast ahead and read all that in a super fast I would love to go be able to talk to a I would love to go be able to talk to a I would love to go be able to talk to a machine that was able to go attend to machine that was able to go attend to machine that was able to go attend to well every book ever written and it's well every book ever written and it's well every book ever written and it's short-term memory and uh I think you're short-term memory and uh I think you're short-term memory and uh I think you're going to get to a point where you can't going to get to a point where you can't going to get to a point where you can't >> a theme in models right now is this >> a theme in models right now is this >> a theme in models right now is this focus on something called dynamism which focus on something called dynamism which focus on something called dynamism which is this ability to control the level of is this ability to control the level of is this ability to control the level of computation and memory spent at a per computation and memory spent at a per computation and memory spent at a per token or per user level when doing token or per user level when doing token or per user level when doing attention uh as well as this ability to attention uh as well as this ability to attention uh as well as this ability to dynamically in your chip on the fly send dynamically in your chip on the fly send dynamically in your chip on the fly send data to other chips for different models data to other chips for different models data to other chips for different models doing certain types of operations and doing certain types of operations and doing certain types of operations and you know the reason is fundamentally as you know the reason is fundamentally as you know the reason is fundamentally as we are scaling context length as we're we are scaling context length as we're we are scaling context length as we're scaling model size uh as we're scaling scaling model size uh as we're scaling scaling model size uh as we're scaling the amount of computation per user we're the amount of computation per user we're the amount of computation per user we're looking for ways to be more efficient so looking for ways to be more efficient so looking for ways to be more efficient so you know the first thing is Like Gavin you know the first thing is Like Gavin you know the first thing is Like Gavin says, you know, mixture of experts, you says, you know, mixture of experts, you says, you know, mixture of experts, you know, architectures where, you know, know, architectures where, you know, know, architectures where, you know, maybe we don't need every parameter maybe we don't need every parameter maybe we don't need every parameter being used for every token. Uh, but being used for every token. Uh, but being used for every token. Uh, but maybe there are things where even at a maybe there are things where even at a maybe there are things where even at a token level, we can say, well, this token level, we can say, well, this token level, we can say, well, this token needs this context from this other token needs this context from this other token needs this context from this other token. They can share that memory, so we token. They can share that memory, so we token. They can share that memory, so we don't have to have overhead of of using don't have to have overhead of of using don't have to have overhead of of using the memory as much. Uh, maybe this token the memory as much. Uh, maybe this token the memory as much. Uh, maybe this token is really important, so we should spend is really important, so we should spend is really important, so we should spend more compute. We should have longer more compute. We should have longer more compute. We should have longer context on that token. So hardware that context on that token. So hardware that context on that token. So hardware that really accelerates these types of very really accelerates these types of very really accelerates these types of very dynamic computations extremely dynamic computations extremely dynamic computations extremely important. And you you can imagine important. And you you can imagine important. And you you can imagine current hardware that was designed current hardware that was designed current hardware that was designed before those types of architectures uh before those types of architectures uh before those types of architectures uh have lots of overheads in doing them. So have lots of overheads in doing them. So have lots of overheads in doing them. So you basically end up in these really bad you basically end up in these really bad you basically end up in these really bad worlds where you have inefficient worlds where you have inefficient worlds where you have inefficient hardware at doing this dynamism. Uh so hardware at doing this dynamism. Uh so hardware at doing this dynamism. Uh so therefore you can't run it very well or therefore you can't run it very well or therefore you can't run it very well or you have these very blocky architectures you have these very blocky architectures you have these very blocky architectures that are kind of applying blunt force to that are kind of applying blunt force to that are kind of applying blunt force to many different tokens that all need more many different tokens that all need more many different tokens that all need more or less uh computation. or less uh computation. or less uh computation. >> I have two questions about the future. >> I have two questions about the future. >> I have two questions about the future. Uh we've talked a lot about what you Uh we've talked a lot about what you Uh we've talked a lot about what you built so far and how you built it. The built so far and how you built it. The built so far and how you built it. The first is about the new ways that people first is about the new ways that people first is about the new ways that people might start using these systems. Uh the might start using these systems. Uh the might start using these systems. Uh the tech the raw technology, log of run tech the raw technology, log of run tech the raw technology, log of run times, you know, things of this nature. times, you know, things of this nature. times, you know, things of this nature. When inference gets gets much cheaper, When inference gets gets much cheaper, When inference gets gets much cheaper, faster, more accessible, there's more faster, more accessible, there's more faster, more accessible, there's more total supply and it's better. What are total supply and it's better. What are total supply and it's better. What are the things that you think people will the things that you think people will the things that you think people will use that capacity to do that are the use that capacity to do that are the use that capacity to do that are the most interesting, exciting to both of most interesting, exciting to both of most interesting, exciting to both of you? Yeah, there was a viral tweet by you? Yeah, there was a viral tweet by you? Yeah, there was a viral tweet by Noan Brown where he said that as these Noan Brown where he said that as these Noan Brown where he said that as these models are having longer and longer time models are having longer and longer time models are having longer and longer time horizons, they can do tasks that take horizons, they can do tasks that take horizons, they can do tasks that take say six months and there's often not say six months and there's often not say six months and there's often not enough time to go and evaluate them uh enough time to go and evaluate them uh enough time to go and evaluate them uh for such a long period of time because for such a long period of time because for such a long period of time because by that point you'll have a new model by that point you'll have a new model by that point you'll have a new model app you'll want to go evaluate instead. app you'll want to go evaluate instead. app you'll want to go evaluate instead. And with tech like we built our cluster And with tech like we built our cluster And with tech like we built our cluster scale memory, you can go ahead and run scale memory, you can go ahead and run scale memory, you can go ahead and run that six-month job much faster. But that six-month job much faster. But that six-month job much faster. But there's a second piece of this too there's a second piece of this too there's a second piece of this too talking to him about it. He's now an talking to him about it. He's now an talking to him about it. He's now an angel as well where it's not just the angel as well where it's not just the angel as well where it's not just the time, it's also the number of people or time, it's also the number of people or time, it's also the number of people or agents who are working on this. If agents who are working on this. If agents who are working on this. If you're trying to go and evaluate can a you're trying to go and evaluate can a you're trying to go and evaluate can a human build a rocket, you will find that human build a rocket, you will find that human build a rocket, you will find that the answer is no. No one person can go the answer is no. No one person can go the answer is no. No one person can go build a rocket. Instead, you have to go build a rocket. Instead, you have to go build a rocket. Instead, you have to go put a team together. And I believe the put a team together. And I believe the put a team together. And I believe the same thing will be true of agents too. same thing will be true of agents too. same thing will be true of agents too. If you want to go ask can agent go out If you want to go ask can agent go out If you want to go ask can agent go out and build some crazy futuristic piece of and build some crazy futuristic piece of and build some crazy futuristic piece of software, you'll probably need a very software, you'll probably need a very software, you'll probably need a very large team. uh maybe that's 10, maybe large team. uh maybe that's 10, maybe large team. uh maybe that's 10, maybe that's a million. You have to go and that's a million. You have to go and that's a million. You have to go and have this enormous amount of both uh have this enormous amount of both uh have this enormous amount of both uh cluster scale memory to go ahead and cluster scale memory to go ahead and cluster scale memory to go ahead and have that very short time per token and have that very short time per token and have that very short time per token and a huge amount of flops to be able to go a huge amount of flops to be able to go a huge amount of flops to be able to go run that whole fleet. I'm going to be a run that whole fleet. I'm going to be a run that whole fleet. I'm going to be a little futuristic. I firmly believe we little futuristic. I firmly believe we little futuristic. I firmly believe we are on a global march of inference are on a global march of inference are on a global march of inference becoming majority of global GDP. Uh and becoming majority of global GDP. Uh and becoming majority of global GDP. Uh and it may may take more than 10 years, but it may may take more than 10 years, but it may may take more than 10 years, but it's it's going to happen. Uh, and right it's it's going to happen. Uh, and right it's it's going to happen. Uh, and right now we measure productivity as a society now we measure productivity as a society now we measure productivity as a society as GDP per capita, but really it's going as GDP per capita, but really it's going as GDP per capita, but really it's going to look much more like agents per to look much more like agents per to look much more like agents per megawatt or maybe agents per gigawatt by megawatt or maybe agents per gigawatt by megawatt or maybe agents per gigawatt by then. And while we're being futuristic, then. And while we're being futuristic, then. And while we're being futuristic, I think this is the second to last year I think this is the second to last year I think this is the second to last year where a majority of the workforce is where a majority of the workforce is where a majority of the workforce is going to be human. I think in 2027, going to be human. I think in 2027, going to be human. I think in 2027, you're going to see there's going to be you're going to see there's going to be you're going to see there's going to be more agents doing knowledge work than more agents doing knowledge work than more agents doing knowledge work than humans. And it's going to be extremely humans. And it's going to be extremely humans. And it's going to be extremely interesting to see what happens. You interesting to see what happens. You interesting to see what happens. You could imagine a world where um for could imagine a world where um for could imagine a world where um for countries a majority of their energy countries a majority of their energy countries a majority of their energy ends up going into data centers doing ends up going into data centers doing ends up going into data centers doing inference and the energy efficiency of inference and the energy efficiency of inference and the energy efficiency of those data centers basically governs how those data centers basically governs how those data centers basically governs how many agents and therefore you know how many agents and therefore you know how many agents and therefore you know how big their workforce is. So you're going big their workforce is. So you're going big their workforce is. So you're going to see like as Gavin is saying you know to see like as Gavin is saying you know to see like as Gavin is saying you know uh right now we have you know uh one uh right now we have you know uh one uh right now we have you know uh one agent or a team of five to 10 agents agent or a team of five to 10 agents agent or a team of five to 10 agents working on group projects for a couple working on group projects for a couple working on group projects for a couple days. So you can do pretty cool stuff days. So you can do pretty cool stuff days. So you can do pretty cool stuff because they're smart but it's not going because they're smart but it's not going because they're smart but it's not going to be civilization scale. What happens to be civilization scale. What happens to be civilization scale. What happens when you have countries that can have when you have countries that can have when you have countries that can have literally a billion concurrent agents, literally a billion concurrent agents, literally a billion concurrent agents, like a billion people in the workforce like a billion people in the workforce like a billion people in the workforce working 247 concurrently on the same working 247 concurrently on the same working 247 concurrently on the same stuff. I mean, it's it's just kind of stuff. I mean, it's it's just kind of stuff. I mean, it's it's just kind of unfathomable what's going to happen. Uh, unfathomable what's going to happen. Uh, unfathomable what's going to happen. Uh, and it's going to be the biggest and it's going to be the biggest and it's going to be the biggest proliferation of technology humanity's proliferation of technology humanity's proliferation of technology humanity's ever seen. ever seen. ever seen. >> I think as well, like when you have >> I think as well, like when you have >> I think as well, like when you have these huge huge amounts of demand, you these huge huge amounts of demand, you these huge huge amounts of demand, you get this idea of economies of scale get this idea of economies of scale get this idea of economies of scale again. again. again. >> Yep. >> Yep. >> Yep. >> Or if you think about people, I have a >> Or if you think about people, I have a >> Or if you think about people, I have a brain. I'm not using the whole thing all brain. I'm not using the whole thing all brain. I'm not using the whole thing all at the same time. That only a part of is at the same time. That only a part of is at the same time. That only a part of is going to be active. And this is the way going to be active. And this is the way going to be active. And this is the way healthy brains work. And for MOE models, healthy brains work. And for MOE models, healthy brains work. And for MOE models, it works much the same way. On MOE it works much the same way. On MOE it works much the same way. On MOE model, only a small fraction of the model, only a small fraction of the model, only a small fraction of the parameters is being used for any given parameters is being used for any given parameters is being used for any given token at any given moment. But if you token at any given moment. But if you token at any given moment. But if you have a large number of users on a piece have a large number of users on a piece have a large number of users on a piece of hardware, you can go kind of take of hardware, you can go kind of take of hardware, you can go kind of take that brain, cut it up into many that brain, cut it up into many that brain, cut it up into many different experts on many different different experts on many different different experts on many different servers, and run a huge amount of volume servers, and run a huge amount of volume servers, and run a huge amount of volume through it. So you'll have a bunch of through it. So you'll have a bunch of through it. So you'll have a bunch of different pieces of traffic. You'll have different pieces of traffic. You'll have different pieces of traffic. You'll have many of them using each part of the many of them using each part of the many of them using each part of the brain at any given point in time. And brain at any given point in time. And brain at any given point in time. And you'll also make the cost per thought, you'll also make the cost per thought, you'll also make the cost per thought, cost per token way, way lower. cost per token way, way lower. cost per token way, way lower. >> So I think you're going to end up with >> So I think you're going to end up with >> So I think you're going to end up with these giant scale distributed brains these giant scale distributed brains these giant scale distributed brains which is going to look like the real the which is going to look like the real the which is going to look like the real the form factor of this is a big data center form factor of this is a big data center form factor of this is a big data center with a bunch of chips, a huge amount of with a bunch of chips, a huge amount of with a bunch of chips, a huge amount of flops, and a huge amount of scale up flops, and a huge amount of scale up flops, and a huge amount of scale up interconnect. interconnect. interconnect. >> You think we'll see a trillion dollar >> You think we'll see a trillion dollar >> You think we'll see a trillion dollar individual data center? individual data center? individual data center? >> Absolutely. It is a matter of time. It's >> Absolutely. It is a matter of time. It's >> Absolutely. It is a matter of time. It's like asking what you see a billion like asking what you see a billion like asking what you see a billion dollar fab or a10 billion dollar fab or dollar fab or a10 billion dollar fab or dollar fab or a10 billion dollar fab or hundred billion dollar fab. It is hundred billion dollar fab. It is hundred billion dollar fab. It is inevitable that the economies of scale inevitable that the economies of scale inevitable that the economies of scale don't stop at oh $40 billion is the don't stop at oh $40 billion is the don't stop at oh $40 billion is the magic number for fabs. No, the cost magic number for fabs. No, the cost magic number for fabs. No, the cost wafer keeps going down as you keep wafer keeps going down as you keep wafer keeps going down as you keep spending more money. And the same thing spending more money. And the same thing spending more money. And the same thing will be true of say plants that go out will be true of say plants that go out will be true of say plants that go out and make steel or plants that go out and and make steel or plants that go out and and make steel or plants that go out and make tokens. a very smart alien lands on make tokens. a very smart alien lands on make tokens. a very smart alien lands on on Earth and wants to know from each of on Earth and wants to know from each of on Earth and wants to know from each of you how you would frame up this you how you would frame up this you how you would frame up this opportunity that that you guys are opportunity that that you guys are opportunity that that you guys are tackling. What what do you say to them? tackling. What what do you say to them? tackling. What what do you say to them? >> Frame it as thinking is really valuable >> Frame it as thinking is really valuable >> Frame it as thinking is really valuable that every company in the world runs on that every company in the world runs on that every company in the world runs on thinking and we are entering this really thinking and we are entering this really thinking and we are entering this really unique moment in time where you have unique moment in time where you have unique moment in time where you have machines that can go think almost as machines that can go think almost as machines that can go think almost as good and as soon as good and as soon good and as soon as good and as soon good and as soon as good and as soon better than the best humans can. better than the best humans can. better than the best humans can. Building these machines is going to go Building these machines is going to go Building these machines is going to go be a huge opportunity. But more be a huge opportunity. But more be a huge opportunity. But more important than that, the way in which important than that, the way in which important than that, the way in which you go ahead and run this kind of you go ahead and run this kind of you go ahead and run this kind of thinking is going to be very very thinking is going to be very very thinking is going to be very very different as demand goes higher and different as demand goes higher and different as demand goes higher and higher and higher and higher. There's a higher and higher and higher. There's a higher and higher and higher. There's a unique moment right now to go build a unique moment right now to go build a unique moment right now to go build a new set of solutions, a new road map for new set of solutions, a new road map for new set of solutions, a new road map for how do you run the future quadrillion how do you run the future quadrillion how do you run the future quadrillion parameter models for a billion people parameter models for a billion people parameter models for a billion people all at the same time on a gigantic all at the same time on a gigantic all at the same time on a gigantic scaleup cluster. We are in a new era of scaleup cluster. We are in a new era of scaleup cluster. We are in a new era of intelligence where um the cost of intelligence where um the cost of intelligence where um the cost of producing intelligence is dramatically producing intelligence is dramatically producing intelligence is dramatically so much cheaper than the value of the so much cheaper than the value of the so much cheaper than the value of the intelligence that we are in a many year intelligence that we are in a many year intelligence that we are in a many year probably many decade supply shortage of probably many decade supply shortage of probably many decade supply shortage of these tokens. And basically any chip or these tokens. And basically any chip or these tokens. And basically any chip or any system that can produce tokens is any system that can produce tokens is any system that can produce tokens is likely to be extremely valuable. And you likely to be extremely valuable. And you likely to be extremely valuable. And you should find some part of the supply should find some part of the supply should find some part of the supply chain of the token. Uh it can be chain of the token. Uh it can be chain of the token. Uh it can be everything from model training down to everything from model training down to everything from model training down to what we're doing in the silicon and what we're doing in the silicon and what we're doing in the silicon and otherwise uh to spend time on and push otherwise uh to spend time on and push otherwise uh to spend time on and push the frontier. Uh and that the companies the frontier. Uh and that the companies the frontier. Uh and that the companies that are the largest are frankly going that are the largest are frankly going that are the largest are frankly going to be the companies that produce most of to be the companies that produce most of to be the companies that produce most of the global supply of tokens and own a the global supply of tokens and own a the global supply of tokens and own a majority of the supply chain of that majority of the supply chain of that majority of the supply chain of that token. And importantly, it's people who token. And importantly, it's people who token. And importantly, it's people who build systems that as they get more and build systems that as they get more and build systems that as they get more and more chips put together cheaper. That more chips put together cheaper. That more chips put together cheaper. That the way you want this to scale is not the way you want this to scale is not the way you want this to scale is not that, oh, if I want to go serve 10 times that, oh, if I want to go serve 10 times that, oh, if I want to go serve 10 times more tokens, I buy 10 times more more tokens, I buy 10 times more more tokens, I buy 10 times more servers. It must be some solution where servers. It must be some solution where servers. It must be some solution where I want to go serve 10 times more tokens, I want to go serve 10 times more tokens, I want to go serve 10 times more tokens, then I get some economies of scale then I get some economies of scale then I get some economies of scale benefit with my cluster scale memory benefit with my cluster scale memory benefit with my cluster scale memory tech that allows me to then not charge tech that allows me to then not charge tech that allows me to then not charge as much as 10 times more for those as much as 10 times more for those as much as 10 times more for those stacked tokens. What a ridiculously stacked tokens. What a ridiculously stacked tokens. What a ridiculously exciting future that you guys are exciting future that you guys are exciting future that you guys are building to enable. When I did this with building to enable. When I did this with building to enable. When I did this with Gavin last time, I asked him my Gavin last time, I asked him my Gavin last time, I asked him my traditional closing question. So, this traditional closing question. So, this traditional closing question. So, this time I'll ask you, what is the kindest time I'll ask you, what is the kindest time I'll ask you, what is the kindest thing that anyone's ever done for you? thing that anyone's ever done for you? thing that anyone's ever done for you? >> During my cancer treatment, there was a >> During my cancer treatment, there was a >> During my cancer treatment, there was a big decision I had to make. Um, the big decision I had to make. Um, the big decision I had to make. Um, the doctors came to me and said, "It's time doctors came to me and said, "It's time doctors came to me and said, "It's time for you to decide. Do you want to get for you to decide. Do you want to get for you to decide. Do you want to get surgery or do you want to get surgery or do you want to get surgery or do you want to get radiation?" Here's the trade-off. If you radiation?" Here's the trade-off. If you radiation?" Here's the trade-off. If you get surgery, you're more likely to live, get surgery, you're more likely to live, get surgery, you're more likely to live, but you have to assume you'll never be but you have to assume you'll never be but you have to assume you'll never be able to walk again. if you get able to walk again. if you get able to walk again. if you get radiation, you'll be able to walk again, radiation, you'll be able to walk again, radiation, you'll be able to walk again, but it's not the same probability that but it's not the same probability that but it's not the same probability that you'll live. You may die. What do you you'll live. You may die. What do you you'll live. You may die. What do you want to do? And I was 16. Uh, and my want to do? And I was 16. Uh, and my want to do? And I was 16. Uh, and my parents, I said, you have to make this parents, I said, you have to make this parents, I said, you have to make this decision for yourself. Um, and I thought decision for yourself. Um, and I thought decision for yourself. Um, and I thought a about it for a long time and decided a about it for a long time and decided a about it for a long time and decided I'm going to do the surgery. I get the I'm going to do the surgery. I get the I'm going to do the surgery. I get the surgery. One of the things you do when surgery. One of the things you do when surgery. One of the things you do when you get a tumor resection is they do you get a tumor resection is they do you get a tumor resection is they do something called a necrosis analysis something called a necrosis analysis something called a necrosis analysis where they look at all the different where they look at all the different where they look at all the different cells and say is a cell dead or alive cells and say is a cell dead or alive cells and say is a cell dead or alive because if you have a bunch of cancer because if you have a bunch of cancer because if you have a bunch of cancer cells that alive you have a problem. And cells that alive you have a problem. And cells that alive you have a problem. And they looked at it and they said you know they looked at it and they said you know they looked at it and they said you know you usually want 98 99% necrosis uh for you usually want 98 99% necrosis uh for you usually want 98 99% necrosis uh for us to say you're in the clear. You're us to say you're in the clear. You're us to say you're in the clear. You're below that. Um you should go get below that. Um you should go get below that. Um you should go get radiation. radiation. radiation. Um, and there was only a few machines in Um, and there was only a few machines in Um, and there was only a few machines in in in the world that actually uh could in in the world that actually uh could in in the world that actually uh could do the type of radiation I needed. Uh, do the type of radiation I needed. Uh, do the type of radiation I needed. Uh, one of them was in Boston. I was in a one of them was in Boston. I was in a one of them was in Boston. I was in a wheelchair uh and I needed to move to wheelchair uh and I needed to move to wheelchair uh and I needed to move to Boston for multiple months. And both of Boston for multiple months. And both of Boston for multiple months. And both of my parents uh decided to move out and my parents uh decided to move out and my parents uh decided to move out and drop everything they were doing and and drop everything they were doing and and drop everything they were doing and and live with me. Um, and I'm eternally live with me. Um, and I'm eternally live with me. Um, and I'm eternally grateful. grateful. grateful. beautiful. Thanks, guys. Amazing beautiful. Thanks, guys. Amazing beautiful. Thanks, guys. Amazing conversation. Your finance team isn't losing money on Your finance team isn't losing money on big mistakes. It's leaking through a big mistakes. It's leaking through a big mistakes. It's leaking through a thousand tiny decisions nobody's thousand tiny decisions nobody's thousand tiny decisions nobody's watching. Ramp puts guardrails on watching. Ramp puts guardrails on watching. Ramp puts guardrails on spending before it happens. Real-time spending before it happens. Real-time spending before it happens. Real-time limits, automatic rules, zero limits, automatic rules, zero limits, automatic rules, zero firefighting. Try it at ramp.com/invest. firefighting. Try it at ramp.com/invest. firefighting. Try it at ramp.com/invest. As your business grows, Vant scales with As your business grows, Vant scales with As your business grows, Vant scales with you, automating compliance and giving you, automating compliance and giving you, automating compliance and giving you a single source of truth for you a single source of truth for you a single source of truth for security and risk. Learn more at security and risk. Learn more at security and risk. Learn more at vant.com/invest. Ridgeline is redefining asset management Ridgeline is redefining asset management technology as a true partner, not just a technology as a true partner, not just a technology as a true partner, not just a software vendor. They've helped firms 5x software vendor. They've helped firms 5x software vendor. They've helped firms 5x in scale, enabling faster growth, in scale, enabling faster growth, in scale, enabling faster growth, smarter operations, and a competitive smarter operations, and a competitive smarter operations, and a competitive edge. Visit ridgelineapps.com to see edge. Visit ridgelineapps.com to see edge. Visit ridgelineapps.com to see what they can unlock for your firm. what they can unlock for your firm. what they can unlock for your firm. Every investment firm is unique, and Every investment firm is unique, and Every investment firm is unique, and generic AI doesn't understand your generic AI doesn't understand your generic AI doesn't understand your process. Rogo does. It's an AI platform process. Rogo does. It's an AI platform process. Rogo does. It's an AI platform built specifically for Wall Street, built specifically for Wall Street, built specifically for Wall Street, connected to your data, understanding connected to your data, understanding connected to your data, understanding your process, and producing real your process, and producing real your process, and producing real outputs. Check them out at outputs. Check them out at outputs. Check them out at rogo.ai/invest. The best AI and software companies from The best AI and software companies from OpenAI to cursor to Perplexity use work OpenAI to cursor to Perplexity use work OpenAI to cursor to Perplexity use work OS to become enterprise ready overnight, OS to become enterprise ready overnight, OS to become enterprise ready overnight, not in months. Visit works.com to skip not in months. Visit works.com to skip not in months. Visit works.com to skip the unglamorous infrastructure work and the unglamorous infrastructure work and the unglamorous infrastructure work and focus on your product.