Why Top Founders Are Racing Into AI Infrastructure

Key Takeaway

Treat AI agents like new junior employees: start today by giving one agent a bounded, reversible task—such as sorting a small batch of emails or researching options—define the desired outcome, require approval before it takes consequential actions, and review its work. This captures the upside of delegation while managing the risks the speakers highlight: wasted tokens and money, hallucinations, forgotten context, and security issues.

Episode Overview

A16Z partners discuss why AI’s accelerating demand is turning infrastructure—from chips and memory to power, cooling, construction, and grid connections—into the central constraint on progress. They argue that AI infrastructure is a long-term opportunity for systems-minded founders because existing hardware and data centers were not designed for AI workloads. The conversation also explores AI agents as a new kind of worker and the operational discipline required to deploy them safely.

Full Transcript

Transcript of Why Top Founders Are Racing Into AI Infrastructure from A16Z. Auto-generated from episode audio; may contain minor errors.

We have a completely new technology, the most important technology in history, and it requires a completely new infrastructure. Usually, when we talk about infrastructure, we mean servers, data storage, and networks. Here, everything goes all the way to the copper mines. Lots of lights, but this thing will. In the past, when you built something, it was an engineering challenge . And here it seems that it is truly a resource constraint. So, whether it's tokens or not, we're putting a lot of money into systems that deliver results, and now we're running into the limited ability of the system to actually absorb the resources that are being poured into them.

The memory manufacturer stated that they would need 3 years of production capacity to meet the demand they have today . If this fund achieves what we hope for, what will the world look like in 5-10 years? America would win the infrastructure game , and that would be great. Ben, Martin, Raghu, congratulations. Thank you. Okay, thank you. I would like to start with a quote from Mark to introduce this new fund. This is the biggest technological revolution in my lifetime. This is clearly bigger than the internet.

This could be compared to a microprocessor, a steam engine, electricity, or perhaps a wheel. " Guys, Machine Age Foundation , please introduce it. Ben , start with us. Well, what actually happened is that we have a whole new technology. The most important technology of all . And every time you have a radically new way of using all these things that we love, infrastructure, the same thing happens . You need a whole new infrastructure. And it's never been more impactful than it is in this case. So we need not only new chips, new system software, we need new power methods, we need to replace copper.

I mean, it's literally everything. This is a very exciting time. So, you know, especially in the hardware aspects of this new era, we needed a new approach. Yeah, I agree. Usually, at least in computing, when we talk about the infrastructure world, we mean servers, storage, and networks. It goes all the way down to the copper mines . That's how big it's going to be. This First of all. And second of all , I think over the last 3 years we've seen a steady increase in the capabilities of models.

Where the model is no longer the bottleneck. And indeed, with AI, these models are getting better faster, faster, faster. Now the bottleneck is everything that I call south of the model. And that's why we need to work on that . Okay. You know, the only thing I'll add briefly is that we tend to follow the founders. We've seen over the last few years that the number of very strong teams working on complex hardware problems has increased. I don't know the exact numbers, but I was trying to estimate them over the weekend.

I think we used to get maybe 5 % of the deals from the best founders in the hardware space, and now it's over 20% or 30%. So the founder community, which is usually smarter than the VC community , has identified this as a very active area for innovation and is responding to that. I think 5%—that’s still pretty generous. Yes, it was very low . It was very low. Yes, 3%, exactly. Explain some of the macro conditions that led to this transition, this shift in the number of founders who are pursuing these ideas.

What do they see as making that possible? Well, obviously the demand for AI is practically infinite, and as a result, the whole supply chain is under pressure. I mean everything, including the materials used to make things, like memory. That’s very interesting, but there ’s something unique about AI because the demand is infinite and the growth is infinite. You usually worry about the margins of companies, how efficient it is. You usually worry about growth. Like, can I just get people to buy these things? You do n’t have to worry about that here .

The question is, can I do it profitably? And a lot of the efficiency is really just a physical limitation of the hardware. So even the AI ​​wave business model puts a lot of strain on existing systems because they weren't built for AI. They weren't built for these workloads. And I think there's this global observation that we really need to change the core components to get that efficiency, to drive growth and value for the business. Yeah. And how do we know that demand is really outpacing supply and not just another hype?

Well, yeah, there's a lot of things you can say. Um, first of all, some of the smallest demand judges are cutting back on huge orders. I mean, if you look at the hyperscalers, actually. Yeah. Their capex is skyrocketing . Next year, it's expected to hit a trillion dollars combined for all the big hyperscalers. This year, it's about $700 billion or so, right? And if you think about the position of hyperscalers in the industry, they see demand everywhere, right? They see, of course, that advanced labs need compute.

They see companies that are working with data for AI. They see the enterprise. They see the U.S. market and the international market. So if anyone has any vision, it's them. And they're ramping up their capital spending like never before, right? So that's a clear, very clear signal. And secondly, if you look at the companies that we see every day, they're just bursting, all the application companies. The growth is crazy. Advanced labs, the growth is crazy . It's documented. So I would say that on the demand side, the signals have never been so clear that this is not just hype.

And on top of that, it's all just And prices are going up. We've never seen chip prices go up like this. GPU prices go down, and then Yeah, yeah, yeah. They always go down. And if you look at the price curve, it goes like this, then back up and then like this. Yeah. And we know that today, only 5 to 10 percent of the market is captured. So the supply, if you look at it as a whole, is almost completely scheduled for 2028. It's so serious that we've even seen multi-day auctions for a few thousand GPUs.

Um, the other side of that is of course the demand, and as we said, it's the fastest growth in the history of the industry. And also the unit of work that AI can do, the value of that unit of work is constantly increasing. But hidden from view is the number of tokens consumed increasing by orders of magnitude, right? If for a chat it's one token, then for an agent it's thousands of tokens, right? So you have an expansion of demand on both sides. One, the unit of work is getting more saturated, yes.

In tokens, and two, the number of people who will benefit from it . It's not just developers, but all knowledge workers. And then everyone else. That's what we're seeing. You said that key components are sold out by 2027 , maybe by 2028. By 2028, yes. What does it mean when the entire industry sold out for such a long time? I don't know if this has ever happened before. Do you remember ? I mean, remember the early days of the Internet, when we were doing massive expansion, and most of what was being laid was speculative and not used.

Remember "dark fiber"? And here, practically every GPU that was built was already sold in advance . So, we were in a slightly different situation. So there was... there was a bandwidth shortage around 98-99. But there was no real demand for it, because there were n't that many people on the Internet at the time. It was a two-pronged problem: companies were all rushing in and theoretically needed more traffic, but there were no users to consume it. And to consume a lot of traffic, you had to do things that required high speed, like video, which at the time was not very viable for a number of reasons unrelated to the power of the data centers.

So it was a bit similar, but not quite. Now we're working on full, people are reselling GPUs for four times more than they bought them for, and that's all . And we're running out of power and cooling capacity, and that's what we're doing. And it's also very difficult to build because of the incredible political hurdles. It's really unprecedented in my career that we're going to have something like this. Yeah. Let me give you a quick example. I was talking to a CFO of a large public company that has historically been very resistant to going to the cloud.

They had a lot of servers, and when they did an inventory, they found that the cost of memory in their servers had gone up to the point where they could finance a full migration to the cloud. So I think we're in a very unusual situation right now. Well, that's true. We're running out of a lot of things. Power, cooling, memory , GPUs—you name it, we're running out of it. You know, the flagship conference for the industry is going on at Stanford right now called Hot Chips .

And a leading memory vendor said that to please today's demand, they're going to need 3 years of manufacturing capacity. And that's just for today. That's not even future demand. So, as for the simultaneous shortage of everything, is it because people just underestimated how good and useful these models would be and just couldn't foresee the demand? Well, I don't even think that's the point . I mean, this all came out of nowhere, right? We've only been doing this for 4 years. So even if we had perfect foresight, once everything was up and running, I don't think we could have ramped up the capacity .

We couldn't have built that capacity. It's impossible. We're talking about chip development cycles that are typically 3 to 4 years. We're talking about site preparation and data center construction, which takes, you know, 4 to 5 years. And also the wiring, the setup, and the power supply. So you either have to build your own power, or usually both. You have to provide your own to power itself and have power sources, and that's not easy. So you have an industry that's used to growing at 20% to 30%.

That's a great growth rate, right? And it's connecting to the AI software industry, which is growing three times as fast, just as a base. So you see that gap, right? So it's just getting bigger. And why didn't this fund exist, you know, 5 or 7 years ago, or why wasn't this a great category to invest in in the same way, Brad? Well, I would say that we're probably -- I'd like to think we're on time, but you know, it would have been appropriate to have created it at least a couple of years ago.

I mean, you could point to every era and every independent company that came into being at that time. Right? Obviously, in the transition from mainframes to client-server systems, we saw a bunch of companies emerge. The transition to the Internet -- that's when Cisco and Juniper came into being. Even in mega data centers, which, by the way, was largely driven by the verticalization of existing cloud providers. You saw Arista come in. So there was an opportunity to invest, you know, in silicon and hardware, but it was relatively small because the changes were small: one company making chips, one making switches.

And here, it's the other way around. So I agree with Ben, we could probably start a little earlier. But the amount of change is so big now that it's just an obvious step. The other thing is, the demand for intelligence is so vertical, and there's no end in sight. I mean, every company that's implemented it is ramping up usage very quickly. And most companies haven't fully implemented it yet . And consumers are just getting started. So the demand for tokens is probably going to grow by almost 1,000% a year, and you can't ramp up supply that fast , we're not...

The amount of work that we're going to have to do to get to the level of infrastructure development that fast, is just huge. So I think there's a lot of investment opportunity ahead. And by the way , another thing: all the hardware architectures were built for a completely different era of computing. So we do n't just need more power— there's a lot of opportunity to build completely different types of infrastructure. Yeah. They're all reaching the physical limits of what they were designed for. Right? Like what Ben was talking about: copper, etc.

If you go through each of those categories, you can see: here's the limit for this type of technology. So now you need technical breakthroughs to get to the next level. Yeah. I want to go into the demand side for a moment. From chatbots to reasoning, agents to multi-agent systems, each step has multiplied the number of tokens for a single task by orders of magnitude. Yeah, nobody likes using AI more than AI itself. Why does this continue instead of stabilizing? You see, it's going to go on forever, just going on...

Well, there's a couple of aspects to it. The first is because right now, if you look at the way we do scaling, we do it through a lot of inference. That is, through a lot of tokens, right? If you think about what RL is, it 's, you know, very much inference. If you think about the chain of thought, it's also very much inference. And if you think about long-lived agents, of course, it's also inference. So that's actually one of the approaches that we're taking to scale.

I think if you step back and look at the macro trend, in the past, when you built something, it was an engineering problem. You just brought in a bunch of engineers, but it doesn't scale, and there are natural laws of engineering physics. That's where the "Mythical Man-on-the-Moon" came from. And it seems like it's really a resource constraint. So regardless of the tokens, we're putting a lot of money into systems , and they're delivering results. And now we're limited by the ability of those systems to actually match the resources that we're putting into them .

So I think that tokens—that's probably where we are now on the scaling curve , but we don't have the natural regulator that we used to have . I think we should expect that to continue, and we have to create a supply to support that. So, the way of thinking is that any problem can be solved with enough infrastructure. And money, basically. And money. So until we run out of problems, we're not going to run out of demand. And that's exactly the challenge. The answer to AI getting better and better is to use more AI, right?

The bottom line is—it's a basic building block that we keep using over and over again. That's why these tokens keep multiplying. Yeah, it's even a self-catalytic effect. So even the idea of ​​using AI to build more AI, like building a GPU core, of course, is just using more AI as part of the process. So another way of thinking about it is: before, there was money coming in , there was an engineering challenge, we knew it would take 2 years, usually failed, that was a natural limiter, and then you got a product out there.

There's nothing between the money you put in and how the hardware creates intelligence. So now we're limited only by our ability to create the offering. It's a very, very different dynamic. More GPUs and everything . Yeah, that's right. So it's just like, because you have the money, the GPUs and the data, you know, in the foreseeable future you're going to be able to scale these things. And that's exciting because, you know, for the last decade, it seemed like so many people were like , there's too much money being put into startups.

We're funding these startups too much. There's too much money in VC. Is that the same thing or something else, because there used to be a certain skepticism that the more money you put into an industry, the bigger the results are going to be. And now, you know, we're saying that's not quite true, to some extent the market is as big as we all collectively create it. Yeah, Look, the one thing we all knew in the startup world was that if I have a two-year head start on you and you try to catch up with me by hiring a thousand engineers, you're going to destroy your company.

That never works. That's the mythical man- month. Nine women can't have a baby in a month. That's it . That never works. Okay, now it works. But it's not about hiring a thousand engineers. It's about taking three billion dollars and launching a great cluster. And suddenly, you know, Grok or whatever can come out of nowhere, and suddenly it becomes a reality, or Kimmy or whatever. It's just the benefits, you can throw money at a problem, and you can throw money at almost any problem, and it works.

And so it's radically different from anything we've ever experienced. By the way, we're all psychologically conditioned to that. The ChatGPT app has a billion active users per week. There are about 30 million developers, who use a fairly significant portion of the computing power. How do we now and in the future think about the computing power requirements in terms of what people actually do with it? That's the progression, is n't it? ChatGPT was an accidental alternative to programming for professionals, right? Now, using programming, I've built amazing tools for knowledge workers.

So that's the next frontier. And now there are over a billion knowledge workers in the world , right? So the demand for those tools is going to be a long list. And by the way, the work they do involves automation and so on. And then you move into the back office, where all the agents are. So gradually each of those things opens up, I would say, orders of magnitude more demand. And we're just at the beginning of that journey. Well, and now you have GrokBot, which, you know, what happened with coding is happening to some extent with all of the computing through GrokBot.

So we're in a completely different wave of demand, and certainly there's more to come. So it really seems limitless at this point. And we haven't even gotten to the embodied AI or robots that will be another source of demand. Mark Cuban is an expert, but from what I understand, GrokBot uses a computer in the same way that a human sits at it and types. I literally used it on the weekend to update my credit card information on a bunch of services that I was too lazy to get to and cancel a bunch of subscriptions.

I mean, it's not coding or anything like that. It's real computer use. Suddenly you're creating what seems like a knowledgeable staff, except they're all sitting inside a computer doing work. I really think Marc Andreessen is right. It's like a steam engine or electricity, that's why. We've introduced this new thing that can be directed to work. And there are some very obvious applications right now. But it's probably 30 to 40 years of solutions ahead. computational tasks, anything that has a clear reward signal. And we're just getting started.

We have language and code. That's it . And we're just getting started with the use of computing. But what else are we looking at? We're looking at science, materials, biology. Of course, creativity is a huge use case. So listen, we're at the very beginning of a very long journey. And we've removed this key bottleneck, traditional software development. Now, of course, the bottlenecks will shift, and there will be more complexity in other areas. But I think we're at the very beginning of a long journey of solving computational problems.

So let's expect this need for computation to persist for decades. By the way, since we mentioned it, Marty, tell us about GrokBot, because we were discussing at the field meeting what exactly struck you about it . Obviously, we're involved in every possible way that you can be involved in. But what, you know, what exactly did you find so interesting about it? So, I think we as an industry have gone through a few e It's a bit of an awareness of how AI is entering our lives, right?

At first we thought: okay, you add AI to the product, it's just a search bar, you talk to it, and it responds. So, this is a traditional way of working. And then Open Clock appeared at the beginning of the year. And with Open Clock, I said, okay, maybe it's just like Google, but better. Perhaps this is not a complete implementation of the idea. How about this being a separate tool that becomes your extension, knows your passwords, and does the work for you, right? It's kind of like your digital extension, but more human-like.

And I think the Grok bot did the right thing: no, why ca n't it just be an employee? So now you have an entity that doesn't need special access to your keys or anything like that. He has his own computer, his own browser, and because these are the smartest models in the world, he can do everything an employee does. And it's interesting, because now, if I want to do something, the first thing I think about is whether Grok can do it for me. And often the answer is "yes" , even if you didn't expect it.

Obvious things include managing your calendar or booking appointments . But there are also non-obvious tasks. For example, I can ask him to look through my mail and sort it. I don't teach him how to do it, but he knows that he should consult with me before performing the sorting. These systems are quite complex, so you can give them high- level tasks and they will do a complex job. Ben, I know you think a lot about this and how it works in an organization. You, of course, think a lot about culture.

What are your thoughts on this ? Well, I think if you look at us, it's like the emergence of a new type of employee. And there will be a lot of them, and we have to, well, you know, like with ours. We spent many, many, many years learning how to work with regular human workers, and now we have a different type of worker. And, you know, you still have to learn how to deal with them. So, they can spend a bunch of tokens and money without doing anything useful.

They may forget things. They can invent things that didn't exist. They, you know, can behave well, they can behave badly, they can create security issues. So there are all these aspects, but at the same time they can be extremely productive. So I think we're just learning how to integrate them so they work seamlessly with the people they interact with. I don't want to stand here and say that I've cracked the code, that we have a great cycle, that the entire firm is fully automated, and I'm going to get rid of all the people little by little because I can.

No, we are not at that stage at all. We're more concerned with how to make all our people superhuman without destroying everything around them, because bots could get out of control. Yes. It's interesting because we tried a few different ways of how best to implement agents into the system. In the end, Martin just decided to treat them like people and get the job done. This is what we do, and it has proven to be the most viable way to launch this system in an organization. I want to go back to the supply side and dive deeper into the bottlenecks.

You know, we talked about how data centers, chip architecture, system software, the buildings themselves—none of this was designed with AI in mind. What would it look like if they were designed with AI in mind? What is the mental model for thinking about what this could mean? So, so I'll say Let's start with your statement: "Hey, the original model of infrastructure and entity models has to change." You can go through the categories and see where the problems arise, right? And then you start to eliminate bottlenecks in each of these aspects.

Eventually, you have to come to a point where you consider what exactly the output mechanism does, right? It takes up a lot of memory . It generates new tokens. It loads the computer. So you can just think about how to optimize all of this? What should memory be like? What should a computer be like? How should they interact with each other? How much energy does each of them need? And if they need so much energy, how do you cool them, right? And then, how do you combine these things together?

This is a task that the entire industry is currently working on, along with many founders. They break the problem down into its fundamental components and ask, "What exactly is the nature of the computations being performed?" How can matrix multiplication provide us with that? How to optimize your computer for such a scenario? And they all gradually need memory to generate tokens. What's the best way to hierarchically organize this memory, huh? What about energy consumption? I mean, then you have to connect all of this. What are the ways to connect on a single chip, between chips, and between data centers?

How much energy does each of these data transfers consume? So, we need to gradually dismantle everything and rebuild it based on these basic building blocks. And that's exactly what we see in where we're observing them now . Let me offer an interesting mental model for understanding how the landscape is changing. Today, building a cutting-edge model costs, say, $3 to $5 billion , right? And that's just the cost of her education. So the withdrawal should cover at least this amount, of course. To make all of this viable.

Let's say twice as much . So, let's assume that now the withdrawal should bring in $10 billion. If you can improve efficiency by 20%, that's $2 billion. And you can easily build an ASIC for $2 billion, right? We have reached an interesting point in the industry where creating a separate ASIC for each model becomes justified due to the amount of capital investment in that model. And then, unlike traditional software, which has a lot of states and is very dynamic. These models are fixed. The model weights are fixed.

So we don't know if the world will move to specialized ASICs for each model, but it gives a great concept of how to evolve the architecture to be much more specialized for these huge capital investments that we're making. It seems like in the history of the industry, we've never created a digital product that had nearly $5 billion invested directly. So I think this will put the highest demands on hardware that we've ever seen. With this in mind, rack power requirements increase from approximately 5–10 kilowatts to 100–250 kilowatts.

The computational density increases by about 70 times. Cooling changes from air to liquid as a mandatory requirement. What investment opportunities arise as a result? Well, first of all, when you get to that level of power per rack, AC is no longer suitable. This is a rather strange thing. So now you're switching to DC, which , by the way, also needs its own cooling. And this, by the way, is incredibly dangerous. Which is quite ironic, because Edison promoted direct current while arguing how dangerous alternating current was.

And demonstrating it. For example, electrocuting animals and the like. Yes. Um, horse, yes. He was right, but about his own type of energy, which is extremely powerful , and that's good news. So starting with energy, it's going to be very, very different. I think, in terms of cooling, we are moving from air to liquid. I think we already use liquid cooling for any modern data center. This has already become the norm. But given the political situation, liquid cooling is no longer enough. It must be environmentally safe.

And, you know, direct current is no longer enough. We need energy that benefits society, not takes it away. There are data centers that did not behave in the best way. A small percentage, probably 10% actually. They waste a lot of water. Of course, not as much as pistachios or almonds, which are written about on the Internet, but they could be much more effective. And there are , you know, some kind of energy parasites who give nothing in return. I think all this will come to an end.

This must end, because we have already passed the point of no return on this issue. This requires a level of engineering that many have not yet invested in. So that's ahead. And also, if the racks are so dense, there are other nuances, such as the floor structure that has to withstand that weight. This, you know, is a very serious problem. And I think that a lot of everything is needed . Therefore, the equipment will operate very loudly. You will have to build data centers with thicker walls, otherwise you will disturb the peace in the area, and this is unacceptable.

I don't think any state would allow this. So many of the ways people designed buildings themselves are now completely obsolete. Once we reach Feynman's level, a much smaller percentage of current data centers will be able to operate. In fact, everyone is talking about memory prices, but one of the areas where prices are rising the fastest is concrete and cement. It's another thing when data centers supply 800 volts to a rack: firstly, it's very dangerous, and secondly, we lack electricians who have experience working with 800 volts inside a data center.

Because it's high voltage. Only 2% of electricians in the US are certified to work with DC. This gives you some idea of ​​the scale. Meta now has a whole staff training program, and it's great. It's like an employment center where people are trained for this job for free. But it's pretty funny. AI is taking all the jobs. AI will create many new jobs for electricians. Yes, we are also doing something in this area. Big companies with huge data centers are all actively experimenting with robots, right?

To perform work on assembling or installing servers in data centers, etc. So you'll see how this will grow as AI develops. By the way, to be clear about the fund we're raising, our focus is on computer science infrastructure. So, everything that the model works on is computer science, right? So keep in mind: chips, networks, interconnects, storage, and probably even electricity itself. Yes. And tell us more about the field of robotics: what exactly will we be doing, compared, perhaps, to microdynamics, or how do you think about it?

Yes, yes, definitely. So, again, we believe that any platform that AI runs on... One of the great things about AI is that it allows computers to interact with the physical world, right? He can see, hear, talk, right? And that means new platforms, right? The simplest thing people say is "peripheral device," but that doesn't really mean anything, right? That is, it could be a mobile device, a CDN, a laptop, but it could also be an embodied device that moves around. And again, as infrastructure-focused investors, we don't work with highly regulated or highly specialized industries, but we are interested in any computing platform that advances AI further.

Yes. Returning to data centers: by 2028, new centers will need about 44 gigawatts of additional capacity, compared to the expected 25 gigawatts that the grid will add. Wait. Wait. We use this word — gigawatt . No, no, it's like... Well, what about 100 gigawatts? Martin, what is a gigawatt? I mean, how big is it? It's like several football fields. That is, it is huge. Is this for 50,000 people? What do you mean, what does it feed on? Well, how... The equivalent is about 50,000 houses? 50,000 homes .

50,000 homes? I grew up in Flagstaff, Arizona. It is a city with a population of 40,000 to 60,000 people, depending on the university. We consume less than a gigawatt of electricity. So that's... So you can basically power and air-condition an entire city with one gigawatt. Yes, that's just ridiculous. around them. No, but by the way, everyone is talking about gigawatts. There are very few gigawatt data centers that actually work. We still have a lot ahead of us . To understand why utilities and hyperscalers can't build faster?

Oh, there are so many things. Well, first of all, right now you need people to build them. So, there is a normal construction. But, in addition, you need permissions. You need access to electricity that you can plug into. So you're either trying to access energy, which is a huge regulatory and tendering battle. There are a very limited number of resources that can be connected to within national gas, power grids, etc. But then you also need to build your own energy sources. And you know what? We have a shortage of transformers, turbines, and everything needed for this.

So you need to get all this equipment. This is not a software problem . It's not just that a bunch of engineers can't work on weekends or anything. It doesn't work that way, but there are real bottlenecks, and these deadlines are not so easy to shorten. And look, the best minds in the world are trying to figure out how to reduce them, but it's not easy. It's not easy , and demand is not slowing down. So, we are already falling behind. Demand is growing, you know , 10 times a year, and supply simply can't grow that fast .

By the way, the situation is so bad that now new companies are looking for GPUs in Mexico, Australia, or other countries, because it's too difficult in the US. So Yes, we are creating huge jobs and long-term economic opportunities in other countries by banning data centers here. I think it would be right to set a standard by which a data center contributes to community development. Yes. So that the energy supply improves, there is no noise, no water problems, and new jobs appear. This is what the standard should be .

And everyone must adhere to this standard. And, by the way, there are data centers that are already doing this. This is not some futuristic dream or anything like that. Energy tariffs there are falling every year, and the reason is that they are self-sufficient . They provide energy to the state during the day. And at night, they take energy from the state when it doesn't need it, because the power plants are always operating at peak capacity. Since the data center has stable power day and night, and the city's consumption increases significantly during the day and drops at night, it is a symbiotic relationship.

If you look at it more broadly, why do we think…we discussed the name for a while. Why do we think " machine age" is an apt term for what we're doing here ? Well , listen. Let me, let me try to explain. First, I think Ben is absolutely right. " Artificial intelligence" was the wrong word. We shouldn't have called it that. This is machine intelligence. Ahem. Do you want to say more about this? Why is that so? Because it's not exactly what people think, right? I mean, it's an archive of human thinking, a collection of human thoughts, but today we don't know how to take an AI without knowledge, release it into the world , and have it reproduce language, right?

We did n't do that, did we ? We created something that can learn from everything we've already learned and then use it productively. And listen, AI is a general term that has been around in computer science for 70 years, it refers to many different things and, of course, has a certain baggage of associations—whether from science fiction or from the works of Nick Bostrom. So the first thing is just acknowledging that this is indeed machine intelligence. And then you want to emphasize the machine part. There's a deep irony — this is from the people who said "software is taking over the world" — that we've reached a point where you invest money but you're limited by the actual machines at the heart of everything.

So I take this as a hint that hardware is playing such a significant role in this wave, and we want to acknowledge that. Yes, I think it's the quality of the machines on which everything is based that will create the next technological breakthroughs . That, in fact, is the reason for the name. Plus, the name is cool. "Machine Age" sounds good. Futuristic. Yes. Given how much has already been spent on AI infrastructure, and how capital-intensive this business can be, are we past the point where new companies can break into this market on a serious level?

Do you know why market leaders like Nvidia, Cerebras, etc. just don't capture the lion's share of these markets? Well, they're doing pretty well. There's no doubt about it, right? But back to our discussion: you need fundamentally new innovations to sustain growth or the pace of improvement, whether it's tokens per second, per dollar, or per watt. Uh, tokens on the counter, right? Or power, take any indicator. If you want to increase these figures by 10 times, you need new innovations. And new innovations traditionally come from brilliant founders who think differently about solving a problem from first principles , right?

And that's exactly what's needed here for the next leap in innovation. I mean, that's the law of markets, right? Let's assume that the existing silicon giants have a market capitalization of many trillions of dollars, which is quite realistic . Even 5% of that is a huge private company. A huge private company, right? We're talking, you know, about annual income. You might say, well, but Nvidia could do it. They could, but why would they? If they focus on the things that make up the 90% , that also provide the same growth.

And you always ask these questions. We asked these questions in the days of cloud technology, right? For example, why would n't Amazon do this? You ask these questions in the Microsoft era, why wouldn't Microsoft do this? It's a natural law of markets: once you reach a certain scale, there are huge opportunities for innovation at the periphery. Yes, there's a funny quote from our partner Alex Rempell, when he had a startup called TrialPay. He tried to sell it or his services to Meta, then Facebook. And Dan Rose, who was the head of corporate development at the time , said, " Alex, that's great.

It looks like you can collect a lot of silver bars, but I have so many gold bars that I can't even lift them all." Yes. So the last thing I'm going to do is look at a silver bar. And I think Nvidia is in exactly that situation. 100%. Yes. We talked about the thing about modelers: if you, you know, fall into the middle ground of what OpenAI or Anthropic can do , that might be, you know, their core areas of interest—that might be the right place, but anything outside of those three to five areas , it might, you know...

As markets expand, they fragment, right? And this happens all the time . Think back to the early days of Ford, 1913, the Rouge River plant. Literally, you know, it was a place where machines were made: water, coal, and rubber trees came in, and cars came out . By the way, he bought an entire rubber plantation in the Amazon jungle. That's right. And there's a great book called " Fordlandia," because he wanted to own the whole vertical loop, so he created a city called Fordlandia in the Amazon jungle, where everything was Americanized: variety shows, ice cream, and all that stuff.

And it actually worked for a while, until he got people to come to work on time, and then they said, "[ __ ] that , go to hell." So if you look at the automotive industry now, of course there are multi-level suppliers, there are a bunch of companies, and that's always the case . So, you know, when markets expand, they fragment. There are many...and then when that growth slows down, they tend to consolidate. This consolidation can happen through acquisitions or through the emergence of new competitors, and it's, you know, the eternal cycle of private markets.

Yes, because the use cases are multiplying, and there is no chance that even the largest company will be able to cover all the big cases; there are so many of them, and they are all, as Martin said, very valuable, so they are just difficult to implement properly. Yes, you have influence over your own simple architecture, right? Um, that's not the case anymore, everything has become too complicated now. It's inevitable that you can't optimize things differently. By the way, here's a very interesting point: people often don't understand that margins once arose from the way software was created as a standard , right?

It wasn't really a technological problem. When the business was starting out, you usually had pretty good margins because that's how software works . Of course, when you supplied it, but even as a service. And with AI, that's not necessarily the case. We may actually be entering an era where hardware optimization is absolutely essential to business success, something we haven't seen before. So, there are many possibilities here. Let's dig deeper and talk about the types of companies we will invest in. Maybe we could start by illustrating the sub-sectors or talking about a few investments we've already made.

I know there are some that have n't been announced yet, but Ravi, do you want to get started? Yes, I mean, the subsectors that we talked about are each of these categories , right? The obvious ones are computer chips. But today it is not enough to simply create a chip. You need to create a complete system, right ? So what goes into the system has potential innovations in memory, networking innovations, power supplies, and so on. So each of these categories are industries where public- level companies can emerge.

And that's all we're looking at now. And then when you put it all together, there's a software layer to automate all of this, manage fleets of devices, and so on. So this is another important area. These things constantly complement each other, but each of these categories is important. Tell us a little about how these companies differ from ordinary ones. I mean, one thing you notice about the companies that we've announced is their huge first rounds, you know, hundreds of millions. Is this a different type of founder?

Or what else is different when we think about the practice of building and investing in such businesses compared to our traditional software? Well, I think the main thing is that you hit on one of the key things: a lot of money is invested even before the product appears. And that's just the nature of this kind of business. Although this is true for large models as well, I would say that the path there is a little more familiar. There's a little more risk and a little more money involved than some of the other things we've done, but a lot of the founders of chip companies are people from the past.

You know, for example, those who know how to create memory. They are not young. Yes. Well, you know, this part is different too, but it's quite exciting, you know. Yes, one more thing about these founders— they all have to be system architects. I mean, you can't just be a researcher or a great computer scientist, right? You have to be able to design and develop a chip or a system, or whatever it is. Then you have to think about how it will all actually be produced, right?

Who will supply it ? And a whole bunch of related things that you don't usually have to think about if you're building software . So, the best founders — of course, Jensen is like Michael Jordan in this, right? They think in terms of the entire ecosystem from the very beginning, before they even start designing the chip, right ? Because of the nature of " bottlenecks" and all these factors that have to come together. So this is an important characteristic that makes a significant difference. There are two environmental factors that are important to you.

The first is that labs are so desperate for solutions that they are willing to collaborate with startups. That's why we get a lot of early signals , because labs are signing deals with companies even before the finished equipment is available . And that's a big shift from five years ago, right? Previously, you simply couldn't come in and sell your , say, imperfect prototype to Google or anyone else. So this is really a change. The second factor is that the availability of capital has increased significantly. I think there is a general consensus that now is the time to transform this area.

So there is plenty of capital available for the next rounds of investment , and of course it is worth investing in those areas where it is available. So, the atmosphere around is also just different. Patrick Collison noted a few years ago that there seem to be fewer young founders these days, like Zuckerberg, who created Facebook in college, or Gates with Microsoft. And, of course, there are people like Michael Troll. There are still young founders building iconic companies, but as you said, it seems like there are more older founders, and it's not just twenty-somethings anymore.

I'm curious if this resonates with you and why. I think Raghu is right: if you're creating something with very complex logistics, where you have to manufacture things and there are technical difficulties, some experience doesn't hurt. If you look at Elon or Travis Kalanick, their companies were initially purely software-based. Even the best of them needed experience in building companies and developing technologies to move into much more complex or, I would say, larger areas. There are a lot more moving parts, so when you're learning to build a company, it's hard even when you fully understand the product.

If you don't understand the product perfectly and are forced to learn it while building a business, it's too steep a learning curve for a novice entrepreneur. So I think what we're talking about is that, on the one hand, there's Michael—a guy who's very young and brilliant, but he built a purely software-based AI product. And on the other hand, there is Elon or Travis, who already have enough experience. I think Michael could probably do it in ten years, but it would be difficult today. It's important to remember that the entire industry and academia has paid little attention to this for the last 20 years, right?

It just wasn't as promising an opportunity. The opportunity was there, but it was never a growth area. The growth areas were software, network technologies, and so on. And I also think that we simply don't have enough people from universities or with experience in large companies who have already been through this. I mean, there really aren't many of them. You don't go on an internship to design a chip. But a lot is changing now. Listen, we're going to create a whole generation of founders who will come out of these new companies, know how it's done, and get hired much earlier.

I will say more: one of Elon's greatest legacies in this is not only the great companies he created, but also the number of entrepreneurs who have emerged from SpaceX and are changing the industrial complex; perhaps it is an even more important legacy. And I think we'll see the same thing, uh, in computer science and hardware. Actually, when we started, one of these projects was founded by two founders, but if you go into the offices, you will see experienced professionals there as well . So it's like a local combination, yeah.

Yes. Yes, it is not necessary for the founder himself to have experience, but he must be able to use this experience. Attract people with experience, yes. Yes, yes. Well, and then you have to be able to work with them , and they have to be good specialists, and all that other stuff. This is difficult. By the way, about experience: this is a large new fund that we are launching, and there are no new general partners in it. The reason we're bringing them together is because you have a lot of experience, and the rest of the group is in this area that has been somewhat hidden and inactive.

Well, that's pretty funny. I think we almost had to be warned against it, just because our views are formed...it's hard. And I think the reason we needed a reminder is because all of this is so ingrained in our careers and our work at Harbor that we're kind of used to it. So, let's say we've clearly invested in Harbor for many years, right? We at Spiffy Socks, we at Anduril. and so on. this is a very early investment. We are in Astro, Strauss, Waymo. You know, that's why even in the early stages we made a number of these investments.

But, as you know, that's because it's in our DNA . So I don't think it will necessarily increase the depth of competencies, it's just a bonus. If this fund does what we expect, how do we see the world changing or what it will look like in 5-10 years? Well, you know, hopefully America wins the infrastructure game . And we're going to have lots of super-green and efficient data centers, and lots and lots of chips, lots of memory, and lots of energy. And, you know, that would be great.

And I think it would be like... we... you know, it goes back to the fact that we really believe that America is a special place, and we're open not only to everyone here, but to anyone in the world who wants to contribute and do something more than just for themselves. This is the best place to come with nothing and do something meaningful. So, we would like to continue this. And I think that will stop if we lose technological leadership. I think it will be someone else or another country, perhaps with different values regarding this.

Thanks for the summary. A wonderful completion to your fund. Our team, Ben, Ardoin, thank you. Thank you. Ben, thank you too.

Go deeper

• All A16Z episode guides