Mike Volpi on Why AI Infrastructure Will Outlast the Hype Cycle

We live in a world where demand for AI capacity tokens is near infinite and supply is incredibly constrained. So another way to put it is we all just live in Jensen's world. You were doing back then M&A for the most valuable company on the planet. The retro on the M&A process that I think worked well for Cisco was we were not afraid to fail and we did it with high enough frequency that we got a few outliers. You bought this thing for 500 million and like it's doing a billion in earnings two years later. When you see the AI build out how do you look back on your previous experiences? If you look at the revenue ramp of Anthropic, OpenAI are definitely experiencing growth rates that were far superior to what we saw at the time. Do I think the build out that's happening now is useful and will be fully utilized if not more 100%? Will it be owned by the same people that own it today? The last great infrastructure boom built the Internet. This one is building the infrastructure for intelligence. What can the first tell us about the second? In this episode A16Z's Raghuram Raghuram and Guino Appenzeller sit down with Hanabi Capital general partner and former Cisco executive Mike Volpe. They compare today's investment in GPUs at data centers with the fiber boom of the late 1990s and why Mike thinks the infrastructure will remain valuable even if a correction reshuffles who owns it. They also discuss NVIDIA, custom AI chips, networking and what Cisco's acquisition strategy can teach founders and investors today. Mike's view? The opportunity underneath this build out could ultimately be even larger than the Internet boom. Great to have you here on the podcast Mike and it's a very very opportune time to have you because you were one of the architects of the great Internet build out when you were at Cisco. And now of course we've got the great AI build out happening. So Guino and I and the audience would love to get your thoughts on then and now from all different dimensions from the actual build out, the capitalization of it, the technology differences, etc. etc. So let's start for those that don't know or remember. Mike was a senior exec at Cisco overseeing a lot of the M&A and strategic moves that Cisco made in the late 90s. And of course that was the time the Internet got built out. So Mike was in the middle of all of it. So clearly let's start from there. So you were there when the Internet started taking off and then you guys rushed into a lot of great things and the industry rushed to build a lot of bandwidth etc. etc. which for some time was not useful. And then it turned out it was actually too little. Right. And there are some loose similarities to what's going on now. So when you see the AI build out, how do you look back on your previous experiences? Well, first of all, it's an absolute privilege to be here with two legends here, both you Raghu and the case of Guino and I, we got the chance to work together at least once. So it's fantastic to be here. And as you said, an interesting time because history doesn't repeat itself, but it definitely rhymes. Yes. So there are some similarities, maybe useful to give some context on some of the observations of the late 90s build out and what some of those things might apply in today's world versus not. Some of the kind of similar themes was that we were building out this technology called the Internet and the web and all the things that wrote on the Internet. We had expectations that this would be massively transformative to the world. And we imagined a wide range of applications that would ultimately massively benefit from this build out. Applications were different media types. Right. So first we just had pictures and text and then we had audio and we had video and then we had high quality video applications like commerce, applications like social networking. All of these things were imagined in the 90s, but didn't exist. In parallel with that, there was a very big surge in demand. At a minimum, there was a surge in demand for connectivity. And on top of that, a reasonable surge in demand for bandwidth. And in parallel with that, there was a great influx of capital of two shapes and forms, equity and debt. So if you look back at sort of what happened, first of all, in the bandwidth world, and in particular in fiber optic bandwidth, the elements, the increments of increase of the amount of bandwidth are quite different than compute. That's because we invented this technology called WDM that essentially takes a piece of fiber and said, if you used to put one gigabit on there or 10 gigabits on there, by putting in more colors of light in that same piece of fiber, you can now have 40 times or 80 times that capacity. So essentially, capacity increased quite dramatically with the advent of technological change. That's actually different because compute ultimately is bound by something that resembles Moore's law. So the pace of improvement is more moderated, hard to believe, but it is actually more moderated and compute than it was at the time. And the other component that is, I think, different is that the practical applicability of AI, I believe today is higher than the practical applicability at the time of network bandwidth. And that represents itself by the revenue ramp of the providers. So if you look at the revenue ramp of Anthropic, OpenAI, I don't know what the numbers look like within Google, even the providers of the open source models like Fireworks, Together, Modal, et cetera, et cetera, are definitely experiencing growth rates, maybe not quite at the same margins, but certainly growth rates that were far superior to what we saw at the time. Now, then what happens? At some point, there is a shock that some of these businesses that are built on the internet are not good enough. And then that ripples through the industry. And ultimately, the players that were most saddled with debt went bankrupt. Now, it's interesting to note that debt was buying assets, those fiber optic cables, and those assets actually turned out to be quite useful in the future, except they were operated by somebody else because the guys that went bankrupt didn't have them anymore. So the first thing to recognize that the underlying assets that were built at the time of this bandwidth, and I suspect this compute this time, it'll be useful. There's no question that'll be useful. Whether it will be owned by the same companies or not, is an interesting economic question. So that's a long winded answer to say, do I think the build out that's happening now is useful and will be fully utilized, if not more? 100%. Will it be owned by the same people that own it today? A lot of interesting dependencies and a lot of things that in fact are outside of the control because they're macroeconomic and geopolitical and so forth. It's really outside of the control of the builders themselves. And in particular, when I talk about the builders, some of the balance sheets are pretty healthy, right? Last count when I was talking to folks at NVIDIA, there are 200 Neo clouds. And those balance sheets are not good. Yes. So I predict that there'll be some of them better than others. But broadly, I expect that there will be an economic correction in the market, but the underlying assets will be quite useful for a long period of time. Isn't that one other big difference from how it was back then, which is that everything is now much, much larger. I mean, back then we were investing a small fraction of a percent in infrastructure for the Internet. Today, we're investing 2% or so of our GDP in AI infrastructure. Yeah. Well, it is much bigger. But it's an interesting concept, right? Because in the 100,000 feet abstraction, what is it that is being built, right? Back then, it was communication that was being built. Today, we're building intelligence. And the economic value of intelligence is probably higher than the economic value of communication. Probably. Right. So I think it's unsurprising. I also think that, believe it or not, I mean, it's been nearly 30 years. The markets are more efficient now, right? Which is why so much more risk capital is willing to make its way into AI, which even back then was not able to. So I'm actually more optimistic about this cycle. But I also believe that there will be some correction and some cleaning up of the industry structure. The role of IT has also massively increased, right? Back then, I mean, in the late 90s, PCs were somewhat common, but not ubiquitous today. So to speak, software has eaten the world. And it's sort of, for any company out there, it's a very central part of what they do. Cost of distribution of intelligence. Oh, for sure. And not just any company, but 7 billion people have a supercomputer in their pocket. So there's much more evenly spread consumption of the underlying benefit that's being produced. Yeah. Don't fully recall all the puts and takes in the 90s and early 2000s. On the financial side, here, obviously, one of the recent shocks of the system was the rising interest rates and the scale of capital causing people to say, hey, this is going to cloud out everything else, and we may not even have enough capital. During the bandwidth buildout and the internet buildout, that was not the case, correct? There was no sudden increase in interest rates, was it? No, there wasn't a massive increase in interest rate. There's a couple of big differences, right? One is that I would say on a proportional basis, I believe more of the capital came from the balance sheet of the equipment vendors than today. Today, it's a more capital markets driven buildout. So, you know, it's private debt, it's public equities, etc., etc. There is NVIDIA vendor. I don't think you're supposed to call it vendor financing, but it looks a lot like it. So there's a little bit of that, and there's some corporate financing, but it's largely equity. And it's not tied to specific equipment purchases and so forth. So I'd say the capital structures are a little cleaner now. The one thing that I think, interestingly, though, favors the 90s is a lot of the money went into the ground in the form of fiber optic. And when you put a fiber optic in the ground, it's good for a long, long time. It's like it's a usable asset for 20, 30 years. I think one of the deterministic factors is the compute you put in the ground. What is its actual life? And this is a fascinating phenomenon, right? Because the dominant part of that compute is chips. It's GPUs. 80 plus percent. Yeah, it's the big chunk of it. All right. So I remember back when people were very excited about Hopper, H100s, right? They were like, OK, well, maybe it has a five year useful life and you're renting it out at $1.80, $2 per GPU hour. And that was four years ago. So the chip should be defunct now, like near defunct. Not at all defunct. Not only that, it rents at a higher rate. It's $250 or $300. So the kind of the interesting part is we thought this asset that have a five year life, partly because of AI itself, is a useful asset for quite a bit longer. And in that context, I mean, if you're in the business of financing an asset and that asset ends up having a much more longer useful life, then you're the beneficiary of that because your business models were constructed on a short. So in some sense, I actually think this, the fact, an unintended consequence of AI being good at software is that the underlying asset life is getting longer, thereby helping it to be more financeable. And that's an interesting phenomenon that I didn't quite expect when this thing got underway. Although the investors are not, the dead investors are not broadly accepted the extended life of a GPU. But the Deerclouds clearly believe that. I'm not sure I have either. I mean, like right now, it seems what's mostly happening is that we have a massive scarcity of an asset. So even previous generations are still worth a lot. If we ever get to a point where we have an abundance of GPUs, that might completely change. You might go back to those numbers. So I think the key metric is how much token revenue can you extract per watt? Maybe, yes. So the Hopper is going to be a lot worse than Blackwell is going to be a lot worse than Verarubin, right? They're all just more efficient at getting dollars out of a watt. So in theory, at some point, when you have sufficient Verarubins, it doesn't really make sense to run H100s anymore, because like it's just the worst way of converting electricity into dollars. Correct. Right. But the problem is right now that the demand is so high that even if the watt to dollar conversion is low, you're going to still use the asset. And right now, the scarcity is so significant. And ultimately, you know, at some point in the conversation where we get there, but ultimately, the scarcity is in the manufacturing of the semiconductor itself. And that doesn't look like it's changing anytime soon. Correct. Yeah. And so then scarcity persists, then the life of the asset continues to be long. Yeah, I think it's correct. Yeah. And the other big difference, there was a period of time where there was unused bandwidth. Today, there is no such thing as an unused GPU, right? There's a bit of unused energy. And like, you could probably, you know, I don't know if we're going to talk about networking, but with better networking, you could get more flops out of a given GPU. Yeah, I was just going to say that. I'm a fused manufacturer in certain use cases and how the people have built it, but certainly, generally not. Yeah. Back when you were at Cisco, were you still there when Cisco was the most valuable company on the planet? Yes, indeed, I was there. So you were doing, back then, M&A for the most valuable company on the planet, and it worked pretty well, I want to say, looking back, right? I was very happy with it, yeah. What would be your words of advice to NVIDIA in 2026? Like, whoever runs M&A there is in exactly the same spot as you are, correct? Yeah, Vishal's in the same spot I am. Look, the retro on the M&A process that I think worked well for Francisco was that fundamentally, uh, we were okay. With making enough acquisitions that we got a statistically significant outcome on those acquisitions, meaning some worked really well, some work. So, so some didn't work at all. And, you know, if you kind of drew it out one in one, in each of those three sectors, a third of them worked really well, a third of them were whatever, and a third of them failed. Um, the benefit that we did is we were not afraid to fail. And we did it with high enough frequency that we got a few outliers, uh, Andy Bechtolsheim's company, uh, uh, Maria Mazzola's company, uh, a bunch of these things that were sort of, you know, even for us, we're venture like returns. So you bought this thing for 500 million and like it's doing a billion in earnings two years later. So, and so it ended up that during the period that I was there, uh, by I joined in 94 and I was like the one biz dev hire in 2000, uh, in 2000, roughly, uh, 50% of Cisco's revenue came from acquired sources. So that was incredibly useful. And we actually never really spent much more on a given acquisition than four or 5% of our market cap. Most of the acquisitions were actually quote cheap relative to our market cap standards. How much would that mean for NVIDIA? I don't even want to think about that. Yeah. Well, my, my rationale at the time was like, if your stock trades more than the price you paid for the acquisition in one day, then you can do the acquisitions as many as you want. Nobody, nobody will care. All of them done with equity or was all equity? Almost everything was equity. A handful of the small ones were cash, but it's Cisco. I mean, they became major product lines. So I was not like, this was a little tuck in that added a feature somewhere, but this was of a new product line that in some cases became the leading product line in the industry. Now you're a networking guy. So you'll appreciate this. The one thing that's a little different about a networking acquisition versus other ones is that networking operates on open protocols, which means that I could buy a switch company made by somebody else. And it immediately talks to my switches or routers, whatever, freely, because all the protocols are open. So, you know, at Cisco, we had a couple of proprietary protocols that we enjoyed, but by and large, all the OSPF, BGP, ISIS, Ethernet switching, 802 dot whatever, they all work. So you could immediately integrate the product into your sales team and have your sales team, like literally you just change the badge on the thing. And so that allowed us to be like super high velocity. Okay. How does that contrast to Jensen's world today? One, they've made some really clever acquisitions, you know, like Mellanox is the world's largest networking company, right? It's like genius. Now they're making other acquisitions that sometimes I scratch my head. I'm not entirely sure what hugging face is all about, for example, like maybe there's a rationale, hard to me to see it. I don't think it matters. I think it's fine for them to do some of these and take risk and some of them will work. And if they work as well as Mellanox, holy cow, and some of them are not going to work. And I think the market has granted them a moment of incredible optionality, right? But basically they get to do whatever they want, which is the optimal moment for them to take these risks. So I think they're being very clever and you can always critique any given acquisition. Trust me, I got critiqued endlessly at Cisco cause I bought Stratacom and now Cascade and I, you know, bought an optical networking company from Vinod that went nowhere and whatever. But in the end, and you're like, all right, well, I did, I did 70, I think our team did 70 acquisitions that I was there and those turned into half the company's revenue and we were doing 30 billion in revenue. So it's 15 billion in revenue. The strategy worked. And I think that, I think that the team and NVIDIA is, is, is, is doing that. I think that's interesting and wise the degree to which they are financing the industry. We'll see. Yeah. Cause that's different, right? That, that, in that sense, NVIDIA is basically, I think they're basically trying to shape what an industry looks like. Yes. And I think industries are powerful and they find their own shape. It's hard to reshape them. Let me, let me, let me defend NVIDIA a little bit, having been at Intel and, and having seen how the big hyperscalers essentially disintermediated Intel from the end customers. Right. And at the end, they could build better chips than Intel, just because Intel had no idea anymore what the customer was doing with the chips. Right. I have some sympathy for wanting to have more than three customers to sell to or four maybe. As do I. So, and you could argue, what is an efficient capital allocation strategy? Is it giving it back to investors or could it be at this time, which is probably once in a time in history, put it all to work on future innovation, right? So. Well, I think both of you make very valid points. The thing I would say, Guido, is that an industry wants to look a certain way because when it does, it creates maximum utility, maximum economic value. Why are there three hyperscalers? Cause it's a business where scale matters. Software. Yeah. Right. It's a business where like scale matters and therefore the industry wants to concentrate the, the, no matter how much you finance them, the little ones are always weaklings. So, and obviously if you're NVIDIA, you don't like that. You don't want three customers. You want 50 customers. So you fund them, but at some point they're going to die. It just, because the industry, whenever it's a scale business wants to consolidate, you know, we could fund a hundred airlines in America, but in the end you're going to have three. So I, my point is not that the, um, not that it's a bad idea. It's just that it's futile. Yeah. So, so, so noble intentions, but, uh, but, uh, now to your point, like, uh, it could be a very good return on it. I don't know that they, I don't know if they actually care about the return on capital. They, they, they, they, they might. Um, and if they get out at the right time. Yeah. It's not even about the return on capital. It's to some degree shaping industries, uh, when shaping the, deciding the shape of the industry. But on the other hand, it is, uh, enabling a lot of innovation as well, because our startups and lots of other startups do get access to that capacity now. Yeah. Right. Yeah. Yeah. Um, no, no, look, we're, uh, overall I get why they're doing it. And, and, and, you know, many of our companies are beneficiaries of it. So thank you, Jensen. So, um, I mean, at VMware, we did probably not as many as you did, but, uh, maybe 50 plus acquisitions over the full lifetime of the company. And, uh, the observation was whether it didn't matter whether you were acquiring a 20% company or a 2000% company, the amount of post acquisition work in terms of integration, culture, business models, sales team, et cetera, et cetera, was all always a lot of work. How did you guys pull it off at that time at Cisco? Um, okay. Structurally, the way we did it was that we had a, a very small, when I say small, less than 10 people, uh, deal team that was jobless to acquire, and we had a somewhat larger, uh, integration project management team that had sort of rules and guidelines about how we did integrations and, uh, uh, they, they had colleagues, those people that project manager had colleagues in the various business units and divisions and functional units within Cisco that collaborated with them. So the important thing to recognize there is that, uh, comp businesses, companies are good at processes. Uh, they're very bad at one time events. And so essentially if we were going to do a large number of acquisitions, we needed to have repeatable processes that were, uh, performed over and over again. And we had a centralized group that was the administrator of those processes. Um, more often than not, and in the beginning at Cisco, we did this a lot. There was nobody that was formally assigned to this. So it'd be like, Raghu, it's, it's, it's your job. You, you go have fun with that integration and let me know when it's done. Right. And you're like, where, where do I even start, you know, that was like the modality and we're like, okay, this is bad. So we, we created this process and it, and it worked pretty well. Um, where it got challenging is we liked to buy small things and small things became too small to become significant to the overall business because the business got so big that like, you know, it just didn't make a difference anymore. And then when they tried to, this was a little bit after my time, because I stopped doing the M&A stuff in 2000 and switched over to running a business unit. But, uh, after my time, they started buying bigger things and then you get into the problem that you're talking about, big things are very, very hard to integrate, they have their, they've been around a long time, they have a long culture, they like, you know, we bought like scientific Atlanta, which was a disaster. We bought like a bunch of things that didn't work very well. We also got into the mode of like, oh, we're not fully integrating them. We're like partially integrating them. It's terrible. Yeah. It doesn't work very well. So, um, so, so in other words, the mitigation is to have a process, but then when you stop respecting the process, you do have big problems like you talked about. Yeah. And what about from the portfolio company side? I mean, you got portfolio companies, we've got portfolio companies that at least in recent memory, we have had some exits into being other large companies buying them. Two questions there. One is if you're a portfolio company CEO, what would you watch out for, right? And then sitting on the boards of these companies, how do you advise your portfolio companies these days about exits in general? Um, look, when you're, when you're in our business of being on boards and you're facing a situation where you need to sell a bit, you know, there's an opportunity to sell a business. It's one of three outcomes. One, your company's doing great and it's a really bad idea to try to sell it. So then you try to convince the CEO, no, don't do it. Two, the company's struggling. We don't have a lot of finances. It's a nice way to get out. Everybody's happy. Then you just help the deal happen. You sort of manage a process. Then you have the tweeners. Uh, we are not totally sure. And, uh, I think that there are a couple of important indicators that help you determine what to do in that moment. The first is, does the CEO, the founder of the founding team have the fire? Like, do they want to do this? Do they, do they, do they believe they can get through this sort of intermediary period and, uh, uh, bust open into the next phase? Do they, do they want it? Because if you have a CEO or a leader, that's kind of like, man, you know, maybe I should consider this. I'm a little tired. It's been a long haul. Then let's just sell this thing and, and kind of move on. Um, and then the second factor is you have to be pretty cold and objective about like, is this going to be a interesting enough business? Not like, will it double from here? Because at the end of the day, if it's double, I'll take the money today. But will it 10 X from here? And that I would say gets to be a really hard decision because it's a little bit emotional also for you. Like you're, you know, you're not a founder, but you know, you've been working with this company for four or five, six years, your heart's in it. You've, you've told your good friends to go work there and, and yet you have to step, step away from it and go like, objectively, this is not 10 X from here. And then you help steer the ship. But I think the most important parameter is where's the founder's head at? Yeah. Yeah. And. So there is a certain element of mystery or unknown for a lot of the portfolio company founders on how do these things actually happen, right? People think one day somebody calls another company and say, Hey, I want to buy you versus, and then there's other thing where people build relationships for a long time. How did you approach it at Cisco and how would you advise your portfolio companies, even if they're expecting a 10 X? Yeah. Did you just call them up and say, hi, I want to buy you? Is that? No. Well, obviously you do as, as a good a job as you can building a network and knowing what's going on. I mean, it's basically cataloging the inventory is the way we used to think about it, but every company in the market is a potential acquisition. So you catalog them and, you know, where, where are they in their growth rate? What kind of founders they have? Who are the backers? What are their aspirations? Who are their customers? Just like be informed. So, and the, and, uh, you know, uh, the business, the corporate development group had to do that, but I kind of considered everybody's job at the company to be constantly informed about what's happening around them. So, uh, and then essentially, uh, we, uh, at the company, we would try to weave it at Cisco, we would try to weave in the inventory against a strategy statement that was there. For example, we're a router company, but there are going to be more switches than routers. Therefore we need to have a switch. What are the switching companies that are available, right? Or optical networking is becoming important because of bandwidth, blah, blah, blah. What are the companies that are available? Uh, you often have the, uh, the other way where companies come to you and says, hi, I'd like to get bought. I think they're broadly a bad idea. Right. because there was either an auction going on, or you should have thought about it earlier if you were interested in it, but basically, you almost never pay attention. We never tried to pay attention to inbound. Flip side, you're now a small company or a startup, and how do you think about that? I always encourage always the founders of our companies to talk to big companies, to talk to competitors, and to always be in dialogue. Some people will say, well, are you shopping yourself? No, it's an inverse intelligence-gathering mechanism. I even encourage my CEOs to talk to their direct competitors. Go to coffee, all right? Like, yeah, if you're clueless, you might leak some information towards them. Hopefully, you're not clueless, and they might tell something to you that's interesting, or they might shape your view on an industry or something, so I encourage people to be out there always talking and seeing what's happening in the market, and usually, you kind of have a pretty good idea of like, okay, my company's struggling a little bit. I think I'd like to sell. I kind of already know them because I've been in conversations with who's where, so I would say a CEO of a startup shouldn't spend their entire day talking, but going on one or two coffees a week with the powers that be at NVIDIA, the powers that be at Amazon, OpenAI, et cetera, et cetera, when the right people are talking, do it. It's fine. The information exchange, you don't lose that much in that. They're already thinking about it, but you gain a lot by having that relationship. I think this is really good advice. I keep telling my portfolio CEOs the same thing. If you don't have the cell phone number of the CEOs of each of your competitors, that's a problem. There will be a day when you need it, right? Absolutely. So it's a network you need to build. Yeah, yeah. Well, enough of this M&A stuff. Let's get back to actual technology. You spent 30 plus years in infrastructure. Very few people know more about infrastructure than what you've done in the past. Summarize the landscape for us right now. When you think about infrastructure opportunities versus where is the technology gaps that you see and what founders should be doing and what are they looking at? Well, super interesting topic. I would say you want to think about, I think, the current shape of this industry in two dimensions, right? One is the vertical stack that starts with EUV equipment and goes all the way up to applied AI and applications and everything that sits in between that sandwich. And then you want to think about, particularly the infrastructure side, what's sort of the value chain of it? The first thing you would say is right now we live in a world where demand for AI capacity, tokens, whatever, is near infinite and supply is incredibly constrained. So another way to put it is we all just live in Jensen's world. He's king of the hill, there's no question. I think the shape of the industry will begin to change when either the characteristics of demand change or the characteristics of supply change. And I think that my view is right now the supply side, because so much of the limitation, well, there are some power limitations for sure, but the dominant limitation is silicon. And I would say that unless there's more fab capacity, that doesn't change, which means it's not likely to really change in the next two or three years, maybe five years out. You could imagine that there would be more fab capacity, you could put more wafers through the same machines or you can make more ASML machines and so forth. But that's probably a governing factor of our industry, which means we live in a reasonably scarce world and those who have fab capacity are actually kind of king makers of the infrastructure. On the demand side, I do think that the world is gonna get smarter about the type of tokens they consume. I think we've been in a kind of an era over the last couple of years that everybody just wanted the super high-end frontier token. And I think the topic of conversation today is model routers and how do I move stuff to the economic difference between a top-end Anthropic token and a top-end Kimi token is like five to one or seven to one or whatever. It's a massive economic difference. So people are gonna get start smarter about consuming what. And I do think that that will begin to at least put some price dependency on demand. And that's more likely to happen in the near term. So those are, first of all, I think the dynamics that are gonna govern our industry and everybody that sits in between, if you're a Neo cloud or you're an inference cloud or you're a model builder, you leave between those two poles of supply and demand. Now for the infra itself, I think right now we still live in a world where you need to think of the entire piece of infra as a single system, right? So it's quite different, I think, than the networking days, for example, of old, where you would assume a very variable amount of traffic going from one to be types of traffic, times of day, lots and lots of variability, much easier to statistically multiplex that workload. Today, basically you have lots of GPUs that are all working all the time, always connected with much more constant flows. And because the interrelationship between the various components are largely still proprietary, you're on NVLink or whatever, it's very hard to swap one thing out for the other. In fact, today, I would argue that probably the best architecture is the Google TPU architecture because it is literally thought of as a single system. And I think an NVIDIA style architecture is migrating in the direction of where the TPU architecture is but it still has a few pieces that you can kind of swap out. And then you have everybody else who's like, I'm gonna take a Cerebrus chip and throw an Arista switch on top of it and like all that good stuff. And at least in today's world, those systems have inefficiencies because they're not constructed as a system, they're more of a patchwork. So I think that industry will evolve, but today, people that can think of the whole thing as a single system and optimize the architecture of it through that have a competitive advantage. Yeah, I mean, it seems we're moving from a card as the minimum unit to a rack as the minimum unit or even a few racks in some cases. A few racks. Yeah, that's true. The NVIDIA part that they announced recently, that is the minimum unit mostly. Yeah, well, but if you look at how Google does this thing with these like toroidal pass-through optical MEMS switches, I mean, it's literally like the whole fucking data center is the minimum unit. So the row is the minimum unit? Yeah, exactly. But my sense is that's where NVIDIA wants to go also. Actually. So the two dominant providers are all going to that systems approach versus. I mean, even the smaller startups are heading that direction. Jumping on that, how do you see AI chips evolving? And right now, Jensen, King of the Hill, incredibly dominant, right? Google TPU, very successful. Currently mostly sold by Google. It's expected to change. Not enough TPUs. But not enough TPUs, exactly. How's the future looking like? Will we see more competitive silicon? I think for sure. We all want that. Yeah, we all want it and we all want to invest in it. I would say, first of all, what I find interesting is that model architecture is becoming reasonably diverse, right? So you have some people using FP4, FP8. Some people are using mixture of experts with lots and lots of experts. Some have fewer experts. Some are jamming up the parameter counts, some not so much. And I actually think that there is a inner relationship between your chip design and the model that's optimized around it, right? So, in some sense, to your point, like, is there going to be more diversity in chips? And I think, yes, because there's going to be more diversity in model designs. And depending on what kind of model you want to do, what kind of things you want that model to do, you can certainly select a different chip architecture for that. Now, reasoning intensity has dominated to date what chip architecture should be because reasoning intensity equals big parameter counts and big KV cache. And that means that you need to talk a lot between memory and chip. And when the only way you can talk between chips are the edges of it, it's just less connections than if you can use the top and bottom of the chip. So, I'm a big fan of 3D DRAM architectures. And some of the new stuff that's coming up with hybrid copper bonding is even more interesting because you can have even higher density. So, I think that architecture has huge promise so long as we continue to see these KV cache, in particular, KV cache intensive use cases. And it seems like the most value-added labor that occurs, labor or AI that occurs, is reasoning-driven labor, right? So, and therefore, that should favor architectures that have very good memory bandwidth. We're investors in Cerebrus. Cerebrus did very well to go fast, but if the model doesn't fit in the SRAM. You need a lot of them. Yeah, well, yeah, you need a lot of them. Because once you get off board of Cerebrus, it's not gonna go super fast, right? And also, because the chip, the wafer is so large, the actual, you're actually at a significant disadvantage in terms of the memory bandwidth once you go off chip. So, now they run Sol on the Cerebrus chip. So, that's a big model. So, they've figured out how to fit it in. But my guess is, OpenAI had to do some hard work to get that model to fit in the chip. Yeah, no, we are fans of 3D architectures as well, but the manufacturing will be a lot more complex with thousands of effects and so on and so forth. Yeah, no, I'm a big fan. I'm a mechanical engineer that studied thermodynamics. So, Sheet is my friend. That makes you one of the very few people who would say that. Yeah, so. All the packaging divisions of TSMC and everybody else are calling you right now. Yeah, yeah, no, I agree. The packaging of those chips are gonna be complex, and especially, obviously, the thermal attributes of it. And then, when we get to a world where you have both a logic chip and a memory chip in the same package, the different coefficients of, well, they have the same coefficient of thermal extension, but logic chips get a lot harder. So, the ability to put that all into the same package will be non-trivial, shall we say. Chips are hot again, and they're getting hotter. The other possibility, of course, is to keep them still separate, but have optical interconnectivity between them. Are you a believer in that? Yeah, I think yes. Optics, though, are very finicky things that need to be experimented with a lot. So, when you heat up optics, they don't like it. Photons don't like getting hot. And, I mean, in theory, you would have a substrate that had waveguides in it or something, but the minute the logic chip starts getting hotter and you get sort of outside of the comfortable temperature range, you're gonna have a variety of connectivity issues and so forth. I think optics are interesting. If you can do it with copper, I'd do it with copper, is my view. But if you, yeah. Even for scale-up racks, not just interconnect now, but even for. No, no, I mean, obviously, for scale-up racks, that's a different story. That just simplifies the cabling and so forth. I'm talking about it at the module. Inside the server. Yeah, yeah, yeah. For, when you start going top of rack and all that. Yeah, you need it. Definitely, no question. You just can't, you don't have enough bandwidth. So, Cisco back then, most valuable company on the planet, early hardware leader in networking, which sort of powered all of the internet. At some point, it wasn't the most valuable company on the planet anymore. What triggered that? What can we learn from that? What tends to happen when you have a bubble in demand, in perceived demand, is that customers over-order because they know they're gonna get cut back. And so, every part of our product line was out of capacity. We were, we couldn't, we didn't have, we couldn't make enough switches, we couldn't make, we had orders on top of orders on top of orders on top of orders. And then, we were, of course, putting orders on all of our supply chain, and because the demand felt so robust, we were taking the financial risk on those orders. In other words, we were like, here's the money, keep the money, just give us this stuff as soon as possible. What could possibly go wrong with that, yeah. Yeah, so then, when demand dries up, it's not like people stop buying, they cancel all their orders. They're like, yeah, I ordered twice as much as I wanted to anyway, so I need half of half of that. So, the orders went to like one quarter of what they were. And then, A, you've taken the risk on the supply chain, and B, your demand falls through the roof. And so, I think. Inc, we ended up in the December quarter and a particular, this is kind of memorable because it happened in December quarter. So we are one month off. So, uh, November, December, January, and usually what tends to happen with businesses that are a month off, you, you know, this right, like December soft. And then the first week of January, you start seeing it coming back. And then by the end of January, you've made it up. So we go through December, it's soft and it's like, no big deal. December is always soft. And then week one, January, nothing happens. Week two, January, nothing happens. So the quarter came in, I don't know, maybe we missed by 35, 40%. Wow. Yeah. Massive MIT for a company, by the way, that publicly traded for a decade, never missed a quarter. Yeah. Uh, and, uh, yeah. And then that's how it all corrected. And it was, uh, it was difficult. Stock went from 80 some dollars a share to $9 a share, I want to say. So it was like literally one 10th the value over a period of six months, very tough. Yeah. So, so look at every time when the workload changes enough at the top layer, the, the underlying infrastructure, software and hardware needs to change with it, right? With AI, we've seen this, we talked about compute, right? What's going to happen in networking? I mean, we've seen big changes in networking. We no longer have one network. We don't have suddenly the second backup network. Where did this come from? You just say you thought it was dead. There we go. Ethernet is a framing mechanism. It's coming back. It was dead, right? Here, here, I think, I think that the core tenants, the world of telecom started with circuit switching, right? Like we made the phone and the idea with the phone is that you had starting point A, starting point B, constant bandwidth between two, you connect the line and off it goes, and then we found out, well, this is like, uh, reasonably unreliable because one line cut and you lose it. Uh, and, uh, obviously it doesn't scale very well in a world where you have uneven traffic patterns. So then we decided let's go to packets and packets are very convenient because they allow stat muxing, uh, which is the big, big benefit of it. David and Andy back then, right? Yeah. She'll eating it. Yep. So, so, uh, that, and, uh, you know, variable length packets and all of these, you know, IP packets essentially. So essentially the world grew up building networking equipment, assuming that you had uneven arrivals and departures into a device that spread out the workload and you would under provision the device to connect more bandwidth because not everybody all communicates at the same time through the thing and the outputs of that inbound are highly variable. Okay. That, that was, that's the internet. That's the workloads that we live with. That was almost dogma, right? For a long time. Okay. So fast forward today. What does an AI data center look like? It does not look like that. You don't have random arrival. Um, you, links are consistently fully utilized. GPUs want to be fully utilized. Traffic patterns are actually much more predictable than they ever were before. Particularly in training. They're incredibly predictable. There are less predictable in inference, but you can see the traffic pattern, especially if you do like good batching. Good chunk of it is still very predictable. Yeah. Especially if you do like big batching, like you can definitely make it more predictable. So the fundamental tenant of the workload has changed. Yet, if you look at the dominant networking architecture today, which is the Broadcom, uh, uh, what's it? Not Trident, Tomahawk, Tomahawk, uh, which is the doing acquisition that they did is a StatMux packet switching architecture. Yes. So I think that at some point for data center networking, we're going to have a new kind of architecture that assumes a high degree of predictability and a full utilization as opposed to an unpredictable, a stochastic StatMux approach. Um, So essentially back to circuits is what you're saying. Yeah. And optical circuits, which is. Yeah. So, Or, I mean, time division, multiplexing works as well, right? Or combinations of those. Yeah. You know, you can have, you can have many modalities of multiplexing, but they're, they're not stochastic, right? They're, they're, uh, uh, constantly timed somehow, like you can do TDM, you can do, uh, whatever else you want, but basically, so, and I think these architectures, the, the, the issue is the switches you buy today have the correct amount of bandwidth and the incorrect architecture, not incorrect, but, uh, less adapted architecture. Uh, and I think the data center switches of the future at some point will look more like that. Will look more, uh, like, you know, potentially you could do it with MEMS. To bring ATM back into the data center. But even ATM had StatMux stochastic cells, right? Straight up circuits. Yeah. Um, so now networking vendors, including my former employer, this is bad news. Because when things don't change, you can't charge a lot of money for them. Right? Like the whole point of a switch is that it's a complicated animal to have a stochastically, statistically multiplexed traffic flow. It's a hard thing to have. Oh, my one packet came in here. I have to throw it into memory. I have to do a lookup, do the lookup. I got to send it out of this gate. That's like hard to do. So you make the thing expensive, but in a world where it's relatively static, you don't need that much intelligence. Don't need buffers anymore. Bam, send it right through. Right? So I think that there's an opportunity for somebody to, to sort of rethink network architecture, given this workload and make it way cheaper. It's like classic innovators dilemma, because no matter what vendor you are today, this is not a good trend. You're not going to want to do that. But if a newcomer and you say, you know what, forget all that stat muxing shit. Like the world's different. I'm going to go with my thing. Actually, you might be able to build something interesting. Um, now it could be optical. It could be electrical. It could be a combination of the two. Um, but I think we're going to, what will emerge. And I think you can see clues of that from how Google does its network. Yeah. Yep. I mean, I think Amine has, has been the pioneer of specifically optical circuit switching, right? Because they were the first ones. And that's where like the toroidal thing. Yeah. Yeah. And I don't understand why it's so slow, by the way. It takes them like a minute to switch. So that one, I don't totally understand. I think to some degree it's because of the use case that right now you use it to set up fairly static. I mean, when it is static, what do you care? Fair. I mean, for availability reasons, you don't want to. Give me a few milliseconds, not a second. Yeah. But you probably could, honestly. Right. I mean, the technology I think is there. It's just a use case. It's not there. Yeah. Yeah. So yeah, no, I've been looking at a bunch of these, whether it's MEMS or, uh, is, uh, Max Zander thingies, the MZIs and whatnot to see if there's something in there, but, you know, somebody has to emerge with an architecture and then the issue is going to be who embraces it because you can't just build a switch without the rest of the system coming together. The, I mean, that's the other thing. The universe dilemma extends to the user's inertia as well. Exactly. Right. Well, this is what the network to some degree, coming back to the, you know, ship me a row concept of infrastructure today, the network has become an integral component of your architecture. So this innovation to some degree almost has to come out of the GPU vendors. Yeah. Yeah. But it's interesting, right? Because there's only NVIDIA, maybe AMD will get there at some point, but only NVIDIA and Google right now have the ability to ship you a rack system. That's right. Yeah. So today they would have to adopt it. Uh, it'd be interesting to see, like, you know, as more chip vendors evolve, is there, uh, you know, is there an alternative architecture that they can all agree on, um, sort of the shit that, you know, we did back in the days, back in the day, right? So like, the thing is like, you have two dominant vendors, the ITF 2.0, well, they have two vendors. They're dominant. Uh, they have their own approach of doing things. They're not likely to shift off of that. Yeah. And can you have a collection of everybody else that says, no, actually this is a standardized network architecture that we're all going to agree upon. And that's what we're going to do. The Rebel Alliance. I like it. Yeah, the buyers have to agree. That's the case. Yeah. Yeah. So Mike, you've seen a lot of these cycles now, some bigger, some larger, that you've seen the last really big one. How does this compare? As you probably know, I've started a new venture fund called Anabi Capital. Technically speaking, I retired in 2023 and I was supposed to get a yacht. And it didn't work out very well. Why? I actually think this is the most fascinating time to be doing our job and to be in our industry, not just because the change is big and we all know that, you know, how transformative the changes, but because it requires more. Uh, intellectual exertion to understand how, what next? It's like, you don't have a crystal ball. It's not obvious what the next move is. Yeah. Sorry. So you have to keep thinking and you have to think at a systems level. Like, I don't think I've ever been in an industry before that I had to think about how light gets concentrated on a Zeiss mirror to figure out whether my chatbot works well. And like, it's all, it's this amazing interconnection of things where there's bottlenecks and this changes and this change with that change. So I find it just the most intellectually fascinating time. And because the end prize is so significant by figuring out how these different parts unlock, you can create a company here and a company there, and this one can become worth a trillion dollars and this one can become 500 billion and so on and so forth. And I honestly, I did not expect to encounter another time like this in my career. And by far, I find, I find this the most dynamic and interesting time to be doing what we're doing. And I think we should all feel quite fortunate about it. Yeah, this tops 99. Oh, yeah, easily. Totally, totally agree. And we are all in the same business for very much the same reason. This has been a fascinating hour. Thank you so much. And we could be continuing this conversation endlessly, but perhaps we'll do it another time. This was super fun. Thank you guys. Thanks for coming, Mike. We appreciate it. Thanks for listening to this episode of the A16Z podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts, and Spotify. Follow us on X, A16Z, and subscribe to our sub stack at A16Z.substack.com. Thanks again for listening, and I'll see you in the next episode. As a reminder, the content here is for informational purposes only. It should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see A16Z.com forward slash disclosures.