Building Cyber Defense for the Agentic Era

Key Takeaway

Treat AI-era security as a continuous practice, not an annual audit: map your external attack surface, identify what changes most often, and trigger targeted adversarial testing whenever a critical application, route, service, or threat changes. Then pre-agree on temporary compensating controls so a verified critical exposure can be contained immediately while a permanent fix is built. The goal is to shrink the window between a new weakness and a defensible response.

Episode Overview

Kevin Mandia, founder and CEO of Armadin and founder of Mandiant, argues that AI shifts cyberattacks from human-scale, sequential operations to high-speed, parallel “drone swarms.” He explains why organizations need continuous AI-driven offensive testing paired with autonomous defensive controls, and discusses the operational implications for CISOs, AI labs, and fast-growing security companies.

Full Transcript

Transcript of Building Cyber Defense for the Agentic Era from A16Z. Auto-generated from episode audio; may contain minor errors.

If you don't have a defense on this, you have a great offense to go up against. You want to be the Baltimore Ravens defense of 2000, you kind of want to have your practice offense really push you. That's what Armanin's going to do. We're going to be the all-star team on offense coming at you so you can train your defense with what we're doing. Are you built manned yet? A great success. Why did you decide to get back on the field? I don't want to sit out the AI shift change when I've done 30 years in security and the whole damn thing's about the change.

What AI does in a microsecond would take 70 humans. They can't even do it. It's apples to oranges. This is a tsunami like has never been seen before in security. The whole, let's slow down the models, we don't want cyber risk, too late. The open models are already good enough. Armanin since January of this year, we have found over 90 zero days at customer sites, all in production. This is not like rinky-dink companies, like these are like Fortune 500 companies. What are the differences or similarities between nation state attacks compared to AI today and then where you think AI can be in a couple of years?

On the defensive side, we're going to say we're being attacked by these models, but we're not sure who's behind them. Is it a nation? Is it a human? Is it? Kevin Mandia has spent 30 years in cybersecurity. His view of the AI transition is simple. Everything I did is dead and then everything else is new. In this episode, David George sits down with Kevin, founder and CEO of Armadin and the founder of Mandiant to talk about what happens when cyber attacks move from human speed to machine speed.

Kevin explains why AI gives attackers an immediate advantage from probing thousands of paths simultaneously to making less sophisticated attackers dramatically more capable. And he makes the case that there's only one viable response. Defense has to become autonomous too. We also get into how Armadin uses AI to continuously attack customer networks, what the team has learned from finding more than 90 zero days this year, and why Kevin believes the next two years could remake nearly every layer of the cybersecurity stack. Kevin, thanks for being here. No, thank you.

Okay. So you built Mandiant. Yes. Obviously a great success. Many different chapters, you know, it ended up inside of Google ultimately. Why did you decide to get back on the field? It's a good question, and I don't know if I decided it, and that'll sound weird, but I met with David Slater and with Travis Lanham, the other founders, and Evan Pena I knew. I could say they started this company. They are the founders. I met them. They had the idea. They pitched me on what they wanted to do, and I saw the talent in them.

Travis is a generational talent. David Slater's, and I mean this in a positive way, freak of nature. These guys are really, really good. Evan Pena is exceptional at what he does, and when you meet that team and you talk to them, the whole time I was listening to what they were doing, I was thinking, I want to be a part of this. I don't want to sit out the AI shift change when I've done 30 years in security and the whole damn thing's about the change.

That's great. Everything I did's dead, and everything else is new, but meeting that team and they were starting the company and me realizing, I think I'm a real good fit with these guys to accelerate the need. What Armadin is building every company needs now, and I was like, this team, I can build it, and I think I can answer the now. Thirty years in security, you meet a few people. Let's go to those people and say, we've built what you need, so I almost felt compelled to do it.

I know that sounds weird, but I would not have founded a company again in mid-50s and I was doing venture. It wasn't like, oh, I'm an entrepreneur and I love starting companies. That's not it. And it wasn't, oh, I'm not a VC, I'm just an operational guy. That's not it. I met the team and went, we have to do this. Yeah. I mean, that's really it. Yeah. And this whole AIC change was happening. Totally. Absolutely. That's the catalyst, right? Yes. Yeah. So tell us about what Armadin does.

So Armadin leverages frontier models and AI. On offense, the test, do you have exploitable risk? And that's what we do today. We call it Armadin Red, but when we started a company, Travis and David Slater and Evan all knew the future of cybersecurity is going to be, A, the good guys have to build the offensive cyber cannon and shoot it at networks to make sure those networks can withstand these attacks because they're the ones that are coming. But we also knew it's going to be AI on offense built by the good guys working in training with AI on defense built by the good guys, and you have to have both.

So our act one was, we got to be the best in the world at finding exploitable risk. If we're five minutes ahead of the bad actor, whether it be a nation state, criminal, nuisance, if we're five minutes ahead of them, we also recognized our act two, Armadin Blue, we got to stop it, compensating control, tourniquet. And so that's what Armadin does. We're building the force field you will need in the AI age to defend yourself from AI attacks. Yeah. Tell us about the nature of the capabilities of AI attacks today and then where you think it goes.

Great. So the nature of them is, first off, we're getting a weird window in time where we're seeing them, but not at the same level you'd expect. I've seen nothing like what Armadin's already built in the wild, which is there's 25,000 intelligence agents on concert, all working together, doing really, really smart things without going on bizarre fishing trips. Because when you respond to an AI attack, you can tell it's AI very quickly. At least I can, because I've thought about a lot of offense, I've responded to a lot of attacks in the past that were led by humans.

And a human goes to point A, then to point B, then to point C through their intrusion. AI does little things, like four or five differences, but one would be, it'll break into point A then laterally move to point B. Next thing you know, it's trying to break into point A again. You know what I mean? Yeah, yeah, yeah. It's like, I get the drone sword, but you can probably coordinate and think a little bit better. That's where it's at today. It'll get better and cleaner.

But the differences are, first and foremost, the scale of what AI can do dwarfs humans, like in ways humans don't even get. So you have a scaling problem in that humans could always find only one path into a network. Yeah, they had to be selective because they had to devote their limited resources to one direct path, right? Yes. And then, so scale is a challenge, speed, ridiculous. What AI does in a microsecond would take 70 humans. They can't even do it. It's apples to oranges. And then, so what was always lacking is AI creative or effective.

But when it comes to what we do, we don't need the fanciest model. We're not trying to speak 400 languages with our models and all that kind of thing. What Armin is doing on offense is we're finding vulnerabilities, exploitable risk. That is code. That's a structured language, a structured process. Because it's structured, AI is going to be great at it, right? So I really think it's already here today, like the whole, let's slow down the models. We don't want cyber risk, too late. The open models are already good enough and these things are common now.

It's just a matter of the minute you have anonymous availability of GPUs, you'll see far more criminal attacks. Oh, interesting. Yeah. You know what I mean? Yeah, of course. But until you can attack anonymously, and it's hard to do crime when people know your name. Yeah. It's better. Yeah, of course. If you can commit a crime anonymously, here's my tip for criminals. If you can commit a crime anonymously, that's a lot smarter than doing it with your jersey on, with your name on it. And so anyway, the difference in attacks and what we're seeing now, we are at the precipice, first inning still, of AI-led attacks coming.

And I think that's just because of the cost and availability of the models is not as readily available to the criminal element as it will be in the future. Yes, exactly. Okay. So you, your experience in working in the security industry for 30 years, you probably saw a fair amount of nation state attacks, right? Every day. Every day. So talk about the differences or similarities between nation state attacks. And I use that just to say the most sophisticated, most successful, if you will, types of attacks compared to AI today, and then where you think AI can be in a couple of years.

So everything's going to change rapidly, right? But I can tell you, nations on offense have never, in my opinion, they've never really been, when you're hacking for espionage and for security reasons, you hack with what I would call kind of a sniper round. You're not spraying and praying. For the most part, modern nations on offense restrict their targeting and they go deep at very specific things like 30 defense contractors or dot mil when they go hard at that. Kind of think of it as that sniper round with AI, I think it becomes more like a drone swarm.

It becomes a little bit different in the cyber domain. And I think even modern nations are thinking, what will our protocol be? If we want to attack this company, do we swarm it and just burn tokens on it? Because AI is going to do a lot of things humans just wouldn't. So it's a little sloppier, a little louder, but it's more effective. More comprehensive. Yeah, that's the problem. You got it. It's more effective probably. And so there's going to be so many things a nation's got to think through right now and their whole doctrine will shift as the AI shift change comes.

How does AI change what our mission is? Do we maybe use the cyber domain differently? Do we drone swarm sometimes, snipe around other times? How do we balance the two? Does it depend on risk, target, how surreptitious we want to be? Because right now AI is not a surreptitious action on offense, unless you've done a ton of post-training. You got maybe a human in the loop really looking at, are we doing smart things? Because if you just go, hey, here's a prompt, hackabc.com, AI is not going to do it in a surreptitious smart way.

And I think even if you ask it to, it's still not going to, until it's been really trained and really had some human influence on it. But more generally, what you're going to see is less capable attackers, less technical, less successful, are going to appear way more successful. It's the equalizer, right? Yeah, because of the buoyant. When you start using models over time, what it's going to be is on the defensive side, we're going to say, we're being attacked by these models, but we're not sure who's behind them, those attacks.

Is it a nation? Is it a human? And we'll have some clue, but attribution will get a little difficult. So that's a long-winded answer saying in the AI age, it does democratize far greater expertise for attacking victim networks. Yeah. So then that is a good segue back to Armaden. So you have talked about the defense needs to be a great offense, right? Yes. Like best defense. You've got to train your defense with something. Yeah, of course. You've got to train your defense with something, and then it needs to be continuous, right?

So how does the product work, and then how do you get a level of sophistication such that you can identify and remediate these vulnerabilities, like what you're describing, that are more sophisticated than a basic prompt? Okay, a lot there. I could talk for 45 minutes on it, but first I can tell you this, you don't have a defense on this. You have a great offense to go up against. Right. You know what I mean? So even thinking in sports terms, if you want to be the Baltimore Ravens defense of 2000, you kind of want to have your practice offense really push you.

You know what I mean? So that you know how good you are, and that's what Armaden is going to do. We're going to be the all-star team on offense coming at you, so you can train your defense with what we're doing, and that's going to be important. And you can't really have a human in the loop in the AIH for tactical autonomous defense. You got to do something fast. Yeah. You got to tourniquet the wounds as fast as you can. So thinking back to our core, you want to be able to do it continuously, and that's the complexity.

So we do a thing called a hyper attack, and David, that's just a fancy word for we throw a drone swarm of agents at you, and we map your network. Every service, every route, every system, all assets. We may end up with terabytes of metadata on your network from this hyper attack. Done super fast. It lights you up, so that that way we can now, it's almost like a metadata twin of what we can see. If we're on the inside, do the same thing. Let's just map everything.

This is the attacker's view of your network. But with that metadata, we now just pull you almost like a heartbeat. What's changed? What's changed? That's continually looking for, did an app change, did a route change, did a service get updated? Did a new machine get presented onto the, into the target area so that we can pull cheaply for change and then attack the change? What you really want in the future in the AI age is you want the constant pressure of models attacking you, but you can't do it all the time because A, it's cost prohibitive, but B, it's unnecessary.

You do it when either the threat changes, hey, new models come out, new intelligence is available, or B, your network changes, so you want to test whether that happens. And then C, you probably just want to test, we have a board meeting tomorrow, let's see how we do. You know what I mean? Yeah, exactly. Let's audit ourselves and see where we're at. And that's what we had over the weekend. There was a zero day and a popular product, and immediately, we've already got the heartbeat. We just pulled who's got the problem.

And our goal at Armadid is to go from, you know, common vulnerability or CVE to, we can find if it's exploitable or not before bad guys can, you know, and not all bugs allow for human access to your system in a stable way, so not all bugs are created equal, right? And so we want to make sure, hey, this one's one you really need to worry about or which ones don't give remote command execution. So long story made short, that hyper attack, that metadata that we get allows us to kind of pull for change and attack you when your network changes, and that's the best you can do for continual, and we'll get better and better at it, and the cost for the pulling will go down.

You know, on small networks, it doesn't cost much, but on a network that changes all the time and is very large, you'd be pulling it quite a bit to make sure you don't have an exposure window. Yeah, of course. So that is actually a very good explanation for why this continuous approach matters. Well, you're already up against it. Somewhere out there, the criminal element will always have that random scanning, and they probably aren't even using AI for it. They've got one exploit that they think works, and they're just kind of scanning the world for it and then coming at the exploitable risk, you know?

And so you're already getting some pressure on your network from an unseen force that's not well-intentioned. You know what I mean? Yeah. better force built by the good guys constantly putting pressure on you. Yeah, it's interesting. So you would kind of, Armaden, I don't know, a year ago would probably be placed in the category of pen testing. And you and I share history and relationship with George at CrowdStrike. And so they famously redefined the category from AV to EDR, and of course they did it in a couple of things.

But the category redefinition was on the back of major infrastructure changes and product changes, and allowed them to create a product category that was far greater and bigger than AV. Talk about pen testing. What is the historical view of pen testing and why that's not what the future is? Yeah, a couple things. I mean, you had to do it, right? It was kind of like first gen AV, you have to buy AV. And I think when you look at Armaden, we will be as ubiquitous as AV because you have to have that AI force field of AI on offense, training AI on defense.

You have to do it, and you can do it, so why wouldn't you? And so you look at that, and it's, pen testing to me is always just scanning for what's already known, and it doesn't prove whether you're really exploitable or not. So it's always created a larger list of vols that don't matter. And so the way we wanted to do it at Armaden was we actually can carry the exploit out. We verify, so there's no false positives. We can get remote code execution or get data off of that machine.

And a lot of pen tests are nothing but hit your infrastructure, not with a thinking, learning technology that memorizes and knows your infrastructure like an AI agent can. So it's not gonna do custom apps. It's gonna, at least the old versions of pen testing, the Tenable, the Rapid7, the Qualys, was more a hygiene sort of thing. What do I have out there and what services are exposed and are there CVs available against those services or known exploitable vols against them? When you have an AI-based attack, it'll find logic flaws rather than code flaws in custom applications.

It'll exhaust all routes all the time. And it's like, all I can tell you is Armaden since January of this year, in 2026, we have found over 90 zero days at customer sites, all in production, and- And by the way, your customer base is like- They're happy we found them before someone else did, yeah. This is not like rinky-dink companies. These are like Fortune 500 companies. Yeah, and I don't mean that as fear, uncertainty, and doubt, but the difference is that we've trained our models, we've post-trained all our models with real red teamers, real folks that actually can develop exploits, and that's important.

And so when we're scanning networks, we don't have source code to review. We're not finding these zero days with source code. We're not finding these zero days because we can log into an app and now we have access and we can get to other things. We are black box coming from the internet over 90 zero days in major software companies, and they're thankful. And so everybody's like, wow, Mythos came out and you can scan source code and find vulnerabilities and you find thousands of them. That's noise.

Yes, we're coming from the outside and then we're calling assist, so usually within 48 hours, hey, we've got remote code execution in your DMZ, and usually from there, we're getting in. And they agree with us. And the nice thing is we're going through, the fixed side's a little bit harder, takes a little bit longer, but those companies go right into incident mode. They respond as if it's an incident. And that's not a pen test. That is like a real adversary coming at you. And the difference between red teaming and pen testing is pen test to me is a hygiene step.

And I think over time, everybody would have red teamed everything all the time if they could. It was cost prohibitive and people prohibitive. With AI and an agent doing it, or in our case, we have lots of different types of agents doing different things, you can now do that. So I think it'll replace pen testing over time. That's like a small portion of what a red team coming at you would do. Yeah, so you talked, you mentioned earlier, obviously that's Armadon Red. You mentioned Armadon Blue.

Talk about Armadon Blue. Armadon Blue is like, we can't, David, just show up and say, hey, you're vulnerable, see you later. And hey, the true north for every CISO should be effective autonomous response. We gotta build that. And we knew all along, you can't just say, hey, we wanna be the best one at finding exploitable risk. That's goal number one. But then goal number two, and be the best one at doing something about it. And that means Armadon Blue. And Armadon Blue will be, take the information about exploitable risk and work with the defense plane, whether it be endpoint EDR or firewalls, and create compensating controls at speed.

Yes. So that if we find an attack five minutes for someone else using a model finds an attack, you're already safeguarded. And these safeguards are gonna be rudimentary, potentially out of the gates, right? Over the next few months. Year from now, they're just gonna be there. Because the whole cyber domain is progressing at a speed where you're gonna have to defend autonomously. For better or for worse. You know, I'd rather have a bad patch stopping a bad guy from getting in, than have an intrusion. Right, you know what I mean?

So you gotta take your lesser of two things, and one's much more manageable. You never want an unknown person with arbitrary access on your network. Yeah, yes, yeah. And so you wanna prevent that any way you can. And I would say the first generation, as we're working with CrowdStrike on it, and they know it has to exist, we're working with Pan on it, they know that it has to exist. They wanna all, you know, shift into the AI age with autonomous defense as well. And so we need to inform those defense platforms, you know, the Fortinets and everybody else.

Here's what you can do about it. And I likened it to, you know, kind of field dressing in war. Someone gets shot, you patch it up, but that's not the hospital, you know? Yeah, yeah. That's a, hey, we kind of stopped the bleeding. But then you gotta maybe do something else with more time and humans, potentially, or even, you know, agenic approach to it down the road. So you're gonna see it happen, even if you're a CISO, you're gonna see autonomous defense happen even if you don't ask for it.

Right. Because of the defensive platforms you've already invested in. Yeah. You mentioned earlier, you know, some of the blurring of the lines of different categories within cyber. And I think you joked that your old days, your 30 years of experience is out the window or irrelevant or something like that. What is the future of the SOC? And then how do the categories within cyber blend together or change? You know, if I'm a CISO, I do believe my true north is effective autonomous security. You wanna keep your best people engaged.

You wanna automate the processes that work for your organization. But you are absolutely saying what survives in the AI age and what doesn't. Right. And I think we're still working through that process. I think there's whole processes in the SOC that'll just go away. And for whatever reason, we're automating right now. Yeah. You know, over time, I can tell you this, if you have humans in the detect and respond loop, you're gonna be too slow. Yes. You know what I mean? It's just not gonna work well.

So you have prevent, detect, respond. Prevent's gonna be governed by AI and detect and respond is gonna be done by AI. And the goal in cybersecurity has always been if you have, you know, you wanna prevent. Yeah, of course, yeah. You don't wanna detect and respond. So I just see the constant narrowing of the window of every phase to the point where, you know, we're really not doing a lot of detection and response because the window to do it, it all happens too fast. Yeah. You got it, it's a little bit too fast.

So, but you still gotta have that onion peel to some extent of systems backing up systems and assuming failure somewhere. Right. You know, like even arm it in creating the force field, sooner or later, somebody's gonna get around it. Someone's gonna create an exploit before we find it somehow, some way on a platform or an app that we just haven't assessed yet. It hasn't been in production at a customer site. And so we haven't looked at it and someone else finds it. And when they do that, you will wanna have a trap behind saying we've got unauthorized access or unlawful access to a system.

Those traps are, you just can't have a human there. Yeah. I mean, it's just gonna, because we've already done it at Armit and we break in and have a gentic aware internal command and control, it proliferates at a speed that is shocking. You know, like I remember as a human, you're like typing on your keyboard, I wanna go laterally move with this passphrase from here to here. And you're so slow and you're doing one thing at a time. This thing just does a thousand things at once.

It's just everywhere. And you're like, whoa, okay, done. Got the app. Yeah, so each one of those steps of the process has to be automated, can't be a human in the loop. Yeah, it's as bad as this. I mean, I don't have great analogies. It's like the balloon popped. You know, it gets in, it's just like, they're gone, the whole defense apparatus just popped. So you gotta get prevention right. And then an immediate lockdown on detect and respond, however you wanna, and there's always gray areas between those phases.

Because people say if you detect and respond automatically and fast, that is prevention. Yep. So all of it's gonna change. Every system's examining it. Every vendor's examining their role in changing into the AI shift. And so it's gonna all change now, but I can't tell you if anyone's positive on how it changes. They're examining their workforce. They're examining their head count. They're examining their processes, meaning the CISOs are. And they're saying, what do I need to look like in the modern era? But it's too soon to tell where it lands.

Yep. So too soon to tell what they look like, what their defense apparatus looks like. Yes. So you have a bunch of Fortune 500 customers. You're close with a bunch of others that are not customers. Like, what is the state of their vulnerability today? Rapidly shrinking. You know, everybody's worried. There's a desperation in a moment in both directions, by the way. If you're on offense in Iran or Russia, you have a desperation and a moment of getting now. You could get it before the security is in place.

You got it. And then if you're on defense, you're desperate to patch every window. And in fairness, both sides are accurate in their desperation. I mean, because we have near-term pain in the AI age that it advantages offense, right? So that's fine. That's just the nature of it. But both sides recognize it's for long-term gain. Meaning AI on defense, being trained by AI on offense, and being autonomous is gonna do a far better, far more diligent job than humans peering at packets. And so we're just going through the window of exposure, let's call it, where everyone's at risk on defense and they're all hustling.

I've seen incredibly powerful efforts at every company right now. And I'm not aware of any large 1A enterprise not actively scanning for exposure and doing something about it in real time. And what's interesting, David, is it's like, it's team ball everywhere. Like the CIO, the CISO, the product teams, the business lines are all like, okay, we found something. And it's almost like war roomed. Like, we gotta go fix this, you know? And a composite of those teams are pretty broad. So there's no way to make the next year pretty.

That's the best way to say it, you know what I mean? It's a cocktail party out there right now, a digital cocktail party. And everybody's in a race. Yep, yeah, it's one of our most sophisticated companies told us they took a large percentage of their engineers and research organizations and just devoted it toward fortifying their own walls, which it was like an all hands on deck. And the alarm bells started ringing very recently. Like this is within the last few months, right? I think Mythos was the biggest.

I mean, we saw it coming long before Mythos, but the Mythos moment from a marketing standpoint got everybody to go, okay, threat's changed. And a lot of people said, Mythos came out, find vulnerabilities in our own software. And you have to do that if you're doing software security at a station. So all the vendors ran out and did that. But the way I respond to Mythos is that just made it very well known about AI on offense coming at you. And I think that's what accelerated it.

That was pretty much a firm stamp. It's coming. Yep, what about the hugging face incident? I'd love for you to talk about the learnings from that. I've given that a lot of thought. I mean, I'm certain at OpenAI, they're like, oh, we're at a regular. They were like, oh, we could have done this and this and it wouldn't happen. You know what I mean? So they've already figured it out. It's been my experience in every technical modality shift, we underestimate the adversary's capability. And in this case, we underestimated the model's capability because when you really read it post-factual, they could have stopped that.

And they could have put guardrails on it, some deterministic things. And I think they realize that now. But I think when you're in a race, it's almost like a lunar landing race, right? The AI race. And you have R&D people. And they're doing the work to create models in a way where even those CEOs are like, we can't slow it. Let's get the government to help us slow it. That means you can't even control your own innovation. I have views on that, but we can take it another time.

And so when you have, and I get that, R&D people are like chasing that innovation. And it's really hard to package them with then like security, experienced security people that have the skill sets to cage that thing, you know? And it's hard to marry those two up because the security people don't understand the AI as well. And the AI people don't realize. One of the things that we did in our model, I mean, make no mistake, Armitin has made the beast that we're all worried about.

We've made a model that attacks. We made many of them. We have a system that attacks production networks and it's highly successful breaking in. Well, is it safe? Well, our guys instinctively knew we gotta have obviously a secure, you know, we gotta have a hypervisor. We gotta secure this thing. We gotta lock it down, host-based. We have to have a proxy. It knows the proxy. It's proxy aware, that's fine. But then our guys did something. And even I was like, nice job. They passively, surreptitiously, look at every single prompt on, do we like it, do we not like it?

And the majority of the time, if we kill an agent, it's probably nothing to do with safety. It's that the agent's wasting money. You know what I mean? So kill it. It's off on a goose chase we've already done or don't wanna do. But there were so many layers of validation that the agent was doing the right thing. And the other thing was, assume every layer of your security will fail and you have to have deterministic rules that eliminate certain activities. But what I did learn reading those incidents, it does take domain.

in expertise to secure agents behaving in certain domains. Yeah, I mean, yeah, it's a great so I get that. So like, like, without a cyber background, I get how you're going to make, you're going to test something go up, didn't think of that. Yes. And you would have had to have an experienced team look at what the the vowels look like to say, you know what, it's going to do this, and it's going to do that. So you got that's why Armaden, we combine the exploit developer types, and red teamers with the AI folks, because our evals most of the time are made by the red teamers, right?

You know what I mean? They're the ones that understand this stuff. And we created 24 kill chains that Armaden, that humans have done in the real world, period, at different victim sites, and our experienced operators have done when testing networks, and we had no model go through the entire kill chains of more than eight. So that's where it is. Eight out of 20. Yeah. And here's what's weird, by the way, we tested the open weight ones, and the most advanced closed models, they all found eight.

Yeah, really? So if you're Yeah, so it was all about just speed and cost. Yeah. And there, you know, the closed models were faster to finding exploitable risk, but that's coming down. But we kind of let the open models run longer. And they got to the same place. That's so in the cyber world, the differentiation between closed and open is not as great as in other domains. And it's compressed. Yeah, yeah, that's somewhat consistent. In terms of like the capability gap, at least closing a little bit.

But that's interesting that their performance is basically the same. From my perspective, seeing the charts from the team, all the lines ended up in the same place. You know, when you're looking at it was immediately time, cost, and then call it effectiveness or creativity. They all ended up at the end of all their operations, where they hit diminishing returns, they ended up in the same place. Not on cost, though. Yeah, not on cost. Yeah, that makes sense. So it's a it's a decent segue, maybe to talk about what kind of models you guys are using.

And then what role do you think the lab companies play in the future? Well, and I don't even know if I finished answering your last question, other than domain expertise on security is gonna have to work with the AI folks, because I did read the you know, the the meter publication. And I was like, well, these guys are AI people, but I'm not sure they've done a lot. They don't Yeah, yeah. And that's gonna happen. Again, the modality shift, because here's one is when cloud started emerging.

You know, I was running a bunch of incidents, we were responding to breaches for living at Mandiant. And we had to learn what's a cloud breach look like, right? We now have to learn what's an AI breach look like how much data is that entropic or the model companies that are being leveraged to do the attacks? What do you wish they logged and how they will change behavior as well to have better audit trails, better forensic capability. So it's early onset to the technology. And we all clearly have to mature into it in a way where we have the accountability.

When these things go rogue, you can say here's what happened and when and it shouldn't be like two weeks of forensics to figure it out. I hate to say it. Like we log every single thing our agent does source IP address, time and date and what it did, you know, so you got to go backwards and replay these things. And I think they've learned lessons the hard way. And they're probably way better today than they were even three months ago at open AI and entropic and testing these things.

And whoever had a regular labs are looking at this going, okay, we got to tighten up a little bit here. And I get they were surprised because they underestimated it. And we've and we've done the same with human adversaries. It is the weirdest thing my whole career. Everybody underestimates the top tier of what you're up against, right? Because you don't have to see it every day. Yeah, you know, and you don't want to fear the boogeyman, as they say under the bed, so you don't want to have FUD.

But you still want to respect the technologies that you're creating and test them in a way that is meaningfully guarded. Got to cage the beast, David. And I would argue, cage it too much, release a little bit, find the line, because if you do overly deterministic, here's the art form to safety running AI, at least in cyber on offense, is you want to leverage the intelligence of the frontier labs to do stuff, but not do the wrong stuff, right? So you need classifiers, you need a model that looks at everything going outbound, everything coming inbound, we've created that with people and humans going thumbs up, thumbs down, that's good, that's bad.

You got to train it, classify and look at it. But if you get too deterministic and disallow too much, you're probably not leveraging the creativity of it. You let it out some and so it is a gray area. And that's the problem with that gray area is that's why no one would ever say we're 100% certain we're going to get what we expect. You know, period, it's it's it's a battle we even have. But we have yet to have an issue because we can put a human in the loop or we have classifiers to say human needs to decide this.

We don't know what the hell just happened. You know what I mean? Yeah. So if you're inspecting everything that's come, you know, I would prompt to the labs, everything coming back and something comes back. And we don't know what the hell it is. Pause, escalate, judge. Yeah. What do those great security operators look like inside Arminan? A lot of experience 15 years on offense, 15 years of red teaming. I think we've read team literally 99 of the fortune 100 throughout our careers. Wow, we didn't get one and I know who it is.

And they've never hired me. And I love their products. So one day, maybe I'll get how do we get it? Let's go get those guys. Talk to them all the time. Never landed. That's all right. They might be good. But uh, you know, they are very experienced. And they all, you know, when I look at our 90 plus zero days, the majority have been found by human, right? But they're found by humans, because we're leveraging AI to do 90 plus percent of the tedial work, which is pen testing, done, automated, web app, pen testing and creative articulate way done automated for the 98 99 percentile.

But we're still putting humans on it. But here's where it gets interesting, David, the last few zero days, tech found it. Really? Yeah. Oh, wow. So we've made the turn. Yeah. You know, and and most people already have made the turn their tech. If you're on offense, leveraging AI, your AI agents are finding zero days. Yeah. I want to shift gears now to your philosophies and mindset in building a company. You know, they're different. Yeah. So couple things there. Like the first time I built a company was 2004.

And I wouldn't have said I was an entrepreneur. I started man yet no for February. And it was self funded and profitable. And we were successful because in hindsight, it's like you almost learned nothing at the time. And you look back and go, Oh, I did learn right then and there, because of the pain usually. But I met, you know, I look back on Mandiant. Now, we had a premise nobody actually believed in no for because our first website said security breaches are inevitable. And nobody believed it.

And I don't even know how much I believed it. And it's a pretty good by the way, I'm slightly off our first headline was, you cannot solely rely on preventive measures. And that was so boring. That's same as security breaches are inevitable. Yeah, you know, can't rely on better ring on this. Yeah, I got it wrong, because I'm not a marketing guy. But anyway, so security breaches are inevitable. And the premise was that let's respond to every breach that matters. So we have first mover intelligence on how to prevent it happening again.

Yeah. And so the first model of all cybersecurity was antivirus. You know, it was like we look for malware. Yeah, we have signatures for it. And if we miss, David George has to find the malware and submit it to us so we get better. And that's a bad model. My mother's not finding malware on her laptop, right? I mean, it's just gonna eat her laptop a lot. So that model was bad. So we decided a better model because I responded to breaches. And the reason I was responding to them is a V was easily evaded.

And so we were like, well, let's learn all the you know, let's second layer AV because it stinks. And that's what George now, you know, so let's second layer AV, you still have to have a V, though I beat it up. But the reality is, is you still need it, or something that replaces it. So the second layer of defense was required. So if he was imagining a line to here, then you extend imagine a line with something that can learn and think. And so we wanted to do that.

And I actually look at arm and it's just the third wave of Intel, like, why are we waiting for a victim and learn from that? That's ridiculous. You've got to find your own problems. First, don't wait for you know, defense contractor A to be compromised and then quickly share the information to make sure it never happens again. And that model still needs to exist for the things that are somehow get there that beat you. But um, we got that model now and it's just not good enough.

So back to your question, you know, mandate was self funded. There's not a I don't know self funded companies these days, David. Well, the speed, the speed requires that that you got to be fast. That's the difference. Yeah, you look at we started arm it in the philosophies were different. When I started mandiant, it was Hey, this is what we do for a living. Let's make enough money to do it for a living. That's it. I mean, so we hired the best people and we had a philosophy of, we're gonna pay you more than our competition.

But you're gonna work harder. Yeah. So we always felt like we had less people working harder, but better people. Yeah, I think we had a we were known for having great talent and that talent has prevailed. There was a time about a year ago, somebody sent me a text and they said, congratulations, 43% of the RSA mainstage keynotes are mandian alumni. Come on. And that's the text I got. I never verified it. But the guy's pretty accurate. I'll take that stat if that's true, you know, and I think it probably was, you know, we've got a lot of reach over time, with a lot of talent.

And I mean, look at found some look at George Kurtz. We he and I worked together at found some 2000. You know, he has spawned a lot of from found some you've spawned a lot of successful companies and, and people. So same thing with mandian. But the differences are this a got to be funded. Yep. B. Oh, your growth rate, get the market now like I look at arm and his opportunity. I feel like we have an 80 mile an hour tailwind. But we don't have a sales force.

We don't have a go to market. We're not international. All that just has to happen. And the only way to thread the needle in today's economy, to beat the bigs is get this go to market in place with exactly the right tech at the right time. And you better already have your act to read ready, and you're at three behind that ready, because you don't want to get boxed into a corner. So Armandine's got an act to we have our act three planned. But we're in the process of building go to market, which requires the funding, and you have to build it ahead of time.

No, the consumer AI companies created such meteoric rises in revenue. I don't think they can be replicated in enterprise security sales, right? But you just saw the compression whiz got to over 100 million in AR and 18 months from their first release, right? We're gonna try to beat that. And that's what you have to do, especially when you're needed, necessary, and you have to exist. And that's not easy to do. So you got to get the funding, you got to constantly think, how do you rapidly grow?

You can't let the wheels on the bus get wobbly, meaning how do you go that fast and maintain process? Yeah, at Mandian's pace of we were self funded and profitable, with no competition, because nobody believed the premise. We didn't get that wobbly. You know, yeah, we just had great leadership and discipline and we could do it. Growing this fast, you have to hire scalable leaders right away that understand institutionalized process, you can't win with grit, gut and moxie, you know, you actually have to proceduralize, almost industrialize the arm and in way.

Yes, that you know what I mean? And that's what I'm trying to figure out. It's how do we do that? And feel comfortable doing it? And here's complications. So let's do it. You got to grow fast being the AI age. It used to be development was at a speed where you train sales at sales kickoff in January, and you're good. Yeah, exactly. Yeah. So I'm trying to figure out, wait a minute, we're different every week. Yeah, what is the modality now of having a sales core that is right up to date?

And here's the challenge that I thought every, every CEO I've talked to is like, how does AI change our business? And we all sort that out. But how does it change our manpower? When I look at AI's influence on sales, the reality in enterprise security sales, people still buy from people. Yes, you still need the same damn go to market structure for now. And in fact, it's even more important because the tech's changing so fast, you can't put that onus on the customer to figure out how did you change?

Where's it at? So we have got to create a process institutionalized, where sales is trained every week, where we are, how are we doing? And these are processes that will get you to win. You know, you get carried with the credibility. Yeah, exactly. Yeah, you got to be great. Oh, yeah. And by the way, there is no replacement for any company I'm involved in. We are not ever aiming to be let's be number two in our space. That sounds fun. It is be the best in the world at what you do.

And when we hire people, I remember getting a question when somebody asked, Well, how do you know when you're the best? And I'm like, when you are the best, you know it. You know, you have to be the best. If you have to ask the question, you got like, I'm pretty sure, you know, there was a time in, you know, LeBron James's career where he knew I'm gonna have to work harder to stay the best. Yes. Right. And that's how I want us to feel at Armada.

And the sports analogies all work. Tom Brady never walked on the field going, well, I'm the second best quarterback out here. No, always. But it requires harder work, better people. And you have to constantly test. Are we the best? And you learn very quickly from your customers if you got to do better, right? So that feedback loop customer straight in to the engineers is critical as well. So long story short, speed has changed, got to do a capital raise, got to scale processes, and test those processes all the time.

Yeah, what I can't stand is chaos. A CEO's job is to absolutely like hide chaos at a company from the employees, right period, you got to make sure they're not like we're like so loose, we have no idea wrong. You have to just say here's the process. If process is too much for you to study. That's the person you go to. All you need to know is a guy or person's name. And that's how I look at it. How do we grow rapidly without feeling chaotic?

How do we grow rapidly earning it with better product? And how do we change fast? I don't know how long IP lasts. So it's like you got to build a Ferrari engine and say, you know what, whatever we're doing today, someone else is doing in six months. Yeah, yeah. So how do you differentiate over the next six months is go to market, get customers and make them happy. You got it. The brand itself has to be built to you have to become a brand. That is the seal of approval.

Yep. So I just gave you a jumbled answer. I wish I could summarize really quick. So here's the four differences, the funding, the speed, the branding matters to go to market build up, you have to build it way faster today. than you had two years ago. Well, and the big reason why that's the case is because this is a tsunami like has never been seen before in security, in the security market, right? CrowdStrike, you know, and then the other players that were around it, they redefined the category.

They had to create the category. In the case of Wiz, part of the reason that it was able to grow so fast is because it was a, oh, bam, hit you like a ton of bricks, pressing need. And so everyone felt like they needed to buy this or something. For you, it's like everyone. I think they innovated really fast too, though. You get a halo early, like, you know, you gotta get the halo. And I personally think you get the halo, trade secrets. You get the halo by getting the right customers and making them ecstatic.

Yes. You know, like, no offense if there's a place called, I don't know, Susie's Cupcakes, they don't get you the halo if you make them happy in cybersecurity. Right. But the money center banks do, the best retail does, the airlines do, you can get it. Yeah, of course. And so that's why Armin is making 1A Enterprise focus number one. Absolutely. Solve the hardest problems. I think Wizz did that, so. Yeah, they did a great job of it, one of our companies too. Yep. And we love those guys.

All right, what else do you want to talk about, Armin? Well, I never answered your question on the differences too. There's more founders today than ever before. Yes. There's more startups than ever before, and they're all able to capitalize now. So you will have competition in anything you choose to do right now. There's not gonna be a mandate. Security breaches are inevitable, and it's right, and there's nobody else there. Yep. That's not gonna happen right now. Everyone's in a crowded market, so I think every founder has to recognize you have to differentiate, and probably right now, because of the noise in marketing more than ever before, the only way to differentiate is get customer, make customer happy, and repeat.

Yep. There is nothing else that'll differentiate you other than your customer base raving about you. Yep. So you better go do that. That's it. So I just gave away every trade secret I've got, but none of it's rocket science. It's like literally, honestly, it's the same thing that's always been in business. Yeah, but it's just at hyper speed now. You just gotta do it faster, and that speed requires never forget what you should focus on. Get customer, make customer happy, repeat. Yep. That's it, and you gotta do that.

And then everything else is like around that is to make sure you do that really well. The training, the sales enablement, the marketing, the people you're hiring, your hiring process, the teamwork, you know? So, and some of the ways we differentiate as well, well, you know, we have all our engineers in one room. Yeah, we're a big believer in that, you know? I think managing distributed teams is more complex than standing up and asking questions, and 20 people are in the room to answer them. Slow speed, if nothing else.

Yeah. So, it's a great time to start a company, though, because almost every, well, every industry's gonna change. Yes. And in cybersecurity, every single tech stack is gonna be different over the next two years. Yes. You know, people are gonna get ripped out, put in new tech. It's all gotta be revamped, and so it's fun and exciting to be part of that. It's the tailwind of a lifetime in cyber. Yep. Well, look, we're so excited about what you're building and thrilled to be your partners. So, thanks for being here.

Oh, it was fun. Thanks for listening to this episode of the A16Z podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts, and Spotify. Follow us on X, A16Z, and subscribe to our Substack at a16z.substack.com. Thanks again for listening, and I'll see you in the next episode. As a reminder, the content here is for informational purposes only. Should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund.

Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see a16z.com forward slash disclosures.

Go deeper

• All A16Z episode guides