1% better

How Real-Time AI Video Is Changing How Creators Work

Move from “prompt and hope” to a controlled creative loop: storyboard one short scene today, create a rough blocking pass in Blender or with reference images, then generate several AI-video variations from that reference. Specify the camera movement, lighting, character entrance, and motion you need

39m
A16Z

Key Takeaway

Move from “prompt and hope” to a controlled creative loop: storyboard one short scene today, create a rough blocking pass in Blender or with reference images, then generate several AI-video variations from that reference. Specify the camera movement, lighting, character entrance, and motion you need rather than relying on a single text prompt. Treat AI video as a fast iteration layer on top of your creative direction—not as a replacement for it.

Episode Overview

Jennifer Li speaks with FAL co-founder Gorka Mirdzevin and head of engineering Batuan Tashkaya about H3Max, a post-trained generative-video model designed for dramatically lower latency and cost. They explore how real-time generation enables continuous, interactive video, while new controls for references, motion, lip sync, lighting, and camera position make the technology more useful for creators and professional studios.

Key Insights

Speed creates entirely new creative formats

Generating a five-second clip in roughly 1.5 seconds changes video AI from an offline rendering tool into an interactive medium. The team argues that real-time speed enables live streams, audience-controlled scenes, and voice-directed video experiences that were impractical with slower models.

Model and systems optimization compound

The reported performance gains did not come from one improvement. FAL combined post-training that reduces diffusion steps with kernel optimization, hardware utilization improvements, prompt-expansion optimization, decoding improvements, and newer hardware.

Control matters more than raw generation for professionals

Professional creators need reliable control over the components of a shot, not merely impressive outputs from a text prompt. References, camera paths, lighting, lip synchronization, and motion transfer move AI video closer to a production workflow where creators can repeatedly obtain the intended result.

Use conventional tools for structure, AI for polish and iteration

The Blender workflow illustrates a pragmatic division of labor: build the composition and movement with a controllable 3D scene, then use an AI-video model to transform that reference into a polished result. This can provide near-complete control while preserving the speed to test many options in parallel.

Open weights can accelerate specialized creative tooling

Because H3 was open source, FAL could post-train and optimize it for specific capabilities. The speakers see this as a way to close the gap between what general-purpose model labs build and the precise point solutions studios and creators require.

Frameworks or Models

Reference-Driven AI Video Workflow

1. Build a rough scene with traditional tools such as Blender, including the desired composition and motion. 2. Render a low-resolution version of that scene. 3. Supply the render as a reference to an AI-video model. 4. Generate and iterate on polished outputs while retaining the underlying scene structure and control.

Structured Camera Controls

1. Define the desired camera position or angle at the start of the clip. 2. Specify additional camera positions or angles at later timestamps in a structured JSON input. 3. Condition the model on this camera path as the source of truth. 4. Generate the video with the camera following the prescribed movement rather than inferring its own path.

Notable Quotes

"Generative media is, I would say, along with the coding agent market, what we call is token market fit."

— Gorka Mirdzevin

"We have a version called H3 Max Turbo that's public that can generate a five-second video in 1.5 seconds, which is insane."

— Gorka Mirdzevin

"Let's fix the speed and let's try to push for quality and controllability of these models."

— Gorka Mirdzevin

"You essentially supply a structured description of where your camera needs to be at any point in time and then the model is like perfectly conditioned to regard it as like the only source of truth and it doesn't like hallucinate where the camera should go."

— Batuan Tashkaya

"These small point solutions that actually make them more efficient push out more video and AI can actually close that gap very nicely."

— Gorka Mirdzevin

Action Items

  • 1
    Build one reference-driven test scene

    Choose a five-second scene you want to make. Create a simple reference image or rough scene layout, then generate multiple video variations from it instead of starting from a text-only prompt.

  • 2
    Write direction as production constraints

    Before generating, list the non-negotiables: character identity, action, shot framing, camera movement, lighting, and timing. Use these constraints in the tool inputs available rather than evaluating only on visual novelty.

  • 3
    Run a fast variation sprint

    Generate several versions of the same shot with one variable changed per version—such as camera angle, lighting, or character motion. Compare outputs against your intended result and retain the best direction for the next iteration.

  • 4
    Prototype an interactive video concept

    If you create content, design a simple live or social format in which viewers choose the next scene action. Start with limited, pre-approved choices so audience participation adds direction without making the experience unmanageable.

Full Transcript

Transcript of How Real-Time AI Video Is Changing How Creators Work from A16Z. Auto-generated from episode audio; may contain minor errors.

Generative media is along with the coding agent market, what we call is token market fit. Everyone's waiting for a large consumer moment in AI. I believe H3Max makes it possible. Were you surprised by the speed up and the gain you could get from post-training this model? We have a version called H3Max Turbo that's public that can generate like a 5 second video in like 1.5 seconds. From a cost standpoint it's also like 2x less. People are starting creating these beautiful scenes using an LLM model GPT-Astra in Blender.

And all of a sudden it unlocked the whole new workflow for Hollywood and professional people. We have been very, very focused towards speed, performance, quality. And now we have a really good base model. The next month or two is going to be fully focused on. What happens when AI video becomes fast enough to generate in real time? A16Z general partner Jennifer Li sits down with FAL co-founder Gorka Mirdzevin and head of engineering Batuan Tashkaya to discuss H3Max and the rapidly changing generative video stack. They unpack how post-training and systems optimization made video generation significantly faster, opening up new experiences where video can run continuously, remember previous scenes, and respond to direction as it plays.

But speed is only part of the story. They also discuss the push toward greater control over camera angles, lighting, characters, and motion, and why those tools could make generative video more useful for professional creative workflows. Welcome Gorka and Batuan to our podcast again. We did the last one last year. This is long overdue. And we have such an exciting model to talk about, which is FAL's H3Max. The day when it came out, I was calling it, it's really in the league of its own. It's so funny to see the benchmarks where you have the dot of this model on the far left or far right, and then everything else is on the other half.

And that graph is actually log scale. It's actually further, but we had to fit it in. We had to do log scale. That is hilarious. The time portion, the quality is not. For sure. The internet noticed, for sure. There are so many viral tweets about it, like people really played around with this model. Maybe just give us the backstory of what inspired you to post train this open weight model from Minimax. And how did you get the quality and speed to where it is? First of all, the Minimax H3 model is the first truly open source, very capable, like latest generation video model out there.

So even though we work with some of the other model labs to run inference for them, we never had this capability, like had the right to add this capability on top of it. So Minimax came up with their very capable open source model that is truly last generation, can take references, like very familiar architecture to any other video model. We thought this is a great opportunity to go all in and see what we can do. And again, we did like many different things that we are going to talk about that combined gave the results that you show on the graphs.

But the biggest reason why everything came together for this particular moment was because H3 was the first truly next generation video model that's open source. What is the idea, given like thought has been known to be like a general media inference serving platform, like what is the idea to get into post training open weight model? Like you talk quite a bit about it in the blog of like combining the system work with the model itself. Maybe talk more about the work behind that. Generative media is, I would say, along with the coding agent market, what we call is token market fit.

And the way we define it is as can a single person productively spend a lot of tokens. And the amount is like 10k a month, something like that. So there is incredible amount of demand in the market to generate video, to generate many things at the same time. A person who is doing this for their daily job, they spend in front of a computer and do this all day long. And they spend thousands of dollars, lots of tokens. And since around April, the whole industry and file itself, we've been compute constant.

We are growing as much as we are adding compute. Like there are things we do here and there, but the whole industry has been compute constraint. And we've always been looking for efficiencies where we can relieve that a little bit so people can use this more. So that has been the idea behind everything we've been doing since April. And this just came at the right time because this makes everything maybe an order of magnitude more efficient. So it gives more compute for other models or even like more tokens can be generated using H3Max.

I think Botuan would agree on that. Yeah. Like just from a system wide optimizations, which is what we have been doing for the past three or four years, you can maybe make the model 2x, 3x faster while producing the same quality. Because it's at the end of the day, same model, same architecture. You have the same constraints. You're just trying to optimize what you can get out of the chip itself. And there is a roof line there. And we have been approaching that roof line more and more, especially lately because our entire team has been focusing on how do we get out more video pixels from a single chip as much as possible.

And this new set of post-training related optimizations with system slash model co-design enables us to go beyond that roof line by an order of magnitude. And we just felt the pressure. We have been working on it on top of open source image models before the video models. We did one version with ideogram. We did one version with flux. So we have been experimenting with how can we build post-training infrastructure to take an existing model, build kernels and systems design around it to run it very, very fast for a specialized version that can beat anything else that we would get just by running the model itself.

And a combination of that plus just getting a frontier video model on our hands and all this expertise, we were able to go by an order of magnitude in terms of speed. Incredible. Let's dig into that. I may get some of the numbers wrong, but. There's like efficiency numbers, there's cost numbers, there's speed up numbers. Right. Not everything means efficiency, but it all adds up to be very efficient. Yeah, I guess what is stunning to me is there is like a magnitude lower cost and also much faster.

I think it was like 35x speed up. Yeah, compared to the original Minimax H3 endpoint. Well, at the Elo score, you didn't really sacrifice quality. So yeah, just reveal a little more of the secret sauce behind. Is this more of the type of system work you have done? Like, did you have to do like model architecture change? Is it system work that really brought down the cost and latency? And how about like the next generation of chips like GB200 fits into the whole story? It's just a compounding effect of like multiple different optimization variables that we have been targeting.

The first one is obviously, OK, you go from like the base model to a model that's like post-trained to be like more efficient. For diffusion models, this is just essentially how do you go from running 50 steps to running something like 20 steps, right? Like you're just trying to optimize that pipeline. But as soon as you go from 50 steps to 20 steps, you lose quality. So you need to target in the optimization scene. OK, I want to improve the quality and then I want to apply the optimization.

So we have like checkpoints of this that are significantly higher quality, but obviously slower. So what we initially did was, OK, let's run our post-training NRL pipelines so that we can improve the model's quality and then apply the optimization stack on top of it so that the end result gets you to the same quality or like even like higher quality than the original model. But at the same time, you're like an order of magnitude faster. So most of the gains come from post-training this model to be like compatible that you can run this on like less amount of steps.

But on top of that, you add like all the kernels and systems engineering work that you do that brings your like hardware utilization from 30, 40 percent, which is like standard in like many inference workloads to like 70, 80 percent. And 70, 80 percent on theoretical MFU, which is like impossible to reach. So you're essentially at the roof line of what you can get out. And then these models are not just like a single or you just give a prompt and you get a video back.

They're actually pipelines underneath. You need to take a prompt. You need to like run an LLM, like a very large LLM to go expand that prompt to a format that the model was initially trained that generate the video in the latent space and then decode those latency back into pixels. And then depending on the workload, there might be an upscaling component involved. So there's like multiple components and every single component by default is unoptimized. There's still like lots to be gained there. And like we just looked at it from a perspective of we are going to get the maximum out of every single component.

This made us go around LLMs at super high speeds, right? There is that component. But for a different workload, this is not like something like an agent decoding LLM workload, where you have very high cache rates, where you have higher sessions. It's a single shot. Give a prompt, you get a prompt back and there's no caching. You're operating at low batch sizes. So there's a completely different set of optimizations on the prompt expansion side, completely different set of optimizations on the diffusion model, completely different set of optimizations on the VAE that you take from latency to pixels.

And you just combine all of these to have an effect that compounds. From a hardware standpoint, going from something like hoppers to black walls, you see something like 2 to 3x improvement by itself. But from a cost standpoint, it's pretty comparable because the cost is also in that league. So I would say it only reduces your wall clock time, but not just the efficiency itself. But it obviously helps if you want to go significantly beyond real time. If you want to generate five seconds of video in less than three, two seconds, then you need some of these latest generation hardware today to unlock that possibility.

Maybe this is a detail of a question. Is the model being served on a single GPU? It is. The majority of the video models today run in a single node configuration, which is eight GPUs, because once you start scaling beyond eight GPUs, the efficiency gets less and less because of the communication overhead. And existing Minimax H3 endpoints, as well as other video models, are probably getting served that single node configuration. Same with this. It's running in parallel across eight GPUs. And do you think there will be more efficiency gains in there?

That you can either optimize more of the steps in between by narrowing down the user experiences, let's say the different types of inputs and outputs? Or as you're thinking of parallelism, is there more to squeeze? We released a turbo version of H3 Max. The initial idea was calling this H3 Turbo. And we were like, we don't want to call this Turbo because the quality is better than the original one. This needs to signify how good of an achievement it is. So we released H3 Max, but a week later, our team was like, we can run this 2x faster at 97th percentile of quality.

We run evals, they're almost the same. There's a small noticeable loss in quality, but we have a version called H3 Max Turbo that's public that can generate a five-second video in 1.5 seconds, which is insane. And that's also 2x, from a cost standpoint, 2x less. So it depends on how okay you are with losing quality. You can go down. And today, these models are so cheap and so fast that I don't think people need any faster or any cheaper. It's already at a point where, from a cost standpoint, compared to the Frontier itself, it's an order of magnitude cheaper.

Compared from a speed perspective, it's more than an order of magnitude faster. And you just enable all the experiences. I think we would need to see what else people would need. But my bet today is we just need to improve quality more than the speed at these speeds. Let's fix the speed and let's try to push for quality and controllability of these models, which is what we have been pushing in the past two or three weeks. I think controllability is key. When we first did it, we did text-to-video and then image-to-video.

And then references came later, which adds a ton of controllability. And it's basically the default mode, how people use these models these days, references. And then we are now adding different Loras, fine-tunes of the base model as well. We are working on a lip-syncing version. We are working on a different camera angle Lora, different style Loras. So, again, open source adds a whole ecosystem around the model and it really, really helps. Were you surprised by the speed up and the gain you could get from post-training this model?

I saw it as a little bit of a surprise. One day, I think it was a Saturday, you launched the model. And the Sunday, people put it on Twitch. It become a real-time model. That's the interesting part of when you release. We did evals. We spent a ton of money doing evals on our own. I don't know, tens of thousands of dollars even. And the results were unbelievable. And then the plan was to just release the model without doing external evals. And then, okay, we decided, let's hold off.

Let's not tell people that this is so much faster and so much better before we have some external validation. So, we waited three, four days to all these other eval platforms to actually run the evals. So, we matched the results that we have externally as well. And that's how we launched it. Because, as you said, the results were a little too good to be true. And it was. I guess, were you taken by surprise that the real-time use case that came out of it? Or what are some examples that you think this model has unlocked of the experiences that prior models couldn't?

This happens once in every couple of months where the whole company gets hold of something and the creativity just explodes and everyone is just working on a new little app or a different optimization, Laura, whatever it might. The whole company gathered around this model and some front-end engineers started working on interesting applications. We can talk about our world model accelerator team, which is brand new. They started working on the live experience. The WebRTC live experience. So, there were five, six different parallel little projects within the company.

I think we broke a record on Slack that day. How many messages were sent. We have a distributed team, we have people all around the world, mostly in San Francisco, but it's incredible when you see the 24-hour development, when people work 16, 17 hours and then someone else wakes up and picks up that, and that went on for three, four days and that's when we released all these projects. Take me into that. It's so interesting, because you imagine a model or product launch being planned out, having all these eval vendors being ready, lined up, and ship something out, and then you let the world or the external users take it and then experiment and build experiences, put online.

It seems like people internally who are very creative just took this, dropped everything they were doing, launched an experience that got really popular on Twitter. Do you want to tell us about that one? Yeah, of course. One of our engineers, Rehan, just by himself, completely, started streaming a live stream of continuous generations of H3 Max from his laptop. His computer. His computer, exactly. He was doing some prompt tricks, trying to keep a coherent story, and then he started live streaming that on Twitch. In parallel, Levels.io, famous Twitter influencer at this point, had a similar idea and he reached out to us that he has a website ready already.

He wants to host the streaming himself and have a website that does infinite streaming. Internally also, we had another team who was working on a continuous version of H3 Max. So H3 Max is like the Rehan's version and Levels.io version were independent clips. It's still very fast, but the clip starts, it ends, and then you take the last frame of the clip, try to put it in the next one, and try to create a continuous. And you put some work into the last frame. There's no memory, like the second clip doesn't really remember anything from the first clip other than the last frame.

Internally, the ML team was working on a version where the transition is more seamless. There's like two minutes of memory. So you're in a scene and when you direct the model or someone else enters the room, it's actually like everyone looks at that person entering and the scene is continuous. So internally, we were working on that. And then another team was working on an experience we called Fall Live for the continuous version. So we had three parallel efforts going on that were all independently going viral on Twitter.

And these were all like spontaneous, like you didn't plan for it at all. You didn't plan for any of them. Exactly. And they just became products and experiences in the following days. But Tom, let's talk about how we made the model more continuous. That was very surprising to me because I've never seen that actually work on a video model before. Yeah. So going back, we have been very, very focused towards role models and essentially like action controlled or action driven real time continuous streams of video.

And the problem until something like H3Max was, quality was not good enough at all. It was just like, you know, it degraded a lot. It didn't remember the past before. But we built the infrastructure. We built the infrastructure that we can go stream video, have people control it in real time, being able to like multiplex it to multiple people, very low latency. And at the same time, our ML team was essentially trying to take every single video model and try to apply this set of optimizations and tricks to, OK, how can we make this generate instead of a five second video, 15 seconds, 30 second video.

But you are always like below the real time factor where, you know, you were always like, you know, you like you never could generate like five seconds under five seconds. Once H3Max unlocked it, the ML team was like, this is insane, which are like separate teams internally. We have a research team. We have an inference team. We have an ML team. They're like, they saw this. And like, this is insane. We can apply all these like set of learnings that we had in previous models where we attempted to do this, where instead of trying to generate a five second chunk, let's try to generate, you know, like a 50, like 10 second video.

And then the five seconds from previous one is still attended. We still remember it. And like as the video goes up, we can like extend that memory up to two minutes and you need to do extremely clever optimizations because attending to a two minute video is just extremely, extremely compute intensive and just like it goes up exponentially from like a compute standpoint. So like we did like lots of optimizations there, but at the end of the state, we were able to, OK, we can remember back to two minutes, which is generally good enough from a memory perspective.

And then obviously with like prompt tricks, you can still continuously remember more finer grain details above the two minute mark and you can essentially stream infinitely. We kept it at an hour from that perspective. And then that team just like released that model under H3 Max Director, which is public for people to use. And I think it's the only model that can generate like, you know, up to 60 minutes continuous videos that is action controlled. You can like, you know, start with a prompt, say like there's like an office setting and someone is like, you know, working and then like 30 seconds later, it just imagines by itself.

30 seconds later, you can like say a woman walks in through the door, like it can take the prompt and reflect it immediately, which is the most fun part. And the office is still the same office. Same office. The camera can pan back to the original person and the original person is still there in the same state. Yeah. So, you know, we released that and it got like we did this like fall live website to just like demonstrate it because it's like people need to see how cool this is, right?

This is a new technology. I don't think people are like really aware. And it got also like very viral immediately because we also let people vote on what the next action is. It was like, you know, like a form of crowd source, like the chat was controlling whatever was happening, which is fun. But obviously, you know, we limited on like the options and then they could pick, oh, like a banana enters the office instead of a moon. And it's like more fun. And then people start like, you know, having this and we start adding more channels and like every channel had a concept.

There's like a channel where it's like full chaos. There's a channel where it's like cartoons from like 80s. And like the model is like extremely capable and it just like remembers like so many different concepts. And I was like, it has like a big, big memory from from like styles and like, you know, concepts perspective. So it just became like a very fun experience underneath. Again, like there are so many really incredible experiences coming out of this, like H3Max director was just another huge surprise to me.

It's like I found it interesting in the in the media market that you it's not like, you know, like language model, you have like this linear graph of like just continuously compounding on like, you know, intelligence capability and so on, like feels like in the field you're operating in. It's always like a few months of like sort of quiet time, but like a lot of things are bubbling. But like in a very short period of time, like everything bursts, like all these things come in combination, come together of like the base model being good enough, like you can get the latency down to the point where you can like references.

Yeah. Yeah. Like get the real time experience, but also like apply controllability on top of that real time experience. Like this just opens so many, you know, opportunities of like live experiences where like end user can control what's happening on the screen, which is incredible. Like we have imagined a lot of these experiences, but never been able to like really play around with it. Maybe just like tell us more about what you're seeing from the market of like how are people using like the director capability?

Like what are you seeing creators are creating that you haven't seen before? And what do you think that unlocks as far as, you know, what people can do with this medium? Yeah. It's been like almost three weeks since we released H3 Max and already it is the most popular video model on the platform, on the file platform by like double, almost like more than double in terms of like volume. So in a lot of other platforms, it's also becoming the default model that people interact with because it's so fast, so cheap.

It just makes sense if you if you come to a platform, this is the experience that you want to see. So in terms of like popularity and volume, it's taking over at least from our vantage point. And for H3 Max director, again, there has been, I don't know, tens of different versions of these live streams. Some of them are still going on and like becoming more and more popular. We are trying to work with some like AI IP holders, people who have like AI shows on Instagram and TikTok and do train a Laura on their style and do a live version of their show.

So we have a couple lined up already. So that's going to be very exciting. And like the way people like if you talk to a creative technologist, prompting with voice has already become something that like they use all the time is like using WhisperFlow or the chat GPT voice mode. And now like you can keep talking to the model and it's like almost as if it's a real director in a real movie set directing like the camera, directing people where to go. You can do that.

And like our creative engineers started using these models like that. So we'll see like a lot of interesting experiences are built as we speak. Very interesting. As in like the video is playing. Video is playing. You are like talking to the video and what's being displayed changes accordingly. That's incredible. And talking about like how the memory piece holds now, like again, this may be a technical detail, like. The capability of remembering what happened in the last scene or in the last couple of minutes of scene, like are you remembering that through like the frames, the images, or is it like through a text?

It essentially, no, it's essentially like it remembers through our video, obviously very, very compressed because you can't attend to the full video. But it essentially knows like most of the details happened in the past two minutes from its own generations. And above the two minute mark, it has like think of it as like it has this evolving system prompt on top of the two minute mark from two to 60 minutes where it knows like the overall structure, overall detail. So it remembers like the last few scenes.

If you think a scene is like 15, 30 seconds, then it remembers like the last four to eight scenes. And then on top of that, there's like a continuously evolving, gradually evolving system prompt that like keeps remembering the overall coherence of the world. Everyone's waiting for a large consumer moment in AI. Now it's like good enough and cheap enough that like a truly novel social AI experience can be built on top of it. Maybe let's talk more about the economic side of this. Like what is the, I guess, one, just like talking about serving cost for like same minutes of video with its three Macs.

And how has it changed your thinking around like your footprint of like inventory of chips? Like how do you want to have like different steps of experiences serving to the end user? But I mentioned this a little bit like everyone talks about how complex the next generation LLMs are. But video models are actually very complex as well because the pipeline has different components and sometimes they require different hardware configuration for efficiency, things like that. So if you were to do this even more efficient, it's called maybe even cheaper.

We would probably run different parts of the pipeline in different types of hardware. Another interesting thing would be to run it on consumer hardware for people to run it in their own machines at home. Like optimizations don't translate 100 percent, but translate somewhat close to that. And then we can do extra work to translate more of it. So doing these optimizations in different types of hardware and combining the pipeline in a way that it's even more efficient. I think that's what we are going to do in the next coming weeks.

Amazing. So you will have people like Rohan that can stream partially of the experience from his computer, but also having like the director and the control plane more living on the cloud. Makes sense. So we talk about all the consumer experiences this model could unlock and it seems like Botuan is happy with all the efficiency squeeze out of the GPUs. Now we're talking more about how do we improve quality and controllability of these models so that the high end of the market, the Hollywood creators, directors, can take this to the next level.

I saw some demos. Coincidentally, this model came out the same week or week prior to Astra. People were combining the Blender experience with H3 Max from Fowl to talk about how it's going to impact the Hollywood world. Using Blender with one of these AI models together is an extremely popular workflow for professional work. Basically, you render a low resolution of your scene, what you want to do using using Blender, previous like non AI technology. And then once you add that video as a reference to an AI model, you basically get close to 100 percent controllability.

And this is an incredibly popular workflow for VFX artists, people who are doing this professionally because they want to get exactly what they put into the model. And as you mentioned, a week after we launched H3 Max, people starting creating, generating these beautiful scenes using an LLM model GPT Astra in Blender and all of a sudden it unlocked a whole new pipeline. plan using an LLM to create a Blender scene and then passing that to the H3 Max model or any video model but it works very well with H3 Max because it's extremely fast and you can like try many things all at once in parallel and that that unlocked a whole new workflow for Hollywood and professional people and it gets you to like close to 100% controllability.

As I said we have been very very focused towards speed performance quality and now we have a really good base model. I think the next month or two is going to be fully focused on okay how much controllability we can add to these models so that professionals at studios, professionals who want to actually produce content that fits their use cases perfectly can leverage these models. The team has been working on an amazing you know like a lip synchronization model where you know you can just supply the audio, you can supply like a video or an image reference and then it can like synchronize the lips perfectly.

Same with like motion controls, you can just take a motion of someone dancing and apply it to like your AI-generated character and it fits perfectly. You can get these results with like basic prompting and you're gonna get like 80%, 90% reliability. What we are targeting is like 99.9% reliability in the outputs so that you can actually trust the model did every single aspect of this generation perfectly and that's like what we've been pushing. One big launch that we had last week was the camera controls which is essentially you can direct where the camera is going within the video perfectly to the degree.

And this is by like describing in the prompt or like generating the... You just essentially like underneath you give a JSON of like I want camera at like 000 at t0. I want camera at like 90 degrees angle at t1. You essentially supply a structured description of where your camera needs to be at any point in time and then the model is like perfectly conditioned to regard it as like the only source of truth and it doesn't like hallucinate where the camera should go. And it's just like you can essentially reconstruct 3D scenes from a single input because the model itself is a very good video model but at the same time you know it's like perfectly adheres to the camera itself.

And this is because the base model itself already has the understanding of the camera angle that you can... It doesn't respect it. You need to tune the model. You need to tune the model to a significant degree and this is like what enables like at large scale you know post-training infrastructure. We now have the infrastructure to take h3 max, add any capability to it. Same applies for any new model. If there's a new video model, we essentially spend most of the time building it as an infrastructure than just like one of training runs so that we can build like services around this for not just like you know open source models but for like frontier closed source models as well because we see in the market this is like the biggest gap.

It's just how controllable these models are. First we start with text-to-video where you put a prompt, you got a video back. It was good but like you never could describe the perfect character for you. And we had image-to-video where you used an image editing model and then generated the first scene and then the model was obviously much more fitting. But you still couldn't say, oh I want this new character to appear at like second tree. You need to put it to the first frame or you can prompt it but it's never perfect.

And then we reference the video where you can provide an initial starting frame and you can also provide, I want these characters with these voices. That's also like a big unlock where you can essentially say this is the voice for this character. And now like you know we are adding, oh within the scene I want camera to look at this degree at like t0. I want camera to look at this degree at like t3. And then we are adding lighting controls where you essentially say where the light is coming from.

These are all compounding on top of each other and we just have the unified infrastructure to just apply this to any model at this point. That's incredible. Hollywood is our fastest growing segment and there's a lot of noise about how AI might disrupt Hollywood but Hollywood usage was non-existent a year ago. And in the past year it grew and now it's the fastest growing segment. Like Amazon, MGM studios in their conference they released their Nara tool. It's mostly backed by file infrastructure behind the scenes and we are seeing incredible incredible pull coming from Hollywood and exactly what they need.

These like small point solutions rather than generating everything from scratch. They want to be able to extend the video a little bit. They want to be able to change the camera controls. They want to change the lighting and someone has to build these solutions for them. What Hollywood needs and what the creators actually need and what the research labs are working on. There's a little bit of a disconnect there and we believe we can come in and do these little post-training projects to close that gap because we work with all the Hollywood studios and we hear from them what they need and these are exactly the things they need.

These small point solutions that actually make them more efficient push out more video and AI can actually close that gap very nicely. Maybe say in a little bit different way like we have been staring at this problem for the last three years as well. We see companies trying to build a movie director video model by either pre-train or post-train on the video side. But what I'm hearing is different people expressing the way they want the output to come out very differently. Consumers talk about it and then write the prompt and generate the results very differently from a Hollywood director, which is obvious.

Professionals want to talk about these camera angles. They want to talk about the lighting. You have built a library or a collection of post-training data and toolkits that can apply these on any model that you can grab the weights on so that they are adapted to a different audience where they can express their creativity in a bit different fashion to control the model when it unlocks a lot of capability underneath. And half the problem was capabilities of these models. We are solving that. The other half of the problem was legal and data residency, things like that.

We made a ton of progress there as well. We now have a system where people can apply with their own IP and we unlock their own IP in the models. We are going to grow that and that's going to be a very powerful thing we do with Hollywood Studios. Also, we now have C-Dance US hosted as well. We already had previously other Chinese models. C-Dance was the missing part. Every Hollywood studio wanted us to have it US hosted. Now that's available and so there are no obstacles in front of these Hollywood Studios.

Now everything is ready and we believe they are going to 10x, 100x their AI usage in the coming months. It's such an exciting world for movie lovers, consumers, people who consume a lot of video and creative content. We have our conference, Generative Media Conference next week. This is the second time we are doing it. Last year, it was mostly consumer AI. There were maybe a couple of Hollywood executives here and there just curious about it and now it's dominated by studios. New AI studios who are like offshoots of the bigger studios trying to do only AI shows, but also the biggest of the Hollywood Studios are also there because now they have big plans integrating AI into their workflows, into their existing systems.

So, you can see the change in the attendance of the conference as well. That's awesome. Well, for the audience, check out the content coming out of the Gen Media Conference. It's going to be very, very exciting and thank you so much, Gorkim and Batuan, for coming on to our show. It's a super exciting time for Gen Media. Thank you. Thanks for listening to this episode of the A16Z podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family.

For more episodes, go to YouTube, Apple Podcasts, and Spotify. Follow us on X at A16Z and subscribe to our Substack at a16z.substack.com. Thanks again for listening, and I'll see you in the next episode. As a reminder, the content here is for informational purposes only. It should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast.

For more details, including a link to our investments, please see A16Z.com forward slash disclosures.