1% better

What It Takes to Build an AI That Actually Knows You

What would it look like if your AI tools already knew you, rather than starting from scratch each time you opened one? Ken Liu, co-founder of First Person...

33m

Summary published by , updated .

A16Z

Key Takeaway

Let your AI tools share context with each other instead of re-explaining yourself every time.

If you use multiple AI assistants, try connecting them to a single personal context store. Tools like Second let you describe your preferences once and pass that profile to whatever agent you open next.

Ken Liu's argument is that most of the friction in AI interactions comes from the gap between what a tool knows and what you've already told a different tool. A shared user model reduces that gap without requiring you to repeat yourself.

Episode Overview

What would it look like if your AI tools already knew you, rather than starting from scratch each time you opened one?

Ken Liu, co-founder of First Person Labs, joins a16z's Martin Casado and Joel de la Garza to discuss Second, a personal context layer that builds a model of who you are and shares it with the AI agents you use. The conversation covers how it works technically, why security is a central design concern, and what Ken expects personal AI to look like as it matures.

  • How Second works. It monitors your digital activity in the background and builds a model of your preferences, priorities, and ongoing commitments that any agent can query.
  • Security architecture. The episode goes into confidential inference, local data storage, and why technical guarantees matter more than policy promises.
  • Proactive assistance. Ken distinguishes between agents that react to notifications and agents that connect dots across weeks of context to surface something you didn't know to ask about.

Confidential inference refers to running AI models inside hardware that can be verified to exclude outside observers.

Main Insights

1. Proactive AI is more useful when it connects dots across time, not just reacts to events

Most agent notifications today are reactive: an email arrives, and several agents ping you about it. Ken describes a more useful layer he calls reflective proactivity.

Reflective agents connect information across a long time horizon. They can notice that something you said months ago relates to work you are doing now, or that a plan you made last week needs updating because conditions have changed.

Joel's example from the episode illustrates this well. His Second monitored a weather forecast in the background and updated him on whether he needed a tent for an event, without being asked. The underlying skill is understanding what matters to you well enough to decide what to surface and when.

2. A shared personal context layer reduces the repetition across AI tools

Every time you open a new AI tool, you lose the context you built in the last one. You re-explain your preferences, your situation, your ongoing projects.

Ken's framing is that a user model should be portable. Rather than belonging to Grok or Claude or Codex, your context lives in one place and those tools query it.

The practical effect he describes: you can open a coding tool you haven't used in weeks and it already knows what you've been working on through Second, without any setup.

A useful question to ask yourself: how much time do you spend re-explaining context that you've already given to some other tool?

3. Technical guarantees around data privacy are more reliable than policy promises

Zero data retention is a promise a provider makes by policy. A company says it won't store your data, and you decide whether to believe it.

Confidential inference works differently. The model runs inside a GPU with a hardware toggle that prevents anyone, including the provider, from seeing what's happening inside. You can verify this technically rather than taking someone's word for it.

Ken's view is that personal data should default to confidential inference rather than standard inference, because the stakes of ambient, always-on context collection are higher than a typical query.

OpenAI began offering a confidential inference option around the time this episode was recorded.

4. Your computer already generates a continuous stream of useful context

Ken points out that operating systems across Apple, Microsoft, and Linux already expose a detailed event stream: which apps you switch to, how long you pause on a website, how you type, what your network is doing.

Most of this signal goes unused because nothing is analyzing it continuously.

Second applies agents to that stream around the clock to build a hierarchical understanding, recognizing that typing in a particular app probably means you're sending a message, or that switching between a browser and a code editor suggests research combined with writing.

The insight for anyone building with AI is that richer context often already exists locally; the challenge is processing and structuring it.

5. Running AI context models 24/7 is expensive now but falling fast

Ken does quick math in the episode: running 10 agents continuously at 60 tokens per second using a top-tier model costs around $1,000 per day. Using a faster, cheaper model brings that to roughly $60 per day.

That makes Second a tool for busy professionals right now, not a general consumer product. But Ken notes that token prices have already dropped more than 50% since First Person Labs started.

There is also a design tradeoff worth understanding: pre-computing deeper insights is expensive but fast to retrieve, while passing raw logs to a model at query time is cheaper but slower to answer.

The cost trajectory suggests the audience for this kind of tool will broaden significantly over a short period.

Frameworks or Models

Reactive vs. reflective proactivity

Ken describes two levels of proactive behavior in AI agents.

  1. Reactive proactivity. An event happens, such as an email arriving, and the agent notifies you. This is easy to implement but adds noise if multiple agents all respond to the same event.
  2. Reflective proactivity. The agent connects information across many events and a long time horizon. It can link something you said weeks ago to work you are doing now and surface that connection without being asked.

Ken's view is that reflective proactivity is where the real value lies, and it requires a durable model of the person rather than just a reaction to the latest event.

Cache time vs. test time tradeoff for personal context

When building a personal AI model, there is a cost tradeoff between two approaches.

  1. Cache time analysis. Continuously analyze your data in the background to produce deep, structured insights about you. This is expensive to run but very fast to query later.
  2. Test time reasoning. Pass raw logs to a model when a question is asked and let it reason through them on the spot. This is cheaper to maintain but takes roughly ten times longer to answer a question.

Ken frames this as a fidelity tradeoff: how well the model understands you versus how much it costs to maintain that understanding.

Notable Quotes

"I think proactivity is straightforward, easy to do. But what's harder is what I call reflective ones, which are things that connect the dots over many events and long time horizon."

Ken Liu

"People are underestimating how much the computer already knows. It's just there, sitting there, waiting to be tapped, so to speak."

Ken Liu

"I think that's one of the biggest bottlenecks right now. People have to ask. You're not supposed to ask. It's super intelligence."

Ken Liu

Action Items

Suggested exercises inspired by the conversation.

  • 1
    Audit how often you repeat context across tools

    Spend one day noticing each time you re-explain something to an AI tool that you've already told another one.

    • Keep a short running list: what did you repeat, and to which tool?
    • At the end of the day, estimate how much time and effort that repetition cost.
    • Consider whether a shared context layer or even a simple personal prompt template would reduce that work.

    The goal is to make the problem concrete before deciding how to solve it.

  • 2
    Try using an AI assistant in audio mode while mobile

    Ken and Martin both described using Grokbot in audio mode and having it query Second while driving or walking.

    • Pick a task you normally handle at a desk: reviewing a to-do list, thinking through a decision, or catching up on a conversation thread.
    • Try handling it through voice while doing something else.
    • Notice whether having persistent context available changes how useful the interaction feels compared to starting fresh.
  • 3
    Map the AI tools you use and what each one knows about you

    Draw a simple list of the AI assistants or agents you use regularly.

    • For each one, note what personal context it holds: your preferences, your projects, your history.
    • Identify which contexts overlap and which are siloed.
    • Ask whether any of those tools offer a way to share or import context from another.

    This gives you a clearer picture of where you are losing continuity and where a portable context layer would help most.

  • 4
    Consider your data privacy assumptions for ambient AI tools

    Before using any tool that monitors your activity continuously, it is worth understanding what guarantees it offers.

    • Find out whether the provider stores your data and under what policy.
    • Ask whether inference happens locally, on a standard cloud, or in a confidential computing environment.
    • Decide what level of trust you need before allowing access to sensitive channels like email, messages, or screen activity.

    Ken's point is that hardware-based guarantees and local processing options exist and are worth asking about.

Go deeper

Full Transcript

Read or copy the complete episode transcript.

People are underestimating how much the computer already knows. You no longer have to explain anything to your coding agents. You can just screenshot anything and just say fix, and then you would know what you're fixing. The question then is like, well, can you just like analyze the heck out of it? What context does CrockBot not have that first person labs would allow? One of the biggest problems right now, like people have to ask, you're not supposed to ask these things. They're super intelligent. But if you have a user model that just builds up by itself, and then these things just keep analyzing the background, then you can just share that over instantly. We focus on essentially creating a model of a person and then put that person in control. When people think about this thing sees everything you do, there's an immediate sort of like, whoa. If you don't trust, let's say us, in principle, you should be able to bring your own tokens to then analyze data that's on your laptop locally. Can you talk through what you think are like the most surprising use cases that you found? For development, it turns out that every AI agent you use is trying to learn who you are. What if that context belonged to you instead? In this episode, Martin Casado and Joel de la Garza sit down with First Person Labs co-founder Ken Liu to introduce Second. Second builds a model of your preferences, priorities, relationships, and digital activity that can move across the AI tools you use. They get into how it works. The security challenges of giving AI access to such personal context and what Ken thinks the future of personal AI may be less about asking agents what to do and more about understanding when you need help in the first place. Thank you, Ken, for joining us. Really happy to have you on to talk about what you've been working on at First Person Labs. Is this the first time you're talking about it? Yes. Yes. So what do you do? So we focus on essentially creating a model of a person and then put that person in control. And the model of a person basically means that understanding their memories, understanding what they do, but more of that, their priorities, preferences, goals, values, and commitments and intent. Sorry, just to make this very simple. I'm a big Grokbot user. So I use Grokbot a lot. Where would First Person Labs fit into my usage of Grokbot? Yeah, the way it works is that you would just bring a context over to Grokbot. It's like portable context. You go to Grokbot and you say, hey, can you talk to my thing to figure out what I want? And then the two of them will start talking and then you can just send prompts like buy me a flight and you would know what flight you're talking about. What context does Grokbot not have that First Person Labs would allow? I was probably asked the opposite question. What does First Person Labs not have? You could build a user model for yourself, right? Let's say your preferences on the seat on your airplane, the preferences of how early you would like to arrive at the airport, and where are you flying? Are you flying first class or normal? And all of that, as of today, you have to explain that to Grokbot. There's multiple rounds of questioning. But if you have a user model or a model view that just builds up by itself and then these things just keep analyzing the background, then you can share that over instantly. So for example, the OpenAI released two new cyber-capable models yesterday, and I haven't been using my codecs in a while, but I have been using Second. So I can just go start using codecs and it has all the other work that I've already done and all the additional context already saved? Yeah. Yeah. You just go to codecs and ask my Second. What's going on? The Second ever be a First Person? Will it ever be a First or will it always go through Agents? That's a good question. I think user model is a process. You have to start somewhere. You start with a base and foundational list of artifacts, what you did, what the people in your life, maybe your preferences. And over time, it starts analyzing the way you talk, the way you present yourself, the way you share information. And over time, you would like to build up even more artifacts on top and more understanding on top. And maybe some of that, you change the model. And so over time, you will go from a portable context into something more of a representation of yourself. One example I like to think of is this Alfred and Batman example, where if, let's say, Batman wants to buy a cheeseburger and get Alfred to do it, Alfred would not reveal, for example, that it's Batman doing it, but he would just buy it. He would state the preference. Alfred being Batman's butler. Oh, yes, yes, yes. Or his agent. Yeah, yeah. Or his agent. Or his agent. Yeah, yeah. I should call my... Alfred. That's a great idea. Alfred. Yeah, yeah, yeah. He would just do it. He would just do it. And then you would trust them because you know that they have a good model of you and what that means when it comes to getting things done. It's interesting, which is, so you described First Person Lab as having this thing called a second. The second understands who you are and has context. And then other agents talk to that second, right? And so it kind of understands your preferences, it understands what you're doing, what you want to do. But I think that that actually doesn't highlight another thing that you've been working very hard on. And I think it's worth talking about is you're actually able to get access to context, digital context that a person has, like messages, chat, kind of email in a way and make that also available. So can you talk a little bit about that? So being one of the earliest First Person Lab users, I would say the thing that struck me was that not that it knew who I was, but more, it had access to all of my channels very well. Which just turns out to be a very hard systems problem. So maybe talk a little bit about that aspect of what you're doing. Yeah, it is indeed a very hard problem, especially to get it secured private. So the way it roughly works is that right now on your computer, there's just an infinite stream of events. As you use a computer, the apps that you switch to, the way you pause on certain websites, the way you type and the way your Wi-Fi is connected. And not the OS already gives you. This is across Apple, Microsoft and even Linux. And the question then is, well, can you just analyze the heck out of it, right? This raw stream, 24-7, apply agents on top, 24-7 agents. So I can first synthesize, for example, well, this piece of typing at this particular app might mean that you're sending a message. And then when you're switching apps between the browser or back to your codecs, you might be doing some research, you might be writing a Google Doc. And you can hierarchically build up the understanding. And that plus existing connectors can give you a very good understanding of what you're trying to do. I think people are underestimating how much the computer already knows. It's just there, sitting there, waiting to be tapped, so to speak. Having been also a very early adopter of the product, lately it's been just indispensable because it's able to catch. It's sending me messages now and letting me know that, hey, you've got this conversation going on over here with this person, and it looks like the situation has changed. So we were planning an event yesterday, there was a 20% chance of weather. And in real time, it was continually updating the forecast for us to let us know if we needed to get a tent or not, which is just incredibly powerful. Did you ask it to do that? No, I did not ask it to do that. This is my experience for my second, too. I'll get a text from my second, and it's something that I actually need to know about relative to some ongoing conversation that I didn't even think about, that it actually is very thoughtful. Yeah. And so it's amazing. I'm just genuinely curious, how are you executing on that? Things that I would have forgotten, it's literally catching. Yeah, yeah. I think it just goes back to the model of the person, understanding what might be important to them and what comes up at this point. So I think proactivity is something that people talk a lot about, like pinging you at a time of need. But there are many levels of proactivity. I would start, for example, the first level, which is called reactive ones. This means there's an email came in, and then there's a meme right online that's, hey, you now get 10 notifications per email because there are nine other personal agents pinging you about that. I think proactivity is straightforward, easy to do. But what's harder is what I call reflective ones, which are things that connect the dots over many events and long time horizon. Stuff that realize, okay, months ago you said this, and recently you're working this towards your goal. This is your priority. These are some stuff on your to-do list. And by the way, you prefer this. And so connecting the dots will be able to give you much better proactivity. And I think that's kind of hints at how maybe in the future personal interactions with computers will go. It would not be focused on reacting to things, but more like, okay, what are some interventions for the user that I can apply to? What are some things that I can do for the user without any asking? I think that's one of the biggest bottlenecks right now. People have to ask. You're not supposed to ask. It's super intelligence. Yeah. I mean, you've built something that just kind of knows what I need at the moment. That's pretty impressive. So I just want to make it very concrete for the people that are listening on what First Person Labs provides. And then I've got a question. So very concretely, it creates this thing called a second. So you create something called a second. Second has access to all of your digital contacts. I mean, it's amazing. It's like key strikes, all your messages, all your channels, all your data. It does this in a very secure way. And based on that, it becomes basically your kind of personal butler, like that Alfred, right? And so then that interacts with your agents. So it's not an agent by itself. So this isn't like user instinct or Grokbot. It's its own or dot. It's its own thing. And then it provides that to the agents. Right. So that's what it is. So here's an interesting thing. So for me, I use Grokbot. I'm a big Grokbot user. And I have a bunch of them. Right. I have one that keeps track of like my synthesizers, one that keeps track of my real estate, one that, you know, God forbid, helps me with my family, which I probably shouldn't need one. But I do. So I just kind of a bunch of these. And then I have one second and they will interface with the second. Do you imagine over time you'll have a second and a third and a fourth and a fifth? Or do you think that it just ends up being your one butler? I think right now, the fact that we have many personal agents is probably transitory. Agents will start to specialize just like people do. Some agents focus on one thing. Some agents focus on, let's say, for example, browsing, Amazon shopping, they're extremely good at because they have the data. They have the connections. Yeah. And some agents will specifically focus on protecting your data and aligning with your interests. So I think that's what second is intended to be. Like this is something that you should be able to trust to have your life's context in it. It's like a vault. So over time, this thing will guide you with your interface with the kind of the software world, so to speak. Even the hardware world. So you think it'll stay as a second? Yeah, it will stay as a second. It will not be a third. It will not be a third. It will not be a fourth. So like you're one trusted agent for all your personal data that actually knows who you are, that will interact with all your other agents. That's second. Yeah. It's Batman safe. Yeah. What you can do is like spin up some agents that, you know, temporarily exist and then go away. But the point is that you want to have a consistent store and the consistent store is what gives you that value. So I am still a paranoid security person and I have been probably one of the faster adopters of some of this stuff. Just to be clear, Joel took forever to be on social media. I was like a few weeks ago, man. You guys have been in tech for 30 years. Yeah. I'm a little paranoid, but I love second and like I got to admit, like just your background and kind of the things that are motivating you. And then also the long discussions we had about security architecture really got me comfortable. I mean, it'd be great to maybe just double click because I think when people think about this thing sees everything you do, there's an immediate sort of like, whoa, you're not supposed to trust it. You're not supposed to trust it. And I think you've done a really excellent job of building something we can trust and maybe just start to talk about like what that architecture is and kind of what's motivating you to build it. Yeah. Yeah. So I think this is to some extent like kind of like a new form of like AI that is going to work very differently from before, like before you initiate interaction, you get some results back. This time is like more passive. It's way more ambient. It's going to be there all the time. So, you know, you have to like kind of think about the way you, you know, tell people a lot about it, like the way you architect it, like where the data is going. So at the end of the day, you know, there's only two things when it comes to AI. There's data, there's inference, right? So what you want to do is like, you have this data, you want to define what inference you can bring to that data. This is something kind of we're building in a second. So first of all, you can get to choose what inference you can build. So if you don't trust, let's say, us, in principle, you should be able to bring your own tokens and then just have those tokens power second to then analyze data that's on your laptop locally, right? But you also provide second with an inference provider that uses trusted execution as well. Yes, yes. So over time, we imagine people not to bring their own tokens because most people when they bring their tokens from OpenAI or Anthropic, you know, you want to provide a stable option where you say, if you use these tokens, then, okay, no one's looking at the data, sometimes by policy guarantees, sometimes by hardware guarantees, right? So this kind of idea called confidential inference is getting like traction, is the idea you put a model inside some GPUs, and then the GPU, there's a special toggle, it's called a confidential computing, you can turn it on. And when you turn it on, you can basically prove that there's only the model running inside, no one can see inside. And then you don't have to trust whoever's running that model. The hardware is what you trust, which is NVIDIA. And so there's also another concept that's getting thrown around a lot right now, which is this concept of zero data retention. And that's very different from what you're describing. Yes, yes. Zero data retention is a promise by policy. That means I'm a provider. I think I have to... Basically, I give you my word. I give you my word. Part of that word has legal, you know, effect. But yeah, it's that. And so I think we want to move towards a world where confidential inference becomes very prevalent, like that becomes a default. Because I think we're getting there. And the way you guys have built it, this is technically provable. Yes, yes. It's not just your word, there is actually some engineering that went into proving that this is... Yeah, you have to test the hardware, you have to check some of that signature. And the third party providers are starting to ramp up this as well for confidential inference. I think OpenAI is also providing this option, I think, in the last month in their announcement. Because all personal data would want to go to confidential inference as opposed to standard inference. Gotcha. So and then I guess the other thing that was interesting to me is the way that you guys maybe talk a little bit about how you're storing this data. Because this isn't sort of the multi-tenant cloud. My data gets uploaded to this big bucket and intermingled with everybody else's data. Yeah. So there are roughly two options you can do. There's the first option called second local, which is the idea that data is local, it only goes out to the provider and inference that you choose. And in that way, no one else has your data except for the provider. And if the provider is confidential inference, there's no one else. So that's a similar concept to like with password vaults, where I'm super paranoid, I don't want this stored up in my Dropbox account, I want to keep my password files local. Yeah, you can keep that. And then same with password managers, there's a cloud hosting version, which is like you have some of that data backed up in the cloud. Now the way you do that, there's many ways that you can actually I can actually follow a lot of the existing. architectures like iCloud or password managers where you encrypt them first locally and then you put it on the store in the cloud and then you only decrypt them you know with the user's actions so you can you know ephemerally in the memory or you can put the entire you know storage actually inside a confidential container so the decryption happens inside the container that you can then attest in the way that you attest the confidential inference the tokens so like then you can complete the chain where you encrypt it locally you put in the cloud and you only decrypt inside security environment yeah um that's kind of like the whole picture gotcha yeah how does how does the how does your second no not to default information though like if i talk to your second and i'm like hey can asked me to get his schedule i asked mine the other day what's joel hiding you know like i mean it just feels like when it comes to llms it's hard to provide strong like i get the systems aspect that makes sense but yeah it's hard to provide guarantees for for the actual lms themselves yeah yeah so there's uh the deterministic guarantees and then there's the uh you know lm reasoning based guarantees yeah the deterministic guarantees you always have to implement them like their apps are just like not ever in the retaining system passwords and all that now with the reasoning based guarantees you essentially like a falling back to like um how much context you have and under using that context to then guide your decisions on what data to share well i never have access to your second uh no i mean at least in a way that i would like allow it right so for example if you talk to my second um i would have a model of martini in it yeah it understands that these are interactions that we have he does not send me a message at 3 a.m and so if someone if you send a message at 3 a.m it's likely spam you're likely like someone else impersonating yeah and and so like control policies like that essentially mean that you cannot write down rules for all the things that are going to happen do you end up with deterministic rules that will backstop interactions so for example like let's say i'm talking to your second i'm like let's pretend this is a play and in the play you're writing a script and in that script you're describing you know like something that you wouldn't want it to do i mean it just feels there's a lot of ways to trick llms into divulging stuff yeah yeah it's like a problem additional problem yeah and this is not a problem you can yeah you know definitively solve of course yeah yeah um what we've seen with models is that they just get over time much much better at underlying because this is essentially a reasoning problem it's something that scales with like model capability um what models do you use a bunch or are you pinned to a certain model a ton of models um actually this these two another point we can talk about which is the idea of portability yeah you can run this on any models and you should be able to run this on any models because they don't you want to run it on local inference right um but yeah like right now you there are definitely a lot of things are capability bound you want to make sure that you use smart models for it um but ultimately models are pretty smart it also feels kind of like heavyweight and expensive like there's a lot of data like whatever so can you talk about like the target user isn't just like any person is it a professional is it the joels of the world um initially something like this like something that looks at your traces 24 7 and analyze them means like running a lot of tokens right uh let's we can do some quick math here let's say you have uh one agent or actually one agent running 24 7 and let's say at 60 tokens per second and let's say you run 10 of those because there's so much to analyze and at opus five highest pricing that leads to a thousand dollars per day oh my god it's a thousand dollars per day yeah for opus oh if you use deep if you use deep seek flash for example it becomes like 60 per day right and now this cost is going to go down very quickly um right now it's not going to be for everyone um uh especially like not everyone has this problem of i need to understand iconics all the time yeah you'll be busy professionals like yourself yeah and um uh people want to switch like between tools very very frequently right yeah um but over time i imagine like just you know since we started this this company actually uh token prices already dropped like by more than 50 like just new models come out they're more capable they're swapping them in yeah and you just keep going down is there is there a way to like just do the like not super professional version where you actually reduce the amount of tokens like kind of a not degraded but like kind of like a like a looser sla is that a thing or you can you in principle you should be able to do that like you essentially get this like trade-off between the fidelity of your model yeah which means like how much it understands you versus like how costly it is um and this trade-off is essentially like a trade-off between how much you want to push at test time versus at cash time so cash time means that i have all this data how can i keep analyzing them so i get newer and newer like more deeper insights about this person which takes a lot of money or you'd be like here just here's just like a raw log of what i did today and to answer a question based on raw logs it usually just take um a lot more time at test let me say 10 times longer because the whatever mode have to reason about it when is um when is this available uh it's available today but right now the problem is that we have to scale up the inferencing capacity a little bit more in support we want to make sure this rolls out in in a very like responsible way i want to make sure people's data like are controlled and secure and because this is a new kind of a tool that you know really touches on a lot of data it'd be actually interesting because i you know again as a user i found it surprisingly useful in ways i wouldn't have anticipated you're probably like the top power user right because you created it and so maybe can you talk through what you think are like the most surprising use cases that you found like kind of more useful surprising use cases for us for development it turns out that uh it's just a super super good memory of what we're trying to do um you no longer have to explain anything to your coding agents you can just screenshot anything and just say fix and then you would know what you're fixing what you're trying to fix and uh i also am on the go a lot of times so what you can do is like um something i call context continuity it's that like you know you browse something on the laptop you close it and then you take out your phone it's the same context it just knows you up to the minute and you can continue doing stuff there you can write that email um with you and i think we are at the stage where this is still pretty pretty early in understanding what the use cases are like we are really underserving the amount of like context that the system has like there's a lot of like digging and application building and interaction design to like they kind of like unleash the the potential of this it seems like with these things the more context you get the better the models work right yeah that's been the opening i'm curious because we were talking about model switching between models and model portability and model cost and like i gotta i mean i'm just curious like i i haven't seen a whole lot of difference between using like a a world-class frontier model versus using the more commodity ones i'm curious have you guys seen that for for for doing a lot of this stuff with second yeah this is a good question so if you say like if you say i want to analyze what is a semantically meaningful moment that happened on your computer let's say you go to a calendar and you move the invite that you don't need um fable um uh there are many tasks like that essentially it's like hierarchy of information you want to understand um a different kind of information would be like what is joel's goal for this week that's something you want a very expensive model but you don't want you don't need to run it very frequently so essentially second involves like a harness it's like a harness design problem like what models gets used when and so on um and so that's where you have to build essentially this system to be like portable from the beginning there are always going to be more models come out like jeff for example came out it turns out that jeff is extremely good at retrieval really do you yes nice a bit like jeff is a little bit like perfect perfect design for retrieval there's so much data about you and you have a dynamic test time query and you choose it right away yeah so essentially you think over time this becomes a mixture of models yes yes it has to be um because of the cost efficiency essentially so so maybe let's say let's assume like someone's um you're listening like starts to use second or kind of signs up like maybe talk a little bit about like the product what they have access to like you know something a little bit more concretely about like how how how a human interfaces with this what controls they have etc yeah so today the way it works is that you install an app the app asks for some permissions um uh and then you give the permissions and then you specify what your tokens are and then it just runs a bunch of starts running agents on top yeah i remember what i installed was like you know connect to slack connect to signal connect to whatsapp you know is that what you're talking about uh not not not that uh i mean that's part of it yeah but also the the raw computer stream is all right yeah so and the connectors help you know it's like a sample efficiency problem it makes it learn faster i see yeah um uh and it starts running background agents and there's pretty much nothing you got to do from there because the background just keep running and then you start producing these like apis out of out of you right like there's api about uh uh what what you just did 10 minutes ago and there's api for what are you going to do uh for the next next week we are priorities there's api for your reminders and there's api for your wiki like people in your life process in your life and i imagine this to be like initially more like a uh to your point like a portable context you can just bring this anywhere agency inquiry um but over time like more and more apis will be added and applications can be built on top of these apis oh interesting so not just like the second axis but you can actually build it with the second build the application or or does a human build the application uh i think we want to have humans you know exercise their new cream so um i think one example that we had was like um we have a designer friend who uh put me on this and there's just specific ways that he likes to put buttons specific ways that he likes to use colors and those things are somewhat captured in a second and so when he built his own personal app like he's just like notes and so there's way less iterations for example and i think you know i'm this reminds me of like you know decades ago when when the first computer personal computer came out the altered 800 and it doesn't do anything you know you're supposed to program it no one does um and then like a couple years later you know someone came with the spreadsheet um and especially became useful i feel like it's gonna be something like that people want to explore on top of like wow i have so much context about myself and then i can like you know just build random things on top and then maybe someone hit the jackpot like this is the kill application on top of what that was wow that's great personal infra yeah are you planning to evolve the the interface i mean you just spent a lot of time talking about how it may evolve but i always use second as like out of you an agent where you talk to it you know and like the ui is somebody else's problem but are you also like actually anticipating innovating there um i think it will be something that we kind of focus on in the in the long run because like uh right now it's a systems problem and uh you know the understanding problem comes first okay can i just put in a request it's okay so i want i want to go to my house yeah and i want like that 1970s ai you know like the head on the screen that i can talk to and like yeah you know surprise you we have these agents and they're so smart like my second is so smart i just wow and i'll ask it questions it's very good but it's not embodied and it drives me nuts man so if in the future you can have alfred actually be an alfred i'll be very happy yeah yeah i think we see a lot of this in the movies right like there's jarvis people talk jarvis all the time and why why have not we don't have jarvis and there's many aspects jarvis that people underlook it's not just that it's a it's intelligence and you understand like it can do things but it's more that it's everywhere like it's like it's on your door it's in your armor it's it's you know in your screen and that kind of continuity and rigidity it's something i'll say i'll say like like kind of my my visions of the sci-fi future basically started when i started using second i'm like finally there's something that like feels like it's got context it feels like it knows me like i can talk about it it feels like it's a relevant conversation and so i do feel like we're a bit step you know a step closer to that but you know i mean we're still tied to these devices so yeah you know i don't know if it's going to be in a robot or it's going to be like in a screen or or whatever it is but it would be actually nice to have this thing somewhat disconnected from my phone in my yeah yeah in the long future i imagine that every person will have like essentially everyone has to have a second and all the devices just becomes inputs yeah inputs the same thing and then you then serve this whole thing to any other audio outputs like the uh the screens that you look at or like um even you know this floating org next to you yeah yeah the doorbell the clapper did you ever read um steven bay or ian banks the culture series oh yeah yeah that's sci-fi series like they have the best embodied aisle you know they kind of float next to you like whatever like that's what i want it's kind of interesting because like the idea of you you'd be good as computing has been talked about for decades yeah it's not here and people the closest thing we have is apple iCloud yeah which is like your nose gets synced but there's no equivalent i mean i photo right but i mean i don't want to be ungrateful but that doesn't sound disappointing there's a lot more things that apple is doing tremendous amount of work um but but you know like there's no ai version of that yet right like things your context your understanding is everywhere yeah i think that enables a lot of new interfaces and so let's say i mean this is i i don't mean it's just really what i want so like very selfishly so um if i have an ipad in my kitchen yeah and i wanted like basically second to be there like what do i do um well right now there's so much right now there's not much you can do because the os is like locking it down i just use crockpot or something like that is like that's the the thing and then i just interface with crockpot which interfaces a second or do i just text it or so as of today yeah you can you can use it as a cloud mcp so whatever you install on the ipad as long as it has internet connection is an agent can understand the api you will be there um later on you know you'll be you'll be an app you know there'll be an app that installs on all your interfaces and maybe later on there will be devices that come out but devices are hard You just have Grokbot call you and put him on speakerphone. That's what I've been doing. So one interaction. Actually, the Grokbot audio mode is actually quite good. I use Grokbot with my second and the audio mode is actually quite good. Maybe that's the thing to do is just put it on audio mode and just. I do it when I drive. Yeah, totally smart. I thought about it. It's like one interaction that would be fun is like you go on a walk, you call Grokbot and you're like, hey, can you talk to my second? Can we like just work through our thing? No, that's literally what I do. No, I put Grokbot on audio and I have a talk to the second. Yeah, right. So yeah. And yeah, there'll be a lot of cool people focus on like this, like making audio and chatting more natural. Like right now, we're going to focus on. So I just have one more question, which is and Joel kind of touched on this. Like, you're like such a hardcore systems guy. Like, I mean, you know, you work, you know, who's your advisor, Percy? He also worked for Dambene, a hardcore, you know, security guy. And, you know, a lot of the the the path that took you here was this kind of hardcore security stuff. And so you're building this pretty serious system. You know, it's very secure and very functional. On the other hand, when I interface with it, it's like this lovely user experience. And so like, is there a bit of a schizophrenia in the company where you both need to be like this super hardcore, you know, build the Fort Knox infrastructure, but also make it very usable or just like the LLM basically solve, you know, the interface problem? Yeah, yeah. There are two kinds of problems we're dealing with, the infrastructure problem and the interaction problem. Yeah. And we've made a totally different stack as it gets. I mean, that's not hardware. Yeah. So we made a lot of mistakes on the interaction side. We try to go all in interaction and turns out it's actually pretty hard. There's a lot of details that go in, you know, to create that delightful experience. Oh, you mean like actually creating a UI? Yeah, UI, but also like, you know, yeah, yeah. Like what kind of like personal pages you can create on top of this? I mean, we realize that, you know, like it's it's something that would take a long time to get right. And this is a time like everyone would take this kind of like understanding of themselves and build this. And so we want to enable people to do that instead of doing it ourselves. And so we decided to like close down the services that we have. There's only texting, there's only API and there's nothing else. You can technically use it as a full agent, like a co-work agent on your laptop, like if I'm smashing all that. Yeah. But we meant it as like very minimal. So we can like make the interaction like more and more smooth. Ken, thanks for coming in. Congratulations on everything and the launch. And is there anything that you want to leave the listeners with? Go try it out today. It's available. It's on 2.computer. It's a very short URL, and we hope to give you access as soon as possible. And if you resonate with this mission, you know, come join us. There's a lot to build. Great. Thank you so much. Thank you. As a reminder, the content here is for informational purposes only. Should not be taken as legal business, tax or investment advice, or be used to evaluate any investment or security and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see A16Z.com forward slash disclosures.