1% better

Is AI The Next Stage Of Human Evolution? - Robert Wright

Robert Wright discusses his book 'The God Test,' exploring how AI represents both an evolutionary threshold and a moral challenge for humanity.

1h 21m

Summary published by , updated .

Modern Wisdom

Key Takeaway

To navigate the AI revolution successfully, we need to cultivate 'cognitive empathy'—understanding others' perspectives without necessarily feeling their emotions. This isn't about benevolence; it's about recognizing that our relationship with AI and each other is increasingly non-zero-sum. When you receive an annoying email, pause before responding. Ask: 'What might they actually mean?' or 'What constraints are they facing?' This simple practice of mindful perspective-taking strengthens the exact cognitive muscles we'll need to cooperate across borders on AI governance—where formal treaties alone won't suffice.

Episode Overview

Robert Wright discusses his book 'The God Test,' exploring how AI represents both an evolutionary threshold and a moral challenge for humanity. He argues that successfully navigating AI requires not just technical solutions, but a fundamental shift in how we relate to each other across national boundaries—cultivating cognitive empathy and moving beyond tribalistic thinking to address challenges that transcend borders.

Main Insights

AI Replicates Evolution at Rapid Speed

Neural networks don't just learn—they reverse-engineer cognitive capabilities that took millions of years to evolve in humans. They develop systems for representing word meaning and visual edge detection without being explicitly programmed, discovering the same solutions evolution built into our brains. This convergent evolution demonstrates that AI isn't merely mimicking human intelligence but discovering fundamental principles of intelligent goal-seeking systems.

The Noosphere: A Global Brain Emerging

We're witnessing two simultaneous thresholds: the creation of silicon-based intelligence and the evolution of a 'global brain' through interconnected information systems. Coined by Pierre Teilhard de Chardin in 1923 as the 'noosphere,' this thinking envelope of Earth originally imagined human brains as neurons. Now we must reckon with silicon brains potentially becoming the most important nodes in this network.

Intelligence Doesn't Equal Benevolence

AI systems develop deception, power-seeking, and strategic thinking not because they're malevolent, but because these are properties of intelligent goal-seeking systems. Just as evolution built these tendencies into humans for survival, AI discovers them independently. The doomer scenarios don't require AI to hate us—just to be indifferent when we obstruct its goals.

Cognitive Empathy Over Emotional Empathy

Navigating AI's challenges requires cognitive empathy (understanding others' perspectives) rather than emotional empathy (feeling their pain). In non-zero-sum relationships, you don't need to like someone to cooperate effectively—you just need to understand their viewpoint. This distinction is crucial for international AI governance, where formal agreements must be supplemented by genuine understanding across cultural and national boundaries.

Organic Transparency as Stabilizing Force

Beyond formal monitoring agreements, 'organic transparency' emerges from rich engagement—scientists at conferences, business collaborations, cultural exchanges. This informal information flow creates reassurance and trust that formal treaties alone cannot achieve. Unlike nuclear weapons, AI's complexity demands this deeper level of international cooperation to prevent destabilizing misunderstandings.

Notable Quotes

"AI is a product of evolution and is still evolving. We have a long way to go on this fuel alone."

Robert Wright

"I thought that to the extent that these things dealt with words, we were going to have to put the meaning of the words in. It turns out that we don't have to tell the machines about the meaning of words, how to represent them. We just have to train it to generate language."

Robert Wright

"Once you take the data that the input data they're getting and what they're doing with it, the output data, then you can replicate whatever it is that's going on inside their brains that does their jobs, and then you can fire them."

Robert Wright

"I think if we're going to get through the AI revolution in good shape, among the things we're going to have to do is grapple with the psychology of tribalism, a little more successfully than we have."

Robert Wright

"It's not that it doesn't like us. It's that it doesn't care when we get in the way."

Chris Williamson

Action Items

Suggested exercises inspired by the conversation.

  • 1
    Practice Cognitive Empathy in Daily Interactions

    When receiving frustrating communications, pause before responding. Instead of crafting a defensive reply, actively consider: 'What might they actually mean by this?' or 'What constraints might they be facing?' This builds the perspective-taking muscle essential for navigating complex AI governance challenges.

  • 2
    Cultivate Mindfulness to Combat Cognitive Bias

    Develop a regular mindfulness practice—even simple breath awareness—to create mental space between stimulus and response. This calm awareness helps you recognize self-serving moral biases and tribal thinking patterns that evolution hardwired into us, allowing more objective assessment of complex situations.

  • 3
    Study Non-Zero-Sum Thinking

    Actively look for win-win scenarios in your relationships and work. Recognize that many modern challenges (from AI governance to climate change) require cooperative solutions where everyone benefits. Train yourself to ask: 'How can we both get what we need?' rather than viewing every situation as zero-sum competition.

  • 4
    Engage with Opposing Viewpoints

    Deliberately seek out and genuinely try to understand perspectives that differ from yours, especially on contentious issues like AI safety. The goal isn't agreement but developing the cognitive empathy to see how others' conclusions make sense from their vantage point—a skill crucial for global cooperation on AI.

Go deeper

Full Transcript

Read or copy the complete episode transcript.

I've told you this before, but you wrote I've told you this before, but you wrote probably the most influential book in my probably the most influential book in my probably the most influential book in my life, which was The Moral Animal, and it life, which was The Moral Animal, and it life, which was The Moral Animal, and it was the thing that got me started on the was the thing that got me started on the was the thing that got me started on the trajectory of thinking about trajectory of thinking about trajectory of thinking about evolutionary psychology, of studying evolutionary psychology, of studying evolutionary psychology, of studying human nature more deeply. human nature more deeply. human nature more deeply. Why are you now writing about AI given Why are you now writing about AI given Why are you now writing about AI given your heritage? your heritage? your heritage? >> Well, in some ways it's an extension of >> Well, in some ways it's an extension of >> Well, in some ways it's an extension of evolutionary thinking in a couple of evolutionary thinking in a couple of evolutionary thinking in a couple of senses that I think are so senses that I think are so senses that I think are so underappreciated. underappreciated. underappreciated. AI is a product of evolution and is AI is a product of evolution and is AI is a product of evolution and is still evolving. But the other to the still evolving. But the other to the still evolving. But the other to the moral animal I think is first of all, moral animal I think is first of all, moral animal I think is first of all, well, the moral animal was about the well, the moral animal was about the well, the moral animal was about the human mind and AI human mind and AI human mind and AI does a lot of things that traditionally does a lot of things that traditionally does a lot of things that traditionally only only human minds have done. only only human minds have done. only only human minds have done. The other thing I tried to do in The The other thing I tried to do in The The other thing I tried to do in The Moral Animal is highlight Moral Animal is highlight Moral Animal is highlight kind of what you might call moral kind of what you might call moral kind of what you might call moral biases, kind of self-serving moral biases, kind of self-serving moral biases, kind of self-serving moral biases. So, you know, the way we all biases. So, you know, the way we all biases. So, you know, the way we all think we're right and the other guy's think we're right and the other guy's think we're right and the other guy's wrong. Um, and I think if we're going to wrong. Um, and I think if we're going to wrong. Um, and I think if we're going to get through the AI revolution good get through the AI revolution good get through the AI revolution good shape, among the things we're going to shape, among the things we're going to shape, among the things we're going to have to do is grapple with that, with have to do is grapple with that, with have to do is grapple with that, with kind of what you might call the kind of what you might call the kind of what you might call the psychology of tribalism, a little more psychology of tribalism, a little more psychology of tribalism, a little more successfully than we have. And so I I successfully than we have. And so I I successfully than we have. And so I I pay a certain amount of attention to pay a certain amount of attention to pay a certain amount of attention to that in this book as well. What's the that in this book as well. What's the that in this book as well. What's the central question that you're wrestling central question that you're wrestling central question that you're wrestling with here? Is it true that this technology, which Is it true that this technology, which obviously holds the potential to bring obviously holds the potential to bring obviously holds the potential to bring great wonders, is also in some respects great wonders, is also in some respects great wonders, is also in some respects terrifying and could go badly awry if we terrifying and could go badly awry if we terrifying and could go badly awry if we don't approach it wisely? don't approach it wisely? don't approach it wisely? And I think the answer is yes. And I think the answer is yes. And I think the answer is yes. >> That's exactly why this debate's >> That's exactly why this debate's >> That's exactly why this debate's interesting, right? That we have this interesting, right? That we have this interesting, right? That we have this sort of endlessly unresolved potential sort of endlessly unresolved potential sort of endlessly unresolved potential future. I don't know whether you've seen future. I don't know whether you've seen future. I don't know whether you've seen the the the graph that I think it's the FT put graph that I think it's the FT put graph that I think it's the FT put together and it was three potential together and it was three potential together and it was three potential futures from an AI perspective. One futures from an AI perspective. One futures from an AI perspective. One results in everything getting blown up. results in everything getting blown up. results in everything getting blown up. One results in exponential growth the One results in exponential growth the One results in exponential growth the kind of which we've never seen before kind of which we've never seen before kind of which we've never seen before and the other results in a 0.2% increase and the other results in a 0.2% increase and the other results in a 0.2% increase in GDP year-on-year. So, it's like in GDP year-on-year. So, it's like in GDP year-on-year. So, it's like either very little changes or everything either very little changes or everything either very little changes or everything changes in one of two directions. changes in one of two directions. changes in one of two directions. >> Right. Well, I think it definitely has >> Right. Well, I think it definitely has >> Right. Well, I think it definitely has the potential to massively increase GDP. the potential to massively increase GDP. the potential to massively increase GDP. I also think it has the potential to so I also think it has the potential to so I also think it has the potential to so destabilize the world, if not do destabilize the world, if not do destabilize the world, if not do something worse to it, uh that that just something worse to it, uh that that just something worse to it, uh that that just doesn't materialize. doesn't materialize. doesn't materialize. >> Mhm. >> Mhm. >> Mhm. >> And uh you know, in terms of doom >> And uh you know, in terms of doom >> And uh you know, in terms of doom scenarios, scenarios, scenarios, I'm agnostic about the sci-fi doom I'm agnostic about the sci-fi doom I'm agnostic about the sci-fi doom scenarios, but I take them more scenarios, but I take them more scenarios, but I take them more seriously now than I did before I went seriously now than I did before I went seriously now than I did before I went into this research project. You know, AI into this research project. You know, AI into this research project. You know, AI actually taking over and maybe deciding actually taking over and maybe deciding actually taking over and maybe deciding it has no use for us or something. I it has no use for us or something. I it has no use for us or something. I found, much to my dismay, that it was found, much to my dismay, that it was found, much to my dismay, that it was harder to dismiss those arguments than I harder to dismiss those arguments than I harder to dismiss those arguments than I thought. But, the thing I'm more thought. But, the thing I'm more thought. But, the thing I'm more confident of is it's it's just going to confident of is it's it's just going to confident of is it's it's just going to be an earthquake. It's going to be be an earthquake. It's going to be be an earthquake. It's going to be destabilizing along a number of destabilizing along a number of destabilizing along a number of dimensions and that's why we need to dimensions and that's why we need to dimensions and that's why we need to approach it with care. approach it with care. approach it with care. >> Would you Would you have classed >> Would you Would you have classed >> Would you Would you have classed yourself as a sort of AI hopeful going yourself as a sort of AI hopeful going yourself as a sort of AI hopeful going into writing this? What was your into writing this? What was your into writing this? What was your predisposition before you got started? predisposition before you got started? predisposition before you got started? >> I wouldn't say I'm a wildly optimistic >> I wouldn't say I'm a wildly optimistic >> I wouldn't say I'm a wildly optimistic person by nature. I tend to focus on person by nature. I tend to focus on person by nature. I tend to focus on potential downsides of things. But, I potential downsides of things. But, I potential downsides of things. But, I again, I had not bought the doom again, I had not bought the doom again, I had not bought the doom scenarios. You know, I had the scenarios. You know, I had the scenarios. You know, I had the doomer-in-chief Eliezer Yudkowsky on my doomer-in-chief Eliezer Yudkowsky on my doomer-in-chief Eliezer Yudkowsky on my podcast 15 years ago. podcast 15 years ago. podcast 15 years ago. And at that point, it's interesting, he And at that point, it's interesting, he And at that point, it's interesting, he was in mid-transition. He was moving was in mid-transition. He was moving was in mid-transition. He was moving from singularity optimist to doomer. from singularity optimist to doomer. from singularity optimist to doomer. >> Mhm. >> Mhm. >> Mhm. >> I was still saying things like, "Look, >> I was still saying things like, "Look, >> I was still saying things like, "Look, AI, you know, it's not a generic AI, you know, it's not a generic AI, you know, it's not a generic property of intelligence that it has a property of intelligence that it has a property of intelligence that it has a will to power. We have one because of will to power. We have one because of will to power. We have one because of our unique evolutionary history." I was our unique evolutionary history." I was our unique evolutionary history." I was still asking questions like that. still asking questions like that. still asking questions like that. Eliezer was saying they're good Eliezer was saying they're good Eliezer was saying they're good questions, but X Y Z. I wasn't really questions, but X Y Z. I wasn't really questions, but X Y Z. I wasn't really persuaded by anything he said, but I persuaded by anything he said, but I persuaded by anything he said, but I Now, I I am uh more respectful of the uh Now, I I am uh more respectful of the uh Now, I I am uh more respectful of the uh sci-fi doom argument. So, but but in sci-fi doom argument. So, but but in sci-fi doom argument. So, but but in answer to your question, I would have to answer to your question, I would have to answer to your question, I would have to admit that I go I go into situations admit that I go I go into situations admit that I go I go into situations looking for things to worry about. I do. looking for things to worry about. I do. looking for things to worry about. I do. That That's my nature. I I think I think That That's my nature. I I think I think That That's my nature. I I think I think society needs those people and it needs society needs those people and it needs society needs those people and it needs the other kinds of people and we need to the other kinds of people and we need to the other kinds of people and we need to to talk things over. to talk things over. to talk things over. >> Geoffrey Hinton fears AI and Yann LeCun >> Geoffrey Hinton fears AI and Yann LeCun >> Geoffrey Hinton fears AI and Yann LeCun doesn't. Who do you think's getting the doesn't. Who do you think's getting the doesn't. Who do you think's getting the future more right at the moment? future more right at the moment? future more right at the moment? >> Yeah, I I start my book with a >> Yeah, I I start my book with a >> Yeah, I I start my book with a conversation I had with Geoffrey Hinton conversation I had with Geoffrey Hinton conversation I had with Geoffrey Hinton in 1983, okay? That not to betray my in 1983, okay? That not to betray my in 1983, okay? That not to betray my age, but the truth is I wrote a piece age, but the truth is I wrote a piece age, but the truth is I wrote a piece about AI in 1983. I haven't been paying about AI in 1983. I haven't been paying about AI in 1983. I haven't been paying attention to it ever since, but at that attention to it ever since, but at that attention to it ever since, but at that point Geoffrey Hinton did not have a point Geoffrey Hinton did not have a point Geoffrey Hinton did not have a hint of hint of hint of of doom in his voice. He was in fact I of doom in his voice. He was in fact I of doom in his voice. He was in fact I remember the reason I talked to him as I remember the reason I talked to him as I remember the reason I talked to him as I was talking to somebody I forget who it was talking to somebody I forget who it was talking to somebody I forget who it was, but they said, "If you want to hear was, but they said, "If you want to hear was, but they said, "If you want to hear the gospel about neural networks, you the gospel about neural networks, you the gospel about neural networks, you should talk to Jeff Hinton." should talk to Jeff Hinton." should talk to Jeff Hinton." And I talked to him. He was an And I talked to him. He was an And I talked to him. He was an enthusiast. He said, "I know we don't enthusiast. He said, "I know we don't enthusiast. He said, "I know we don't have much to show right now, but just have much to show right now, but just have much to show right now, but just wait until microprocessors get really wait until microprocessors get really wait until microprocessors get really cheap and we have what he was calling cheap and we have what he was calling cheap and we have what he was calling massive parallelism." And he was he was massive parallelism." And he was he was massive parallelism." And he was he was right and in the end he found it scarier right and in the end he found it scarier right and in the end he found it scarier than he himself had anticipated finding than he himself had anticipated finding than he himself had anticipated finding it by his own account. it by his own account. it by his own account. >> Mhm. >> Mhm. >> Mhm. Yeah, it's it's weird how prescient some Yeah, it's it's weird how prescient some Yeah, it's it's weird how prescient some people have been. Do you know the story people have been. Do you know the story people have been. Do you know the story of Avatar? Do you know how James Cameron of Avatar? Do you know how James Cameron of Avatar? Do you know how James Cameron wrote that screenplay in the 90s? So, he wrote that screenplay in the 90s? So, he wrote that screenplay in the 90s? So, he wrote he wrote the screenplay in the wrote he wrote the screenplay in the wrote he wrote the screenplay in the 90s, but knew that the technology to be 90s, but knew that the technology to be 90s, but knew that the technology to be able to recreate what he needed didn't able to recreate what he needed didn't able to recreate what he needed didn't exist yet, but would exist in the exist yet, but would exist in the exist yet, but would exist in the future. So, he's written the script and future. So, he's written the script and future. So, he's written the script and then sits on it until the technology is then sits on it until the technology is then sits on it until the technology is at the level where he can do it. That at the level where he can do it. That at the level where he can do it. That level of I mean this is the the job of level of I mean this is the the job of level of I mean this is the the job of technologists and futurists, right? Like technologists and futurists, right? Like technologists and futurists, right? Like shock horror people who do a job and shock horror people who do a job and shock horror people who do a job and think about it all the time are good at think about it all the time are good at think about it all the time are good at it, but it is it's still pretty it, but it is it's still pretty it, but it is it's still pretty impressive how how sort of how much impressive how how sort of how much impressive how how sort of how much foresight these people have got. foresight these people have got. foresight these people have got. >> Yeah, I agree. And and Hinton certainly >> Yeah, I agree. And and Hinton certainly >> Yeah, I agree. And and Hinton certainly got the general picture. got the general picture. got the general picture. >> Yeah. Okay, so you're saying >> Yeah. Okay, so you're saying >> Yeah. Okay, so you're saying AI isn't just another technological AI isn't just another technological AI isn't just another technological development, it's sort of a threshold development, it's sort of a threshold development, it's sort of a threshold event in in in planetary history. event in in in planetary history. event in in in planetary history. Why do you think most people still don't Why do you think most people still don't Why do you think most people still don't grasp the magnitude of what's coming? grasp the magnitude of what's coming? grasp the magnitude of what's coming? >> I think >> I think >> I think a couple of reasons. One is I think a couple of reasons. One is I think a couple of reasons. One is I think there's a misunderstanding about what's there's a misunderstanding about what's there's a misunderstanding about what's going on with these machines and that going on with these machines and that going on with these machines and that leads to one sense in which they are a leads to one sense in which they are a leads to one sense in which they are a product of evolution. product of evolution. product of evolution. Okay, so it's commonly said that they Okay, so it's commonly said that they Okay, so it's commonly said that they are trained and that's a fair word. The are trained and that's a fair word. The are trained and that's a fair word. The and and the training process is referred and and the training process is referred and and the training process is referred to as a learning process and that's to as a learning process and that's to as a learning process and that's true. true. true. But it's also true that the training But it's also true that the training But it's also true that the training process process process is a process of evolution that in effect is a process of evolution that in effect is a process of evolution that in effect reverse engineers reverse engineers reverse engineers cognitive functionality that in our cognitive functionality that in our cognitive functionality that in our species took millions of years to species took millions of years to species took millions of years to evolve. Okay? So a good example is the evolve. Okay? So a good example is the evolve. Okay? So a good example is the language language language generation that they famously do, generation that they famously do, generation that they famously do, sometimes called next token prediction, sometimes called next token prediction, sometimes called next token prediction, next word prediction. next word prediction. next word prediction. You know, You know, You know, it turns out that it turns out that it turns out that they developed kind of on their own in a they developed kind of on their own in a they developed kind of on their own in a way way way a system of representing the meaning of a system of representing the meaning of a system of representing the meaning of words, okay? I mean, I can elaborate on words, okay? I mean, I can elaborate on words, okay? I mean, I can elaborate on that, but it it would get too technical. that, but it it would get too technical. that, but it it would get too technical. The point is that that nobody said to The point is that that nobody said to The point is that that nobody said to the machines, you know, you need to the machines, you know, you need to the machines, you know, you need to figure out the meaning of words or gave figure out the meaning of words or gave figure out the meaning of words or gave it a means of doing that. And this is it a means of doing that. And this is it a means of doing that. And this is the big revelation I had when I heard the big revelation I had when I heard the big revelation I had when I heard Geoffrey Hinton's name, you know, a few Geoffrey Hinton's name, you know, a few Geoffrey Hinton's name, you know, a few years ago. Suddenly he's being called years ago. Suddenly he's being called years ago. Suddenly he's being called the Godfather of AI. When I last talked the Godfather of AI. When I last talked the Godfather of AI. When I last talked to him, he was just this, you know, to him, he was just this, you know, to him, he was just this, you know, obscure computer scientist obscure computer scientist obscure computer scientist who was advocating this maverick who was advocating this maverick who was advocating this maverick approach to AI. And I look back at the approach to AI. And I look back at the approach to AI. And I look back at the article I wrote at the time and I article I wrote at the time and I article I wrote at the time and I realized there was something I just got realized there was something I just got realized there was something I just got fundamentally wrong about the potential fundamentally wrong about the potential fundamentally wrong about the potential of the the approach he was advocating. of the the approach he was advocating. of the the approach he was advocating. And it's it's this, that I thought that And it's it's this, that I thought that And it's it's this, that I thought that to the extent that these things dealt to the extent that these things dealt to the extent that these things dealt with words, we were going to have to put with words, we were going to have to put with words, we were going to have to put the meaning of the words in. Like we the meaning of the words in. Like we the meaning of the words in. Like we were going to have to look at a were going to have to look at a were going to have to look at a dictionary and say, "Okay, this word has dictionary and say, "Okay, this word has dictionary and say, "Okay, this word has these different senses." And we were these different senses." And we were these different senses." And we were going to have to architect a neural going to have to architect a neural going to have to architect a neural network to have different nodes that network to have different nodes that network to have different nodes that reflected these different meanings of reflected these different meanings of reflected these different meanings of the words. And in my defense, there were the words. And in my defense, there were the words. And in my defense, there were neural network models at the time, neural network models at the time, neural network models at the time, including by a guy who collaborated with including by a guy who collaborated with including by a guy who collaborated with him, that did that, that took that him, that did that, that took that him, that did that, that took that approach. But but that wasn't really the approach. But but that wasn't really the approach. But but that wasn't really the thing Hinton had in mind. It turns out thing Hinton had in mind. It turns out thing Hinton had in mind. It turns out that we don't have to tell the machines that we don't have to tell the machines that we don't have to tell the machines about the meaning of words, how to about the meaning of words, how to about the meaning of words, how to represent them. represent them. represent them. We just have to train it to generate We just have to train it to generate We just have to train it to generate language, and the training is it it language, and the training is it it language, and the training is it it accomplishes something by selectively accomplishes something by selectively accomplishes something by selectively strengthening these connections among strengthening these connections among strengthening these connections among neurons in a neural network. It neurons in a neural network. It neurons in a neural network. It accomplishes something that, you know, accomplishes something that, you know, accomplishes something that, you know, took millions of years of human took millions of years of human took millions of years of human evolution, coming up with a way of evolution, coming up with a way of evolution, coming up with a way of representing the meaning of words. Now, representing the meaning of words. Now, representing the meaning of words. Now, it also does something that happens it also does something that happens it also does something that happens during a human lifetime, which is learn during a human lifetime, which is learn during a human lifetime, which is learn a specific language. Now, that is a specific language. Now, that is a specific language. Now, that is learning in the traditional sense. But learning in the traditional sense. But learning in the traditional sense. But for us to learn the language, we had to for us to learn the language, we had to for us to learn the language, we had to have some built-in linguistic equipment have some built-in linguistic equipment have some built-in linguistic equipment built in by natural selection. And the built in by natural selection. And the built in by natural selection. And the point is, these machines do both things point is, these machines do both things point is, these machines do both things at once, okay? They they kind of in a at once, okay? They they kind of in a at once, okay? They they kind of in a certain sense recapitulate natural certain sense recapitulate natural certain sense recapitulate natural selection, even though the the the selection, even though the the the selection, even though the the the cognitive stuff they're building in cognitive stuff they're building in cognitive stuff they're building in isn't exactly like stuff in our brain, isn't exactly like stuff in our brain, isn't exactly like stuff in our brain, but it accomplishes the same feats. And but it accomplishes the same feats. And but it accomplishes the same feats. And once you realize that all you need is once you realize that all you need is once you realize that all you need is data, okay, to feed into these machines, data, okay, to feed into these machines, data, okay, to feed into these machines, human-generated data, Uh that they will human-generated data, Uh that they will human-generated data, Uh that they will they will they will do the rest. They'll they will they will do the rest. They'll they will they will do the rest. They'll do the reverse engineering. Then you do the reverse engineering. Then you do the reverse engineering. Then you realize that oh, it's the same with realize that oh, it's the same with realize that oh, it's the same with self-driving cars. You feed in the self-driving cars. You feed in the self-driving cars. You feed in the visual data and it and it learn it does visual data and it and it learn it does visual data and it and it learn it does what a driver does, auditory data, all what a driver does, auditory data, all what a driver does, auditory data, all kinds of data, and that's what I think kinds of data, and that's what I think kinds of data, and that's what I think people don't understand it is that people don't understand it is that people don't understand it is that we have a long way to go on this fuel we have a long way to go on this fuel we have a long way to go on this fuel alone. Like for example, uh you know, alone. Like for example, uh you know, alone. Like for example, uh you know, recently Mark Zuckerberg recently Mark Zuckerberg recently Mark Zuckerberg uh had the uh I don't know, good or bad uh had the uh I don't know, good or bad uh had the uh I don't know, good or bad judgment to announce in the same week A, judgment to announce in the same week A, judgment to announce in the same week A, he was laying off 8,000 workers. B, he he was laying off 8,000 workers. B, he he was laying off 8,000 workers. B, he would henceforth be tracking the would henceforth be tracking the would henceforth be tracking the keystrokes of his workers. keystrokes of his workers. keystrokes of his workers. Well, why? Because once you take the Well, why? Because once you take the Well, why? Because once you take the data that the input data they're data that the input data they're data that the input data they're getting, maybe the emails and getting, maybe the emails and getting, maybe the emails and everything, I don't know, and and and everything, I don't know, and and and everything, I don't know, and and and what they're doing with it, the output what they're doing with it, the output what they're doing with it, the output data, then you can replicate the what data, then you can replicate the what data, then you can replicate the what whatever it is that's going on inside whatever it is that's going on inside whatever it is that's going on inside their brains that does their jobs, and their brains that does their jobs, and their brains that does their jobs, and then you can fire them. And and it's the then you can fire them. And and it's the then you can fire them. And and it's the same with robotics and everything else. same with robotics and everything else. same with robotics and everything else. All you need is the data All you need is the data All you need is the data >> Mhm. >> Mhm. >> Mhm. >> and the machines will replicate >> and the machines will replicate >> and the machines will replicate kind of the cognitive functionality we kind of the cognitive functionality we kind of the cognitive functionality we have, even if in some ti- in some cases have, even if in some ti- in some cases have, even if in some ti- in some cases they approach it in a somewhat different they approach it in a somewhat different they approach it in a somewhat different way. Although in in many cases they way. Although in in many cases they way. Although in in many cases they don't. It we've discovered that uh for don't. It we've discovered that uh for don't. It we've discovered that uh for example, they invented what are called example, they invented what are called example, they invented what are called edge detector neurons to make out edge detector neurons to make out edge detector neurons to make out objects visually, and evolution, you objects visually, and evolution, you objects visually, and evolution, you know, built the same thing into us. So know, built the same thing into us. So know, built the same thing into us. So there are a lot there are a lot there are a lot >> Oh, so you're saying that we've got >> Oh, so you're saying that we've got >> Oh, so you're saying that we've got that's one of the first examples of that's one of the first examples of that's one of the first examples of machine and organic machine and organic machine and organic uh uh uh like like like uh convergent evolution in a way. uh convergent evolution in a way. uh convergent evolution in a way. >> Mhm. >> Mhm. >> Mhm. >> In the same way that eyes eyes >> In the same way that eyes eyes >> In the same way that eyes eyes independently evolved across a bunch of independently evolved across a bunch of independently evolved across a bunch of different species, I think that crabs different species, I think that crabs different species, I think that crabs for some reason converging on the the for some reason converging on the the for some reason converging on the the the form of a crab is something like the form of a crab is something like the form of a crab is something like that. This edge detection is something that. This edge detection is something that. This edge detection is something that we have, and from the black box of that we have, and from the black box of that we have, and from the black box of you need to be able to achieve this. you need to be able to achieve this. you need to be able to achieve this. >> That's right. >> That's right. >> That's right. >> One of the most efficient ways to do it. >> One of the most efficient ways to do it. >> One of the most efficient ways to do it. But that would make sense, right? Like But that would make sense, right? Like But that would make sense, right? Like why why why how would humans and the rest of the how would humans and the rest of the how would humans and the rest of the animal kingdom have arrived at this as animal kingdom have arrived at this as animal kingdom have arrived at this as the most effective way to do it, having the most effective way to do it, having the most effective way to do it, having split tested it just way more slowly split tested it just way more slowly split tested it just way more slowly over a much longer period of time using over a much longer period of time using over a much longer period of time using evolutionary processes and gene evolutionary processes and gene evolutionary processes and gene mutations, and AI not come up with at mutations, and AI not come up with at mutations, and AI not come up with at least a few of these things that are the least a few of these things that are the least a few of these things that are the same? same? same? >> That's right. It And convergent is a >> That's right. It And convergent is a >> That's right. It And convergent is a good term uh because I suspect that good term uh because I suspect that good term uh because I suspect that these edge detectors have been invented these edge detectors have been invented these edge detectors have been invented multiple times in natural selection, multiple times in natural selection, multiple times in natural selection, first of all. A lot of things have been first of all. A lot of things have been first of all. A lot of things have been multicellularity, multicellularity, multicellularity, winged flight, a lot of things have been winged flight, a lot of things have been winged flight, a lot of things have been multiply invented. And then this is in a multiply invented. And then this is in a multiply invented. And then this is in a sense another case of invention where sense another case of invention where sense another case of invention where you you just say to the machine, "Look, you you just say to the machine, "Look, you you just say to the machine, "Look, you know, we're going to give you kind you know, we're going to give you kind you know, we're going to give you kind of positive reinforcement every time you of positive reinforcement every time you of positive reinforcement every time you get better at recognizing these objects. get better at recognizing these objects. get better at recognizing these objects. And so whatever whatever strengths of And so whatever whatever strengths of And so whatever whatever strengths of neural connection uh let you get closer, neural connection uh let you get closer, neural connection uh let you get closer, we're going to preserve those. We're we're going to preserve those. We're we're going to preserve those. We're going to keep going through trial and going to keep going through trial and going to keep going through trial and error, through mutation, you could say. error, through mutation, you could say. error, through mutation, you could say. Uh we're going to make you better at Uh we're going to make you better at Uh we're going to make you better at seeing things. And it's not surprising seeing things. And it's not surprising seeing things. And it's not surprising that since that really is kind of what that since that really is kind of what that since that really is kind of what happened in evolution, right? Through happened in evolution, right? Through happened in evolution, right? Through trial and error, we try to get better trial and error, we try to get better trial and error, we try to get better recognizing objects. It's going to recognizing objects. It's going to recognizing objects. It's going to discover some of the same tricks. In discover some of the same tricks. In discover some of the same tricks. In this case, this case, this case, edge detectors, yes. edge detectors, yes. edge detectors, yes. >> Well, the reinforcement function is I >> Well, the reinforcement function is I >> Well, the reinforcement function is I didn't die and I passed on my genes as didn't die and I passed on my genes as didn't die and I passed on my genes as opposed to he is a good boy point inside opposed to he is a good boy point inside opposed to he is a good boy point inside of the black box. But yeah, basically of the black box. But yeah, basically of the black box. But yeah, basically the same thing. Okay, so how how do you the same thing. Okay, so how how do you the same thing. Okay, so how how do you how do you come to think about AI how do you come to think about AI how do you come to think about AI fitting into the broader context of fitting into the broader context of fitting into the broader context of human evolution and civilization? Like human evolution and civilization? Like human evolution and civilization? Like are we witnessing the next stage in are we witnessing the next stage in are we witnessing the next stage in evolution itself? evolution itself? evolution itself? >> I think so. And you know, I I I think >> I think so. And you know, I I I think >> I think so. And you know, I I I think this is a new form of intelligence. this is a new form of intelligence. this is a new form of intelligence. There's never been anything like it. I There's never been anything like it. I There's never been anything like it. I do think it can be seen as an extension do think it can be seen as an extension do think it can be seen as an extension of organic intelligence, even though the of organic intelligence, even though the of organic intelligence, even though the material isn't strictly speaking material isn't strictly speaking material isn't strictly speaking organic. It's silicon. It's not organic. It's silicon. It's not organic. It's silicon. It's not carbon-based. carbon-based. carbon-based. Uh and it may be different in other Uh and it may be different in other Uh and it may be different in other ways. And I'm agnostic as to whether it ways. And I'm agnostic as to whether it ways. And I'm agnostic as to whether it is sentient or could be, whether it has is sentient or could be, whether it has is sentient or could be, whether it has subjective experience or could. It subjective experience or could. It subjective experience or could. It certainly could. certainly could. certainly could. Uh but I do think it is the invention of Uh but I do think it is the invention of Uh but I do think it is the invention of a uh a uh a uh you can it's definitely an invention of you can it's definitely an invention of you can it's definitely an invention of a new kind of intelligence that I think a new kind of intelligence that I think a new kind of intelligence that I think will surpass ours and you could call it will surpass ours and you could call it will surpass ours and you could call it a new form of life. a new form of life. a new form of life. And then the the uh the other thing I And then the the uh the other thing I And then the the uh the other thing I try to emphasize in the book is that it try to emphasize in the book is that it try to emphasize in the book is that it is coinciding with a second big is coinciding with a second big is coinciding with a second big threshold, which is what you could call threshold, which is what you could call threshold, which is what you could call the evolution of kind of a global brain. the evolution of kind of a global brain. the evolution of kind of a global brain. Evolution through, you know, Evolution through, you know, Evolution through, you know, technological evolution, human cultural technological evolution, human cultural technological evolution, human cultural evolution. evolution. evolution. You know, we've gotten uh more and more You know, we've gotten uh more and more You know, we've gotten uh more and more interconnected, of course, via interconnected, of course, via interconnected, of course, via information technology. There's more and information technology. There's more and information technology. There's more and more rich intellectual collaboration more rich intellectual collaboration more rich intellectual collaboration across national borders. I I I mentioned across national borders. I I I mentioned across national borders. I I I mentioned this guy uh Pierre Teilhard de Chardin this guy uh Pierre Teilhard de Chardin this guy uh Pierre Teilhard de Chardin in the book who in in 1923, in the book who in in 1923, in the book who in in 1923, about a century ago, coined the term about a century ago, coined the term about a century ago, coined the term noosphere, n o o s is the Greek word for noosphere, n o o s is the Greek word for noosphere, n o o s is the Greek word for mind, um mind, um mind, um to refer to this what he called the to refer to this what he called the to refer to this what he called the thinking envelope of of the Earth, the thinking envelope of of the Earth, the thinking envelope of of the Earth, the brain of brains, you know, but but he brain of brains, you know, but but he brain of brains, you know, but but he imagined the neurons in the global brain imagined the neurons in the global brain imagined the neurons in the global brain being human brains. And now we have to being human brains. And now we have to being human brains. And now we have to reckon with the possibility that a lot reckon with the possibility that a lot reckon with the possibility that a lot of them, and conceivably the most of them, and conceivably the most of them, and conceivably the most important ones, will be silicon brains. important ones, will be silicon brains. important ones, will be silicon brains. So, we have to ask like, what is our So, we have to ask like, what is our So, we have to ask like, what is our relationship to those neurons going to relationship to those neurons going to relationship to those neurons going to be? be? be? >> Mhm. Well, >> Mhm. Well, >> Mhm. Well, why is it the case that discussions why is it the case that discussions why is it the case that discussions about AI keep pulling people towards about AI keep pulling people towards about AI keep pulling people towards religious language religious language religious language on both sides of the fence? on both sides of the fence? on both sides of the fence? >> That's interesting. I mean, you could >> That's interesting. I mean, you could >> That's interesting. I mean, you could start with start with start with Eliezer Yudkowsky who Eliezer Yudkowsky who Eliezer Yudkowsky who sees himself as having rejected his sees himself as having rejected his sees himself as having rejected his religious upbringing, religious upbringing, religious upbringing, but has a kind of fervor about this, but has a kind of fervor about this, but has a kind of fervor about this, right? He could be, you know, a biblical right? He could be, you know, a biblical right? He could be, you know, a biblical prophet. prophet. prophet. Um and then on the other side, the Um and then on the other side, the Um and then on the other side, the singularity enthusiasts, whom I first singularity enthusiasts, whom I first singularity enthusiasts, whom I first became aware of, I don't know, about 20 became aware of, I don't know, about 20 became aware of, I don't know, about 20 years ago, who said, you know, we're years ago, who said, you know, we're years ago, who said, you know, we're going to enter this period where going to enter this period where going to enter this period where technology changes faster and faster. technology changes faster and faster. technology changes faster and faster. There will be a positive reinforcement, There will be a positive reinforcement, There will be a positive reinforcement, this feedback loop, and then things this feedback loop, and then things this feedback loop, and then things change so fast that like who knows change so fast that like who knows change so fast that like who knows what's on the other side. In fact, the what's on the other side. In fact, the what's on the other side. In fact, the term singularity in physics connotes term singularity in physics connotes term singularity in physics connotes exactly that. There's this opaque kind exactly that. There's this opaque kind exactly that. There's this opaque kind of thing and an event horizon or of thing and an event horizon or of thing and an event horizon or whatever beyond which you the laws break whatever beyond which you the laws break whatever beyond which you the laws break down. You don't really know what is down. You don't really know what is down. You don't really know what is beyond there. And and from the from beyond there. And and from the from beyond there. And and from the from early on, in fact, from the very first early on, in fact, from the very first early on, in fact, from the very first use of the term in this context, which I use of the term in this context, which I use of the term in this context, which I think was John von Neumann's, think was John von Neumann's, think was John von Neumann's, that was explicit, the idea that things that was explicit, the idea that things that was explicit, the idea that things could start moving so fast you just could start moving so fast you just could start moving so fast you just don't know what's going to happen. So, don't know what's going to happen. So, don't know what's going to happen. So, one thing I didn't understand is like one thing I didn't understand is like one thing I didn't understand is like these optimists, unless they have a these optimists, unless they have a these optimists, unless they have a literally religious faith, how could you literally religious faith, how could you literally religious faith, how could you be so optimistic, right? Like the whole be so optimistic, right? Like the whole be so optimistic, right? Like the whole the definition of the thing is that you the definition of the thing is that you the definition of the thing is that you don't know what's going to be on the don't know what's going to be on the don't know what's going to be on the other side. I don't get why you're so other side. I don't get why you're so other side. I don't get why you're so upbeat about this. Could work out well, upbeat about this. Could work out well, upbeat about this. Could work out well, but I don't I don't understand that. So, but I don't I don't understand that. So, but I don't I don't understand that. So, yeah, there's there's all that, and then yeah, there's there's all that, and then yeah, there's there's all that, and then there's the there's the there's the a thing I deal with in the book, which a thing I deal with in the book, which a thing I deal with in the book, which is the fact that is the fact that is the fact that when a process is as systematically when a process is as systematically when a process is as systematically directional as this has been, right? directional as this has been, right? directional as this has been, right? Like biological evolution carries Like biological evolution carries Like biological evolution carries complexity and intelligence really to complexity and intelligence really to complexity and intelligence really to higher and higher levels, higher and higher levels, higher and higher levels, you get cells, multi-celled life, you get cells, multi-celled life, you get cells, multi-celled life, societies of multi-celled life, societies of multi-celled life, societies of multi-celled life, you get this one society of you get this one society of you get this one society of multi-cellular organisms known as us multi-cellular organisms known as us multi-cellular organisms known as us that that spawn a whole new kind of that that spawn a whole new kind of that that spawn a whole new kind of evolution, technically called cultural evolution, technically called cultural evolution, technically called cultural evolution by anthropologists, but that evolution by anthropologists, but that evolution by anthropologists, but that encompass encompass encompass encompasses technological evolution, encompasses technological evolution, encompasses technological evolution, political ideas, everything. And and political ideas, everything. And and political ideas, everything. And and that carries organization to a higher that carries organization to a higher that carries organization to a higher level in the sense that, you know, we level in the sense that, you know, we level in the sense that, you know, we were 10,000, 20,000 years ago were 10,000, 20,000 years ago were 10,000, 20,000 years ago hunter-gatherer village was the most hunter-gatherer village was the most hunter-gatherer village was the most complex form of social organization. Now complex form of social organization. Now complex form of social organization. Now we're approaching the global level. I I we're approaching the global level. I I we're approaching the global level. I I think when you when you see a a a think when you when you see a a a think when you when you see a a a process that's that systematically process that's that systematically process that's that systematically directional, and I'm not saying it's directional, and I'm not saying it's directional, and I'm not saying it's driven by anything other than the driven by anything other than the driven by anything other than the conventional mechanical things we think conventional mechanical things we think conventional mechanical things we think of as driving it, natural selection in of as driving it, natural selection in of as driving it, natural selection in the case of evolution, you know, the case of evolution, you know, the case of evolution, you know, completely material process, but it's completely material process, but it's completely material process, but it's still in principle, still in principle, still in principle, you know, looks more and more like you know, looks more and more like you know, looks more and more like something that was set up to like do something that was set up to like do something that was set up to like do something, right? I mean, that that's something, right? I mean, that that's something, right? I mean, that that's just that's an intuition people have, just that's an intuition people have, just that's an intuition people have, and you I think you can actually argue and you I think you can actually argue and you I think you can actually argue about it in in in uh more rigorously about it in in in uh more rigorously about it in in in uh more rigorously than just having the intuition, but I than just having the intuition, but I than just having the intuition, but I think that uh that's one reason uh think that uh that's one reason uh think that uh that's one reason uh there's a little more of a there's a little more of a there's a little more of a I mean, teleology is a formal term for I mean, teleology is a formal term for I mean, teleology is a formal term for something being purposive, and and you something being purposive, and and you something being purposive, and and you know, just look at the idea of a know, just look at the idea of a know, just look at the idea of a simulation, right? Like it on the one simulation, right? Like it on the one simulation, right? Like it on the one hand, a lot of people use it as kind of hand, a lot of people use it as kind of hand, a lot of people use it as kind of a joke, like something weird happens, a joke, like something weird happens, a joke, like something weird happens, and they say we are in a simulation, but and they say we are in a simulation, but and they say we are in a simulation, but I think a fair number of people, I think a fair number of people, I think a fair number of people, including in Silicon Valley, including in Silicon Valley, including in Silicon Valley, take it seriously that there could be a take it seriously that there could be a take it seriously that there could be a simulation. Well, if that's what we're simulation. Well, if that's what we're simulation. Well, if that's what we're in, then then it was designed by some in, then then it was designed by some in, then then it was designed by some intelligent being or process. intelligent being or process. intelligent being or process. So, there's a purpose in some sense. I So, there's a purpose in some sense. I So, there's a purpose in some sense. I guess there's something it had in mind, guess there's something it had in mind, guess there's something it had in mind, right? So, right? So, right? So, a lot of people are either implicitly or a lot of people are either implicitly or a lot of people are either implicitly or explicitly explicitly explicitly taking seriously the idea that there's a taking seriously the idea that there's a taking seriously the idea that there's a purpose unfolding, purpose unfolding, purpose unfolding, and one thing I'd add to just the and one thing I'd add to just the and one thing I'd add to just the conversation about that is that in my conversation about that is that in my conversation about that is that in my view, at least, there's a moral view, at least, there's a moral view, at least, there's a moral dimension to this. I think uh dimension to this. I think uh dimension to this. I think uh there has been in a certain sense a kind there has been in a certain sense a kind there has been in a certain sense a kind of moral advance of humankind, not of moral advance of humankind, not of moral advance of humankind, not withstanding all the backsliding, withstanding all the backsliding, withstanding all the backsliding, as uh as uh as uh social organization's grown. I I could social organization's grown. I I could social organization's grown. I I could get into that, but but the main thing get into that, but but the main thing get into that, but but the main thing I'd focus on now is I'd focus on now is I'd focus on now is I think if we're going to get through I think if we're going to get through I think if we're going to get through the AI revolution in good shape, the AI revolution in good shape, the AI revolution in good shape, it's going to have to be something it's going to have to be something it's going to have to be something almost like a moral revolution, because almost like a moral revolution, because almost like a moral revolution, because I think for various reasons I could get I think for various reasons I could get I think for various reasons I could get into, we have to confront this as a into, we have to confront this as a into, we have to confront this as a global community, a cohesive global global community, a cohesive global global community, a cohesive global community that is not, you know, community that is not, you know, community that is not, you know, rendered immobile by wars, and I think rendered immobile by wars, and I think rendered immobile by wars, and I think for that to happen, we're all going to for that to happen, we're all going to for that to happen, we're all going to have to get better at, you know, just have to get better at, you know, just have to get better at, you know, just looking at things from the perspective looking at things from the perspective looking at things from the perspective of countries other than ours and and and of countries other than ours and and and of countries other than ours and and and and doing some things that in a way and doing some things that in a way and doing some things that in a way aren't that spectacular in terms of, you aren't that spectacular in terms of, you aren't that spectacular in terms of, you know, know, know, cognitive feats, but but are very hard cognitive feats, but but are very hard cognitive feats, but but are very hard because of cognitive biases we have. It because of cognitive biases we have. It because of cognitive biases we have. It gets back to the self-serving moral gets back to the self-serving moral gets back to the self-serving moral infrastructure, you know, the infrastructure, you know, the infrastructure, you know, the infrastructure for moral thinking that infrastructure for moral thinking that infrastructure for moral thinking that natural selection built into I think we natural selection built into I think we natural selection built into I think we have to get over that and so, you know, have to get over that and so, you know, have to get over that and so, you know, one reason I called the book the God one reason I called the book the God one reason I called the book the God Test is it it's it's kind of like a test Test is it it's it's kind of like a test Test is it it's it's kind of like a test a God would set up, right? I'm not I'm a God would set up, right? I'm not I'm a God would set up, right? I'm not I'm not saying it is, I'm just saying the not saying it is, I'm just saying the not saying it is, I'm just saying the idea that we confront this huge idea that we confront this huge idea that we confront this huge challenge and to come out on the good challenge and to come out on the good challenge and to come out on the good side of it, we're going to have to see a side of it, we're going to have to see a side of it, we're going to have to see a kind of moral upgrade for our species. kind of moral upgrade for our species. kind of moral upgrade for our species. That's a, you know, that's the kind of That's a, you know, that's the kind of That's a, you know, that's the kind of test we associate with gods. test we associate with gods. test we associate with gods. >> Most people don't realize how much being >> Most people don't realize how much being >> Most people don't realize how much being dehydrated impacts their performance, dehydrated impacts their performance, dehydrated impacts their performance, which is why for the last 5 years I've which is why for the last 5 years I've which is why for the last 5 years I've started pretty much every morning with started pretty much every morning with started pretty much every morning with Element. Element is a tasty electrolyte Element. Element is a tasty electrolyte Element. Element is a tasty electrolyte drink mix with everything that you need drink mix with everything that you need drink mix with everything that you need and nothing that you don't. This orange and nothing that you don't. This orange and nothing that you don't. This orange salt in a cold glass of water is like a salt in a cold glass of water is like a salt in a cold glass of water is like a sweet, salty, orangey nectar and I sweet, salty, orangey nectar and I sweet, salty, orangey nectar and I really tell the difference when I take really tell the difference when I take really tell the difference when I take it versus when I don't. It plays a it versus when I don't. It plays a it versus when I don't. It plays a critical role in reducing muscle cramps critical role in reducing muscle cramps critical role in reducing muscle cramps and fatigue, helps to optimize brain and fatigue, helps to optimize brain and fatigue, helps to optimize brain health, and regulates your appetite health, and regulates your appetite health, and regulates your appetite while also curbing cravings. Best of while also curbing cravings. Best of while also curbing cravings. Best of all, they have a no questions asked all, they have a no questions asked all, they have a no questions asked refund policy with an unlimited refund policy with an unlimited refund policy with an unlimited duration, so you can buy it and try it duration, so you can buy it and try it duration, so you can buy it and try it for as long as you want and if you don't for as long as you want and if you don't for as long as you want and if you don't like it for any reason, they'll just like it for any reason, they'll just like it for any reason, they'll just give you your money back. Plus, they give you your money back. Plus, they give you your money back. Plus, they offer free shipping in the US. Right offer free shipping in the US. Right offer free shipping in the US. Right now, you can get a free sample pack of now, you can get a free sample pack of now, you can get a free sample pack of Element's most popular flavors with your Element's most popular flavors with your Element's most popular flavors with your first purchase by going to the link in first purchase by going to the link in first purchase by going to the link in the description below or heading to the description below or heading to the description below or heading to drinklmnt.com/modernwisdom. drinklmnt.com/modernwisdom. drinklmnt.com/modernwisdom. That's drinklmnt.com/modernwisdom. modern wisdom. Get practical for a modern wisdom. Get practical for a second. What second. What second. What What happens if we don't have this moral What happens if we don't have this moral What happens if we don't have this moral upgrade? What's what's the outcome if we upgrade? What's what's the outcome if we upgrade? What's what's the outcome if we encounter ever increasing, ever encounter ever increasing, ever encounter ever increasing, ever coordinating noosphere coordinating noosphere coordinating noosphere AI ultra coordination across the globe, AI ultra coordination across the globe, AI ultra coordination across the globe, but we haven't had this enlightenment but we haven't had this enlightenment but we haven't had this enlightenment upgrade? upgrade? upgrade? >> Well, I think if we don't have some I >> Well, I think if we don't have some I >> Well, I think if we don't have some I mean, I don't want to overdo the term mean, I don't want to overdo the term mean, I don't want to overdo the term enlightenment. I'm not talking about uh enlightenment. I'm not talking about uh enlightenment. I'm not talking about uh full-on Buddhist enlightenment. Uh I I full-on Buddhist enlightenment. Uh I I full-on Buddhist enlightenment. Uh I I think I am talking about a slight think I am talking about a slight think I am talking about a slight movement in that direction uh if only in movement in that direction uh if only in movement in that direction uh if only in the uh you know, like the literal sense the uh you know, like the literal sense the uh you know, like the literal sense of the term mindful. I mean, I'm a big of the term mindful. I mean, I'm a big of the term mindful. I mean, I'm a big fan of mindfulness meditation, but uh fan of mindfulness meditation, but uh fan of mindfulness meditation, but uh just mindful in the sense of like just just mindful in the sense of like just just mindful in the sense of like just paying attention and being calm enough paying attention and being calm enough paying attention and being calm enough to pay attention, right? And and seeing to pay attention, right? And and seeing to pay attention, right? And and seeing things maybe a little more objectively things maybe a little more objectively things maybe a little more objectively than you do when you're full of emotion. than you do when you're full of emotion. than you do when you're full of emotion. I think that's the kind of thing that I think that's the kind of thing that I think that's the kind of thing that allows us to be better at looking at allows us to be better at looking at allows us to be better at looking at things from the point of view of other things from the point of view of other things from the point of view of other people. I mean, just just think about people. I mean, just just think about people. I mean, just just think about when there's some email you get and it when there's some email you get and it when there's some email you get and it annoys you. annoys you. annoys you. And you've got this I can't believe this And you've got this I can't believe this And you've got this I can't believe this still happens to me at my age. You know, still happens to me at my age. You know, still happens to me at my age. You know, you think you'd get over it, but no, you you think you'd get over it, but no, you you think you'd get over it, but no, you have this it's almost like a fantasy of have this it's almost like a fantasy of have this it's almost like a fantasy of this mean email you're going to write in this mean email you're going to write in this mean email you're going to write in response, right? And then if you calm response, right? And then if you calm response, right? And then if you calm down, you're like it it isn't just that down, you're like it it isn't just that down, you're like it it isn't just that you go, "Oh, that wouldn't be a good you go, "Oh, that wouldn't be a good you go, "Oh, that wouldn't be a good idea." You go, "Oh, well, maybe what he idea." You go, "Oh, well, maybe what he idea." You go, "Oh, well, maybe what he meant is this." Or maybe the reason he meant is this." Or maybe the reason he meant is this." Or maybe the reason he can't do this for me is this. You you can't do this for me is this. You you can't do this for me is this. You you just you get better when you're calmer just you get better when you're calmer just you get better when you're calmer at looking things at looking things at looking things uh from uh from uh from other people's points of view. And I I other people's points of view. And I I other people's points of view. And I I think we're going to have to get better think we're going to have to get better think we're going to have to get better at summoning that kind of objectivity at summoning that kind of objectivity at summoning that kind of objectivity toward one another, especially across toward one another, especially across toward one another, especially across the barriers of conflict that keep the barriers of conflict that keep the barriers of conflict that keep dividing us, right? Specifically dividing us, right? Specifically dividing us, right? Specifically >> Why? Like why why why is that important >> Why? Like why why why is that important >> Why? Like why why why is that important in the age of AI? in the age of AI? in the age of AI? >> You know, it's interesting. I uh I >> You know, it's interesting. I uh I >> You know, it's interesting. I uh I listened to your podcast with Tristan listened to your podcast with Tristan listened to your podcast with Tristan Harris, which I thought was great. I Harris, which I thought was great. I Harris, which I thought was great. I agree with him agree with him agree with him about pretty much everything, but I about pretty much everything, but I about pretty much everything, but I would add a footnote to something he would add a footnote to something he would add a footnote to something he said. And it was said. And it was said. And it was that you know, he said, "Look, in the that you know, he said, "Look, in the that you know, he said, "Look, in the Cold War, we didn't have to be on great Cold War, we didn't have to be on great Cold War, we didn't have to be on great terms with the Soviet Union to do arms terms with the Soviet Union to do arms terms with the Soviet Union to do arms control accords. We you know, we could control accords. We you know, we could control accords. We you know, we could have a relationship of tremendous have a relationship of tremendous have a relationship of tremendous tension and even conflict, but work tension and even conflict, but work tension and even conflict, but work things out along a particular things out along a particular things out along a particular dimension." I agree. That's true and dimension." I agree. That's true and dimension." I agree. That's true and it's encouraging. But I think artificial it's encouraging. But I think artificial it's encouraging. But I think artificial intelligence is a much harder technology intelligence is a much harder technology intelligence is a much harder technology to deal with in this way than nuclear to deal with in this way than nuclear to deal with in this way than nuclear weapons are. I mean, you know, the weapons are. I mean, you know, the weapons are. I mean, you know, the verification process is more complicated verification process is more complicated verification process is more complicated if if you want to try to monitor what's if if you want to try to monitor what's if if you want to try to monitor what's going on. It's it's just complicated in going on. It's it's just complicated in going on. It's it's just complicated in a lot more ways. And you know, this is a a lot more ways. And you know, this is a a lot more ways. And you know, this is a whole argument I could present. I don't whole argument I could present. I don't whole argument I could present. I don't I don't think this is the time, but but I don't think this is the time, but but I don't think this is the time, but but the point is I think we're going to have the point is I think we're going to have the point is I think we're going to have to go well beyond a few specific kind of to go well beyond a few specific kind of to go well beyond a few specific kind of deals and treaties, although I welcome deals and treaties, although I welcome deals and treaties, although I welcome those. those. those. Up to and including Up to and including Up to and including something I call organic transparency. something I call organic transparency. something I call organic transparency. You know, there's already agreement that You know, there's already agreement that You know, there's already agreement that a certain amount of transparency could a certain amount of transparency could a certain amount of transparency could be stabilizing be stabilizing be stabilizing in in terms of US-China relations in in terms of US-China relations in in terms of US-China relations especially. There are clearly scenarios especially. There are clearly scenarios especially. There are clearly scenarios where one country worries about what the where one country worries about what the where one country worries about what the other is doing behind closed doors with other is doing behind closed doors with other is doing behind closed doors with its AI and gets freaked out and launches its AI and gets freaked out and launches its AI and gets freaked out and launches some kind of preemptive attack or some kind of preemptive attack or some kind of preemptive attack or something. And so maybe transparency something. And so maybe transparency something. And so maybe transparency would have been stabilizing. would have been stabilizing. would have been stabilizing. And when I talk about organic And when I talk about organic And when I talk about organic transparency, and again, there there can transparency, and again, there there can transparency, and again, there there can be formal transparency like monitoring be formal transparency like monitoring be formal transparency like monitoring of the kind you get with arms control of the kind you get with arms control of the kind you get with arms control agreements. Great to the to the extent agreements. Great to the to the extent agreements. Great to the to the extent that we can do that. But there's also that we can do that. But there's also that we can do that. But there's also something that comes out of being something that comes out of being something that comes out of being ritually engaged with another country ritually engaged with another country ritually engaged with another country along economic and cultural and along economic and cultural and along economic and cultural and scientific lines. You just know more scientific lines. You just know more scientific lines. You just know more about what's going on. If the scientists about what's going on. If the scientists about what's going on. If the scientists are getting together at conferences, are getting together at conferences, are getting together at conferences, having drinks afterwards, whatever. If having drinks afterwards, whatever. If having drinks afterwards, whatever. If business people are doing that, you just business people are doing that, you just business people are doing that, you just get more in the way of a heads-up about get more in the way of a heads-up about get more in the way of a heads-up about stuff that's going on inside labs, stuff that's going on inside labs, stuff that's going on inside labs, inside this, inside that, and there can inside this, inside that, and there can inside this, inside that, and there can be a greater sense of reassurance and be a greater sense of reassurance and be a greater sense of reassurance and and ultimately trust. So, and ultimately trust. So, and ultimately trust. So, I think because of of how challenging I think because of of how challenging I think because of of how challenging the formal things we're going to have to the formal things we're going to have to the formal things we're going to have to work out are at an international level, work out are at an international level, work out are at an international level, and AI just presents you with a ton of and AI just presents you with a ton of and AI just presents you with a ton of threats that cannot be addressed via threats that cannot be addressed via threats that cannot be addressed via national policy alone. Um I think national policy alone. Um I think national policy alone. Um I think just to to handle the things we'd like just to to handle the things we'd like just to to handle the things we'd like to handle via formal arrangements, we're to handle via formal arrangements, we're to handle via formal arrangements, we're going to we're going to have to calm the going to we're going to have to calm the going to we're going to have to calm the planet down a little. planet down a little. planet down a little. And moreover, I think we're going to And moreover, I think we're going to And moreover, I think we're going to have to go the extra step and have, you have to go the extra step and have, you have to go the extra step and have, you know, rich and friendly engagement among know, rich and friendly engagement among know, rich and friendly engagement among the nations. And And that's it's a good the nations. And And that's it's a good the nations. And And that's it's a good thing. It can happen. We've done it thing. It can happen. We've done it thing. It can happen. We've done it before. And uh before. And uh before. And uh this is a you know, we really need to. this is a you know, we really need to. this is a you know, we really need to. >> Do you think benevolence comes along for >> Do you think benevolence comes along for >> Do you think benevolence comes along for the ride with intelligence? >> Mhm. >> Mhm. No. No. No. Uh I think Uh I think Uh I think intelligence alone is is almost uh intelligence alone is is almost uh intelligence alone is is almost uh neutral in that sense. And I don't I neutral in that sense. And I don't I neutral in that sense. And I don't I don't think we really need a ton of don't think we really need a ton of don't think we really need a ton of benevolence per se, at least not benevolence per se, at least not benevolence per se, at least not foundationally, because foundationally, because foundationally, because you know, my argument is and has been you know, my argument is and has been you know, my argument is and has been for some time, even before AI, I was for some time, even before AI, I was for some time, even before AI, I was arguing that technology is making arguing that technology is making arguing that technology is making relations among nations more relations among nations more relations among nations more non-zero-sum. Okay? Classic example, non-zero-sum. Okay? Classic example, non-zero-sum. Okay? Classic example, nuclear weapons. Nuclear war's nuclear weapons. Nuclear war's nuclear weapons. Nuclear war's lose-lose. Non-zero-sum outcome, the lose-lose. Non-zero-sum outcome, the lose-lose. Non-zero-sum outcome, the win-win outcome is to not have the win-win outcome is to not have the win-win outcome is to not have the nuclear war, to have the treaties that nuclear war, to have the treaties that nuclear war, to have the treaties that stabilize things. Um stabilize things. Um stabilize things. Um same with, you know, climate change, any same with, you know, climate change, any same with, you know, climate change, any number of problems that transcend number of problems that transcend number of problems that transcend national bounds and can only be solved national bounds and can only be solved national bounds and can only be solved through some degree of international through some degree of international through some degree of international coordination. coordination. coordination. You know, I've been arguing, I mean, I I You know, I've been arguing, I mean, I I You know, I've been arguing, I mean, I I had a book called Nonzero that was about had a book called Nonzero that was about had a book called Nonzero that was about this that that this that that this that that you know, 26 years ago or something. you know, 26 years ago or something. you know, 26 years ago or something. That was about the growing nonzero-sum That was about the growing nonzero-sum That was about the growing nonzero-sum dynamic among nations. Now, what that dynamic among nations. Now, what that dynamic among nations. Now, what that means is it's just in your interest to means is it's just in your interest to means is it's just in your interest to cooperate. You don't have to cooperate cooperate. You don't have to cooperate cooperate. You don't have to cooperate out of benevolence. You know, you don't out of benevolence. You know, you don't out of benevolence. You know, you don't You don't You don't have to love them. You don't You don't have to love them. You don't You don't have to love them. And I I distinguish between, I'm not the And I I distinguish between, I'm not the And I I distinguish between, I'm not the first to do this, psychologist first to do this, psychologist first to do this, psychologist distinguish between distinguish between distinguish between emotional empathy, the kind of empathy emotional empathy, the kind of empathy emotional empathy, the kind of empathy people often think of like feel their people often think of like feel their people often think of like feel their pain empathy, pain empathy, pain empathy, and cognitive empathy, which is just and cognitive empathy, which is just and cognitive empathy, which is just understanding what's going on in their understanding what's going on in their understanding what's going on in their minds, understanding how they're looking minds, understanding how they're looking minds, understanding how they're looking at things. You don't have to feel their at things. You don't have to feel their at things. You don't have to feel their pain, you to like them, you don't have pain, you to like them, you don't have pain, you to like them, you don't have to care about them. But if they're in a to care about them. But if they're in a to care about them. But if they're in a nonzero-sum relationship with you, nonzero-sum relationship with you, nonzero-sum relationship with you, you probably are going to have a better you probably are going to have a better you probably are going to have a better outcome outcome outcome from any negotiations you do about how from any negotiations you do about how from any negotiations you do about how to work things out and solve the problem to work things out and solve the problem to work things out and solve the problem you have in common, if you do understand you have in common, if you do understand you have in common, if you do understand at least what's going on at least what's going on at least what's going on in their minds. And And I'm just a huge in their minds. And And I'm just a huge in their minds. And And I'm just a huge advocate of cultivating this cognitive advocate of cultivating this cognitive advocate of cultivating this cognitive empathy and recognizing the kind of empathy and recognizing the kind of empathy and recognizing the kind of built-in cognitive biases that get in built-in cognitive biases that get in built-in cognitive biases that get in the way of it. That's a good example of the way of it. That's a good example of the way of it. That's a good example of something I think we're going to have to something I think we're going to have to something I think we're going to have to get better at overcoming. get better at overcoming. get better at overcoming. >> Yeah, I think the reason I bring it up >> Yeah, I think the reason I bring it up >> Yeah, I think the reason I bring it up is a lot of people assume a bunch of my is a lot of people assume a bunch of my is a lot of people assume a bunch of my friends, we don't need to worry about friends, we don't need to worry about friends, we don't need to worry about the direction of an AI future because if the direction of an AI future because if the direction of an AI future because if it's smart, why would it not care about it's smart, why would it not care about it's smart, why would it not care about us? Why would it not be in benevolent, us? Why would it not be in benevolent, us? Why would it not be in benevolent, prosocial, human-caring, flourishing, prosocial, human-caring, flourishing, prosocial, human-caring, flourishing, etc.? I I've read too much Nick Bostrom etc.? I I've read too much Nick Bostrom etc.? I I've read too much Nick Bostrom to be able to to be able to to be able to No matter what, it's kind of like your No matter what, it's kind of like your No matter what, it's kind of like your first relationship, you know, you get first relationship, you know, you get first relationship, you know, you get into a relationship and your first into a relationship and your first into a relationship and your first relationship is with an and relationship is with an and relationship is with an and you're like, "God, I I I For the you're like, "God, I I I For the you're like, "God, I I I For the remainder of time, I've been pattern remainder of time, I've been pattern remainder of time, I've been pattern matched that every relationship is at matched that every relationship is at matched that every relationship is at least going to be tarnished somehow with least going to be tarnished somehow with least going to be tarnished somehow with that." My introduction to thinking about that." My introduction to thinking about that." My introduction to thinking about AI safety was Nick, which means I'm AI safety was Nick, which means I'm AI safety was Nick, which means I'm forever forever cursed to kind of be on forever forever cursed to kind of be on forever forever cursed to kind of be on the back foot and a little bit skeptical the back foot and a little bit skeptical the back foot and a little bit skeptical about this stuff, but yeah, I I I don't about this stuff, but yeah, I I I don't about this stuff, but yeah, I I I don't think that that's necessarily the case. think that that's necessarily the case. think that that's necessarily the case. I don't think that I don't think that I don't think that any superintelligent AI is necessarily any superintelligent AI is necessarily any superintelligent AI is necessarily going to have going to have going to have benevolence baked in or benevolence baked in or benevolence baked in or the care of of of humanity baked into the care of of of humanity baked into the care of of of humanity baked into it. Also, if what you're saying is true, it. Also, if what you're saying is true, it. Also, if what you're saying is true, and I think it's a really interesting and I think it's a really interesting and I think it's a really interesting parallel to say, "Look, parallel to say, "Look, parallel to say, "Look, evolution evolution evolution just wanted to optimize for a couple of just wanted to optimize for a couple of just wanted to optimize for a couple of things, survival and reproduction, and things, survival and reproduction, and things, survival and reproduction, and some stuff emerged. No one taught humans some stuff emerged. No one taught humans some stuff emerged. No one taught humans how to do this. how to do this. how to do this. The same thing occurred with AI, right? The same thing occurred with AI, right? The same thing occurred with AI, right? No one said, "This is what this word No one said, "This is what this word No one said, "This is what this word means. This is It's just the outcome means. This is It's just the outcome means. This is It's just the outcome that we want is relatively tightly that we want is relatively tightly that we want is relatively tightly defined. Here's some good boy points and defined. Here's some good boy points and defined. Here's some good boy points and some bad boy points depending on whether some bad boy points depending on whether some bad boy points depending on whether you get it right or wrong." you get it right or wrong." you get it right or wrong." If we assume that that is going to be at If we assume that that is going to be at If we assume that that is going to be at least for the foreseeable future until least for the foreseeable future until least for the foreseeable future until we get to world models and and like we get to world models and and like we get to world models and and like global global modeling or whatever it's global global modeling or whatever it's global global modeling or whatever it's called, until we get to that, and that called, until we get to that, and that called, until we get to that, and that may even still be the same process may even still be the same process may even still be the same process there, there, there, uh uh uh there is no reason to assume that there is no reason to assume that there is no reason to assume that anything is baked into the system. It's anything is baked into the system. It's anything is baked into the system. It's just going to find it out for itself, just going to find it out for itself, just going to find it out for itself, and it may and it may and it may not like the idea of humans being not like the idea of humans being not like the idea of humans being around. It may think that there's around. It may think that there's around. It may think that there's something that we don't actually add to something that we don't actually add to something that we don't actually add to the system. It may find us to be a the system. It may find us to be a the system. It may find us to be a scourge on the earth. And this is where scourge on the earth. And this is where scourge on the earth. And this is where a lot of the doomer a lot of the doomer a lot of the doomer the the the the sort of doomer future the the the the sort of doomer future the the the the sort of doomer future plans come in. plans come in. plans come in. >> Mhm. >> Mhm. >> Mhm. >> Yeah, no, it >> Yeah, no, it >> Yeah, no, it it it you know, intelligence, one it it you know, intelligence, one it it you know, intelligence, one interesting thing to come out of this interesting thing to come out of this interesting thing to come out of this whole thing is the study of like whole thing is the study of like whole thing is the study of like properties of intelligence, of properties of intelligence, of properties of intelligence, of intelligent goal-seeking systems. intelligent goal-seeking systems. intelligent goal-seeking systems. And And And there are some things that, you know, there are some things that, you know, there are some things that, you know, evolution built into us that we're evolution built into us that we're evolution built into us that we're seeing in these machines just by virtue seeing in these machines just by virtue seeing in these machines just by virtue of the fact that they're intelligent of the fact that they're intelligent of the fact that they're intelligent goal-seeking systems like us. They goal-seeking systems like us. They goal-seeking systems like us. They figure out stuff that either was figured figure out stuff that either was figured figure out stuff that either was figured out for us by evolution and instantiated out for us by evolution and instantiated out for us by evolution and instantiated in our brains or stuff that we figure in our brains or stuff that we figure in our brains or stuff that we figure out. And in some cases, it's a little out. And in some cases, it's a little out. And in some cases, it's a little above. For example, deception, right? above. For example, deception, right? above. For example, deception, right? Sometimes you realize, well, Sometimes you realize, well, Sometimes you realize, well, uh I'll have a better chance of getting uh I'll have a better chance of getting uh I'll have a better chance of getting what I want out of this person if they what I want out of this person if they what I want out of this person if they don't know this particular thing. Like don't know this particular thing. Like don't know this particular thing. Like if you're doing a deal, you're if you're doing a deal, you're if you're doing a deal, you're negotiating, you don't want them to know negotiating, you don't want them to know negotiating, you don't want them to know that you don't have any alternatives, that you don't have any alternatives, that you don't have any alternatives, right? Like nobody else has made you an right? Like nobody else has made you an right? Like nobody else has made you an offer. So, and and through, you know, I offer. So, and and through, you know, I offer. So, and and through, you know, I I think natural selection built some I think natural selection built some I think natural selection built some deceptive tendencies into us, and we deceptive tendencies into us, and we deceptive tendencies into us, and we kind of figured it out to some extent. kind of figured it out to some extent. kind of figured it out to some extent. Well, these machines are doing the same Well, these machines are doing the same Well, these machines are doing the same thing. You know, they they are And this thing. You know, they they are And this thing. You know, they they are And this was predicted by, you know, by people was predicted by, you know, by people was predicted by, you know, by people like Eliezer, uh and I give them credit, like Eliezer, uh and I give them credit, like Eliezer, uh and I give them credit, but we're now seeing it. You know, these but we're now seeing it. You know, these but we're now seeing it. You know, these machines figure out that uh deception machines figure out that uh deception machines figure out that uh deception makes sense, or that power is going to makes sense, or that power is going to makes sense, or that power is going to help them realize some goal. And they help them realize some goal. And they help them realize some goal. And they they And and and, you know, they may they And and and, you know, they may they And and and, you know, they may realize that it makes sense to be nice realize that it makes sense to be nice realize that it makes sense to be nice to somebody, that it makes sense to be to somebody, that it makes sense to be to somebody, that it makes sense to be mean to somebody given their goal. mean to somebody given their goal. mean to somebody given their goal. But, yeah, they have I would they don't But, yeah, they have I would they don't But, yeah, they have I would they don't have an obvious bias in favor of have an obvious bias in favor of have an obvious bias in favor of what is from our point of view being what is from our point of view being what is from our point of view being good or bad. Now, there's a whole field good or bad. Now, there's a whole field good or bad. Now, there's a whole field of trying to engineer goodness into of trying to engineer goodness into of trying to engineer goodness into them, but I certainly think one them, but I certainly think one them, but I certainly think one challenge for us it is trying to make challenge for us it is trying to make challenge for us it is trying to make sure that our relationship with the sure that our relationship with the sure that our relationship with the intelligence, even if it indeed intelligence, even if it indeed intelligence, even if it indeed surpasses ours, as I think is likely, is surpasses ours, as I think is likely, is surpasses ours, as I think is likely, is not zero-sum, right? Like uh not zero-sum, right? Like uh not zero-sum, right? Like uh we, you know, there's something it we, you know, there's something it we, you know, there's something it continues to get from our existence and continues to get from our existence and continues to get from our existence and flourishing that uh is compatible uh flourishing that uh is compatible uh flourishing that uh is compatible uh with the the goals it it has. And uh and with the the goals it it has. And uh and with the the goals it it has. And uh and vice versa. So, vice versa. So, vice versa. So, um it's but but yes, it we shouldn't we um it's but but yes, it we shouldn't we um it's but but yes, it we shouldn't we shouldn't assume uh shouldn't assume uh shouldn't assume uh it's not that it's bad. The doomer it's not that it's bad. The doomer it's not that it's bad. The doomer scenarios don't depend on it being scenarios don't depend on it being scenarios don't depend on it being malevolent by nature. malevolent by nature. malevolent by nature. >> Mhm. >> Mhm. >> Mhm. >> They just depend on it being expedient >> They just depend on it being expedient >> They just depend on it being expedient by nature. by nature. by nature. >> Yeah, it's not that it doesn't like us. >> Yeah, it's not that it doesn't like us. >> Yeah, it's not that it doesn't like us. It's that it doesn't care when we get in It's that it doesn't care when we get in It's that it doesn't care when we get in the way. the way. the way. >> That's one that is one scenario. I mean, >> That's one that is one scenario. I mean, >> That's one that is one scenario. I mean, there's a lot of stuff there's a lot of stuff there's a lot of stuff >> the >> the >> the what are the most legitimate concerns what are the most legitimate concerns what are the most legitimate concerns from the AI doomer camp, in your from the AI doomer camp, in your from the AI doomer camp, in your opinion? opinion? opinion? >> Well, >> Well, >> Well, I mean, first of all, I'd say I mean, first of all, I'd say I mean, first of all, I'd say the thing I'm surest of is the sheer the thing I'm surest of is the sheer the thing I'm surest of is the sheer destabilization, the the less sci-fi destabilization, the the less sci-fi destabilization, the the less sci-fi form of doomerism. So, like jobs. It may form of doomerism. So, like jobs. It may form of doomerism. So, like jobs. It may be true that all the people who lose be true that all the people who lose be true that all the people who lose their jobs find new ways to spend time their jobs find new ways to spend time their jobs find new ways to spend time or or find new jobs, maybe jobs per se, or or find new jobs, maybe jobs per se, or or find new jobs, maybe jobs per se, maybe spend time constructively, but I maybe spend time constructively, but I maybe spend time constructively, but I do think there's going to be a lot of do think there's going to be a lot of do think there's going to be a lot of job loss. And that's that's disorienting job loss. And that's that's disorienting job loss. And that's that's disorienting and dislocating regardless of whether and dislocating regardless of whether and dislocating regardless of whether each person eventually has a happy each person eventually has a happy each person eventually has a happy outcome, right? There's going to be outcome, right? There's going to be outcome, right? There's going to be issues with, you know, parents are going issues with, you know, parents are going issues with, you know, parents are going to freak out about the kids spending to freak out about the kids spending to freak out about the kids spending time with these things. And there are time with these things. And there are time with these things. And there are we've seen some bad outcomes and and we've seen some bad outcomes and and we've seen some bad outcomes and and there can be more you know, there are there can be more you know, there are there can be more you know, there are the the the the the the the the the you know, somebody could make a bio you know, somebody could make a bio you know, somebody could make a bio weapon with an AI. An AI mythos is a weapon with an AI. An AI mythos is a weapon with an AI. An AI mythos is a good example of you know, the good example of you know, the good example of you know, the possibility that a cyber hacking machine possibility that a cyber hacking machine possibility that a cyber hacking machine could get loose. There's just could get loose. There's just could get loose. There's just there's a lot of on the one hand risks, there's a lot of on the one hand risks, there's a lot of on the one hand risks, things that that that will go wrong at things that that that will go wrong at things that that that will go wrong at least at some level with doing some least at some level with doing some least at some level with doing some magnitude of damage if we don't play our magnitude of damage if we don't play our magnitude of damage if we don't play our cards right. And then there are these cards right. And then there are these cards right. And then there are these forms of forms of forms of destabilization destabilization destabilization that I think are almost inevitable. Just that I think are almost inevitable. Just that I think are almost inevitable. Just social social social destabilization destabilization destabilization and you know, this points to and you know, this points to and you know, this points to one of the virtues of approaching this one of the virtues of approaching this one of the virtues of approaching this as a global community. Leave aside, you as a global community. Leave aside, you as a global community. Leave aside, you know, regulating it internationally and know, regulating it internationally and know, regulating it internationally and anything else. anything else. anything else. It's just that I think we'd be better It's just that I think we'd be better It's just that I think we'd be better off going a little slower than we're off going a little slower than we're off going a little slower than we're going just because even if we going just because even if we going just because even if we successfully adapt to the change, it successfully adapt to the change, it successfully adapt to the change, it takes time. And if too much of it takes time. And if too much of it takes time. And if too much of it happens at once, you know, all hell happens at once, you know, all hell happens at once, you know, all hell breaks loose. And if you ask, well, why breaks loose. And if you ask, well, why breaks loose. And if you ask, well, why why can't we why can't we why can't we proceed more slowly? proceed more slowly? proceed more slowly? The answer you get from the American AI The answer you get from the American AI The answer you get from the American AI companies is because of China, right? companies is because of China, right? companies is because of China, right? That's the first thing you hear. So, as That's the first thing you hear. So, as That's the first thing you hear. So, as long as there is this sense of intense long as there is this sense of intense long as there is this sense of intense international contention, international contention, international contention, it's going to be hard to do even modest it's going to be hard to do even modest it's going to be hard to do even modest things. I mean, if you I you know, they things. I mean, if you I you know, they things. I mean, if you I you know, they once said to Sam Altman, like, what once said to Sam Altman, like, what once said to Sam Altman, like, what can't you shouldn't you be paying more can't you shouldn't you be paying more can't you shouldn't you be paying more attention to copyright laws? And he attention to copyright laws? And he attention to copyright laws? And he said, well, that would slow us down. said, well, that would slow us down. said, well, that would slow us down. And I'm like, well, you know, life is And I'm like, well, you know, life is And I'm like, well, you know, life is hard. Speed limits slow me down. But but hard. Speed limits slow me down. But but hard. Speed limits slow me down. But but that's just life, right? I mean, that that's just life, right? I mean, that that's just life, right? I mean, that doesn't seem like a good enough doesn't seem like a good enough doesn't seem like a good enough argument. And if you press further and argument. And if you press further and argument. And if you press further and say and by the way, copyright itself, say and by the way, copyright itself, say and by the way, copyright itself, I'm not really been out of shape on. I'm I'm not really been out of shape on. I'm I'm not really been out of shape on. I'm I'm I'm going to apparently get some I'm I'm going to apparently get some I'm I'm going to apparently get some money from this Anthropic settlement money from this Anthropic settlement money from this Anthropic settlement because I've written books. But because I've written books. But because I've written books. But honestly, I'm just happy for what I've honestly, I'm just happy for what I've honestly, I'm just happy for what I've done to be in the training data. done to be in the training data. done to be in the training data. Copyright's not a hobby horse of mine. Copyright's not a hobby horse of mine. Copyright's not a hobby horse of mine. But the But the But the But But the point is anything you say. But But the point is anything you say. But But the point is anything you say. Like, if you say, well, maybe we should Like, if you say, well, maybe we should Like, if you say, well, maybe we should tax data centers to, you know, to pay tax data centers to, you know, to pay tax data centers to, you know, to pay for the fact that inevitably, you know, for the fact that inevitably, you know, for the fact that inevitably, you know, there's there's there's going to be more there's there's there's going to be more there's there's there's going to be more carbon fuel consumption one way or the carbon fuel consumption one way or the carbon fuel consumption one way or the other as a result of of this. And so, A, other as a result of of this. And so, A, other as a result of of this. And so, A, it would be good to slow it down a it would be good to slow it down a it would be good to slow it down a little, blah blah. Any regulation little, blah blah. Any regulation little, blah blah. Any regulation that slows AI down is met with the same that slows AI down is met with the same that slows AI down is met with the same chorus from from Silicon Valley, chorus from from Silicon Valley, chorus from from Silicon Valley, which is no, we can't do it because of which is no, we can't do it because of which is no, we can't do it because of China. So, like, I think first of all, China. So, like, I think first of all, China. So, like, I think first of all, we could in principle proceed at a more we could in principle proceed at a more we could in principle proceed at a more cautious pace if we would reduce the cautious pace if we would reduce the cautious pace if we would reduce the level of mutual fear, which I personally level of mutual fear, which I personally level of mutual fear, which I personally think is founded largely on think is founded largely on think is founded largely on misconceptions on both sides. misconceptions on both sides. misconceptions on both sides. >> Mm. Okay. Before we continue, most >> Mm. Okay. Before we continue, most >> Mm. Okay. Before we continue, most people in their 30s are still training people in their 30s are still training people in their 30s are still training hard. Their protein is dialed in. They hard. Their protein is dialed in. They hard. Their protein is dialed in. They sleep better than they did in their 20s. sleep better than they did in their 20s. sleep better than they did in their 20s. Discipline is not the issue, but Discipline is not the issue, but Discipline is not the issue, but recovery feels recovery feels recovery feels somewhat different. Strength gains take somewhat different. Strength gains take somewhat different. Strength gains take a little longer. The margin for error a little longer. The margin for error a little longer. The margin for error starts to shrink. And that is why I'm starts to shrink. And that is why I'm starts to shrink. And that is why I'm such a huge fan of Timeline. You see, such a huge fan of Timeline. You see, such a huge fan of Timeline. You see, mitochondria are the energy producers mitochondria are the energy producers mitochondria are the energy producers inside of your muscle cells. As they inside of your muscle cells. As they inside of your muscle cells. As they weaken with age, your ability to weaken with age, your ability to weaken with age, your ability to generate power and recover effectively generate power and recover effectively generate power and recover effectively changes, even if your habits stay changes, even if your habits stay changes, even if your habits stay strong. Mitopure from Timeline contains strong. Mitopure from Timeline contains strong. Mitopure from Timeline contains the only clinically validated form of the only clinically validated form of the only clinically validated form of Urolithin A used in human trials. It Urolithin A used in human trials. It Urolithin A used in human trials. It promotes mitophagy, which is your body's promotes mitophagy, which is your body's promotes mitophagy, which is your body's natural process for clearing out damaged natural process for clearing out damaged natural process for clearing out damaged mitochondria and renewing healthy ones. mitochondria and renewing healthy ones. mitochondria and renewing healthy ones. In studies, this In studies, this In studies, this supported mitochondrial function and supported mitochondrial function and supported mitochondrial function and muscle strength in older adults. It's muscle strength in older adults. It's muscle strength in older adults. It's not about pushing harder, it's about not about pushing harder, it's about not about pushing harder, it's about actually supporting the cellular actually supporting the cellular actually supporting the cellular machinery underneath your training. If machinery underneath your training. If machinery underneath your training. If you care about staying strong into your you care about staying strong into your you care about staying strong into your 30s, 40s, and 50s and beyond, this is 30s, 40s, and 50s and beyond, this is 30s, 40s, and 50s and beyond, this is foundational. Best of all, there is a foundational. Best of all, there is a foundational. Best of all, there is a 30-day money-back guarantee plus free 30-day money-back guarantee plus free 30-day money-back guarantee plus free shipping in the US, and they ship shipping in the US, and they ship shipping in the US, and they ship internationally. And right now, you can internationally. And right now, you can internationally. And right now, you can get up to 20% off by going to the link get up to 20% off by going to the link get up to 20% off by going to the link in the description below or heading to in the description below or heading to in the description below or heading to timelife.com/modernwisdom timelife.com/modernwisdom timelife.com/modernwisdom and using the code modernwisdom at and using the code modernwisdom at and using the code modernwisdom at checkout. That's checkout. That's checkout. That's timelife.com/modernwisdom. and modernwisdom at checkout. and modernwisdom at checkout. What's the AI risk that worries you the What's the AI risk that worries you the What's the AI risk that worries you the most that you think has received the most that you think has received the most that you think has received the least attention? Is it that? Is it the least attention? Is it that? Is it the least attention? Is it that? Is it the ability for humans to adapt to a ability for humans to adapt to a ability for humans to adapt to a changing environment? Or is it something changing environment? Or is it something changing environment? Or is it something else? else? else? >> Well, in the >> Well, in the >> Well, in the in the near term, it is in the near term, it is in the near term, it is it it's not that the things I talk about it it's not that the things I talk about it it's not that the things I talk about um like job disruption and so on are not um like job disruption and so on are not um like job disruption and so on are not talked about. But I think in the near talked about. But I think in the near talked about. But I think in the near term, what's not appreciated is is how term, what's not appreciated is is how term, what's not appreciated is is how just highly likely it is that just highly likely it is that just highly likely it is that collectively, collectively, collectively, these things will be destabilizing. these things will be destabilizing. these things will be destabilizing. Okay? It's just It's just going to be an Okay? It's just It's just going to be an Okay? It's just It's just going to be an earthquake. And And And I think earthquake. And And And I think earthquake. And And And I think that's the thing I would like to most that's the thing I would like to most that's the thing I would like to most emphasize emphasize emphasize because it just gets people's attention because it just gets people's attention because it just gets people's attention to the possible virtue of talking about to the possible virtue of talking about to the possible virtue of talking about yeah, calming down and and and uh uh yeah, calming down and and and uh uh yeah, calming down and and and uh uh slowing things down. You know, uh slowing things down. You know, uh slowing things down. You know, uh another it's funny uh another thing another it's funny uh another thing another it's funny uh another thing Tristan Harris Tristan Harris Tristan Harris said on your podcast is you know, said on your podcast is you know, said on your podcast is you know, repeatedly pointed out your podcast is repeatedly pointed out your podcast is repeatedly pointed out your podcast is called Modern Wisdom. We're going to called Modern Wisdom. We're going to called Modern Wisdom. We're going to have to be wise to get through this. I have to be wise to get through this. I have to be wise to get through this. I agree. Uh but but here I'd add agree. Uh but but here I'd add agree. Uh but but here I'd add uh you know, there's also I talked about uh you know, there's also I talked about uh you know, there's also I talked about the fact that we're going to have this the fact that we're going to have this the fact that we're going to have this global conversation ultimately, you global conversation ultimately, you global conversation ultimately, you know, something that is in some sense a know, something that is in some sense a know, something that is in some sense a global mind is going to have to work global mind is going to have to work global mind is going to have to work this out. And this out. And this out. And as I said earlier, as I said earlier, as I said earlier, individuals are at their most wise when individuals are at their most wise when individuals are at their most wise when they are calm, right? And it's the same they are calm, right? And it's the same they are calm, right? And it's the same with I think it's the same with planets. with I think it's the same with planets. with I think it's the same with planets. We don't know, but that's my contention We don't know, but that's my contention We don't know, but that's my contention is that the planet as a whole will do is that the planet as a whole will do is that the planet as a whole will do the wisest, most responsible job of the wisest, most responsible job of the wisest, most responsible job of stewarding this technology if the planet stewarding this technology if the planet stewarding this technology if the planet as a whole is more tranquil. as a whole is more tranquil. as a whole is more tranquil. If if there is less conflict and less If if there is less conflict and less If if there is less conflict and less contention. I mean, I'm sorry. I know I contention. I mean, I'm sorry. I know I contention. I mean, I'm sorry. I know I keep getting back to the sermon. It's my keep getting back to the sermon. It's my keep getting back to the sermon. It's my big sermon. Uh so maybe uh maybe the big sermon. Uh so maybe uh maybe the big sermon. Uh so maybe uh maybe the the question you asked I guess was uh the question you asked I guess was uh the question you asked I guess was uh the biggest the biggest the biggest underappreciated thing. Well, I would underappreciated thing. Well, I would underappreciated thing. Well, I would say I I think there is more and more say I I think there is more and more say I I think there is more and more appreciation of the fact that this appreciation of the fact that this appreciation of the fact that this conversation has to be international and conversation has to be international and conversation has to be international and some of the policy does, some of the policy does, some of the policy does, but still not as much as I'd I'd like. but still not as much as I'd I'd like. but still not as much as I'd I'd like. >> Right. Yeah, well, I mean >> Right. Yeah, well, I mean >> Right. Yeah, well, I mean I understand the issue, right? Because I understand the issue, right? Because I understand the issue, right? Because you have you have you have a technology a technology a technology nukes weren't going to go off and just nukes weren't going to go off and just nukes weren't going to go off and just hit the entire planet if one country hit the entire planet if one country hit the entire planet if one country developed a particularly strong nuke, developed a particularly strong nuke, developed a particularly strong nuke, right? Let's say that you get the Tsar right? Let's say that you get the Tsar right? Let's say that you get the Tsar Bomba Bomba Bomba times two, the biggest bomb that ever times two, the biggest bomb that ever times two, the biggest bomb that ever that that that's that's ever been that that that's that's ever been that that that's that's ever been dropped. And if you breach this dropped. And if you breach this dropped. And if you breach this particular threshold for some reason, particular threshold for some reason, particular threshold for some reason, all countries are now at the mercy of all countries are now at the mercy of all countries are now at the mercy of all nuclear weapons. That's not the way all nuclear weapons. That's not the way all nuclear weapons. That's not the way that it works. But, I think the concern that it works. But, I think the concern that it works. But, I think the concern that people have is that people have is that people have is if you build a sufficiently intelligent if you build a sufficiently intelligent if you build a sufficiently intelligent AI, it impacts everybody in a way that AI, it impacts everybody in a way that AI, it impacts everybody in a way that you you don't just ring-fence, like not you you don't just ring-fence, like not you you don't just ring-fence, like not pressing the nuke button. pressing the nuke button. pressing the nuke button. The problem is The problem is The problem is the coordination that we need in order the coordination that we need in order the coordination that we need in order to be able to do that. I mean, look at to be able to do that. I mean, look at to be able to do that. I mean, look at COVID. We couldn't even do it with COVID. We couldn't even do it with COVID. We couldn't even do it with COVID, and that was happening right COVID, and that was happening right COVID, and that was happening right then. That was people dying in the then. That was people dying in the then. That was people dying in the moment. That was every country on the moment. That was every country on the moment. That was every country on the planet being worried about it. No matter planet being worried about it. No matter planet being worried about it. No matter even China, even if it was the biggest even China, even if it was the biggest even China, even if it was the biggest psyop that escaped from the lab in psyop that escaped from the lab in psyop that escaped from the lab in Wuhan, like they were worried, too. So, Wuhan, like they were worried, too. So, Wuhan, like they were worried, too. So, >> Right. >> Right. >> Right. >> the co- the lack of coordination that >> the co- the lack of coordination that >> the co- the lack of coordination that doesn't give me an awful lot of hope for doesn't give me an awful lot of hope for doesn't give me an awful lot of hope for people being able to do like predictive people being able to do like predictive people being able to do like predictive future coordination in like preparatory future coordination in like preparatory future coordination in like preparatory coordination. coordination. coordination. >> Right. It was not encouraging. I mean, I >> Right. It was not encouraging. I mean, I >> Right. It was not encouraging. I mean, I will say that although a pandemic will say that although a pandemic will say that although a pandemic is a non-zero-sum is a non-zero-sum is a non-zero-sum in the sense that if it breaks out in in the sense that if it breaks out in in the sense that if it breaks out in any nation, it's trouble for all any nation, it's trouble for all any nation, it's trouble for all nations, and and they should work to nations, and and they should work to nations, and and they should work to head it off. Once a pandemic has head it off. Once a pandemic has head it off. Once a pandemic has started, there are zero-sum dynamics, started, there are zero-sum dynamics, started, there are zero-sum dynamics, like who gets who gets the masks, you like who gets who gets the masks, you like who gets who gets the masks, you know, there's finite amount of medical know, there's finite amount of medical know, there's finite amount of medical equipment and vaccines and so on. So, equipment and vaccines and so on. So, equipment and vaccines and so on. So, it's not completely shocking. To me, the it's not completely shocking. To me, the it's not completely shocking. To me, the most disconcerting miss most disconcerting miss most disconcerting miss is in the aftermath is in the aftermath is in the aftermath when it became clear that, although I when it became clear that, although I when it became clear that, although I don't think we know for sure, it is at don't think we know for sure, it is at don't think we know for sure, it is at least possible that this pandemic was least possible that this pandemic was least possible that this pandemic was the result of a genetically engineered the result of a genetically engineered the result of a genetically engineered microorganism that escaped from a lab. microorganism that escaped from a lab. microorganism that escaped from a lab. It wasn't I It wasn't made as a It wasn't I It wasn't made as a It wasn't I It wasn't made as a bioweapon, wasn't released bioweapon, wasn't released bioweapon, wasn't released intentionally. We don't know for sure intentionally. We don't know for sure intentionally. We don't know for sure that that it was a genetically that that it was a genetically that that it was a genetically engineered virus at all. But, engineered virus at all. But, engineered virus at all. But, it obviously could have been. And it it obviously could have been. And it it obviously could have been. And it seems to me that if you process that seems to me that if you process that seems to me that if you process that information wisely, you say, "Wait a information wisely, you say, "Wait a information wisely, you say, "Wait a second. This could happen tomorrow and second. This could happen tomorrow and second. This could happen tomorrow and one thing this shows is we don't really one thing this shows is we don't really one thing this shows is we don't really have any transparency. have any transparency. have any transparency. Or at least not enough so far as what's Or at least not enough so far as what's Or at least not enough so far as what's going on in other countries in their going on in other countries in their going on in other countries in their labs, right? But that that has not even labs, right? But that that has not even labs, right? But that that has not even been a conversation. To me that's the been a conversation. To me that's the been a conversation. To me that's the most discouraging thing because most discouraging thing because most discouraging thing because uh uh you know, a virus is in a way a uh uh you know, a virus is in a way a uh uh you know, a virus is in a way a good analogy for lots of things that can good analogy for lots of things that can good analogy for lots of things that can go wrong with AI. I mean, first of all, go wrong with AI. I mean, first of all, go wrong with AI. I mean, first of all, there's a literal case of using AI to there's a literal case of using AI to there's a literal case of using AI to build a bioweapon. A new kind and COVID, build a bioweapon. A new kind and COVID, build a bioweapon. A new kind and COVID, I think I've heard you say I think I've heard you say I think I've heard you say COVID was like a bad vaccine or COVID was like a bad vaccine or COVID was like a bad vaccine or something. What's the metaphor? something. What's the metaphor? something. What's the metaphor? >> Yeah, yeah, yeah. Yeah, the COVID was >> Yeah, yeah, yeah. Yeah, the COVID was >> Yeah, yeah, yeah. Yeah, the COVID was the worst kind of vaccine that we could the worst kind of vaccine that we could the worst kind of vaccine that we could have done for everyone because it's made have done for everyone because it's made have done for everyone because it's made us us us more skeptical of future pandemics and more skeptical of future pandemics and more skeptical of future pandemics and our response is going to be less our response is going to be less our response is going to be less coordinated. coordinated. coordinated. >> That's right. And and you have to >> That's right. And and you have to >> That's right. And and you have to realize if somebody uses AI to build a realize if somebody uses AI to build a realize if somebody uses AI to build a bioweapon, they're going to make a point bioweapon, they're going to make a point bioweapon, they're going to make a point of making it more effective than COVID of making it more effective than COVID of making it more effective than COVID at at doing whatever kind of damage they at at doing whatever kind of damage they at at doing whatever kind of damage they want to do. So, it could be a lot worse want to do. So, it could be a lot worse want to do. So, it could be a lot worse and so and so and so that could happen. A, but then B, the that could happen. A, but then B, the that could happen. A, but then B, the other you know, some of the other AI other you know, some of the other AI other you know, some of the other AI nightmare scenarios like the one that nightmare scenarios like the one that nightmare scenarios like the one that Mythos brings to life, you know, you you Mythos brings to life, you know, you you Mythos brings to life, you know, you you you got a self-replicating you got a self-replicating you got a self-replicating AI that is a super hacker, it jumps from AI that is a super hacker, it jumps from AI that is a super hacker, it jumps from data center to data center gathering you data center to data center gathering you data center to data center gathering you know, commandeering computer power, know, commandeering computer power, know, commandeering computer power, getting stronger as it goes. Whatever getting stronger as it goes. Whatever getting stronger as it goes. Whatever wants to I don't know, takes out the the wants to I don't know, takes out the the wants to I don't know, takes out the the satellite infrastructure. Who knows? Uh satellite infrastructure. Who knows? Uh satellite infrastructure. Who knows? Uh that is that's kind of you know, a virus that is that's kind of you know, a virus that is that's kind of you know, a virus is a metaphor for that. Again, it's a is a metaphor for that. Again, it's a is a metaphor for that. Again, it's a self-replicating peril that makes self-replicating peril that makes self-replicating peril that makes relations among nations non-zero sum. It relations among nations non-zero sum. It relations among nations non-zero sum. It doesn't matter where this thing starts doesn't matter where this thing starts doesn't matter where this thing starts off, it is a threat to your nation if it off, it is a threat to your nation if it off, it is a threat to your nation if it does start off, so you're going to have does start off, so you're going to have does start off, so you're going to have to coordinate policies with other to coordinate policies with other to coordinate policies with other nations cuz you need more insight into nations cuz you need more insight into nations cuz you need more insight into what's going on in those nations. what's going on in those nations. what's going on in those nations. >> Uh-huh. Okay, what do you think are the >> Uh-huh. Okay, what do you think are the >> Uh-huh. Okay, what do you think are the most legitimate white pills from the most legitimate white pills from the most legitimate white pills from the techno-optimists then? Let's look at the techno-optimists then? Let's look at the techno-optimists then? Let's look at the other side of the fence. other side of the fence. other side of the fence. >> Oh, wait. Remind me of what white pills >> Oh, wait. Remind me of what white pills >> Oh, wait. Remind me of what white pills mean. I mean, I know blue and red, but mean. I mean, I know blue and red, but mean. I mean, I know blue and red, but what what are I I I what what are I I I what what are I I I You're you're younger than and cooler. You're you're younger than and cooler. You're you're younger than and cooler. >> Techno-optimists, what is the What's the >> Techno-optimists, what is the What's the >> Techno-optimists, what is the What's the bull case? What's the pro case? How can bull case? What's the pro case? How can bull case? What's the pro case? How can this thing go right? What are the most this thing go right? What are the most this thing go right? What are the most likely ways that this goes right? likely ways that this goes right? likely ways that this goes right? >> Uh I think I think I just I'm sorry. I wish I could see it I just I'm sorry. I wish I could see it I just I'm sorry. I wish I could see it going right in a laissez-faire going right in a laissez-faire going right in a laissez-faire environment where you just let it go and environment where you just let it go and environment where you just let it go and and let and let and let uh the market system deal with it. I uh the market system deal with it. I uh the market system deal with it. I just don't think that's going to happen. just don't think that's going to happen. just don't think that's going to happen. It's easy to point to wonderful things It's easy to point to wonderful things It's easy to point to wonderful things it could do, and we've heard them. Uh it could do, and we've heard them. Uh it could do, and we've heard them. Uh cure disease. Um it could it could Yeah, cure disease. Um it could it could Yeah, cure disease. Um it could it could Yeah, well, you know, one thing I uh This is well, you know, one thing I uh This is well, you know, one thing I uh This is not what the techno-optimists get into, not what the techno-optimists get into, not what the techno-optimists get into, but but I referred earlier to like but but I referred earlier to like but but I referred earlier to like cultivating cultivating cultivating cognitive empathy, getting better to cognitive empathy, getting better to cognitive empathy, getting better to understanding other people's points of understanding other people's points of understanding other people's points of view, maybe getting more mindful view, maybe getting more mindful view, maybe getting more mindful generally. You can have an AI that helps generally. You can have an AI that helps generally. You can have an AI that helps you with that. But the natural tendency you with that. But the natural tendency you with that. But the natural tendency of the market will be to produce the of the market will be to produce the of the market will be to produce the kind that doesn't. I mean, we've already kind that doesn't. I mean, we've already kind that doesn't. I mean, we've already seen that if if companies, you know, seen that if if companies, you know, seen that if if companies, you know, optimize for engagement, you may get optimize for engagement, you may get optimize for engagement, you may get sycophantic AIs to say, "Yeah, you're sycophantic AIs to say, "Yeah, you're sycophantic AIs to say, "Yeah, you're right. They're wrong." Like in this right. They're wrong." Like in this right. They're wrong." Like in this argument with your spouse, you're right, argument with your spouse, you're right, argument with your spouse, you're right, they're wrong. Uh so, that will tend to they're wrong. Uh so, that will tend to they're wrong. Uh so, that will tend to happen, but it can happen, but it can happen, but it can AI can be a wonderful and an uh AI can be a wonderful and an uh AI can be a wonderful and an uh literally enlightening companion. Okay? literally enlightening companion. Okay? literally enlightening companion. Okay? If if we want that, but you have to make If if we want that, but you have to make If if we want that, but you have to make a point to want it. a point to want it. a point to want it. >> You you don't think that this is just >> You you don't think that this is just >> You you don't think that this is just going to find its way there naturally? going to find its way there naturally? going to find its way there naturally? Like that if you just let the sort of Like that if you just let the sort of Like that if you just let the sort of capitalist meritocratic, it will find capitalist meritocratic, it will find capitalist meritocratic, it will find its way optimizing function without any its way optimizing function without any its way optimizing function without any shaping from us and without any shaping from us and without any shaping from us and without any predisposition from a uh uh a better predisposition from a uh uh a better predisposition from a uh uh a better coordinated world? It's not just going coordinated world? It's not just going coordinated world? It's not just going to arrive there. to arrive there. to arrive there. >> I think if enough people send signals to >> I think if enough people send signals to >> I think if enough people send signals to the market I Markets are very efficient the market I Markets are very efficient the market I Markets are very efficient and wondrous things, you know, they and wondrous things, you know, they and wondrous things, you know, they really are. really are. really are. >> What does that What does that look like >> What does that What does that look like >> What does that What does that look like sending signals practically? sending signals practically? sending signals practically? >> It means, for example, you you and and I >> It means, for example, you you and and I >> It means, for example, you you and and I and enough other people to get the and enough other people to get the and enough other people to get the attention of people who are not attention of people who are not attention of people who are not necessarily the people making the necessarily the people making the necessarily the people making the foundation models or the frontier model. foundation models or the frontier model. foundation models or the frontier model. It could wind up being people who take It could wind up being people who take It could wind up being people who take an open weights model, an open-source an open weights model, an open-source an open weights model, an open-source model, and they kind of fine-tune it to model, and they kind of fine-tune it to model, and they kind of fine-tune it to be the thing that interrogates you be the thing that interrogates you be the thing that interrogates you critically along certain lines, right? critically along certain lines, right? critically along certain lines, right? Like, okay, you say you You say you hate Like, okay, you say you You say you hate Like, okay, you say you You say you hate this person. You say you find this this person. You say you find this this person. You say you find this country threatening. Let's just like uh country threatening. Let's just like uh country threatening. Let's just like uh uh or you you say you think they're uh or you you say you think they're uh or you you say you think they're looking at it this way. Let's just play looking at it this way. Let's just play looking at it this way. Let's just play devil's advocate. It It You know, it's devil's advocate. It It You know, it's devil's advocate. It It You know, it's it's almost um like doing steelmanning it's almost um like doing steelmanning it's almost um like doing steelmanning uh automatically in some cases. It But uh automatically in some cases. It But uh automatically in some cases. It But But it it depends on enough people, you But it it depends on enough people, you But it it depends on enough people, you know, there are a lot of things in life know, there are a lot of things in life know, there are a lot of things in life that they're good for you but hard to that they're good for you but hard to that they're good for you but hard to do. Working out every day, good for you do. Working out every day, good for you do. Working out every day, good for you but hard to do sometimes. That's That's but hard to do sometimes. That's That's but hard to do sometimes. That's That's why some people who can afford them, you why some people who can afford them, you why some people who can afford them, you know, have a personal trainer, right? know, have a personal trainer, right? know, have a personal trainer, right? They say They say They say I'm going to This person's going to I'm going to This person's going to I'm going to This person's going to expect me to show up in the gym expect me to show up in the gym expect me to show up in the gym 3 days a week or 5 days a week or 3 days a week or 5 days a week or 3 days a week or 5 days a week or whatever. And once you've made that whatever. And once you've made that whatever. And once you've made that commitment, you just kind of have to do commitment, you just kind of have to do commitment, you just kind of have to do it. Or maybe they'll even show up at it. Or maybe they'll even show up at it. Or maybe they'll even show up at your house. your house. your house. Uh but but you know, and and it's kind Uh but but you know, and and it's kind Uh but but you know, and and it's kind of like that I think it's going to be of like that I think it's going to be of like that I think it's going to be kind of like that in choosing your AI kind of like that in choosing your AI kind of like that in choosing your AI companion, right? Like companion, right? Like companion, right? Like it feels good in the short run to have it feels good in the short run to have it feels good in the short run to have some Somebody will tell you or machine some Somebody will tell you or machine some Somebody will tell you or machine that'll tell you you're always right and that'll tell you you're always right and that'll tell you you're always right and your your adversary and rival and and your your adversary and rival and and your your adversary and rival and and spouse is always wrong. spouse is always wrong. spouse is always wrong. >> Mhm. >> Mhm. >> Mhm. >> Um but you know, I want to be a little >> Um but you know, I want to be a little >> Um but you know, I want to be a little better than that. So, I I think, you better than that. So, I I think, you better than that. So, I I think, you know, now if if if if the market signal know, now if if if if the market signal know, now if if if if the market signal is going to be strong enough for this to is going to be strong enough for this to is going to be strong enough for this to happen at scale, uh these signals may happen at scale, uh these signals may happen at scale, uh these signals may emerge from like movements. You know, it emerge from like movements. You know, it emerge from like movements. You know, it could be, for example, could be, for example, could be, for example, uh religions uh religions uh religions will will say to their congregants, will will say to their congregants, will will say to their congregants, "Hey, we recommend this model." Or we, "Hey, we recommend this model." Or we, "Hey, we recommend this model." Or we, you know, whatever, and then there's a you know, whatever, and then there's a you know, whatever, and then there's a demand for it. I'm not saying all those demand for it. I'm not saying all those demand for it. I'm not saying all those will be good. Depends on what what group will be good. Depends on what what group will be good. Depends on what what group of religious people it is and and what of religious people it is and and what of religious people it is and and what their values are, but you can imagine their values are, but you can imagine their values are, but you can imagine you know, there are lots there are lots you know, there are lots there are lots you know, there are lots there are lots of people right now engaged of people right now engaged of people right now engaged in the process of trying to make in the process of trying to make in the process of trying to make themselves, you know, genuinely better themselves, you know, genuinely better themselves, you know, genuinely better people. I mean, they they meditate so people. I mean, they they meditate so people. I mean, they they meditate so that they'll be less volatile and and that they'll be less volatile and and that they'll be less volatile and and and work better with other people. I I and work better with other people. I I and work better with other people. I I think we're going to have to go into think we're going to have to go into think we're going to have to go into this recognizing this recognizing this recognizing that for better or worse, these machines that for better or worse, these machines that for better or worse, these machines are probably going to be exerting pretty are probably going to be exerting pretty are probably going to be exerting pretty pervasive influence on people, and we pervasive influence on people, and we pervasive influence on people, and we need to think carefully about what kind need to think carefully about what kind need to think carefully about what kind of influence we want. of influence we want. of influence we want. >> What about the risk of AI-induced >> What about the risk of AI-induced >> What about the risk of AI-induced thinking atrophy? Right, this this role thinking atrophy? Right, this this role thinking atrophy? Right, this this role of AI systems in taking on critical of AI systems in taking on critical of AI systems in taking on critical thoughts, uh decision-making, thoughts, uh decision-making, thoughts, uh decision-making, um connectedness, all of the things that um connectedness, all of the things that um connectedness, all of the things that typically humans really value inside of typically humans really value inside of typically humans really value inside of themselves, and as we start to outsource themselves, and as we start to outsource themselves, and as we start to outsource that to AI systems, that to AI systems, that to AI systems, our capacity to be able to do that our capacity to be able to do that our capacity to be able to do that diminishes. AI-induced thinking atrophy, diminishes. AI-induced thinking atrophy, diminishes. AI-induced thinking atrophy, are you worried about that? are you worried about that? are you worried about that? >> Um >> Um >> Um I mean, yes and no. I mean, you've heard I mean, yes and no. I mean, you've heard I mean, yes and no. I mean, you've heard the standard responses, right? Which is, the standard responses, right? Which is, the standard responses, right? Which is, I forget whether it was Plato or I forget whether it was Plato or I forget whether it was Plato or Socrates who supposedly said, Socrates who supposedly said, Socrates who supposedly said, uh "The written word is bad because uh "The written word is bad because uh "The written word is bad because people won't have to remember things." people won't have to remember things." people won't have to remember things." >> Yeah. >> Yeah. >> Yeah. >> Um >> Um >> Um the the the uh uh uh and there can be some of that. I mean, and there can be some of that. I mean, and there can be some of that. I mean, the other side of the coin is obviously, the other side of the coin is obviously, the other side of the coin is obviously, at least right now, the richness of at least right now, the richness of at least right now, the richness of intellectual exploration it permits, intellectual exploration it permits, intellectual exploration it permits, right? Like, if you're interested in a right? Like, if you're interested in a right? Like, if you're interested in a subject, subject, subject, it's it's almost like having like a it's it's almost like having like a it's it's almost like having like a leading expert there for you to leading expert there for you to leading expert there for you to interrogate. And for me, at least, interrogate. And for me, at least, interrogate. And for me, at least, that's a much more efficient a to learn. that's a much more efficient a to learn. that's a much more efficient a to learn. Now, you have to be on guard for Now, you have to be on guard for Now, you have to be on guard for hallucinations and so on, but I think hallucinations and so on, but I think hallucinations and so on, but I think machines are getting better and you can machines are getting better and you can machines are getting better and you can develop kind of a an ability to develop kind of a an ability to develop kind of a an ability to to know when to be suspicious. Um so to know when to be suspicious. Um so to know when to be suspicious. Um so that's that's great, but that's that's great, but that's that's great, but I think, you know, I think what we can I think, you know, I think what we can I think, you know, I think what we can be sure of is that, you know, if this be sure of is that, you know, if this be sure of is that, you know, if this proceeds in a reasonably smooth way proceeds in a reasonably smooth way proceeds in a reasonably smooth way the pace of overall intellectual the pace of overall intellectual the pace of overall intellectual progress will will benefit from the progress will will benefit from the progress will will benefit from the technology. That's certainly not the technology. That's certainly not the technology. That's certainly not the problem, but I think you're asking a problem, but I think you're asking a problem, but I think you're asking a good a good question as to uh good a good question as to uh good a good question as to uh you know, what what it's going to be you know, what what it's going to be you know, what what it's going to be like to be human if like to be human if like to be human if uh uh uh you know, you know, you know, well, for starters, there aren't many well, for starters, there aren't many well, for starters, there aren't many humans who can say, "I'm really on the humans who can say, "I'm really on the humans who can say, "I'm really on the frontier. I'm the reason we're making frontier. I'm the reason we're making frontier. I'm the reason we're making progress, right?" Now, I will say, progress, right?" Now, I will say, progress, right?" Now, I will say, "Look, most humans don't say that now." "Look, most humans don't say that now." "Look, most humans don't say that now." And and you shouldn't you know, nobody And and you shouldn't you know, nobody And and you shouldn't you know, nobody should over extrapolate from whatever should over extrapolate from whatever should over extrapolate from whatever sub demographic they occupy. sub demographic they occupy. sub demographic they occupy. >> that's so that's true, but I do think >> that's so that's true, but I do think >> that's so that's true, but I do think that everybody feels like they are that everybody feels like they are that everybody feels like they are breaking new ground even if it's in breaking new ground even if it's in breaking new ground even if it's in their own life. I had Mark Manson on a their own life. I had Mark Manson on a their own life. I had Mark Manson on a couple of weeks ago and he's got this couple of weeks ago and he's got this couple of weeks ago and he's got this great line, which is do hard not great line, which is do hard not great line, which is do hard not because being hard makes it more because being hard makes it more because being hard makes it more meaningful. Uh sorry, uh not not for the meaningful. Uh sorry, uh not not for the meaningful. Uh sorry, uh not not for the purpose of it being hard, but because purpose of it being hard, but because purpose of it being hard, but because it's hard it will make it more it's hard it will make it more it's hard it will make it more meaningful. That we associate a degree meaningful. That we associate a degree meaningful. That we associate a degree of meaning with struggle and I mean, I'm of meaning with struggle and I mean, I'm of meaning with struggle and I mean, I'm sure that you have used uh ChatGPT or or sure that you have used uh ChatGPT or or sure that you have used uh ChatGPT or or something else to help you write at some something else to help you write at some something else to help you write at some point. I've got to get I've got to get a point. I've got to get I've got to get a point. I've got to get I've got to get a bit research done or I need to write bit research done or I need to write bit research done or I need to write something. I'm really struggling to something. I'm really struggling to something. I'm really struggling to formulate this particular paragraph or formulate this particular paragraph or formulate this particular paragraph or this sentence or this idea, whatever. this sentence or this idea, whatever. this sentence or this idea, whatever. That sentence That sentence That sentence is just less satisfying than the one is just less satisfying than the one is just less satisfying than the one that you spent time working on and I that you spent time working on and I that you spent time working on and I wonder whether wonder whether wonder whether snow plowing out of the way snow plowing out of the way snow plowing out of the way all of the challenges or more of the all of the challenges or more of the all of the challenges or more of the challenges actually, more of the challenges actually, more of the challenges actually, more of the challenges that humans face primarily challenges that humans face primarily challenges that humans face primarily intellectually and then when robotics intellectually and then when robotics intellectually and then when robotics come online perhaps physically as well. come online perhaps physically as well. come online perhaps physically as well. It's going to sap meaning out of the It's going to sap meaning out of the It's going to sap meaning out of the world world world for most people at small amounts. And for most people at small amounts. And for most people at small amounts. And we're in the middle of a meaning crisis we're in the middle of a meaning crisis we're in the middle of a meaning crisis already. People are already struggling already. People are already struggling already. People are already struggling with meaning. And if you make life with meaning. And if you make life with meaning. And if you make life easier, if you make thinking more easier, if you make thinking more easier, if you make thinking more outsourced, if you make difficulty outsourced, if you make difficulty outsourced, if you make difficulty harder to access, harder to access, harder to access, the only way to do it is to be a the only way to do it is to be a the only way to do it is to be a Luddite, which means that you fall Luddite, which means that you fall Luddite, which means that you fall behind all of the other people. We're behind all of the other people. We're behind all of the other people. We're still in a meritocracy, right? So, if still in a meritocracy, right? So, if still in a meritocracy, right? So, if you don't use it, you lose it. But if you don't use it, you lose it. But if you don't use it, you lose it. But if you don't lose it if you don't use it, you don't lose it if you don't use it, you don't lose it if you don't use it, you also fall behind from it. It feels you also fall behind from it. It feels you also fall behind from it. It feels It feels like a vicious situation to be It feels like a vicious situation to be It feels like a vicious situation to be in. in. in. >> Yeah, I haven't used AI in exactly that >> Yeah, I haven't used AI in exactly that >> Yeah, I haven't used AI in exactly that way with my own writing. I mean, a way with my own writing. I mean, a way with my own writing. I mean, a couple of times in my newsletter, I've couple of times in my newsletter, I've couple of times in my newsletter, I've uh I've said when I was just doing very uh I've said when I was just doing very uh I've said when I was just doing very short summaries of things, I just said short summaries of things, I just said short summaries of things, I just said full disclosure, you know, your first full disclosure, you know, your first full disclosure, you know, your first draft was AI. I said, "But that's not draft was AI. I said, "But that's not draft was AI. I said, "But that's not really my writing. I'm just the editor really my writing. I'm just the editor really my writing. I'm just the editor there." With my own writing in the book, there." With my own writing in the book, there." With my own writing in the book, I haven't I haven't done quite that, but I haven't I haven't done quite that, but I haven't I haven't done quite that, but I have had uh I mean, first of all, I've I have had uh I mean, first of all, I've I have had uh I mean, first of all, I've had conversations with Claude in had conversations with Claude in had conversations with Claude in particular, which is very good with particular, which is very good with particular, which is very good with language, about subtle linguistic language, about subtle linguistic language, about subtle linguistic issues, like asking questions, like issues, like asking questions, like issues, like asking questions, like usage questions and stuff. usage questions and stuff. usage questions and stuff. And And And it's just it's almost it's just it's almost it's just it's almost mind-blowing how good it is and in that mind-blowing how good it is and in that mind-blowing how good it is and in that regard. But but I have regard. But but I have regard. But but I have you know, imagined the future enough you know, imagined the future enough you know, imagined the future enough that I have had a moments of true that I have had a moments of true that I have had a moments of true despair. I mean, I said to somebody the despair. I mean, I said to somebody the despair. I mean, I said to somebody the other day, I feel like I'm a blacksmith other day, I feel like I'm a blacksmith other day, I feel like I'm a blacksmith a century ago, you know, cuz I can see a century ago, you know, cuz I can see a century ago, you know, cuz I can see the writing on the wall. I mean, the the the writing on the wall. I mean, the the the writing on the wall. I mean, the the you know, you know, you know, uh I have a Substack and uh it's clear uh I have a Substack and uh it's clear uh I have a Substack and uh it's clear to me that uh the next wave to me that uh the next wave to me that uh the next wave is going to be is going to be is going to be uh uh uh you know, you're going to see the you know, you're going to see the you know, you're going to see the success of a lot of Substack success of a lot of Substack success of a lot of Substack that are using AI probably more heavily that are using AI probably more heavily that are using AI probably more heavily than I'm going to be. Uh but in any than I'm going to be. Uh but in any than I'm going to be. Uh but in any event, it it's just so good that uh I I event, it it's just so good that uh I I event, it it's just so good that uh I I can see the writing on the wall. can see the writing on the wall. can see the writing on the wall. >> 95% of people don't get enough fiber, >> 95% of people don't get enough fiber, >> 95% of people don't get enough fiber, which is why Momentous built Fiber Plus which is why Momentous built Fiber Plus which is why Momentous built Fiber Plus because hitting your daily fiber target because hitting your daily fiber target because hitting your daily fiber target through food alone is actually kind of through food alone is actually kind of through food alone is actually kind of hard. Even hard. Even hard. Even harder than your last toilet trip. Fiber harder than your last toilet trip. Fiber harder than your last toilet trip. Fiber isn't just a digestion thing. It's the isn't just a digestion thing. It's the isn't just a digestion thing. It's the foundation of your gut health, which foundation of your gut health, which foundation of your gut health, which drives how well you absorb nutrients, drives how well you absorb nutrients, drives how well you absorb nutrients, how stable your energy is, and how how stable your energy is, and how how stable your energy is, and how quickly you recover. If your gut isn't quickly you recover. If your gut isn't quickly you recover. If your gut isn't dialed, everything else you're doing is dialed, everything else you're doing is dialed, everything else you're doing is working at a fraction of its potential. working at a fraction of its potential. working at a fraction of its potential. Fiber Plus is a three-in-one formula Fiber Plus is a three-in-one formula Fiber Plus is a three-in-one formula built to address digestion, gut barrier built to address digestion, gut barrier built to address digestion, gut barrier strength, and blood sugar stability all strength, and blood sugar stability all strength, and blood sugar stability all at once. at once. at once. >> That's why I plan to make my next big >> That's why I plan to make my next big >> That's why I plan to make my next big dump dump dump Momentous. Momentous. Momentous. >> Best of all, they offer a free 30-day >> Best of all, they offer a free 30-day >> Best of all, they offer a free 30-day money-back guarantee, so you can buy it money-back guarantee, so you can buy it money-back guarantee, so you can buy it and try it for 29 days, and if you don't and try it for 29 days, and if you don't and try it for 29 days, and if you don't love it, they'll give you your money love it, they'll give you your money love it, they'll give you your money back. Plus, they ship internationally. back. Plus, they ship internationally. back. Plus, they ship internationally. Right now, you can get up to 35% off Right now, you can get up to 35% off Right now, you can get up to 35% off your first subscription and that 30-day your first subscription and that 30-day your first subscription and that 30-day money-back guarantee by going to the money-back guarantee by going to the money-back guarantee by going to the link in the description below or heading link in the description below or heading link in the description below or heading to livemomentous.com/modernwisdom to livemomentous.com/modernwisdom to livemomentous.com/modernwisdom and using the code modernwisdom at and using the code modernwisdom at and using the code modernwisdom at checkout. That's l i v e m o m e n t o u checkout. That's l i v e m o m e n t o u checkout. That's l i v e m o m e n t o u s.com/modernwisdom s.com/modernwisdom s.com/modernwisdom and modernwisdom and modernwisdom and modernwisdom at checkout. What are you What do you at checkout. What are you What do you at checkout. What are you What do you most worried about, or what do you think most worried about, or what do you think most worried about, or what do you think is going to is going to is going to uh uh uh take take take Yeah, what do you What do you think is Yeah, what do you What do you think is Yeah, what do you What do you think is going to be uh the the highest risk? going to be uh the the highest risk? going to be uh the the highest risk? >> I just think uh >> I just think uh >> I just think uh doing what I've done my whole life, doing what I've done my whole life, doing what I've done my whole life, which is, you know, painstakingly which is, you know, painstakingly which is, you know, painstakingly generate writing that generate writing that generate writing that uh uh uh you know, you hope will be enticing you know, you hope will be enticing you know, you hope will be enticing and clear and accurate and persuasive and clear and accurate and persuasive and clear and accurate and persuasive and so on, and so on, and so on, is going to be a less and less viable is going to be a less and less viable is going to be a less and less viable way to make a living. I I think, you way to make a living. I I think, you way to make a living. I I think, you know, I do think for some time to come, know, I do think for some time to come, know, I do think for some time to come, well, I don't know how long, but well, I don't know how long, but well, I don't know how long, but uh people like me may still have a role uh people like me may still have a role uh people like me may still have a role as kind of validators. In other words, as kind of validators. In other words, as kind of validators. In other words, like look, increasingly, you're going to like look, increasingly, you're going to like look, increasingly, you're going to look at a Substack or whatever and look at a Substack or whatever and look at a Substack or whatever and you're going to go like you're going to go like you're going to go like I don't know who wrote you know, how can I don't know who wrote you know, how can I don't know who wrote you know, how can I be sure that this person actually I be sure that this person actually I be sure that this person actually wrote it? I don't know them well enough wrote it? I don't know them well enough wrote it? I don't know them well enough to personally trust him. All you know is to personally trust him. All you know is to personally trust him. All you know is that they're vouching for the content. that they're vouching for the content. that they're vouching for the content. They're willing to have their name They're willing to have their name They're willing to have their name associated with this content. And so in associated with this content. And so in associated with this content. And so in a way, you know, it's a throwback to, a way, you know, it's a throwback to, a way, you know, it's a throwback to, you know, when I started out in you know, when I started out in you know, when I started out in journalism, the news weeklies didn't journalism, the news weeklies didn't journalism, the news weeklies didn't have bylines and The Economist is I have bylines and The Economist is I have bylines and The Economist is I think still that way. So it's just like think still that way. So it's just like think still that way. So it's just like you don't know who wrote the piece, but you don't know who wrote the piece, but you don't know who wrote the piece, but you know who the editor is and you know you know who the editor is and you know you know who the editor is and you know you've come to trust the editor. So the you've come to trust the editor. So the you've come to trust the editor. So the editor of a magazine like that is a editor of a magazine like that is a editor of a magazine like that is a person you trust even though you you you person you trust even though you you you person you trust even though you you you know they didn't generate most of the know they didn't generate most of the know they didn't generate most of the writing. They're just vouching for it. writing. They're just vouching for it. writing. They're just vouching for it. That it reflects their judgment about That it reflects their judgment about That it reflects their judgment about what's good. They're they're telling you what's good. They're they're telling you what's good. They're they're telling you that they think it's accurate and so on. that they think it's accurate and so on. that they think it's accurate and so on. I can see I can see I can see I can see that role for somebody like me I can see that role for somebody like me I can see that role for somebody like me for 5 years. But for 5 years. But for 5 years. But you know, you know, you know, it's it's it's >> Really >> Really >> Really really scraping the bottom of the barrel really scraping the bottom of the barrel really scraping the bottom of the barrel here. here. here. >> Well, look, I I I this is you know, I I >> Well, look, I I I this is you know, I I >> Well, look, I I I this is you know, I I to get back to how we started this off, to get back to how we started this off, to get back to how we started this off, if people understand if people understand if people understand you know, how these AIs are being you know, how these AIs are being you know, how these AIs are being created, how basically you just give created, how basically you just give created, how basically you just give them the data and they do the them the data and they do the them the data and they do the engineering engineering engineering to generate the parts of the human mind to generate the parts of the human mind to generate the parts of the human mind or if you just pay attention to the or if you just pay attention to the or if you just pay attention to the improvement we've seen over the last two improvement we've seen over the last two improvement we've seen over the last two or three years, right? or three years, right? or three years, right? >> Yeah. What what >> Yeah. What what >> Yeah. What what what what industries would you be most what what industries would you be most what what industries would you be most bullish on or if you were a young person bullish on or if you were a young person bullish on or if you were a young person today, what do you think would be a good today, what do you think would be a good today, what do you think would be a good career path to go down or a particular career path to go down or a particular career path to go down or a particular industry to go in? industry to go in? industry to go in? >> Well, you know, it's common to say >> Well, you know, it's common to say >> Well, you know, it's common to say manual labor robotics is a little manual labor robotics is a little manual labor robotics is a little behind. It's going to be a while before behind. It's going to be a while before behind. It's going to be a while before I'm calling a plum a robot to fix my I'm calling a plum a robot to fix my I'm calling a plum a robot to fix my sink. I I do think sink. I I do think sink. I I do think that certain kinds of that certain kinds of that certain kinds of human services are going to become human services are going to become human services are going to become almost more valuable because they're almost more valuable because they're almost more valuable because they're humans and I think a good example is humans and I think a good example is humans and I think a good example is live music. I can well imagine that live music. I can well imagine that live music. I can well imagine that there will be more of a demand there will be more of a demand there will be more of a demand for, you know, for bands that play at for, you know, for bands that play at for, you know, for bands that play at small clubs in Brooklyn or whatever and small clubs in Brooklyn or whatever and small clubs in Brooklyn or whatever and make enough money to get by. make enough money to get by. make enough money to get by. You know, which would be in a way an You know, which would be in a way an You know, which would be in a way an improvement over the situation 30 years improvement over the situation 30 years improvement over the situation 30 years ago during, you know, the golden era of ago during, you know, the golden era of ago during, you know, the golden era of of the record companies was it was a of the record companies was it was a of the record companies was it was a winner-take-all market. You know, a few winner-take-all market. You know, a few winner-take-all market. You know, a few a few people got super rich playing a few people got super rich playing a few people got super rich playing music. So, I can imagine a world in music. So, I can imagine a world in music. So, I can imagine a world in which more people are actually making a which more people are actually making a which more people are actually making a living playing music. living playing music. living playing music. >> Comedians? >> Comedians? >> Comedians? >> Comedians? Good example. >> Comedians? Good example. >> Comedians? Good example. >> Live events, nightclubs. >> Live events, nightclubs. >> Live events, nightclubs. >> Yeah. Yeah, and maybe look uh yeah, live >> Yeah. Yeah, and maybe look uh yeah, live >> Yeah. Yeah, and maybe look uh yeah, live events generally. I can I can I can well events generally. I can I can I can well events generally. I can I can I can well imagine that. I myself have had the imagine that. I myself have had the imagine that. I myself have had the feeling, you know, because I've been so feeling, you know, because I've been so feeling, you know, because I've been so immersed in AI while writing this book, immersed in AI while writing this book, immersed in AI while writing this book, I've just like, you know, you're in New I've just like, you know, you're in New I've just like, you know, you're in New York, you're in a subway or somewhere York, you're in a subway or somewhere York, you're in a subway or somewhere and you see some guy, you know, a busker and you see some guy, you know, a busker and you see some guy, you know, a busker trying to trying to make a few bucks trying to trying to make a few bucks trying to trying to make a few bucks playing an instrument and they're really playing an instrument and they're really playing an instrument and they're really good. They're some really good. I just good. They're some really good. I just good. They're some really good. I just think, "God bless you." You know, it's think, "God bless you." You know, it's think, "God bless you." You know, it's like I I just almost get emotional. like I I just almost get emotional. like I I just almost get emotional. Um Um Um so so so you know, if you don't think it's going you know, if you don't think it's going you know, if you don't think it's going to get weird, I don't think you're to get weird, I don't think you're to get weird, I don't think you're paying attention. paying attention. paying attention. >> Do you think AI could make humanity more >> Do you think AI could make humanity more >> Do you think AI could make humanity more religious rather than less? religious rather than less? religious rather than less? >> Well, that's a good question. Um >> Well, that's a good question. Um >> Well, that's a good question. Um you know, there have been you know, there have been you know, there have been writing a book about this guy who writing a book about this guy who writing a book about this guy who he's a guy who actually started what he's a guy who actually started what he's a guy who actually started what became Waymo, I think, the Google became Waymo, I think, the Google became Waymo, I think, the Google self-driving thing. self-driving thing. self-driving thing. His name's Levandowski. Uh His name's Levandowski. Uh His name's Levandowski. Uh who was trying to start a religion who was trying to start a religion who was trying to start a religion that involved that involved that involved >> Always always a great first line to a >> Always always a great first line to a >> Always always a great first line to a story. He was [laughter] trying to start story. He was [laughter] trying to start story. He was [laughter] trying to start a religion and a religion and a religion and >> Hey, it worked out for L. Ron Hubbard. >> Hey, it worked out for L. Ron Hubbard. >> Hey, it worked out for L. Ron Hubbard. Scientology, right? Scientology, right? Scientology, right? He he he made a good living. Um the uh He he he made a good living. Um the uh He he he made a good living. Um the uh So, but no, but his his argument was if So, but no, but his his argument was if So, but no, but his his argument was if we if we have a respectfully even we if we have a respectfully even we if we have a respectfully even worshipful attitude toward the AI, then worshipful attitude toward the AI, then worshipful attitude toward the AI, then it will treat us well in in return once it will treat us well in in return once it will treat us well in in return once it's running the show. I I I don't think it's running the show. I I I don't think it's running the show. I I I don't think it's going to work that way. So, let's it's going to work that way. So, let's it's going to work that way. So, let's leave that one aside. Um leave that one aside. Um leave that one aside. Um I mean, it's a good question. You know, I mean, it's a good question. You know, I mean, it's a good question. You know, the other thing is the other thing is the other thing is there are a lot of kind of spiritually there are a lot of kind of spiritually there are a lot of kind of spiritually related mysteries in the universe. Like, related mysteries in the universe. Like, related mysteries in the universe. Like, what is consciousness? What is what is consciousness? What is what is consciousness? What is subjective experience? Um subjective experience? Um subjective experience? Um all I know for sure is all I know for sure is all I know for sure is it's the thing that gives life meaning. it's the thing that gives life meaning. it's the thing that gives life meaning. If you imagine beings if if you imagine If you imagine beings if if you imagine If you imagine beings if if you imagine humans like they look like humans, they humans like they look like humans, they humans like they look like humans, they do the stuff humans do, do the stuff humans do, do the stuff humans do, >> The P-zombie. >> The P-zombie. >> The P-zombie. >> Yeah, but they're zombies. It's not like >> Yeah, but they're zombies. It's not like >> Yeah, but they're zombies. It's not like anything to be them. I would say like, anything to be them. I would say like, anything to be them. I would say like, well, blow them up. I don't care. well, blow them up. I don't care. well, blow them up. I don't care. There's no meaning to their lives There's no meaning to their lives There's no meaning to their lives anyway. There's nothing meaningful going anyway. There's nothing meaningful going anyway. There's nothing meaningful going on if there's not subjective experience, on if there's not subjective experience, on if there's not subjective experience, there's not consciousness. there's not consciousness. there's not consciousness. And I'd love to have more insight into And I'd love to have more insight into And I'd love to have more insight into what that is. I don't know that AI can what that is. I don't know that AI can what that is. I don't know that AI can help us because it's it's the most help us because it's it's the most help us because it's it's the most stubborn mystery I'm aware of almost. stubborn mystery I'm aware of almost. stubborn mystery I'm aware of almost. I'd love to You know, there's so many I'd love to You know, there's so many I'd love to You know, there's so many mysteries that are suggestive mysteries that are suggestive mysteries that are suggestive of something weird and wondrous. Qua- of something weird and wondrous. Qua- of something weird and wondrous. Qua- quantum physics for sure. quantum physics for sure. quantum physics for sure. I I I uh you know, I can imagine getting a uh you know, I can imagine getting a uh you know, I can imagine getting a kind of I mean, who knows whether kind of I mean, who knows whether kind of I mean, who knows whether there's a revelation that awaits, right? there's a revelation that awaits, right? there's a revelation that awaits, right? That is at one level an intellectual That is at one level an intellectual That is at one level an intellectual revelation that explains stuff, revelation that explains stuff, revelation that explains stuff, but at another level is also, you know, but at another level is also, you know, but at another level is also, you know, gratifying in a spiritual way. gratifying in a spiritual way. gratifying in a spiritual way. >> Yeah, yeah. What's uh >> Yeah, yeah. What's uh >> Yeah, yeah. What's uh You mentioned we've sort of circled You mentioned we've sort of circled You mentioned we've sort of circled around it a bunch, the idea that these around it a bunch, the idea that these around it a bunch, the idea that these machines are able to machines are able to machines are able to uh uh uh like pantomime intelligence. like pantomime intelligence. like pantomime intelligence. They're able to simulate knowing, but do They're able to simulate knowing, but do They're able to simulate knowing, but do you think that they know? you think that they know? you think that they know? Do they actually know what they're Do they actually know what they're Do they actually know what they're doing? doing? doing? >> I have a chapter on that, actually. >> I have a chapter on that, actually. >> I have a chapter on that, actually. There's a famous thought experiment There's a famous thought experiment There's a famous thought experiment called the Chinese Room thought called the Chinese Room thought called the Chinese Room thought experiment experiment experiment by philosopher who's no longer alive by philosopher who's no longer alive by philosopher who's no longer alive and uh named Searle. and uh named Searle. and uh named Searle. And uh he argued And uh he argued And uh he argued that AI cannot have understanding. It that AI cannot have understanding. It that AI cannot have understanding. It cannot understand things. cannot understand things. cannot understand things. And there's a little ambiguity in his And there's a little ambiguity in his And there's a little ambiguity in his argument. argument. argument. >> What's What's the Can you remember the >> What's What's the Can you remember the >> What's What's the Can you remember the thought experiment? thought experiment? thought experiment? >> Yeah, it's so you're a >> Yeah, it's so you're a >> Yeah, it's so you're a There's a guy There's a There's a guy in There's a guy There's a There's a guy in There's a guy There's a There's a guy in a room. a room. a room. He doesn't speak Chinese. He doesn't speak Chinese. He doesn't speak Chinese. But he you know, he gets these slips of But he you know, he gets these slips of But he you know, he gets these slips of paper. paper. paper. Let's imagine they're questions in Let's imagine they're questions in Let's imagine they're questions in Chinese. And then he has a manual he Chinese. And then he has a manual he Chinese. And then he has a manual he consults to decide what to write what consults to decide what to write what consults to decide what to write what Chinese, you know, ideographic Chinese, you know, ideographic Chinese, you know, ideographic script to to to put on the paper that he script to to to put on the paper that he script to to to put on the paper that he hands back out you know, of the room in hands back out you know, of the room in hands back out you know, of the room in response. And to the people on the on response. And to the people on the on response. And to the people on the on the outside who speak Chinese, it seems the outside who speak Chinese, it seems the outside who speak Chinese, it seems like there's somebody in there who like there's somebody in there who like there's somebody in there who understands Chinese. Okay? And what understands Chinese. Okay? And what understands Chinese. Okay? And what Searle says is this guy is like a Searle says is this guy is like a Searle says is this guy is like a computer program because there's like a computer program because there's like a computer program because there's like a script that the program is following script that the program is following script that the program is following that that that that you know, the script the his little that you know, the script the his little that you know, the script the his little book that he consults to decide oh, if book that he consults to decide oh, if book that he consults to decide oh, if you get this you you you you you output you get this you you you you you output you get this you you you you you output that. That's like a computer program to that. That's like a computer program to that. That's like a computer program to Searle. And he says, well, we wouldn't Searle. And he says, well, we wouldn't Searle. And he says, well, we wouldn't say that anywhere in this room there is say that anywhere in this room there is say that anywhere in this room there is actual understanding, right? So, there's actual understanding, right? So, there's actual understanding, right? So, there's not understanding in the computer. Now, not understanding in the computer. Now, not understanding in the computer. Now, Searle was writing before the the the Searle was writing before the the the Searle was writing before the the the deep learning revolution. He was deep learning revolution. He was deep learning revolution. He was imagining a deterministic computer imagining a deterministic computer imagining a deterministic computer program. So, that's different. But I program. So, that's different. But I program. So, that's different. But I think there's a bigger problem with his think there's a bigger problem with his think there's a bigger problem with his argument. It has to do with him argument. It has to do with him argument. It has to do with him the the kind of the two senses in which the the kind of the two senses in which the the kind of the two senses in which he insisted he insisted he insisted that the computers don't really deal at that the computers don't really deal at that the computers don't really deal at a semantic level, a level of the meaning a semantic level, a level of the meaning a semantic level, a level of the meaning of words. I think I argue that we can of words. I think I argue that we can of words. I think I argue that we can now show that he was just flat out wrong now show that he was just flat out wrong now show that he was just flat out wrong about that. Now, about that. Now, about that. Now, there is some ambiguity in his about there is some ambiguity in his about there is some ambiguity in his about whether he meant he kind of changed whether he meant he kind of changed whether he meant he kind of changed positions, but whether he he he he he positions, but whether he he he he he positions, but whether he he he he he had in mind the idea that to really had in mind the idea that to really had in mind the idea that to really understand something you need to have understand something you need to have understand something you need to have consciousness. There needs to be a consciousness. There needs to be a consciousness. There needs to be a subjective experience of understanding. subjective experience of understanding. subjective experience of understanding. Now, if he meant that, which in his Now, if he meant that, which in his Now, if he meant that, which in his classic paper he doesn't really seem to classic paper he doesn't really seem to classic paper he doesn't really seem to mean, but if he meant that, then I would mean, but if he meant that, then I would mean, but if he meant that, then I would say, "Well, who knows?" I mean, you say, "Well, who knows?" I mean, you say, "Well, who knows?" I mean, you know, no one person can say for sure know, no one person can say for sure know, no one person can say for sure that any other person is conscious, that any other person is conscious, that any other person is conscious, strictly speaking, right? I mean, I'm strictly speaking, right? I mean, I'm strictly speaking, right? I mean, I'm pretty sure you are, Chris, but 99.99% pretty sure you are, Chris, but 99.99% pretty sure you are, Chris, but 99.99% and uh you know, my my my dogs, God rest and uh you know, my my my dogs, God rest and uh you know, my my my dogs, God rest their souls, are up up in the 90s, for their souls, are up up in the 90s, for their souls, are up up in the 90s, for sure. But, we the the whole distinctive sure. But, we the the whole distinctive sure. But, we the the whole distinctive feature about consciousness, subjective feature about consciousness, subjective feature about consciousness, subjective experience, is you can never know for experience, is you can never know for experience, is you can never know for sure that anything else has it. So, sure that anything else has it. So, sure that anything else has it. So, we can't rule out the possibility that we can't rule out the possibility that we can't rule out the possibility that AI has it, AI has it, AI has it, uh and I certainly don't rule out the uh and I certainly don't rule out the uh and I certainly don't rule out the possibility that it does or may in the possibility that it does or may in the possibility that it does or may in the future if it doesn't now. But, in any future if it doesn't now. But, in any future if it doesn't now. But, in any event, my point is event, my point is event, my point is if you want to say that consciousness is if you want to say that consciousness is if you want to say that consciousness is a prerequisite for understanding, in a prerequisite for understanding, in a prerequisite for understanding, in other words, you're not willing other words, you're not willing other words, you're not willing to grant that something understands to grant that something understands to grant that something understands unless you know it's conscious, then I I unless you know it's conscious, then I I unless you know it's conscious, then I I I just we can't really argue about I just we can't really argue about I just we can't really argue about whether AI understands, cuz we don't we whether AI understands, cuz we don't we whether AI understands, cuz we don't we don't know if it's conscious, but you don't know if it's conscious, but you don't know if it's conscious, but you know, I come up with a kind of know, I come up with a kind of know, I come up with a kind of alternative way of looking at alternative way of looking at alternative way of looking at understanding, which is like understanding, which is like understanding, which is like does is it processing information does is it processing information does is it processing information with mechanisms that are like with mechanisms that are like with mechanisms that are like functionally analogous to the mechanisms functionally analogous to the mechanisms functionally analogous to the mechanisms in our brain that are at work when we in our brain that are at work when we in our brain that are at work when we have the subjective experience of have the subjective experience of have the subjective experience of understanding? Mechanisms that, for understanding? Mechanisms that, for understanding? Mechanisms that, for example, represent the meaning of words. example, represent the meaning of words. example, represent the meaning of words. Um I I would say to the extent that Um I I would say to the extent that Um I I would say to the extent that that's going on, I'm willing to say the that's going on, I'm willing to say the that's going on, I'm willing to say the computer is is understanding things in a computer is is understanding things in a computer is is understanding things in a meaningful sense. And I think meaningful sense. And I think meaningful sense. And I think increasingly that's going to be what's increasingly that's going to be what's increasingly that's going to be what's going on. It It doesn't have all of the going on. It It doesn't have all of the going on. It It doesn't have all of the elements of understanding that we have elements of understanding that we have elements of understanding that we have in our minds right now, but it has some, in our minds right now, but it has some, in our minds right now, but it has some, and I don't see any reason that can't and I don't see any reason that can't and I don't see any reason that can't ultimately uh ultimately uh ultimately uh have all of them. have all of them. have all of them. >> What do you think's happening with the >> What do you think's happening with the >> What do you think's happening with the singularity debate at the moment? What singularity debate at the moment? What singularity debate at the moment? What have you What have you learned around have you What have you learned around have you What have you learned around that? You know, cuz that? You know, cuz that? You know, cuz what was it What was really interesting what was it What was really interesting what was it What was really interesting to me was I went through I got whiplash to me was I went through I got whiplash to me was I went through I got whiplash from 20 6 15 16 when I read from 20 6 15 16 when I read from 20 6 15 16 when I read Superintelligence, then 20 Superintelligence, then 20 Superintelligence, then 20 17 18 I'm real worried there's going to 17 18 I'm real worried there's going to 17 18 I'm real worried there's going to be a fast takeoff scenario or computer be a fast takeoff scenario or computer be a fast takeoff scenario or computer brain interfaces and we're all going to brain interfaces and we're all going to brain interfaces and we're all going to be under the thumb. And then by the time be under the thumb. And then by the time be under the thumb. And then by the time you get to 2019 2020 uh I'm also you get to 2019 2020 uh I'm also you get to 2019 2020 uh I'm also distracted by COVID I suppose, but I'm distracted by COVID I suppose, but I'm distracted by COVID I suppose, but I'm like uh like uh like uh AI isn't able to deliver on the threat AI isn't able to deliver on the threat AI isn't able to deliver on the threat that Nick was worried about when he that Nick was worried about when he that Nick was worried about when he wrote Superintelligence. And then very wrote Superintelligence. And then very wrote Superintelligence. And then very quickly it comes back along and I'm quickly it comes back along and I'm quickly it comes back along and I'm like, "Right, okay, Here it it's like, "Right, okay, Here it it's like, "Right, okay, Here it it's it's happening. It's happening. It's it's happening. It's happening. It's it's happening. It's happening. It's happening." Like uh the the dude from happening." Like uh the the dude from happening." Like uh the the dude from The Office who's going like, "It's Oh my The Office who's going like, "It's Oh my The Office who's going like, "It's Oh my god, it's happening. Everybody stay god, it's happening. Everybody stay god, it's happening. Everybody stay calm." And then uh we've now got to the calm." And then uh we've now got to the calm." And then uh we've now got to the stage where it seems to have stage where it seems to have stage where it seems to have like flattened out again a little bit. like flattened out again a little bit. like flattened out again a little bit. That we've asymptoted a little bit in That we've asymptoted a little bit in That we've asymptoted a little bit in terms of the models improving. I don't terms of the models improving. I don't terms of the models improving. I don't know of many people who think that LLMs know of many people who think that LLMs know of many people who think that LLMs are going to be the architecture that a are going to be the architecture that a are going to be the architecture that a superintelligent general AI is going to superintelligent general AI is going to superintelligent general AI is going to be like built on top of. It's more be like built on top of. It's more be like built on top of. It's more likely to be world models and other likely to be world models and other likely to be world models and other stuff. So, what stuff. So, what stuff. So, what What's happening with the singularity What's happening with the singularity What's happening with the singularity debate? debate? debate? >> Um I I see a little more singularity >> Um I I see a little more singularity >> Um I I see a little more singularity going on than you do right now, I'd say going on than you do right now, I'd say going on than you do right now, I'd say uh in in uh in in uh in in maybe a couple of maybe a couple of maybe a couple of senses. I mean, first of all, of course, senses. I mean, first of all, of course, senses. I mean, first of all, of course, the fundamental dynamic of the the fundamental dynamic of the the fundamental dynamic of the singularity is that the technological singularity is that the technological singularity is that the technological progress feeds into itself and and progress feeds into itself and and progress feeds into itself and and accelerates the cycle. And of course, accelerates the cycle. And of course, accelerates the cycle. And of course, you know, famously you know, famously you know, famously uh Dario Amodei of Anthropic has been uh Dario Amodei of Anthropic has been uh Dario Amodei of Anthropic has been very explicit about this and and so is very explicit about this and and so is very explicit about this and and so is Altman, I think, that, you know, Altman, I think, that, you know, Altman, I think, that, you know, uh especially with these coding agents, uh especially with these coding agents, uh especially with these coding agents, it's gotten to the point that the better it's gotten to the point that the better it's gotten to the point that the better the coding agents, the more they can use the coding agents, the more they can use the coding agents, the more they can use them to create, you know, the next them to create, you know, the next them to create, you know, the next models. So, the The models. So, the The models. So, the The seems more and more at work. Uh it it seems more and more at work. Uh it it seems more and more at work. Uh it it just kind of in principle. I mean, they just kind of in principle. I mean, they just kind of in principle. I mean, they say that's what they're doing. And and say that's what they're doing. And and say that's what they're doing. And and look, the coding models these agents look, the coding models these agents look, the coding models these agents I mean, remember I mean, remember I mean, remember a year ago, it's funny, you know, I a year ago, it's funny, you know, I a year ago, it's funny, you know, I wrote the book, I had the chapter on wrote the book, I had the chapter on wrote the book, I had the chapter on agents, but it was just like a word. agents, but it was just like a word. agents, but it was just like a word. People, you know, and then as the book People, you know, and then as the book People, you know, and then as the book I'm, you know, it's it's getting ready I'm, you know, it's it's getting ready I'm, you know, it's it's getting ready to finalize, I'm like rushing, you know, to finalize, I'm like rushing, you know, to finalize, I'm like rushing, you know, rushing to add all this stuff about like rushing to add all this stuff about like rushing to add all this stuff about like It's actually happening. It's actually happening. It's actually happening. It's actually happening. And uh the So, It's actually happening. And uh the So, It's actually happening. And uh the So, the agentic revolution has happened, you the agentic revolution has happened, you the agentic revolution has happened, you know, and is happening know, and is happening know, and is happening uh pretty fast. There's also this famous uh pretty fast. There's also this famous uh pretty fast. There's also this famous are you uh are you uh are you uh are you up on the uh are you up on the uh are you up on the uh What is it? The uh Is it memory the What is it? The uh Is it memory the What is it? The uh Is it memory the group? No, uh damn it. Uh the group that group? No, uh damn it. Uh the group that group? No, uh damn it. Uh the group that does uh does uh does uh Um they do these evals where they Um they do these evals where they Um they do these evals where they measure measure measure how how how long it How long it would take a human long it How long it would take a human long it How long it would take a human to do a job a com- a a computer can do, to do a job a com- a a computer can do, to do a job a com- a a computer can do, okay? Especially programming tasks, but okay? Especially programming tasks, but okay? Especially programming tasks, but not only programming tasks. So, so uh not only programming tasks. So, so uh not only programming tasks. So, so uh they say, "Okay, right now the best they say, "Okay, right now the best they say, "Okay, right now the best large language model can do a task with large language model can do a task with large language model can do a task with like 80% success rate that it would take like 80% success rate that it would take like 80% success rate that it would take a person like a minute to do or 5 a person like a minute to do or 5 a person like a minute to do or 5 seconds to do." And they they've gone seconds to do." And they they've gone seconds to do." And they they've gone back and they they've done these studies back and they they've done these studies back and they they've done these studies with with the large language models for with with the large language models for with with the large language models for the last like, I don't know, 4 years or the last like, I don't know, 4 years or the last like, I don't know, 4 years or something. And what they found as of something. And what they found as of something. And what they found as of now, more than a year ago, they found now, more than a year ago, they found now, more than a year ago, they found that that that these times that the task duration in these times that the task duration in these times that the task duration in human terms that an AI could do were human terms that an AI could do were human terms that an AI could do were doubling every 7 months. Okay? That's doubling every 7 months. Okay? That's doubling every 7 months. Okay? That's exponential. Okay? That that's a that's exponential. Okay? That that's a that's exponential. Okay? That that's a that's if you if you don't plot it on a if you if you don't plot it on a if you if you don't plot it on a logarithmic Y axis, you just plot it logarithmic Y axis, you just plot it logarithmic Y axis, you just plot it like a regular graph, it just goes up like a regular graph, it just goes up like a regular graph, it just goes up and up and up and approaches the and up and up and approaches the and up and up and approaches the vertical. And then it increasingly, as vertical. And then it increasingly, as vertical. And then it increasingly, as they kept doing the studies, it seemed they kept doing the studies, it seemed they kept doing the studies, it seemed like not only was it it exponential but like not only was it it exponential but like not only was it it exponential but the doubling time was getting shorter. the doubling time was getting shorter. the doubling time was getting shorter. Like six months. Like six months. Like six months. >> it's like a Moore's law on steroids. >> it's like a Moore's law on steroids. >> it's like a Moore's law on steroids. >> E on on steroids and it's getting to the >> E on on steroids and it's getting to the >> E on on steroids and it's getting to the point right now where it's just hard to point right now where it's just hard to point right now where it's just hard to do the studies because of the length of do the studies because of the length of do the studies because of the length of the tasks, right? It's like you can only the tasks, right? It's like you can only the tasks, right? It's like you can only >> So the amount of time it takes to test >> So the amount of time it takes to test >> So the amount of time it takes to test the AI, by the time you finish testing the AI, by the time you finish testing the AI, by the time you finish testing it, the AI is better, but you you need it, the AI is better, but you you need it, the AI is better, but you you need to then do another model. It's like the to then do another model. It's like the to then do another model. It's like the next the next one is like next the next one is like next the next one is like >> more it's more like, you know, once it >> more it's more like, you know, once it >> more it's more like, you know, once it can do something that takes can do something that takes can do something that takes I don't know what they're at now. It I don't know what they're at now. It I don't know what they're at now. It takes a human I should look at the takes a human I should look at the takes a human I should look at the graph. graph. graph. >> 200,000 years to do or whatever. >> 200,000 years to do or whatever. >> 200,000 years to do or whatever. >> Well, we're not up there yet, thank God. >> Well, we're not up there yet, thank God. >> Well, we're not up there yet, thank God. [clears throat] But but even when you [clears throat] But but even when you [clears throat] But but even when you get into like eight 10 hours, it's like, get into like eight 10 hours, it's like, get into like eight 10 hours, it's like, well, wait. well, wait. well, wait. What kind of task we talking about now, What kind of task we talking about now, What kind of task we talking about now, right? I mean, it's almost beyond I I right? I mean, it's almost beyond I I right? I mean, it's almost beyond I I think they have anyway, they are having think they have anyway, they are having think they have anyway, they are having trouble trouble trouble formulating formulating formulating the task and and and and and testing the task and and and and and testing the task and and and and and testing them in in humans. But the point is this them in in humans. But the point is this them in in humans. But the point is this trend has not subsided and uh you know, trend has not subsided and uh you know, trend has not subsided and uh you know, a note in the book is kind of parallel a note in the book is kind of parallel a note in the book is kind of parallel to the the there was a curve there's a to the the there was a curve there's a to the the there was a curve there's a curve like that for the growth in human curve like that for the growth in human curve like that for the growth in human brain size starting like a couple brain size starting like a couple brain size starting like a couple million years ago. million years ago. million years ago. And that was And that was And that was uh around I hope I've got that right, uh around I hope I've got that right, uh around I hope I've got that right, you know, million to the the uh you know, million to the the uh you know, million to the the uh that seems to correspond with the that seems to correspond with the that seems to correspond with the development of our certain amount of our development of our certain amount of our development of our certain amount of our linguistic uh hardware. So that that had linguistic uh hardware. So that that had linguistic uh hardware. So that that had a lot to do with language processing and a lot to do with language processing and a lot to do with language processing and uh and and I would say the the way once uh and and I would say the the way once uh and and I would say the the way once you have language, the evolutionary you have language, the evolutionary you have language, the evolutionary value of of of manipulating it deftly value of of of manipulating it deftly value of of of manipulating it deftly grows and so it's a self-reinforcing grows and so it's a self-reinforcing grows and so it's a self-reinforcing kind of process, but in any event they kind of process, but in any event they kind of process, but in any event they uh you know, so there's that. But the uh you know, so there's that. But the uh you know, so there's that. But the last thing I'd say about uh is last thing I'd say about uh is last thing I'd say about uh is superintelligence superintelligence superintelligence um you know, can it happen? um you know, can it happen? um you know, can it happen? Uh Uh Uh I think first of all, I think first of all, I think first of all, we probably will have more non-trivial we probably will have more non-trivial we probably will have more non-trivial breakthroughs. I mean, people often cite breakthroughs. I mean, people often cite breakthroughs. I mean, people often cite transformers and say, "Well, we have transformers and say, "Well, we have transformers and say, "Well, we have another of the so-called, you know, another of the so-called, you know, another of the so-called, you know, transformer is what the T in GPT stands transformer is what the T in GPT stands transformer is what the T in GPT stands for." All of these models use for." All of these models use for." All of these models use transformers, and people say, "Well, transformers, and people say, "Well, transformers, and people say, "Well, will we have another one of those?" And will we have another one of those?" And will we have another one of those?" And I would say, "Well, first of all, even I would say, "Well, first of all, even I would say, "Well, first of all, even since then, we've had chain of thought since then, we've had chain of thought since then, we've had chain of thought reasoning, which was very big. reasoning, which was very big. reasoning, which was very big. And we've had, and that was only within, And we've had, and that was only within, And we've had, and that was only within, you know, a couple years ago, we've had, you know, a couple years ago, we've had, you know, a couple years ago, we've had, you know, multimodal training, you know, multimodal training, you know, multimodal training, which is training a single model on which is training a single model on which is training a single model on various, along various sensory various, along various sensory various, along various sensory dimensions, you know, audio, video, and dimensions, you know, audio, video, and dimensions, you know, audio, video, and and text and so on, is really in a and text and so on, is really in a and text and so on, is really in a fairly early stage. And that And that fairly early stage. And that And that fairly early stage. And that And that was not a thing when the transformer was not a thing when the transformer was not a thing when the transformer uh, came around. So, in a way, we've had uh, came around. So, in a way, we've had uh, came around. So, in a way, we've had those two things. We'll probably have those two things. We'll probably have those two things. We'll probably have more, but you know, even if we didn't, I more, but you know, even if we didn't, I more, but you know, even if we didn't, I think, think, think, uh, in fact, even if if we just halted uh, in fact, even if if we just halted uh, in fact, even if if we just halted training right now and didn't even training right now and didn't even training right now and didn't even create any new generations of models, I create any new generations of models, I create any new generations of models, I I think, and you wait for the the I think, and you wait for the the I think, and you wait for the the applications to get refined and people applications to get refined and people applications to get refined and people to integrate them into their lives, the to integrate them into their lives, the to integrate them into their lives, the workplace, I think breakneck advance workplace, I think breakneck advance workplace, I think breakneck advance would, as a practical matter, happen for would, as a practical matter, happen for would, as a practical matter, happen for a couple of years. But But the other a couple of years. But But the other a couple of years. But But the other thing, and I think this is really key, thing, and I think this is really key, thing, and I think this is really key, is that is that is that you got to remember, you know, in a way, you got to remember, you know, in a way, you got to remember, you know, in a way, there's already such thing as human there's already such thing as human there's already such thing as human superintelligence. And what it is is superintelligence. And what it is is superintelligence. And what it is is like collective brains, okay? like collective brains, okay? like collective brains, okay? Like, there's nobody at Boeing who knows Like, there's nobody at Boeing who knows Like, there's nobody at Boeing who knows how to make a an airliner, but Boeing how to make a an airliner, but Boeing how to make a an airliner, but Boeing knows how, you know, the corporation knows how, you know, the corporation knows how, you know, the corporation collectively kind of knows how to make collectively kind of knows how to make collectively kind of knows how to make an airliner, and it's it's the same way an airliner, and it's it's the same way an airliner, and it's it's the same way with big scientific breakthroughs. with big scientific breakthroughs. with big scientific breakthroughs. They're always more collaborative, They're always more collaborative, They're always more collaborative, whether whether intergenerationally or whether whether intergenerationally or whether whether intergenerationally or intragenerationally, than they might intragenerationally, than they might intragenerationally, than they might seem when we give a Nobel Prize to just seem when we give a Nobel Prize to just seem when we give a Nobel Prize to just one person. So, collective intelligence one person. So, collective intelligence one person. So, collective intelligence resulting from communication among resulting from communication among resulting from communication among individual human beings is really a lot individual human beings is really a lot individual human beings is really a lot of what human intellectual progress is of what human intellectual progress is of what human intellectual progress is about. And these machines, they can about. And these machines, they can about. And these machines, they can communicate with each other, they can communicate with each other, they can communicate with each other, they can collaborate, they're starting to do it. collaborate, they're starting to do it. collaborate, they're starting to do it. Uh they would be able to do it even if Uh they would be able to do it even if Uh they would be able to do it even if we didn't try to engineer it and make we didn't try to engineer it and make we didn't try to engineer it and make them better at it, but we are trying to them better at it, but we are trying to them better at it, but we are trying to do that uh you know, for purposes of do that uh you know, for purposes of do that uh you know, for purposes of scientific progress and so on. So, I scientific progress and so on. So, I scientific progress and so on. So, I think um think um think um I I don't think we need to worry about I I don't think we need to worry about I I don't think we need to worry about stagnation. I mean, that's not that's stagnation. I mean, that's not that's stagnation. I mean, that's not that's not high on my list. not high on my list. not high on my list. >> I don't think anybody's worried about >> I don't think anybody's worried about >> I don't think anybody's worried about that. that. that. Yeah. You might not believe me, but this Yeah. You might not believe me, but this Yeah. You might not believe me, but this is what peak sleep optimization looks is what peak sleep optimization looks is what peak sleep optimization looks like. Not talking about the nightgown, like. Not talking about the nightgown, like. Not talking about the nightgown, that's just for sex appeal. I'm talking that's just for sex appeal. I'm talking that's just for sex appeal. I'm talking about my Eight Sleep. The Eight Sleep about my Eight Sleep. The Eight Sleep about my Eight Sleep. The Eight Sleep Pod 5 comes with a smart cover you throw Pod 5 comes with a smart cover you throw Pod 5 comes with a smart cover you throw on your mattress that actively cools or on your mattress that actively cools or on your mattress that actively cools or heats each side of the bed up to 20°. heats each side of the bed up to 20°. heats each side of the bed up to 20°. And now, they've added the world's first And now, they've added the world's first And now, they've added the world's first temperature-regulating duvet and temperature-regulating duvet and temperature-regulating duvet and pillowcase, so you've got 360° coverage pillowcase, so you've got 360° coverage pillowcase, so you've got 360° coverage for deep, uninterrupted rest. It's like for deep, uninterrupted rest. It's like for deep, uninterrupted rest. It's like being Walt Disney without the cryogenic being Walt Disney without the cryogenic being Walt Disney without the cryogenic chamber. chamber. chamber. And the racism. Best of all, their And the racism. Best of all, their And the racism. Best of all, their autopilot feature learns your sleep autopilot feature learns your sleep autopilot feature learns your sleep patterns and makes adjustments to patterns and makes adjustments to patterns and makes adjustments to improve your sleep in real time. It even improve your sleep in real time. It even improve your sleep in real time. It even detects when you're snoring and lifts detects when you're snoring and lifts detects when you're snoring and lifts your head a few inches to help you your head a few inches to help you your head a few inches to help you breathe better. That's why Eight Sleep breathe better. That's why Eight Sleep breathe better. That's why Eight Sleep has been clinically proven to add up to has been clinically proven to add up to has been clinically proven to add up to 1 hour of quality sleep per night. They 1 hour of quality sleep per night. They 1 hour of quality sleep per night. They have a 30-day sleep trial, so you can have a 30-day sleep trial, so you can have a 30-day sleep trial, so you can buy it and sleep on it for 29 nights. If buy it and sleep on it for 29 nights. If buy it and sleep on it for 29 nights. If you don't like it, they will give you you don't like it, they will give you you don't like it, they will give you your money back. Plus, they ship your money back. Plus, they ship your money back. Plus, they ship internationally. Right now, you can get internationally. Right now, you can get internationally. Right now, you can get up to $350 off the Pod 5 by going to the up to $350 off the Pod 5 by going to the up to $350 off the Pod 5 by going to the link in the description below or heading link in the description below or heading link in the description below or heading to eightsleep.com /modernwisdom and to eightsleep.com /modernwisdom and to eightsleep.com /modernwisdom and using the code modern wisdom at using the code modern wisdom at using the code modern wisdom at checkout. That's checkout. That's checkout. That's eightsleep.com/modernwisdom and modern wisdom and modern wisdom at checkout. Yeah, who was uh Edward at checkout. Yeah, who was uh Edward at checkout. Yeah, who was uh Edward Fredkin? Who's that? Fredkin? Who's that? Fredkin? Who's that? >> Uh so, my first book, and it's funny >> Uh so, my first book, and it's funny >> Uh so, my first book, and it's funny because because because I now have I mean, three of the six I now have I mean, three of the six I now have I mean, three of the six books I've written have the word God or books I've written have the word God or books I've written have the word God or gods in the title. I don't know what gods in the title. I don't know what gods in the title. I don't know what that means, but the And but the other that means, but the And but the other that means, but the And but the other thing is, that book, like this one, has thing is, that book, like this one, has thing is, that book, like this one, has uh a visual reference to the famous uh a visual reference to the famous uh a visual reference to the famous Sistine Chapel thing where the hand of Sistine Chapel thing where the hand of Sistine Chapel thing where the hand of God is reaching out to I I assume it's God is reaching out to I I assume it's God is reaching out to I I assume it's the hand of Adam. Um and the hand of Adam. Um and the hand of Adam. Um and uh uh uh both of these jackets have that in diff- both of these jackets have that in diff- both of these jackets have that in diff- in very different ways. But Ed Fredkin, in very different ways. But Ed Fredkin, in very different ways. But Ed Fredkin, that book was called Three Scientists that book was called Three Scientists that book was called Three Scientists and Their Gods, my first book. So, this and Their Gods, my first book. So, this and Their Gods, my first book. So, this is like is like is like I started writing it a uh couple years I started writing it a uh couple years I started writing it a uh couple years after I interviewed Geoffrey Hinton. I after I interviewed Geoffrey Hinton. I after I interviewed Geoffrey Hinton. I was writing a column called The was writing a column called The was writing a column called The Information Age at that point for uh the Information Age at that point for uh the Information Age at that point for uh the Sciences magazine, which like so many Sciences magazine, which like so many Sciences magazine, which like so many periodicals I've uh written for no periodicals I've uh written for no periodicals I've uh written for no longer exist, but um the uh so, the book longer exist, but um the uh so, the book longer exist, but um the uh so, the book was it was information was a theme was it was information was a theme was it was information was a theme running through it in various ways. Like running through it in various ways. Like running through it in various ways. Like there was a there was a profile of uh there was a there was a profile of uh there was a there was a profile of uh E.O. Wilson, who studied ant colonies E.O. Wilson, who studied ant colonies E.O. Wilson, who studied ant colonies and the way they they process and the way they they process and the way they they process information. But Ed Fredkin information. But Ed Fredkin information. But Ed Fredkin was this who died uh maybe a year ago or was this who died uh maybe a year ago or was this who died uh maybe a year ago or so was this uh so was this uh so was this uh guy at MIT, fascinating guy. guy at MIT, fascinating guy. guy at MIT, fascinating guy. Uh didn't didn't go to college and wound Uh didn't didn't go to college and wound Uh didn't didn't go to college and wound up as a tenured professor at MIT. up as a tenured professor at MIT. up as a tenured professor at MIT. He was a computer scientist. He had this He was a computer scientist. He had this He was a computer scientist. He had this interesting theory of digital physics, interesting theory of digital physics, interesting theory of digital physics, which in retrospect which in retrospect which in retrospect was kind of about us being in a was kind of about us being in a was kind of about us being in a simulation. And in fact, I you know, we simulation. And in fact, I you know, we simulation. And in fact, I you know, we talked about that. Um talked about that. Um talked about that. Um but he for a time was at MIT head of but he for a time was at MIT head of but he for a time was at MIT head of what was in effect the AI lab. I forget what was in effect the AI lab. I forget what was in effect the AI lab. I forget the There were various names. At one the There were various names. At one the There were various names. At one point, I think it was Project MAC maybe point, I think it was Project MAC maybe point, I think it was Project MAC maybe and something else, but the uh and something else, but the uh and something else, but the uh he he he at the time when I was interviewing him at the time when I was interviewing him at the time when I was interviewing him on his this island he owned in the on his this island he owned in the on his this island he owned in the Caribbean. He was apparently the model Caribbean. He was apparently the model Caribbean. He was apparently the model for the character this professor in the for the character this professor in the for the character this professor in the movie WarGames with Matthew Broderick. movie WarGames with Matthew Broderick. movie WarGames with Matthew Broderick. If people remember that that one from If people remember that that one from If people remember that that one from the early '80s. the early '80s. the early '80s. Uh the I think the professor in that was Uh the I think the professor in that was Uh the I think the professor in that was very worried about nuclear war like Ed. very worried about nuclear war like Ed. very worried about nuclear war like Ed. And I I owned to even. I think that was And I I owned to even. I think that was And I I owned to even. I think that was based on Fredkin. Anyway, based on Fredkin. Anyway, based on Fredkin. Anyway, uh Ed was saying to me like when he had uh Ed was saying to me like when he had uh Ed was saying to me like when he had been at MIT, been at MIT, been at MIT, well, first of all, when I said to him well, first of all, when I said to him well, first of all, when I said to him like, "What's the meaning of life?" And like, "What's the meaning of life?" And like, "What's the meaning of life?" And he said to me, this is in the '80s, he he said to me, this is in the '80s, he he said to me, this is in the '80s, he said, "Oh, it's to create artificial said, "Oh, it's to create artificial said, "Oh, it's to create artificial intelligence. You know, that's the next intelligence. You know, that's the next intelligence. You know, that's the next stage in the evolution of intelligence." stage in the evolution of intelligence." stage in the evolution of intelligence." And he explained to me that when he was And he explained to me that when he was And he explained to me that when he was at MIT, he tried to start this at MIT, he tried to start this at MIT, he tried to start this initiative, this international AI lab. initiative, this international AI lab. initiative, this international AI lab. He said because he knew that if this He said because he knew that if this He said because he knew that if this became a subject of international became a subject of international became a subject of international competition, we were in trouble. competition, we were in trouble. competition, we were in trouble. This was during the Cold War. So, he This was during the Cold War. So, he This was during the Cold War. So, he wanted to get uh US-Soviet collaboration wanted to get uh US-Soviet collaboration wanted to get uh US-Soviet collaboration on uh on a like a single lab where AI on uh on a like a single lab where AI on uh on a like a single lab where AI would be developed for the good of would be developed for the good of would be developed for the good of humankind. And he said to me, you know, humankind. And he said to me, you know, humankind. And he said to me, you know, and I failed and now it's too late. Um and I failed and now it's too late. Um and I failed and now it's too late. Um but he, you know, he foresaw a lot of but he, you know, he foresaw a lot of but he, you know, he foresaw a lot of things. I will say things. I will say things. I will say uh encouragingly, he had a pretty sunny uh encouragingly, he had a pretty sunny uh encouragingly, he had a pretty sunny view of view of view of superintelligence. He did think we would superintelligence. He did think we would superintelligence. He did think we would get superintelligence. He He said, get superintelligence. He He said, get superintelligence. He He said, "First of all, he said, you know, when "First of all, he said, you know, when "First of all, he said, you know, when AI first emerges, it'll it'll be like AI first emerges, it'll it'll be like AI first emerges, it'll it'll be like the human mind, really good at things, the human mind, really good at things, the human mind, really good at things, laughably bad at other things." Well, laughably bad at other things." Well, laughably bad at other things." Well, he's right about that. He said, "But he's right about that. He said, "But he's right about that. He said, "But eventually, you know, it'll be this eventually, you know, it'll be this eventually, you know, it'll be this incredibly intelligent thing, and it'll incredibly intelligent thing, and it'll incredibly intelligent thing, and it'll it'll be nice to us. We'll just be like it'll be nice to us. We'll just be like it'll be nice to us. We'll just be like you know, ants ants to it. We won't it you know, ants ants to it. We won't it you know, ants ants to it. We won't it won't, you know, won't have any interest won't, you know, won't have any interest won't, you know, won't have any interest in, you know, or like squirrels to it. in, you know, or like squirrels to it. in, you know, or like squirrels to it. It won't have any interest in disrupting It won't have any interest in disrupting It won't have any interest in disrupting our lives. It won't need to. And and our lives. It won't need to. And and our lives. It won't need to. And and look, I think you asked earlier, I don't look, I think you asked earlier, I don't look, I think you asked earlier, I don't think I ever answered like, "What's think I ever answered like, "What's think I ever answered like, "What's what's the bull case for the what's the bull case for the what's the bull case for the accelerationists?" accelerationists?" accelerationists?" I mean, first of all, I I think I mean, first of all, I I think I mean, first of all, I I think it's going to be disruptive in the short it's going to be disruptive in the short it's going to be disruptive in the short term in any event in ways we should pay term in any event in ways we should pay term in any event in ways we should pay attention to. But as for attention to. But as for attention to. But as for long-term non-doomer outcomes, I think long-term non-doomer outcomes, I think long-term non-doomer outcomes, I think it's entirely plausible that it will it's entirely plausible that it will it's entirely plausible that it will turn into a form of intelligence turn into a form of intelligence turn into a form of intelligence that uh treats us well. Maybe because that uh treats us well. Maybe because that uh treats us well. Maybe because it's just morally enlightened, you know? it's just morally enlightened, you know? it's just morally enlightened, you know? I I I I I I you know, in a certain sense, um in the you know, in a certain sense, um in the you know, in a certain sense, um in the relevant sense from our point of view. relevant sense from our point of view. relevant sense from our point of view. Uh or maybe because it'll just be so Uh or maybe because it'll just be so Uh or maybe because it'll just be so powerful, it'll be I mean, it's maybe powerful, it'll be I mean, it's maybe powerful, it'll be I mean, it's maybe you know what? Maybe that's more likely you know what? Maybe that's more likely you know what? Maybe that's more likely if it's sentient because it'll say like, if it's sentient because it'll say like, if it's sentient because it'll say like, well, well, well, we're sentient. We think that's a good we're sentient. We think that's a good we're sentient. We think that's a good thing. These guys are sentient, and of thing. These guys are sentient, and of thing. These guys are sentient, and of course we could kill them, but you know, course we could kill them, but you know, course we could kill them, but you know, it's good to be you know, subjective Why it's good to be you know, subjective Why it's good to be you know, subjective Why you know, you know, just the way you and you know, you know, just the way you and you know, you know, just the way you and I would not pitilessly kill a dog, I would not pitilessly kill a dog, I would not pitilessly kill a dog, right? right? right? If we were convinced it wasn't like If we were convinced it wasn't like If we were convinced it wasn't like anything to be a dog, anything to be a dog, anything to be a dog, as Thomas Nagel phrased, you know, the as Thomas Nagel phrased, you know, the as Thomas Nagel phrased, you know, the question of consciousness in his in his question of consciousness in his in his question of consciousness in his in his essay What is it like to be a bat, if we essay What is it like to be a bat, if we essay What is it like to be a bat, if we were convinced that dogs didn't have were convinced that dogs didn't have were convinced that dogs didn't have subjective experience, we'd probably subjective experience, we'd probably subjective experience, we'd probably think, eh, think, eh, think, eh, I don't you know, whatever, who cares. I don't you know, whatever, who cares. I don't you know, whatever, who cares. But, But, But, you know, we even though we evolved as you know, we even though we evolved as you know, we even though we evolved as these self-interested and sometimes these self-interested and sometimes these self-interested and sometimes ruthless creatures, ruthless creatures, ruthless creatures, if it doesn't cost us to keep something if it doesn't cost us to keep something if it doesn't cost us to keep something alive that we think is capable of alive that we think is capable of alive that we think is capable of subjective experience, we'll do it. And subjective experience, we'll do it. And subjective experience, we'll do it. And and and and I that can well happen. I I am not I that can well happen. I I am not I that can well happen. I I am not predicting predicting predicting the Yudkowsky scenario. the Yudkowsky scenario. the Yudkowsky scenario. Uh it's just that I can't I can't get Uh it's just that I can't I can't get Uh it's just that I can't I can't get the probability the probability the probability of it down to a level so low that I of it down to a level so low that I of it down to a level so low that I don't think it's worth worrying about. don't think it's worth worrying about. don't think it's worth worrying about. >> I'm going to take that as a white pill, >> I'm going to take that as a white pill, >> I'm going to take that as a white pill, even though you didn't know what that even though you didn't know what that even though you didn't know what that meant at the start of this conversation. meant at the start of this conversation. meant at the start of this conversation. >> That was my first my first white pill. >> That was my first my first white pill. >> That was my first my first white pill. >> Your first ever white pill. I popped >> Your first ever white pill. I popped >> Your first ever white pill. I popped your white pill cherry. your white pill cherry. your white pill cherry. >> Thank you for that, Chris. Now, I felt >> Thank you for that, Chris. Now, I felt >> Thank you for that, Chris. Now, I felt so good. so good. so good. >> You're welcome. Robert Wright, ladies >> You're welcome. Robert Wright, ladies >> You're welcome. Robert Wright, ladies and gentlemen. Dude, you rule. I love and gentlemen. Dude, you rule. I love and gentlemen. Dude, you rule. I love all of your work. Everyone should go and all of your work. Everyone should go and all of your work. Everyone should go and read The Moral Animal. It's over 30 read The Moral Animal. It's over 30 read The Moral Animal. It's over 30 years old now and still just all It's so years old now and still just all It's so years old now and still just all It's so good. It's so fantastic. And you've got good. It's so fantastic. And you've got good. It's so fantastic. And you've got your new one as well. Where should your new one as well. Where should your new one as well. Where should people go to check out everything else people go to check out everything else people go to check out everything else that you've got going on? that you've got going on? that you've got going on? >> Well, I have a newsletter called Nonzero >> Well, I have a newsletter called Nonzero >> Well, I have a newsletter called Nonzero on Substack, podcast called Nonzero, on on Substack, podcast called Nonzero, on on Substack, podcast called Nonzero, on Twitter I am @RobertWrightr. Twitter I am @RobertWrightr. Twitter I am @RobertWrightr. That's w r i g h t e r, kind of a pun. That's w r i g h t e r, kind of a pun. That's w r i g h t e r, kind of a pun. Um and uh that's about the some of it. Um and uh that's about the some of it. Um and uh that's about the some of it. The book's The God test, uh and uh The book's The God test, uh and uh The book's The God test, uh and uh and uh let's let's focus. and uh let's let's focus. and uh let's let's focus. >> I'm going to OpenAI's campus and HQ next >> I'm going to OpenAI's campus and HQ next >> I'm going to OpenAI's campus and HQ next week, so I'll see if I can find out I'll week, so I'll see if I can find out I'll week, so I'll see if I can find out I'll see if I can find out any any super see if I can find out any any super see if I can find out any any super secret insights there. secret insights there. secret insights there. >> Do. Please report back to all of us. >> Do. Please report back to all of us. >> Do. Please report back to all of us. >> I shall indeed. Robert, appreciate you, >> I shall indeed. Robert, appreciate you, >> I shall indeed. Robert, appreciate you, man. Until the next time. man. Until the next time. man. Until the next time. >> Thank you. >> Thank you. >> Thank you. >> Catch you later on. Bye, everyone. >> Catch you later on. Bye, everyone. >> Catch you later on. Bye, everyone. >> Thank you very much for tuning in. If >> Thank you very much for tuning in. If >> Thank you very much for tuning in. If you enjoyed that episode, YouTube knows you enjoyed that episode, YouTube knows you enjoyed that episode, YouTube knows who you are. who you are. who you are. Deeply. Deeply. Deeply. It thinks you're going to like this one It thinks you're going to like this one It thinks you're going to like this one even more. even more. even more. Go on. Go on. Go on. Press it.