Why Scale Will Not Solve AGI | Vishal Misra - The a16z Show

Anthropic makes great products. Clot Anthropic makes great products. Clot code is fantastic. Co-work is fantastic. code is fantastic. Co-work is fantastic. code is fantastic. Co-work is fantastic. But they are grains of silicon doing But they are grains of silicon doing But they are grains of silicon doing matrix multiplication. They don't have matrix multiplication. They don't have matrix multiplication. They don't have consciousness. They don't have an inner consciousness. They don't have an inner consciousness. They don't have an inner monologue. You take an LLM and train it monologue. You take an LLM and train it monologue. You take an LLM and train it on pre 1916 or 1911 physics and see if on pre 1916 or 1911 physics and see if on pre 1916 or 1911 physics and see if it can come up with the theory of it can come up with the theory of it can come up with the theory of relativity. If it does, then we have relativity. If it does, then we have relativity. If it does, then we have AGI. AGI. AGI. >> Just today, by the way, Daario allegedly >> Just today, by the way, Daario allegedly >> Just today, by the way, Daario allegedly said that you can't rule out that said that you can't rule out that said that you can't rule out that they're conscious. You can rule out they're conscious. You can rule out they're conscious. You can rule out their cost. I think I mean come on to their cost. I think I mean come on to their cost. I think I mean come on to get to what is called AGI. I think there get to what is called AGI. I think there get to what is called AGI. I think there are two things that need to happen. One are two things that need to happen. One are two things that need to happen. One is >> Michelle. It's great to have you in >> Michelle. It's great to have you in again. again. again. >> Great to be back. This is one of my >> Great to be back. This is one of my >> Great to be back. This is one of my favorite topics which is um how do LLM favorite topics which is um how do LLM favorite topics which is um how do LLM actually work? actually work? actually work? >> And I think that uh you in my opinion >> And I think that uh you in my opinion >> And I think that uh you in my opinion you've done kind of the best work on you've done kind of the best work on you've done kind of the best work on this modeling it out. this modeling it out. this modeling it out. >> Thank you. For those that did not see >> Thank you. For those that did not see >> Thank you. For those that did not see the original um one, maybe it's probably the original um one, maybe it's probably the original um one, maybe it's probably worth doing just a quick background on worth doing just a quick background on worth doing just a quick background on kind of what led you to this point and kind of what led you to this point and kind of what led you to this point and then we'll just go into the current work then we'll just go into the current work then we'll just go into the current work that you've been doing. that you've been doing. that you've been doing. >> 5 years ago when GPD3 was first >> 5 years ago when GPD3 was first >> 5 years ago when GPD3 was first released, released, released, >> uh I got early access to it and I >> uh I got early access to it and I >> uh I got early access to it and I started playing with it and I was trying started playing with it and I was trying started playing with it and I was trying to solve a problem related to quering a to solve a problem related to quering a to solve a problem related to quering a cricket database. cricket database. cricket database. >> Yeah. And I got GPD3 >> Yeah. And I got GPD3 >> Yeah. And I got GPD3 to do in context learning, few short to do in context learning, few short to do in context learning, few short learning. And you know it was kind of learning. And you know it was kind of learning. And you know it was kind of the the the first at least to to me it was the first first at least to to me it was the first first at least to to me it was the first known uh implementation of rag retrieval known uh implementation of rag retrieval known uh implementation of rag retrieval augmented generation which I used to augmented generation which I used to augmented generation which I used to solve this problem of uh querying solve this problem of uh querying solve this problem of uh querying getting GPD3 to translate natural getting GPD3 to translate natural getting GPD3 to translate natural language into something that could be language into something that could be language into something that could be used to query a database that GPD3 had used to query a database that GPD3 had used to query a database that GPD3 had no idea about. I had no access to GPD3's no idea about. I had no access to GPD3's no idea about. I had no access to GPD3's internals, but I was still able to use internals, but I was still able to use internals, but I was still able to use it to solve that problem. So, it it it it to solve that problem. So, it it it it to solve that problem. So, it it it worked beautifully. Uh we we deployed uh worked beautifully. Uh we we deployed uh worked beautifully. Uh we we deployed uh this uh in production at ESPN in this uh in production at ESPN in this uh in production at ESPN in September 21. But September 21. But September 21. But >> Wow. Wow. You >> Wow. Wow. You >> Wow. Wow. You you did the first implementation of Frag you did the first implementation of Frag you did the first implementation of Frag in 2021. in 2021. in 2021. >> No, no, no. In 2020. >> No, no, no. In 2020. >> No, no, no. In 2020. >> 2020. >> 2020. >> 2020. >> 2020. I got it working and by the time >> 2020. I got it working and by the time >> 2020. I got it working and by the time you talked to all the lawyers at ESPN you talked to all the lawyers at ESPN you talked to all the lawyers at ESPN and you know, productionize it, it took and you know, productionize it, it took and you know, productionize it, it took it took a while. But October 2020 we had it took a while. But October 2020 we had it took a while. But October 2020 we had well I had this architecture working but well I had this architecture working but well I had this architecture working but after I got it to work I was amazed that after I got it to work I was amazed that after I got it to work I was amazed that it worked. I wanted to understand how it it worked. I wanted to understand how it it worked. I wanted to understand how it worked worked worked >> and I looked at >> and I looked at >> and I looked at you know the attention is all your deep you know the attention is all your deep you know the attention is all your deep papers and all the other sort of deep papers and all the other sort of deep papers and all the other sort of deep learning architecture papers and I learning architecture papers and I learning architecture papers and I couldn't understand why it worked. couldn't understand why it worked. couldn't understand why it worked. >> Yeah. >> Yeah. >> Yeah. >> So then I started getting sort of deep >> So then I started getting sort of deep >> So then I started getting sort of deep into building a mathematical model. into building a mathematical model. into building a mathematical model. >> Yeah. And now you published a series of >> Yeah. And now you published a series of >> Yeah. And now you published a series of papers. The first one that I read was papers. The first one that I read was papers. The first one that I read was the one where you had kind of your the one where you had kind of your the one where you had kind of your matrix kind of abstraction. So maybe matrix kind of abstraction. So maybe matrix kind of abstraction. So maybe we'll talk about that and then we'll we'll talk about that and then we'll we'll talk about that and then we'll talk about the more recent talk about the more recent talk about the more recent >> Yeah. >> Yeah. >> Yeah. >> work. >> work. >> work. >> So perhaps we'll just start with the >> So perhaps we'll just start with the >> So perhaps we'll just start with the first one which is first one which is first one which is >> you were trying to you were trying to >> you were trying to you were trying to >> you were trying to you were trying to describe you're trying to come up with a describe you're trying to come up with a describe you're trying to come up with a mathematical model of how LLM works. mathematical model of how LLM works. mathematical model of how LLM works. >> Yeah. >> Yeah. >> Yeah. >> And you had which was very helpful to me >> And you had which was very helpful to me >> And you had which was very helpful to me which was um and at the time you were which was um and at the time you were which was um and at the time you were actually trying to like figure out how actually trying to like figure out how actually trying to like figure out how incontext learning was working. incontext learning was working. incontext learning was working. >> Yes. Yeah. >> Yes. Yeah. >> Yes. Yeah. >> And you came up with an abstraction for >> And you came up with an abstraction for >> And you came up with an abstraction for LLMs which is basically this very very LLMs which is basically this very very LLMs which is basically this very very large matrix and you use that to large matrix and you use that to large matrix and you use that to describe. So maybe you can kind of walk describe. So maybe you can kind of walk describe. So maybe you can kind of walk through that work very quick. through that work very quick. through that work very quick. >> Sure. >> Sure. >> Sure. >> Yeah. So so what you do is you you >> Yeah. So so what you do is you you >> Yeah. So so what you do is you you imagine this huge gigantic matrix where imagine this huge gigantic matrix where imagine this huge gigantic matrix where every row of the matrix corresponds to a every row of the matrix corresponds to a every row of the matrix corresponds to a prompt. prompt. prompt. And the way these LLMs work is given a And the way these LLMs work is given a And the way these LLMs work is given a prompt they construct a distribution of prompt they construct a distribution of prompt they construct a distribution of probabilities of the next token. Next probabilities of the next token. Next probabilities of the next token. Next token is next word. So every LLM has a token is next word. So every LLM has a token is next word. So every LLM has a vocabulary, you know, GPD and its vocabulary, you know, GPD and its vocabulary, you know, GPD and its variants have a vocabulary of about variants have a vocabulary of about variants have a vocabulary of about 50,000 tokens. 50,000 tokens. 50,000 tokens. >> So given a prompt, it'll come up with a >> So given a prompt, it'll come up with a >> So given a prompt, it'll come up with a distribution of what the next token distribution of what the next token distribution of what the next token should be. And then all these models should be. And then all these models should be. And then all these models sample from that distribution. sample from that distribution. sample from that distribution. >> Yeah. So that's the posterior >> Yeah. So that's the posterior >> Yeah. So that's the posterior distribution. distribution. distribution. >> That's the posterior distribution, >> That's the posterior distribution, >> That's the posterior distribution, right? That that's how LLM work. And so right? That that's how LLM work. And so right? That that's how LLM work. And so the idea of this matrix is matrix is for the idea of this matrix is matrix is for the idea of this matrix is matrix is for every possible combination of tokens every possible combination of tokens every possible combination of tokens which is a prompt, there's a row. which is a prompt, there's a row. which is a prompt, there's a row. >> Yeah. Yeah. >> Yeah. Yeah. >> Yeah. Yeah. >> And the columns are a distribution over >> And the columns are a distribution over >> And the columns are a distribution over the vocabulary. the vocabulary. the vocabulary. >> So if you have like a vocabulary of >> So if you have like a vocabulary of >> So if you have like a vocabulary of 50,000 possible tokens, it's a 50,000 possible tokens, it's a 50,000 possible tokens, it's a distribution over distribution over distribution over >> those 50,000 tokens. >> those 50,000 tokens. >> those 50,000 tokens. >> And by distribution, it's just the >> And by distribution, it's just the >> And by distribution, it's just the probability probability probability >> just the probability. Sorry. Yeah. Just >> just the probability. Sorry. Yeah. Just >> just the probability. Sorry. Yeah. Just the probability that the next token the probability that the next token the probability that the next token should be this versus that. should be this versus that. should be this versus that. >> Y. >> Y. >> Y. >> Uh so that that's sort of the idea and >> Uh so that that's sort of the idea and >> Uh so that that's sort of the idea and and when you start viewing it that way, and when you start viewing it that way, and when you start viewing it that way, it makes things at least clearer to it makes things at least clearer to it makes things at least clearer to people like me who want to model it. uh people like me who want to model it. uh people like me who want to model it. uh what what's happening? So concretely what what's happening? So concretely what what's happening? So concretely let's say you have an example that let's say you have an example that let's say you have an example that uh let's say your prompt is just one uh let's say your prompt is just one uh let's say your prompt is just one word protein. Yeah. So if you look at word protein. Yeah. So if you look at word protein. Yeah. So if you look at the distribution of the next word next the distribution of the next word next the distribution of the next word next token after that uh most of the uh token after that uh most of the uh token after that uh most of the uh probabilities would be zero but you'd probabilities would be zero but you'd probabilities would be zero but you'd have non zero non-trivial probabilities have non zero non-trivial probabilities have non zero non-trivial probabilities on let's say two words one is synthesis on let's say two words one is synthesis on let's say two words one is synthesis the other is shake the other is shake the other is shake >> right and now the LLM is going to sample >> right and now the LLM is going to sample >> right and now the LLM is going to sample synthesis sample uh this next token and synthesis sample uh this next token and synthesis sample uh this next token and man pick synthesis or shake man pick synthesis or shake man pick synthesis or shake >> or you as a human will give the prompt >> or you as a human will give the prompt >> or you as a human will give the prompt protein shake protein shake protein shake >> or protein synthesis. Now, depending on >> or protein synthesis. Now, depending on >> or protein synthesis. Now, depending on whether you pick synthesis or shake, the whether you pick synthesis or shake, the whether you pick synthesis or shake, the next that row looks very different, next that row looks very different, next that row looks very different, right? If you pick protein synthesis, right? If you pick protein synthesis, right? If you pick protein synthesis, the terms that would have a high the terms that would have a high the terms that would have a high probability would be all concerned with probability would be all concerned with probability would be all concerned with biology, right? But if you pick protein biology, right? But if you pick protein biology, right? But if you pick protein shake, it'll all be about gyms and shake, it'll all be about gyms and shake, it'll all be about gyms and exercise and all, you know, bodybuilding exercise and all, you know, bodybuilding exercise and all, you know, bodybuilding stuff. So, that synthesis or shake stuff. So, that synthesis or shake stuff. So, that synthesis or shake completely changes what comes next. completely changes what comes next. completely changes what comes next. >> Yeah. So this is an example of uh you >> Yeah. So this is an example of uh you >> Yeah. So this is an example of uh you can say bijian updating. can say bijian updating. can say bijian updating. You start with protein you have a prior You start with protein you have a prior You start with protein you have a prior that after protein this is going to that after protein this is going to that after protein this is going to happen. As soon as you get new evidence happen. As soon as you get new evidence happen. As soon as you get new evidence then the next term is synthesis or shake then the next term is synthesis or shake then the next term is synthesis or shake you completely update the distribution. you completely update the distribution. you completely update the distribution. So now you can imagine that the whole So now you can imagine that the whole So now you can imagine that the whole the the entirety of LLM is this giant the the entirety of LLM is this giant the the entirety of LLM is this giant matrix where you have every row protein matrix where you have every row protein matrix where you have every row protein protein shake protein synthesis the cat protein shake protein synthesis the cat protein shake protein synthesis the cat sat on the sat on the sat on the >> you know Humpty Dumpty blah blah blah >> you know Humpty Dumpty blah blah blah >> you know Humpty Dumpty blah blah blah >> now given uh the vocabulary of uh these >> now given uh the vocabulary of uh these >> now given uh the vocabulary of uh these LLM let's say 50,000 and the context LLM let's say 50,000 and the context LLM let's say 50,000 and the context window so GPD for instance chat GPD the window so GPD for instance chat GPD the window so GPD for instance chat GPD the first version had a context window of first version had a context window of first version had a context window of 8,000 tokens. Yeah, if you look at all 8,000 tokens. Yeah, if you look at all 8,000 tokens. Yeah, if you look at all possible combinations of 8,000 tokens possible combinations of 8,000 tokens possible combinations of 8,000 tokens and 50,000 uh vocabulary, the number of and 50,000 uh vocabulary, the number of and 50,000 uh vocabulary, the number of rows rows rows in this matrix is more than the number in this matrix is more than the number in this matrix is more than the number of electrons across all galaxies. Right? of electrons across all galaxies. Right? of electrons across all galaxies. Right? >> So, so there's no way that these LLMs >> So, so there's no way that these LLMs >> So, so there's no way that these LLMs can represent it exactly now. can represent it exactly now. can represent it exactly now. Fortunately, this matrix is very sparse. Fortunately, this matrix is very sparse. Fortunately, this matrix is very sparse. Why? Because you know an arbitrary Why? Because you know an arbitrary Why? Because you know an arbitrary combination of these tokens is combination of these tokens is combination of these tokens is gibberish. We're not never going to use gibberish. We're not never going to use gibberish. We're not never going to use that in natural in real life. that in natural in real life. that in natural in real life. >> Yeah. >> Yeah. >> Yeah. >> Also, the columns are also mainly zero. >> Also, the columns are also mainly zero. >> Also, the columns are also mainly zero. >> Yeah. >> Yeah. >> Yeah. >> Right. If you have protein, then you >> Right. If you have protein, then you >> Right. If you have protein, then you won't have lots of, you know, you won't won't have lots of, you know, you won't won't have lots of, you know, you won't have arbitrary numbers or arbitrary have arbitrary numbers or arbitrary have arbitrary numbers or arbitrary words after that. It's very sparse both words after that. It's very sparse both words after that. It's very sparse both in rows and in columns. So I in kind of in rows and in columns. So I in kind of in rows and in columns. So I in kind of an abstract way what all these LLMs are an abstract way what all these LLMs are an abstract way what all these LLMs are doing is coming coming up with a doing is coming coming up with a doing is coming coming up with a compressed representation compressed representation compressed representation of this matrix and when you give a of this matrix and when you give a of this matrix and when you give a prompt they try to approximate what the prompt they try to approximate what the prompt they try to approximate what the true distribution should have been and true distribution should have been and true distribution should have been and try to generate it that that's what uh try to generate it that that's what uh try to generate it that that's what uh in my mind at least it boils up to in my mind at least it boils up to in my mind at least it boils up to >> just from my understanding. So if you >> just from my understanding. So if you >> just from my understanding. So if you have a row of uh protein have a row of uh protein have a row of uh protein and then you have one with protein shake and then you have one with protein shake and then you have one with protein shake >> Mhm. >> Mhm. >> Mhm. >> is protein shake a subset of protein or >> is protein shake a subset of protein or >> is protein shake a subset of protein or is it different? is it different? is it different? >> It's different. It's a continuation from >> It's different. It's a continuation from >> It's different. It's a continuation from >> I see. >> I see. >> I see. >> Yeah. >> Yeah. >> Yeah. >> Right. No, but I'm just saying like the >> Right. No, but I'm just saying like the >> Right. No, but I'm just saying like the actual the actual posterior distribution actual the actual posterior distribution actual the actual posterior distribution is that a subset? is that a subset? is that a subset? >> You you can say it's a subset, right? Uh >> You you can say it's a subset, right? Uh >> You you can say it's a subset, right? Uh if you have protein then protein shake if you have protein then protein shake if you have protein then protein shake and protein synthesis are all and protein synthesis are all and protein synthesis are all continuations from protein. So both continuations from protein. So both continuations from protein. So both synthesis and shake have non-zero synthesis and shake have non-zero synthesis and shake have non-zero probabilities. So you can yeah probabilities. So you can yeah probabilities. So you can yeah you can think of it as somewhat a subset you can think of it as somewhat a subset you can think of it as somewhat a subset >> right >> right >> right >> you you know you use this approach to >> you you know you use this approach to >> you you know you use this approach to describe how in context learning works describe how in context learning works describe how in context learning works and so maybe first describe what in and so maybe first describe what in and so maybe first describe what in context learning is and then kind of the context learning is and then kind of the context learning is and then kind of the conclusion that you came from that. So conclusion that you came from that. So conclusion that you came from that. So eight context learning is when you eight context learning is when you eight context learning is when you uh show the LLM something it has kind of uh show the LLM something it has kind of uh show the LLM something it has kind of never seen before. You give it a few never seen before. You give it a few never seen before. You give it a few examples of this is what it wants uh examples of this is what it wants uh examples of this is what it wants uh this is what you're trying to do. Then this is what you're trying to do. Then this is what you're trying to do. Then you give a new problem which is related you give a new problem which is related you give a new problem which is related to the examples that you have shown to the examples that you have shown to the examples that you have shown >> and the LLM learns in real time what >> and the LLM learns in real time what >> and the LLM learns in real time what it's supposed to do and solves that it's supposed to do and solves that it's supposed to do and solves that problem. And by the way, the first time problem. And by the way, the first time problem. And by the way, the first time I saw this, it absolutely blew my mind. I saw this, it absolutely blew my mind. I saw this, it absolutely blew my mind. And I actually I actually use your DSL And I actually I actually use your DSL And I actually I actually use your DSL >> when I was like first learning about it. >> when I was like first learning about it. >> when I was like first learning about it. So maybe like kind of like So maybe like kind of like So maybe like kind of like the DSL thing is just just crazy this the DSL thing is just just crazy this the DSL thing is just just crazy this works at all. works at all. works at all. >> It's absolutely, you know, mind-blowing >> It's absolutely, you know, mind-blowing >> It's absolutely, you know, mind-blowing that it works. And so going back to that that it works. And so going back to that that it works. And so going back to that cricket problem was you know cricket problem was you know cricket problem was you know in the mid '90s uh I was part of a group in the mid '90s uh I was part of a group in the mid '90s uh I was part of a group that had created this uh cricket portal that had created this uh cricket portal that had created this uh cricket portal called cricket info. called cricket info. called cricket info. >> Yeah. >> Yeah. >> Yeah. >> Uh cricket uh is a very start sport. You >> Uh cricket uh is a very start sport. You >> Uh cricket uh is a very start sport. You know you think baseball multiply by a know you think baseball multiply by a know you think baseball multiply by a thousand. at all kinds of stats and we thousand. at all kinds of stats and we thousand. at all kinds of stats and we had created this uh had created this uh had created this uh online searchable database called stats online searchable database called stats online searchable database called stats guru where you could search for anything guru where you could search for anything guru where you could search for anything any stat related to cricket and has been any stat related to cricket and has been any stat related to cricket and has been available since 2000 available since 2000 available since 2000 >> but because you can query for anything >> but because you can query for anything >> but because you can query for anything everything was be made available and how everything was be made available and how everything was be made available and how do you make something like that do you make something like that do you make something like that available to the general public well available to the general public well available to the general public well they're not going to write SQL queries they're not going to write SQL queries they're not going to write SQL queries >> the next best thing at that time was to >> the next best thing at that time was to >> the next best thing at that time was to create a web form unfortunately ally create a web form unfortunately ally create a web form unfortunately ally everything was crammed into that web everything was crammed into that web everything was crammed into that web form. So as a result you had like 20 form. So as a result you had like 20 form. So as a result you had like 20 drop downs, 15 checkboxes, 18 different drop downs, 15 checkboxes, 18 different drop downs, 15 checkboxes, 18 different text fields. It looked like a very text fields. It looked like a very text fields. It looked like a very complicated, daunting interface. So as a complicated, daunting interface. So as a complicated, daunting interface. So as a result, even though it could solve or it result, even though it could solve or it result, even though it could solve or it could answer any query, almost no one could answer any query, almost no one could answer any query, almost no one used it. A vanishingly small percentage used it. A vanishingly small percentage used it. A vanishingly small percentage of cricket fans use it because it it of cricket fans use it because it it of cricket fans use it because it it just looked intimidating. And then ESPN just looked intimidating. And then ESPN just looked intimidating. And then ESPN bought that site uh in 2007. bought that site uh in 2007. bought that site uh in 2007. I still know people who uh run the site I still know people who uh run the site I still know people who uh run the site and I always told them you know why and I always told them you know why and I always told them you know why don't you do something about stats guru don't you do something about stats guru don't you do something about stats guru and in January 2020 uh the and in January 2020 uh the and in January 2020 uh the editor-inchief of cricket info Sambbal editor-inchief of cricket info Sambbal editor-inchief of cricket info Sambbal he's he's a friend so he came to New he's he's a friend so he came to New he's he's a friend so he came to New York and we had gone out for rings and York and we had gone out for rings and York and we had gone out for rings and again I told him you know why don't you again I told him you know why don't you again I told him you know why don't you do something about stats guru so he do something about stats guru so he do something about stats guru so he looks at me and says why don't you do looks at me and says why don't you do looks at me and says why don't you do something about stats guru he was joking something about stats guru he was joking something about stats guru he was joking but uh that idea kind of stayed with me but uh that idea kind of stayed with me but uh that idea kind of stayed with me and when GP3 was released I thought and when GP3 was released I thought and when GP3 was released I thought maybe I could use stats guru maybe I could use stats guru maybe I could use stats guru use GP3 to create a front end for stats use GP3 to create a front end for stats use GP3 to create a front end for stats guru. guru. guru. >> And so what I did was uh I designed a >> And so what I did was uh I designed a >> And so what I did was uh I designed a DSL a domain specific language which uh DSL a domain specific language which uh DSL a domain specific language which uh converted queries about cricket stats in converted queries about cricket stats in converted queries about cricket stats in natural language into this DSL. natural language into this DSL. natural language into this DSL. >> No. >> No. >> No. >> And to be clear you created this it >> And to be clear you created this it >> And to be clear you created this it wasn't like part of like any training wasn't like part of like any training wasn't like part of like any training that was online that like could have that was online that like could have that was online that like could have seen. seen. seen. >> Nothing GPD could have seen. I created >> Nothing GPD could have seen. I created >> Nothing GPD could have seen. I created it. I thought okay this makes sense. So it. I thought okay this makes sense. So it. I thought okay this makes sense. So I designed that DSL and then I did that I designed that DSL and then I did that I designed that DSL and then I did that few short learning things. So I would so few short learning things. So I would so few short learning things. So I would so I created about a database of about I I created about a database of about I I created about a database of about I would say 1500 natural language queries would say 1500 natural language queries would say 1500 natural language queries and the DSL corresponding to that query. and the DSL corresponding to that query. and the DSL corresponding to that query. So when a new query came in, somebody's So when a new query came in, somebody's So when a new query came in, somebody's asking a stats question in English. What asking a stats question in English. What asking a stats question in English. What I would do is I would go through the I would do is I would go through the I would do is I would go through the natural language queries, do a semantic natural language queries, do a semantic natural language queries, do a semantic search, pick the most closely matching search, pick the most closely matching search, pick the most closely matching top few. top few. top few. >> Yeah. >> Yeah. >> Yeah. >> Uh and then use that natural language >> Uh and then use that natural language >> Uh and then use that natural language query and its DSL and send that as a query and its DSL and send that as a query and its DSL and send that as a prefix. Now GPD3, if you recall, had a prefix. Now GPD3, if you recall, had a prefix. Now GPD3, if you recall, had a context window of only 2,000 tokens. context window of only 2,000 tokens. context window of only 2,000 tokens. >> Yeah. So you had to be very judicious >> Yeah. So you had to be very judicious >> Yeah. So you had to be very judicious about which examples that you picked. about which examples that you picked. about which examples that you picked. But you pick that and then you send the But you pick that and then you send the But you pick that and then you send the new query and GP3 would complete it in new query and GP3 would complete it in new query and GP3 would complete it in the DSL that I had designed which until the DSL that I had designed which until the DSL that I had designed which until milliseconds ago it had never seen. milliseconds ago it had never seen. milliseconds ago it had never seen. >> Yeah. >> Yeah. >> Yeah. >> And I had no access to internals of >> And I had no access to internals of >> And I had no access to internals of GPD3. I had no access to the weights. GPD3. I had no access to the weights. GPD3. I had no access to the weights. >> Yeah. >> Yeah. >> Yeah. >> But still it worked. So that that's how >> But still it worked. So that that's how >> But still it worked. So that that's how so so so >> so so it's not obvious to me given your >> so so it's not obvious to me given your >> so so it's not obvious to me given your matrix example of like a prompt and then matrix example of like a prompt and then matrix example of like a prompt and then a distribution how something like in a distribution how something like in a distribution how something like in context learning context learning context learning >> works >> works >> works >> would work and so like I think your >> would work and so like I think your >> would work and so like I think your first paper first paper first paper >> tackled this problem >> tackled this problem >> tackled this problem >> right >> right >> right >> um and so maybe you could walk through >> um and so maybe you could walk through >> um and so maybe you could walk through your understanding of how LLMs your understanding of how LLMs your understanding of how LLMs do in context learning. do in context learning. do in context learning. >> Yeah. So, so when you think about what >> Yeah. So, so when you think about what >> Yeah. So, so when you think about what in context learning is is that in context learning is is that in context learning is is that as you see evidence. So, so you know in as you see evidence. So, so you know in as you see evidence. So, so you know in the first paper what I also did was I I the first paper what I also did was I I the first paper what I also did was I I took this cricket DSL example. took this cricket DSL example. took this cricket DSL example. >> Yeah. >> Yeah. >> Yeah. >> And I uh >> And I uh >> And I uh I depicted the next token probabilities I depicted the next token probabilities I depicted the next token probabilities >> mhm >> mhm >> mhm >> of the model as it was shown more and >> of the model as it was shown more and >> of the model as it was shown more and more examples. So the first time you more examples. So the first time you more examples. So the first time you show it this DSL the natural language show it this DSL the natural language show it this DSL the natural language and the DSL the probabilities of the DSL and the DSL the probabilities of the DSL and the DSL the probabilities of the DSL tokens were were extremely low because tokens were were extremely low because tokens were were extremely low because GP3 had never seen this thing. When it GP3 had never seen this thing. When it GP3 had never seen this thing. When it saw the cricket question saw the cricket question saw the cricket question in its mind it was trying to continue it in its mind it was trying to continue it in its mind it was trying to continue it with an English answer. with an English answer. with an English answer. So the probabilities that were high were So the probabilities that were high were So the probabilities that were high were all English words. all English words. all English words. >> Yeah. Once it saw my prompt where I had >> Yeah. Once it saw my prompt where I had >> Yeah. Once it saw my prompt where I had the question and the DSL, the next time the question and the DSL, the next time the question and the DSL, the next time I had the question in the next row, the I had the question in the next row, the I had the question in the next row, the probabilities of the DSL token started probabilities of the DSL token started probabilities of the DSL token started going up going up going up with every example, it went up and with every example, it went up and with every example, it went up and finally when I gave the new query, it finally when I gave the new query, it finally when I gave the new query, it was like it had almost 100% probability was like it had almost 100% probability was like it had almost 100% probability of getting the right token. of getting the right token. of getting the right token. >> Yeah. >> Yeah. >> Yeah. >> So this is an example of in real time >> So this is an example of in real time >> So this is an example of in real time the model was updating its posterior the model was updating its posterior the model was updating its posterior probability. It was upgrading its probability. It was upgrading its probability. It was upgrading its knowledge that okay I've seen evidence knowledge that okay I've seen evidence knowledge that okay I've seen evidence this is what I'm supposed to do. Now this is what I'm supposed to do. Now this is what I'm supposed to do. Now this is a colloquial way of saying what this is a colloquial way of saying what this is a colloquial way of saying what Beijian Beijian Beijian >> inference is. Beijian updating basically >> inference is. Beijian updating basically >> inference is. Beijian updating basically is you start with a prior when you see a is you start with a prior when you see a is you start with a prior when you see a new evidence you update your posterior. new evidence you update your posterior. new evidence you update your posterior. That's the mathematical division. But That's the mathematical division. But That's the mathematical division. But but in in English it's basically you see but in in English it's basically you see but in in English it's basically you see something you see new evidence you something you see new evidence you something you see new evidence you update your belief about what's update your belief about what's update your belief about what's happening. happening. happening. >> Yeah. >> Yeah. >> Yeah. >> Right. So it was clear to me that LLMs >> Right. So it was clear to me that LLMs >> Right. So it was clear to me that LLMs are doing something which resembles are doing something which resembles are doing something which resembles Beijian updating. So in that first paper Beijian updating. So in that first paper Beijian updating. So in that first paper I had this matrix formulation and I I had this matrix formulation and I I had this matrix formulation and I showed that you know what it's doing. It showed that you know what it's doing. It showed that you know what it's doing. It looks like Beijian updating. looks like Beijian updating. looks like Beijian updating. >> Yeah. >> Yeah. >> Yeah. >> Then we can come to the sort of next >> Then we can come to the sort of next >> Then we can come to the sort of next series of papers. series of papers. series of papers. >> That's right. So okay so I mean it it it >> That's right. So okay so I mean it it it >> That's right. So okay so I mean it it it seemed pretty conclusive to me at that seemed pretty conclusive to me at that seemed pretty conclusive to me at that time and then you went quiet for a while time and then you went quiet for a while time and then you went quiet for a while and then I still remember the WhatsApp and then I still remember the WhatsApp and then I still remember the WhatsApp text. You said Martin I know exactly how text. You said Martin I know exactly how text. You said Martin I know exactly how these things are working now. these things are working now. these things are working now. >> Yeah. Well >> Yeah. Well >> Yeah. Well >> and then and then and then listen you >> and then and then and then listen you >> and then and then and then listen you dropped a series of papers that kind of dropped a series of papers that kind of dropped a series of papers that kind of broke the internet. like you went super broke the internet. like you went super broke the internet. like you went super viral on Twitter like I mean people viral on Twitter like I mean people viral on Twitter like I mean people really noticed. Um uh and so I I want to really noticed. Um uh and so I I want to really noticed. Um uh and so I I want to get to that in just a second. But before get to that in just a second. But before get to that in just a second. But before that, um that, um that, um >> I remember when your first paper came >> I remember when your first paper came >> I remember when your first paper came out, people would be like, out, people would be like, out, people would be like, >> you know, these things are definitely >> you know, these things are definitely >> you know, these things are definitely not Beijian. Like, you know, you know, not Beijian. Like, you know, you know, not Beijian. Like, you know, you know, anything could be considered to be anything could be considered to be anything could be considered to be Beijian, but they're not. Like, why do Beijian, but they're not. Like, why do Beijian, but they're not. Like, why do you think that there was this reaction you think that there was this reaction you think that there was this reaction to like, to like, to like, >> you know, there's something new, they're >> you know, there's something new, they're >> you know, there's something new, they're not Beijian? I mean, I felt like there's not Beijian? I mean, I felt like there's not Beijian? I mean, I felt like there's almost kind of a backlash just because almost kind of a backlash just because almost kind of a backlash just because they're being characterized as they're being characterized as they're being characterized as >> Yeah. Yeah. I I think this whole world >> Yeah. Yeah. I I think this whole world >> Yeah. Yeah. I I think this whole world of uh uh probability and machine of uh uh probability and machine of uh uh probability and machine learning that there have been camps of learning that there have been camps of learning that there have been camps of Beijian and frequentists. Beijian and frequentists. Beijian and frequentists. >> Yes. >> Yes. >> Yes. >> And I don't want to get in the middle of >> And I don't want to get in the middle of >> And I don't want to get in the middle of that sort of political battle, but that sort of political battle, but that sort of political battle, but Beijian has become like almost like Beijian has become like almost like Beijian has become like almost like people had a reaction to that. It's it's people had a reaction to that. It's it's people had a reaction to that. It's it's part of that war. part of that war. part of that war. >> I see. >> I see. >> I see. >> So, it's like the old Beijian >> So, it's like the old Beijian >> So, it's like the old Beijian frequentist type battle. Yeah. So, so frequentist type battle. Yeah. So, so frequentist type battle. Yeah. So, so people just had oh no you can say people just had oh no you can say people just had oh no you can say anything is Beijian right? So I said anything is Beijian right? So I said anything is Beijian right? So I said okay maybe they have a point maybe what okay maybe they have a point maybe what okay maybe they have a point maybe what we are saying is not really Beijian how we are saying is not really Beijian how we are saying is not really Beijian how do we prove that it's Beijian do we prove that it's Beijian do we prove that it's Beijian >> right >> right >> right >> so then first I have to thank you and >> so then first I have to thank you and >> so then first I have to thank you and and Harovitz for this uh and Harovitz for this uh and Harovitz for this uh >> you know when I when I when I said that >> you know when I when I when I said that >> you know when I when I when I said that I in my first paper I showed these I in my first paper I showed these I in my first paper I showed these probabilities probabilities probabilities >> uh it was because open AI had in its >> uh it was because open AI had in its >> uh it was because open AI had in its chat uh interface chat uh interface chat uh interface uh this option to displays those uh this option to displays those uh this option to displays those probabilities then they stopped probabilities then they stopped probabilities then they stopped >> so we could not peer inside what's going >> so we could not peer inside what's going >> so we could not peer inside what's going what's happening for some reason they what's happening for some reason they what's happening for some reason they stopped openai stopped openai stopped openai I'm not going to get into I'm not going to get into I'm not going to get into >> the open and close but but they stopped >> the open and close but but they stopped >> the open and close but but they stopped so then we developed our own interface so then we developed our own interface so then we developed our own interface which could let you look not only at uh which could let you look not only at uh which could let you look not only at uh the probabilities but also the entropy the probabilities but also the entropy the probabilities but also the entropy of the next token was this on top of an of the next token was this on top of an of the next token was this on top of an open source model open source model open source model >> yeah yeah so so you can load any sort of >> yeah yeah so so you can load any sort of >> yeah yeah so so you can load any sort of open source model but you know being an open source model but you know being an open source model but you know being an academia academia academia We didn't have access to compute. Thanks We didn't have access to compute. Thanks We didn't have access to compute. Thanks to to to >> your generous uh uh donation, we got uh >> your generous uh uh donation, we got uh >> your generous uh uh donation, we got uh the clusters the clusters the clusters >> to run uh over what it's called token >> to run uh over what it's called token >> to run uh over what it's called token probe. So you can go to probe. So you can go to probe. So you can go to tokenprobe.cchs.colia.edu. tokenprobe.cchs.colia.edu. tokenprobe.cchs.colia.edu. >> Is it still running? >> Is it still running? >> Is it still running? >> It's still running. It's still running >> It's still running. It's still running >> It's still running. It's still running and people come to it. Uh I use it in my and people come to it. Uh I use it in my and people come to it. Uh I use it in my classes uh to get students to do classes uh to get students to do classes uh to get students to do assignments. They write their own DSLs assignments. They write their own DSLs assignments. They write their own DSLs and you know they say that that it and you know they say that that it and you know they say that that it really helps them understand how these really helps them understand how these really helps them understand how these LLMs work. So I literally my LLMs work. So I literally my LLMs work. So I literally my understanding of LMS came from token pro understanding of LMS came from token pro understanding of LMS came from token pro just you know sit there and just look at just you know sit there and just look at just you know sit there and just look at the the distribution as as you filled the the distribution as as you filled the the distribution as as you filled out a prompt. It's actually very very out a prompt. It's actually very very out a prompt. It's actually very very enlightening. So for those of you that enlightening. So for those of you that enlightening. So for those of you that are listening um are listening um are listening um >> what's the URL again? >> what's the URL again? >> what's the URL again? >> Token probe >> Token probe >> Token probe >> token probe.cs.colia.edu. >> token probe.cs.colia.edu. >> token probe.cs.colia.edu. >> Yeah check it out. It's actually very >> Yeah check it out. It's actually very >> Yeah check it out. It's actually very very useful way to I can actually see very useful way to I can actually see very useful way to I can actually see how the probability distribution gets how the probability distribution gets how the probability distribution gets updated as as you fill out a prompt. updated as as you fill out a prompt. updated as as you fill out a prompt. >> Right. Yeah. >> Right. Yeah. >> Right. Yeah. >> But then I cheated. >> But then I cheated. >> But then I cheated. Oh, Oh, Oh, >> I you know it was running >> I you know it was running >> I you know it was running >> but I also had access to the GPUs that >> but I also had access to the GPUs that >> but I also had access to the GPUs that were powering it. were powering it. were powering it. >> Mhm. >> Mhm. >> Mhm. >> And then along with colleagues at >> And then along with colleagues at >> And then along with colleagues at Colombia and one of them now is uh is at Colombia and one of them now is uh is at Colombia and one of them now is uh is at deep mind we started to sort of think deep mind we started to sort of think deep mind we started to sort of think about how do you really prove that it's about how do you really prove that it's about how do you really prove that it's Beijing Beijing Beijing to prove to prove to prove >> Can you just explain it? Actually I I >> Can you just explain it? Actually I I >> Can you just explain it? Actually I I actually don't know the answer to this. actually don't know the answer to this. actually don't know the answer to this. >> Yeah. It seemed to me you proved it in >> Yeah. It seemed to me you proved it in >> Yeah. It seemed to me you proved it in the first paper like what was missing. the first paper like what was missing. the first paper like what was missing. >> Well, in the first paper we showed it. >> Well, in the first paper we showed it. >> Well, in the first paper we showed it. It was empirical It was empirical It was empirical >> and you could see I see I see you could >> and you could see I see I see you could >> and you could see I see I see you could see not a mathematical because it was see not a mathematical because it was see not a mathematical because it was obvious to me that obvious to me that obvious to me that >> yeah it was even obvious to me but to >> yeah it was even obvious to me but to >> yeah it was even obvious to me but to convince uh you you could say you know convince uh you you could say you know convince uh you you could say you know people who dismiss oh anything can be people who dismiss oh anything can be people who dismiss oh anything can be based in based in based in >> I see I see >> I see I see >> I see I see >> we had to show it precisely >> we had to show it precisely >> we had to show it precisely mathematically. mathematically. mathematically. >> Got it. Got it. So then we came up with >> Got it. Got it. So then we came up with >> Got it. Got it. So then we came up with this idea you know my colleagues at this idea you know my colleagues at this idea you know my colleagues at Namanagaral and Sedhad Dalal we the Namanagaral and Sedhad Dalal we the Namanagaral and Sedhad Dalal we the series of papers were were written with series of papers were were written with series of papers were were written with them. We came up with this idea of a them. We came up with this idea of a them. We came up with this idea of a Beijian wind tunnel. Okay so what's a Beijian wind tunnel. Okay so what's a Beijian wind tunnel. Okay so what's a wind tunnel? Well wind tunnel in the wind tunnel? Well wind tunnel in the wind tunnel? Well wind tunnel in the aerospace industry is where you test an aerospace industry is where you test an aerospace industry is where you test an aircraft in an isolated environment. you aircraft in an isolated environment. you aircraft in an isolated environment. you don't fly it and you test test it don't fly it and you test test it don't fly it and you test test it against all sorts of uh uh you know against all sorts of uh uh you know against all sorts of uh uh you know aerodynamic pressure then you see what aerodynamic pressure then you see what aerodynamic pressure then you see what what it'll withstand what kind of what it'll withstand what kind of what it'll withstand what kind of altitude pressure blah blah blah right altitude pressure blah blah blah right altitude pressure blah blah blah right you don't want to do it up in the air you don't want to do it up in the air you don't want to do it up in the air testing testing testing >> so we said okay why don't we create an >> so we said okay why don't we create an >> so we said okay why don't we create an environment where we take these environment where we take these environment where we take these architectures and we tested transformers architectures and we tested transformers architectures and we tested transformers mamba LSTMs MLPS all architectures we mamba LSTMs MLPS all architectures we mamba LSTMs MLPS all architectures we say why don't we create take a blank say why don't we create take a blank say why don't we create take a blank architecture. architecture. architecture. Give it a task where it's impossible for Give it a task where it's impossible for Give it a task where it's impossible for the architecture to memorize what the the architecture to memorize what the the architecture to memorize what the solution to that task should be. solution to that task should be. solution to that task should be. The space is combinatorily The space is combinatorily The space is combinatorily impossible for given the number of impossible for given the number of impossible for given the number of parameters and we took very small parameters and we took very small parameters and we took very small models. models. models. So it's difficult enough that they So it's difficult enough that they So it's difficult enough that they cannot memorize it cannot memorize it cannot memorize it >> but it's tractable enough that we know >> but it's tractable enough that we know >> but it's tractable enough that we know precisely what the the Beijian posterior precisely what the the Beijian posterior precisely what the the Beijian posterior should be. You can calculate it should be. You can calculate it should be. You can calculate it analytically. analytically. analytically. So we gave these models a bunch of tasks So we gave these models a bunch of tasks So we gave these models a bunch of tasks where again we show that it's impossible where again we show that it's impossible where again we show that it's impossible to memorize. We trained these models and to memorize. We trained these models and to memorize. We trained these models and we found that the transformer got the we found that the transformer got the we found that the transformer got the precise Beijian posterior down to 10 ^ precise Beijian posterior down to 10 ^ precise Beijian posterior down to 10 ^ minus 3 bits accuracy. It was matching minus 3 bits accuracy. It was matching minus 3 bits accuracy. It was matching the distribution perfectly. So it is the distribution perfectly. So it is the distribution perfectly. So it is actually doing Beijian in the actually doing Beijian in the actually doing Beijian in the mathematical sense given a task mathematical sense given a task mathematical sense given a task >> where it has to update its belief. Uh >> where it has to update its belief. Uh >> where it has to update its belief. Uh Mamba also does it reasonably well. Mamba also does it reasonably well. Mamba also does it reasonably well. LSTMs can do one of the things. So the LSTMs can do one of the things. So the LSTMs can do one of the things. So the in the papers we have a taxonomy of in the papers we have a taxonomy of in the papers we have a taxonomy of Beijing task. Transformer does Beijing task. Transformer does Beijing task. Transformer does everything. Mamba does most of it. LSTMs everything. Mamba does most of it. LSTMs everything. Mamba does most of it. LSTMs do only partially and MLPs fail do only partially and MLPs fail do only partially and MLPs fail completely. completely. completely. >> So is this a reflection of the data that >> So is this a reflection of the data that >> So is this a reflection of the data that it's trained on or is it more a it's trained on or is it more a it's trained on or is it more a reflection of the mechanism? reflection of the mechanism? reflection of the mechanism? >> It's the mechanism. It's the >> It's the mechanism. It's the >> It's the mechanism. It's the architecture. architecture. architecture. The data decides what tasks it learns. The data decides what tasks it learns. The data decides what tasks it learns. >> Right? So in the first paper we had >> Right? So in the first paper we had >> Right? So in the first paper we had these beijian wind tunnels and we show these beijian wind tunnels and we show these beijian wind tunnels and we show that you know it's doing the job where that you know it's doing the job where that you know it's doing the job where different tasks in the second paper we different tasks in the second paper we different tasks in the second paper we show why it does it. So we look at the show why it does it. So we look at the show why it does it. So we look at the transformers we look at the gradients transformers we look at the gradients transformers we look at the gradients and we show how the gradients actually and we show how the gradients actually and we show how the gradients actually shape this geometry shape this geometry shape this geometry which enables this basin updating to which enables this basin updating to which enables this basin updating to happen. happen. happen. Then in the third paper what we did we Then in the third paper what we did we Then in the third paper what we did we take we took these frontier production take we took these frontier production take we took these frontier production LLMs which have open weights so that we LLMs which have open weights so that we LLMs which have open weights so that we could look inside them and we did our could look inside them and we did our could look inside them and we did our testing and we saw that the geometries testing and we saw that the geometries testing and we saw that the geometries that we saw in the small models that we saw in the small models that we saw in the small models persisted in models which are you know persisted in models which are you know persisted in models which are you know hundreds of millions of parameters the hundreds of millions of parameters the hundreds of millions of parameters the same signature existed. The only thing same signature existed. The only thing same signature existed. The only thing is that uh because they are trained on is that uh because they are trained on is that uh because they are trained on all sorts of data, it's a little bit all sorts of data, it's a little bit all sorts of data, it's a little bit dirty or messy. dirty or messy. dirty or messy. >> Yeah. >> Yeah. >> Yeah. >> But you can see the same structure. So >> But you can see the same structure. So >> But you can see the same structure. So the the whole idea behind the Beijian the the whole idea behind the Beijian the the whole idea behind the Beijian wind tunnel was unlike these production wind tunnel was unlike these production wind tunnel was unlike these production LLMs where you don't know what they have LLMs where you don't know what they have LLMs where you don't know what they have been trained on, been trained on, been trained on, >> right? >> right? >> right? >> So you cannot mathematically compute the >> So you cannot mathematically compute the >> So you cannot mathematically compute the posterior. posterior. posterior. >> So again, how do you prove it? >> So again, how do you prove it? >> So again, how do you prove it? >> I mean it looks based in you know from >> I mean it looks based in you know from >> I mean it looks based in you know from the first paper. the first paper. the first paper. >> From the first it looks Beijian, but you >> From the first it looks Beijian, but you >> From the first it looks Beijian, but you know. So the wind tunnel sort of solved know. So the wind tunnel sort of solved know. So the wind tunnel sort of solved that problem for us. We said okay let's that problem for us. We said okay let's that problem for us. We said okay let's start with a blank architecture. Give it start with a blank architecture. Give it start with a blank architecture. Give it a task where we know what the answer is. a task where we know what the answer is. a task where we know what the answer is. It cannot memorize it. Let's see what it It cannot memorize it. Let's see what it It cannot memorize it. Let's see what it does. And does. And does. And >> so do you think this provides any sort >> so do you think this provides any sort >> so do you think this provides any sort of like indication of how humans think of like indication of how humans think of like indication of how humans think or do you think that these things are or do you think that these things are or do you think that these things are totally independent? totally independent? totally independent? >> No no it it does provide right. So you >> No no it it does provide right. So you >> No no it it does provide right. So you know human beings also know human beings also know human beings also uh update our beliefs as we see new uh update our beliefs as we see new uh update our beliefs as we see new evidence. Right. So we do in some sort evidence. Right. So we do in some sort evidence. Right. So we do in some sort of in some sense uh Beijian updating but of in some sense uh Beijian updating but of in some sense uh Beijian updating but we do something more than that I'll come we do something more than that I'll come we do something more than that I'll come to that but uh these transformers uh or to that but uh these transformers uh or to that but uh these transformers uh or even mamba do this beijian updating even mamba do this beijian updating even mamba do this beijian updating >> and uh but but but the difference with >> and uh but but but the difference with >> and uh but but but the difference with humans is humans is humans is you know we we'll update our posterior you know we we'll update our posterior you know we we'll update our posterior when we see some new evidence but the when we see some new evidence but the when we see some new evidence but the way our brains have evolved evolved over way our brains have evolved evolved over way our brains have evolved evolved over hundreds of millions of years is our hundreds of millions of years is our hundreds of millions of years is our optimization objective has been don't optimization objective has been don't optimization objective has been don't die and reproduce. Right? That's been die and reproduce. Right? That's been die and reproduce. Right? That's been sort of the driving force and our brains sort of the driving force and our brains sort of the driving force and our brains have learned to adjust and so when we have learned to adjust and so when we have learned to adjust and so when we see some danger see some danger see some danger there's some something rustling in that there's some something rustling in that there's some something rustling in that bush. Don't go near. We know how to bush. Don't go near. We know how to bush. Don't go near. We know how to react to that danger. We know how to uh react to that danger. We know how to uh react to that danger. We know how to uh save ourselves. save ourselves. save ourselves. We internalize that We internalize that We internalize that learning and our brain cells or our learning and our brain cells or our learning and our brain cells or our synapses remain plastic throughout our synapses remain plastic throughout our synapses remain plastic throughout our lifetime. What happens with LLM is once lifetime. What happens with LLM is once lifetime. What happens with LLM is once the training is done those weights are the training is done those weights are the training is done those weights are frozen. frozen. frozen. when you're doing an inference for when you're doing an inference for when you're doing an inference for instance in context learning or anything instance in context learning or anything instance in context learning or anything during that conversation during that conversation during that conversation okay you're doing bijian inference but okay you're doing bijian inference but okay you're doing bijian inference but then you forget then you forget then you forget the next time a new conversation starts the next time a new conversation starts the next time a new conversation starts with zero context you don't retain any with zero context you don't retain any with zero context you don't retain any learning that happened in the previous learning that happened in the previous learning that happened in the previous instance so so for instance with the instance so so for instance with the instance so so for instance with the cricket DSL that I was doing every cricket DSL that I was doing every cricket DSL that I was doing every invocation of it was fresh invocation of it was fresh invocation of it was fresh it did not remember the last time I sent it did not remember the last time I sent it did not remember the last time I sent a query what the DSL looked a query what the DSL looked a query what the DSL looked So that's one difference between uh So that's one difference between uh So that's one difference between uh how humans uh uh use sort of beijan how humans uh uh use sort of beijan how humans uh uh use sort of beijan updating which is we remain plastic all updating which is we remain plastic all updating which is we remain plastic all our lives our lives our lives >> whereas uh LMS are frozen and there's >> whereas uh LMS are frozen and there's >> whereas uh LMS are frozen and there's another uh sort of another uh sort of another uh sort of difference which uh if you want me to difference which uh if you want me to difference which uh if you want me to get get get >> tell me yeah yeah yeah >> tell me yeah yeah yeah >> tell me yeah yeah yeah >> so so the other difference is uh u >> so so the other difference is uh u >> so so the other difference is uh u well first you know our objective ive is well first you know our objective ive is well first you know our objective ive is don't die reproduce. LLM's objective is don't die reproduce. LLM's objective is don't die reproduce. LLM's objective is predict the next token as accurately as predict the next token as accurately as predict the next token as accurately as possible. Right? So all these uh possible. Right? So all these uh possible. Right? So all these uh scary stories that you you read about scary stories that you you read about scary stories that you you read about that oh the LLM tried to deceive and it that oh the LLM tried to deceive and it that oh the LLM tried to deceive and it tried to prevent itself from being shut tried to prevent itself from being shut tried to prevent itself from being shut down. That's not a function of the down. That's not a function of the down. That's not a function of the architecture. architecture. architecture. That's a function of the training data. That's a function of the training data. That's a function of the training data. >> It has been fed you know articles on >> It has been fed you know articles on >> It has been fed you know articles on Reddit or SMO or whatever. I mean, just Reddit or SMO or whatever. I mean, just Reddit or SMO or whatever. I mean, just today, by the way, Daario today, by the way, Daario today, by the way, Daario >> allegedly said that uh you can't rule >> allegedly said that uh you can't rule >> allegedly said that uh you can't rule out that they're conscious. out that they're conscious. out that they're conscious. >> You can rule out their I mean, come on. >> You can rule out their I mean, come on. >> You can rule out their I mean, come on. And I said, you know, Antropic makes And I said, you know, Antropic makes And I said, you know, Antropic makes great products. Cloud code is fantastic. great products. Cloud code is fantastic. great products. Cloud code is fantastic. Coco work is fantastic, Coco work is fantastic, Coco work is fantastic, but they are grains of silicon doing but they are grains of silicon doing but they are grains of silicon doing matrix multiplication. matrix multiplication. matrix multiplication. They don't have consciousness. They They don't have consciousness. They They don't have consciousness. They don't have an inner monologue. They don't have an inner monologue. They don't have an inner monologue. They don't uh they're not driven by the same don't uh they're not driven by the same don't uh they're not driven by the same objective function. Don't die, objective function. Don't die, objective function. Don't die, reproduce, right? They're driven by reproduce, right? They're driven by reproduce, right? They're driven by don't make a mistake on the next token. don't make a mistake on the next token. don't make a mistake on the next token. And that's driven entirely by the And that's driven entirely by the And that's driven entirely by the training data, training data, training data, right? You train the LLM with stories of right? You train the LLM with stories of right? You train the LLM with stories of ASMO or Reddit where you know to survive ASMO or Reddit where you know to survive ASMO or Reddit where you know to survive it's going to do this or that. It'll it's going to do this or that. It'll it's going to do this or that. It'll reproduce that. So it's it it's a reproduce that. So it's it it's a reproduce that. So it's it it's a reflection. It's not a mind. And and the reflection. It's not a mind. And and the reflection. It's not a mind. And and the results, just to say it for the 10th results, just to say it for the 10th results, just to say it for the 10th time, are perfectly vision. time, are perfectly vision. time, are perfectly vision. >> Perfectly. Yeah. >> Perfectly. Yeah. >> Perfectly. Yeah. >> To the to the to the digit. >> To the to the to the digit. >> To the to the to the digit. >> To the digit. Yeah. I mean, I I trained >> To the digit. Yeah. I mean, I I trained >> To the digit. Yeah. I mean, I I trained it for 150,000 steps it for 150,000 steps it for 150,000 steps >> and uh the accuracy was 10 ^ minus 3 >> and uh the accuracy was 10 ^ minus 3 >> and uh the accuracy was 10 ^ minus 3 bits. bits. bits. >> I could have trained it for you know >> I could have trained it for you know >> I could have trained it for you know this happened in half an hour this happened in half an hour this happened in half an hour >> on the infrastructure that you provided >> on the infrastructure that you provided >> on the infrastructure that you provided for token pro in the background. I could for token pro in the background. I could for token pro in the background. I could use those APUs to train. But uh so thank use those APUs to train. But uh so thank use those APUs to train. But uh so thank you again for that. But so no human you again for that. But so no human you again for that. But so no human beings coming back to it, we we are beings coming back to it, we we are beings coming back to it, we we are >> Beijian, >> Beijian, >> Beijian, >> but we do something else. You know when >> but we do something else. You know when >> but we do something else. You know when I when I when I throw this pen at you, I when I when I throw this pen at you, I when I when I throw this pen at you, what will you do? what will you do? what will you do? >> Dodge it or >> Dodge it or >> Dodge it or >> do it? Yeah. >> do it? Yeah. >> do it? Yeah. >> Why will you dodge it? >> Why will you dodge it? >> Why will you dodge it? >> To avoid being hit. >> To avoid being hit. >> To avoid being hit. >> Avoid being hit. But your head is not >> Avoid being hit. But your head is not >> Avoid being hit. But your head is not doing a Beijian calculation of okay, doing a Beijian calculation of okay, doing a Beijian calculation of okay, this pen is coming. The probability that this pen is coming. The probability that this pen is coming. The probability that it hits me, it'll cause this much pain it hits me, it'll cause this much pain it hits me, it'll cause this much pain or all that. or all that. or all that. >> Correct. What you're essentially doing >> Correct. What you're essentially doing >> Correct. What you're essentially doing in your head is you're doing a in your head is you're doing a in your head is you're doing a simulation. simulation. simulation. >> You see the uh the the the pen coming >> You see the uh the the the pen coming >> You see the uh the the the pen coming and you know that it'll come and hit me. and you know that it'll come and hit me. and you know that it'll come and hit me. Your mind simulates and you dodge it. Your mind simulates and you dodge it. Your mind simulates and you dodge it. Right? So Right? So Right? So all of deep learning is uh doing all of deep learning is uh doing all of deep learning is uh doing correlations. It's not doing causation. It's not doing causation. >> Yeah. Causal models are the ones that >> Yeah. Causal models are the ones that >> Yeah. Causal models are the ones that are able to do simulations and are able to do simulations and are able to do simulations and interventions. So you know Judea has interventions. So you know Judea has interventions. So you know Judea has this whole uh causal hierarchy this whole uh causal hierarchy this whole uh causal hierarchy >> where the first hierarchy and the first >> where the first hierarchy and the first >> where the first hierarchy and the first hierarchy is association which is you hierarchy is association which is you hierarchy is association which is you build these correlation models. Deep build these correlation models. Deep build these correlation models. Deep learning is beautiful. It it's extremely learning is beautiful. It it's extremely learning is beautiful. It it's extremely powerful. I mean you see every day all powerful. I mean you see every day all powerful. I mean you see every day all these models are like amazingly good. these models are like amazingly good. these models are like amazingly good. >> They do association. The second is >> They do association. The second is >> They do association. The second is intervention intervention intervention in the hierarchy. Yeah, in the hierarchy. Yeah, in the hierarchy. Yeah, >> deep learning models do not do that. >> deep learning models do not do that. >> deep learning models do not do that. Third is counterfactual. Third is counterfactual. Third is counterfactual. So both intervention and counterfactual So both intervention and counterfactual So both intervention and counterfactual you can imagine it it it's some sort of you can imagine it it it's some sort of you can imagine it it it's some sort of simulation. simulation. simulation. You you build a model of causal model of You you build a model of causal model of You you build a model of causal model of what's happening and then you are able what's happening and then you are able what's happening and then you are able to simulate. So our brains do that. to simulate. So our brains do that. to simulate. So our brains do that. The current architectures don't do that. The current architectures don't do that. The current architectures don't do that. Another example I think which will make Another example I think which will make Another example I think which will make it clear is uh the difference between it clear is uh the difference between it clear is uh the difference between I'll use these technical term Shannon I'll use these technical term Shannon I'll use these technical term Shannon entropy entropy entropy >> and kmogrove complexity. >> and kmogrove complexity. >> and kmogrove complexity. >> Sure. >> Sure. >> Sure. >> So if you look at the Shannon entropy of >> So if you look at the Shannon entropy of >> So if you look at the Shannon entropy of the digits of pi the digits of pi the digits of pi >> it's infinite. >> it's infinite. >> it's infinite. >> Sure. >> Sure. >> Sure. >> It's impossible to predict and learn >> It's impossible to predict and learn >> It's impossible to predict and learn what digit will come after. Yeah. So what digit will come after. Yeah. So what digit will come after. Yeah. So that's the definition of Shannon entropy that's the definition of Shannon entropy that's the definition of Shannon entropy and Shannon entropy sort of tries to and Shannon entropy sort of tries to and Shannon entropy sort of tries to build a correlation. It tries to learn build a correlation. It tries to learn build a correlation. It tries to learn the correlation. Deep learning does the the correlation. Deep learning does the the correlation. Deep learning does the Shannon entropy. Shannon entropy. Shannon entropy. >> Gulmagraph complexity on the other hand >> Gulmagraph complexity on the other hand >> Gulmagraph complexity on the other hand is the is the length of the shortest is the is the length of the shortest is the is the length of the shortest program. program. program. >> Yeah. >> Yeah. >> Yeah. >> Which will reproduce >> Which will reproduce >> Which will reproduce uh the string that you that is under uh the string that you that is under uh the string that you that is under question. question. question. >> Yeah. >> Yeah. >> Yeah. >> Now the program to get the digits of pi >> Now the program to get the digits of pi >> Now the program to get the digits of pi are very small. are very small. are very small. >> Yeah. Thanks to Raman Jim and others you >> Yeah. Thanks to Raman Jim and others you >> Yeah. Thanks to Raman Jim and others you know there all sorts of really small know there all sorts of really small know there all sorts of really small program that can reproduce it exactly. program that can reproduce it exactly. program that can reproduce it exactly. So the colograph complexity of pi is So the colograph complexity of pi is So the colograph complexity of pi is very small. Shannon entropy is infinite. very small. Shannon entropy is infinite. very small. Shannon entropy is infinite. >> I think deep learning is still in the >> I think deep learning is still in the >> I think deep learning is still in the Shannon entropy world. It has not Shannon entropy world. It has not Shannon entropy world. It has not crossed over to the colog complexity and crossed over to the colog complexity and crossed over to the colog complexity and the causal world. the causal world. the causal world. >> Wow interesting. >> Wow interesting. >> Wow interesting. >> Right. So uh do you to what extent do >> Right. So uh do you to what extent do >> Right. So uh do you to what extent do you think this provides us research you think this provides us research you think this provides us research directions to kind of improve the state directions to kind of improve the state directions to kind of improve the state of the so let me just give you a of the so let me just give you a of the so let me just give you a specific example you talked about specific example you talked about specific example you talked about >> human beings don't actually update you >> human beings don't actually update you >> human beings don't actually update you know the matrix they don't kind of know the matrix they don't kind of know the matrix they don't kind of update their weights update their weights update their weights >> but right now there's a lot of research >> but right now there's a lot of research >> but right now there's a lot of research on continual learning you know so on continual learning you know so on continual learning you know so >> does your work provide some guidance of >> does your work provide some guidance of >> does your work provide some guidance of how you might approach those problems how you might approach those problems how you might approach those problems and and in particular I've always had and and in particular I've always had and and in particular I've always had this question which is we use so much this question which is we use so much this question which is we use so much data and so much compute. data and so much compute. data and so much compute. >> Yeah. >> Yeah. >> Yeah. >> To create these models like is it even >> To create these models like is it even >> To create these models like is it even reasonable to think that you can update reasonable to think that you can update reasonable to think that you can update the weights and actually have a the weights and actually have a the weights and actually have a meaningful impact you know with in in meaningful impact you know with in in meaningful impact you know with in in real time. I mean it just seems like you real time. I mean it just seems like you real time. I mean it just seems like you just need so much more data in order to just need so much more data in order to just need so much more data in order to do that. So can you start answering do that. So can you start answering do that. So can you start answering these questions? these questions? these questions? >> You you can start answering some of >> You you can start answering some of >> You you can start answering some of these questions and and one of the these questions and and one of the these questions and and one of the misconceptions that exists today is that misconceptions that exists today is that misconceptions that exists today is that scale will solve everything. Scale will scale will solve everything. Scale will scale will solve everything. Scale will not solve everything. you you you need a not solve everything. you you you need a not solve everything. you you you need a different kind of architecture and this different kind of architecture and this different kind of architecture and this continual learning is a difficult continual learning is a difficult continual learning is a difficult problem. You have to balance the fact problem. You have to balance the fact problem. You have to balance the fact that you will learn something new that you will learn something new that you will learn something new against the risk of catastrophic against the risk of catastrophic against the risk of catastrophic forgetting. forgetting. forgetting. >> Right. >> Right. >> Right. >> Right. >> Right. >> Right. >> Right. >> Right. >> Right. >> If you update the weights and you forget >> If you update the weights and you forget >> If you update the weights and you forget what what was important and what you what what was important and what you what what was important and what you have already learned then then you are have already learned then then you are have already learned then then you are you know you're not making progress. you know you're not making progress. you know you're not making progress. Then it'll just be some sort of random Then it'll just be some sort of random Then it'll just be some sort of random chaotic model. So to solve that problem chaotic model. So to solve that problem chaotic model. So to solve that problem is difficult. That's one aspect of it. is difficult. That's one aspect of it. is difficult. That's one aspect of it. So, so, so you know to get to what is So, so, so you know to get to what is So, so, so you know to get to what is called AGI, I think there are two things called AGI, I think there are two things called AGI, I think there are two things that need to happen. One is this that need to happen. One is this that need to happen. One is this plasticity plasticity plasticity which has to be implemented through which has to be implemented through which has to be implemented through container learning. container learning. container learning. >> Secondly, we have to move from >> Secondly, we have to move from >> Secondly, we have to move from correlation to causation. correlation to causation. correlation to causation. >> Yeah, >> Yeah, >> Yeah, >> that's uh uh I how much is this similar >> that's uh uh I how much is this similar >> that's uh uh I how much is this similar to what Yan Lun talks about with the to what Yan Lun talks about with the to what Yan Lun talks about with the >> so Yan Lun causality planning? >> so Yan Lun causality planning? >> so Yan Lun causality planning? >> Yeah. you know predicting like how your >> Yeah. you know predicting like how your >> Yeah. you know predicting like how your action would action would action would >> it is it is related you know he he's >> it is it is related you know he he's >> it is it is related you know he he's coming at it from a different angle than coming at it from a different angle than coming at it from a different angle than the jp model right the jp model right the jp model right >> but it is related the the other thing is >> but it is related the the other thing is >> but it is related the the other thing is uh you know the first time I came on uh you know the first time I came on uh you know the first time I came on this podcast I I mentioned this test of this podcast I I mentioned this test of this podcast I I mentioned this test of AGI AGI AGI >> the Einstein test >> the Einstein test >> the Einstein test >> I don't remember >> I don't remember >> I don't remember >> so I said you know uh uh you take an LLM >> so I said you know uh uh you take an LLM >> so I said you know uh uh you take an LLM >> and train it on pre 1916 or 1911 physics >> and train it on pre 1916 or 1911 physics >> and train it on pre 1916 or 1911 physics >> and see if it can come up with the >> and see if it can come up with the >> and see if it can come up with the theory of relativity. theory of relativity. theory of relativity. >> Yeah, >> Yeah, >> Yeah, >> if it does then we have AGI. I mean it's >> if it does then we have AGI. I mean it's >> if it does then we have AGI. I mean it's a high bar but you know we should have a high bar but you know we should have a high bar but you know we should have high bars. It won't. And this is the high bars. It won't. And this is the high bars. It won't. And this is the same test that I think Demis uh same test that I think Demis uh same test that I think Demis uh mentioned at uh the India AI summit mentioned at uh the India AI summit mentioned at uh the India AI summit couple of weeks ago. It's created a lot couple of weeks ago. It's created a lot couple of weeks ago. It's created a lot of news. But why why is that and how is of news. But why why is that and how is of news. But why why is that and how is that related to this idea of Shannon that related to this idea of Shannon that related to this idea of Shannon versus Kro? versus Kro? versus Kro? So at the time of Einstein there were a So at the time of Einstein there were a So at the time of Einstein there were a lot of clues lot of clues lot of clues that Newtonian mechanics there was that Newtonian mechanics there was that Newtonian mechanics there was something missing. something missing. something missing. >> Yeah. >> Yeah. >> Yeah. >> Right. Uh people knew that Mercury's >> Right. Uh people knew that Mercury's >> Right. Uh people knew that Mercury's orbit didn't make sense. There was orbit didn't make sense. There was orbit didn't make sense. There was something off about it. Then there were something off about it. Then there were something off about it. Then there were these experiments done uh the these experiments done uh the these experiments done uh the Michaelelsson Mley experiments where Michaelelsson Mley experiments where Michaelelsson Mley experiments where they were trying to figure out uh they were trying to figure out uh they were trying to figure out uh uh this uh medium called uh the ether uh this uh medium called uh the ether uh this uh medium called uh the ether through which light travels. through which light travels. through which light travels. And they felt that if you know you And they felt that if you know you And they felt that if you know you bounce light in different directions bounce light in different directions bounce light in different directions uh the speed might change and they they uh the speed might change and they they uh the speed might change and they they could detect a change in the speed of could detect a change in the speed of could detect a change in the speed of light. light. light. They tried several experiments. They had They tried several experiments. They had They tried several experiments. They had really precise instruments which could really precise instruments which could really precise instruments which could measure the speed and they found measure the speed and they found measure the speed and they found nothing. They found that that speed of nothing. They found that that speed of nothing. They found that that speed of light did not change at all. light did not change at all. light did not change at all. Then there were there's a whole issue of Then there were there's a whole issue of Then there were there's a whole issue of black holes. black holes. black holes. >> Yeah. >> Yeah. >> Yeah. >> Then gravitational lensing. So there >> Then gravitational lensing. So there >> Then gravitational lensing. So there were a lot of these signs that Newtonian were a lot of these signs that Newtonian were a lot of these signs that Newtonian mechanics mechanics mechanics is not really explaining everything. is not really explaining everything. is not really explaining everything. >> Yeah. >> Yeah. >> Yeah. But until Einstein came up with a new But until Einstein came up with a new But until Einstein came up with a new representation of the space-time representation of the space-time representation of the space-time container, container, container, >> right, >> right, >> right, >> we were stuck. >> we were stuck. >> we were stuck. >> So if you had a model that just looked >> So if you had a model that just looked >> So if you had a model that just looked at correlations and so uh sees all of at correlations and so uh sees all of at correlations and so uh sees all of this, you know, this, you know, this, you know, all of these uh pieces of individual all of these uh pieces of individual all of these uh pieces of individual evidence and put together, it would not evidence and put together, it would not evidence and put together, it would not have come up with the beautiful equation have come up with the beautiful equation have come up with the beautiful equation that Einstein came up with. you know uh that Einstein came up with. you know uh that Einstein came up with. you know uh I'm forgetting exactly what it is g muv= I'm forgetting exactly what it is g muv= I'm forgetting exactly what it is g muv= 8 pi t muv some something like that 8 pi t muv some something like that 8 pi t muv some something like that >> where you know uh the the the equation >> where you know uh the the the equation >> where you know uh the the the equation of uh the rel the space-time continum of uh the rel the space-time continum of uh the rel the space-time continum that the tensor that the tensor that the tensor >> so he came up with a new formulation >> so he came up with a new formulation >> so he came up with a new formulation >> so he kind of rejected the existing >> so he kind of rejected the existing >> so he kind of rejected the existing axioms he came up with a very short axioms he came up with a very short axioms he came up with a very short colograph representation of colograph representation of colograph representation of >> interesting >> interesting >> interesting >> the world >> the world >> the world >> one equation from that equation >> one equation from that equation >> one equation from that equation everything else follows everything else follows everything else follows >> right whether you're talking about >> right whether you're talking about >> right whether you're talking about gravitational waves or black holes or gravitational waves or black holes or gravitational waves or black holes or mercury or how GPS works you know GPS mercury or how GPS works you know GPS mercury or how GPS works you know GPS the GPS that we use every day in our the GPS that we use every day in our the GPS that we use every day in our phones it uses the equation of phones it uses the equation of phones it uses the equation of relativity so do does this end up relativity so do does this end up relativity so do does this end up becoming like becoming like becoming like um um um you you you almost have to ignore the you you you almost have to ignore the you you you almost have to ignore the majority of previous data in order to do majority of previous data in order to do majority of previous data in order to do it which LLM can't because they're it which LLM can't because they're it which LLM can't because they're trained on the majority of previous trained on the majority of previous trained on the majority of previous data. It's like you almost have like data. It's like you almost have like data. It's like you almost have like this kind of data gravity that's pulling this kind of data gravity that's pulling this kind of data gravity that's pulling you back. It's like it's like everybody you back. It's like it's like everybody you back. It's like it's like everybody said it's X. said it's X. said it's X. >> There's a little bit of evidence that >> There's a little bit of evidence that >> There's a little bit of evidence that it's Y, but because everybody said it's it's Y, but because everybody said it's it's Y, but because everybody said it's X, like the LM will always say it's X. X, like the LM will always say it's X. X, like the LM will always say it's X. >> It'll always say it'll treat that Y as >> It'll always say it'll treat that Y as >> It'll always say it'll treat that Y as an anomaly. Actually this is actually a an anomaly. Actually this is actually a an anomaly. Actually this is actually a very nice way to say it which is like very nice way to say it which is like very nice way to say it which is like >> it's like >> it's like >> it's like >> I so now okay now I get your Shannon >> I so now okay now I get your Shannon >> I so now okay now I get your Shannon entropy versus like one of them is like entropy versus like one of them is like entropy versus like one of them is like >> the total amount of information there >> the total amount of information there >> the total amount of information there that will always be bound to the total that will always be bound to the total that will always be bound to the total amount of information there which is amount of information there which is amount of information there which is what happens right now. what happens right now. what happens right now. >> Yeah. where you can actually describe >> Yeah. where you can actually describe >> Yeah. where you can actually describe another another motion. You can describe another another motion. You can describe another another motion. You can describe everything with a shorter description everything with a shorter description everything with a shorter description with the new data, which would be a with the new data, which would be a with the new data, which would be a totally different motion, which would be totally different motion, which would be totally different motion, which would be like like like >> you need a new representation, right? >> you need a new representation, right? >> you need a new representation, right? Yeah. You know, another way that I've Yeah. You know, another way that I've Yeah. You know, another way that I've always thought about these, I thought always thought about these, I thought always thought about these, I thought you articulated it well in the last time you articulated it well in the last time you articulated it well in the last time we talked about it, which is the we talked about it, which is the we talked about it, which is the universe is this very, very complex universe is this very, very complex universe is this very, very complex space and then, you know, somehow humans space and then, you know, somehow humans space and then, you know, somehow humans map it into a manifold. map it into a manifold. map it into a manifold. >> Mhm. >> Mhm. >> Mhm. >> That's less complex. >> That's less complex. >> That's less complex. >> Yeah. >> Yeah. >> Yeah. >> And then that gets kind of written down >> And then that gets kind of written down >> And then that gets kind of written down and then the LLM. So that's kind of some and then the LLM. So that's kind of some and then the LLM. So that's kind of some some distribution, some you know, it's some distribution, some you know, it's some distribution, some you know, it's still a very large space, but it's it's still a very large space, but it's it's still a very large space, but it's it's a bounded space. And the LM learn that a bounded space. And the LM learn that a bounded space. And the LM learn that manifold and then they kind of use, manifold and then they kind of use, manifold and then they kind of use, you know, Beijian inference to move up you know, Beijian inference to move up you know, Beijian inference to move up and down that manifold, but they're kind and down that manifold, but they're kind and down that manifold, but they're kind of bound to that manifold. of bound to that manifold. of bound to that manifold. >> Yeah. >> Yeah. >> Yeah. >> And then again, I don't want to put >> And then again, I don't want to put >> And then again, I don't want to put words in your mouth. And then, but like words in your mouth. And then, but like words in your mouth. And then, but like what they can't do is is generate a new what they can't do is is generate a new what they can't do is is generate a new manifold, right? Which requires manifold, right? Which requires manifold, right? Which requires understanding the way that the universe understanding the way that the universe understanding the way that the universe works and then coming up with a new works and then coming up with a new works and then coming up with a new representation of the universe. representation of the universe. representation of the universe. >> And this is what relativity is, right? >> And this is what relativity is, right? >> And this is what relativity is, right? >> Yeah. Exactly. >> Yeah. Exactly. >> Yeah. Exactly. >> Einstein had to create a new manifold. >> Einstein had to create a new manifold. >> Einstein had to create a new manifold. >> Yeah. If you just stuck with the old >> Yeah. If you just stuck with the old >> Yeah. If you just stuck with the old manifold of the Newtonian physics, manifold of the Newtonian physics, manifold of the Newtonian physics, >> then you would see these correlations >> then you would see these correlations >> then you would see these correlations but you could not come up with a but you could not come up with a but you could not come up with a manifold that explained them. So you manifold that explained them. So you manifold that explained them. So you need to come up with a new need to come up with a new need to come up with a new representation. So to me you know there representation. So to me you know there representation. So to me you know there are lots of definitions of AGI uh you are lots of definitions of AGI uh you are lots of definitions of AGI uh you know Turing test we have already passed know Turing test we have already passed know Turing test we have already passed that you know performing economically that you know performing economically that you know performing economically useful work every day you see you know useful work every day you see you know useful work every day you see you know LLMs are doing that. LLMs are doing that. LLMs are doing that. >> Do we I don't know. No, I mean they are >> Do we I don't know. No, I mean they are >> Do we I don't know. No, I mean they are >> I mean I mean without human >> I mean I mean without human >> I mean I mean without human intervention. intervention. intervention. >> No no no. So that that's different but >> No no no. So that that's different but >> No no no. So that that's different but still you know it's like a car can run still you know it's like a car can run still you know it's like a car can run faster than humans, right? faster than humans, right? faster than humans, right? >> I mean that's a that's the that's a >> I mean that's a that's the that's a >> I mean that's a that's the that's a that's a very shallow definition. that's a very shallow definition. that's a very shallow definition. >> Yeah. So all these definitions do useful >> Yeah. So all these definitions do useful >> Yeah. So all these definitions do useful >> you know maybe you know in 6 months >> you know maybe you know in 6 months >> you know maybe you know in 6 months you'll have cloud or what a gemini do you'll have cloud or what a gemini do you'll have cloud or what a gemini do without intervention cing tasks which without intervention cing tasks which without intervention cing tasks which are well defined well scoped are well defined well scoped are well defined well scoped >> that's possible but to me AGI will >> that's possible but to me AGI will >> that's possible but to me AGI will happen when these two problems get happen when these two problems get happen when these two problems get solved solved solved >> elasticity continual learning properly >> elasticity continual learning properly >> elasticity continual learning properly and building a causal model from you and building a causal model from you and building a causal model from you know know know uh in a more data efficient manner Yeah, uh in a more data efficient manner Yeah, uh in a more data efficient manner Yeah, we we are hearing people now talking we we are hearing people now talking we we are hearing people now talking about you know seeing generality like about you know seeing generality like about you know seeing generality like Donald Kuth for example in the last few Donald Kuth for example in the last few Donald Kuth for example in the last few days right you know had this you know days right you know had this you know days right you know had this you know this you know aha moment apparently that this you know aha moment apparently that this you know aha moment apparently that kind of made went viral on X so do you kind of made went viral on X so do you kind of made went viral on X so do you think that that suggests that we're think that that suggests that we're think that that suggests that we're seeing generality or seeing generality or seeing generality or >> No no no so so that actually >> No no no so so that actually >> No no no so so that actually to me it validates what I've been to me it validates what I've been to me it validates what I've been talking about for a while now how talking about for a while now how talking about for a while now how >> so so if if you read what he did uh with >> so so if if you read what he did uh with >> so so if if you read what he did uh with the help of uh you know a colleague he the help of uh you know a colleague he the help of uh you know a colleague he got the LLMs to solve this particular got the LLMs to solve this particular got the LLMs to solve this particular problem of finding Hamiltonian cycles problem of finding Hamiltonian cycles problem of finding Hamiltonian cycles odd numbers we won't get into that and odd numbers we won't get into that and odd numbers we won't get into that and he got the LLMs to keep solving for one he got the LLMs to keep solving for one he got the LLMs to keep solving for one odd number after the other right odd number after the other right odd number after the other right >> what he also got to do is after it found >> what he also got to do is after it found >> what he also got to do is after it found a solution for a particular value of m a solution for a particular value of m a solution for a particular value of m he made the LLM update its memory he made the LLM update its memory he made the LLM update its memory with exactly what it learned in solving with exactly what it learned in solving with exactly what it learned in solving that problem. So the LLM's tried many that problem. So the LLM's tried many that problem. So the LLM's tried many different things. Yeah. different things. Yeah. different things. Yeah. >> You know, something worked, update the >> You know, something worked, update the >> You know, something worked, update the the memory. So that's kind of like the memory. So that's kind of like the memory. So that's kind of like hacking together plasticity. hacking together plasticity. hacking together plasticity. >> Yeah. >> Yeah. >> Yeah. >> Right. It's learning what it has done as >> Right. It's learning what it has done as >> Right. It's learning what it has done as we went along. Again, it's it's a hacked we went along. Again, it's it's a hacked we went along. Again, it's it's a hacked version of it. You're not changing the version of it. You're not changing the version of it. You're not changing the weights. You're just sort of improving weights. You're just sort of improving weights. You're just sort of improving the context. the context. the context. >> Right. >> Right. >> Right. >> Right. But you as you learned and even >> Right. But you as you learned and even >> Right. But you as you learned and even after that so this whole space of after that so this whole space of after that so this whole space of Hamiltonian cycles and the associated Hamiltonian cycles and the associated Hamiltonian cycles and the associated math is well represented in the math is well represented in the math is well represented in the manifolds that these LLMs have been manifolds that these LLMs have been manifolds that these LLMs have been trained on trained on trained on >> right you just had to find the right >> right you just had to find the right >> right you just had to find the right connection and LLMs I know compute you connection and LLMs I know compute you connection and LLMs I know compute you throw enough compute they will find the throw enough compute they will find the throw enough compute they will find the right connection so can was able to find right connection so can was able to find right connection so can was able to find the LLM's the LLM's the LLM's attempts And eventually it needed him attempts And eventually it needed him attempts And eventually it needed him to put together what he saw into a to put together what he saw into a to put together what he saw into a solution. It definitely helped him get solution. It definitely helped him get solution. It definitely helped him get to the solution but he had to create the to the solution but he had to create the to the solution but he had to create the new sort of manifold new sort of manifold new sort of manifold >> to come to the solution. The LLMs were >> to come to the solution. The LLMs were >> to come to the solution. The LLMs were after a while stuck right he you read after a while stuck right he you read after a while stuck right he you read what he has written. I mean it just hot what he has written. I mean it just hot what he has written. I mean it just hot up the press I think two days ago. up the press I think two days ago. up the press I think two days ago. >> Two days ago days ago but uh eventually >> Two days ago days ago but uh eventually >> Two days ago days ago but uh eventually he used the solutions and he came up he used the solutions and he came up he used the solutions and he came up with uh the proof. with uh the proof. with uh the proof. >> Yeah. >> Yeah. >> Yeah. >> Right. So it's like you know it's like >> Right. So it's like you know it's like >> Right. So it's like you know it's like Einstein saw all these evidences Einstein saw all these evidences Einstein saw all these evidences then he thought what will explain then he thought what will explain then he thought what will explain he came up with a causal model. he came up with a causal model. he came up with a causal model. >> Yeah. So canut and his brain is sort of >> Yeah. So canut and his brain is sort of >> Yeah. So canut and his brain is sort of the the the >> that's in the chimograph is the human >> that's in the chimograph is the human >> that's in the chimograph is the human right and the llms are extremely right and the llms are extremely right and the llms are extremely efficient at doing the shannon part of efficient at doing the shannon part of efficient at doing the shannon part of it. It found all the solutions by trying it. It found all the solutions by trying it. It found all the solutions by trying you know various things and learning you know various things and learning you know various things and learning more and more more and more more and more >> clever way to decompose it. I'm >> clever way to decompose it. I'm >> clever way to decompose it. I'm wondering like do you think this again wondering like do you think this again wondering like do you think this again I'm going to ask the same question again I'm going to ask the same question again I'm going to ask the same question again which is do you think this provides some which is do you think this provides some which is do you think this provides some sort of insight on like the next problem sort of insight on like the next problem sort of insight on like the next problem to tackle like yeah to tackle like yeah to tackle like yeah >> like like is there a mechanism that will >> like like is there a mechanism that will >> like like is there a mechanism that will get the kagarov complexity get the kagarov complexity get the kagarov complexity >> or not like is this it tells us which >> or not like is this it tells us which >> or not like is this it tells us which direction direction direction >> but clearly not how to do it like >> but clearly not how to do it like >> but clearly not how to do it like >> not how to do but even colograph >> not how to do but even colograph >> not how to do but even colograph complexity has largely remained a sort complexity has largely remained a sort complexity has largely remained a sort of a theoretical construct of a theoretical construct of a theoretical construct >> yeah for sure there's no algorithm >> yeah for sure there's no algorithm >> yeah for sure there's no algorithm There's no there haven't been practical There's no there haven't been practical There's no there haven't been practical implementations of finding implementations of finding implementations of finding >> the shortest program. >> the shortest program. >> the shortest program. >> We know it exists. You know, you can >> We know it exists. You know, you can >> We know it exists. You know, you can argue about it. It but so so that's argue about it. It but so so that's argue about it. It but so so that's where I think where I think where I think >> it's my bias. That's where our energy >> it's my bias. That's where our energy >> it's my bias. That's where our energy >> should be focused not larger models with >> should be focused not larger models with >> should be focused not larger models with more tokens. more tokens. more tokens. >> Can you and can you can you tie the two >> Can you and can you can you tie the two >> Can you and can you can you tie the two things like how does that pair with things like how does that pair with things like how does that pair with doing simulation or is that simulation doing simulation or is that simulation doing simulation or is that simulation totally orthogonal? totally orthogonal? totally orthogonal? >> No, simulation is is related, right? So >> No, simulation is is related, right? So >> No, simulation is is related, right? So you think it like basically you do you think it like basically you do you think it like basically you do simulation and somehow that is a step simulation and somehow that is a step simulation and somehow that is a step towards doing the kagra complexity. towards doing the kagra complexity. towards doing the kagra complexity. >> It it's it's the simulator is the is the >> It it's it's the simulator is the is the >> It it's it's the simulator is the is the program that we create. It may not be program that we create. It may not be program that we create. It may not be the perfect program. the perfect program. the perfect program. >> Oh I see. >> Oh I see. >> Oh I see. >> But in our heads we create this uh >> But in our heads we create this uh >> But in our heads we create this uh simulator that when I'm throwing the pen simulator that when I'm throwing the pen simulator that when I'm throwing the pen you know that it's coming at you right you know that it's coming at you right you know that it's coming at you right and you duck. So, so you're not and you duck. So, so you're not and you duck. So, so you're not computing the probabilities as it goes, computing the probabilities as it goes, computing the probabilities as it goes, but but you have, you know, you build a but but you have, you know, you build a but but you have, you know, you build a very physical thing versus we were very physical thing versus we were very physical thing versus we were talking more conceptually. talking more conceptually. talking more conceptually. >> Conceptually, but but it's the same >> Conceptually, but but it's the same >> Conceptually, but but it's the same because of the same mechanism. because of the same mechanism. because of the same mechanism. >> It's the same mechanism really. >> It's the same mechanism really. >> It's the same mechanism really. >> Yeah. You have to build a causal model. >> Yeah. You have to build a causal model. >> Yeah. You have to build a causal model. >> Yeah. >> Yeah. >> Yeah. >> Right. >> Right. >> Right. >> I see. For most things, right? >> I see. For most things, right? >> I see. For most things, right? >> So, you have to move from correlation to >> So, you have to move from correlation to >> So, you have to move from correlation to causation. I mean, we've heard this causation. I mean, we've heard this causation. I mean, we've heard this term. term. term. >> Yeah. you know >> Yeah. you know >> Yeah. you know add infinitum but here it it's making a add infinitum but here it it's making a add infinitum but here it it's making a difference in the way we view difference in the way we view difference in the way we view intelligence intelligence intelligence >> how how how has the last three papers >> how how how has the last three papers >> how how how has the last three papers been received been received been received >> no I don't know there well I mean I mean >> no I don't know there well I mean I mean >> no I don't know there well I mean I mean the archive the archive the archive >> versions will let me tell you it I mean >> versions will let me tell you it I mean >> versions will let me tell you it I mean >> um lot of great reception a lot of >> um lot of great reception a lot of >> um lot of great reception a lot of people read it I'm just wondering like people read it I'm just wondering like people read it I'm just wondering like what kind of feedback that you've got what kind of feedback that you've got what kind of feedback that you've got >> I'm getting good feedback but I'm an >> I'm getting good feedback but I'm an >> I'm getting good feedback but I'm an outsider in this field right that's outsider in this field right that's outsider in this field right that's right like networking guy. right like networking guy. right like networking guy. >> I'm a networking guy. Why is he writing >> I'm a networking guy. Why is he writing >> I'm a networking guy. Why is he writing about you know learning and machine about you know learning and machine about you know learning and machine learning and deep learning and basian so learning and deep learning and basian so learning and deep learning and basian so but but people who have actually taken but but people who have actually taken but but people who have actually taken the time to read those papers I'm the time to read those papers I'm the time to read those papers I'm getting really good feedback uh there getting really good feedback uh there getting really good feedback uh there was a recent paper by Google was a recent paper by Google was a recent paper by Google research research research >> which tried to teach uh LLM by some sort >> which tried to teach uh LLM by some sort >> which tried to teach uh LLM by some sort of RLF to do Beijian learning properly. of RLF to do Beijian learning properly. of RLF to do Beijian learning properly. >> Yeah. >> Yeah. >> Yeah. >> And that's going in this direction. And >> And that's going in this direction. And >> And that's going in this direction. And I think people are coming around to the I think people are coming around to the I think people are coming around to the view that okay LLMs are doing Beijian view that okay LLMs are doing Beijian view that okay LLMs are doing Beijian learning. I know that some people also learning. I know that some people also learning. I know that some people also looked at the Beijian vent tunnel paper looked at the Beijian vent tunnel paper looked at the Beijian vent tunnel paper the archive version and they reproduced the archive version and they reproduced the archive version and they reproduced the experiments. the experiments. the experiments. >> That's great. >> That's great. >> That's great. >> Did they just saw what was written and >> Did they just saw what was written and >> Did they just saw what was written and they they did the trading and they saw they they did the trading and they saw they they did the trading and they saw yeah this is actually happening. So yeah this is actually happening. So yeah this is actually happening. So >> that's great. So what's next? >> that's great. So what's next? >> that's great. So what's next? uh what's next is uh you know these two uh what's next is uh you know these two uh what's next is uh you know these two parallel uh tracks I hope to make parallel uh tracks I hope to make parallel uh tracks I hope to make progress there plasticity and causal progress there plasticity and causal progress there plasticity and causal >> because today you've taken an existing >> because today you've taken an existing >> because today you've taken an existing mechanism mechanism mechanism >> and you've created a formal model how it >> and you've created a formal model how it >> and you've created a formal model how it works works works >> and so now you're actually interested in >> and so now you're actually interested in >> and so now you're actually interested in improving in creating a new mechanism improving in creating a new mechanism improving in creating a new mechanism >> and do you think it's an entirely >> and do you think it's an entirely >> and do you think it's an entirely different architecture different architecture different architecture >> I or do you think do you think LLMs are >> I or do you think do you think LLMs are >> I or do you think do you think LLMs are like part of the solution like part of the solution like part of the solution >> I think LLMs are definitely part of the >> I think LLMs are definitely part of the >> I think LLMs are definitely part of the solution. solution. solution. >> I see. >> I see. >> I see. >> But but there has to be something more >> But but there has to be something more >> But but there has to be something more >> and other. >> and other. >> and other. >> So you know I was not interested in sort >> So you know I was not interested in sort >> So you know I was not interested in sort of cataloging what all these of cataloging what all these of cataloging what all these >> LLMs can do. >> LLMs can do. >> LLMs can do. >> Was more interested in why are they and >> Was more interested in why are they and >> Was more interested in why are they and how are they doing it. how are they doing it. how are they doing it. >> I think now we have a good grip on >> I think now we have a good grip on >> I think now we have a good grip on >> the why and how. >> the why and how. >> the why and how. >> And the next step is to you know >> And the next step is to you know >> And the next step is to you know >> move them to the next level. We we now I >> move them to the next level. We we now I >> move them to the next level. We we now I I think we have a fairly good I think we have a fairly good I think we have a fairly good understanding of what the limits are. understanding of what the limits are. understanding of what the limits are. >> Yeah. >> Yeah. >> Yeah. Now how do you uh go to the next step? Now how do you uh go to the next step? Now how do you uh go to the next step? >> Is there an is there an equivalent kind >> Is there an is there an equivalent kind >> Is there an is there an equivalent kind of theoretical of theoretical of theoretical framework for causality that applies framework for causality that applies framework for causality that applies here like similar to like Beijian for here like similar to like Beijian for here like similar to like Beijian for inference? inference? inference? >> Well the Japal's whole causal hierarchy >> Well the Japal's whole causal hierarchy >> Well the Japal's whole causal hierarchy I think I think I think >> I think that's the right one. >> I think that's the right one. >> I think that's the right one. >> That's that's a very good one. You know >> That's that's a very good one. You know >> That's that's a very good one. You know the whole do calculus uh approach the whole do calculus uh approach the whole do calculus uh approach I think it's a good way to think about I think it's a good way to think about I think it's a good way to think about it. you know the the sort of association it. you know the the sort of association it. you know the the sort of association intervention counterfactuals. intervention counterfactuals. intervention counterfactuals. >> Yeah. >> Yeah. >> Yeah. >> It takes you from correlation to >> It takes you from correlation to >> It takes you from correlation to actually simulation. actually simulation. actually simulation. >> Yeah. >> Yeah. >> Yeah. >> In a mathematical way. >> In a mathematical way. >> In a mathematical way. >> That's great. >> That's great. >> That's great. >> All right. Well, listen, really >> All right. Well, listen, really >> All right. Well, listen, really appreciate you coming. This is awesome. appreciate you coming. This is awesome. appreciate you coming. This is awesome. So, we had you here for the first paper So, we had you here for the first paper So, we had you here for the first paper where you had the empirical results. where you had the empirical results. where you had the empirical results. >> Mhm. And then we had you back when you >> Mhm. And then we had you back when you >> Mhm. And then we had you back when you actually have like the formal proof and actually have like the formal proof and actually have like the formal proof and hopefully the next time you come back hopefully the next time you come back hopefully the next time you come back you will have a proposal for the you will have a proposal for the you will have a proposal for the mechanism that uh mechanism that uh mechanism that uh >> that actually provides the next step. >> that actually provides the next step. >> that actually provides the next step. >> Hopefully. >> Hopefully. >> Hopefully. >> All right. Cool. Thank you for coming >> All right. Cool. Thank you for coming >> All right. Cool. Thank you for coming in. in. in. >> Thank you for having me.