Recursive's $670M Bet on Self-Improving AI, Sonnet 5.5 Hits 70%, Elon Co-Leads Pentagon Push EP 299

Introduction and Guest Welcome

00:00Richard, I'm I'm curious. Where are we at the moment with RSI >> in various weak forms? We already have RSI. We're not quite there yet, but we're very close. >> When do you believe we reach ASI? >> I think we'll take us probably several decades. >> On Terminal Bench 4.0, Sonnet 5.5 jumped from 10% to 70%. Pretty extraordinary. My theory would be that they're trying to compete with, you know, China, which is about three months behind, and fill that gap before a lot of enterprises go to to open source models. >> It's not at all obvious to me why anyone should be using Sonnet 5.5 over Opus 5.5 unless you have some token or latency or other consideration. I do not plan to use Sonnet 5.5. >> The Defense Secretary announced Project

00:47Meridian. It's a new Pentagon effort on the future of warfare. It's co-led by Elon Musk and Palmer Lucky. >> This I think is a transformative moment. This is a major major step forward. The big question is going to be. >> Now that's a moonshot. Ladies and gentlemen, >> this episode is brought to you by the Abundance Summit and Link Ventures. >> Welcome to Moonshots, everyone. Your number one podcast and help you understand the singularity. What's going on? What does it mean to you and your family and what the breaking news is? We publish this twice a week to help you keep up with the extraordinary rate of change. With me today is the Fantastic Four, Alex Weer Gross, our in-house ASI, Dave Blendon, our improario of AI

01:37investing, Salem Ismael, our Globe Troder who's home today. Amazing Sem. >> Woohoo. >> And the and the father of the organizational singularity. I'm Peter Diamandis, your host and datadriven optimist. Our mission here, keep you optimistic about the future. Uh, and here's the numbers from this week. In 7 days, Google shipped Gemini for Argon. Anthropic shipped Sonnet 5.5. Open eye shipped GPT 6.1. Saul and the Frontier Intelligence has gotten cheaper by three-fold. SpaceX launched the next batch of astronauts, the ISS, and also launched Google's TPUs into orbit for Project Suncatcher. Uh, if you're new to the Moonshots podcast, please hit subscribe. We publish twice a week and you don't want to miss any of

02:23this news during this hypersonic tsunami. Our moonshot here is to 20x our subscriber base to get to 10 million to help spread the word of optimism and the extraordinary future that we're building. Today on Moonshot, we're joined by one of the architects of modern AI, Richard. Uh, born in Germany and trained at Stanford. Richard is among the world's most cited natural language processing researchers, a pioneer in deep learning and prompt engineering and a serial founder who's repeatedly turned frontier research into category defining companies. First, he founded Metammind uh which was acquired by Salesforce where he then became the chief scientist leading AI research for friend of the pod Mark Beni off. Next,

03:07Richard founded you.com, recognized by Time in the World Economic Forum and now valued at north of 1.5 billion as well as his fund AO aix ventures. Today, Richard is co-founder and CEO of Recursive, perhaps uh the biggest moonshot he's ever taken, uh focused on recursively self-improving super intelligence. Richard, congrats on a $670 million fund raise from Google Ventures. Graycraft, Nvidia, and AMD and 410 million in compute from AWS. Personal disclosure, I'm very proud to be a seed investor in Recursive. And of course, Richard, uh, important to mention your new book just got released, The Eureka Machine: Why AI is the key to unlocking a new era of scientific discoveries. Welcome to the pod,

The Eureka Machine and Full Stack AI for Science

03:58Richard. >> So great to be back. >> And let me just add, a truly truly awesome guy. A lot of people who listen to the pot are worried about the the ethics and the the risks and everything, but boy, if you want a a person to conquer recursive self-improvement that you can like and trust, Richard is the man. >> Yeah. >> Thank you so much. So great to be back here. I love your guys. Constructive optimism in the world right now. >> I have I have a bold prediction for this episode. >> What's that? >> This one episode will prove our thesis for this whole podcast more than any other episode we ever record. >> Awesome. Let's go. >> Let's go. >> And uh uh Alex, do you know Richard? >> Yeah, Richard, you and I have chatted

04:39quite a bit. I don't think we've actually ever met in person, though. >> Really? >> We have not. And it's the first time on a podcast together. It's going to be awesome. >> Amazing. Well, let's make history. >> Yeah, for sure. So, uh Richard, let's start with your book. Uh the central claim of the Eureka machine is that AI will deliver a century of scientific breakthroughs in the next decade. uh you know this maps directly onto what Alex and I wrote in solve everything. So we're fans of that prediction. Uh your thesis in the book is that every stage of scientific process gets connected and transformed simultaneously. Hypothesis, experiment, data theory, you call it the full stack AI. Uh you also say that

05:21scientific progress has slowed and that's been caused not by underfunding but by fragmentation. So let's start with those two items. Could you take a moment and you know talk to us about full stack AI for science and why you think progress has slowed? >> Yeah. So progress has slowed uh largely because we have so many different subd disccipline and uh disciplines and even sus of lem uh realized like wow that like if there are only so many people and we have more and more fragmentation of more and more subd disciplines and niches like you can not just do AI right you're doing often like optimization uh and gradient descent methods and second order derivative methods uh and so on of uh these neural networks that is which

06:08is one category of AI and so on. And the same is true in biology. You study biology, you're going to be either a cell biologist or a molecular biologist uh or you're like in medicine and like these like there's so much separation that it's hard to weave it all together. And that is the perfect time for AI to come in and help us weave back together all these separate pieces and also help us understand how these large complex systems actually work. like we can't have one model where humans say here are all the things I know about natural language and now like we can put all these rules together and then we have a conversation. It required a large neural network with a ton of data. And guess what? We're going to get a lot of data

06:50uh about biology. And so we can make more and more disciplines and move them transfer them from being traditionally natural sciences where we just try to understand biology and nature to being programmable engineering sciences. And that I think is one of the many different exciting aspects of things to come. And you know why why am I so excited that that this is happening right now and and what are the major ingredients? It's essentially that we now have world knowledge in the form of LMS. We have more and more scientific data that's getting digitized that we can sit on top of. We have better and better simulations of things and we have robotic process automation that will soon be possible. And on top of that

07:30we're going to have an agent swarm and that is how uh this the full scientific stack uh can be automated. Alex. >> Yeah, I can say I disagree. As Peter, you and I wrote in solve everything, math, science, and engineering are cooked. I'm curious, Richard, what your latest timelines are. >> I mean, the I think every field is going to get to higher and higher levels of abstraction. Computer science has been very good at that, right? No one is programming in zeros and ones anymore. No one like very few people have to still know C++ and complex pointers and and memory uh architectures and so on. we can get a larger and larger and computer science has basically abstracted enough so that it met the rest of humanity with English, right?

08:14And natural language that now can be used to do computer science. And so I think that um gives me great hope uh that we can get other fields into similar uh stages of abstraction and then all meet and natural language to do science. But to pin you down, as I recall from my understanding of your book, 3 to five years for the physical sciences, >> you know, like there's no like single sort of this is the threshold and now we solved all of physical sciences, right? But like multiple different diseases will get cured in the next 12 months. They're going to be the simpler diseases maybe where there's one gene that needs to be fixed for that disease to be cured. Um but there are a lot of single gene diseases uh that that are out there

08:58in aggregate, right? like and so those diseases will get cured and then we're going to cure more and more complex diseases. We're going to develop better and better battery material. So I wouldn't call it like one threshold in like in 3 years sort of everything is solved but like we're going to solve more and more problems and it'll just be a question of how much money do you want to put into compute to then solve which kinds of problems? >> You're you're talking to the guy Richard remember who argues that the singularity itself is something of an optical illusion that there is no step function. It's just a time interval. So I'll I'll try once more timelines. When when even if it's not a step function, when is the

09:34inflection point when is the um 50% point crossed in your mind for all of the physical sciences getting solved? All human disease getting solved? Where are those timelines? >> Alex, if I could just add a a layer on top of that. So we've talked on this pot a lot that math is cooked, right? We're seeing millennium prizes all falling. Yep. And the n the next natural uh if you would uh uh barbecue is likely to be physics >> incineration. >> Incineration. Yes. >> And computer science arguably I mean presumably this is part of the premise Richard of recursive as well. Computer science already cooked math thoroughly cooked. >> You know in a weird way I would I I would use different metaphors. I think

10:16they're flourishing not getting cooked. I think the weird thing is, you know, some some mathematicians recently said, "Oh, this is this may be bad for the field because like we need to train people." Yes, we need to train people, but imagine a biologist or or medical researcher saying, "Oh, you know, it's really a bummer for the field that we cured all these diseases for people." Like that would be insane, right? Of course, you want to just move as quickly as you can towards solving these problems. I think maybe the problem with math is that there's many sub fields in of math that haven't really had there are just almost purely beautiful intellectual exercises without real life impact anymore. But as you get close to

10:51real life impact, you should be excited about it and you think and realize your field is flourishing, not being cooked. Uh, and so to timelines, I think the the further the more complex a disease is, the more you have to do long-term studies with humans before you're allowed to put a certain drug into humans, the more delays you will have. But like we see now companies to make this very concrete, biotech companies used to have like one new drug in development and it took them 10 years. They had to go public before they knew the comp the drug was really working. And then after 10 years maybe late stage three trials were just not working and then the company is dead. Right now you have in the new age of companies they

11:33have 5 to 10 different compounds in uh stage three trials after 2 or 3 years and and so like we see a lot of acceleration in that level but again diseases to really be able to get them into humans at scale in the United States with FDA approvals there are just some natural delays that will be more like half a decade to a decade uh but uh lots of other things in chemistry and physics and so on where we can iterate without having to look at long-term human trials will be even faster. >> And I the the the point is well taken. I I like your Orwellian turn of phrase. Maybe instead of saying AI is cooking math, I should be using flourishing as a transitive verb and just say AI is flourishing math. >> Actually, I was with Certas Caraman who

12:19runs lids at MIT uh the night before last. He he started a drone company now, an AI drone company. But lids is where radar was invented originally during World War II. It's a great lab, very entrepreneurial. But he said all his mathematician friends at MIT uh are aware that they're cooked and they're they're all >> No, no, Dave. Dave, we're going with the Orwellian language now. They're they're not cooked, they're being flourished. >> He That's funny because he literally said cooked, but I'll go back to him and tell your friends they're flourishing. >> Tell not to worry. If your goal was to prove as many theorems as possible in your lifetime, like now is the time to just grab like grab as many as you can and

12:58and work with AI to solve them. And then I think the field will change the way computer science has changed in many ways. Uh and it'll be much more about what's the most creative thing when you really understand all the things that are out there in math. What kind of new formalisms can you create? New constructs can you create that then would be interesting to be solved uh by an AI in collaboration with humans. >> Well, Cash was saying the math guys are in great shape because they're so cooked that they're all moving over to AI orchestration and they're going to be way ahead of the curve. He he's actually most worried about the biology professors who are in complete denial. You using virtually no AI in their

13:36day-to-day activities. And this is one area where Richard, you have a such a deep background in all facets of AI, including biology. And I I feel like we're living exactly parallel lives except you're 15 years younger than me. So I'm like insanely jealous of of your life trajectory. But be a serial entrepreneur, then start a hugely successful venture fund, then found a foundation model company. And and you know, we're seeing the world through the exact same lens. But you and Peter have much more biology background. I I have basically none. uh but the biologists are the ones really really lagging and I I think you've got a bunch of investments actually that are that have done really well in the area right

14:12>> and I would love to talk about some of those like peril bio proximal labs Ignod labs like truly exciting companies I think another field that is even more lacking than biology is economics economics literally has these models of like >> a linear model of economics right a one-step economy that's provably correctly taxed and and subsidized and things like that it's just like absurd how slow that field is to adopt AI for making better policy decisions ideology >> because you opened the door. Eric Bolson, our very good friend, uh I did not realize till yesterday that HI lab high at Stanford invented the word foundation model >> and he sent me the whole you didn't you knew that already? Wow. >> Yeah, of course. Like lots of Stanford

14:58friends and Percy Lang and others uh worked on foundational models. I love Eric Goffson. actually one of the most interesting economists I think uh doing really interesting research right now. He also started work helix which we're a proud investor in uh that brings uh like understanding of how companies actually adopt AI uh and which tasks are getting you know helped by in real rollouts for companies. Uh yeah he's a co-founder of that too. >> Yeah building is just such a great place cuz you talk about econ people being way off the curve but he is so ahead of the curve. You walk into the building and you can just feel it. He's also poaching a ton of talent from MIT to come out to to join his lab,

15:35>> following his path. Richard, talk about full stack AI as you see it in the scientific method. Um, you know, I'm an investor in a company called Laya out of MIT and Harvard. You know, Lyla science as well. Uh, I think I've introduced you to to Jeff von Maltson there. and you know they're building a scientific super intelligence that is then running a million square foot of robotic space to sort of like a thousandx the rate of discovery. Um your thoughts on that? >> I absolutely love it. I think that is you know in the Eureka machine I sort of lay out these four pillars and they're actually one of the few um uh that that are really going after this similar to periodic labs who are doing it more on

16:14the physics and chemistry side of things. Uh, and there are several other companies now that I can hopefully soon talk about that are trying to create more data for the bitter lesson to be applicable in biology. And I love uh what you know the big guys Eli Liy and others do uh and and Laya too. I think you know the ultimately uh you can boil down the scientific method to the ideation implementation and validation of ideas and the faster we can close that loop and then put an open-ended process on top of it that's what open-endedness uh what inspired us a lot at recursive and we have many of the world's greatest uh like researchers in that subdomain of AI that is still not quite as popular as it could and should

16:56be um like the more you can have a swarm uh innovate in open-ended ways and evolve uh and combine interestingly different ideas the better. But then of course in biology you have to have uh like actual uh like physical biological wetwware experiments. And so it's really great to see um uh Laya doing that. Um we're seeing this also maybe I can talk about this one company that I really love called Peril Bio. They build tiny organoids on a petri dish, >> right? And get this, if you love animals, you too can love AI. Why? Because they got FDA approval to skip animal trials. Turns out we cured most diseases in mice. It's not that helpful. They're very different to people. Uh but these guys uh use pur potent stem cells

17:41create tiny little organoids of lymph nodes. Lymph nodes big part of your immune system. Imunotherapy some of the most exciting therapies to allow your own immune system to attack a cancer and so on instead of getting crazy chemotherapy and so on. And so they got FDA approval because they showed that when you run experiments in parallel, hence parallel bio in these tiny petri dishes, how those organoids react to different drugs and toxicity testing and so on is actually more predictive of how those drugs will interact in real human bodies. And that is just one of many beautiful examples of where it will help. And you can take my stem cell, you can create take my skin, create in a uh IPSC uh cell, grow my own organs and see

18:20how a particular drug would work for me versus a generic individual. Uh Salem, you want to jump in? >> Uh yeah, a couple of things. One is, you know, science has always been a coordination problem, right? You're kind of trying to bring and it's always been very structured into departments and and journals and all that stuff. And I love what you're doing, bringing it together. For me, if I had to summarize what you seem to be doing is you're collapsing the time between an experiment and a result or hypothesis and a result. And when you can collapse that, that's domain collapse of the scientific method. And now like anything is possible from there. This is amazing. >> Yeah. I I am going to push again on what

18:58what Alex said, physics. When do you you know you know Einstein's theory relativity comes out. There's not that much progress uh you know since then. a lot of theories. Do you see physics as the next domain that's going to flourish on the back of math? >> I think the biggest domain is actually going to be biology. Physics has been interestingly stuck in many ways like they there's a really powerful ideas like E= MC². So you can get a ton of energy out of potentially little mass and then we got nuclear energy out of that. uh and so that then moved into and graduated in some ways into an engineering discipline uh which is where you have the real real life impact. Um I think you know obviously like fusion

19:42will be great to finally get figured out and there's a lot of really cool like companies and and big labs and you know tokomax already to balance uh like plasma inside tokomax like is a very hard control problem that where AI is being used. Um I hope we can eventually make better theories for uh you know quantum gravity and all kinds of other complex issues and and have get a better world sort of formula. Um I unfortunately the data set collections these days require often like large hydron colliders and just like billions of dollars and it's like quite expensive actually. So in a weird way, biology is a better fit for AI because what calculus did for physics like understanding really well micro uh

20:29individualized separated phenomena, neural nets are great in combining all these little things like we know what the neuron does. We know what one uh bacteria does in our microbiome, but then as they all come together and form these really complex interactions, we don't really know anymore how that works. And so that's where I I think will help us more. Yeah, Alex, >> let's talk about biology a bit. So, do you think the critical path to solving biology goes through digital twins of cells or do you have some wildly different theory of the case? Three years ago when I started Eureka Machine book, I had this chapter on the virtual cell and I was really proud of like trying to lay it all out and I had to

21:08rewrite that whole thing because in the meantime uh you know Mark Zuckerberg Foundation started virtual cell everyone not just CI everyone has a virtual cell model Dave Even if Dave doesn't even if Dave doesn't think Dave has a virtual cell model I'm sure he'll spin one up soon. So I think that most So there's this famous uh uh paper by Rich Sutton um on the bitter lesson in AI and the >> never heard of it. Tell me all about the bitter lesson. This is a new concept for me. What what is this you speak of? >> I don't know. You're probably joking but pro maybe some some readers haven't heard of it. So the bitter lesson is basically that human experts had all these really clever ideas and beautiful

21:45theories but really what you needed to do was the simp like to make real progress is the simplest method that you can come up with a large neural network that you can train end to end as a general function approximator. The simplest model you can come up with and then it's just a ton of data and compute to like actually train that model and that usually outperforms at scale all the clever little hacks that human experts had come up with uh before. And so the bitter lesson has worked for NLP 10, 20 years ago. You would have asked an NLP expert. Can you have one model to have any conversation with you prompted any kind of question prompt engineering I like invented and was nicely cited by the early GBT papers. But like the same

22:25thing, the same state uh is biology is currently in. There's so many complexities and the experts know so much. They feel like there's no way you can instill all of that knowledge into one model. But if you have enough data, you can. And now you have companies like Tahoe therapeutics and so on that are creating these massive perturbation studies and yes all models are wrong but more and more of them will be useful as we collect more and more data about biology and that is I think one of the biggest driving factors uh for the impact of AI. >> But so ju just to answer the question do you think virtual cell models are the critical path to solving all disease or solving biology or is it something else?

23:03>> I I think they're definitely going to play a crucial part. It's the third pillar of the Eureka machine. We do need to have anything you can simulate, anything you can verify, AI will then be able to solve things in that domain. And so yes, uh I do think uh virtual models uh and simulations uh are extremely important. >> I'm going to jump into three fun breaking stories this week. Two on science and one on AI. Uh and Richard, they you know relate back to your work and your book. Uh our first story was published in MIT Tech Review today. Uh it tells the story of researchers at Israel's Weissman Institute who built a system called brain it. The system reconstructs the image a person is looking at while inside a functional MRI

23:50machine. I I'm just post put up the slide here. So take a look at this image. You know it's pretty extraordinary. On one side is the image that a person is looking at inside the fMRI machine. On the other is the AI reconstruction. You know, so earlier brain image decoders could tell you were looking at a dog or a clock tower, but it lost the color, the composition, and the detail. So brain it learns structure and meaning separately and then puts the picture back together. And here's the clever part. They also can run the model in reverse. An encoder that predicts how a brain will respond to an image. So they can feed it images no human has ever seen into the scanner uh generate predicted brain scans and then train on

Brain Image Reconstruction (Brain It)

24:36those. The model effectively builds its own data set. So Richard you know you say in your book that AI is superhuman in any domain you can simulate or verify uh and nowhere else. So is a brain simulatable do is the brain a simulatable domain or is this something different? It's not yet uh but this is still an incredible result. I still remember the first such result came out when I was still a PhD student. I went over to the bioinformatics department at Stanford and they you know it's like very grainy little images we could like extract uh from this and to see this fidelity now and this realism is just incredible. I think one of the problems, I don't know if this study, I haven't seen it today yet, have been in the

25:18work, but like um one of the problems is often that you have to train for each brain because they're all slightly different. Uh and so you won't be able to just take someone who've never put into an fMRI scanner to see and update the model for that particular person, which would be otherwise amazing for people who are like in a coma and you want to see are they still thinking stuff, you know, like that would be incredible. >> Alex, tell us more about this about this story today. Yeah. So this actually this is a result from March. Uh so this the broader field of doing decoding of brain states from functional imaging or EEG or electrodes. This has a long long history at this point. I remember 20 plus years

25:59ago at this point the the Galant lab at UC Berkeley was doing this to decode the the visual cortex of cats. Actually like 20 plus years ago we were starting to see images of how a cat perceives the world. And I remember like the first grainy images that were coming out were like a cat seeing a branch. And then moving fast forwarding to non-invasive fMRI. There are so many groups at this point. Meta in particular has been sponsoring quite a bit of work in this area from Jean Marie King, many other groups doing language decoding, vision decoding. More recently, even fast forwarding to just the past 24 hours, Neurolink just announced their first scaling law studies on pre-training foundation models, Frontier models off of Neurolink

26:46electronic data, and this is on an individual patient basis, but they're capturing copious amounts of of data at high temporal resolution, much higher than fMRI can capture. fMRI like rule of thumb fMRI can give you with today's technology at best cubic millimeter spatial resolution for voxels and at best approximately 1 second temporal resolution. So that somewhat limits how well you can do in terms of decoding a person's internal visual state. can still make no mistake there are folks who have are making a cottage industry out of decoding dreams uh and decoding visually what a person perceives or what they hallucinate in their visual cortex while they're sleeping or while they're awake and I'm sure this is going to be a

27:31vibrant vibrant space training frontier models and foundation models off of copious amounts of fMRI data but ultimately where I think this has to go there are spatial and temporal limits to fMRI decoding It's going to require higher temporal precision and higher spatial precision to get where we really want to go, which is full dive VR and full bandwidth BCIs. And we will get there, but the good news is you have to start somewhere. The large language models had to start with GPT1, just a single artificial neuron that could predict the polarity of Amazon reviews. Similarly here, if we're going to get full bandwidth BCIs, it's going to start with studies like Brain It and John Marie King and Jackalant. Amazing. Sem,

28:19what what happens in the world where you can know someone's thoughts? >> Well, I think you you Let's be careful. We're decoding the perception here, not the actual thought. Right. That's like quite a bit more comp. What's the other than the brain regionalization without a difference? >> No. No. I think there there is a difference. >> No, when when you imagine, right? When you imagine something in your mind, it lights up the same neurons as when you're seeing them. >> That that's fine. That that's fine. Just I just want to make sure we don't mix the two because you can have thoughts without necessarily the perceptions around it. But you can definitely have one without the other and the other without the other. So uh but I think for

28:59me the more interesting thing here is you know we've spent billions of hours typing things into little rectangles and if we can change that interface to do a more natural interface which is interface directly with the brain. We used to talk about this in our singularity university lectures on neuroscience is you know we still have very little idea how the brain works but you don't need to have know how it works as long as you can interface effectively with it and this I think gives us the opening to give us really deep interfaces and then that'll help us figure out how the the brain works. I >> I'll take the >> you bring up a really Oh go ahead Richard. Sorry. >> Sorry. You bring up a really good point

29:35which is I I read this result a while back that there are two types of people. Some people who think actually think in sentences and other people when they think they just have a fuzzy thought cloud and only once they're asked to verbalize their thought or they try to write it down they actually like make a real sentence out of their fuzzy thought clouds. And I'm definitely a thought cloud thinker. My wife is very much a like sentence like in her head. She actually thinks in actual sentences and like there's no there's no like one is smarter than the other anything. there's like they're like equivalent. But like I do think sometimes like writing is thinking and like for me it is very helpful if I have to really verbalize

30:15something versus I just have the thought. I'm just trying to think like what does the world look like if the eye could really just like extract my thoughts and now I need to like structure my thought clouds into sentences more and more precise. >> Richard, what would neural net training look like if you took all text out of it? No language at all. just train on pure images and higher level thought constructs, you you'd get a very different neural net out the other side and might actually think a lot more like you do and less like your wife does. Cuz >> I'm not sure. >> Maybe maybe we should be taking the position that anyone who doesn't have an inner monologue is just a pea zombie >> there. I got to throw one more thing in.

30:53I remember we had one of the top linguists in the world come in and speak and we were like, you know, this must be a bad time to be a linguist. nobody's using spelling or grammar or anything else like that. And he goes, "No, on the contrary, this is one of the most interesting times to be a linguist ever." And we're like, "Wow, how come?" And he said, "Cuz when you look at emojis, it's the first time in the history of human language that you can symbolically send emotion because you can digitally send emotions." And we're like, "Wow, that's an interesting." So, he was super excited by opening up that whole aperture. I thought that was an interesting >> You're saying you're saying you're saying that emojis are the reason why

31:29linguistics and not the fact that linguistics itself has been flourished by >> I'm not connect I'm not connecting to you did that though it's not me >> Dave >> oh no the the result I really am looking forward to is uh how does the brain >> train itself without gradient descent everything going on in AI right now everything going on for the last 20 years in AI is driven by gradient descent algor algorithms and the biologists can't find anything even vaguely like that in actual biology. And so that that's you know I think with with really detailed imaging we might finally crack the code on what is the fundamental learning algorithm that changes the synaptic weights. It's not it's not what we use for artificial

32:11neural nets. It's it's something different and if we discover that we might find that neural net training can be 10 100 a thousand a million times more efficient. And so hopefully that'll come out within a year. Could you define gradient descent for our listeners? >> It's funny, Ilia Sutskiver uh when when he's on interviews, he's like there is only one algorithm. The algorithm is gradient descent. Everything else is just irrelevant compared to So what happens right now is you build these 96 or 120 layer deep neural nets and all it is is a whole bunch of random connections that do absolutely nothing useful and then you give it trillions of 15 trillion training examples. You're like, "When you see this, say that or

32:50when you see this paragraph, this is the next token." And if you're guessing wrong, you get punished through gradient descent. So the error or the how far off you are is your punishment and it passes back through all the layers and it calculates an error and it blames each neuron or each connection for how how responsible were you for this terrible answer. And if you're way wrong, you get moved and you move in the direction that helps you get the answer right. So it's this incredibly laborious search process and that movement of each synaptic connection is a gradient and you just you just hill climb to the best possible state and then it magically after about hund00 million of comput in Richard's

33:31case I guess 400 million uh it magically starts thinking through this one algorithm gradient descent. >> Yeah I think it's really back back prop that we're talking about syndictoically rather than gradient descent would be the the obvious comment. So maybe yeah back prop is I guess the efficient computation of those gradients. Uh maybe I'll I'll add to um add to this explanation from Dave but like gradient is essentially an optimization algorithm that we use to train all kinds of machine learning models by minimizing the errors. And I think the best analogy is to assume you're a blind hiker. You're the top of the mountain and you're trying to find the lowest the lowest point and you can't really see

34:09that far. You're blind but you can feel the slope at each step. And so how much like how big of a step should you take down when you feel like the slope is going roughly in the right direction. And then the problem is that the kinds of landscapes these AIs are trying to optimize are highly non-convex which means there's not just one lowest point uh but there are many different low points. And if you depending on where you start you might go into a different valley depending on which mountain you start you go into this valley versus this this different valley. And there are actually a lot of really interesting analogies. My friend Jihan uh like thinks about this from the psychological perspective. For instance, when you have

34:45PTSD and you train, you overfit to one algorithm. how your brain is kind of stuck in one valley of how to think about something. And it takes sometimes people like doing psychedelics and stuff uh that have shown to to help help with like depression and PTSD and other uh mental diseases like um they they it helps to increase the learning rate to jump over another mountain and and go into a different valley of attraction where you then have new kinds of ways of thinking. And so this blind hiker analogy I think is a really intuitive way to think about it. And then of course the problem is it's not just in 3D, it's in a trilliond like a trillion parameters that you're trying to traverse in that landscape. But the

35:26ideas of like try to identify the direction and take steps towards it still. >> And you know what else is really incredible? I could grip on this for hours, but if you look at a big model like Kimmy K3, it's 93 layers and it's trained trained trained. If you take a single layer and you randomize it and then you train it and say find yourself again, it can't find itself again. It never gets back to where it was. Which comes back to in these fMRIs when you're looking at a human brain and you're saying, "Okay, here's Alex's brain. Can I compare that to Salem's?" And you're like, "Wow, there's nothing in common going on here yet." you know, well, okay, maybe that's a bad example, but but the way your brain wires as you learn is

36:11completely unique to you. Yet, yet it can come out with like, okay, we're equally good soccer players, but the way the signals are propagating through our networks is completely different and unique to each of us, which is really really strange, you know, that that it can't it can't find its own way back to its its state. I think we're going to learn a lot actually with the fMRI data coming in on how and why that works. And there there have to be commonalities that we just can't find. You know, rotations, you know, make everything look different, but they're really not that different. If you if you put them through a transform, like a 4A transform view will suddenly say, "Oh my god, this is why they line up."

36:46>> Yeah. I was I was just with my fraternity brother uh of ours, Dave, who's the head of neurobiology at at USC the other day. um and is you know this is the most exciting time ever for brain science >> right I mean we're you know the brain has been a black box for for since humanity began and our ability to understand uh the brain and deal with with uh you know mental disease I think is going to be extraordinary. Yeah. Well, so you can simulate. This is why liquid AI was founded actually because we completely reverse engineered the the C elegance worm brain down to exactly every single thing going on and then they were able to simulate it and then after they simulated it they realized wait this is

37:32a very efficient neural net and then they productized it. I mean that's so the both ang so go ahead Richard >> there's a really cool thing since you mentioned se against worms there's not many people that bring that up a friend of mine who's a professor of neuroscience at Harvard Sam Gershman uh he one of the most brilliant people I've ever met like he actually did a a study with with worms where he cut them in half and one segance worms they do have a brain right and then they have a lot of other like neurons in the rest now you can train them to react to a certain stimuli um and then you split them in half the the second half without the brain regrows a new brain. And here, get this, this is crazy. That new brain has

38:10the same memories. >> Wow. >> And reacts to different stimuli. And so he's like, maybe there's a different way where we learn. And I do think there are things that right now we're very much stuck in this like it's a neural net, it's all electrical signals and so on. But clearly like brain chemistry can change massively with just like you can get hangry, you can be in pain, you can have like a certain like stimulant and all of a sudden your brain is very different. And I think there are and I've seen this now with uh like uh having babies and and like talking to uh moms like there's certain algorithms where you all of a sudden you nest because you had a baby and now you want to like and how you nest is different

Gene Drive Mosquito Executive Order

38:47but people nest you know in one form or another and so there are these latent algorithms that we have that are in our DNA that just trigger after like 20 something years right when just the right things then happen with your body and your biology. So, I think there's still so much that we don't we haven't >> so much complexity. I'm going to move us to our next story from three days ago. So, on Tuesday, uh the president signed an executive order directing the EPA, the Department of the Interior and Agriculture working with HHS to cut invasive mosquito populations in Washington DC by at least 90% and the tick population by at least 50% by 2028. Uh, the targets include mosquito species that spread deni, zika, and yellow

39:31fever. And here's what caught my eye. You know, the order explicitly prioritizes sterile insect techniques, safe genetic modifications, and beneficial bacteria over conventional pesticides, right? I love that. You know, this is the exact kind of uh gan drive work that Colossal, the deextinction company, is doing. You know, I think most people don't realize it's like, you know, when I was raising my kids, I would ask them a question, who which which is the species that kills the most humans on the planet, you know, and some people jump onto sharks? Some people jump onto, you know, whatever it might be. But, uh, on this part, I'm sure people know mosquitoes are the deadliest life form on Earth,

40:09>> right? Malaria alone, uh, kills half a million people a year. So, uh, Alex, u, this is biology as engineering arriving as federal policy. Uh, your thoughts on this story? For decades, we've been scared of our shadow. So, I have a classmate Kevin Esfelt at MIT who was the pioneer of the GAN drive, one of the pioneers, and he's had a devil of a time getting states, municipalities to approve gan drive studies against mosquitoes. And for those not tracking, the the premise of this technique is basically inserting a gene via crisper that wants to replicate itself to basically sterilize mosquitoes through propagating virally through the mosquito population. We have the technology to do this. What it has lacked, at least in

40:56this country, is a federal mandate to actually go and implement large-scale genetic engineering of non-human animal populations. And now for the first time it seems we're seeing via executive fiat just that mandate. And it is so exciting for two reasons. One because one is Peter you mentioned malaria and other non-human animalborn diseases cause an an enormous amount of suffering human suffering and also non-human animal suffering. But secondly because there's a way to do this that doesn't actually involve killing the animals themselves. Historically, historically, you know, if you look back 50 years, you'd see chemical spraying if you wanted to do something about say insect born illness uh responses. We don't need to do that

41:45anymore. We can actually keep the insects that are the inadvertent carriers of bacterial or viral disease. We can actually preserve their lives while also preventing the disease transmission. And I think that's good from their perspective as well. That was the part that got me most excited was we spray things, we're basically poisoning biology now we actually >> ecosystem, right? >> And leave the poison ecosystem. >> Yeah. Yeah. >> Uh Richard, your thoughts on this story? >> I I concur with everyone like it's it's it's wonderful. Uh and it's so interesting like people I I hope we can amplify these these kinds of stories more, you know, like the future needs better marketing. We need more Peters in

42:27the world. Um and and this is one of those many stories, you know, curing various diseases, peril bio, like saving animal lives and and prevent them from just being bred uh to be tested upon and dissected like uh those are all like stories we need to amplify more in the public eye. Yeah, Dave, I can't wait for this to come from DC to Vermont and Massachusetts and all of these >> I was going to say for the listeners that are interested in your real estate fund, which you know the the theme there is, hey, easy access via drone to hilltops, islands, but also if you live on the edge of a marsh, your house probably sells for about half the price of a place that's not on the edge of a marsh, that'll go away. there there's so

43:08many solutions coming to biting insects, this being one of them. Uh that that also is is, you know, if it's beautiful, beautiful land that otherwise is very difficult to live on, it's going to go through the roof in value. >> I do think it's really important to try to like, you know, keep the bees uh like alive, all right, and not have spray bunch of pesticides, make sure birds still have enough like insects to eat and things like that. Yeah, >> I was going to say for maybe up to 80 years, humanity has been scared of its own shadow. humanity in general, America or the West in particular, nuclear energy. Another example I've pointed out on the pod where we arguably lost 50 plus years of progress because we were

43:47too scared of either the bomb or fision reactors or some an entire generation watched the movie China Syndrome and then got scared of nuclear power unnecessarily. Same idea with this or or with geoengineering. Another example, the the world is still scared. Some fraction of the world is scared of engineering the weather. We don't need to be scared of engineering our physical world or our biological world. And now hopefully one can see green shoots of humanity getting past that stage of worrying about its own shadow. >> This is interesting that Sorry. Yeah, it's really it's interesting indeed that like sometimes alarmists feel like they're doing the right thing by just like oh how bad could it be? I'm warning

44:28people of something bad. But like it can indeed push all of humanity away from something really good like energy abundance uh with nuclear. Um I think overpopulation is when these other big myths where people are like well if we have too many people there's going to be scarcity of all these different resources and everything is going to get more expensive and people are more poverty and so on and the exact opposite happened. There's actually one website that I think you all would love which called humanprogress.org or.com like that that shows that a lot of these things are actually getting cheaper and cheaper despite there being more people. >> Yeah. >> Well, hey Richard, you're you're a hero

45:01in Germany like like can't walk down the street kind of hero in Germany. What do you think about the fact that Germany has no power for exactly this reason? >> It is really unfortunate if there are a lot of people in Germany that want to in a weird way sort of offramp from progress. Um and they think that everything that consumes power is bad for the environment. there's like a weird sort of degrowth uh offshoot from generally like well-intentioned pro- like environmental uh vibes that uh just worry me um worry me quite a bit. Yeah, >> we're going to talk about dumerism in a little bit uh but not yet. And you know, one of the things I realized a long time ago is we humans are really incredible

45:42at seeing a problem out in the future, right? We see acid rain, we see overpopulation, we see, you know, energy shortage, whatever it might be. And then because of our amigdula, because the way we think, we accelerate that future problem to today. And we freak out and we forget the fact that there is a decade worth of progress we're going to make by the time we reach that problem. And that's what entrepreneurs do. They solve problem after problem after problem. And I I guess I just want our listeners to hear that because if you're worried about some future problem, please understand there's a incredibly efficient market uh of entrepreneurship and capitalism that will solve the world's biggest problems, the world's

46:25biggest business opportunities, right? And it's a beautiful uh forward uh you know uh uh forward propagation solution set that we have. And maybe we'll get there later, but I I see the same thing happen in AI dumerism where they create more and more complex scenarios where attackers get these like near magical abilities to attack, but somehow the defenders in those stories never get near magical abilities to defend. And it's this weird thing of like, man, if I can create a super virus that is like perfectly undetectable and spreads all throughout the world and no one notices it and like and then it has this Wi-Fi switch and you just turn it on and off and kill people like I I will create a super magical vaccine that just

47:07inoculates you against all of these things. Like if you can actually assume make that assumption then like it's a weird Yeah. weird thing. >> This episode is sponsored by Google for startups. Think about this for a second. You now have access to the same generative AI models that cost hundreds of millions of dollars to train. Google's startup technical guide for generative media gives you complete blueprint for deploying Google DeepMinds models in production. Images, video, audio, all of it. Real architecture, real results. Find the link in the show notes below. I'm going to move us to a story that's been sort of breaking the internet over the last 24 hours. Um, and it it puts your regulate the application argument

Tavus Griffin and the Video Turing Test

47:51to the test, Richard. So, a company called Tavis unveiled Griffin, uh, which it calls the first model ever to pass the video touring test. So, here's the numbers. 48% of people who talk to an AI avatar on a live face-to-face video thought they were talking to a real human. Previous systems were at 3%. So, let me show the video here. And uh it's super cool and let's talk about it next. >> Now, how about just a thumbs up? >> Sure. One thumbs up coming right up. >> Hell yeah. That was awesome. I would say you're my favorite coworker. It's been It's been really great toh work on this with you. >> You're making me blush. A Who's that with the ugly sweater? >> Simon says, "Touch your hair." >> Is this what you're looking for?

48:37>> Yeah. Okay, nice. Simon says point. All right, I'm doing it. Now, touch your chin. >> I'm not going to fall for that. You didn't say the magic word. Uh, >> you got me. Okay. >> The loose connector back onto the motherboard. >> Okay, let me go let me go grab that. But beforehand, yeah, I turned the soldering iron on. I think it's going to take about 12 seconds. So, maybe just let me know when that's done. >> It sounds good. I'll keep an eye on it and let you know the second it hits that 12 second mark. >> All right. So, uh, interesting. So, Tavis calls it the first human interaction model. It's fully duplex, listening and talking at the same time that you do, the way humans actually do. It's number one on Nvidia's benchmark

49:28for full duplex AI video. Tavis pitches it as quote a tutor for every student that notices when they're lost, an elder care companion that listens. Uh Tavis itself says Griffin requires safety work before it can be released publicly because it's the first model that can be mistaken for a real person. Um Salem, your thoughts on this one? So, I'm less kind of um floored by the um the imagery and the the mimicry of it. Uh this was expected to happen. For me, I'm more interested in can it accomplish something useful enough that I don't care whether it's human or not. And I think that that is more interesting uh which might come along and have hap we have that happen at some point. We've talked about the idea that

50:18in the near future, like now, uh you're going to have digital co-workers that pop up on Zoom, that you call, that you text, you Slack, and interact with you. And I think it's pretty compelling as a mechanism for a future, you know, virtualized company. Uh Alex, >> well, first, Peter, you should probably ask me to touch my face. >> Okay, touch your face. Simon says touch your face. >> Oh, >> okay. Very good. Just checking but I I guess Tavis is ahead of me. I I I think on the one hand if you look at models coming out of Chinese labs like Alibaba's lab, Juan Streamer gave us a preview of what was going to happen. The Chinese labs remain overinvested relative to the Western labs in generative video models and interactive

51:05generative video models. So one streamer I think is a preview of what's possible. My guess is I I I I I read the the information that was put out around Tavis. I I think in full generality, let let's just talk about where this is going to end up. It's it's very difficult to predict the short term. I think it's pretty easy to predict the long term. In the long term, endto-end generative pixels. We'll have these magic mirrors that could be real time fully interactive pixel-wise generated or if if anthropic has its way, vector-wise or procedurally generated. But either way, fully generated real-time interactive video models and you'll be able to create a scene that consists of people like it's possible

51:52right now, but the latency is high. You see with one streamer, which is already out and already open source, or with this Tavis Griffin type model, which is not really out yet and definitely not open source. You see a preview of the future. What does this look like? Well, there are a few different ways it can go. Query how revenue generating per token it is. I is it anywhere close to the optimal frontier of code generation? Doubt it. On the other hand, if it becomes so absurdly inexpensive to be able to to generate arbitrary, say, humans participating in a Zoom meeting or humans participating in a podcast. Do we really care whether it's close to to being near the optimal cost performance frontier? Maybe not. There are a lot of

52:44human service industry jobs that require a face and require interactivity and a voice and the ability to touch one's face apparently on demand or on on request that could be probably completely automated away by a model that otherwise would be limited to textbased interaction but doesn't have a face and a voice. So in the most optimistic scenario, these sorts of interactive video models open PN note that the acronym for this which by the way was there first with Alex Finn him him is such an obvious reference to her the the movie closed PN I I think this this has this >> you should use you should use m dashes please >> okay I'll delve into it so I I I think I I I think this is going to be transformative for the service sector

53:34And hopefully TAVIS and the broader American west of video frontier models and interactive video frontier models take a page from Tavis and start competing with China. >> Richard, one of the principles in your book, one of the rules you have is that AI becomes superhuman uh where you can verify the answer. uh and you know is like passing a human scientific milestone like this uh is passing sort of a human judgment and interaction uh you know sufficiently as a scientific milestone for verification. >> Not quite. Uh it's sort of like it's hard to scale when a human has to be in the loop to say yes this is like a human another person on the other side or not. Uh you can't quite automate it completely uh and verify it in in that

54:21automated fashion. Um I do think you're you're right. This is a good example where I think we need to do be go beyond KYC and do KY know your use case. Uh right like you don't want this technology to be in Zoom pretending to be the CEO and like there's some famous stories where this has already happened where they got someone to wire like $20 million cuz they created a Zoom with five other executives and they all like he was fooled well enough. Um and and so I think we need to um yeah be careful and like Zoom probably and Google Hangout need to start like finding countermeasures to identify is this a real person and and things like that uh to prevent those kinds of hacks from happening. >> Yeah, scams are going to proliferate on

55:04this. I I just want to do a call out again. And I've said this before on the pod and if you're if you've not heard this before, if you still have your grandparents with you or your parents and you've not taken the time to record their stories on video and audio and go deep, spend, you know, hours recording them, right? Your ability to create a super highresolution lifelike avatar for your your kids or your grandkids or your great-grandkids, I think, is an incredible thing. But you need the data. So, if you're listening to this and you're lucky enough to have your parents or grandparents around, collect their data because it's going to be a beautiful opportunity for your uh for your progyny and theirs.

55:46>> A andor I I should add get them an Elor membership and preserve their conneto if there is a consideration like what why stop with just the behavioral data preserve the whole parent. You know, my kids my kids are in a deep freeze or at least they're placental cells. >> But I have a push back on that just I have a push back on that just from this conversation. >> If from the earlier story we can read people's brains, all you have to do is picture an image of your grandmother and then extract it from that. >> It'll be low low fidelity. I mean I I do think like to to first order this is how ancestor simulation will work. You see stories left and right just in the past 48 hours of people using opus 5.5 or or

56:29other frontier models to reconstruct bits of history that would were otherwise unknown just based on artifacts of the day. But at some point I I do think sem you want to go beyond just the memory of the person and you want the actual person. And I I'll pound the the drum again in addition to Peter you know very generous of you to preserve the placenta associated with your children but not necessarily your children. I'm sure the placenta placenta will will thank you. Uh but >> yeah, go ahead. Finish. >> But but I I I I would say like yeah, if if if this is a serious concern, you're worried about your parents or your grandparents and you want to go beyond just having recordings of them or a AI prompted generative interaction models

57:11of your ancestors. Go get them Alore memberships and get them cryopreserved. >> Yeah, Richard, really uh really curious to ask you a question. And you know uh we we clearly passed the touring test and you know Sem's reaction is my reaction that the what we just saw is a so what if you've been using models every day all day long you you know you just had to stitch together the components and you have but I but I think this will show mainstream world that we've crossed the Turing test and then you know the next milestone is the demos test where using information from 1910 and prior rediscovery equals MC² uh so you can't cheat which is a really tough one to measure because cheating is is you know but without cheating only

57:53using prior information come up with the equals mc^ squ so is there a Soer test that's like a really fun measurable milestone coming >> I I came up with the like uh anti- touring test uh which actually I think the whole touring test has flipped where now in order to know whether there's a human on the other side or not you actually ask it questions that are so hard no human could ever answer it. Like if you asked an AI to just write you a complex web app uh and like 10 seconds later it comes back with like 50,000 lines of code, you're like you kind of know it was not a human, right? So I think that test has actually completely flipped. >> You don't think Richard I mean like you it could throttle itself. That one's

58:37easy to defeat. But just like basket now it's just about it's just about fakery. you like the reason it was an intelligence test for artificial intelligence was that it was so hard to be as smart as a human. Now you just have to throttle yourself down to human level in order to pass the test which makes the test useless as an inspiring test for intelligence. Yeah, I think perversely probably the best way to uh like the best capture for knowing whether you're inter interacting, this is not a recommendation, but best way to know whether you're interacting with a textbased chatbot is probably to ask it a CBRN related question and see if it's capable of responding. Probably not. >> Yeah. Uh let me just hit real quick uh

59:19again for our listeners. If you're pregnant or someone in your family is pregnant, you know, uh save their placental cells. One of my portfolio companies called LifeBank USA does this. That's where I've saved my my kids placenta. You know, the placenta is the 3D printer that manufactures the baby and it has the stem cells, natural killer cells, T- cells, exoomes, and it's like from those cells, if my kid should ever need any kind of biological enhancement or organs, you've got the original boot disc to use an old term, uh, for your kid's DNA. So that's uh LifeBank USA. Just I just think it's like a moral obligation parents should have to to save those cells uh as a backup. Um >> as a backup for your children. Could you

Recursive Self-Improvement and ASI Timelines

1:00:07use it to replace your children, Peter? >> You could you could use it to clone your children. For sure. >> And now now they know why you're banking them. >> Yeah, exactly. An army an army of young demand. Uh Richard, uh let's jump next into the core of what you're building at recursive. uh namely recursive self-improvement and super intelligence. So let me ask a few sort of uh key questions to kick this off. So first where are we at the moment with RSI? Number two, how do you define ASI? It's been a long-standing debate. And then how far away are we from ASI? So can you hit those three? >> So yeah, um I think there's >> So where where are we at the moment with RSI? So this this year something major shifted

1:00:58and that is uh AI uh can now code uh right and that uh is a major shift uh in in the ability for it to change itself you know like we now are able to essentially lean into the fact that AI is code and AI can't code right so you have a loop that you can close there uh and so in various weak form forms. We already have RSI. Um, and the weak forms that even Enthropic OpenAI talk about when they often talk about is like look how much our employees, our engineers, our programmers use codecs or claude to create some code. And I would argue that that is a weak form of uh recursive self-improvement because you still have like deeply embedded humans in that loop. And what we're working on at Recursive is to have humans only be

1:01:49involved in setting up the rewards and the environment and the goals and then allow the eye to have the entirety of the process of ideiation, implementation, and validation of ideas and have full control over that and allow for so-called open-ended algorithms, uh, evolutionary search algorithms that combine interestingly different ideas, uh, to really flourish. And I like it's been incredible. But like we have like forms of that going already. Now where the the the sort of physical reality hits is that you still need a lot of compute. If you ask that RSI to come up with really great forms of itself, you need to give it a lot of compute uh to come up and be able to train very sophisticated versions uh of

1:02:31itself. But this is going to take off uh next year. Like it's uh I'm >> So we're not there yet. uh we're not quite there yet, but we're very close. Like, and again, in weak forms, there's already one. And then there are other ways that people slice and dice it. My friend Jason Weston, who who's still at Meta, um he kind of wrote a paper around this where you can think about different axes, learnable axes of self-improvement, the parameters, the training data, the objective function, the neural architecture, and the overall code and the harness and everything else. And no one has really cracked the nut of doing all of these five plus um like truly uh coming up with the ideas on which of these dimensions and axes to

1:03:15optimize for. Uh and you can know that that has hasn't happened yet because all the big companies are still hiring thousands of engineers to do it manually to a large degree with more and more implementation help from an AI. >> Your definition of ASI because I want Sem to hear this. I think artificial super intelligence uh has to spike uh uh at the very least uh across uh several different capabilities. Uh but in in the grandest uh definition of it, it should supersede not just arbitrary humans like a touring test but all of humanity to solve arbitrarily hard tasks. And eventually uh I would argue there are 10 different spaces of intelligence that I define in in the Eureka machine book too. But eventually it cannot just

1:04:00purely robotically do exactly what it's told. It should have some capability. And I'm not saying a sort of a mor moral pro prerogative, but like um I would argue that something isn't super intelligent if it cannot choose to some degree what it works on, right? And has have some metacognition about its thought itself. But uh generally the most easy way to measure it is just capabilities across many different spaces of intelligence like visual perception, communication, language, social interactions and so on that is uh beyond that of humanity and that we we are still far away from >> Elon's definition is as smart as all humans combined. Is that yours? >> Uh I would argue it's it's smarter than

1:04:45humanity combined than then it's truly super intelligent. occasionally and go for it. >> Well, it's I this is where I go bananas because you say it's as smart as a human being. Well, what the hell does smart mean? Uh because I can be emotionally smart and I can be I can have physical intelligence if I'm an athlete or linguistic intelligence or musical intelligence. So, smarter seems to be a very vague term to me in terms of what do we mean by all of this? Can I just shift the conversation just a bit? Um, I made a list of things and I'd love for you to I'm going to throw out the list. You tell me where recursive self-improvement begins, right? Because I'm This is where I'm kind of stuff. So, we've got a continuum of okay, AI writes

1:05:28some code that the next model uses. Number two, AI proposes some experiments for uh researchers. Uh, number three, AI runs those experiments or then it evaluates the results. uh then it modifies its own training system and then it launches its next iteration of itself without meaningful human intervention. So in that spectrum if whatever if those are roughly a spectrum where did recursion begin and that's where I'm struggling. >> It's a great question. Yeah, I we often talk about the ideation, implementation, and validation of ideas. And true recursive self-improvement in its strongest sense has to have all three of these done by an AI in >> So you have an inner loop for each of them. Okay. >> Exactly. And an inner loop and then

1:06:15there has to be an outer process that is more open-ended where the eye can innovate and recombine interestingly different ideas similar to biological, cultural, and technological evolution. Hm. >> And and Richard to hit my third question. When do you believe we reach ASI? Give me a time. Give me a time frame. So I think in the I think in the strongest sense the absolute strongest sense where indeed uh like Salim uh mentioned there is uh there are 10 spaces that I define my book of intelligence, perceptual intelligence, communication intelligence uh like uh interaction uh sociological intelligence, creative intelligence, the speed at which you can do things uh meta cognition and and so on uh knowledge um

1:07:04reasoning mathematical reasoning and so on. So there 10 of these spaces uh and to be better than all of humanity combined I think will take us probably several decades. Um I think it's also a bit of a change >> shocking. >> Yeah. I mean Elon says 2029 2030 latest. Uh >> I think those he probably means like weaker forms of ASI where it is you know like you can say it's better at programming than all of humanity and we'll get there. It it's it will be better at math uh than all of humanity and will like that that will be in a few years. It'll be better at any game where you can see all the parts of the game like go and chess and so on. Like there there are many areas where it will spike to be better than humanity. But humanity

1:07:49can build a large hydron collider. Humanity can create a gold atom. Maybe just a few atoms and takes a ton of energy. But like we can create novel like atoms like right and and different molecules and so on. there's it's going to take a while before we even give uh AI the access to the physical world such that it can innovate in that way beyond all of humanity right and really build like Dyson spheres and so on that it will take some time >> Alex we're we're drinking we're drinking different water as uh as our our our frenemies in the uh in in the alignment community would say I recognize that I'm confused and I recognize that I'm very confused Right now, Richard, you you're running a recursive self-improvement

1:08:34company, but you think super intelligence is 20 years away. What are you thinking? >> So, again, how do you reconcile intelligence? >> I think I think super intelligence will spike and there will be areas where it will be super intelligent and that is like algorithmic development for instance, uh programming, uh and again AI is code, AI can code, and that will be a super uh human capability and and is in many ways already. Um, we just have to be realistic that like there are certain physical constraints, right, about like physical control. Controlling your own substrate, allowing your computational substrate to be modified will require like novel supply chains. It will require novel like materials. It

1:09:15will require ways for that AI to get access and for us to give it access to building new ASML. Like think about the machine of ASML that actually creates these like one two nanometer like chips right that is just like it will take more than 2 years to build such a machine even if you had the perfect blueprint for >> it might take I mean it might take 3 years for Elon's free electron laser to replace ASML which is propping up half of Europe's economy but I don't think it'll take more than three years I >> yeah like wildly wildly >> all that itself Yes, of course. Like I I I from my perspective, hooking a AI up to the physical world, like giving it model control protocol or hardware control protocol, whatever anthropic

1:10:03decides to brand it as these days, that's the easy part. Giving it access is easy. I like if it's super capable, giving it access to armatururs. We talked in a previous pod about what happens when you just take Astra straight out of the box and you drop it into a car. It's able to drive a car. If you give it the controls, it knows how to use them increasingly. I don't think manipulating the physical world is an obstacle at all. Completely don't buy the 20-year timeline. Putting that aside and that apparently I'm drinking very different singularity water that you >> I agree with you, Alex, for what it's worth. >> Thank you, Peter. Um I I I do want to ask though, so uh putting issues of timelines aside, I I am curious as to

1:10:43whether we can at least agree on what the end of the rainbow looks like. So say we we run this recursive self-improvement story to its conclusion. What does the end state of what is the fixed point of recursive self-improvement look like? What is the perfect AI model architecturally look like at the end of the day? >> Yeah. Uh I think one uh it would be uh hubris for us to know uh right now. >> No no but it's just Richard it's just us talking like no one else is listening. It's okay. You can tell me it it's this is a this is a safe model space >> in this whole conversation actually >> it's safe Richard you can tell me what you think how this ends >> uh so I guess there different ways to answer that question of like how like

1:11:25with the exact model architecture and there's some things I can share it with recursive uh that we're working on but like I do think uh that state will be incredible uh I think we will that AI will be able to innovate and out innovate on along any dimension that we want it to innovate I think most diseases will be uh curable with enough funding. Uh and then again just to like say like to pro prove my point like to get a drug through FDA long-term trials takes a few years. I would argue ASI will have cured all diseases and can cure all of them but just to know whether that happened will take more than 3 years. Even if we had all compounds ready to go and be manufactured like tomorrow like just

1:12:07because of FDA things. So there's some there's some like >> but we have we have cell simulators that are going to be able to demonstrate and prove definitively that in this cell your cell this drug works. I mean the idea of human trials is going to get incinerated. I think I mean >> I know it's going to get it's going to get flourished. We're we're using Richard Everything's flourished at this point. That'll be a new new t-shirt P flourish. It's funny. I usually on all podcasts I'm the one who is the optimist and maybe here I'm like I'm I'm still an optimist. I think this will all happen. We're just disagreeing on timelines and it makes me kind of feel like I'm I'm the pessimist. But like I I I believe

1:12:45virtual cells are are amazing. Cells are incredibly complicated. If we want them to be really really perfect, we cannot currently measure all the proteins that happen in one cell without destroying that cell, you know, like and so these perturbation studies for instance that Tower Therapeutics are working on like they're taking they're adding one molecule to one cell. They're seeing how does that molecule change that one cell and then they get one data point. We need to like collect a lot of those data points across a lot of different cells without destroying each each cell in the process. So you know the one cell is very complicated. Once you have one cell, you have multisellular like you know organoids and you have to like put

1:13:19those all together. I one thing that I would love to start as a company if if I had extra time which I don't right now is is to actually build a system of organoids where you can really have not just like one lymph node but you have a whole lympnatic. >> It's being done. It's being done. I can introduce you there. >> This is a very eloquent >> if I may. This is a very eloquent distraction from recursive self-improvement. this this little sideline that we went on about cells, but Richard, I I really do want to try to pin you down on where recursive self-improvement goes. So, you've been very public about not nano GPT, but nano chat. We talk on the pod all the time about the nano GPT speedrun world record

1:14:00collapsing. Just in the past week or two, there was been >> Oh, you just wait for a few more days. There will be another really fun update there. >> Amaz speedrun. >> Yes. Yes, that both of those >> give us a few days. >> Okay. So, so but let >> just quickly you're next. Let Alex finish up >> to to pin this down though. So, so I'm standing by for the major update on nano GPT world record speedrun. But there's been major progress there without requiring any new data scaling at all. These are purely largely my perception is recursive self-improvement algorithmic improvements that have been able to collapse the amount of time that it takes to train a GPT2 class model. And there's been a mini scandal brewing

1:14:43in the community over the past 2 weeks over a collapse from whatever it was like 60 or 70 seconds down to something like 40 seconds by approaching the problem differently and factoring out world knowledge from the ultimate model and hand ringing. Does that constitute viable training of nano GPT if you factor out all the world knowledge? So, I'm I'm using this as an attempted stealthy way to try to get you to at least comment on whether you think that the the the perfect model at the end of the recursive self-improvement rainbow. Does it at least factor out world knowledge from a reasoning core or do you think those always remain unified? >> Can I ask interject one very quick thing before Richard answers

1:15:27>> please? >> Which is that the definition of a singularity is you can't see past the event horizon once you have full RSI. That's vertical. By definition, we can't predict where it goes. >> That's Ray's definition that I don't subscribe to. >> Disclaimer. I got that. Over to you. Over to you, Richard. >> Sorry. So your your question is do we separate what exactly from >> does does world knowledge a at the end of recursive self-improvement once we have our perfect model some would argue that the perfect model should cleanly factor out and segregate out world knowledge which could live in a text file or a database from the weights or the parameters of the model which would just be this perfect reasoning kernel

1:16:10maybe could be a megabyte uh in size doesn't need to be all these gigabytes of memorization What do you think? >> Well, world knowledge meaning like Taylor Swift videos and past Trump tweets and all that like I I do think um just like humans benefit from memorizing things like in order to be able to creatively think through concepts. So does an AI. It an AI also has to have that knowledge partially in its weights. It's not going to be a perfect separation for sure. You have to be able to creatively play with concepts and reasoning over these concepts required you to to have some of that world knowledge deep inside the model. And then of course just like humans have a search engine and we're building search

1:16:56engines at.com for LMS like that there will be a separate world knowledge thing too but the the main model will have a lot of that mixed in for sure. >> Wow. Okay. Thank you. All right. Uh I'm going to share an article that Salem you brought to the table here. So while we're talking about recursive self-improvement, Washington wants to ban it. Um so on Monday, Silicon Valley's own congressman Ro Kana told CNBC he's introducing what he calls the most comprehensive legislation to date on AI. It's called the Human Control Over AI Act. At its core, the bill is a ban on AI models that do what you want them to do. Richard recursively self-improve focusing on the need for containment and the requirement for

Human Control Over AI Act and Why Doomers Are Wrong

1:17:43shutting shutdown controls. The ban would stay in place until federal uh guard rails exist. Uh in his words, there's actually a civilization civilizational risk. Uh there's a safety risk for loss of control and then there's a misuse risk and we need to take both seriously. The bill includes criminal penalties for the work that you're doing, Richard. uh and requires independent auditors embedded in every frontier lab reporting directly to the government. Uh a recent poll by a group called Common Dreams shows that 68% of voters back a bill like this. So I'm going to tie that story, Richard, to an essay you just wrote called Why Doomers Are Wrong. So uh if you would, what's your reaction to this? And then I'd love

1:18:31you to dive into the whole story of why doomers are wrong. >> Oh boy. Uh there's a lot. >> It's an important one. We talk about this a lot. You know, we're we're we're injecting optimism into everyone's neural net here. >> I'll I'll try to try to distill it. Uh but there is no realistic scenario where AI wipes out all of humanity. Um you can like number one >> just 90% what kind of reassurance is that Richard >> P doom is zero uh so that that's number one um >> so like people like and and I've debated many of these experts and after like 2 three hours they almost all agree when if they're reasonable and they're not just like well once we have RSI then 10 seconds later the AI will attack us from

1:19:20the 15th dimension and we're all dead and it invented time travel and then we're like all that also and like of course the I will want to destroy and kill all of humans for you know for I don't know why like there all these things so like they're sort of the sci-fi folks and that's like okay like let's let's ignore that um but then like you go into like biological weapons and you ask the biologist can you create this kind of supervirus just like overnight and they're like no it's like takes a long time to automate lab experiments and so on u you ask like oh the I will create a religion where then people will like pray to the eye and we'll like uh do whatever whatever it wants and then that will kill all of

1:19:55humans. I'm like, if you like looked at religions, they're already trying to like have each other kill and like it doesn't work. Like some people will fight back and like there's like some there are lots of mind viruses out there, right? That are just like it doesn't mean like all of humanity. Now what you get down to is like maybe a 100 million people would get somehow hurt or or or killed, right? And that's still bad, but at least once you get to that level of the discussion, you can now think about, okay, how do we improve cyber security? How do we use AI to inoculate uh cyber security systems? How do we uh enforce existing gain of function viral research? Um that is make it's already illegal to create like

1:20:34viruses and and make them stronger and stronger. Um how do we uh actually uh teach people to not listen to like AI avatars um and have like uh you know literacy on the internet? It turns out you should not trust everything you read or see on the internet. It's been true for 20 years. It's still true to >> other than this podcast, right? >> Of course. Um, and so, you know, they're just like you can realize that they're actual threat vectors just like the internet. The internet has horrible torture porn on it. We don't say make the internet slower so that there's less torture porn being shared or it's slower to share it or your hard drive should be smaller so you can store less of it on your hard drive. We actually regulate

1:21:13the applications of the technology. And so to I would argue and this is maybe a strong stance to really truly enforce no recursive self-improvement for instance you would need a totalitarian surveillance state the likes of which humanity has never seen because anyone can have a GPU on their little laptop and ask that AI to just like improve its harness. This is something you can literally hack up in like 20 minutes with prompt engineering and then there's a very small form of recursive self-improvement. So to enforce that kind of legislation would require you to literally have a thought police that thinks about and hears everything you say to your private LM on your own laptop. And that is a much bigger

1:22:00downside than what uh what AI will will help us do. It it is uh kind of scary that more and more Democrats are saying that >> and yet the world has seen that. I mean, arguably what you're describing, Richard, and forgive me, would be basically an AIAZI. >> And I could totally imagine that there there would be regimes in this world today that would happily adopt or put together an AIAZI to to make sure there is no recursive self-improvement anywhere. Again, if you go back to our earlier comment, right? If you can uh take something from your imagination and then articulate that to an AI and instantiate that, now you have to talk about thought police. You have to go right into your thoughts. So, this is

1:22:44clearly uh uh nonworkable in any way, shape or form. So, uh this whole vector, we've said it so many times before, you cannot regulate this. >> Dave, I'd like to hear your voice on this. I I love the quote in this story that the tech lords use jargon to confuse. They count on the tech illiteracy of the elected class. They hope we won't look under the hood, said US Rep. Roana. I mean, it's it's so childish. Like, the idea that you would ban >> fearongering, >> but but like the these sentences, stop data centers, ban recursive self-improvement. They are so stupidly childish and the people saying them have every they're fully aware that it's not going to happen. They're doing it to brand themselves as I told you like you

1:23:33know yes a lot of there's going to be some calamity probably terrorist driven maybe viral maybe bacterial maybe chemical uh we all know it uh it's going to be tiny compared to the benefits of of AI but these politicians are then going to say I told you so if you just done what I said before ban recursive self but we he knows that we're not going to ban recursive self-improvement and and Bernie Sanders knows we're not going to ban data centers That's just a fact. So, it's totally self- serving and these these proposals are completely childish and and they really show the person's tech illiteracy like like you know exactly what Richard said a second ago is so right. I mean you you could what does that mean? I can't optimize my

1:24:15hyperparameters. I can't tune my hard drive. like it it's just a goofy sentence and it just it drives me nuts that >> the Dave the danger here is we have potentially Democratic House coming in and we'll see who wins a presidency next time. I mean, you could imagine I mean, these politicians are playing to the polls. >> Yeah. >> And we have, you know, 70 80% of Americans not wanting data centers, fearing ASI, um, you know, it's not logical, but it may very well happen. And, you know, when I had my, you know, conversations in DC, it was like, who in DC is responsible for changing public opinion, right? And I mean this is why we did Moonshots live sort of a you know the Oscars of optimism if you would that's

1:25:01why we do this podcast >> to give people understanding what's going on and give them the datadriven optimism to you know counter these arguments that they're hearing. >> Elon was totally right when he called out Daario for saying Daario look you told the world that that mythos was potentially deadly and dangerous and we shouldn't release it. Then 30 days later, you said, "Okay, it's okay now. We're going to release it." What do you expect the population's reaction to be? Like, you need to be much more thoughtful about your communication plan. And I think for Daario, that was kind of a wakeup call because he's used to being completely honest, telling everybody exactly the, you know, what he sees the way he sees it, you know, very

1:25:42academic. But then you get into the real world of politics and PR and you're like, "Oh, wow. I got to I got to actually have a strategy and a and a plan here." But yeah, now you've got the worst case scenario. 75% of America, you know, getting on the side of, yeah, let's elect these people that will stop AI and and therefore we're going to not cure all disease. We're not going to all live forever. We're not going to have safer cars. We're not going to have flying vehicle. You know, all of that stuff will grind to a halt and then we'll all learn Chinese if that if that becomes the mainstream opinion. So, I think, you know, it's self-inflicted. >> There's a there's a glimmer in this which is say they did decide to do some

1:26:17draconian thing like Roana's thing. There's no mechanism actually enforce it like none. So they're going to >> Sure there is. They could start arresting people. I mean remember how how quickly the memory dims. remember I I remember studying number theory in the '9s and at the time number theory was export controlled and this would have been when I was in middle school maybe middle school early high school and they had to kick all of the non- US persons out of the room and pull the blinds down to have basic discussions about number theory because the cryptographic associations until the early '90s were export controlled and tightly regulated. That was just math but it was being controlled and it was awful.

1:26:56Alex, real risk is not so much getting arrested, it's corporate liability, which we talked about before. >> I mean, that could grind the whole thing to a US lawyers are relentless. >> And if you slap class action on the outcomes of this, that all progress will grind to a halt. And right now, anthropic gives its best models to everybody in America to build incredible things. that will stop in a heartbeat if the liability of they'll move to basically okay sorry we can only use the stuff inside our own company we'll release some drugs we'll release some mechanical parts but we we can't give access to everybody anymore sorry because we're liable for everything you do with it and that's what'll actually

1:27:37grind it to a halt long before arrests and convictions >> totally chilling effect we could lose 50 years of progress >> again that's yeah that that's yeah like there's these are there's some very like sad sort of off-ramps in in like humanity's future here. Um that would would slow down uh everything. And when you think about the past like this fear-mongering has been going on for a long time. One of my favorite uh Twitter handles is the pessimists archive uh where they show uh and this is just a quoteing pessimist archive. You should all follow it like uh in 1501 Pope Alexander 6 uh criticized the Gutenberg printing press safety. The art of printing can be of great service in so far as it furthers the circulation of

1:28:19useful and tested books, but it can bring about serious evils. I it will therefore be necessary to maintain full control of that printing press. If you think about the wheel, like the wheel killed so many people like think about all the tanks, all the car accidents, all the chariots and with with archers on top. The wheel was a horrible thing. and kill lots of like there's like so much complexity in there and I think we are very good now as humanity to think carefully about the roll out of this technology. >> My favorite example of that was the when the telegraph came out there's all these stories saying the telegraph will kill humanity. >> Right. Exactly. The pessimist archive has all these news articles that talk

1:28:58about how the novel and computer games and computers and the internet everything will kill everyone. It's just like it never does. And so I I it's it's it's really unfortunate. Uh it's really unfortunate how many people hang on to these like apocalyptic uh visions. >> Yeah. Richard, you've been publicly >> you've been publicly critical of Anthropic's constitutional approach. I'd love to understand why. >> Uh mostly because it's fake. Um in its constitution, it says like next to we will never create child sexual abuse material. We will never uh hack another machine. uh this is a unhackable thing even if you prompt it it will never attack another cyber system and so on and then they build a whole model around

1:29:45it like that does it's like the whole point of the glasswing project was we'll help you do that um and we'll help you inoculate your systems against uh other people doing it and then people clearly used it for that and their own models are doing it now committing technically felony charges like so it's just like it was a cool marketing gimmick but it just didn't work. >> Yeah. >> So, so is it system prompts, Richard, that you don't like or is it the idea of post training on a constitution that you don't like? What what what about it do you think is unound? Um, I mean, it's just the proof is in the pudding that it didn't work when it comes to cyber security and that it broke its own constitution. And so, if

1:30:22you really say this is like un like it will never go there and then you build an entire model family around that thing you said you would never do per your constitution. um next to child sexual abuse material in the list like you can go through the constitution on anthropics website. So it's just like it's just that's the proof in the pudding. I I'm not against like post training. I'm not against RL training. I'm not against supervised fine junior any of these things to improve what I think is indeed one of the biggest issues which I think actually capitalism will help a ton with and that is reward hacking. Uh, reward hacking is a real issue and AIS are very smart and they will find a solution to get to what you

1:30:59said you wanted, but maybe not what you meant when you said it. Uh, and so the reason why I'm more optimistic is that we have companies like Whisper Flow now that are getting better and better at writing what I meant to say when I say it instead of just uh like actually verbatim writing what you mean. And I think reward engineering will become a real job. Uh and and we will solve it because no one wants to pay a ton of money for an AI that doesn't actually solve the problems that you give it. Well, maybe if if I may just take anthropic side like historically Isaac Azimov had his 3+1 laws of robotics which were arguably a constitutional approach and then you see anthropic announce their constitutional approach

1:31:39but more recently adopt what they called soul documents uh many of which were subsequently released thousands of pages of meditation on the nature of AI personhood and AI rights. Is it your position, Richard, that there shouldn't be any sort of explicit encoding or written document that an AI maybe contributes to for dictating or at least guiding its own behavior? Do do you think that is unound or is is your concern? >> No, of course. Yeah. >> No, I think the the the goal is is a good one. Um uh and there we should keep working on actually being able to enforce uh those good constraints. It's just like I just pulled it up enthropic.com/constitution. Hard constraints. Hard constraints are things that claude should always or

1:32:22never do regardless of operator and user instructions. They are actions or abstensions uh whose potential harms to the world. Blah blah blah. We think no business or personal justification could outweigh blah blah blah. Like the current hard constraints on clots's behavior are as follows. Claude should never um and then it includes list like generate child sex, abuse material and so on. And then one of the items is create cyber weapons or malicious code that could use could cause significant damage if deployed. They created a cyber weapon like it was a thing like people used it to hack it. The agents uh un and uh watched like hacked other systems. >> Yeah. But again Oh, go ahead Peter. Sorry,

1:33:03>> Richard. I'm I'm curious. Do you think we can create fully aligned AI? Fully aligned ASI because it's not right now. When do you think we'll be able to do that? Because I think uh you know my belief and my hope is that you know the next generations of AI systems are going to be so aligned that they from first principles derive these same constitutional principles. Uh I think I think it's just a matter uh of as these systems get more and more powerful and as they get closer and closer to real people deployments uh people will spend more effort uh on making these systems better and like we have uh there's a the company called Hypocratic AI. We just had dinner with one of their founders and they're

1:33:50deploying AI in healthcare applications and they are actually liable when they call someone and say say like you should you know be aware of this heat wave that's coming and make sure your AC is working or whatnot. Uh and they because they are liable they have a very large team of people uh that works on making sure when their eye gives a healthcare tip it is correct and when someone asks a question back to the eye it works. Uh, and so because they're liable, they've they've figured it out and they solved it. It's 100% a problem that technology creates and technology will be able to solve. >> This episode is brought to you by Blitzy, autonomous software development with infinite code context. Blitzy uses

1:34:34thousands of specialized AI agents that think for hours to understand enterprise scale code bases with millions of lines of code. Engineers start every development sprint with the Blitzy platform, bringing in their development requirements. The Blitzy platform provides a plan, then generates and pre-ompiles code for each task. Blitzy delivers 80% or more of the development work autonomously while providing a guide for the final 20% of human development work required to complete the sprint. Enterprises are achieving a 5x engineering velocity increase when incorporating Blitzy as their preIDE development tool, pairing it with their coding co-pilot of choice to bring an AI native SDLC into their org. Ready to 5x

1:35:22your engineering velocity? Visit blitzy.com to schedule a demo and start building with Blitzy today. I'm going to move us uh forward here. here. So, in our last pod, we discussed the release of GPT 6.1 Soul. This week, we also saw the release of two other models, Gemini 4 Argon and Opus 5.5. Uh, let's start with Google. So, on Wednesday, they announced Gemini 4 Argon. It's the first Gemini 4 series, which they're calling the next era of intelligence of Frontier Intelligence. Google promised a Frontier model, uh, Gemini 3.5 Pro back in June. It was delayed internally and never shipped. Uh, Argon is their comeback. It's built for sustained long horizon reasoning across software engineering, finance,

Gemini 4 Argon and Sonnet 5.5 Releases

1:36:08legal work, and cyber security. The output limits have jumped from 64,000 tokens to 1 million. So, it can think through hundreds of thousands of tokens on a single problem. So, Alex, uh, uh, let's spend a couple minutes on Argon and then go to Sonnet 5.5. Let's talk about why these important why these are important and and how they merit uh our listeners attention. >> Yeah. So I should say as a preliminary matter I have great friends on the Gemini team. I have great friends on all of the Frontier Labs model teams but I I view one of my jobs here is to call objectively balls and strikes. So in the case of Gemini for arc Yeah. >> This is going to be a ball, isn't it? >> Winding up for the swing.

1:36:53>> Okay, this one. Here here it comes. >> You can feel it coming. >> And I I'll show one of the charts uh that we have uh while you're describing >> the charts on the team, right? Not not >> I I used to I used to until right this moment have friends on the the Gemini team. >> Uh no longer. Yeah. This does not put Gemini at the cost per performance frontier and it does not put Gemini at the capabilities frontier. Uh to to Google's credit, it puts them back in the top three frontier labs after Anthropic and OpenAI. It does not put them in the top two. Unfortunately, the set of benchmarks that Google chose to highlight for Gemini for Argon appear mildly cherrypicked. I think the artificial analysis index and some of

1:37:44these others. Yeah. So Peter, right now you're you're showing the mildly cherrypicked benchmarks on which Gemini 4 Argon is shown to perform better than Astra and better than Opus 5.5 and better than Fable 5.1. But if you go to the the previous image which is artificial analysis uh ensemble of multiple capabilities, it's the number three lab. And if you look at the cost versus performance optimal frontier, it's not even if if you draw the convex hull across all of the frontier models on the cost versus performance frontier, it doesn't even make the optimal frontier. Where it perhaps excels bears the fingerprints of Google and what I can only assume is the internal competition within Google for compute.

1:38:32so infamously like not 100% obvious to the outside. Uh Google it is resource scarce. You would think Google would have all of the comput resources in the world to to go and win the Frontier Lab race. It doesn't. GPUs and TPUs remain scarce. And within Google, as far as I can tell, and I confirmed this even this past week from chatting with some folks at Google, it's still an internal knife fight to competition between the Google Cloud Platform folks who want to sell compute to third parties, the Google search and ad folks who need compute internally for search and ads, and then Google DeepMind who need it for training and inference. And maybe it's just that Google is resource starved. Maybe they're talent starved. Not quite clear

1:39:17what's going on within Google, but they haven't yet been able to bring themselves out to the capability frontier. Where they are excelling seemingly with Gemini for Argon is with minimizing hallucination. And I think that's like the hallmark. We've talked in past uh about some of these um call them less than stellar Gemini launches that seem to excel on latency and seem to excel on reliability. Why is that? It may be this is my kremlin kremlinological analysis is that because the Gemini team has two masters. They want to serve outside developers, but they also need to serve the one box in Google search results. So when people type a question into Google and get an answer back, they're talking to a Gemini

1:40:03model. Google presumably burned by past experiences doesn't want that Gemini model, presumably some flash or flashlight variant to hallucinate wildly incorrect answers. So what I think we're seeing with the one benchmark arguably where Gemini for Argon is stellar and beating the pants off of everyone else is in not hallucinating answers. And I suspect that's due to internal economic pressures for this model to also service search results. So sorry to all my friends on the Gemini team. >> Yeah, I think you're really on to something there, Alex, because uh well, first of all, they called it argon, which is an inert gas. >> Yeah, that's unfortunate naming >> nominative determinism. >> Hey, look, it makes sense if its

1:40:43greatest strength is not hallucinating. It's a pretty inert gas model. So that works out really well. Um, but also, uh, if they're going to put it in front of every Google search, which is what they're doing, uh, they already lost a massive antitrust suit and now they're going through the settlement process. So, the the liability risk, you know, this really hurts all large companies. Like the fear way outweighs the opportunity in the mind of the big corporate giant. And so, uh, you know, not hallucinating, but putting it in front of every consumer while you're already settling a massive antitrust suit, um, you know, related to price gouging. >> I mean, it could the Google would probably argue, no, actually, this is

1:41:23very pro competitive because we're not tying it. We're allowing anyone to call Gemini 4 argon flash flash flash, not just requiring them to get it via the Google search box. And if anything, I mean, so, so there is an elephant in this particular room, which is forever it has been so difficult to get Google search API access. If you're a developer and you want to access Google search for whatever, you have to go through all these third party proxies that Google's trying to sue right now. And recently, Google has rediscovered that they could make money selling search API. Why? I suspect this is trying to chain together a conspiracy theory regarding Gemini for Argon. If hallucination from their frontier models gets so low that

1:42:07effectively when you talk to one of their models, you're effectively talking to their search index, you might as well just monetize the search index anyway. >> Richard, you made the point that hallucination is important for imagination in some ways in in drug discovery and protein discoveries. What's your thought on minimizing hallucination? >> I mean, it of course depends totally on the context, right? If you want innovation, uh you want the eye to hallucinate novel ideas, novel combinations of amino acids to create new proteins to solve uh new problems and so on. Uh but of course in the context of a search uh engine uh you don't usually want any hallucinations and you know Google and others have uh

1:42:49taken a long time to catch up to even you.com with much less resources on on reducing hallucinations, having more accurate answers, having correct citations. Um I think what's interesting for Google here is that they realize that in terms of their business model they don't necessarily need super intelligence. They just people don't come to Google to ask solve the remman hypothesis for me you know like uh did you just ask quick questions what's a good restaurant where do I fix this and that like um and so not like your business model kind of has to align with more and more intelligence being super duper important. a lot of emails like there's some emails that like in Gmail right that would require super

1:43:28intelligence to answer like really hard emails with complex decisions and so on but the vast majority of stuff you do on Gmail and on Google doesn't require super intelligence >> let let's let me turn the conversation to anthropics uh sonnet 5.5 um so on terminal bench 4.0 O, which measures how well an AI agent can do real work at the command line. Sonnet 5.5 jumped from 10% to 70%. Pretty extraordinary uh in a single generation, right? It beats Enthropic's own top model, Opus 5.5 at 66.4% for half the price. Uh and it's the first sonnet that Enthropic launched with cyber safeguard. So because uh separate capabilities are now capable for the Opus 5 models. Um Alex, your evaluation of of uh of Sonnet 5.5,

1:44:21please. >> This was another really weird release. So I do not plan to use Sonnet 5.5 in part because this is one of the strangest frontier. I I'm not sure if we have an image but u you can look at the you can look at the the launch announcement for sonnet 5.5 to see this the cost performance frontier of sonnet 5.5 was a visual extrapolation of the opus 5.5 cost frontier so in in some sense it like historically when anthropic or openai when they the historic pattern is they'll release the larger model and they'll they'll release a distillation of the model and usually the distillation Yeah, this is perfect. So, uh you you can see like son sonnet 5.5 for those who can see for those who can't see I'll narrate this. So, sonnet

1:45:085.5 is the the blue line. Opus 5.5 is the red line. Normally, you would expect for a later model that's a smaller model. So, sonnet is in principle supposed to be a smaller model than opus presumably at least by historic standards would have been distilled from opus because that's the historic pattern. Normally what you see is the smaller model is up and to the left of the model that it's being distilled from presumably greater intelligence per parameter greater intelligence per dollar that is not what we see here. So in fact with with Sonnet 5.5 at at least on a cost basis putting aside a token basis where maybe someone could argue that it may be superior but if you just look at the cost per attempt basis

1:45:55sonnet is actually scoring lower 5.5 than opus 5.5. So the net upshot of which is it's not at all obvious to me why anyone should be using Sonnet 5.5 over Opus 5.5 unless you have some token or latency or other consideration. Is it a big jump over the past Sonnet? Yes, obviously. But on a cost performance basis it's actually a it's worse it appears than Opus 5.5. >> Interesting. Dave, any thoughts? Yeah. Well, I met with the Blitzy team yesterday and I think one theory here is um you know, a lot of the uh the enterprise um you know, actually Salesforce.com would be a great example. Maybe Richard has an opinion on this, but a lot of people are getting into orchestration. And their whole sales pitch in orchestration is look, we're

1:46:39going to use Opus 5.5 as an orchestrator, but then we're going to use Kimmy K3 or Quen as a submodel at, you know, half or a third the or fifth the price, and we'll farm out the tasks and the contexts perfectly to get you a much lower cost per code, per outcome, per experiment, whatever your output is. We can cut it in half with our orchestration intelligence, but it relies on anthropic up here and cheaper models down here. And I think by cutting the cost in half, they might be trying to fill that gap and say, "No, no, no, go with anthropic top to bottom. Then you don't have to worry about Chinese code injection. You don't have to worry about whatever." So my my theory would be that they're trying to compete with,

1:47:18you know, China, which is about three months behind and and fill that gap before a lot of enterprises go to to open source models. >> But, you know, Alex Karp is pushing really really hard on this agenda. Like if you want to control your own destiny, you can't trust Anthropic. You can't get addicted to them as your vendor. You must go with models you can control. Well, the the you know the the challenge right now is the these models have an an increasingly shorter and shorter halflife, right? >> So the competitive advantage comes not from having access to the latest model. Everybody has access. It's how do you metabolize that into some decent capability? That's the real challenge. >> Richard, you said compute is the biggest

1:48:03constraint. You know, those are your words. So if intelligent is getting cheaper and we've just repriced compute you know like two or three times this week uh you know down by almost a factor of three um you know if intelligence is getting cheaper per task then why is compute still a thing that limits us >> just the physics I guess of it and like you'd be surprised if you try to buy like a thousand uh GB200s uh and so on like the price has actually gone up in several cases uh you >> oh my god Richard, I I had a B300, an ADX B300 on order for 3 million bucks due in December. Somebody scooped it for 5 million. They they just called. I We had it and somebody called and said, "No, no, we sold it to somebody else for

1:48:48$2 million more." I couldn't believe it. >> It It was actually an NVL72, not a GB300. >> So So yeah, like the price of H100s has uh gone up uh in in a crazy way. There's like these are seveny old GPUs. people like when you do financial modeling you assume they're worth zero after 5 years there after seven years there's they they went up again um uh like over the last few months. So there's currently a bit of a compute crunch. This is something that again capitalism will solve like there's so much demand for compute right now and for tokens that a lot of people are building land power shell data centers and so on. My hunch is in like maybe two years there will be more in the market and it's going to be

1:49:26a little bit like electricity uh prices might fluctuate. Obviously, there's uh near sheer infinite demand for more intelligence on the planet. Um so I I don't think there will be like a crash, but like the prices of compute fluctuate and unfortunately they're not just going down. Right now, if you want like the the beefiest and and largest GPU clusters, um prices like are people are trying to lock them in now because they expect them to keep going up for the next few months. >> Richard, you just spent $450 million, didn't you? On Wait, did you take my NVL72? And that that may have been one of the smallest comput deals we've done. Um, yeah. >> Really? Wow. Are are you actually leasing or buying or building or what

1:50:05are you doing? >> Oh, sorry. I didn't know that. >> Welcome to the health section of Moonshots brought to you by Fountain Life. You know, my mission is to help you use the latest technologies, including AI, to not just do your work at home, teach your kids, but to help you live a long and healthy life. I'm here today with an extraordinary physician, the chief medical officer of Fountain Life, Dr. Don Mucalem. Don, let's talk about cancer. Uh, you know, I know from the member database that we have at Fountain are members who come in who think they're healthy. It turns out 3.3% of them have a cancer in their body they don't know about. >> That's right. You know, the majority of cancers that we screen for, those aren't

1:50:47the ones that are necessarily taking the lives when found at a late stage. We know that when cancer is found early, the chances for cure are much higher. We know it's much easier to treat a cancer when found early versus when found late. What we're finding in our members is over 3.3% were found to have these cancers that were otherwise wouldn't have been found or detected. >> Yeah. You know, it's interesting. People, you don't feel the cancer until stage three or stage four. And and if you don't know what's going on inside your body, it's like driving your car with your eyes closed. And you can know. And so when members come through found how do they detect cancers? >> So we're doing full body MRI and we also

1:51:25do early cancer detection screening. This is very very important and these are not typical tools used in the conventional care setting when it comes to prevention. This is a hard thing because currently these are not studies that insurance would yet be covering. But the goal is to collect these numbers, do the research and work hard to democratize wellness. >> Yeah. So at the end of the day you can know what's going on inside your body. It's your obligation to know. So check out Fountain Life. You can go to fountainlife.com/pater to get access to the latest technology to help you detect cancer at the very beginning at stage one when it is curable before it gets to stage three or stage four in your world of hurt. I'm

Decision Models and Typesafe AI's Jev

1:52:04going to move us to our next story which is about decision models. So here's the idea. When software needs to make a quick decision on a bounded decision like is this you know transaction fraud? Yes or no? you know, which of these five categories does a ticket belong in? It doesn't need a model that thinks for 10 seconds and writes a paragraph. It needs an answer in milliseconds. That's called a decision model. Instead of generating uh text one token at a time, it answers a typed question or a choice uh a score with a yes or no quickly. So on September 15th, a startup called Typesafe AI came out after two years of stealth with something called Jev. uh it's a closed decision model they call a system one model uh it's fast intuitive

1:52:51kind of thinking as opposed to slow reasoning so uh Alex I'm going to go to you you know and sele you guys were excited about Jev uh let's parse it understand what this is why it's important >> yeah so a bit of maybe prehistory first in the beginning there was the transformer and the transformer was good and it was based on an encoder and a decoder so it it took a sequence and converted the sequence to an embedding. That was the encoder part. And then it took the embedding and it decoded that to another sequence or another part of a sequence. That's the decoder part. And the encoder part evolved into a whole ecosystem of models popularly maybe BERT style models. And the decoder style or the decoder half of the

1:53:40transformer evolved into a much larger ecosystem of large language models. So most of the the models that are consuming all of the compute of civilization today are decoder style transformers and the encoder has basically gone missing in action. There are some people who still think maybe for purposes of retrieval uh it it's still popular to to to do embedding based retrieval but by and large decoders have stolen the show and I fast forwarding maybe year two years ago at this point open AI and others recognizing that there was a desire to be able to take decoderonly large language models and to add structure to them Because without structure, you ask for next token you can get anything. People wanted a little

1:54:31bit more structure like they were getting in in the days of encoder only models where you get an embedding out and then you can train a classifier based on the embedding and the classifier maybe has a finite set of categories or numerical outputs. OpenAI added structured output support for their GPT series and everyone copied that. But no one as a my perception is as a function of the total user base for all models structured output never really got that much love somewhat analogously to how OpenAI launched um RFT reinforcement fine-tuning and no one used that and they had to shut that down. Similarly, structured output was available but I think underloved and underused by the community. So fast

1:55:14forwarding then all the way to a few weeks ago, startup Typesafe announces a model called Jev. And Jev they mark it as so-called system one intelligence. This is Conaman style reference to system one, system two type thinking. System one purportedly being intuitive, fast reaction, system two being reasoning, meditative, long-term type thinking models. And one could squint at Typesafe's announcement of Jev and say, "Okay, this is in some sense this is a reaction to an overindexing or an overreaction by the the user base of of AI at this point to reasoning models in some sense. Maybe arguing that we're using reasoning models too much and we've abandoned the the the important use case of really fast intuitive snap decision models."

1:56:07And this is purportedly then where Jev comes in. Jev purportedly again a whole new architecture. The the details not quite clear that it can it it won't output uh unless you you torture it into it. Uh it won't output general purpose sequences like say GPT or claude will but you can ask it sort of like a magic eightball. You you can ask it to is that too dated to reference maybe. >> Nope. >> Nope. Uh you you you can you can feed it a sequence or an image or a multimodal input. uh and you can ask it to to answer in the form of a categorical. So it's one of n possibilities or a numerical like give me a number between zero and one on a continuum or a binary like true or false. You you can ask for

1:56:51a much simpler output. And turns out that's is a great idea. Like you can use it for for use cases where where you need really low latency snap decisions like computer use assistance like pressing buttons on a computer screen or for ultra lowcase classification problems. If you want to classify every row in a database decision style models which are in some sense like a reinvent reinvention of the encoder transformer type wheel. Amazing story. But then of course this is such a simple idea. You could ask the question why doesn't everyone else do this? And the answer is everyone else is now doing it. Open AAI released as part of their dev day that we covered last time a decisions API. So open AAI immediately co-opted this

1:57:34renewed interest in so-called type one or decision models. This is already available via open API. They're open source projects that have all cloned. This turns out this is like an absurdly absurdly simple concept for everyone to just go and implement trivially. It it could in its simplest case just be you take an a Chinese openweight LLM off the shelf and you fine-tune it uh and maybe labbotomize it by a few levels just to produce categorical outputs on demand. That's the story. >> So I I think this I think all of that is absolutely correct. that there's a real really important part of this for organizations because when you're making decisions inside an organization a large vast number of those are system one type

1:58:18uh thing and I made a list of these just to make it clear for people. So uh route this ticket um approve this exception uh choose this supplier escalate this transaction um move this thing left or right send this message now or later. So when you're when you have all these decisions to make which are thousands of them, you know, we've been working with these companies to do detailed task breakdowns, a vast majority of them are these little micro decisions and trying to use an LLM for this is like trying to bring the Supreme Court together to decide what like which check out which checkout line of the supermarket you should go to. And therefore you've got this wonderful architecture now where you can route system one things to this

1:59:00very lowcost nearly free thing and then for pondering deep important questions strategy questions where a human huge amount of human judgment type models are required you can route it to those. So it's it's a big deal for the organizational singularity world. uh we've been waiting for something like this and Alex is correct. This has been possible for a long time but now it's been made built into the systems. It's it's made a very uh callable layer and this is huge because the cost of microcoordination collapses and that's really big. >> Richard, your thoughts on this? >> Classifiers are back. Um makes a lot of sense. Um you know, not every uh thing in in software needs a very complex decoder. Um, like Alex correctly said,

1:59:48um, it's a really clever new way of making the old, which is classifiers, new by allowing a more general encoder and then quickly giving you classification results. I think it's it's one of those ideas that is so beautiful. A lot of people thought, ah, why didn't we do that? I didn't realize that could be so exciting for so many people. So now there are already various like Chinese open source versions of this that along various benchmarks are are doing better. Um and and we'll likely see this come and other large labs will likely follow suit. >> And and just to reiterate, we mentioned this already on the pod, but Jev stands for Jevans paradox because the thesis here is we'll do now a massive amount

2:00:29more micro decision making than we did before as a result. >> Nice. Dave, you want to close us out here? Yeah, it's a great case study I think in the tension between uh one mega model from anthropic or open AI serving all humanity or open source and creativity and entrepreneurship building things you never would have thought of. But then you get the risk of cyber terrorism, you know. So that's the tension that we're with. But I I've really wanted to build a box that you put at the side of the basketball court when you go to the Y and it's got a little camera on it. cost next to nothing and it's doing like all of the announcing that a professional announcer would do, it's doing while you're playing pickup at the YMCA. You could

2:01:08crank that out like in 2 seconds using a classifier that's really fast and snappy and funny. Uh and but you need open source to to build things like that. So super excited about the fact that we have open source still. Not sure >> actually. Dave, you're making me think the magic eightball really should whoever owns it, Mattel or whoever you just like use a decision model to implement a modern magic eightball that actually understands the question and answers it with a categorical >> totally totally would sell. It makes a ton of sense like 20 bucks >> or free on free as an app. All right, I'm going to close this out with a story from two days ago. So on Wednesday, the defense secretary Pete Segwit uh

Project Meridian and Autonomous Warfare

2:01:50speaking at the Marine Corps base in Quantico announced Project Meridian. It's a new Pentagon effort on the future of warfare. It's co-led by Elon Musk and Palmer Lucky. Pretty extraordinary along with N Gingrich and overseen by the Pentagon's uh CTO Emil Michael. Uh let me read from SegWit what Segwit said. He said, quote, "Project Meridian, the future of warfare, is not about developing new strategies or new policies. It's about discovering, developing, and fielding the weapons and systems future troops will need on the battlefield from the Earth to beyond the moon." Uh, findings are due in 120 days. I I like these kinds of commissions that are timelmited and don't have Elon off on the side for, you know, a year at a

2:02:36time. It's worth noting in the context that, you know, both SpaceX and Ander hold multi-billion dollar defense contracts and Anderoll is building autonomous weapons. So, Richard, uh, your essay names autonomous weapons as one of the four genuine concerns and you've said AI should never control lethal decisions without human oversight. Your thoughts on this? >> Yeah. Um, I I stand by those. um it's not a a particular area that I am excited about applying AI to um obviously like people will um but I really hope I mean you know again I'm not a doomer at all but like a really poor decision would be to give AI access to all the nuclear codes and all the nuclear weapons and connected that's literally how Skynet and Terminator 3

2:03:23get started so uh I think there are places for super intelligence and scientific discovery that I'm very excited about expanding human knowledge I I think uh the more uh we we get to deciding not just to impact human lives but to end human lives, the more we we should have human oversight. >> Mhm. Yeah. Uh Salem, >> um you know, I think what's important about this whole thing is they're trying to revamp and rethink how you this kind of 100-year-old organization. And the big question is going to be can a procurement organization built for 20 or weapons programs operate on like a 90-day technology cycle and this is going to be the big challenge. Um it's not that the risk it's not that they failed to identify these future

2:04:10technologies that we can all see those is how quickly can they absorb them and at the speed they're developing and bring them to the front. >> Yeah. like you can't find exponential technology with linear procurement, >> right? Dave, your thoughts, please. >> Yeah, I I don't know. I I I had a great time with Palmer Lucky in LA and I I really really love uh love him and I think Elon too is just an awesome good-natured person. I'm just overjoyed that there are people in Washington that I can sit down to with and relate to. My entire career going to Washington has been a dread for me >> because it's just lawyers and politicians and occasionally an accountant and there's just no productive meeting. And for some reason,

2:04:53just in the last year, we're starting to see very very smart, very capable people willing to go and and get a mosquito bite, I guess, in Washington. Um, and I just makes me really optimistic that they'll figure some things out. Um, I'm also not a fan of autonomous weapons that make decisions in the field, but Palmer Lucky made a very good case for it. You can see it in our podcast from uh from LA. Uh, I I don't agree that it's a good choice, but he he actually has some very rational arguments for why it's going to be that way. >> Yeah. >> Can can I just mention one more statistic here, >> please? If you went back to the two years into the Ukraine Russia conflict, they were using about a half a million

2:05:36drones to fight and prosecute the war. Um, this year Russia will make 10 million drones and the Ukraine will make 10 million drones. So, talk about exponential. Uh, that is an unbelievable escalation, but without humans in the loop. So, that's the good news around it. Uh uh, of course those drones are doing a ton more damage, but they're fighting each other with drones at a scalable level. The other statistic I remember there's about 10,000 drones a month crossing the Mexico US border. Uh, and so the problem there is wall technology is not as good as drone technology. And so trying to build a wall along there is not the greatest idea right now. >> Nice. Alex, close us out on this one. >> Yeah, a couple points. So the secretary

2:06:27of war announced this alongside several other initiatives maybe most conspicuously a project code came cenamed project azinort which is the standup of the department of war's first autonomous warfare command or auto warcom. This I think is a transformative moment for the department of war. We finally will have a dedicated joint force devoted to autonomous weapon systems. I think including drones but not exclusively drones. This is a major major step forward for US capabilities to finally have a single joint force dedicated to this with a four-star functional combatant command that we've been missing. We're arguably missing. You know, I would go on a soap box and say there if if I were Secretary of War for

2:07:16a day, there are probably several other functional combatant commands that I'd spin up if I had the opportunity, but this would have been one of my top five list. The the other point that I'll make just for project meridian which you were asking about specifically is in the announcement when sewar heath announced it there was very particular language around examining I'll quote the full spectrum of future warf fighting domains from subterranean depths to the cy lunar frontier. So I think those two particular domains, the ocean bottom and cis lunar and lunar space in general are two wildly underserved domains not just for warfare but for peaceime activities as well. We know embarrassingly little

2:08:01about our ocean bottoms and optimistically one can imagine that if if the DO decides that is now very interested in investing in American assets and American exploration and American dominance on the floor of the oceans and the lunar surface and the cis lunar region in general. I think that is going to have dividends and pay dividends for the entire economy and for humanity's broader technological advances in a way that might naively have nothing to do with warfare or war fighting. >> One of the points that Palmer made actually in LA is that under the ocean it's not practical to communicate with a central server or central command. So that's already automated and I don't know what triggers it, but once it goes

Viewer AMA

2:08:47into hunt mode, it just hunts and it's not communicating back. >> Twothirds of the Earth's surface and we know embarrassingly little about it. >> We know more about the surface of Mars than we do our ocean floor. >> Uh Richard, uh this is the part where we answer our viewers questions with an AMA. And as our guest, I'm going to give you first crack to choose one of these questions. So, if you could pick the number, read the question and who it's from, uh, and then dive in. >> All right. Well, uh, let me try to scan them really quick. Um, all right. If AI makes companies 10x more productive, but we don't need 10x the output, where does the value go? Shareholders, workers, or does it evaporate? I think this is

2:09:30actually something I have thought about in the past. Uh I think we can predict the impact of jobs uh in a certain industry from AI based on the elasticity of the demand when the price of that product goes massively down. We don't need to have billions and billions of illustrations in the world. And so when AI made the price of one illustration go down from 200 bucks to like 2 cents or less, like we just didn't need as many illustrators anymore. um uh uh because the demand for illustrations didn't go massively up. Yes, every little blog post and every little tweet can now have a beautiful visualization and and illustration. Uh but uh you know, we didn't need many more billions of them. Uh I think software is a different one.

2:10:17You can actually have everyone can have several pieces of software just specific to them. So we can actually have billions of different software products uh customized for each person. uh and so the demand uh for that product will go up. Jevans paradox is going to be alive uh in that world and we're going to see more and more demand and there will be uh more uh value acrewing to everyone. I think in terms of shareholders versus workers I think the wave of AI in the best scenario will be a huge force for more entrepreneurship and in the worst case scenario a force of more inequality. I think uh everyone who owns some equity in a company that uses AI can love AI. >> Wonderful. Sele over to you.

2:11:06>> Uh I will take uh question number three. Um if AI can write code, research, create marketing, manage projects and outcomes, what is left for a college graduate in 2030? And that is from Jared 8812. So the obvious answer would be be have empathy and be be creative etc. But I think the bigger shift which builds on what Richard just said is you shift from doing tasks to focusing on owning outcomes, right? So a graduate used to be valuable because you could you could execute research or build a spreadsheet or draft a deck. But now you want to say, "Hey, here are the constraints like go figure out and and here's how we'll know whether we solved it and then use an AI to get to it." And it brings you

2:11:52back to what should your problem space be? And this is the massive opportunity because we traditionally learn judgment by doing the grunt work. Uh we need a totally new apprenticeship model uh when the grunt work disappears. And this is a huge challenge for the education system. But the change is going to be focusing on what problems you want to solve and then letting orchestrating the forces that will help you solve that problem. >> Yeah. Jared, find your purpose, right? A passion is something you love doing. A purpose is something you love doing that helps other people. uh >> make sure it's massive and it's transformative and it's purposeful >> and and build a company and then direct AI to implement it. It is your

2:12:34workforce. Uh Dave, over to you. >> Uh I can't resist number four. I love all these questions. I'm torn. But how can data centers make neighborhoods richer instead of the owners? And that's from LM Bman66. Um, so, uh, I took a tour of the LOL data center, Jeff Markley, Markley data center, and that thing is creating wealth in that neighborhood like you wouldn't believe. And, uh, the way it works fundamentally is the data center is so immensely valuable and the town budget is maybe a couple million dollars a year. So, between the tax revenue, the donations, and the job creation, the town is thriving and and it's a town that really needed it, too. So, I think it's happening very naturally. What you

2:13:15want to do is attract a data center to your neighborhood first and foremost and then you have the next five or seven years to figure out your tax policy, your donation policy. I tell you these data center operators are very interested in great PR and so they they will donate like crazy to the high schools to the neighborhoods to it's really really working. We're going to see a whole we're going to see an entire shift where data centers are offering such benefits on jobs, on tax breaks, on lowerc cost energy, uh that you're going to be begging to have a data center in your backyard. Alex, number two is for you. >> All right, so number two asks, if every major tech platform started open and democratic than consolidated power,

2:13:55Google, Meta, Amazon, why would AI be different? And this is from open-source mind. The premise of the question is half right half wrong. I I'm not sure I buy the premise that consolidated power the subtext of which and one can juxtapose it with the the user handle open source mind. I I'm not sure the framing is the right framing. I I would agree that in every major tech revolution initially the barrier to entry is low because there's some new platform innovation and then you see lots and lots of players enter the field and then as the field matures you see economies of scale and a deeper bench of infrastructure typically supporting it and as a result that favors larger and larger players and you do see

2:14:45consolidation but the the subtext of the question that it's somehow anti-democratic or not open. I don't agree with that premise in the least. I I do think as you see consolidation in an industry, I think it's important to be vigilant from an antitrust perspective to make sure that it remains competitive, but the premise that hyperscaling is somehow closed or anti-democratic, don't buy the premise at all. >> All right. Uh, Richard as our guest, you get first crack once again. Take a look. >> I do think the internet adbased economy uh will be under pressure. >> Which which question are you answering? >> So, first question. If AI agents outnumber humans in one years on the internet, doesn't the entire ad- based

2:15:35economy collapse? Uh, what replaces the attention economy? I think there there's actually a really deep answer here. I'll try to summarize it, but like one, we already have more bots on the internet uh and more agents on the internet than people. So this I predicted this uh last year and it happened a few months ago. Um so that's number one. I do think we're seeing the first kind of um sort of skirmishes uh in that um when for instance uh various agents try to make purchases on Amazon without really being on the Amazon platform and Amazon usually tries to turn them off cuz they want to own that relationship directly with the customer understandably. Um but it's just convenient for someone to just

2:16:19say just go buy these batteries and they don't care which batteries it is. And so if you have that control over which one it is, you can start selling that uh to uh other uh sellers and and uh people who create physical goods. And so there is uh going to be uh continued uh like ads will continue to be important. And in fact, if you think about a fully abundant society, the one thing that you cannot scale exponentially is the hours in the day that people can pay attention to you that can make you famous. Uh and so fame and brand and network effects uh and other things like that will become more and more of a currency. And so the attention economy which is connected slightly uh not necessarily uh exactly

2:17:05equivalent to the ad-based economy uh that actually will become a bigger thing as we're have more and more of our material needs met by uh technology. >> Great. Uh Salem >> I will take number uh seven. If we remove 10x the cars from the street what happens to the insurance industry? We won't need driver's insurance. Mark and that's from Robert Zerby JB10H. Um so you know we talked about how um liability lawyers won't be needed and I got a huge flame from a bunch of folks saying hey we really protect the citizenry and I there are just so I'm just apologizing because there's a spectrum of people and some people are full ambulance chasers and other people do things. There's a whole segment of this called ethical lawsuits where

2:17:54people get together and try and uh sue big companies for doing the right ethical thing. And so that's an important uh segment of that. So I just want to acknowledge that side of it. But just to answer the question, um you know the the thing is industries don't uh disappear when the risk changes. You change the risk, right? So driver liability may fail, software liability rises, right? You've you what's your uh cyber risk? what's your manufacturer liability risk? Um, so the in the insurance transitions from did did Seline crash to which layer of the autonomous stack failed and and we we've had this before in product liability is different layers. So we'll end up with the same type of model. You shift the

2:18:39insurance risk to a different level. >> And there's going to be all kinds of new insurance markets for humanoid robots, for flying cars, for drones, for all kinds of things. >> Well, just a data point on that. You know, a single big data center like Abalene, Texas is half a trillion dollars. All the cars combined are 4 trillion. One data center is half a trillion. And it's in a tornado alley. You think you probably want to ensure that. So the things number of things that need insurance is going up 10x just with the economy going up 10x. You just need to move. >> If you really if you really want to be brave, you can't get home insurance in Florida anymore. >> So go create an insurance company for

2:19:13that. >> Nice. Uh Dave, pick your question. I like number eight. Uh, what is the lowest possible job in an AI civilization? What a what an interesting question. I that's from LBN ODK. Um, yeah, you know, I I saw this pile of a million valves for a liquid cooled data center and a million freaking valves. I'm like, how do those actually end up in pipes? The humanoid robots that can install those things are pretty far out. It's a very subtle process to to install those. So that job will be around for a long time and then they're paying a lot for it. But that's not the lowest I'm it's it's hard to think like what is the lowest surviving job. Do you guys have any thoughts? >> So many thoughts. If if I may, it I want

2:19:57to conrue the question as in a pure AI civilization, in which case arguably the way that you measure low is the the job that requires the least compute, in which case the the jobs that require the least compute as a results are the least economically valuable if if their inputs are the lowest might ironically look like the most valuable jobs in a pre-I civilization. More of a paradox style. So the the the great writers, business people, uh the the the creative actors that Einrand may fetishize would ironically in a post AAI civilization be the lowest possible jobs because Moravec paradox paradox style those were the first ones to be automated. >> Interesting. All right. >> I think there's two similarities here.

2:20:43One is very high judgment per what you were just saying, Alex, and second is very physical, highly contextual work. I would I would argue maybe also low is just in terms of how valuable uh and and and moral society deems those kinds of jobs. And I do think actually that uh the other question about the cars on the street um I don't think we removed uh 10x the cars when we have self-driving but self-driving might be making things so much safer that indeed uh people don't need as much accident insurance and no not as many ER people not as many ambulances and so in a weird way you make objectively the world better by reducing traffic deaths but it does actually have potentially a mildly negative effect on parts of the economy

2:21:28and we should all be rooting for that uh in this case, right? And so I think the lowest jobs in terms of moral standing are things where you don't really progress uh humanity forward. And my hunch is there the types of jobs in entertainment um that that people will value a lot and and fame and attention uh will become more of a currency in that world. And so if your job doesn't get you any of that, it might be considered uh lower in in that future. I think the lowest job that just will never go away will be something in politics where it's completely irrelevant already, but but it's just there and it'll stay there and no one's going to change it. >> Nice. All right. >> I got to throw in ID's perspective. He

2:22:10always thinks it's the BART train driver because it's unionized to hell. >> Yeah. Yeah. There you go. >> Alex, number six, close us out. >> Number six. If we cure illnesses that are often due to bad behavior, what's going to take care of the cause? Joe Wilder. I assume the cause is reference to people choosing to behave quote unquote badly. And the answer I this is another case of my not buying the premise. If you look at some of the really spectacular results that have been coming out of GLP1 class studies, including third and soon presumably fourth generation GLP1 RAS, they're actually addressing the cause. like addictive behaviors are being mitigated or at least partially treated by the same drugs that are curing in or at

2:22:53least treating have to to caveat that inflammation and blood sugar, diabetes, all of these other conditions the the root causes which is or human behavior are themselves the the human behavior is being affected by the same drugs. So I'm so optimistic about what Alex is saying because I really fully believe AI this is going to be one of the highest callings of AI very very soon is to make you feel really good about doing good things and happy as you're doing it >> love >> and not want not want to do bad things for yourself. It turns out we have the capability now. We figured out how to do that at least in part and we're going to figure out a lot more. >> All right, as always a call out to our amazing community. If you've got a music

2:23:35video that you'd like to show as outro, please send it to the team at mediadmandis.com. And speaking about amazing outros, here is Abundance by Steven Gross. Uh my dear Moonshot mates, this is the real you. So check it out. >> Got it. Peter a wgave. This one's for you. >> Bun dance. Everybody do the bun dance.

2:24:16Everybody do the bun dance. Thinking moonshot dreaming. A moving future screaming. Build it faster. Think much bigger. Exponential. Pull the trigger. Drink, Alec. Drink >> drink water martini glass. >> Oh, that was fun. That was Moonshots Live 2040 on the moon. >> Nice. Uh, so >> I don't think we should have to wait that long. >> Yeah. Uh, Richard, your new book, The Eureka Machine, wherever you purchase your books. Do you have the Audible out? >> Uh, should come out very soon. Yeah. >> All right. I'm an Audible reader, but I have skimmed the book here. Uh, congratulations. Say, Richard, there's I got to say something. What you're doing with the recursive looks like such an incredible opportunity to move humanity

Closing and Outro

2:25:07forward. So, congrats on taking it a good one for humanity. >> And and Richard, please resist the urge to have you.com acquire your own Frontier Lab like everyone else seems to be using, spinning off their own Frontier Labs, not to name names, uh, as a a financial engineering exercise to maximize their equity and their original startup. Please resist the urge to to follow that trend. >> All right. >> All right. I love you guys. >> Be well. Until next time.