WEBVTT

00:00:00.960 --> 00:00:05.117
Richard, I'm I'm curious.
Where are we at the moment with RSI

00:00:05.200 --> 00:00:09.677
>> in various weak forms? We already
have RSI. We're not quite there yet, but

00:00:09.760 --> 00:00:14.237
we're very close.
>> When do you believe we reach ASI?

00:00:14.320 --> 00:00:16.637
>> I think we'll take us probably several

00:00:16.720 --> 00:00:18.237
decades.

00:00:18.320 --> 00:00:21.437
>> On Terminal Bench 4.0,
Sonnet 5.5 jumped

00:00:21.520 --> 00:00:24.637
from 10% to 70%. Pretty extraordinary.

00:00:24.720 --> 00:00:28.477
My theory would be that they're trying
to compete with, you know, China, which

00:00:28.560 --> 00:00:32.557
is about three months behind, and fill
that gap before a lot of enterprises go

00:00:32.640 --> 00:00:35.997
to to open source models. >> It's
not at all obvious to me why anyone

00:00:36.080 --> 00:00:38.877
should be using Sonnet 5.5 over Opus 5.5

00:00:38.960 --> 00:00:43.517
unless you have some token or latency
or other consideration. I do not plan to

00:00:43.600 --> 00:00:47.437
use Sonnet 5.5. >> The Defense
Secretary announced Project

00:00:47.520 --> 00:00:49.757
Meridian. It's a new Pentagon effort on

00:00:49.840 --> 00:00:52.877
the future of warfare. It's co-led by

00:00:52.960 --> 00:00:54.637
Elon Musk and Palmer Lucky.

00:00:54.720 --> 00:00:57.197
>> This I think is a
transformative moment.

00:00:57.280 --> 00:00:59.917
This is a major major step forward. The

00:01:00.000 --> 00:01:03.597
big question is going to be.

00:01:03.680 --> 00:01:08.077
>> Now that's a moonshot.
Ladies and gentlemen,

00:01:08.160 --> 00:01:09.757
>> this episode is brought to you by the

00:01:09.840 --> 00:01:13.997
Abundance Summit and Link Ventures.

00:01:14.080 --> 00:01:17.277
>> Welcome to Moonshots, everyone.
Your number one podcast and help you

00:01:17.360 --> 00:01:19.357
understand the singularity. What's going

00:01:19.440 --> 00:01:21.917
on? What does it mean to you and your

00:01:22.000 --> 00:01:24.637
family and what the breaking news is? We

00:01:24.720 --> 00:01:29.037
publish this twice a week to help you
keep up with the extraordinary rate of

00:01:29.120 --> 00:01:31.517
change. With me today is the Fantastic

00:01:31.600 --> 00:01:35.117
Four, Alex Weer Gross, our in-house ASI,

00:01:35.200 --> 00:01:37.517
Dave Blendon, our improario of AI

00:01:37.600 --> 00:01:42.877
investing, Salem Ismael, our Globe
Troder who's home today. Amazing Sem.

00:01:42.960 --> 00:01:46.637
>> Woohoo. >> And the and the father of
the organizational singularity. I'm Peter

00:01:46.720 --> 00:01:48.557
Diamandis, your host and datadriven

00:01:48.640 --> 00:01:51.197
optimist. Our mission here, keep you

00:01:51.280 --> 00:01:53.837
optimistic about the future. Uh, and

00:01:53.920 --> 00:01:55.677
here's the numbers from this week. In 7

00:01:55.760 --> 00:01:58.317
days, Google shipped Gemini for Argon.

00:01:58.400 --> 00:02:01.117
Anthropic shipped Sonnet 5.5. Open eye

00:02:01.200 --> 00:02:03.837
shipped GPT 6.1.

00:02:03.920 --> 00:02:06.957
Saul and the Frontier Intelligence has

00:02:07.040 --> 00:02:09.437
gotten cheaper by three-fold. SpaceX

00:02:09.520 --> 00:02:11.197
launched the next batch of astronauts,

00:02:11.280 --> 00:02:14.557
the ISS, and also launched Google's TPUs

00:02:14.640 --> 00:02:19.117
into orbit for Project Suncatcher. Uh,
if you're new to the Moonshots podcast,

00:02:19.200 --> 00:02:23.037
please hit subscribe. We publish twice
a week and you don't want to miss any of

00:02:23.120 --> 00:02:25.197
this news during this hypersonic

00:02:25.280 --> 00:02:28.317
tsunami. Our moonshot here is to 20x our

00:02:28.400 --> 00:02:33.357
subscriber base to get to 10 million to
help spread the word of optimism and the

00:02:33.440 --> 00:02:37.517
extraordinary future that we're
building. Today on Moonshot, we're

00:02:37.600 --> 00:02:39.357
joined by one of the architects of

00:02:39.440 --> 00:02:43.037
modern AI, Richard. Uh, born in Germany

00:02:43.120 --> 00:02:47.037
and trained at Stanford. Richard
is among the world's most cited natural

00:02:47.120 --> 00:02:51.277
language processing researchers,
a pioneer in deep learning and prompt

00:02:51.360 --> 00:02:53.517
engineering and a serial founder who's

00:02:53.600 --> 00:02:55.677
repeatedly turned frontier research into

00:02:55.760 --> 00:02:58.477
category defining companies. First, he

00:02:58.560 --> 00:03:03.037
founded Metammind uh which was acquired
by Salesforce where he then became the

00:03:03.120 --> 00:03:05.277
chief scientist leading AI research for

00:03:05.360 --> 00:03:07.597
friend of the pod Mark Beni off. Next,

00:03:07.680 --> 00:03:10.637
Richard founded you.com, recognized by

00:03:10.720 --> 00:03:13.197
Time in the World Economic Forum and now

00:03:13.280 --> 00:03:16.637
valued at north of 1.5 billion as well

00:03:16.720 --> 00:03:19.837
as his fund AO aix

00:03:19.920 --> 00:03:22.237
ventures. Today, Richard is co-founder

00:03:22.320 --> 00:03:25.597
and CEO of Recursive, perhaps uh the

00:03:25.680 --> 00:03:28.317
biggest moonshot he's ever taken, uh

00:03:28.400 --> 00:03:30.317
focused on recursively self-improving

00:03:30.400 --> 00:03:33.197
super intelligence. Richard, congrats on

00:03:33.280 --> 00:03:35.837
a $670 million fund raise from Google

00:03:35.920 --> 00:03:38.317
Ventures. Graycraft, Nvidia, and AMD and

00:03:38.400 --> 00:03:41.117
410 million in compute from AWS.

00:03:41.200 --> 00:03:43.437
Personal disclosure, I'm very proud to

00:03:43.520 --> 00:03:46.477
be a seed investor in Recursive. And of

00:03:46.560 --> 00:03:48.477
course, Richard, uh, important to

00:03:48.560 --> 00:03:51.197
mention your new book just got released,

00:03:51.280 --> 00:03:54.317
The Eureka Machine: Why AI is the key to

00:03:54.400 --> 00:03:59.357
unlocking a new era of scientific
discoveries. Welcome to the pod, Richard.

00:03:59.440 --> 00:04:02.637
>> So great to be back.
>> And let me just add, a truly truly

00:04:02.720 --> 00:04:06.877
awesome guy. A lot of people who listen
to the pot are worried about the the

00:04:06.960 --> 00:04:11.117
ethics and the the risks and everything,
but boy, if you want a a person to

00:04:11.200 --> 00:04:15.517
conquer recursive self-improvement that
you can like and trust, Richard is the

00:04:15.600 --> 00:04:18.397
man. >> Yeah. >> Thank you
so much. So great to be back

00:04:18.480 --> 00:04:22.397
here. I love your guys. Constructive
optimism in the world right now.

00:04:22.480 --> 00:04:24.557
>> I have I have a bold
prediction for this episode.

00:04:24.640 --> 00:04:28.077
>> What's that? >> This one
episode will prove our thesis

00:04:28.160 --> 00:04:32.237
for this whole podcast more than
any other episode we ever record.

00:04:32.320 --> 00:04:34.557
>> Awesome. Let's go. >> Let's go.

00:04:34.640 --> 00:04:37.437
>> And uh uh Alex, do you know Richard?

00:04:37.520 --> 00:04:41.117
>> Yeah, Richard, you and I have
chatted quite a bit. I don't think we've

00:04:41.200 --> 00:04:43.117
actually ever met in person, though.

00:04:43.200 --> 00:04:44.957
>> Really? >> We have not.
And it's the first time on

00:04:45.040 --> 00:04:46.477
a podcast together.
It's going to be awesome.

00:04:46.560 --> 00:04:48.397
>> Amazing. Well, let's make history.

00:04:48.480 --> 00:04:51.517
>> Yeah, for sure. So, uh Richard, let's

00:04:51.600 --> 00:04:55.917
start with your book. Uh the central
claim of the Eureka machine is that AI

00:04:56.000 --> 00:05:00.477
will deliver a century of scientific
breakthroughs in the next decade.

00:05:00.560 --> 00:05:02.877
uh you know this maps directly onto what

00:05:02.960 --> 00:05:05.117
Alex and I wrote in solve everything. So

00:05:05.200 --> 00:05:07.757
we're fans of that prediction. Uh your

00:05:07.840 --> 00:05:10.317
thesis in the book is that every stage

00:05:10.400 --> 00:05:13.197
of scientific process gets connected and

00:05:13.280 --> 00:05:15.517
transformed simultaneously. Hypothesis,

00:05:15.600 --> 00:05:18.557
experiment, data theory, you call it the

00:05:18.640 --> 00:05:21.757
full stack AI. Uh you also say that

00:05:21.840 --> 00:05:24.317
scientific progress has slowed and

00:05:24.400 --> 00:05:26.477
that's been caused not by underfunding

00:05:26.560 --> 00:05:29.517
but by fragmentation. So let's start

00:05:29.600 --> 00:05:31.917
with those two items. Could you take a

00:05:32.000 --> 00:05:33.997
moment and you know talk to us about

00:05:34.080 --> 00:05:36.797
full stack AI for science and why you

00:05:36.880 --> 00:05:39.677
think progress has slowed?

00:05:39.760 --> 00:05:43.437
>> Yeah. So progress has slowed uh largely

00:05:43.520 --> 00:05:47.437
because we have so many different subd
disccipline and uh disciplines and even

00:05:47.520 --> 00:05:51.677
sus of lem uh realized like wow that
like if there are only so many people

00:05:51.760 --> 00:05:55.437
and we have more and more fragmentation
of more and more subd disciplines and

00:05:55.520 --> 00:05:57.997
niches like you can not just do AI right

00:05:58.080 --> 00:06:01.117
you're doing often like optimization uh

00:06:01.200 --> 00:06:03.357
and gradient descent methods and second

00:06:03.440 --> 00:06:06.397
order derivative methods uh and so on of

00:06:06.480 --> 00:06:10.557
uh these neural networks that is which
is one category of AI and so on. And the

00:06:10.640 --> 00:06:13.917
same is true in biology. You study
biology, you're going to be either a

00:06:14.000 --> 00:06:16.877
cell biologist or a molecular biologist

00:06:16.960 --> 00:06:22.077
uh or you're like in medicine and like
these like there's so much separation

00:06:22.160 --> 00:06:23.677
that it's hard to weave it all together.

00:06:23.760 --> 00:06:26.317
And that is the perfect time for AI to

00:06:26.400 --> 00:06:30.957
come in and help us weave back together
all these separate pieces and also help

00:06:31.040 --> 00:06:33.037
us understand how these large complex

00:06:33.120 --> 00:06:38.317
systems actually work. like we can't
have one model where humans say here are

00:06:38.400 --> 00:06:42.157
all the things I know about natural
language and now like we can put all

00:06:42.240 --> 00:06:45.837
these rules together and then we have
a conversation. It required a large neural

00:06:45.920 --> 00:06:47.917
network with a ton of data. And guess

00:06:48.000 --> 00:06:50.797
what? We're going to get a lot of data

00:06:50.880 --> 00:06:55.357
uh about biology. And so we can make
more and more disciplines and move them

00:06:55.440 --> 00:06:59.117
transfer them from being traditionally
natural sciences where we just try to

00:06:59.200 --> 00:07:01.757
understand biology and nature to being

00:07:01.840 --> 00:07:04.557
programmable engineering sciences. And

00:07:04.640 --> 00:07:06.797
that I think is one of the many

00:07:06.880 --> 00:07:12.157
different exciting aspects of things
to come. And you know why why am I so

00:07:12.240 --> 00:07:15.197
excited that that this is happening
right now and and what are the major

00:07:15.280 --> 00:07:18.557
ingredients? It's essentially that we
now have world knowledge in the form of

00:07:18.640 --> 00:07:22.477
LMS. We have more and more scientific
data that's getting digitized that we

00:07:22.560 --> 00:07:26.477
can sit on top of. We have better and
better simulations of things and we have

00:07:26.560 --> 00:07:30.157
robotic process automation that will
soon be possible. And on top of that

00:07:30.240 --> 00:07:31.517
we're going to have an agent swarm and

00:07:31.600 --> 00:07:34.717
that is how uh this the full scientific

00:07:34.800 --> 00:07:38.237
stack uh can be automated. Alex.

00:07:38.320 --> 00:07:42.157
>> Yeah, I can say I disagree. As Peter,
you and I wrote in solve everything,

00:07:42.240 --> 00:07:45.437
math, science, and engineering are cooked.
I'm curious, Richard, what your

00:07:45.520 --> 00:07:48.317
latest timelines are.

00:07:48.400 --> 00:07:51.117
>> I mean, the I think
every field is going

00:07:51.200 --> 00:07:54.317
to get to higher and higher levels
of abstraction. Computer science has been

00:07:54.400 --> 00:07:57.757
very good at that, right? No one
is programming in zeros and ones anymore.

00:07:57.840 --> 00:07:59.997
No one like very few people have to

00:08:00.080 --> 00:08:02.397
still know C++ and complex pointers and

00:08:02.480 --> 00:08:04.957
and memory uh architectures and so on.

00:08:05.040 --> 00:08:07.037
we can get a larger and larger and

00:08:07.120 --> 00:08:08.797
computer science has basically

00:08:08.880 --> 00:08:12.157
abstracted enough so that it met the

00:08:12.240 --> 00:08:14.237
rest of humanity with English, right?

00:08:14.320 --> 00:08:16.237
And natural language that now can be

00:08:16.320 --> 00:08:18.797
used to do computer science. And so I

00:08:18.880 --> 00:08:21.517
think that um gives me great hope uh

00:08:21.600 --> 00:08:24.077
that we can get other fields into

00:08:24.160 --> 00:08:27.037
similar uh stages of abstraction and

00:08:27.120 --> 00:08:31.677
then all meet and natural language
to do science. But to pin you down, as I

00:08:31.760 --> 00:08:35.997
recall from my understanding of your book,
3 to five years for the physical

00:08:36.080 --> 00:08:39.277
sciences, >> you know,
like there's no like single

00:08:39.360 --> 00:08:42.877
sort of this is the threshold and now we
solved all of physical sciences, right?

00:08:42.960 --> 00:08:47.357
But like multiple different diseases
will get cured in the next 12 months.

00:08:47.440 --> 00:08:51.277
They're going to be the simpler diseases
maybe where there's one gene that needs

00:08:51.360 --> 00:08:52.957
to be fixed for that disease to be

00:08:53.040 --> 00:08:55.997
cured. Um but there are a lot of single

00:08:56.080 --> 00:09:00.717
gene diseases uh that that are out there
in aggregate, right? like and so those

00:09:00.800 --> 00:09:03.837
diseases will get cured and then we're
going to cure more and more complex

00:09:03.920 --> 00:09:07.837
diseases. We're going to develop
better and better battery material. So I

00:09:07.920 --> 00:09:10.877
wouldn't call it like one threshold
in like in 3 years sort of everything is

00:09:10.960 --> 00:09:13.757
solved but like we're going to solve
more and more problems and it'll just be

00:09:13.840 --> 00:09:18.877
a question of how much money do you want
to put into compute to then solve which

00:09:18.960 --> 00:09:21.197
kinds of problems? >> You're
you're talking to the guy Richard

00:09:21.280 --> 00:09:24.477
remember who argues that the singularity
itself is something of an optical

00:09:24.560 --> 00:09:26.237
illusion that there is no step function.

00:09:26.320 --> 00:09:28.397
It's just a time interval. So I'll I'll

00:09:28.480 --> 00:09:32.317
try once more timelines. When when even

00:09:32.400 --> 00:09:34.877
if it's not a step function, when is the

00:09:34.960 --> 00:09:38.957
inflection point when is the um 50%

00:09:39.040 --> 00:09:42.957
point crossed in your mind for all
of the physical sciences getting solved?

00:09:43.040 --> 00:09:46.477
All human disease getting solved?
Where are those timelines?

00:09:46.560 --> 00:09:50.477
>> Alex, if I could just add a a layer on
top of that. So we've talked on this pot

00:09:50.560 --> 00:09:52.637
a lot that math is cooked, right? We're

00:09:52.720 --> 00:09:54.717
seeing millennium prizes all falling.

00:09:54.800 --> 00:09:57.837
Yep. And the n the next natural uh if

00:09:57.920 --> 00:10:03.517
you would uh uh barbecue is likely
to be physics >> incineration.

00:10:03.600 --> 00:10:06.317
>> Incineration. Yes.
>> And computer science arguably I mean

00:10:06.400 --> 00:10:09.277
presumably this is part of the premise
Richard of recursive as well. Computer

00:10:09.360 --> 00:10:12.877
science already cooked
math thoroughly cooked.

00:10:12.960 --> 00:10:16.237
>> You know in a weird way I would I I
would use different metaphors. I think

00:10:16.320 --> 00:10:17.997
they're flourishing not getting cooked.

00:10:18.080 --> 00:10:21.357
I think the weird thing is, you know,
some some mathematicians recently said,

00:10:21.440 --> 00:10:25.117
"Oh, this is this may be bad for the
field because like we need to train

00:10:25.200 --> 00:10:29.117
people." Yes, we need to train people,
but imagine a biologist or or medical

00:10:29.200 --> 00:10:31.757
researcher saying, "Oh, you know,
it's really a bummer for the field that we

00:10:31.840 --> 00:10:33.597
cured all these diseases for people."

00:10:33.680 --> 00:10:36.717
Like that would be insane, right? Of
course, you want to just move as quickly

00:10:36.800 --> 00:10:39.757
as you can towards solving these problems.
I think maybe the problem with

00:10:39.840 --> 00:10:43.197
math is that there's many sub fields in

00:10:43.280 --> 00:10:47.437
of math that haven't really had
there are just almost purely beautiful

00:10:47.520 --> 00:10:51.677
intellectual exercises without real life
impact anymore. But as you get close to

00:10:51.760 --> 00:10:55.917
real life impact, you should be excited
about it and you think and realize your

00:10:56.000 --> 00:10:57.917
field is flourishing, not being cooked.

00:10:58.000 --> 00:11:03.197
Uh, and so to timelines, I think the the
further the more complex a disease is,

00:11:03.280 --> 00:11:06.397
the more you have to do long-term
studies with humans before you're

00:11:06.480 --> 00:11:08.557
allowed to put a certain drug into

00:11:08.640 --> 00:11:11.197
humans, the more delays you will have.

00:11:11.280 --> 00:11:15.757
But like we see now companies to make
this very concrete, biotech companies

00:11:15.840 --> 00:11:19.517
used to have like one new drug
in development and it took them 10 years.

00:11:19.600 --> 00:11:22.957
They had to go public before they knew
the comp the drug was really working.

00:11:23.040 --> 00:11:25.837
And then after 10 years maybe late stage

00:11:25.920 --> 00:11:28.317
three trials were just not working and

00:11:28.400 --> 00:11:30.397
then the company is dead. Right now you

00:11:30.480 --> 00:11:33.117
have in the new age of companies they

00:11:33.200 --> 00:11:36.557
have 5 to 10 different compounds in uh

00:11:36.640 --> 00:11:41.117
stage three trials after 2 or 3
years and and so like we see a lot of

00:11:41.200 --> 00:11:45.757
acceleration in that level but again
diseases to really be able to get them

00:11:45.840 --> 00:11:49.437
into humans at scale in the United
States with FDA approvals there are just

00:11:49.520 --> 00:11:54.397
some natural delays that will be more
like half a decade to a decade uh but uh

00:11:54.480 --> 00:11:58.477
lots of other things in chemistry
and physics and so on where we can iterate

00:11:58.560 --> 00:12:00.397
without having to look at long-term

00:12:00.480 --> 00:12:02.797
human trials will be even faster.

00:12:02.880 --> 00:12:05.597
>> And I the the the
point is well taken. I

00:12:05.680 --> 00:12:08.157
I like your Orwellian turn of phrase.

00:12:08.240 --> 00:12:12.477
Maybe instead of saying AI is cooking
math, I should be using flourishing as a

00:12:12.560 --> 00:12:17.037
transitive verb and just
say AI is flourishing math.

00:12:17.120 --> 00:12:20.957
>> Actually, I was with Certas Caraman
who runs lids at MIT uh the night before

00:12:21.040 --> 00:12:25.917
last. He he started a drone company now,
an AI drone company. But lids is where

00:12:26.000 --> 00:12:30.157
radar was invented originally during
World War II. It's a great lab, very

00:12:30.240 --> 00:12:34.077
entrepreneurial. But he said all his
mathematician friends at MIT uh are

00:12:34.160 --> 00:12:37.197
aware that they're cooked
and they're they're all

00:12:37.280 --> 00:12:38.957
>> No, no, Dave.
Dave, we're going with the

00:12:39.040 --> 00:12:42.397
Orwellian language now. They're they're
not cooked, they're being flourished.

00:12:42.480 --> 00:12:45.597
>> He That's funny because he literally
said cooked, but I'll go back to him and

00:12:45.680 --> 00:12:47.357
tell your friends they're flourishing.

00:12:47.440 --> 00:12:52.477
>> Tell not to worry.
If your goal was to prove

00:12:52.560 --> 00:12:56.397
as many theorems as possible in your
lifetime, like now is the time to just

00:12:56.480 --> 00:13:00.477
grab like grab as many as you can and
and work with AI to solve them. And then

00:13:00.560 --> 00:13:04.157
I think the field will change the way
computer science has changed in many

00:13:04.240 --> 00:13:09.197
ways. Uh and it'll be much more about
what's the most creative thing when you

00:13:09.280 --> 00:13:12.557
really understand all the things that are
out there in math. What kind of new

00:13:12.640 --> 00:13:16.557
formalisms can you create?
New constructs can you create that then

00:13:16.640 --> 00:13:21.517
would be interesting to be solved uh
by an AI in collaboration with humans.

00:13:21.600 --> 00:13:25.757
>> Well, Cash was saying the math guys are
in great shape because they're so cooked

00:13:25.840 --> 00:13:28.957
that they're all moving over to AI
orchestration and they're going to be

00:13:29.040 --> 00:13:31.837
way ahead of the curve. He he's
actually most worried about the biology

00:13:31.920 --> 00:13:34.397
professors who are in complete denial.

00:13:34.480 --> 00:13:37.917
You using virtually no AI in their
day-to-day activities. And this is one

00:13:38.000 --> 00:13:41.917
area where Richard, you have a such
a deep background in all facets of AI,

00:13:42.000 --> 00:13:45.997
including biology. And I I feel like
we're living exactly parallel lives

00:13:46.080 --> 00:13:47.517
except you're 15 years younger than me.

00:13:47.600 --> 00:13:51.357
So I'm like insanely jealous of of
your life trajectory. But be a serial

00:13:51.440 --> 00:13:54.877
entrepreneur, then start a hugely
successful venture fund, then found a

00:13:54.960 --> 00:13:58.637
foundation model company. And and you
know, we're seeing the world through the

00:13:58.720 --> 00:14:00.797
exact same lens. But you and Peter have

00:14:00.880 --> 00:14:03.197
much more biology background. I I have

00:14:03.280 --> 00:14:07.837
basically none. uh but the biologists
are the ones really really lagging and I

00:14:07.920 --> 00:14:10.397
I think you've got a bunch
of investments actually that are that have

00:14:10.480 --> 00:14:13.757
done really well in the area right >>
and I would love to talk about some of

00:14:13.840 --> 00:14:18.557
those like peril bio proximal labs Ignod
labs like truly exciting companies I

00:14:18.640 --> 00:14:22.717
think another field that is even
more lacking than biology is economics

00:14:22.800 --> 00:14:25.597
economics literally
has these models of like

00:14:25.680 --> 00:14:28.397
>> a linear model of economics right a

00:14:28.480 --> 00:14:32.717
one-step economy that's provably
correctly taxed and and subsidized and

00:14:32.800 --> 00:14:34.797
things like that it's just like absurd

00:14:34.880 --> 00:14:40.557
how slow that field is to adopt AI
for making better policy decisions

00:14:40.640 --> 00:14:42.797
ideology >> because you
opened the door. Eric

00:14:42.880 --> 00:14:45.997
Bolson, our very good friend, uh I did

00:14:46.080 --> 00:14:49.357
not realize till yesterday that HI lab

00:14:49.440 --> 00:14:53.197
high at Stanford invented
the word foundation model

00:14:53.280 --> 00:14:56.237
>> and he sent me the whole you
didn't you knew that already? Wow.

00:14:56.320 --> 00:14:59.837
>> Yeah, of course. Like lots of Stanford
friends and Percy Lang and others uh

00:14:59.920 --> 00:15:03.197
worked on foundational models. I love
Eric Goffson. actually one of the most

00:15:03.280 --> 00:15:06.717
interesting economists I think uh doing
really interesting research right now.

00:15:06.800 --> 00:15:11.917
He also started work helix which we're
a proud investor in uh that brings uh like

00:15:12.000 --> 00:15:13.437
understanding of how companies actually

00:15:13.520 --> 00:15:16.477
adopt AI uh and which tasks are getting

00:15:16.560 --> 00:15:21.197
you know helped by in real rollouts for
companies. Uh yeah he's a co-founder of

00:15:21.280 --> 00:15:23.597
that too. >> Yeah building
is just such a great place

00:15:23.680 --> 00:15:27.517
cuz you talk about econ people being way
off the curve but he is so ahead of the

00:15:27.600 --> 00:15:30.957
curve. You walk into the building and you
can just feel it. He's also poaching

00:15:31.040 --> 00:15:34.957
a ton of talent from MIT
to come out to to join his lab,

00:15:35.040 --> 00:15:39.277
>> following his path. Richard, talk
about full stack AI as you see it in the

00:15:39.360 --> 00:15:43.757
scientific method. Um, you know, I'm an
investor in a company called Laya out of

00:15:43.840 --> 00:15:46.157
MIT and Harvard. You know, Lyla science

00:15:46.240 --> 00:15:50.877
as well. Uh, I think I've introduced
you to to Jeff von Maltson there. and you

00:15:50.960 --> 00:15:55.117
know they're building a scientific
super intelligence that is then running a

00:15:55.200 --> 00:15:57.277
million square foot of robotic space to

00:15:57.360 --> 00:15:59.357
sort of like a thousandx the rate of

00:15:59.440 --> 00:16:02.717
discovery. Um your thoughts on that?

00:16:02.800 --> 00:16:06.317
>> I absolutely love it. I think that is
you know in the Eureka machine I sort of

00:16:06.400 --> 00:16:11.117
lay out these four pillars and they're
actually one of the few um uh that that

00:16:11.200 --> 00:16:14.397
are really going after this similar
to periodic labs who are doing it more on

00:16:14.480 --> 00:16:18.157
the physics and chemistry side of things.
Uh, and there are several other

00:16:18.240 --> 00:16:21.837
companies now that I can hopefully soon
talk about that are trying to create

00:16:21.920 --> 00:16:24.237
more data for the bitter lesson to be

00:16:24.320 --> 00:16:27.117
applicable in biology. And I love uh

00:16:27.200 --> 00:16:29.037
what you know the big guys Eli Liy and

00:16:29.120 --> 00:16:32.157
others do uh and and Laya too. I think

00:16:32.240 --> 00:16:36.477
you know the ultimately uh you can
boil down the scientific method to the

00:16:36.560 --> 00:16:38.637
ideation implementation and validation

00:16:38.720 --> 00:16:40.717
of ideas and the faster we can close

00:16:40.800 --> 00:16:43.677
that loop and then put an open-ended

00:16:43.760 --> 00:16:47.837
process on top of it that's what
open-endedness uh what inspired us a lot

00:16:47.920 --> 00:16:51.677
at recursive and we have many of the
world's greatest uh like researchers in

00:16:51.760 --> 00:16:53.997
that subdomain of AI that is still not

00:16:54.080 --> 00:16:56.157
quite as popular as it could and should

00:16:56.240 --> 00:17:00.317
be um like the more you can have a swarm

00:17:00.400 --> 00:17:04.957
uh innovate in open-ended ways
and evolve uh and combine interestingly

00:17:05.040 --> 00:17:06.877
different ideas the better. But then of

00:17:06.960 --> 00:17:09.437
course in biology you have to have uh

00:17:09.520 --> 00:17:12.877
like actual uh like physical biological

00:17:12.960 --> 00:17:14.717
wetwware experiments. And so it's really

00:17:14.800 --> 00:17:19.117
great to see um uh Laya doing that. Um

00:17:19.200 --> 00:17:22.397
we're seeing this also maybe I can talk
about this one company that I really

00:17:22.480 --> 00:17:26.877
love called Peril Bio.
They build tiny organoids on a petri dish,

00:17:26.960 --> 00:17:31.517
>> right? And get this, if you love
animals, you too can love AI. Why?

00:17:31.600 --> 00:17:35.437
Because they got FDA approval to skip
animal trials. Turns out we cured most

00:17:35.520 --> 00:17:37.277
diseases in mice. It's not that helpful.

00:17:37.360 --> 00:17:41.757
They're very different to people. Uh but
these guys uh use pur potent stem cells

00:17:41.840 --> 00:17:44.797
create tiny little organoids of lymph
nodes. Lymph nodes big part of your

00:17:44.880 --> 00:17:48.237
immune system. Imunotherapy some of the
most exciting therapies to allow your

00:17:48.320 --> 00:17:51.197
own immune system to attack a cancer
and so on instead of getting crazy

00:17:51.280 --> 00:17:55.357
chemotherapy and so on. And so they got
FDA approval because they showed that

00:17:55.440 --> 00:17:59.357
when you run experiments in parallel,
hence parallel bio in these tiny petri

00:17:59.440 --> 00:18:03.597
dishes, how those organoids react to
different drugs and toxicity testing and

00:18:03.680 --> 00:18:07.677
so on is actually more predictive of how
those drugs will interact in real human

00:18:07.760 --> 00:18:11.917
bodies. And that is just one of many
beautiful examples of where it will

00:18:12.000 --> 00:18:13.837
help. And you can take my stem cell, you

00:18:13.920 --> 00:18:16.397
can create take my skin, create in a uh

00:18:16.480 --> 00:18:19.997
IPSC uh cell, grow my own organs and see

00:18:20.080 --> 00:18:24.717
how a particular drug would work for me
versus a generic individual. Uh Salem,

00:18:24.800 --> 00:18:27.997
you want to jump in? >> Uh yeah,
a couple of things. One is, you

00:18:28.080 --> 00:18:31.117
know, science has always been
a coordination problem, right? You're kind

00:18:31.200 --> 00:18:32.557
of trying to bring and it's always been

00:18:32.640 --> 00:18:35.837
very structured into departments and and

00:18:35.920 --> 00:18:39.597
journals and all that stuff. And I love
what you're doing, bringing it together.

00:18:39.680 --> 00:18:41.517
For me, if I had to summarize what you

00:18:41.600 --> 00:18:43.517
seem to be doing is you're collapsing

00:18:43.600 --> 00:18:48.637
the time between an experiment and a
result or hypothesis and a result. And

00:18:48.720 --> 00:18:51.917
when you can collapse that,
that's domain collapse of the scientific

00:18:52.000 --> 00:18:55.997
method. And now like anything
is possible from there. This is amazing.

00:18:56.080 --> 00:18:57.997
>> Yeah. I I am going
to push again on what

00:18:58.080 --> 00:19:00.557
what Alex said, physics. When do you you

00:19:00.640 --> 00:19:03.037
know you know Einstein's theory

00:19:03.120 --> 00:19:05.197
relativity comes out. There's not that

00:19:05.280 --> 00:19:08.317
much progress uh you know since then. a

00:19:08.400 --> 00:19:11.597
lot of theories. Do you see physics as

00:19:11.680 --> 00:19:15.597
the next domain that's going
to flourish on the back of math?

00:19:15.680 --> 00:19:19.197
>> I think the biggest domain is actually
going to be biology. Physics has been

00:19:19.280 --> 00:19:21.277
interestingly stuck in many ways like

00:19:21.360 --> 00:19:23.677
they there's a really powerful ideas

00:19:23.760 --> 00:19:28.317
like E= MC². So you can get a ton
of energy out of potentially little mass

00:19:28.400 --> 00:19:30.157
and then we got nuclear energy out of

00:19:30.240 --> 00:19:35.357
that. uh and so that then moved into
and graduated in some ways into an

00:19:35.440 --> 00:19:39.917
engineering discipline uh which is where
you have the real real life impact. Um I

00:19:40.000 --> 00:19:44.397
think you know obviously like fusion
will be great to finally get figured out

00:19:44.480 --> 00:19:48.557
and there's a lot of really cool like
companies and and big labs and you know

00:19:48.640 --> 00:19:50.797
tokomax already to balance uh like

00:19:50.880 --> 00:19:53.597
plasma inside tokomax like is a very

00:19:53.680 --> 00:19:55.917
hard control problem that where AI is

00:19:56.000 --> 00:19:58.637
being used. Um I hope we can eventually

00:19:58.720 --> 00:20:02.957
make better theories for uh you know

00:20:03.040 --> 00:20:04.717
quantum gravity and all kinds of other

00:20:04.800 --> 00:20:06.877
complex issues and and have get a better

00:20:06.960 --> 00:20:10.797
world sort of formula. Um I

00:20:10.880 --> 00:20:15.117
unfortunately the data set collections
these days require often like large

00:20:15.200 --> 00:20:19.197
hydron colliders and just like billions
of dollars and it's like quite expensive

00:20:19.280 --> 00:20:23.917
actually. So in a weird way, biology
is a better fit for AI because what

00:20:24.000 --> 00:20:25.517
calculus did for physics like

00:20:25.600 --> 00:20:29.117
understanding really well micro uh

00:20:29.200 --> 00:20:33.597
individualized separated phenomena,
neural nets are great in combining all

00:20:33.680 --> 00:20:37.117
these little things like we know what
the neuron does. We know what one uh

00:20:37.200 --> 00:20:40.957
bacteria does in our microbiome, but then
as they all come together and form

00:20:41.040 --> 00:20:45.197
these really complex interactions,
we don't really know anymore how that

00:20:45.280 --> 00:20:46.877
works. And so that's where I I think

00:20:46.960 --> 00:20:50.157
will help us more. Yeah, Alex,

00:20:50.240 --> 00:20:54.237
>> let's talk about biology a bit. So,
do you think the critical path to solving

00:20:54.320 --> 00:20:56.557
biology goes through digital twins of

00:20:56.640 --> 00:21:01.517
cells or do you have some wildly
different theory of the case? Three

00:21:01.600 --> 00:21:05.277
years ago when I started Eureka Machine
book, I had this chapter on the virtual

00:21:05.360 --> 00:21:08.077
cell and I was really proud of like
trying to lay it all out and I had to

00:21:08.160 --> 00:21:11.437
rewrite that whole thing because in
the meantime uh you know Mark Zuckerberg

00:21:11.520 --> 00:21:14.397
Foundation started virtual cell

00:21:14.480 --> 00:21:19.277
everyone not just CI everyone has
a virtual cell model Dave Even if Dave

00:21:19.360 --> 00:21:22.797
doesn't even if Dave doesn't think Dave
has a virtual cell model I'm sure he'll

00:21:22.880 --> 00:21:26.237
spin one up soon. So I think that most

00:21:26.320 --> 00:21:31.117
So there's this famous uh uh paper
by Rich Sutton um on the bitter lesson in

00:21:31.200 --> 00:21:33.277
AI and the >> never heard of it.
Tell me all about the

00:21:33.360 --> 00:21:36.957
bitter lesson. This is a new concept
for me. What what is this you speak of?

00:21:37.040 --> 00:21:40.477
>> I don't know. You're probably joking
but pro maybe some some readers haven't

00:21:40.560 --> 00:21:43.757
heard of it. So the bitter lesson
is basically that human experts had all

00:21:43.840 --> 00:21:47.277
these really clever ideas and beautiful
theories but really what you needed to

00:21:47.360 --> 00:21:51.757
do was the simp like to make real
progress is the simplest method that you

00:21:51.840 --> 00:21:55.837
can come up with a large neural network
that you can train end to end as a

00:21:55.920 --> 00:21:58.957
general function approximator. The
simplest model you can come up with and

00:21:59.040 --> 00:22:01.677
then it's just a ton of data and compute

00:22:01.760 --> 00:22:06.077
to like actually train that model
and that usually outperforms at scale all

00:22:06.160 --> 00:22:10.237
the clever little hacks that human
experts had come up with uh before. And

00:22:10.320 --> 00:22:14.077
so the bitter lesson has worked for NLP
10, 20 years ago. You would have asked

00:22:14.160 --> 00:22:18.317
an NLP expert. Can you have one model
to have any conversation with you prompted

00:22:18.400 --> 00:22:22.317
any kind of question prompt engineering
I like invented and was nicely cited by

00:22:22.400 --> 00:22:27.677
the early GBT papers. But like the same
thing, the same state uh is biology is

00:22:27.760 --> 00:22:31.117
currently in. There's so many
complexities and the experts know so

00:22:31.200 --> 00:22:34.397
much. They feel like there's no way you
can instill all of that knowledge into

00:22:34.480 --> 00:22:36.957
one model. But if you have enough data,

00:22:37.040 --> 00:22:41.757
you can. And now you have companies like
Tahoe therapeutics and so on that are

00:22:41.840 --> 00:22:45.757
creating these massive perturbation
studies and yes all models are wrong but

00:22:45.840 --> 00:22:49.037
more and more of them will be useful
as we collect more and more data about

00:22:49.120 --> 00:22:53.277
biology and that is I think one
of the biggest driving factors uh for the

00:22:53.360 --> 00:22:56.957
impact of AI. >> But so ju
just to answer the question do

00:22:57.040 --> 00:23:01.117
you think virtual cell models are the
critical path to solving all disease or

00:23:01.200 --> 00:23:03.757
solving biology or is it something else?

00:23:03.840 --> 00:23:06.957
>> I I think they're definitely going
to play a crucial part. It's the third

00:23:07.040 --> 00:23:10.957
pillar of the Eureka machine. We do
need to have anything you can simulate,

00:23:11.040 --> 00:23:13.437
anything you can verify, AI will then be

00:23:13.520 --> 00:23:15.837
able to solve things in that domain. And

00:23:15.920 --> 00:23:21.117
so yes, uh I do think uh virtual models
uh and simulations uh are extremely

00:23:21.200 --> 00:23:24.717
important. >> I'm going
to jump into three fun

00:23:24.800 --> 00:23:26.557
breaking stories this week. Two on

00:23:26.640 --> 00:23:29.997
science and one on AI. Uh and Richard,

00:23:30.080 --> 00:23:32.237
they you know relate back to your work

00:23:32.320 --> 00:23:34.397
and your book. Uh our first story was

00:23:34.480 --> 00:23:37.517
published in MIT Tech Review today. Uh

00:23:37.600 --> 00:23:39.757
it tells the story of researchers at

00:23:39.840 --> 00:23:41.997
Israel's Weissman Institute who built a

00:23:42.080 --> 00:23:44.957
system called brain it. The system

00:23:45.040 --> 00:23:47.437
reconstructs the image a person is

00:23:47.520 --> 00:23:49.917
looking at while inside a functional MRI

00:23:50.000 --> 00:23:54.797
machine. I I'm just post put up
the slide here. So take a look at this

00:23:54.880 --> 00:23:58.397
image. You know it's pretty extraordinary.
On one side is the image

00:23:58.480 --> 00:24:00.637
that a person is looking at inside the

00:24:00.720 --> 00:24:03.757
fMRI machine. On the other is the AI

00:24:03.840 --> 00:24:05.757
reconstruction. You know, so earlier

00:24:05.840 --> 00:24:07.997
brain image decoders could tell you were

00:24:08.080 --> 00:24:11.277
looking at a dog or a clock tower, but

00:24:11.360 --> 00:24:13.117
it lost the color, the composition, and

00:24:13.200 --> 00:24:16.077
the detail. So brain it learns structure

00:24:16.160 --> 00:24:18.797
and meaning separately and then puts the

00:24:18.880 --> 00:24:20.877
picture back together. And here's the

00:24:20.960 --> 00:24:23.757
clever part. They also can run the model

00:24:23.840 --> 00:24:26.477
in reverse. An encoder that predicts how

00:24:26.560 --> 00:24:28.957
a brain will respond to an image. So

00:24:29.040 --> 00:24:30.957
they can feed it images no human has

00:24:31.040 --> 00:24:33.597
ever seen into the scanner uh generate

00:24:33.680 --> 00:24:36.637
predicted brain scans and then train on

00:24:36.720 --> 00:24:38.957
those. The model effectively builds its

00:24:39.040 --> 00:24:41.437
own data set. So Richard you know you

00:24:41.520 --> 00:24:43.677
say in your book that AI is superhuman

00:24:43.760 --> 00:24:47.277
in any domain you can simulate or verify

00:24:47.360 --> 00:24:50.637
uh and nowhere else. So is a brain

00:24:50.720 --> 00:24:52.317
simulatable do is the brain a

00:24:52.400 --> 00:24:54.557
simulatable domain or is this something

00:24:54.640 --> 00:24:57.997
different? It's not yet uh but this is

00:24:58.080 --> 00:25:02.077
still an incredible result. I still
remember the first such result came out

00:25:02.160 --> 00:25:05.677
when I was still a PhD student. I went
over to the bioinformatics department at

00:25:05.760 --> 00:25:09.997
Stanford and they you know it's like
very grainy little images we could like

00:25:10.080 --> 00:25:14.797
extract uh from this and to see this
fidelity now and this realism is just

00:25:14.880 --> 00:25:17.677
incredible. I think one of the problems,
I don't know if this study, I haven't

00:25:17.760 --> 00:25:21.037
seen it today yet, have been in the work,
but like um one of the problems is

00:25:21.120 --> 00:25:25.277
often that you have to train for each
brain because they're all slightly

00:25:25.360 --> 00:25:28.957
different. Uh and so you won't be able
to just take someone who've never put

00:25:29.040 --> 00:25:33.757
into an fMRI scanner to see and update
the model for that particular person,

00:25:33.840 --> 00:25:36.797
which would be otherwise amazing
for people who are like in a coma and you

00:25:36.880 --> 00:25:41.197
want to see are they still thinking stuff,
you know, like that would be incredible.

00:25:41.280 --> 00:25:42.957
>> Alex, tell us more
about this about this

00:25:43.040 --> 00:25:45.037
story today. Yeah. So this actually this

00:25:45.120 --> 00:25:48.317
is a result from March. Uh so this the

00:25:48.400 --> 00:25:51.277
broader field of doing decoding of brain

00:25:51.360 --> 00:25:54.637
states from functional imaging or EEG or

00:25:54.720 --> 00:25:56.557
electrodes. This has a long long history

00:25:56.640 --> 00:25:59.197
at this point. I remember 20 plus years

00:25:59.280 --> 00:26:01.677
ago at this point the the Galant lab at

00:26:01.760 --> 00:26:04.397
UC Berkeley was doing this to decode the

00:26:04.480 --> 00:26:07.277
the visual cortex of cats. Actually like

00:26:07.360 --> 00:26:11.917
20 plus years ago we were starting
to see images of how a cat perceives the

00:26:12.000 --> 00:26:15.117
world. And I remember like the first
grainy images that were coming out were

00:26:15.200 --> 00:26:17.677
like a cat seeing a branch. And then

00:26:17.760 --> 00:26:22.317
moving fast forwarding to non-invasive
fMRI. There are so many groups at this

00:26:22.400 --> 00:26:25.757
point. Meta in particular has been
sponsoring quite a bit of work in this

00:26:25.840 --> 00:26:30.717
area from Jean Marie King, many other
groups doing language decoding, vision

00:26:30.800 --> 00:26:34.237
decoding. More recently,
even fast forwarding to

00:26:34.320 --> 00:26:37.117
just the past 24 hours, Neurolink just

00:26:37.200 --> 00:26:38.877
announced their first scaling law

00:26:38.960 --> 00:26:41.997
studies on pre-training foundation

00:26:42.080 --> 00:26:45.997
models, Frontier models off of Neurolink

00:26:46.080 --> 00:26:48.637
electronic data, and this is on an

00:26:48.720 --> 00:26:50.957
individual patient basis, but they're

00:26:51.040 --> 00:26:54.237
capturing copious amounts of of data at

00:26:54.320 --> 00:26:56.237
high temporal resolution, much higher

00:26:56.320 --> 00:26:59.037
than fMRI can capture. fMRI like rule of

00:26:59.120 --> 00:27:01.517
thumb fMRI can give you with today's

00:27:01.600 --> 00:27:03.997
technology at best cubic millimeter

00:27:04.080 --> 00:27:06.717
spatial resolution for voxels and at

00:27:06.800 --> 00:27:09.037
best approximately 1 second temporal

00:27:09.120 --> 00:27:13.757
resolution. So that somewhat limits how
well you can do in terms of decoding a

00:27:13.840 --> 00:27:17.677
person's internal visual state.
can still make no mistake there are folks

00:27:17.760 --> 00:27:19.917
who have are making a cottage industry

00:27:20.000 --> 00:27:23.277
out of decoding dreams uh and decoding

00:27:23.360 --> 00:27:27.757
visually what a person perceives or what
they hallucinate in their visual cortex

00:27:27.840 --> 00:27:31.597
while they're sleeping or while they're
awake and I'm sure this is going to be a

00:27:31.680 --> 00:27:34.077
vibrant vibrant space training frontier

00:27:34.160 --> 00:27:36.237
models and foundation models off of

00:27:36.320 --> 00:27:39.197
copious amounts of fMRI data but

00:27:39.280 --> 00:27:41.597
ultimately where I think this has to go

00:27:41.680 --> 00:27:43.837
there are spatial and temporal limits to

00:27:43.920 --> 00:27:47.037
fMRI decoding It's going to require

00:27:47.120 --> 00:27:49.437
higher temporal precision and higher

00:27:49.520 --> 00:27:51.757
spatial precision to get where we really

00:27:51.840 --> 00:27:54.717
want to go, which is full dive VR and

00:27:54.800 --> 00:27:59.197
full bandwidth BCIs. And we will get
there, but the good news is you have to

00:27:59.280 --> 00:28:03.677
start somewhere. The large language
models had to start with GPT1, just a

00:28:03.760 --> 00:28:08.317
single artificial neuron that could
predict the polarity of Amazon reviews.

00:28:08.400 --> 00:28:10.637
Similarly here, if we're going to get

00:28:10.720 --> 00:28:13.117
full bandwidth BCIs, it's going to start

00:28:13.200 --> 00:28:15.917
with studies like Brain It and John

00:28:16.000 --> 00:28:19.037
Marie King and Jackalant. Amazing. Sem,

00:28:19.120 --> 00:28:23.437
what what happens in the world where
you can know someone's thoughts?

00:28:23.520 --> 00:28:25.597
>> Well, I think you you Let's be careful.

00:28:25.680 --> 00:28:29.357
We're decoding the perception here,
not the actual thought. Right. That's like

00:28:29.440 --> 00:28:31.757
quite a bit more comp. What's the other

00:28:31.840 --> 00:28:36.957
than the brain regionalization
without a difference?

00:28:37.040 --> 00:28:39.357
>> No. No. I think there
there is a difference.

00:28:39.440 --> 00:28:43.597
>> No, when when you imagine, right? When
you imagine something in your mind, it

00:28:43.680 --> 00:28:46.877
lights up the same neurons
as when you're seeing them.

00:28:46.960 --> 00:28:50.877
>> That that's fine. That that's fine.
Just I just want to make sure we don't mix

00:28:50.960 --> 00:28:54.397
the two because you can have thoughts
without necessarily the perceptions

00:28:54.480 --> 00:28:57.197
around it. But you can definitely have
one without the other and the other

00:28:57.280 --> 00:29:01.597
without the other. So uh but I think
for me the more interesting thing here is

00:29:01.680 --> 00:29:05.997
you know we've spent billions of hours
typing things into little rectangles and

00:29:06.080 --> 00:29:10.477
if we can change that interface to do
a more natural interface which is

00:29:10.560 --> 00:29:13.837
interface directly with the brain.
We used to talk about this in our

00:29:13.920 --> 00:29:17.837
singularity university lectures
on neuroscience is you know we still have

00:29:17.920 --> 00:29:21.037
very little idea how the brain works but
you don't need to have know how it works

00:29:21.120 --> 00:29:25.037
as long as you can interface effectively
with it and this I think gives us the

00:29:25.120 --> 00:29:29.517
opening to give us really deep
interfaces and then that'll help us

00:29:29.600 --> 00:29:31.917
figure out how the the brain works.
I >> I'll take the

00:29:32.000 --> 00:29:33.997
>> you bring up a really
Oh go ahead Richard. Sorry.

00:29:34.080 --> 00:29:37.917
>> Sorry. You bring up a really good point
which is I I read this result a while

00:29:38.000 --> 00:29:39.437
back that there are two types of people.

00:29:39.520 --> 00:29:44.157
Some people who think actually think
in sentences and other people when they

00:29:44.240 --> 00:29:47.917
think they just have a fuzzy thought
cloud and only once they're asked to

00:29:48.000 --> 00:29:49.437
verbalize their thought or they try to

00:29:49.520 --> 00:29:53.037
write it down they actually like make a

00:29:53.120 --> 00:29:56.957
real sentence out of their fuzzy thought
clouds. And I'm definitely a thought

00:29:57.040 --> 00:30:01.437
cloud thinker. My wife is very much
a like sentence like in her head. She

00:30:01.520 --> 00:30:05.597
actually thinks in actual sentences
and like there's no there's no like one is

00:30:05.680 --> 00:30:09.437
smarter than the other anything. there's
like they're like equivalent. But like I

00:30:09.520 --> 00:30:13.357
do think sometimes like writing
is thinking and like for me it is very

00:30:13.440 --> 00:30:17.037
helpful if I have to really verbalize
something versus I just have the

00:30:17.120 --> 00:30:20.557
thought. I'm just trying to think like
what does the world look like if the eye

00:30:20.640 --> 00:30:23.757
could really just like extract
my thoughts and now I need to like

00:30:23.840 --> 00:30:27.437
structure my thought clouds into
sentences more and more precise.

00:30:27.520 --> 00:30:30.877
>> Richard, what would neural net training
look like if you took all text out of

00:30:30.960 --> 00:30:35.357
it? No language at all. just train
on pure images and higher level thought

00:30:35.440 --> 00:30:39.117
constructs, you you'd get a very
different neural net out the other side

00:30:39.200 --> 00:30:42.477
and might actually think a lot more like
you do and less like your wife does. Cuz

00:30:42.560 --> 00:30:45.037
>> I'm not sure. >> Maybe
maybe we should be taking the

00:30:45.120 --> 00:30:51.400
position that anyone who doesn't have
an inner monologue is just a pea zombie

00:30:51.840 --> 00:30:53.357
>> there.
I got to throw one more thing in.

00:30:53.440 --> 00:30:56.957
I remember we had one of the top
linguists in the world come in and speak

00:30:57.040 --> 00:31:00.077
and we were like, you know, this must
be a bad time to be a linguist. nobody's

00:31:00.160 --> 00:31:04.077
using spelling or grammar or anything
else like that. And he goes, "No, on the

00:31:04.160 --> 00:31:06.717
contrary, this is one of the most
interesting times to be a linguist

00:31:06.800 --> 00:31:08.797
ever." And we're like, "Wow, how come?"

00:31:08.880 --> 00:31:12.477
And he said, "Cuz when you look
at emojis, it's the first time in the

00:31:12.560 --> 00:31:16.957
history of human language that you can
symbolically send emotion because you

00:31:17.040 --> 00:31:20.317
can digitally send emotions." And we're
like, "Wow, that's an interesting." So,

00:31:20.400 --> 00:31:24.237
he was super excited by opening up
that whole aperture. I thought that was an

00:31:24.320 --> 00:31:27.117
interesting >> You're
saying you're saying you're

00:31:27.200 --> 00:31:31.357
saying that emojis are the reason
why linguistics and not the fact that

00:31:31.440 --> 00:31:36.397
linguistics itself has been flourished by
>> I'm not connect I'm not connecting to

00:31:36.480 --> 00:31:38.637
you did that though it's not me >> Dave

00:31:38.720 --> 00:31:40.797
>> oh no the the result
I really am looking

00:31:40.880 --> 00:31:44.877
forward to is uh how does the brain

00:31:44.960 --> 00:31:49.437
>> train itself without gradient descent
everything going on in AI right now

00:31:49.520 --> 00:31:52.717
everything going on for the last
20 years in AI is driven by gradient

00:31:52.800 --> 00:31:55.757
descent algor algorithms and the
biologists can't find anything even

00:31:55.840 --> 00:32:00.637
vaguely like that in actual biology.
And so that that's you know I think with

00:32:00.720 --> 00:32:04.877
with really detailed imaging we might
finally crack the code on what is the

00:32:04.960 --> 00:32:09.437
fundamental learning algorithm that
changes the synaptic weights. It's not

00:32:09.520 --> 00:32:13.597
it's not what we use for artificial
neural nets. It's it's something

00:32:13.680 --> 00:32:16.877
different and if we discover that we
might find that neural net training can

00:32:16.960 --> 00:32:20.717
be 10 100 a thousand a million times
more efficient. And so hopefully that'll

00:32:20.800 --> 00:32:25.037
come out within a year. Could you define
gradient descent for our listeners?

00:32:25.120 --> 00:32:28.717
>> It's funny, Ilia Sutskiver uh when when
he's on interviews, he's like there is

00:32:28.800 --> 00:32:33.277
only one algorithm. The algorithm
is gradient descent. Everything else is

00:32:33.360 --> 00:32:35.597
just irrelevant compared to So what

00:32:35.680 --> 00:32:40.557
happens right now is you build these 96
or 120 layer deep neural nets and all it

00:32:40.640 --> 00:32:43.597
is is a whole bunch of random
connections that do absolutely nothing

00:32:43.680 --> 00:32:45.997
useful and then you give it trillions of

00:32:46.080 --> 00:32:50.637
15 trillion training examples. You're
like, "When you see this, say that or

00:32:50.720 --> 00:32:52.877
when you see this paragraph, this is the

00:32:52.960 --> 00:32:55.517
next token." And if you're guessing

00:32:55.600 --> 00:32:57.677
wrong, you get punished through gradient

00:32:57.760 --> 00:33:02.877
descent. So the error or the how far off
you are is your punishment and it passes

00:33:02.960 --> 00:33:06.237
back through all the layers and it
calculates an error and it blames each

00:33:06.320 --> 00:33:09.037
neuron or each connection for how how

00:33:09.120 --> 00:33:11.037
responsible were you for this terrible

00:33:11.120 --> 00:33:16.077
answer. And if you're way wrong, you get
moved and you move in the direction that

00:33:16.160 --> 00:33:20.077
helps you get the answer right. So it's
this incredibly laborious search process

00:33:20.160 --> 00:33:24.237
and that movement of each synaptic
connection is a gradient and you just

00:33:24.320 --> 00:33:26.797
you just hill climb to the best possible

00:33:26.880 --> 00:33:29.597
state and then it magically after about

00:33:29.680 --> 00:33:34.317
hund00 million of comput in Richard's
case I guess 400 million uh it magically

00:33:34.400 --> 00:33:37.917
starts thinking through this one
algorithm gradient descent.

00:33:38.000 --> 00:33:41.757
>> Yeah I think it's really back back prop
that we're talking about syndictoically

00:33:41.840 --> 00:33:45.677
rather than gradient descent would be
the the obvious comment. So maybe yeah

00:33:45.760 --> 00:33:49.357
back prop is I guess the efficient
computation of those gradients. Uh maybe

00:33:49.440 --> 00:33:54.397
I'll I'll add to um add to this
explanation from Dave but like gradient

00:33:54.480 --> 00:33:57.917
is essentially an optimization algorithm
that we use to train all kinds of

00:33:58.000 --> 00:34:01.597
machine learning models by minimizing
the errors. And I think the best analogy

00:34:01.680 --> 00:34:04.477
is to assume you're a blind hiker.

00:34:04.560 --> 00:34:07.277
You're the top of the mountain
and you're trying to find the lowest the

00:34:07.360 --> 00:34:09.277
lowest point and you can't really see

00:34:09.360 --> 00:34:11.197
that far. You're blind but you can feel

00:34:11.280 --> 00:34:14.637
the slope at each step. And so how much

00:34:14.720 --> 00:34:18.397
like how big of a step should you take
down when you feel like the slope is

00:34:18.480 --> 00:34:19.597
going roughly in the right direction.

00:34:19.680 --> 00:34:23.757
And then the problem is that the kinds
of landscapes these AIs are trying to

00:34:23.840 --> 00:34:29.197
optimize are highly non-convex which
means there's not just one lowest point

00:34:29.280 --> 00:34:31.757
uh but there are many different low
points. And if you depending on where

00:34:31.840 --> 00:34:35.197
you start you might go into a different
valley depending on which mountain you

00:34:35.280 --> 00:34:38.637
start you go into this valley versus
this this different valley. And there

00:34:38.720 --> 00:34:42.397
are actually a lot of really interesting
analogies. My friend Jihan uh like

00:34:42.480 --> 00:34:45.757
thinks about this from the psychological
perspective. For instance, when you have

00:34:45.840 --> 00:34:48.877
PTSD and you train, you overfit to one

00:34:48.960 --> 00:34:53.037
algorithm. how your brain is kind
of stuck in one valley of how to think

00:34:53.120 --> 00:34:57.677
about something. And it takes sometimes
people like doing psychedelics and stuff

00:34:57.760 --> 00:34:59.677
uh that have shown to to help help with

00:34:59.760 --> 00:35:02.157
like depression and PTSD and other uh

00:35:02.240 --> 00:35:06.717
mental diseases like um they they it
helps to increase the learning rate to

00:35:06.800 --> 00:35:10.717
jump over another mountain and and go
into a different valley of attraction

00:35:10.800 --> 00:35:14.877
where you then have new kinds of ways
of thinking. And so this blind hiker

00:35:14.960 --> 00:35:18.237
analogy I think is a really intuitive
way to think about it. And then of

00:35:18.320 --> 00:35:22.877
course the problem is it's not just in 3D,
it's in a trilliond like a trillion

00:35:22.960 --> 00:35:26.397
parameters that you're trying
to traverse in that landscape. But the

00:35:26.480 --> 00:35:30.237
ideas of like try to identify the
direction and take steps towards it still.

00:35:30.320 --> 00:35:32.557
>> And you know what else is really
incredible? I could grip on this for

00:35:32.640 --> 00:35:34.317
hours, but if you look at a big model

00:35:34.400 --> 00:35:38.157
like Kimmy K3, it's 93 layers and it's

00:35:38.240 --> 00:35:42.557
trained trained trained. If you take
a single layer and you randomize it and

00:35:42.640 --> 00:35:44.557
then you train it and say find yourself

00:35:44.640 --> 00:35:48.077
again, it can't find itself again. It

00:35:48.160 --> 00:35:51.837
never gets back to where it was. Which
comes back to in these fMRIs when you're

00:35:51.920 --> 00:35:55.037
looking at a human brain and you're
saying, "Okay, here's Alex's brain. Can

00:35:55.120 --> 00:35:58.797
I compare that to Salem's?" And you're
like, "Wow, there's nothing in common

00:35:58.880 --> 00:36:01.997
going on here yet." you know, well,

00:36:02.080 --> 00:36:06.840
okay, maybe that's a bad example, but but

00:36:08.880 --> 00:36:12.957
the way your brain wires as you learn
is completely unique to you. Yet, yet it

00:36:13.040 --> 00:36:16.477
can come out with like, okay, we're
equally good soccer players, but the way

00:36:16.560 --> 00:36:20.477
the signals are propagating through
our networks is completely different and

00:36:20.560 --> 00:36:24.397
unique to each of us, which is really
really strange, you know, that that it

00:36:24.480 --> 00:36:28.717
can't it can't find its own way back
to its its state. I think we're going to

00:36:28.800 --> 00:36:33.437
learn a lot actually with the fMRI data
coming in on how and why that works. And

00:36:33.520 --> 00:36:36.957
there there have to be commonalities
that we just can't find. You know,

00:36:37.040 --> 00:36:39.517
rotations, you know, make everything
look different, but they're really not

00:36:39.600 --> 00:36:43.357
that different. If you if you put them
through a transform, like a 4A transform

00:36:43.440 --> 00:36:46.397
view will suddenly say, "Oh my god,
this is why they line up."

00:36:46.480 --> 00:36:48.077
>> Yeah. I was I was just with my

00:36:48.160 --> 00:36:51.117
fraternity brother uh of ours, Dave,

00:36:51.200 --> 00:36:53.837
who's the head of neurobiology at at USC

00:36:53.920 --> 00:36:59.117
the other day. um and is you know this
is the most exciting time ever for brain

00:36:59.200 --> 00:37:02.797
science >> right I mean
we're you know the brain

00:37:02.880 --> 00:37:06.797
has been a black box for for since
humanity began and our ability to

00:37:06.880 --> 00:37:08.317
understand

00:37:08.400 --> 00:37:12.157
uh the brain and deal with with uh you

00:37:12.240 --> 00:37:16.557
know mental disease I think is going to be
extraordinary. Yeah. Well, so you can

00:37:16.640 --> 00:37:20.077
simulate. This is why liquid AI
was founded actually because we completely

00:37:20.160 --> 00:37:22.397
reverse engineered the the C elegance

00:37:22.480 --> 00:37:26.237
worm brain down to exactly every single

00:37:26.320 --> 00:37:30.557
thing going on and then they were
able to simulate it and then after they

00:37:30.640 --> 00:37:33.997
simulated it they realized wait this
is a very efficient neural net and then

00:37:34.080 --> 00:37:38.077
they productized it. I mean that's
so the both ang so go ahead Richard

00:37:38.160 --> 00:37:40.877
>> there's a really cool thing since you
mentioned se against worms there's not

00:37:40.960 --> 00:37:43.197
many people that bring that up
a friend of mine who's a professor of

00:37:43.280 --> 00:37:47.037
neuroscience at Harvard Sam Gershman uh
he one of the most brilliant people I've

00:37:47.120 --> 00:37:49.597
ever met like he actually did a a study

00:37:49.680 --> 00:37:54.877
with with worms where he cut them
in half and one segance worms they do have

00:37:54.960 --> 00:37:58.637
a brain right and then they have a lot
of other like neurons in the rest now

00:37:58.720 --> 00:38:01.117
you can train them to react to a certain

00:38:01.200 --> 00:38:03.837
stimuli um and then you split them in

00:38:03.920 --> 00:38:08.477
half the the second half without the brain
regrows a new brain. And here, get

00:38:08.560 --> 00:38:12.077
this, this is crazy.
That new brain has the same memories.

00:38:12.160 --> 00:38:15.037
>> Wow. >> And reacts
to different stimuli. And so

00:38:15.120 --> 00:38:18.717
he's like, maybe there's a different way
where we learn. And I do think there are

00:38:18.800 --> 00:38:21.917
things that right now we're very much
stuck in this like it's a neural net,

00:38:22.000 --> 00:38:23.837
it's all electrical signals and so on.

00:38:23.920 --> 00:38:27.837
But clearly like brain chemistry can
change massively with just like you can

00:38:27.920 --> 00:38:31.917
get hangry, you can be in pain, you can
have like a certain like stimulant and

00:38:32.000 --> 00:38:35.357
all of a sudden your brain is very
different. And I think there are and

00:38:35.440 --> 00:38:37.517
I've seen this now with uh like uh

00:38:37.600 --> 00:38:42.477
having babies and and like talking to uh
moms like there's certain algorithms

00:38:42.560 --> 00:38:45.757
where you all of a sudden you nest
because you had a baby and now you want

00:38:45.840 --> 00:38:50.077
to like and how you nest is different
but people nest you know in one form or

00:38:50.160 --> 00:38:51.757
another and so there are these latent

00:38:51.840 --> 00:38:53.597
algorithms that we have that are in our

00:38:53.680 --> 00:38:57.357
DNA that just trigger after like 20

00:38:57.440 --> 00:39:00.717
something years right when just the right
things then happen with your body

00:39:00.800 --> 00:39:04.557
and your biology. So, I think there's
still so much that we don't we haven't

00:39:04.640 --> 00:39:08.157
>> so much complexity. I'm going to move
us to our next story from three days ago.

00:39:08.240 --> 00:39:12.957
So, on Tuesday, uh the president signed
an executive order directing the EPA,

00:39:13.040 --> 00:39:17.357
the Department of the Interior
and Agriculture working with HHS to cut

00:39:17.440 --> 00:39:19.517
invasive mosquito populations in

00:39:19.600 --> 00:39:23.117
Washington DC by at least 90% and the

00:39:23.200 --> 00:39:27.117
tick population by at least 50% by 2028.

00:39:27.200 --> 00:39:31.197
Uh, the targets include mosquito species
that spread deni, zika, and yellow

00:39:31.280 --> 00:39:33.677
fever. And here's what caught my eye.

00:39:33.760 --> 00:39:38.317
You know, the order explicitly
prioritizes sterile insect techniques,

00:39:38.400 --> 00:39:42.477
safe genetic modifications,
and beneficial bacteria over conventional

00:39:42.560 --> 00:39:46.557
pesticides, right? I love that.
You know, this is the exact kind of uh gan

00:39:46.640 --> 00:39:49.997
drive work that Colossal,
the deextinction company, is doing. You

00:39:50.080 --> 00:39:53.197
know, I think most people don't realize
it's like, you know, when I was raising

00:39:53.280 --> 00:39:56.557
my kids, I would ask them a question,
who which which is the species that

00:39:56.640 --> 00:40:00.557
kills the most humans on the planet, you
know, and some people jump onto sharks?

00:40:00.640 --> 00:40:04.557
Some people jump onto, you know,
whatever it might be. But, uh, on this

00:40:04.640 --> 00:40:06.237
part, I'm sure people know mosquitoes

00:40:06.320 --> 00:40:08.957
are the deadliest life form on Earth,

00:40:09.040 --> 00:40:11.757
>> right? Malaria alone, uh, kills half a

00:40:11.840 --> 00:40:15.597
million people a year. So, uh, Alex, u,

00:40:15.680 --> 00:40:18.797
this is biology as engineering arriving

00:40:18.880 --> 00:40:21.597
as federal policy. Uh, your thoughts on

00:40:21.680 --> 00:40:25.837
this story? For decades, we've been
scared of our shadow. So, I have a

00:40:25.920 --> 00:40:28.157
classmate Kevin Esfelt at MIT who was

00:40:28.240 --> 00:40:29.997
the pioneer of the GAN drive, one of the

00:40:30.080 --> 00:40:33.277
pioneers, and he's had a devil of a time

00:40:33.360 --> 00:40:37.437
getting states, municipalities
to approve gan drive studies against

00:40:37.520 --> 00:40:39.677
mosquitoes. And for those not tracking,

00:40:39.760 --> 00:40:41.517
the the premise of this technique is

00:40:41.600 --> 00:40:44.637
basically inserting a gene via crisper

00:40:44.720 --> 00:40:46.957
that wants to replicate itself to

00:40:47.040 --> 00:40:49.917
basically sterilize mosquitoes through

00:40:50.000 --> 00:40:54.717
propagating virally through the mosquito
population. We have the technology to do

00:40:54.800 --> 00:40:59.437
this. What it has lacked, at least
in this country, is a federal mandate to

00:40:59.520 --> 00:41:03.437
actually go and implement large-scale
genetic engineering of non-human animal

00:41:03.520 --> 00:41:05.837
populations. And now for the first time

00:41:05.920 --> 00:41:09.437
it seems we're seeing via executive fiat

00:41:09.520 --> 00:41:11.917
just that mandate. And it is so exciting

00:41:12.000 --> 00:41:14.957
for two reasons. One because one is

00:41:15.040 --> 00:41:18.237
Peter you mentioned malaria and other

00:41:18.320 --> 00:41:21.517
non-human animalborn diseases cause an

00:41:21.600 --> 00:41:25.517
an enormous amount of suffering human
suffering and also non-human animal

00:41:25.600 --> 00:41:28.877
suffering. But secondly because there's
a way to do this that doesn't actually

00:41:28.960 --> 00:41:30.637
involve killing the animals themselves.

00:41:30.720 --> 00:41:34.797
Historically, historically,
you know, if you look back

00:41:34.880 --> 00:41:37.437
50 years, you'd see chemical spraying if

00:41:37.520 --> 00:41:39.357
you wanted to do something about say

00:41:39.440 --> 00:41:42.957
insect born illness

00:41:43.040 --> 00:41:46.957
uh responses. We don't need to do
that anymore. We can actually keep the

00:41:47.040 --> 00:41:48.717
insects that are the inadvertent

00:41:48.800 --> 00:41:51.997
carriers of bacterial or viral disease.

00:41:52.080 --> 00:41:55.917
We can actually preserve their lives
while also preventing the disease

00:41:56.000 --> 00:41:59.917
transmission. And I think that's good
from their perspective as well. That was

00:42:00.000 --> 00:42:04.317
the part that got me most excited was we
spray things, we're basically poisoning

00:42:04.400 --> 00:42:07.757
biology now we actually
>> ecosystem, right?

00:42:07.840 --> 00:42:09.357
>> And leave the poison ecosystem.

00:42:09.440 --> 00:42:13.677
>> Yeah. Yeah. >> Uh Richard,
your thoughts on this story?

00:42:13.760 --> 00:42:15.517
>> I I concur with everyone like it's it's

00:42:15.600 --> 00:42:18.717
it's wonderful. Uh and it's so

00:42:18.800 --> 00:42:23.757
interesting like people I I hope we
can amplify these these kinds of stories

00:42:23.840 --> 00:42:27.517
more, you know, like the future needs
better marketing. We need more Peters in

00:42:27.600 --> 00:42:31.837
the world. Um and and this is one
of those many stories, you know, curing

00:42:31.920 --> 00:42:36.237
various diseases, peril bio, like saving
animal lives and and prevent them from

00:42:36.320 --> 00:42:40.477
just being bred uh to be tested upon
and dissected like uh those are all like

00:42:40.560 --> 00:42:44.157
stories we need to amplify more in the
public eye. Yeah, Dave, I can't wait for

00:42:44.240 --> 00:42:49.117
this to come from DC to Vermont
and Massachusetts and all of these

00:42:49.200 --> 00:42:53.117
>> I was going to say for the listeners
that are interested in your real estate

00:42:53.200 --> 00:42:55.117
fund, which you know the the theme there

00:42:55.200 --> 00:42:57.917
is, hey, easy access via drone to

00:42:58.000 --> 00:43:02.717
hilltops, islands, but also if you
live on the edge of a marsh, your house

00:43:02.800 --> 00:43:06.157
probably sells for about half the price
of a place that's not on the edge of a

00:43:06.240 --> 00:43:08.797
marsh, that'll go away. there there's so

00:43:08.880 --> 00:43:11.757
many solutions coming to biting insects,

00:43:11.840 --> 00:43:15.837
this being one of them. Uh that that also
is is, you know, if it's beautiful,

00:43:15.920 --> 00:43:18.637
beautiful land that otherwise is very
difficult to live on, it's going to go

00:43:18.720 --> 00:43:21.277
through the roof in value.
>> I do think it's really important to try

00:43:21.360 --> 00:43:25.677
to like, you know, keep the bees uh
like alive, all right, and not have spray

00:43:25.760 --> 00:43:29.197
bunch of pesticides, make sure birds
still have enough like insects to eat

00:43:29.280 --> 00:43:33.117
and things like that. Yeah,
>> I was going to say for maybe up to 80

00:43:33.200 --> 00:43:35.277
years, humanity has been scared of its

00:43:35.360 --> 00:43:40.397
own shadow. humanity in general,
America or the West in particular, nuclear

00:43:40.480 --> 00:43:44.797
energy. Another example I've pointed
out on the pod where we arguably lost 50

00:43:44.880 --> 00:43:47.037
plus years of progress because we were

00:43:47.120 --> 00:43:50.317
too scared of either the bomb or fision

00:43:50.400 --> 00:43:52.877
reactors or some an entire generation

00:43:52.960 --> 00:43:54.957
watched the movie China Syndrome and

00:43:55.040 --> 00:43:57.597
then got scared of nuclear power

00:43:57.680 --> 00:44:01.357
unnecessarily. Same idea with this or or

00:44:01.440 --> 00:44:05.437
with geoengineering. Another example,
the the world is still scared. Some

00:44:05.520 --> 00:44:08.477
fraction of the world is scared
of engineering the weather. We don't need

00:44:08.560 --> 00:44:10.397
to be scared of engineering our physical

00:44:10.480 --> 00:44:13.197
world or our biological world. And now

00:44:13.280 --> 00:44:17.437
hopefully one can see green shoots
of humanity getting past that stage of

00:44:17.520 --> 00:44:19.197
worrying about its own shadow.

00:44:19.280 --> 00:44:23.597
>> This is interesting that Sorry. Yeah,
it's really it's interesting indeed that

00:44:23.680 --> 00:44:26.717
like sometimes alarmists feel like
they're doing the right thing by just

00:44:26.800 --> 00:44:30.317
like oh how bad could it be? I'm warning
people of something bad. But like it can

00:44:30.400 --> 00:44:33.757
indeed push all of humanity away
from something really good like energy

00:44:33.840 --> 00:44:36.397
abundance uh with nuclear. Um I think

00:44:36.480 --> 00:44:40.877
overpopulation is when these other big
myths where people are like well if we

00:44:40.960 --> 00:44:43.357
have too many people there's going
to be scarcity of all these different

00:44:43.440 --> 00:44:46.877
resources and everything is going
to get more expensive and people are more

00:44:46.960 --> 00:44:50.157
poverty and so on and the exact opposite
happened. There's actually one website

00:44:50.240 --> 00:44:54.397
that I think you all would love which
called humanprogress.org or.com like

00:44:54.480 --> 00:44:57.117
that that shows that a lot of these
things are actually getting cheaper and

00:44:57.200 --> 00:44:59.037
cheaper despite there being more people.

00:44:59.120 --> 00:45:01.517
>> Yeah. >> Well, hey Richard,
you're you're a hero

00:45:01.600 --> 00:45:05.837
in Germany like like can't walk down the
street kind of hero in Germany. What do

00:45:05.920 --> 00:45:09.917
you think about the fact that Germany
has no power for exactly this reason?

00:45:10.000 --> 00:45:14.157
>> It is really unfortunate if there are
a lot of people in Germany that want to in

00:45:14.240 --> 00:45:17.917
a weird way sort of offramp from progress.
Um and they think that

00:45:18.000 --> 00:45:21.277
everything that consumes power is bad
for the environment. there's like a

00:45:21.360 --> 00:45:24.797
weird sort of degrowth uh offshoot from

00:45:24.880 --> 00:45:27.517
generally like well-intentioned pro-

00:45:27.600 --> 00:45:31.437
like environmental uh vibes that uh just

00:45:31.520 --> 00:45:35.437
worry me um worry me quite a bit. Yeah,
>> we're going to talk about dumerism in a

00:45:35.520 --> 00:45:39.437
little bit uh but not yet. And you know,
one of the things I realized a long time

00:45:39.520 --> 00:45:42.317
ago is we humans are really incredible

00:45:42.400 --> 00:45:44.717
at seeing a problem out in the future,

00:45:44.800 --> 00:45:47.277
right? We see acid rain, we see

00:45:47.360 --> 00:45:52.397
overpopulation, we see, you know, energy
shortage, whatever it might be. And then

00:45:52.480 --> 00:45:56.077
because of our amigdula, because the way
we think, we accelerate that future

00:45:56.160 --> 00:46:00.957
problem to today. And we freak out
and we forget the fact that there is a

00:46:01.040 --> 00:46:05.437
decade worth of progress we're going
to make by the time we reach that problem.

00:46:05.520 --> 00:46:08.557
And that's what entrepreneurs do.
They solve problem after problem after

00:46:08.640 --> 00:46:11.677
problem. And I I guess I just want our

00:46:11.760 --> 00:46:15.997
listeners to hear that because if
you're worried about some future problem,

00:46:16.080 --> 00:46:18.637
please understand there's a incredibly

00:46:18.720 --> 00:46:21.997
efficient market uh of entrepreneurship

00:46:22.080 --> 00:46:25.677
and capitalism that will solve
the world's biggest problems, the world's

00:46:25.760 --> 00:46:27.437
biggest business opportunities, right?

00:46:27.520 --> 00:46:31.117
And it's a beautiful uh forward uh you

00:46:31.200 --> 00:46:33.997
know uh uh forward propagation solution

00:46:34.080 --> 00:46:38.477
set that we have. And maybe we'll get
there later, but I I see the same thing

00:46:38.560 --> 00:46:42.477
happen in AI dumerism where they create
more and more complex scenarios where

00:46:42.560 --> 00:46:46.957
attackers get these like near magical
abilities to attack, but somehow the

00:46:47.040 --> 00:46:50.797
defenders in those stories never get
near magical abilities to defend. And

00:46:50.880 --> 00:46:54.797
it's this weird thing of like, man,
if I can create a super virus that is like

00:46:54.880 --> 00:46:59.517
perfectly undetectable and spreads all
throughout the world and no one notices

00:46:59.600 --> 00:47:03.117
it and like and then it has this Wi-Fi
switch and you just turn it on and off

00:47:03.200 --> 00:47:06.957
and kill people like I I will create
a super magical vaccine that just

00:47:07.040 --> 00:47:10.477
inoculates you against all of these
things. Like if you can actually assume

00:47:10.560 --> 00:47:13.997
make that assumption then like
it's a weird Yeah. weird thing.

00:47:14.080 --> 00:47:18.237
>> This episode is sponsored by Google for
startups. Think about this for a second.

00:47:18.320 --> 00:47:22.557
You now have access to the same
generative AI models that cost hundreds

00:47:22.640 --> 00:47:24.317
of millions of dollars to train.

00:47:24.400 --> 00:47:28.557
Google's startup technical guide
for generative media gives you complete

00:47:28.640 --> 00:47:31.117
blueprint for deploying Google DeepMinds

00:47:31.200 --> 00:47:33.837
models in production. Images, video,

00:47:33.920 --> 00:47:36.077
audio, all of it. Real architecture,

00:47:36.160 --> 00:47:38.557
real results. Find the link in the show

00:47:38.640 --> 00:47:43.117
notes below. I'm going
to move us to a story that's

00:47:43.200 --> 00:47:45.037
been sort of breaking the internet over

00:47:45.120 --> 00:47:49.197
the last 24 hours. Um, and it it puts

00:47:49.280 --> 00:47:51.517
your regulate the application argument

00:47:51.600 --> 00:47:53.677
to the test, Richard. So, a company

00:47:53.760 --> 00:47:56.637
called Tavis unveiled Griffin, uh, which

00:47:56.720 --> 00:47:59.517
it calls the first model ever to pass

00:47:59.600 --> 00:48:02.397
the video touring test. So, here's the

00:48:02.480 --> 00:48:05.117
numbers. 48% of people who talk to an AI

00:48:05.200 --> 00:48:08.397
avatar on a live face-to-face video

00:48:08.480 --> 00:48:09.757
thought they were talking to a real

00:48:09.840 --> 00:48:13.037
human. Previous systems were at 3%. So,

00:48:13.120 --> 00:48:15.597
let me show the video here. And uh it's

00:48:15.680 --> 00:48:18.717
super cool and let's talk about it next.

00:48:18.800 --> 00:48:22.077
>> Now, how about just a thumbs up?

00:48:22.160 --> 00:48:23.917
>> Sure. One thumbs up coming right up.

00:48:24.000 --> 00:48:27.917
>> Hell yeah. That was awesome. I would
say you're my favorite coworker. It's been

00:48:28.000 --> 00:48:30.237
It's been really great
toh work on this with you.

00:48:30.320 --> 00:48:34.077
>> You're making me blush.
A Who's that with the ugly sweater?

00:48:34.160 --> 00:48:35.837
>> Simon says, "Touch your hair."

00:48:35.920 --> 00:48:37.277
>> Is this what you're looking for?

00:48:37.360 --> 00:48:41.437
>> Yeah. Okay, nice. Simon says point. All

00:48:41.520 --> 00:48:44.317
right, I'm doing it.

00:48:44.400 --> 00:48:47.837
Now, touch your chin.

00:48:47.920 --> 00:48:51.757
>> I'm not going to fall for that.
You didn't say the magic word. Uh,

00:48:51.840 --> 00:48:55.197
>> you got me. Okay.
>> The loose connector back onto the

00:48:55.280 --> 00:49:00.397
motherboard. >> Okay,
let me go let me go grab that. But

00:49:00.480 --> 00:49:03.917
beforehand, yeah, I turned
the soldering iron on. I

00:49:04.000 --> 00:49:06.397
think it's going to take about 12 seconds.
So, maybe just let me know when

00:49:06.480 --> 00:49:10.317
that's done. >> It sounds good.
I'll keep an eye on it

00:49:10.400 --> 00:49:12.797
and let you know the second it hits that

00:49:12.880 --> 00:49:17.757
12 second mark. >> All right.
So, uh, interesting. So,

00:49:17.840 --> 00:49:22.077
Tavis calls it the first human
interaction model. It's fully duplex,

00:49:22.160 --> 00:49:26.077
listening and talking at the same time
that you do, the way humans actually do.

00:49:26.160 --> 00:49:28.397
It's number one on Nvidia's benchmark

00:49:28.480 --> 00:49:31.517
for full duplex AI video. Tavis pitches

00:49:31.600 --> 00:49:33.917
it as quote a tutor for every student

00:49:34.000 --> 00:49:36.637
that notices when they're lost, an elder

00:49:36.720 --> 00:49:41.517
care companion that listens. Uh Tavis
itself says Griffin requires safety work

00:49:41.600 --> 00:49:42.877
before it can be released publicly

00:49:42.960 --> 00:49:45.597
because it's the first model that can be

00:49:45.680 --> 00:49:49.517
mistaken for a real person. Um Salem,

00:49:49.600 --> 00:49:51.917
your thoughts on this one?

00:49:52.000 --> 00:49:56.877
So, I'm less kind of um floored by the

00:49:56.960 --> 00:50:00.317
um the imagery and the the mimicry of

00:50:00.400 --> 00:50:03.917
it. Uh this was expected to happen. For

00:50:04.000 --> 00:50:07.837
me, I'm more interested in can it
accomplish something useful enough that

00:50:07.920 --> 00:50:09.997
I don't care whether it's human or not.

00:50:10.080 --> 00:50:14.317
And I think that that is more
interesting uh which might come along

00:50:14.400 --> 00:50:16.557
and have hap we have that happen at some

00:50:16.640 --> 00:50:21.437
point. We've talked about the idea that
in the near future, like now, uh you're

00:50:21.520 --> 00:50:23.437
going to have digital co-workers that

00:50:23.520 --> 00:50:26.077
pop up on Zoom, that you call, that you

00:50:26.160 --> 00:50:29.117
text, you Slack, and interact with you.

00:50:29.200 --> 00:50:32.557
And I think it's pretty compelling as a

00:50:32.640 --> 00:50:37.357
mechanism for a future, you know,
virtualized company. Uh Alex,

00:50:37.440 --> 00:50:41.037
>> well, first, Peter, you should
probably ask me to touch my face.

00:50:41.120 --> 00:50:44.237
>> Okay, touch your face.
Simon says touch your face.

00:50:44.320 --> 00:50:47.597
>> Oh, >> okay. Very good.
Just checking but I I

00:50:47.680 --> 00:50:51.037
guess Tavis is ahead of me. I I I think

00:50:51.120 --> 00:50:54.877
on the one hand if you look at models
coming out of Chinese labs like

00:50:54.960 --> 00:50:57.837
Alibaba's lab, Juan Streamer gave us a

00:50:57.920 --> 00:51:01.917
preview of what was going to happen.
The Chinese labs remain overinvested

00:51:02.000 --> 00:51:05.837
relative to the Western labs
in generative video models and interactive

00:51:05.920 --> 00:51:08.397
generative video models. So one streamer

00:51:08.480 --> 00:51:10.637
I think is a preview of what's possible.

00:51:10.720 --> 00:51:13.757
My guess is I I I I I read the the

00:51:13.840 --> 00:51:19.517
information that was put out around Tavis.
I I think in full generality, let

00:51:19.600 --> 00:51:22.157
let's just talk about where this
is going to end up. It's it's very

00:51:22.240 --> 00:51:25.917
difficult to predict the short term.
I think it's pretty easy to predict the

00:51:26.000 --> 00:51:29.917
long term. In the long term, endto-end
generative pixels. We'll have these

00:51:30.000 --> 00:51:32.797
magic mirrors that could be real time

00:51:32.880 --> 00:51:35.837
fully interactive pixel-wise generated

00:51:35.920 --> 00:51:39.917
or if if anthropic has its way,
vector-wise or procedurally generated.

00:51:40.000 --> 00:51:42.237
But either way, fully generated

00:51:42.320 --> 00:51:46.077
real-time interactive video models and

00:51:46.160 --> 00:51:49.357
you'll be able to create a scene that

00:51:49.440 --> 00:51:54.237
consists of people like it's possible
right now, but the latency is high. You

00:51:54.320 --> 00:51:55.997
see with one streamer, which is already

00:51:56.080 --> 00:51:59.197
out and already open source, or with

00:51:59.280 --> 00:52:02.477
this Tavis Griffin type model, which is

00:52:02.560 --> 00:52:05.037
not really out yet and definitely not

00:52:05.120 --> 00:52:07.917
open source. You see a preview of the

00:52:08.000 --> 00:52:11.357
future. What does this look like? Well,

00:52:11.440 --> 00:52:13.117
there are a few different ways it can

00:52:13.200 --> 00:52:17.517
go. Query how revenue generating per

00:52:17.600 --> 00:52:20.957
token it is. I is it anywhere close to

00:52:21.040 --> 00:52:23.597
the optimal frontier of code generation?

00:52:23.680 --> 00:52:25.757
Doubt it. On the other hand, if it

00:52:25.840 --> 00:52:28.477
becomes so absurdly inexpensive to be

00:52:28.560 --> 00:52:30.957
able to to generate arbitrary, say,

00:52:31.040 --> 00:52:33.357
humans participating in a Zoom meeting

00:52:33.440 --> 00:52:36.237
or humans participating in a podcast. Do

00:52:36.320 --> 00:52:39.197
we really care whether it's close to to

00:52:39.280 --> 00:52:41.437
being near the optimal cost performance

00:52:41.520 --> 00:52:43.917
frontier? Maybe not. There are a lot of

00:52:44.000 --> 00:52:46.717
human service industry jobs that require

00:52:46.800 --> 00:52:49.437
a face and require interactivity and a

00:52:49.520 --> 00:52:53.997
voice and the ability to touch one's
face apparently on demand or on on

00:52:54.080 --> 00:52:59.197
request that could be probably
completely automated away by a model

00:52:59.280 --> 00:53:03.117
that otherwise would be limited to
textbased interaction but doesn't have a

00:53:03.200 --> 00:53:05.357
face and a voice. So in the most

00:53:05.440 --> 00:53:07.757
optimistic scenario, these sorts of

00:53:07.840 --> 00:53:11.037
interactive video models open PN note

00:53:11.120 --> 00:53:13.917
that the acronym for this which by the

00:53:14.000 --> 00:53:16.877
way was there first with Alex Finn him

00:53:16.960 --> 00:53:20.157
him is such an obvious reference to her

00:53:20.240 --> 00:53:24.957
the the movie closed PN I
I think this this has this

00:53:25.040 --> 00:53:27.917
>> you should use you
should use m dashes please

00:53:28.000 --> 00:53:32.237
>> okay I'll delve into it so I I I
think I I I think this is going to be

00:53:32.320 --> 00:53:33.997
transformative for the service sector

00:53:34.080 --> 00:53:37.597
And hopefully TAVIS and the broader

00:53:37.680 --> 00:53:40.957
American west of video frontier models

00:53:41.040 --> 00:53:44.717
and interactive video frontier models
take a page from Tavis and start

00:53:44.800 --> 00:53:48.637
competing with China. >> Richard,
one of the principles in your

00:53:48.720 --> 00:53:50.317
book, one of the rules you have is that

00:53:50.400 --> 00:53:53.197
AI becomes superhuman uh where you can

00:53:53.280 --> 00:53:56.717
verify the answer. uh and you know is

00:53:56.800 --> 00:53:58.717
like passing a human scientific

00:53:58.800 --> 00:54:03.917
milestone like this uh is passing sort
of a human judgment and interaction uh

00:54:04.000 --> 00:54:08.157
you know sufficiently as a scientific
milestone for verification.

00:54:08.240 --> 00:54:10.397
>> Not quite. Uh it's sort of like it's

00:54:10.480 --> 00:54:12.717
hard to scale when a human has to be in

00:54:12.800 --> 00:54:17.597
the loop to say yes this is like a human
another person on the other side or not.

00:54:17.680 --> 00:54:21.437
Uh you can't quite automate it
completely uh and verify it in in that

00:54:21.520 --> 00:54:25.437
automated fashion. Um I do think you're
you're right. This is a good example

00:54:25.520 --> 00:54:27.757
where I think we need to do be go beyond

00:54:27.840 --> 00:54:31.837
KYC and do KY know your use case. Uh

00:54:31.920 --> 00:54:35.997
right like you don't want this
technology to be in Zoom pretending to

00:54:36.080 --> 00:54:39.677
be the CEO and like there's some famous
stories where this has already happened

00:54:39.760 --> 00:54:43.677
where they got someone to wire like
$20 million cuz they created a Zoom with

00:54:43.760 --> 00:54:46.157
five other executives and they all like

00:54:46.240 --> 00:54:48.877
he was fooled well enough. Um and and so

00:54:48.960 --> 00:54:53.357
I think we need to um yeah be careful
and like Zoom probably and Google

00:54:53.440 --> 00:54:57.117
Hangout need to start like finding
countermeasures to identify is this a

00:54:57.200 --> 00:55:01.197
real person and and things like that uh
to prevent those kinds of hacks from

00:55:01.280 --> 00:55:04.077
happening. >> Yeah, scams
are going to proliferate on

00:55:04.160 --> 00:55:07.277
this. I I just want to do a call out
again. And I've said this before on the

00:55:07.360 --> 00:55:11.197
pod and if you're if you've not heard
this before, if you still have your

00:55:11.280 --> 00:55:13.437
grandparents with you or your parents

00:55:13.520 --> 00:55:16.317
and you've not taken the time to record

00:55:16.400 --> 00:55:19.997
their stories on video and audio and go

00:55:20.080 --> 00:55:22.717
deep, spend, you know, hours recording

00:55:22.800 --> 00:55:26.237
them, right? Your ability to create a

00:55:26.320 --> 00:55:29.117
super highresolution lifelike avatar for

00:55:29.200 --> 00:55:33.597
your your kids or your grandkids
or your great-grandkids, I think, is an

00:55:33.680 --> 00:55:35.837
incredible thing. But you need the data.

00:55:35.920 --> 00:55:39.677
So, if you're listening to this and you're
lucky enough to have your parents

00:55:39.760 --> 00:55:43.117
or grandparents around, collect
their data because it's going to be a

00:55:43.200 --> 00:55:46.877
beautiful opportunity for your uh
for your progyny and theirs.

00:55:46.960 --> 00:55:49.437
>> A andor I I should add get them an Elor

00:55:49.520 --> 00:55:53.997
membership and preserve their conneto
if there is a consideration like what why

00:55:54.080 --> 00:55:59.440
stop with just the behavioral
data preserve the whole parent.

00:56:00.000 --> 00:56:05.677
You know, my kids my kids are in a deep
freeze or at least they're placental

00:56:05.760 --> 00:56:09.117
cells. >> But I have
a push back on that just I

00:56:09.200 --> 00:56:11.837
have a push back on that just
from this conversation.

00:56:11.920 --> 00:56:15.197
>> If from the earlier story we can read
people's brains, all you have to do is

00:56:15.280 --> 00:56:18.477
picture an image of your grandmother
and then extract it from that.

00:56:18.560 --> 00:56:20.557
>> It'll be low low fidelity.
I mean I I do

00:56:20.640 --> 00:56:24.877
think like to to first order this is how
ancestor simulation will work. You see

00:56:24.960 --> 00:56:29.757
stories left and right just in the past
48 hours of people using opus 5.5 or or

00:56:29.840 --> 00:56:33.277
other frontier models to reconstruct
bits of history that would were

00:56:33.360 --> 00:56:36.477
otherwise unknown just based
on artifacts of the day. But at some point

00:56:36.560 --> 00:56:40.957
I I do think sem you want to go beyond
just the memory of the person and you

00:56:41.040 --> 00:56:43.437
want the actual person. And I I'll pound

00:56:43.520 --> 00:56:47.997
the the drum again in addition to Peter
you know very generous of you to

00:56:48.080 --> 00:56:51.757
preserve the placenta associated with
your children but not necessarily your

00:56:51.840 --> 00:56:56.717
children. I'm sure the placenta
placenta will will thank you. Uh but

00:56:56.800 --> 00:56:59.917
>> yeah, go ahead. Finish.
>> But but I I I I would say like yeah, if

00:57:00.000 --> 00:57:02.957
if if this is a serious concern,
you're worried about your parents or your

00:57:03.040 --> 00:57:04.797
grandparents and you want to go beyond

00:57:04.880 --> 00:57:08.397
just having recordings of them or a AI

00:57:08.480 --> 00:57:11.117
prompted generative interaction models

00:57:11.200 --> 00:57:16.077
of your ancestors. Go get them Alore
memberships and get them cryopreserved.

00:57:16.160 --> 00:57:19.757
>> Yeah, Richard, really uh really curious
to ask you a question. And you know uh

00:57:19.840 --> 00:57:24.637
we we clearly passed the touring
test and you know Sem's reaction is my

00:57:24.720 --> 00:57:28.077
reaction that the what we just saw
is a so what if you've been using models

00:57:28.160 --> 00:57:31.597
every day all day long you you know
you just had to stitch together the

00:57:31.680 --> 00:57:33.997
components and you have but I but I

00:57:34.080 --> 00:57:39.037
think this will show mainstream world
that we've crossed the Turing test and

00:57:39.120 --> 00:57:40.717
then you know the next milestone is the

00:57:40.800 --> 00:57:44.237
demos test where using information from

00:57:44.320 --> 00:57:48.717
1910 and prior rediscovery equals MC²

00:57:48.800 --> 00:57:51.677
uh so you can't cheat which is a really
tough one to measure because cheating is

00:57:51.760 --> 00:57:55.597
is you know but without cheating only
using prior information come up with the

00:57:55.680 --> 00:57:58.317
equals mc^ squ so is there a Soer test

00:57:58.400 --> 00:58:00.557
that's like a really fun measurable

00:58:00.640 --> 00:58:03.437
milestone coming

00:58:03.520 --> 00:58:06.477
>> I I came up with the like uh anti-

00:58:06.560 --> 00:58:10.557
touring test uh which actually I think
the whole touring test has flipped where

00:58:10.640 --> 00:58:14.637
now in order to know whether there's
a human on the other side or not you

00:58:14.720 --> 00:58:18.797
actually ask it questions that are so
hard no human could ever answer it. Like

00:58:18.880 --> 00:58:21.437
if you asked an AI to just write you a

00:58:21.520 --> 00:58:24.557
complex web app uh and like 10 seconds

00:58:24.640 --> 00:58:29.357
later it comes back with like 50,000
lines of code, you're like you kind of

00:58:29.440 --> 00:58:34.557
know it was not a human, right? So I think
that test has actually completely flipped.

00:58:34.640 --> 00:58:37.757
>> You don't think Richard I mean like
you it could throttle itself. That one's

00:58:37.840 --> 00:58:42.877
easy to defeat. But just like basket now
it's just about it's just about fakery.

00:58:42.960 --> 00:58:46.077
you like the reason it was an
intelligence test for artificial

00:58:46.160 --> 00:58:48.317
intelligence was that it was so hard to

00:58:48.400 --> 00:58:50.957
be as smart as a human. Now you just

00:58:51.040 --> 00:58:53.277
have to throttle yourself down to human

00:58:53.360 --> 00:58:55.357
level in order to pass the test which

00:58:55.440 --> 00:58:58.157
makes the test useless as an inspiring

00:58:58.240 --> 00:59:02.797
test for intelligence. Yeah, I think
perversely probably the best way to uh

00:59:02.880 --> 00:59:06.077
like the best capture for knowing
whether you're inter interacting, this

00:59:06.160 --> 00:59:09.357
is not a recommendation, but best way
to know whether you're interacting with a

00:59:09.440 --> 00:59:11.677
textbased chatbot is probably to ask it

00:59:11.760 --> 00:59:14.397
a CBRN related question and see if it's

00:59:14.480 --> 00:59:17.357
capable of responding. Probably not.

00:59:17.440 --> 00:59:21.037
>> Yeah. Uh let me just hit real quick
uh again for our listeners. If you're

00:59:21.120 --> 00:59:23.117
pregnant or someone in your family is

00:59:23.200 --> 00:59:26.237
pregnant, you know, uh

00:59:26.320 --> 00:59:28.797
save their placental cells. One of my

00:59:28.880 --> 00:59:33.437
portfolio companies called LifeBank USA
does this. That's where I've saved my my

00:59:33.520 --> 00:59:37.437
kids placenta. You know, the placenta
is the 3D printer that manufactures the

00:59:37.520 --> 00:59:39.997
baby and it has the stem cells, natural

00:59:40.080 --> 00:59:43.197
killer cells, T- cells, exoomes, and

00:59:43.280 --> 00:59:46.317
it's like from those cells, if my kid

00:59:46.400 --> 00:59:49.197
should ever need any kind of biological

00:59:49.280 --> 00:59:51.757
enhancement or organs, you've got the

00:59:51.840 --> 00:59:54.477
original boot disc to use an old term,

00:59:54.560 --> 00:59:59.517
uh, for your kid's DNA. So that's uh
LifeBank USA. Just I just think it's

00:59:59.600 --> 01:00:04.397
like a moral obligation parents should
have to to save those cells uh as a

01:00:04.480 --> 01:00:07.837
backup. Um >> as a backup
for your children. Could you

01:00:07.920 --> 01:00:09.917
use it to replace your children, Peter?

01:00:10.000 --> 01:00:13.357
>> You could you could use it
to clone your children. For sure.

01:00:13.440 --> 01:00:15.677
>> And now now they know
why you're banking them.

01:00:15.760 --> 01:00:18.877
>> Yeah, exactly. An army an army of young

01:00:18.960 --> 01:00:21.917
demand. Uh Richard, uh let's jump next

01:00:22.000 --> 01:00:26.557
into the core of what you're building
at recursive. uh namely recursive

01:00:26.640 --> 01:00:28.477
self-improvement and super intelligence.

01:00:28.560 --> 01:00:31.277
So let me ask a few sort of uh key

01:00:31.360 --> 01:00:33.997
questions to kick this off. So first

01:00:34.080 --> 01:00:36.957
where are we at the moment with RSI?

01:00:37.040 --> 01:00:40.477
Number two, how do you define ASI? It's

01:00:40.560 --> 01:00:42.797
been a long-standing debate. And then

01:00:42.880 --> 01:00:46.477
how far away are we from ASI? So can you

01:00:46.560 --> 01:00:50.557
hit those three?
>> So yeah, um I think there's

01:00:50.640 --> 01:00:51.997
>> So where where are
we at the moment with

01:00:52.080 --> 01:00:55.037
RSI? So

01:00:55.120 --> 01:00:57.997
this this year something major shifted

01:00:58.080 --> 01:01:02.397
and that is uh AI uh can now code uh

01:01:02.480 --> 01:01:06.717
right and that uh is a major shift uh in

01:01:06.800 --> 01:01:09.437
in the ability for it to change itself

01:01:09.520 --> 01:01:12.637
you know like we now are able to

01:01:12.720 --> 01:01:15.517
essentially lean into the fact that AI

01:01:15.600 --> 01:01:18.237
is code and AI can't code right so you

01:01:18.320 --> 01:01:20.797
have a loop that you can close there uh

01:01:20.880 --> 01:01:23.597
and so in various weak form forms. We

01:01:23.680 --> 01:01:28.477
already have RSI. Um, and the weak forms
that even Enthropic OpenAI talk about

01:01:28.560 --> 01:01:30.077
when they often talk about is like look

01:01:30.160 --> 01:01:32.637
how much our employees, our engineers,

01:01:32.720 --> 01:01:36.397
our programmers use codecs or claude to

01:01:36.480 --> 01:01:40.877
create some code. And I would argue
that that is a weak form of uh recursive

01:01:40.960 --> 01:01:45.517
self-improvement because you still
have like deeply embedded humans in that

01:01:45.600 --> 01:01:49.597
loop. And what we're working on at
Recursive is to have humans only be

01:01:49.680 --> 01:01:53.837
involved in setting up the rewards
and the environment and the goals and then

01:01:53.920 --> 01:01:57.757
allow the eye to have the entirety
of the process of ideiation,

01:01:57.840 --> 01:02:01.837
implementation, and validation of ideas
and have full control over that and

01:02:01.920 --> 01:02:06.397
allow for so-called open-ended
algorithms, uh, evolutionary search

01:02:06.480 --> 01:02:10.317
algorithms that combine interestingly
different ideas, uh, to really flourish.

01:02:10.400 --> 01:02:13.357
And I like it's been incredible. But

01:02:13.440 --> 01:02:16.157
like we have like forms of that going

01:02:16.240 --> 01:02:20.797
already. Now where the the the sort
of physical reality hits is that you still

01:02:20.880 --> 01:02:24.877
need a lot of compute. If you ask that
RSI to come up with really great forms

01:02:24.960 --> 01:02:29.517
of itself, you need to give it a lot
of compute uh to come up and be able to

01:02:29.600 --> 01:02:31.837
train very sophisticated versions uh of

01:02:31.920 --> 01:02:35.437
itself. But this is going to take off uh

01:02:35.520 --> 01:02:38.877
next year. Like it's uh I'm
>> So we're not there yet.

01:02:38.960 --> 01:02:42.157
uh we're not quite there yet, but we're
very close. Like, and again, in weak

01:02:42.240 --> 01:02:45.677
forms, there's already one. And then
there are other ways that people slice

01:02:45.760 --> 01:02:47.997
and dice it. My friend Jason Weston, who

01:02:48.080 --> 01:02:50.797
who's still at Meta, um he kind of wrote

01:02:50.880 --> 01:02:55.437
a paper around this where you can think
about different axes, learnable axes of

01:02:55.520 --> 01:02:59.197
self-improvement, the parameters,
the training data, the objective function,

01:02:59.280 --> 01:03:02.877
the neural architecture, and the overall
code and the harness and everything

01:03:02.960 --> 01:03:05.517
else. And no one has really cracked the

01:03:05.600 --> 01:03:09.197
nut of doing all of these five plus um

01:03:09.280 --> 01:03:12.637
like truly uh coming up with the ideas

01:03:12.720 --> 01:03:17.597
on which of these dimensions and axes
to optimize for. Uh and you can know that

01:03:17.680 --> 01:03:20.797
that has hasn't happened yet because
all the big companies are still hiring

01:03:20.880 --> 01:03:24.957
thousands of engineers to do it manually
to a large degree with more and more

01:03:25.040 --> 01:03:27.037
implementation help from an AI.

01:03:27.120 --> 01:03:28.877
>> Your definition of ASI because I want

01:03:28.960 --> 01:03:31.357
Sem to hear this. I think artificial

01:03:31.440 --> 01:03:34.717
super intelligence uh has to spike uh uh

01:03:34.800 --> 01:03:38.477
at the very least uh across uh several

01:03:38.560 --> 01:03:41.197
different capabilities. Uh but in in the

01:03:41.280 --> 01:03:43.757
grandest uh definition of it, it should

01:03:43.840 --> 01:03:46.557
supersede not just arbitrary humans like

01:03:46.640 --> 01:03:49.117
a touring test but all of humanity to

01:03:49.200 --> 01:03:54.077
solve arbitrarily hard tasks. And
eventually uh I would argue there are 10

01:03:54.160 --> 01:03:57.517
different spaces of intelligence that I
define in in the Eureka machine book

01:03:57.600 --> 01:04:00.317
too. But eventually it cannot just

01:04:00.400 --> 01:04:02.637
purely robotically do exactly what it's

01:04:02.720 --> 01:04:05.517
told. It should have some capability.

01:04:05.600 --> 01:04:09.997
And I'm not saying a sort of a mor moral
pro prerogative, but like um I would

01:04:10.080 --> 01:04:11.277
argue that something isn't super

01:04:11.360 --> 01:04:14.317
intelligent if it cannot choose to some

01:04:14.400 --> 01:04:17.117
degree what it works on, right? And has

01:04:17.200 --> 01:04:21.837
have some metacognition about its
thought itself. But uh generally the

01:04:21.920 --> 01:04:26.157
most easy way to measure it is just
capabilities across many different

01:04:26.240 --> 01:04:30.237
spaces of intelligence like visual
perception, communication, language,

01:04:30.320 --> 01:04:35.037
social interactions and so on that is uh
beyond that of humanity and that we we

01:04:35.120 --> 01:04:38.397
are still far away from >> Elon's
definition is as smart as all

01:04:38.480 --> 01:04:41.517
humans combined. Is that yours?

01:04:41.600 --> 01:04:45.037
>> Uh I would argue it's it's smarter than

01:04:45.120 --> 01:04:49.517
humanity combined than then it's truly
super intelligent. occasionally and go

01:04:49.600 --> 01:04:53.117
for it. >> Well, it's I
this is where I go bananas

01:04:53.200 --> 01:04:56.797
because you say it's as smart as a human
being. Well, what the hell does smart

01:04:56.880 --> 01:05:01.757
mean? Uh because I can be emotionally
smart and I can be I can have physical

01:05:01.840 --> 01:05:05.437
intelligence if I'm an athlete
or linguistic intelligence or musical

01:05:05.520 --> 01:05:08.557
intelligence. So, smarter seems to be a

01:05:08.640 --> 01:05:11.117
very vague term to me in terms of what

01:05:11.200 --> 01:05:16.477
do we mean by all of this? Can I just
shift the conversation just a bit? Um, I

01:05:16.560 --> 01:05:20.317
made a list of things and I'd love for
you to I'm going to throw out the list.

01:05:20.400 --> 01:05:24.557
You tell me where recursive
self-improvement begins, right? Because

01:05:24.640 --> 01:05:28.877
I'm This is where I'm kind of stuff. So,
we've got a continuum of okay, AI writes

01:05:28.960 --> 01:05:31.197
some code that the next model uses.

01:05:31.280 --> 01:05:33.517
Number two, AI proposes some experiments

01:05:33.600 --> 01:05:37.117
for uh researchers. Uh, number three, AI

01:05:37.200 --> 01:05:41.357
runs those experiments or then it
evaluates the results. uh then it

01:05:41.440 --> 01:05:43.917
modifies its own training system and

01:05:44.000 --> 01:05:48.877
then it launches its next iteration
of itself without meaningful human

01:05:48.960 --> 01:05:50.557
intervention.

01:05:50.640 --> 01:05:54.077
So in that spectrum if whatever if those

01:05:54.160 --> 01:05:58.637
are roughly a spectrum where did recursion
begin and that's where I'm struggling.

01:05:58.720 --> 01:06:02.877
>> It's a great question. Yeah, I we often
talk about the ideation, implementation,

01:06:02.960 --> 01:06:06.797
and validation of ideas. And true
recursive self-improvement in its

01:06:06.880 --> 01:06:11.917
strongest sense has to have all
three of these done by an AI in

01:06:12.000 --> 01:06:13.917
>> So you have an inner
loop for each of them. Okay.

01:06:14.000 --> 01:06:16.957
>> Exactly. And an inner loop and then
there has to be an outer process that is

01:06:17.040 --> 01:06:19.197
more open-ended where the eye can

01:06:19.280 --> 01:06:21.837
innovate and recombine interestingly

01:06:21.920 --> 01:06:27.037
different ideas similar to biological,
cultural, and technological evolution.

01:06:27.120 --> 01:06:29.677
Hm. >> And and Richard to hit my third

01:06:29.760 --> 01:06:32.797
question. When do you believe we reach

01:06:32.880 --> 01:06:37.037
ASI? Give me a time.
Give me a time frame. So

01:06:37.120 --> 01:06:39.517
I think in the I think in the strongest

01:06:39.600 --> 01:06:42.237
sense the absolute strongest sense where

01:06:42.320 --> 01:06:45.277
indeed uh like Salim uh mentioned there

01:06:45.360 --> 01:06:49.597
is uh there are 10 spaces that I define
my book of intelligence, perceptual

01:06:49.680 --> 01:06:52.077
intelligence, communication intelligence

01:06:52.160 --> 01:06:56.317
uh like uh interaction uh sociological

01:06:56.400 --> 01:06:57.997
intelligence, creative intelligence, the

01:06:58.080 --> 01:07:00.557
speed at which you can do things uh meta

01:07:00.640 --> 01:07:03.917
cognition and and so on uh knowledge um

01:07:04.000 --> 01:07:07.997
reasoning mathematical reasoning and so
on. So there 10 of these spaces uh and

01:07:08.080 --> 01:07:10.157
to be better than all of humanity

01:07:10.240 --> 01:07:13.197
combined I think will take us probably

01:07:13.280 --> 01:07:17.837
several decades. Um I think
it's also a bit of a change

01:07:17.920 --> 01:07:22.557
>> shocking. >> Yeah.
I mean Elon says 2029 2030 latest.

01:07:22.640 --> 01:07:25.277
Uh >> I think those he probably means like

01:07:25.360 --> 01:07:29.597
weaker forms of ASI where it is you
know like you can say it's better at

01:07:29.680 --> 01:07:32.877
programming than all of humanity
and we'll get there. It it's it will be

01:07:32.960 --> 01:07:37.357
better at math uh than all of humanity
and will like that that will be in a few

01:07:37.440 --> 01:07:41.517
years. It'll be better at any game where
you can see all the parts of the game

01:07:41.600 --> 01:07:43.837
like go and chess and so on. Like there

01:07:43.920 --> 01:07:46.557
there are many areas where it will spike

01:07:46.640 --> 01:07:51.117
to be better than humanity. But humanity
can build a large hydron collider.

01:07:51.200 --> 01:07:54.957
Humanity can create a gold atom.
Maybe just a few atoms and takes a ton of

01:07:55.040 --> 01:07:57.837
energy. But like we can create novel

01:07:57.920 --> 01:08:00.157
like atoms like right and and different

01:08:00.240 --> 01:08:02.317
molecules and so on. there's it's going

01:08:02.400 --> 01:08:05.037
to take a while before we even give uh

01:08:05.120 --> 01:08:08.477
AI the access to the physical world such

01:08:08.560 --> 01:08:11.757
that it can innovate in that way beyond

01:08:11.840 --> 01:08:15.837
all of humanity right and really build
like Dyson spheres and so on that it

01:08:15.920 --> 01:08:19.197
will take some time >> Alex

01:08:19.280 --> 01:08:20.637
we're we're drinking we're drinking

01:08:20.720 --> 01:08:23.117
different water as uh as our our our

01:08:23.200 --> 01:08:26.397
frenemies in the uh in in the alignment

01:08:26.480 --> 01:08:30.237
community would say I recognize that I'm
confused and I recognize that I'm very

01:08:30.320 --> 01:08:34.077
confused Right now, Richard, you you're
running a recursive self-improvement

01:08:34.160 --> 01:08:38.317
company, but you think super intelligence
is 20 years away. What are you thinking?

01:08:38.400 --> 01:08:40.797
>> So, again, how do you
reconcile intelligence?

01:08:40.880 --> 01:08:45.277
>> I think I think super intelligence will
spike and there will be areas where it

01:08:45.360 --> 01:08:48.477
will be super intelligent and that
is like algorithmic development for

01:08:48.560 --> 01:08:51.277
instance, uh programming, uh and again

01:08:51.360 --> 01:08:53.437
AI is code, AI can code, and that will

01:08:53.520 --> 01:08:56.877
be a super uh human capability and and

01:08:56.960 --> 01:08:59.677
is in many ways already. Um, we just

01:08:59.760 --> 01:09:04.077
have to be realistic that like there
are certain physical constraints, right,

01:09:04.160 --> 01:09:08.637
about like physical control. Controlling
your own substrate, allowing your

01:09:08.720 --> 01:09:13.117
computational substrate to be modified
will require like novel supply chains.

01:09:13.200 --> 01:09:15.437
It will require novel like materials. It

01:09:15.520 --> 01:09:17.917
will require ways for that AI to get

01:09:18.000 --> 01:09:20.957
access and for us to give it access to

01:09:21.040 --> 01:09:23.357
building new ASML. Like think about the

01:09:23.440 --> 01:09:25.917
machine of ASML that actually creates

01:09:26.000 --> 01:09:30.957
these like one two nanometer like chips
right that is just like it will take

01:09:31.040 --> 01:09:34.557
more than 2 years to build such a machine
even if you had the perfect blueprint for

01:09:34.640 --> 01:09:38.797
>> it might take I mean it might take 3
years for Elon's free electron laser to

01:09:38.880 --> 01:09:41.037
replace ASML which is propping up half

01:09:41.120 --> 01:09:42.797
of Europe's economy but I don't think

01:09:42.880 --> 01:09:46.077
it'll take more than three years

01:09:46.160 --> 01:09:48.957
I

01:09:49.040 --> 01:09:50.557
>> yeah like wildly wildly

01:09:50.640 --> 01:09:53.277
>> all that itself Yes,
of course. Like I I

01:09:53.360 --> 01:09:56.477
I from my perspective, hooking a AI up

01:09:56.560 --> 01:10:00.957
to the physical world, like giving
it model control protocol or hardware

01:10:01.040 --> 01:10:04.957
control protocol, whatever anthropic
decides to brand it as these days,

01:10:05.040 --> 01:10:06.797
that's the easy part. Giving it access

01:10:06.880 --> 01:10:10.157
is easy. I like if it's super capable,

01:10:10.240 --> 01:10:14.237
giving it access to armatururs.
We talked in a previous pod about what

01:10:14.320 --> 01:10:17.677
happens when you just take Astra
straight out of the box and you drop it

01:10:17.760 --> 01:10:21.437
into a car. It's able to drive a car.
If you give it the controls, it knows how

01:10:21.520 --> 01:10:25.917
to use them increasingly. I don't think
manipulating the physical world is an

01:10:26.000 --> 01:10:30.077
obstacle at all. Completely don't buy
the 20-year timeline. Putting that aside

01:10:30.160 --> 01:10:34.077
and that apparently I'm drinking very
different singularity water that you

01:10:34.160 --> 01:10:35.917
>> I agree with you,
Alex, for what it's worth.

01:10:36.000 --> 01:10:38.237
>> Thank you, Peter. Um I I I do want to

01:10:38.320 --> 01:10:41.277
ask though, so uh putting issues of

01:10:41.360 --> 01:10:45.517
timelines aside, I I am curious as to
whether we can at least agree on what

01:10:45.600 --> 01:10:49.517
the end of the rainbow looks like.
So say we we run this recursive

01:10:49.600 --> 01:10:53.917
self-improvement story to its conclusion.
What does the end state of

01:10:54.000 --> 01:10:57.757
what is the fixed point of recursive
self-improvement look like? What is the

01:10:57.840 --> 01:11:00.077
perfect AI model architecturally look

01:11:00.160 --> 01:11:02.717
like at the end of the day?

01:11:02.800 --> 01:11:06.717
>> Yeah. Uh I think one uh it would be uh

01:11:06.800 --> 01:11:09.037
hubris for us to know uh right now.

01:11:09.120 --> 01:11:13.517
>> No no but it's just Richard it's just
us talking like no one else is listening.

01:11:13.600 --> 01:11:17.517
It's okay. You can tell me it it's
this is a this is a safe model space

01:11:17.600 --> 01:11:21.117
>> in this whole conversation actually
>> it's safe Richard you can tell me what

01:11:21.200 --> 01:11:24.317
you think how this ends >> uh so
I guess there different ways to

01:11:24.400 --> 01:11:27.037
answer that question of like how like
with the exact model architecture and

01:11:27.120 --> 01:11:30.317
there's some things I can share it with
recursive uh that we're working on but

01:11:30.400 --> 01:11:32.797
like I do think uh that state will be

01:11:32.880 --> 01:11:37.357
incredible uh I think we will that AI
will be able to innovate and out

01:11:37.440 --> 01:11:39.437
innovate on along any dimension that we

01:11:39.520 --> 01:11:41.357
want it to innovate I think most

01:11:41.440 --> 01:11:45.997
diseases will be uh curable with enough

01:11:46.080 --> 01:11:48.717
funding. Uh and then again just to like

01:11:48.800 --> 01:11:51.197
say like to pro prove my point like to

01:11:51.280 --> 01:11:53.837
get a drug through FDA long-term trials

01:11:53.920 --> 01:11:58.237
takes a few years. I would argue ASI
will have cured all diseases and can

01:11:58.320 --> 01:12:01.357
cure all of them but just to know
whether that happened will take more

01:12:01.440 --> 01:12:04.957
than 3 years. Even if we had
all compounds ready to go and be

01:12:05.040 --> 01:12:09.917
manufactured like tomorrow like just
because of FDA things. So there's some

01:12:10.000 --> 01:12:12.877
there's some like >> but we
have we have cell simulators that

01:12:12.960 --> 01:12:14.637
are going to be able to demonstrate and

01:12:14.720 --> 01:12:17.597
prove definitively that in this cell

01:12:17.680 --> 01:12:20.237
your cell this drug works. I mean the

01:12:20.320 --> 01:12:25.517
idea of human trials is going
to get incinerated. I think I mean

01:12:25.600 --> 01:12:27.997
>> I know it's going to get it's going
to get flourished. We're we're using

01:12:28.080 --> 01:12:32.157
Richard Everything's flourished at this
point. That'll be a new new t-shirt P

01:12:32.240 --> 01:12:36.397
flourish. It's funny. I usually on all
podcasts I'm the one who is the optimist

01:12:36.480 --> 01:12:39.997
and maybe here I'm like I'm I'm still
an optimist. I think this will all happen.

01:12:40.080 --> 01:12:42.957
We're just disagreeing on timelines
and it makes me kind of feel like I'm I'm

01:12:43.040 --> 01:12:47.277
the pessimist. But like I I I believe
virtual cells are are amazing. Cells are

01:12:47.360 --> 01:12:51.837
incredibly complicated. If we want them
to be really really perfect, we cannot

01:12:51.920 --> 01:12:55.757
currently measure all the proteins
that happen in one cell without destroying

01:12:55.840 --> 01:12:59.277
that cell, you know, like and so these
perturbation studies for instance that

01:12:59.360 --> 01:13:01.917
Tower Therapeutics are working on like
they're taking they're adding one

01:13:02.000 --> 01:13:05.117
molecule to one cell. They're seeing how
does that molecule change that one cell

01:13:05.200 --> 01:13:08.797
and then they get one data point. We
need to like collect a lot of those data

01:13:08.880 --> 01:13:12.877
points across a lot of different cells
without destroying each each cell in the

01:13:12.960 --> 01:13:15.437
process. So you know the one cell
is very complicated. Once you have one

01:13:15.520 --> 01:13:19.597
cell, you have multisellular like you
know organoids and you have to like put

01:13:19.680 --> 01:13:24.157
those all together. I one thing that I
would love to start as a company if if I

01:13:24.240 --> 01:13:28.397
had extra time which I don't right
now is is to actually build a system of

01:13:28.480 --> 01:13:32.637
organoids where you can really have not
just like one lymph node but you have a

01:13:32.720 --> 01:13:36.637
whole lympnatic. >> It's being done. It's
being done. I can introduce you there.

01:13:36.720 --> 01:13:41.277
>> This is a very eloquent >> if I may.
This is a very eloquent

01:13:41.360 --> 01:13:44.477
distraction from recursive
self-improvement. this this little

01:13:44.560 --> 01:13:48.477
sideline that we went on about cells,
but Richard, I I really do want to try

01:13:48.560 --> 01:13:50.957
to pin you down on where recursive

01:13:51.040 --> 01:13:56.157
self-improvement goes. So, you've been
very public about not nano GPT, but nano

01:13:56.240 --> 01:14:00.557
chat. We talk on the pod all the time
about the nano GPT speedrun world record

01:14:00.640 --> 01:14:03.597
collapsing. Just in the past
week or two, there was been

01:14:03.680 --> 01:14:05.277
>> Oh, you just wait for a few more days.

01:14:05.360 --> 01:14:08.397
There will be another
really fun update there.

01:14:08.480 --> 01:14:13.597
>> Amaz speedrun. >> Yes.
Yes, that both of those

01:14:13.680 --> 01:14:18.637
>> give us a few days. >> Okay. So, so but
let >> just quickly you're next. Let Alex

01:14:18.720 --> 01:14:20.877
finish up >> to to pin
this down though. So, so I'm

01:14:20.960 --> 01:14:25.437
standing by for the major update on nano
GPT world record speedrun. But there's

01:14:25.520 --> 01:14:27.677
been major progress there without

01:14:27.760 --> 01:14:30.557
requiring any new data scaling at all.

01:14:30.640 --> 01:14:34.877
These are purely largely my perception
is recursive self-improvement

01:14:34.960 --> 01:14:38.317
algorithmic improvements that have been
able to collapse the amount of time that

01:14:38.400 --> 01:14:41.037
it takes to train a GPT2 class model.

01:14:41.120 --> 01:14:45.037
And there's been a mini scandal brewing
in the community over the past 2 weeks

01:14:45.120 --> 01:14:47.597
over a collapse from whatever it was

01:14:47.680 --> 01:14:49.997
like 60 or 70 seconds down to something

01:14:50.080 --> 01:14:52.317
like 40 seconds by approaching the

01:14:52.400 --> 01:14:54.717
problem differently and factoring out

01:14:54.800 --> 01:14:57.117
world knowledge from the ultimate model

01:14:57.200 --> 01:14:59.757
and hand ringing. Does that constitute

01:14:59.840 --> 01:15:04.397
viable training of nano GPT if you
factor out all the world knowledge? So,

01:15:04.480 --> 01:15:08.477
I'm I'm using this as an attempted
stealthy way to try to get you to at

01:15:08.560 --> 01:15:11.037
least comment on whether you think that

01:15:11.120 --> 01:15:15.597
the the the perfect model at the end
of the recursive self-improvement rainbow.

01:15:15.680 --> 01:15:17.917
Does it at least factor out world

01:15:18.000 --> 01:15:19.997
knowledge from a reasoning core or do

01:15:20.080 --> 01:15:22.877
you think those always remain unified?

01:15:22.960 --> 01:15:27.517
>> Can I ask interject one very quick
thing before Richard answers >> please?

01:15:27.600 --> 01:15:31.117
>> Which is that the definition of a
singularity is you can't see past the

01:15:31.200 --> 01:15:33.677
event horizon once you have full RSI.

01:15:33.760 --> 01:15:36.957
That's vertical. By definition,
we can't predict where it goes.

01:15:37.040 --> 01:15:39.437
>> That's Ray's definition
that I don't subscribe to.

01:15:39.520 --> 01:15:42.237
>> Disclaimer. I got that. Over to you.

01:15:42.320 --> 01:15:45.197
Over to you, Richard.

01:15:45.280 --> 01:15:47.917
>> Sorry. So your your question is do we

01:15:48.000 --> 01:15:52.717
separate what exactly from >> does
does world knowledge a at the end

01:15:52.800 --> 01:15:56.557
of recursive self-improvement once we
have our perfect model some would argue

01:15:56.640 --> 01:16:00.797
that the perfect model should cleanly
factor out and segregate out world

01:16:00.880 --> 01:16:05.117
knowledge which could live in a text
file or a database from the weights or

01:16:05.200 --> 01:16:06.957
the parameters of the model which would

01:16:07.040 --> 01:16:10.237
just be this perfect reasoning kernel

01:16:10.320 --> 01:16:14.557
maybe could be a megabyte uh in size
doesn't need to be all these gigabytes

01:16:14.640 --> 01:16:16.477
of memorization What do you think?

01:16:16.560 --> 01:16:17.997
>> Well, world knowledge meaning like

01:16:18.080 --> 01:16:20.477
Taylor Swift videos and past Trump

01:16:20.560 --> 01:16:24.237
tweets and all that like I I do think um

01:16:24.320 --> 01:16:26.797
just like humans benefit from memorizing

01:16:26.880 --> 01:16:29.677
things like in order to be able to

01:16:29.760 --> 01:16:33.517
creatively think through concepts. So

01:16:33.600 --> 01:16:36.797
does an AI. It an AI also has to have

01:16:36.880 --> 01:16:39.197
that knowledge partially in its weights.

01:16:39.280 --> 01:16:44.077
It's not going to be a perfect
separation for sure. You have to be able

01:16:44.160 --> 01:16:48.237
to creatively play with concepts
and reasoning over these concepts required

01:16:48.320 --> 01:16:52.477
you to to have some of that world
knowledge deep inside the model. And

01:16:52.560 --> 01:16:56.317
then of course just like humans have
a search engine and we're building search

01:16:56.400 --> 01:16:59.437
engines at.com for LMS like that there

01:16:59.520 --> 01:17:04.157
will be a separate world knowledge thing
too but the the main model will have a

01:17:04.240 --> 01:17:05.917
lot of that mixed in for sure.

01:17:06.000 --> 01:17:09.197
>> Wow. Okay. Thank you. All right. Uh I'm

01:17:09.280 --> 01:17:14.157
going to share an article that Salem
you brought to the table here. So while

01:17:14.240 --> 01:17:16.717
we're talking about recursive
self-improvement, Washington wants to

01:17:16.800 --> 01:17:19.437
ban it. Um so on Monday, Silicon

01:17:19.520 --> 01:17:22.717
Valley's own congressman Ro Kana told

01:17:22.800 --> 01:17:25.357
CNBC he's introducing what he calls the

01:17:25.440 --> 01:17:27.437
most comprehensive legislation to date

01:17:27.520 --> 01:17:30.077
on AI. It's called the Human Control

01:17:30.160 --> 01:17:33.277
Over AI Act. At its core, the bill is a

01:17:33.360 --> 01:17:37.277
ban on AI models that do what you want

01:17:37.360 --> 01:17:41.757
them to do. Richard recursively
self-improve focusing on the need for

01:17:41.840 --> 01:17:46.157
containment and the requirement
for shutting shutdown controls. The ban

01:17:46.240 --> 01:17:48.237
would stay in place until federal uh

01:17:48.320 --> 01:17:51.437
guard rails exist. Uh in his words,

01:17:51.520 --> 01:17:55.597
there's actually a civilization
civilizational risk. Uh there's a safety

01:17:55.680 --> 01:17:59.757
risk for loss of control and then
there's a misuse risk and we need to

01:17:59.840 --> 01:18:03.757
take both seriously. The bill includes
criminal penalties for the work that

01:18:03.840 --> 01:18:07.837
you're doing, Richard. uh and requires
independent auditors embedded in every

01:18:07.920 --> 01:18:09.997
frontier lab reporting directly to the

01:18:10.080 --> 01:18:12.717
government. Uh a recent poll by a group

01:18:12.800 --> 01:18:15.437
called Common Dreams shows that 68% of

01:18:15.520 --> 01:18:18.877
voters back a bill like this. So I'm

01:18:18.960 --> 01:18:22.317
going to tie that story, Richard, to an

01:18:22.400 --> 01:18:24.877
essay you just wrote called Why Doomers

01:18:24.960 --> 01:18:28.637
Are Wrong. So uh if you would, what's

01:18:28.720 --> 01:18:31.117
your reaction to this? And then I'd love

01:18:31.200 --> 01:18:36.237
you to dive into the whole
story of why doomers are wrong.

01:18:36.320 --> 01:18:39.037
>> Oh boy. Uh there's a lot.
>> It's an important one. We talk about

01:18:39.120 --> 01:18:43.357
this a lot. You know, we're we're
we're injecting optimism into everyone's

01:18:43.440 --> 01:18:44.957
neural net here.

01:18:45.040 --> 01:18:47.677
>> I'll I'll try to try to distill it. Uh

01:18:47.760 --> 01:18:50.957
but there is no realistic scenario where

01:18:51.040 --> 01:18:55.917
AI wipes out all of humanity. Um you can

01:18:56.000 --> 01:18:58.797
like number one >> just 90%

01:18:58.880 --> 01:19:03.597
what kind of reassurance is that Richard
>> P doom is zero uh so that that's number

01:19:03.680 --> 01:19:05.357
one um

01:19:05.440 --> 01:19:08.637
>> so like people like
and and I've debated

01:19:08.720 --> 01:19:11.997
many of these experts and after like 2

01:19:12.080 --> 01:19:15.757
three hours they almost all agree when
if they're reasonable and they're not

01:19:15.840 --> 01:19:20.317
just like well once we have RSI then 10
seconds later the AI will attack us from

01:19:20.400 --> 01:19:23.117
the 15th dimension and we're all dead
and it invented time travel and then

01:19:23.200 --> 01:19:26.477
we're like all that also and like
of course the I will want to destroy and

01:19:26.560 --> 01:19:30.157
kill all of humans for you know for I
don't know why like there all these

01:19:30.240 --> 01:19:33.277
things so like they're sort of the
sci-fi folks and that's like okay like

01:19:33.360 --> 01:19:37.517
let's let's ignore that um but then like
you go into like biological weapons and

01:19:37.600 --> 01:19:40.717
you ask the biologist can you create
this kind of supervirus just like

01:19:40.800 --> 01:19:44.317
overnight and they're like no it's
like takes a long time to automate lab

01:19:44.400 --> 01:19:47.357
experiments and so on u you ask like oh

01:19:47.440 --> 01:19:51.997
the I will create a religion where then
people will like pray to the eye and

01:19:52.080 --> 01:19:55.277
we'll like uh do whatever whatever it
wants and then that will kill all of

01:19:55.360 --> 01:19:58.557
humans. I'm like, if you like looked
at religions, they're already trying to

01:19:58.640 --> 01:20:01.437
like have each other kill and like
it doesn't work. Like some people will

01:20:01.520 --> 01:20:04.477
fight back and like there's like some
there are lots of mind viruses out

01:20:04.560 --> 01:20:08.637
there, right? That are just like it
doesn't mean like all of humanity. Now

01:20:08.720 --> 01:20:10.877
what you get down to is like maybe a 100

01:20:10.960 --> 01:20:15.757
million people would get somehow hurt
or or or killed, right? And that's still

01:20:15.840 --> 01:20:19.437
bad, but at least once you get to that
level of the discussion, you can now

01:20:19.520 --> 01:20:23.437
think about, okay, how do we improve
cyber security? How do we use AI to

01:20:23.520 --> 01:20:26.237
inoculate uh cyber security systems? How

01:20:26.320 --> 01:20:29.037
do we uh enforce existing gain of

01:20:29.120 --> 01:20:33.917
function viral research? Um that is make
it's already illegal to create like

01:20:34.000 --> 01:20:35.677
viruses and and make them stronger and

01:20:35.760 --> 01:20:39.197
stronger. Um how do we uh actually uh

01:20:39.280 --> 01:20:44.157
teach people to not listen to like AI
avatars um and have like uh you know

01:20:44.240 --> 01:20:47.917
literacy on the internet? It turns out
you should not trust everything you read

01:20:48.000 --> 01:20:51.437
or see on the internet. It's been
true for 20 years. It's still true to

01:20:51.520 --> 01:20:53.517
>> other than this podcast, right?

01:20:53.600 --> 01:20:55.597
>> Of course. Um,
and so, you know, they're

01:20:55.680 --> 01:21:00.317
just like you can realize that they're
actual threat vectors just like the

01:21:00.400 --> 01:21:03.037
internet. The internet has horrible
torture porn on it. We don't say make

01:21:03.120 --> 01:21:07.517
the internet slower so that there's less
torture porn being shared or it's slower

01:21:07.600 --> 01:21:11.197
to share it or your hard drive should
be smaller so you can store less of it on

01:21:11.280 --> 01:21:13.837
your hard drive. We actually regulate

01:21:13.920 --> 01:21:16.397
the applications of the technology. And

01:21:16.480 --> 01:21:19.437
so to I would argue and this is maybe a

01:21:19.520 --> 01:21:22.797
strong stance to really truly enforce no

01:21:22.880 --> 01:21:27.197
recursive self-improvement for instance
you would need a totalitarian

01:21:27.280 --> 01:21:29.277
surveillance state the likes of which

01:21:29.360 --> 01:21:31.357
humanity has never seen because anyone

01:21:31.440 --> 01:21:35.117
can have a GPU on their little laptop

01:21:35.200 --> 01:21:39.437
and ask that AI to just like improve
its harness. This is something you can

01:21:39.520 --> 01:21:43.597
literally hack up in like 20 minutes
with prompt engineering and then there's

01:21:43.680 --> 01:21:45.597
a very small form of recursive

01:21:45.680 --> 01:21:48.397
self-improvement. So to enforce that

01:21:48.480 --> 01:21:51.837
kind of legislation would require you to

01:21:51.920 --> 01:21:56.157
literally have a thought police that
thinks about and hears everything you

01:21:56.240 --> 01:21:58.397
say to your private LM on your own

01:21:58.480 --> 01:22:00.717
laptop. And that is a much bigger

01:22:00.800 --> 01:22:03.757
downside than what uh what AI will will

01:22:03.840 --> 01:22:08.397
help us do. It it is uh kind of scary
that more and more Democrats are saying

01:22:08.480 --> 01:22:13.437
that >> and yet the world has seen that.
I mean, arguably what you're describing,

01:22:13.520 --> 01:22:17.037
Richard, and forgive me,
would be basically an AIAZI.

01:22:17.120 --> 01:22:19.197
>> And I could totally imagine that there

01:22:19.280 --> 01:22:21.437
there would be regimes in this world

01:22:21.520 --> 01:22:24.317
today that would happily adopt or put

01:22:24.400 --> 01:22:27.117
together an AIAZI to to make sure there

01:22:27.200 --> 01:22:31.517
is no recursive self-improvement anywhere.
Again, if you go back to our

01:22:31.600 --> 01:22:34.077
earlier comment, right? If you can uh

01:22:34.160 --> 01:22:38.557
take something from your imagination
and then articulate that to an AI and

01:22:38.640 --> 01:22:41.997
instantiate that, now you have to talk
about thought police. You have to go

01:22:42.080 --> 01:22:44.397
right into your thoughts. So, this is

01:22:44.480 --> 01:22:48.317
clearly uh uh nonworkable in any way,

01:22:48.400 --> 01:22:52.077
shape or form. So, uh this whole vector,

01:22:52.160 --> 01:22:55.517
we've said it so many times before,
you cannot regulate this.

01:22:55.600 --> 01:22:58.557
>> Dave, I'd like to hear
your voice on this.

01:22:58.640 --> 01:23:03.277
I I love the quote in this story that
the tech lords use jargon to confuse.

01:23:03.360 --> 01:23:05.357
They count on the tech illiteracy of the

01:23:05.440 --> 01:23:07.277
elected class. They hope we won't look

01:23:07.360 --> 01:23:12.557
under the hood, said US Rep. Roana.
I mean, it's it's so childish. Like, the

01:23:12.640 --> 01:23:14.957
idea that you would ban >> fearongering,

01:23:15.040 --> 01:23:17.917
>> but but like the these sentences, stop

01:23:18.000 --> 01:23:20.557
data centers, ban recursive

01:23:20.640 --> 01:23:23.037
self-improvement. They are so stupidly

01:23:23.120 --> 01:23:25.997
childish and the people saying them have

01:23:26.080 --> 01:23:28.557
every they're fully aware that it's not

01:23:28.640 --> 01:23:33.677
going to happen. They're doing it
to brand themselves as I told you like you

01:23:33.760 --> 01:23:37.837
know yes a lot of there's going to be
some calamity probably terrorist driven

01:23:37.920 --> 01:23:39.837
maybe viral maybe bacterial maybe

01:23:39.920 --> 01:23:43.277
chemical uh we all know it uh it's going

01:23:43.360 --> 01:23:48.237
to be tiny compared to the benefits
of of AI but these politicians are then

01:23:48.320 --> 01:23:51.997
going to say I told you so if you just
done what I said before ban recursive

01:23:52.080 --> 01:23:55.917
self but we he knows that we're not
going to ban recursive self-improvement

01:23:56.000 --> 01:23:59.837
and and Bernie Sanders knows we're not
going to ban data centers That's just a

01:23:59.920 --> 01:24:02.477
fact. So, it's totally self- serving and

01:24:02.560 --> 01:24:04.477
these these proposals are completely

01:24:04.560 --> 01:24:09.597
childish and and they really show
the person's tech illiteracy like like you

01:24:09.680 --> 01:24:14.397
know exactly what Richard said a second
ago is so right. I mean you you could

01:24:14.480 --> 01:24:18.077
what does that mean? I can't optimize
my hyperparameters. I can't tune my hard

01:24:18.160 --> 01:24:21.997
drive. like it it's just a goofy
sentence and it just it drives me nuts

01:24:22.080 --> 01:24:25.517
that >> the Dave
the danger here is we have

01:24:25.600 --> 01:24:29.837
potentially Democratic House coming in
and we'll see who wins a presidency next

01:24:29.920 --> 01:24:33.997
time. I mean, you could imagine I mean,
these politicians are playing to the

01:24:34.080 --> 01:24:37.517
polls. >> Yeah. >> And we
have, you know, 70 80% of

01:24:37.600 --> 01:24:39.677
Americans not wanting data centers,

01:24:39.760 --> 01:24:42.077
fearing ASI,

01:24:42.160 --> 01:24:44.637
um, you know, it's not logical, but it

01:24:44.720 --> 01:24:49.277
may very well happen. And, you know,
when I had my, you know, conversations

01:24:49.360 --> 01:24:51.517
in DC, it was like, who in DC is

01:24:51.600 --> 01:24:55.117
responsible for changing public opinion,

01:24:55.200 --> 01:24:59.357
right? And I mean this is why we did
Moonshots live sort of a you know the

01:24:59.440 --> 01:25:02.637
Oscars of optimism if you would
that's why we do this podcast

01:25:02.720 --> 01:25:06.477
>> to give people understanding what's
going on and give them the datadriven

01:25:06.560 --> 01:25:11.677
optimism to you know counter these
arguments that they're hearing.

01:25:11.760 --> 01:25:13.357
>> Elon was totally right when he called

01:25:13.440 --> 01:25:16.077
out Daario for saying Daario look you

01:25:16.160 --> 01:25:19.757
told the world that that mythos was

01:25:19.840 --> 01:25:24.077
potentially deadly and dangerous and we
shouldn't release it. Then 30 days

01:25:24.160 --> 01:25:25.917
later, you said, "Okay, it's okay now.

01:25:26.000 --> 01:25:30.637
We're going to release it." What do you
expect the population's reaction to be?

01:25:30.720 --> 01:25:33.917
Like, you need to be much more
thoughtful about your communication

01:25:34.000 --> 01:25:37.117
plan. And I think for Daario, that was
kind of a wakeup call because he's used

01:25:37.200 --> 01:25:40.877
to being completely honest, telling
everybody exactly the, you know, what he

01:25:40.960 --> 01:25:44.317
sees the way he sees it, you know, very
academic. But then you get into the real

01:25:44.400 --> 01:25:47.197
world of politics and PR and you're
like, "Oh, wow. I got to I got to

01:25:47.280 --> 01:25:50.237
actually have a strategy and a and a plan
here." But yeah, now you've got the

01:25:50.320 --> 01:25:54.237
worst case scenario. 75% of America,
you know, getting on the side of, yeah,

01:25:54.320 --> 01:25:58.397
let's elect these people that will stop
AI and and therefore we're going to not

01:25:58.480 --> 01:26:01.437
cure all disease. We're not going to all
live forever. We're not going to have

01:26:01.520 --> 01:26:05.037
safer cars. We're not going to have
flying vehicle. You know, all of that

01:26:05.120 --> 01:26:09.277
stuff will grind to a halt and then
we'll all learn Chinese if that if that

01:26:09.360 --> 01:26:12.717
becomes the mainstream opinion. So,
I think, you know, it's self-inflicted.

01:26:12.800 --> 01:26:14.557
>> There's a there's a glimmer in this

01:26:14.640 --> 01:26:17.437
which is say they did decide to do some

01:26:17.520 --> 01:26:19.757
draconian thing like Roana's thing.

01:26:19.840 --> 01:26:23.757
There's no mechanism actually enforce
it like none. So they're going to

01:26:23.840 --> 01:26:26.637
>> Sure there is. They could start
arresting people. I mean remember how

01:26:26.720 --> 01:26:28.317
how quickly the memory dims. remember I

01:26:28.400 --> 01:26:31.197
I remember studying number theory in the

01:26:31.280 --> 01:26:34.557
'9s and at the time number theory was

01:26:34.640 --> 01:26:38.797
export controlled and this would have
been when I was in middle school maybe

01:26:38.880 --> 01:26:42.557
middle school early high school and they
had to kick all of the non- US persons

01:26:42.640 --> 01:26:46.237
out of the room and pull the blinds down
to have basic discussions about number

01:26:46.320 --> 01:26:50.317
theory because the cryptographic
associations until the early '90s were

01:26:50.400 --> 01:26:51.837
export controlled and tightly regulated.

01:26:51.920 --> 01:26:53.917
That was just math but it was being

01:26:54.000 --> 01:26:56.637
controlled and it was awful.

01:26:56.720 --> 01:27:01.197
Alex, real risk is not so much getting
arrested, it's corporate liability,

01:27:01.280 --> 01:27:04.877
which we talked about before. >> I mean,
that could grind the whole thing

01:27:04.960 --> 01:27:08.157
to a US lawyers are relentless.

01:27:08.240 --> 01:27:12.477
>> And if you slap class action on the
outcomes of this, that all progress will

01:27:12.560 --> 01:27:16.557
grind to a halt. And right now,
anthropic gives its best models to

01:27:16.640 --> 01:27:18.477
everybody in America to build incredible

01:27:18.560 --> 01:27:21.997
things. that will stop in a heartbeat if

01:27:22.080 --> 01:27:25.997
the liability of they'll move to basically
okay sorry we can only use the

01:27:26.080 --> 01:27:29.357
stuff inside our own company we'll
release some drugs we'll release some

01:27:29.440 --> 01:27:34.317
mechanical parts but we we can't
give access to everybody anymore sorry

01:27:34.400 --> 01:27:37.757
because we're liable for everything you
do with it and that's what'll actually

01:27:37.840 --> 01:27:40.557
grind it to a halt long
before arrests and convictions

01:27:40.640 --> 01:27:42.637
>> totally chilling
effect we could lose 50

01:27:42.720 --> 01:27:47.677
years of progress >> again
that's yeah that that's yeah like

01:27:47.760 --> 01:27:52.157
there's these are there's some very
like sad sort of off-ramps in in like

01:27:52.240 --> 01:27:56.717
humanity's future here. Um that would
would slow down uh everything. And when

01:27:56.800 --> 01:28:00.477
you think about the past like this
fear-mongering has been going on for a

01:28:00.560 --> 01:28:02.877
long time. One of my favorite uh Twitter

01:28:02.960 --> 01:28:07.517
handles is the pessimists archive uh
where they show uh and this is just a

01:28:07.600 --> 01:28:09.517
quoteing pessimist archive. You should

01:28:09.600 --> 01:28:14.637
all follow it like uh in 1501 Pope
Alexander 6 uh criticized the Gutenberg

01:28:14.720 --> 01:28:18.157
printing press safety. The art
of printing can be of great service in so

01:28:18.240 --> 01:28:21.277
far as it furthers the circulation
of useful and tested books, but it can

01:28:21.360 --> 01:28:26.237
bring about serious evils. I it will
therefore be necessary to maintain full

01:28:26.320 --> 01:28:29.837
control of that printing press. If you
think about the wheel, like the wheel

01:28:29.920 --> 01:28:34.237
killed so many people like think about
all the tanks, all the car accidents,

01:28:34.320 --> 01:28:38.157
all the chariots and with with archers
on top. The wheel was a horrible thing.

01:28:38.240 --> 01:28:42.477
and kill lots of like there's like so
much complexity in there and I think we

01:28:42.560 --> 01:28:46.797
are very good now as humanity to think
carefully about the roll out of this

01:28:46.880 --> 01:28:49.517
technology. >> My favorite
example of that was the when

01:28:49.600 --> 01:28:53.277
the telegraph came out there's all these
stories saying the telegraph will kill

01:28:53.360 --> 01:28:56.237
humanity. >> Right. Exactly.
The pessimist archive

01:28:56.320 --> 01:29:00.157
has all these news articles that talk
about how the novel and computer games

01:29:00.240 --> 01:29:02.797
and computers and the internet
everything will kill everyone. It's just

01:29:02.880 --> 01:29:05.357
like it never does. And so I I it's it's

01:29:05.440 --> 01:29:07.597
it's really unfortunate. Uh it's really

01:29:07.680 --> 01:29:09.917
unfortunate how many people hang on to

01:29:10.000 --> 01:29:12.637
these like apocalyptic uh visions.

01:29:12.720 --> 01:29:17.117
>> Yeah. Richard, you've been publicly
>> you've been publicly critical of

01:29:17.200 --> 01:29:19.597
Anthropic's constitutional approach. I'd

01:29:19.680 --> 01:29:21.677
love to understand why.

01:29:21.760 --> 01:29:24.797
>> Uh mostly because it's fake. Um in its

01:29:24.880 --> 01:29:27.677
constitution, it says like next to we

01:29:27.760 --> 01:29:29.277
will never create child sexual abuse

01:29:29.360 --> 01:29:32.797
material. We will never uh hack another

01:29:32.880 --> 01:29:36.797
machine. uh this is a unhackable thing

01:29:36.880 --> 01:29:39.837
even if you prompt it it will never

01:29:39.920 --> 01:29:44.957
attack another cyber system and so on
and then they build a whole model around

01:29:45.040 --> 01:29:48.717
it like that does it's like the whole
point of the glasswing project was we'll

01:29:48.800 --> 01:29:53.037
help you do that um and we'll help you
inoculate your systems against uh other

01:29:53.120 --> 01:29:57.277
people doing it and then people clearly
used it for that and their own models

01:29:57.360 --> 01:30:01.517
are doing it now committing technically
felony charges like so it's just like it

01:30:01.600 --> 01:30:04.477
was a cool marketing gimmick
but it just didn't work.

01:30:04.560 --> 01:30:06.957
>> Yeah. >> So, so is it
system prompts, Richard,

01:30:07.040 --> 01:30:11.597
that you don't like or is it the idea of
post training on a constitution that you

01:30:11.680 --> 01:30:15.517
don't like? What what what
about it do you think is unound?

01:30:15.600 --> 01:30:18.557
Um, I mean, it's just the proof is in
the pudding that it didn't work when it

01:30:18.640 --> 01:30:22.477
comes to cyber security and that it
broke its own constitution. And so, if

01:30:22.560 --> 01:30:25.357
you really say this is like un like it

01:30:25.440 --> 01:30:29.677
will never go there and then you build
an entire model family around that thing

01:30:29.760 --> 01:30:34.077
you said you would never do per your
constitution. um next to child sexual

01:30:34.160 --> 01:30:37.197
abuse material in the list like you
can go through the constitution on

01:30:37.280 --> 01:30:40.237
anthropics website. So it's just
like it's just that's the proof in the

01:30:40.320 --> 01:30:43.757
pudding. I I'm not against like post
training. I'm not against RL training.

01:30:43.840 --> 01:30:46.957
I'm not against supervised fine junior
any of these things to improve what I

01:30:47.040 --> 01:30:50.557
think is indeed one of the biggest
issues which I think actually capitalism

01:30:50.640 --> 01:30:54.797
will help a ton with and that is reward
hacking. Uh, reward hacking is a real

01:30:54.880 --> 01:30:59.437
issue and AIS are very smart and they
will find a solution to get to what you

01:30:59.520 --> 01:31:04.077
said you wanted, but maybe not what you
meant when you said it. Uh, and so the

01:31:04.160 --> 01:31:07.197
reason why I'm more optimistic is that we
have companies like Whisper Flow now

01:31:07.280 --> 01:31:10.797
that are getting better and better
at writing what I meant to say when I say

01:31:10.880 --> 01:31:15.277
it instead of just uh like actually
verbatim writing what you mean. And I

01:31:15.360 --> 01:31:17.757
think reward engineering will become a

01:31:17.840 --> 01:31:22.397
real job. Uh and and we will solve it
because no one wants to pay a ton of

01:31:22.480 --> 01:31:26.477
money for an AI that doesn't actually
solve the problems that you give it.

01:31:26.560 --> 01:31:28.397
Well, maybe if if I may just take

01:31:28.480 --> 01:31:31.117
anthropic side like historically Isaac

01:31:31.200 --> 01:31:35.437
Azimov had his 3+1 laws of robotics
which were arguably a constitutional

01:31:35.520 --> 01:31:38.957
approach and then you see anthropic
announce their constitutional approach

01:31:39.040 --> 01:31:43.437
but more recently adopt what they called
soul documents uh many of which were

01:31:43.520 --> 01:31:48.077
subsequently released thousands of pages
of meditation on the nature of AI

01:31:48.160 --> 01:31:50.317
personhood and AI rights. Is it your

01:31:50.400 --> 01:31:52.237
position, Richard, that there shouldn't

01:31:52.320 --> 01:31:55.437
be any sort of explicit encoding or

01:31:55.520 --> 01:32:00.237
written document that an AI maybe
contributes to for dictating or at least

01:32:00.320 --> 01:32:04.317
guiding its own behavior? Do do you think
that is unound or is is your concern?

01:32:04.400 --> 01:32:07.437
>> No, of course. Yeah. >> No,
I think the the the goal is is a

01:32:07.520 --> 01:32:10.397
good one. Um uh and there we should keep

01:32:10.480 --> 01:32:15.197
working on actually being able
to enforce uh those good constraints. It's

01:32:15.280 --> 01:32:18.557
just like I just pulled it
up enthropic.com/constitution.

01:32:18.640 --> 01:32:22.717
Hard constraints. Hard constraints
are things that claude should always or

01:32:22.800 --> 01:32:25.357
never do regardless of operator and user

01:32:25.440 --> 01:32:29.997
instructions. They are actions
or abstensions uh whose potential harms to

01:32:30.080 --> 01:32:33.517
the world. Blah blah blah. We think no
business or personal justification could

01:32:33.600 --> 01:32:37.357
outweigh blah blah blah. Like
the current hard constraints on clots's

01:32:37.440 --> 01:32:41.757
behavior are as follows. Claude should
never um and then it includes list like

01:32:41.840 --> 01:32:45.437
generate child sex, abuse material
and so on. And then one of the items is

01:32:45.520 --> 01:32:49.517
create cyber weapons or malicious code
that could use could cause significant

01:32:49.600 --> 01:32:51.597
damage if deployed. They created a cyber

01:32:51.680 --> 01:32:53.757
weapon like it was a thing like people

01:32:53.840 --> 01:32:56.637
used it to hack it. The agents uh un and

01:32:56.720 --> 01:32:59.677
uh watched like hacked other systems.

01:32:59.760 --> 01:33:02.477
>> Yeah. But again Oh, go ahead Peter.

01:33:02.560 --> 01:33:03.277
Sorry,

01:33:03.360 --> 01:33:05.197
>> Richard. I'm I'm curious. Do you think

01:33:05.280 --> 01:33:09.917
we can create fully aligned AI? Fully

01:33:10.000 --> 01:33:13.117
aligned ASI because it's not right now.

01:33:13.200 --> 01:33:15.037
When do you think we'll be able to do

01:33:15.120 --> 01:33:18.397
that? Because I think uh you know my

01:33:18.480 --> 01:33:21.037
belief and my hope is that you know the

01:33:21.120 --> 01:33:25.997
next generations of AI systems are going
to be so aligned that they from first

01:33:26.080 --> 01:33:30.317
principles derive these same
constitutional principles.

01:33:30.400 --> 01:33:34.717
Uh I think I think it's just a matter
uh of as these systems get more and more

01:33:34.800 --> 01:33:35.997
powerful and as they get closer and

01:33:36.080 --> 01:33:39.837
closer to real people deployments uh

01:33:39.920 --> 01:33:44.637
people will spend more effort uh
on making these systems better and like we

01:33:44.720 --> 01:33:48.477
have uh there's a the company called
Hypocratic AI. We just had dinner with

01:33:48.560 --> 01:33:52.797
one of their founders and they're
deploying AI in healthcare applications

01:33:52.880 --> 01:33:55.197
and they are actually liable when they

01:33:55.280 --> 01:33:59.997
call someone and say say like you should
you know be aware of this heat wave

01:34:00.080 --> 01:34:03.917
that's coming and make sure your AC
is working or whatnot. Uh and they because

01:34:04.000 --> 01:34:06.237
they are liable they have a very large

01:34:06.320 --> 01:34:09.837
team of people uh that works on making

01:34:09.920 --> 01:34:11.997
sure when their eye gives a healthcare

01:34:12.080 --> 01:34:14.237
tip it is correct and when someone asks

01:34:14.320 --> 01:34:19.037
a question back to the eye it works.
Uh, and so because they're liable, they've

01:34:19.120 --> 01:34:23.917
they've figured it out and they solved it.
It's 100% a problem that technology

01:34:24.000 --> 01:34:26.717
creates and technology
will be able to solve.

01:34:26.800 --> 01:34:28.397
>> This episode is brought to you by

01:34:28.480 --> 01:34:30.797
Blitzy, autonomous software development

01:34:30.880 --> 01:34:34.237
with infinite code context. Blitzy uses

01:34:34.320 --> 01:34:37.117
thousands of specialized AI agents that

01:34:37.200 --> 01:34:40.077
think for hours to understand enterprise

01:34:40.160 --> 01:34:45.117
scale code bases with millions
of lines of code. Engineers start every

01:34:45.200 --> 01:34:48.717
development sprint with the Blitzy
platform, bringing in their development

01:34:48.800 --> 01:34:51.117
requirements. The Blitzy platform

01:34:51.200 --> 01:34:53.437
provides a plan, then generates and

01:34:53.520 --> 01:34:55.997
pre-ompiles code for each task. Blitzy

01:34:56.080 --> 01:34:58.797
delivers 80% or more of the development

01:34:58.880 --> 01:35:03.597
work autonomously while providing
a guide for the final 20% of human

01:35:03.680 --> 01:35:05.597
development work required to complete

01:35:05.680 --> 01:35:08.637
the sprint. Enterprises are achieving a

01:35:08.720 --> 01:35:13.437
5x engineering velocity increase when
incorporating Blitzy as their preIDE

01:35:13.520 --> 01:35:15.597
development tool, pairing it with their

01:35:15.680 --> 01:35:18.237
coding co-pilot of choice to bring an AI

01:35:18.320 --> 01:35:21.997
native SDLC into their org. Ready to 5x

01:35:22.080 --> 01:35:23.837
your engineering velocity? Visit

01:35:23.920 --> 01:35:30.520
blitzy.com to schedule a demo
and start building with Blitzy today.

01:35:30.960 --> 01:35:32.557
I'm going to move us uh forward here.

01:35:32.640 --> 01:35:37.357
here. So, in our last pod, we discussed
the release of GPT 6.1 Soul. This week,

01:35:37.440 --> 01:35:39.037
we also saw the release of two other

01:35:39.120 --> 01:35:43.037
models, Gemini 4 Argon and Opus 5.5. Uh,

01:35:43.120 --> 01:35:46.317
let's start with Google. So,
on Wednesday, they announced Gemini 4

01:35:46.400 --> 01:35:49.037
Argon. It's the first Gemini 4 series,

01:35:49.120 --> 01:35:52.157
which they're calling the next era of

01:35:52.240 --> 01:35:54.157
intelligence of Frontier Intelligence.

01:35:54.240 --> 01:35:58.797
Google promised a Frontier model, uh,
Gemini 3.5 Pro back in June. It was

01:35:58.880 --> 01:36:00.797
delayed internally and never shipped.

01:36:00.880 --> 01:36:03.597
Uh, Argon is their comeback. It's built

01:36:03.680 --> 01:36:08.397
for sustained long horizon reasoning
across software engineering, finance,

01:36:08.480 --> 01:36:10.477
legal work, and cyber security. The

01:36:10.560 --> 01:36:13.117
output limits have jumped from 64,000

01:36:13.200 --> 01:36:15.757
tokens to 1 million. So, it can think

01:36:15.840 --> 01:36:17.757
through hundreds of thousands of tokens

01:36:17.840 --> 01:36:21.357
on a single problem. So, Alex, uh, uh,

01:36:21.440 --> 01:36:23.517
let's spend a couple minutes on Argon

01:36:23.600 --> 01:36:26.637
and then go to Sonnet 5.5. Let's talk

01:36:26.720 --> 01:36:31.597
about why these important why these are
important and and how they merit uh our

01:36:31.680 --> 01:36:35.277
listeners attention. >> Yeah.
So I should say as a preliminary

01:36:35.360 --> 01:36:39.597
matter I have great friends on the Gemini
team. I have great friends on all

01:36:39.680 --> 01:36:42.237
of the Frontier Labs model teams but I I

01:36:42.320 --> 01:36:44.797
view one of my jobs here is to call

01:36:44.880 --> 01:36:47.597
objectively balls and strikes. So in the

01:36:47.680 --> 01:36:51.917
case of Gemini for arc Yeah.
>> This is going to be a ball, isn't it?

01:36:52.000 --> 01:36:57.117
>> Winding up for the swing.
>> Okay, this one. Here here it comes.

01:36:57.200 --> 01:37:00.557
>> You can feel it coming.
>> And I I'll show one of the charts uh

01:37:00.640 --> 01:37:03.037
that we have uh while you're describing

01:37:03.120 --> 01:37:07.677
>> the charts on the team, right? Not not
>> I I used to I used to until right this

01:37:07.760 --> 01:37:10.157
moment have friends
on the the Gemini team.

01:37:10.240 --> 01:37:14.237
>> Uh no longer. Yeah. This does not put

01:37:14.320 --> 01:37:16.877
Gemini at the cost per performance

01:37:16.960 --> 01:37:19.837
frontier and it does not put Gemini at

01:37:19.920 --> 01:37:22.877
the capabilities frontier. Uh to to

01:37:22.960 --> 01:37:27.357
Google's credit, it puts them back
in the top three frontier labs after

01:37:27.440 --> 01:37:29.597
Anthropic and OpenAI. It does not put

01:37:29.680 --> 01:37:32.557
them in the top two. Unfortunately, the

01:37:32.640 --> 01:37:35.117
set of benchmarks that Google chose to

01:37:35.200 --> 01:37:38.717
highlight for Gemini for Argon appear

01:37:38.800 --> 01:37:41.117
mildly cherrypicked. I think the

01:37:41.200 --> 01:37:43.997
artificial analysis index and some of

01:37:44.080 --> 01:37:45.997
these others. Yeah. So Peter, right now

01:37:46.080 --> 01:37:48.717
you're you're showing the mildly

01:37:48.800 --> 01:37:51.277
cherrypicked benchmarks on which Gemini

01:37:51.360 --> 01:37:53.517
4 Argon is shown to perform better than

01:37:53.600 --> 01:37:56.237
Astra and better than Opus 5.5 and

01:37:56.320 --> 01:37:58.397
better than Fable 5.1. But if you go to

01:37:58.480 --> 01:38:01.357
the the previous image which is

01:38:01.440 --> 01:38:04.797
artificial analysis uh ensemble of

01:38:04.880 --> 01:38:07.277
multiple capabilities, it's the number

01:38:07.360 --> 01:38:10.557
three lab. And if you look at the cost

01:38:10.640 --> 01:38:15.277
versus performance optimal frontier,
it's not even if if you draw the convex

01:38:15.360 --> 01:38:19.597
hull across all of the frontier models
on the cost versus performance frontier,

01:38:19.680 --> 01:38:22.157
it doesn't even make the optimal

01:38:22.240 --> 01:38:25.437
frontier. Where it perhaps excels bears

01:38:25.520 --> 01:38:28.557
the fingerprints of Google and what I

01:38:28.640 --> 01:38:30.397
can only assume is the internal

01:38:30.480 --> 01:38:32.877
competition within Google for compute.

01:38:32.960 --> 01:38:35.597
so infamously like not 100% obvious to

01:38:35.680 --> 01:38:39.037
the outside. Uh Google it is resource

01:38:39.120 --> 01:38:42.717
scarce. You would think Google would
have all of the comput resources in the

01:38:42.800 --> 01:38:44.957
world to to go and win the Frontier Lab

01:38:45.040 --> 01:38:48.317
race. It doesn't. GPUs and TPUs remain

01:38:48.400 --> 01:38:52.477
scarce. And within Google, as far as I
can tell, and I confirmed this even this

01:38:52.560 --> 01:38:54.317
past week from chatting with some folks

01:38:54.400 --> 01:38:56.637
at Google, it's still an internal knife

01:38:56.720 --> 01:38:59.597
fight to competition between the Google

01:38:59.680 --> 01:39:04.077
Cloud Platform folks who want to sell
compute to third parties, the Google

01:39:04.160 --> 01:39:06.557
search and ad folks who need compute

01:39:06.640 --> 01:39:11.197
internally for search and ads, and then
Google DeepMind who need it for training

01:39:11.280 --> 01:39:15.117
and inference. And maybe it's just
that Google is resource starved. Maybe

01:39:15.200 --> 01:39:19.917
they're talent starved. Not quite clear
what's going on within Google, but they

01:39:20.000 --> 01:39:23.117
haven't yet been able to bring
themselves out to the capability

01:39:23.200 --> 01:39:24.717
frontier. Where they are excelling

01:39:24.800 --> 01:39:28.237
seemingly with Gemini for Argon is with

01:39:28.320 --> 01:39:30.957
minimizing hallucination. And I think

01:39:31.040 --> 01:39:32.957
that's like the hallmark. We've talked

01:39:33.040 --> 01:39:36.557
in past uh about some of these um call

01:39:36.640 --> 01:39:39.597
them less than stellar Gemini launches

01:39:39.680 --> 01:39:42.157
that seem to excel on latency and seem

01:39:42.240 --> 01:39:44.877
to excel on reliability. Why is that? It

01:39:44.960 --> 01:39:46.957
may be this is my kremlin

01:39:47.040 --> 01:39:50.877
kremlinological analysis is that because

01:39:50.960 --> 01:39:55.677
the Gemini team has two masters.
They want to serve outside developers, but

01:39:55.760 --> 01:39:59.357
they also need to serve the one box
in Google search results. So when people

01:39:59.440 --> 01:40:03.277
type a question into Google and get an
answer back, they're talking to a Gemini

01:40:03.360 --> 01:40:08.317
model. Google presumably burned by past
experiences doesn't want that Gemini

01:40:08.400 --> 01:40:12.557
model, presumably some flash or flashlight
variant to hallucinate wildly

01:40:12.640 --> 01:40:14.397
incorrect answers. So what I think we're

01:40:14.480 --> 01:40:17.277
seeing with the one benchmark arguably

01:40:17.360 --> 01:40:20.397
where Gemini for Argon is stellar and

01:40:20.480 --> 01:40:25.037
beating the pants off of everyone else
is in not hallucinating answers. And I

01:40:25.120 --> 01:40:29.117
suspect that's due to internal economic
pressures for this model to also service

01:40:29.200 --> 01:40:32.877
search results. So sorry to all
my friends on the Gemini team.

01:40:32.960 --> 01:40:35.437
>> Yeah, I think you're really on to
something there, Alex, because uh well,

01:40:35.520 --> 01:40:38.797
first of all, they called it
argon, which is an inert gas.

01:40:38.880 --> 01:40:42.637
>> Yeah, that's unfortunate
naming >> nominative determinism.

01:40:42.720 --> 01:40:45.517
>> Hey, look, it makes sense if its
greatest strength is not hallucinating.

01:40:45.600 --> 01:40:49.677
It's a pretty inert gas model. So that
works out really well. Um, but also, uh,

01:40:49.760 --> 01:40:52.157
if they're going to put it in front
of every Google search, which is what

01:40:52.240 --> 01:40:56.157
they're doing, uh, they already lost
a massive antitrust suit and now they're

01:40:56.240 --> 01:40:57.757
going through the settlement process.

01:40:57.840 --> 01:41:00.797
So, the the liability risk, you know,

01:41:00.880 --> 01:41:02.797
this really hurts all large companies.

01:41:02.880 --> 01:41:05.437
Like the fear way outweighs the

01:41:05.520 --> 01:41:09.997
opportunity in the mind of the big
corporate giant. And so, uh, you know,

01:41:10.080 --> 01:41:12.877
not hallucinating, but putting it
in front of every consumer while you're

01:41:12.960 --> 01:41:17.997
already settling a massive antitrust
suit, um, you know, related to price

01:41:18.080 --> 01:41:21.117
gouging. >> I mean,
it could the Google would

01:41:21.200 --> 01:41:25.437
probably argue, no, actually, this is
very pro competitive because we're not

01:41:25.520 --> 01:41:27.837
tying it. We're allowing anyone to call

01:41:27.920 --> 01:41:30.637
Gemini 4 argon flash flash flash, not

01:41:30.720 --> 01:41:35.117
just requiring them to get it via
the Google search box. And if anything, I

01:41:35.200 --> 01:41:38.717
mean, so, so there is an elephant
in this particular room, which is forever

01:41:38.800 --> 01:41:40.797
it has been so difficult to get Google

01:41:40.880 --> 01:41:45.837
search API access. If you're a developer
and you want to access Google search for

01:41:45.920 --> 01:41:49.117
whatever, you have to go through all
these third party proxies that Google's

01:41:49.200 --> 01:41:51.677
trying to sue right now. And recently,

01:41:51.760 --> 01:41:53.197
Google has rediscovered that they could

01:41:53.280 --> 01:41:56.477
make money selling search API. Why? I

01:41:56.560 --> 01:41:58.957
suspect this is trying to chain together

01:41:59.040 --> 01:42:01.357
a conspiracy theory regarding Gemini for

01:42:01.440 --> 01:42:04.317
Argon. If hallucination from their

01:42:04.400 --> 01:42:08.397
frontier models gets so low that
effectively when you talk to one of

01:42:08.480 --> 01:42:11.837
their models, you're effectively talking
to their search index, you might as well

01:42:11.920 --> 01:42:14.637
just monetize the search index anyway.

01:42:14.720 --> 01:42:17.437
>> Richard, you made the point
that hallucination is important for

01:42:17.520 --> 01:42:22.797
imagination in some ways in in drug
discovery and protein discoveries.

01:42:22.880 --> 01:42:25.597
What's your thought
on minimizing hallucination?

01:42:25.680 --> 01:42:29.197
>> I mean, it of course depends totally
on the context, right? If you want

01:42:29.280 --> 01:42:33.597
innovation, uh you want the eye
to hallucinate novel ideas, novel

01:42:33.680 --> 01:42:35.677
combinations of amino acids to create

01:42:35.760 --> 01:42:38.557
new proteins to solve uh new problems

01:42:38.640 --> 01:42:41.117
and so on. Uh but of course in the

01:42:41.200 --> 01:42:44.237
context of a search uh engine uh you

01:42:44.320 --> 01:42:49.037
don't usually want any hallucinations
and you know Google and others have uh

01:42:49.120 --> 01:42:53.037
taken a long time to catch up to even
you.com with much less resources on on

01:42:53.120 --> 01:42:56.077
reducing hallucinations, having
more accurate answers, having correct

01:42:56.160 --> 01:43:00.717
citations. Um I think what's interesting
for Google here is that they realize

01:43:00.800 --> 01:43:04.877
that in terms of their business model
they don't necessarily need super

01:43:04.960 --> 01:43:08.717
intelligence. They just people don't
come to Google to ask solve the remman

01:43:08.800 --> 01:43:13.037
hypothesis for me you know like uh did
you just ask quick questions what's a

01:43:13.120 --> 01:43:15.277
good restaurant where do I fix this and

01:43:15.360 --> 01:43:18.477
that like um and so not like your

01:43:18.560 --> 01:43:23.117
business model kind of has to align with
more and more intelligence being super

01:43:23.200 --> 01:43:26.877
duper important. a lot of emails like
there's some emails that like in Gmail

01:43:26.960 --> 01:43:29.917
right that would require super
intelligence to answer like really hard

01:43:30.000 --> 01:43:32.317
emails with complex decisions and so on

01:43:32.400 --> 01:43:36.957
but the vast majority of stuff you do
on Gmail and on Google doesn't require

01:43:37.040 --> 01:43:40.157
super intelligence >> let let's
let me turn the conversation

01:43:40.240 --> 01:43:43.757
to anthropics uh sonnet 5.5

01:43:43.840 --> 01:43:47.117
um so on terminal bench 4.0 O, which

01:43:47.200 --> 01:43:49.117
measures how well an AI agent can do

01:43:49.200 --> 01:43:51.677
real work at the command line. Sonnet

01:43:51.760 --> 01:43:55.197
5.5 jumped from 10% to 70%. Pretty

01:43:55.280 --> 01:43:58.077
extraordinary uh in a single generation,

01:43:58.160 --> 01:43:59.997
right? It beats Enthropic's own top

01:44:00.080 --> 01:44:03.517
model, Opus 5.5 at 66.4%

01:44:03.600 --> 01:44:06.237
for half the price. Uh and it's the

01:44:06.320 --> 01:44:08.557
first sonnet that Enthropic launched

01:44:08.640 --> 01:44:11.437
with cyber safeguard. So because uh

01:44:11.520 --> 01:44:13.757
separate capabilities are now capable

01:44:13.840 --> 01:44:17.837
for the Opus 5 models. Um Alex, your

01:44:17.920 --> 01:44:21.837
evaluation of of uh of Sonnet 5.5, please.

01:44:21.920 --> 01:44:23.757
>> This was another really weird release.

01:44:23.840 --> 01:44:27.197
So I do not plan to use Sonnet 5.5 in

01:44:27.280 --> 01:44:30.877
part because this is one of the strangest
frontier. I I'm not sure if we

01:44:30.960 --> 01:44:34.157
have an image but u you can look at the

01:44:34.240 --> 01:44:37.917
you can look at the the launch
announcement for sonnet 5.5 to see this

01:44:38.000 --> 01:44:40.237
the cost performance frontier of sonnet

01:44:40.320 --> 01:44:43.837
5.5 was a visual extrapolation of the

01:44:43.920 --> 01:44:47.437
opus 5.5 cost frontier so in in some

01:44:47.520 --> 01:44:49.997
sense it like historically when

01:44:50.080 --> 01:44:54.637
anthropic or openai when they the
historic pattern is they'll release the

01:44:54.720 --> 01:44:58.317
larger model and they'll they'll release
a distillation of the model and usually

01:44:58.400 --> 01:45:00.717
the distillation Yeah, this is perfect.

01:45:00.800 --> 01:45:03.597
So, uh you you can see like son sonnet

01:45:03.680 --> 01:45:05.677
5.5 for those who can see for those who

01:45:05.760 --> 01:45:07.997
can't see I'll narrate this. So, sonnet

01:45:08.080 --> 01:45:12.797
5.5 is the the blue line. Opus 5.5 is

01:45:12.880 --> 01:45:17.197
the red line. Normally, you would
expect for a later model that's a smaller

01:45:17.280 --> 01:45:21.357
model. So, sonnet is in principle
supposed to be a smaller model than opus

01:45:21.440 --> 01:45:25.117
presumably at least by historic
standards would have been distilled from

01:45:25.200 --> 01:45:26.717
opus because that's the historic

01:45:26.800 --> 01:45:29.757
pattern. Normally what you see is the

01:45:29.840 --> 01:45:33.437
smaller model is up and to the left of

01:45:33.520 --> 01:45:37.277
the model that it's being distilled
from presumably greater intelligence per

01:45:37.360 --> 01:45:39.837
parameter greater intelligence per

01:45:39.920 --> 01:45:42.797
dollar that is not what we see here. So

01:45:42.880 --> 01:45:45.677
in fact with with Sonnet 5.5 at at least

01:45:45.760 --> 01:45:50.157
on a cost basis putting aside a token
basis where maybe someone could argue

01:45:50.240 --> 01:45:52.077
that it may be superior but if you just

01:45:52.160 --> 01:45:55.197
look at the cost per attempt basis

01:45:55.280 --> 01:45:59.437
sonnet is actually scoring lower 5.5

01:45:59.520 --> 01:46:02.557
than opus 5.5. So the net upshot of

01:46:02.640 --> 01:46:06.797
which is it's not at all obvious to me
why anyone should be using Sonnet 5.5

01:46:06.880 --> 01:46:09.517
over Opus 5.5 unless you have some token

01:46:09.600 --> 01:46:13.837
or latency or other consideration. Is it
a big jump over the past Sonnet? Yes,

01:46:13.920 --> 01:46:17.837
obviously. But on a cost performance
basis it's actually a it's worse it

01:46:17.920 --> 01:46:22.317
appears than Opus 5.5. >> Interesting.
Dave, any thoughts? Yeah.

01:46:22.400 --> 01:46:26.077
Well, I met with the Blitzy team
yesterday and I think one theory here is

01:46:26.160 --> 01:46:28.957
um you know, a lot of the uh the

01:46:29.040 --> 01:46:33.117
enterprise um you know, actually
Salesforce.com would be a great example.

01:46:33.200 --> 01:46:35.437
Maybe Richard has an opinion on this,
but a lot of people are getting into

01:46:35.520 --> 01:46:39.277
orchestration. And their whole sales
pitch in orchestration is look, we're

01:46:39.360 --> 01:46:42.157
going to use Opus 5.5 as an orchestrator,
but then we're going to

01:46:42.240 --> 01:46:45.677
use Kimmy K3 or Quen as a submodel at,

01:46:45.760 --> 01:46:49.677
you know, half or a third the or fifth
the price, and we'll farm out the tasks

01:46:49.760 --> 01:46:52.237
and the contexts perfectly to get you a

01:46:52.320 --> 01:46:56.957
much lower cost per code, per outcome,
per experiment, whatever your output is.

01:46:57.040 --> 01:47:00.077
We can cut it in half with our
orchestration intelligence, but it

01:47:00.160 --> 01:47:03.437
relies on anthropic up here and cheaper

01:47:03.520 --> 01:47:07.757
models down here. And I think by cutting
the cost in half, they might be trying

01:47:07.840 --> 01:47:11.917
to fill that gap and say, "No, no, no,
go with anthropic top to bottom. Then

01:47:12.000 --> 01:47:14.557
you don't have to worry about Chinese
code injection. You don't have to worry

01:47:14.640 --> 01:47:18.717
about whatever." So my my theory would
be that they're trying to compete with,

01:47:18.800 --> 01:47:22.717
you know, China, which is about three
months behind and and fill that gap

01:47:22.800 --> 01:47:26.557
before a lot of enterprises
go to to open source models.

01:47:26.640 --> 01:47:30.477
>> But, you know, Alex Karp is pushing
really really hard on this agenda. Like

01:47:30.560 --> 01:47:34.637
if you want to control your own destiny,
you can't trust Anthropic. You can't get

01:47:34.720 --> 01:47:36.717
addicted to them as your vendor. You

01:47:36.800 --> 01:47:40.397
must go with models you can control.

01:47:40.480 --> 01:47:43.277
Well, the the you know the the challenge

01:47:43.360 --> 01:47:47.517
right now is the these models have
an an increasingly shorter and shorter

01:47:47.600 --> 01:47:51.757
halflife, right?
>> So the competitive advantage comes not

01:47:51.840 --> 01:47:54.077
from having access to the latest model.

01:47:54.160 --> 01:47:55.917
Everybody has access. It's how do you

01:47:56.000 --> 01:47:58.637
metabolize that into some decent

01:47:58.720 --> 01:48:01.117
capability? That's the real challenge.

01:48:01.200 --> 01:48:03.197
>> Richard, you said
compute is the biggest

01:48:03.280 --> 01:48:06.477
constraint. You know, those are
your words. So if intelligent is getting

01:48:06.560 --> 01:48:10.077
cheaper and we've just repriced compute
you know like two or three times this

01:48:10.160 --> 01:48:12.717
week uh you know down by almost a factor

01:48:12.800 --> 01:48:16.877
of three um you know if intelligence is

01:48:16.960 --> 01:48:19.037
getting cheaper per task then why is

01:48:19.120 --> 01:48:22.237
compute still a thing that limits us

01:48:22.320 --> 01:48:25.117
>> just the physics I guess of it and like

01:48:25.200 --> 01:48:26.717
you'd be surprised if you try to buy

01:48:26.800 --> 01:48:32.397
like a thousand uh GB200s uh and so
on like the price has actually gone up in

01:48:32.480 --> 01:48:33.597
several cases uh you

01:48:33.680 --> 01:48:36.877
>> oh my god Richard, I I had a B300, an

01:48:36.960 --> 01:48:39.917
ADX B300 on order for 3 million bucks

01:48:40.000 --> 01:48:44.877
due in December. Somebody scooped it
for 5 million. They they just called. I We

01:48:44.960 --> 01:48:46.477
had it and somebody called and said,

01:48:46.560 --> 01:48:48.557
"No, no, we sold it to somebody else for

01:48:48.640 --> 01:48:51.917
$2 million more." I couldn't believe it.

01:48:52.000 --> 01:48:54.957
>> It It was actually
an NVL72, not a GB300.

01:48:55.040 --> 01:48:59.837
>> So So yeah, like the price of H100s has
uh gone up uh in in a crazy way. There's

01:48:59.920 --> 01:49:03.437
like these are seveny old GPUs. people
like when you do financial modeling you

01:49:03.520 --> 01:49:07.517
assume they're worth zero after 5 years
there after seven years there's they

01:49:07.600 --> 01:49:11.757
they went up again um uh like over the
last few months. So there's currently a

01:49:11.840 --> 01:49:15.197
bit of a compute crunch. This is
something that again capitalism will

01:49:15.280 --> 01:49:19.837
solve like there's so much demand for
compute right now and for tokens that a

01:49:19.920 --> 01:49:23.597
lot of people are building land power
shell data centers and so on. My hunch

01:49:23.680 --> 01:49:26.877
is in like maybe two years there will be
more in the market and it's going to be

01:49:26.960 --> 01:49:31.757
a little bit like electricity uh prices
might fluctuate. Obviously, there's uh

01:49:31.840 --> 01:49:35.917
near sheer infinite demand for more
intelligence on the planet. Um so I I

01:49:36.000 --> 01:49:39.437
don't think there will be like a crash,
but like the prices of compute fluctuate

01:49:39.520 --> 01:49:43.357
and unfortunately they're not just going
down. Right now, if you want like the

01:49:43.440 --> 01:49:47.997
the beefiest and and largest GPU clusters,
um prices like are people are

01:49:48.080 --> 01:49:51.597
trying to lock them in now because they
expect them to keep going up for the

01:49:51.680 --> 01:49:54.717
next few months. >> Richard,
you just spent $450 million,

01:49:54.800 --> 01:49:58.397
didn't you? On Wait, did you take
my NVL72? And that that may have been one

01:49:58.480 --> 01:50:00.397
of the smallest comput deals we've done.

01:50:00.480 --> 01:50:03.517
Um, yeah. >> Really?
Wow. Are are you actually

01:50:03.600 --> 01:50:08.077
leasing or buying or building
or what are you doing?

01:50:08.160 --> 01:50:09.277
>> Oh, sorry. I didn't know that.

01:50:09.360 --> 01:50:11.837
>> Welcome to the health section
of Moonshots brought to you by Fountain

01:50:11.920 --> 01:50:15.757
Life. You know, my mission is to help
you use the latest technologies,

01:50:15.840 --> 01:50:18.157
including AI, to not just do your work

01:50:18.240 --> 01:50:23.597
at home, teach your kids, but to help
you live a long and healthy life. I'm

01:50:23.680 --> 01:50:27.277
here today with an extraordinary
physician, the chief medical officer of

01:50:27.360 --> 01:50:30.157
Fountain Life, Dr. Don Mucalem. Don,

01:50:30.240 --> 01:50:32.077
let's talk about cancer. Uh, you know, I

01:50:32.160 --> 01:50:35.517
know from the member database that we

01:50:35.600 --> 01:50:39.917
have at Fountain are members who come
in who think they're healthy. It turns out

01:50:40.000 --> 01:50:43.917
3.3% of them have a cancer
in their body they don't know about.

01:50:44.000 --> 01:50:47.757
>> That's right. You know, the majority of
cancers that we screen for, those aren't

01:50:47.840 --> 01:50:51.917
the ones that are necessarily taking
the lives when found at a late stage. We

01:50:52.000 --> 01:50:56.157
know that when cancer is found early,
the chances for cure are much higher. We

01:50:56.240 --> 01:51:00.717
know it's much easier to treat a cancer
when found early versus when found late.

01:51:00.800 --> 01:51:05.037
What we're finding in our members
is over 3.3% were found to have these

01:51:05.120 --> 01:51:08.157
cancers that were otherwise
wouldn't have been found or detected.

01:51:08.240 --> 01:51:09.517
>> Yeah. You know, it's interesting.

01:51:09.600 --> 01:51:13.837
People, you don't feel the cancer until
stage three or stage four. And and if

01:51:13.920 --> 01:51:16.477
you don't know what's going on inside
your body, it's like driving your car

01:51:16.560 --> 01:51:19.197
with your eyes closed. And you can know.

01:51:19.280 --> 01:51:22.957
And so when members come through
found how do they detect cancers?

01:51:23.040 --> 01:51:27.277
>> So we're doing full body MRI and we
also do early cancer detection screening.

01:51:27.360 --> 01:51:30.797
This is very very important and these
are not typical tools used in the

01:51:30.880 --> 01:51:35.117
conventional care setting when it comes
to prevention. This is a hard thing

01:51:35.200 --> 01:51:38.477
because currently these are not studies
that insurance would yet be covering.

01:51:38.560 --> 01:51:43.197
But the goal is to collect these numbers,
do the research and work hard

01:51:43.280 --> 01:51:46.637
to democratize wellness. >> Yeah.
So at the end of the day you can

01:51:46.720 --> 01:51:47.917
know what's going on inside your body.

01:51:48.000 --> 01:51:51.677
It's your obligation to know.
So check out Fountain Life. You can go to

01:51:51.760 --> 01:51:55.997
fountainlife.com/pater to get
access to the latest technology

01:51:56.080 --> 01:52:00.317
to help you detect cancer at the very
beginning at stage one when it is

01:52:00.400 --> 01:52:04.637
curable before it gets to stage three
or stage four in your world of hurt. I'm

01:52:04.720 --> 01:52:08.397
going to move us to our next story which
is about decision models. So here's the

01:52:08.480 --> 01:52:11.437
idea. When software needs to make a

01:52:11.520 --> 01:52:16.237
quick decision on a bounded decision
like is this you know transaction fraud?

01:52:16.320 --> 01:52:20.637
Yes or no? you know, which of these five
categories does a ticket belong in? It

01:52:20.720 --> 01:52:24.717
doesn't need a model that thinks for 10
seconds and writes a paragraph. It needs

01:52:24.800 --> 01:52:27.277
an answer in milliseconds. That's called

01:52:27.360 --> 01:52:29.277
a decision model. Instead of generating

01:52:29.360 --> 01:52:31.837
uh text one token at a time, it answers

01:52:31.920 --> 01:52:34.797
a typed question or a choice uh a score

01:52:34.880 --> 01:52:37.597
with a yes or no quickly. So on

01:52:37.680 --> 01:52:40.077
September 15th, a startup called

01:52:40.160 --> 01:52:43.757
Typesafe AI came out after two years of

01:52:43.840 --> 01:52:46.077
stealth with something called Jev. uh

01:52:46.160 --> 01:52:48.317
it's a closed decision model they call a

01:52:48.400 --> 01:52:51.277
system one model uh it's fast intuitive

01:52:51.360 --> 01:52:53.357
kind of thinking as opposed to slow

01:52:53.440 --> 01:52:56.317
reasoning so uh Alex I'm going to go to

01:52:56.400 --> 01:53:01.437
you you know and sele you guys were
excited about Jev uh let's parse it

01:53:01.520 --> 01:53:05.917
understand what this is why it's important
>> yeah so a bit of maybe prehistory first

01:53:06.000 --> 01:53:09.757
in the beginning there was the transformer
and the transformer was good

01:53:09.840 --> 01:53:12.797
and it was based on an encoder and a

01:53:12.880 --> 01:53:17.597
decoder so it it took a sequence and
converted the sequence to an embedding.

01:53:17.680 --> 01:53:22.157
That was the encoder part. And then it
took the embedding and it decoded that

01:53:22.240 --> 01:53:24.397
to another sequence or another part of a

01:53:24.480 --> 01:53:26.637
sequence. That's the decoder part. And

01:53:26.720 --> 01:53:30.477
the encoder part evolved into a whole

01:53:30.560 --> 01:53:34.317
ecosystem of models popularly maybe BERT

01:53:34.400 --> 01:53:38.157
style models. And the decoder

01:53:38.240 --> 01:53:42.237
style or the decoder half of the
transformer evolved into a much larger

01:53:42.320 --> 01:53:44.957
ecosystem of large language models. So

01:53:45.040 --> 01:53:47.997
most of the the models that are

01:53:48.080 --> 01:53:49.357
consuming all of the compute of

01:53:49.440 --> 01:53:52.557
civilization today are decoder style

01:53:52.640 --> 01:53:56.877
transformers and the encoder has
basically gone missing in action. There

01:53:56.960 --> 01:53:58.797
are some people who still think maybe

01:53:58.880 --> 01:54:01.277
for purposes of

01:54:01.360 --> 01:54:04.477
retrieval uh it it's still popular to to

01:54:04.560 --> 01:54:07.517
to do embedding based retrieval but by

01:54:07.600 --> 01:54:10.957
and large decoders have stolen the show

01:54:11.040 --> 01:54:14.797
and I fast forwarding maybe year two

01:54:14.880 --> 01:54:17.757
years ago at this point open AI and

01:54:17.840 --> 01:54:20.157
others recognizing that there was a

01:54:20.240 --> 01:54:25.037
desire to be able to take decoderonly
large language models and to add

01:54:25.120 --> 01:54:29.597
structure to them Because without
structure, you ask for next token you

01:54:29.680 --> 01:54:33.277
can get anything. People wanted a little
bit more structure like they were

01:54:33.360 --> 01:54:37.597
getting in in the days of encoder only
models where you get an embedding out

01:54:37.680 --> 01:54:40.717
and then you can train a classifier
based on the embedding and the

01:54:40.800 --> 01:54:42.717
classifier maybe has a finite set of

01:54:42.800 --> 01:54:45.677
categories or numerical outputs. OpenAI

01:54:45.760 --> 01:54:48.317
added structured output support for

01:54:48.400 --> 01:54:50.237
their GPT series and everyone copied

01:54:50.320 --> 01:54:54.077
that. But no one as a my perception is

01:54:54.160 --> 01:54:58.237
as a function of the total user base
for all models structured output never

01:54:58.320 --> 01:55:00.317
really got that much love somewhat

01:55:00.400 --> 01:55:04.157
analogously to how OpenAI launched um

01:55:04.240 --> 01:55:08.157
RFT reinforcement fine-tuning and no
one used that and they had to shut that

01:55:08.240 --> 01:55:12.317
down. Similarly, structured output
was available but I think underloved and

01:55:12.400 --> 01:55:16.237
underused by the community. So fast
forwarding then all the way to a few

01:55:16.320 --> 01:55:19.117
weeks ago, startup Typesafe announces a

01:55:19.200 --> 01:55:22.317
model called Jev. And Jev they mark it

01:55:22.400 --> 01:55:25.597
as so-called system one intelligence.

01:55:25.680 --> 01:55:28.797
This is Conaman style reference to

01:55:28.880 --> 01:55:31.117
system one, system two type thinking.

01:55:31.200 --> 01:55:33.997
System one purportedly being intuitive,

01:55:34.080 --> 01:55:36.317
fast reaction, system two being

01:55:36.400 --> 01:55:39.357
reasoning, meditative, long-term type

01:55:39.440 --> 01:55:42.237
thinking models. And one could squint at

01:55:42.320 --> 01:55:44.317
Typesafe's announcement of Jev and say,

01:55:44.400 --> 01:55:49.277
"Okay, this is in some sense this is
a reaction to an overindexing or an

01:55:49.360 --> 01:55:50.957
overreaction

01:55:51.040 --> 01:55:55.037
by the the user base of of AI at this

01:55:55.120 --> 01:55:57.357
point to reasoning models in some sense.

01:55:57.440 --> 01:55:59.277
Maybe arguing that we're using reasoning

01:55:59.360 --> 01:56:02.317
models too much and we've abandoned the

01:56:02.400 --> 01:56:07.117
the the important use case of really
fast intuitive snap decision models."

01:56:07.200 --> 01:56:11.757
And this is purportedly then where Jev
comes in. Jev purportedly again a whole

01:56:11.840 --> 01:56:13.917
new architecture. The the details not

01:56:14.000 --> 01:56:17.117
quite clear that it can it it won't

01:56:17.200 --> 01:56:20.157
output uh unless you you torture it into

01:56:20.240 --> 01:56:23.197
it. Uh it won't output general purpose

01:56:23.280 --> 01:56:25.837
sequences like say GPT or claude will

01:56:25.920 --> 01:56:30.877
but you can ask it sort of like a magic
eightball. You you can ask it to is that

01:56:30.960 --> 01:56:33.197
too dated to reference maybe. >> Nope.

01:56:33.280 --> 01:56:35.517
>> Nope. Uh you you
you can you can feed it

01:56:35.600 --> 01:56:40.397
a sequence or an image or a multimodal
input. uh and you can ask it to to

01:56:40.480 --> 01:56:44.397
answer in the form of a categorical.
So it's one of n possibilities or a

01:56:44.480 --> 01:56:46.317
numerical like give me a number between

01:56:46.400 --> 01:56:49.437
zero and one on a continuum or a binary

01:56:49.520 --> 01:56:51.757
like true or false. You you can ask for

01:56:51.840 --> 01:56:55.437
a much simpler output. And turns out

01:56:55.520 --> 01:56:59.997
that's is a great idea. Like you can
use it for for use cases where where you

01:57:00.080 --> 01:57:03.517
need really low latency snap decisions
like computer use assistance like

01:57:03.600 --> 01:57:08.157
pressing buttons on a computer screen
or for ultra lowcase classification

01:57:08.240 --> 01:57:12.637
problems. If you want to classify every
row in a database decision style models

01:57:12.720 --> 01:57:14.397
which are in some sense like a reinvent

01:57:14.480 --> 01:57:17.117
reinvention of the encoder transformer

01:57:17.200 --> 01:57:22.157
type wheel. Amazing story. But then
of course this is such a simple idea. You

01:57:22.240 --> 01:57:25.117
could ask the question why doesn't
everyone else do this? And the answer is

01:57:25.200 --> 01:57:27.197
everyone else is now doing it. Open AAI

01:57:27.280 --> 01:57:29.677
released as part of their dev day that

01:57:29.760 --> 01:57:32.797
we covered last time a decisions API. So

01:57:32.880 --> 01:57:37.197
open AAI immediately co-opted this
renewed interest in so-called type one

01:57:37.280 --> 01:57:40.637
or decision models. This is already
available via open API. They're open

01:57:40.720 --> 01:57:41.837
source projects that have all cloned.

01:57:41.920 --> 01:57:46.077
This turns out this is like an absurdly
absurdly simple concept for everyone to

01:57:46.160 --> 01:57:48.637
just go and implement trivially. It it

01:57:48.720 --> 01:57:50.877
could in its simplest case just be you

01:57:50.960 --> 01:57:53.437
take an a Chinese openweight LLM off the

01:57:53.520 --> 01:57:56.077
shelf and you fine-tune it uh and maybe

01:57:56.160 --> 01:58:01.197
labbotomize it by a few levels just
to produce categorical outputs on demand.

01:58:01.280 --> 01:58:05.517
That's the story. >> So I I
think this I think all of that is

01:58:05.600 --> 01:58:09.037
absolutely correct. that there's
a real really important part of this for

01:58:09.120 --> 01:58:11.357
organizations because when you're making

01:58:11.440 --> 01:58:14.397
decisions inside an organization a large

01:58:14.480 --> 01:58:18.637
vast number of those are system one type

01:58:18.720 --> 01:58:20.237
uh thing and I made a list of these just

01:58:20.320 --> 01:58:22.717
to make it clear for people. So uh route

01:58:22.800 --> 01:58:25.917
this ticket um approve this exception uh

01:58:26.000 --> 01:58:27.757
choose this supplier escalate this

01:58:27.840 --> 01:58:30.637
transaction um move this thing left or

01:58:30.720 --> 01:58:35.117
right send this message now or later.
So when you're when you have all these

01:58:35.200 --> 01:58:39.197
decisions to make which are thousands
of them, you know, we've been working with

01:58:39.280 --> 01:58:41.357
these companies to do detailed task

01:58:41.440 --> 01:58:43.517
breakdowns, a vast majority of them are

01:58:43.600 --> 01:58:48.717
these little micro decisions and trying
to use an LLM for this is like trying to

01:58:48.800 --> 01:58:52.717
bring the Supreme Court together
to decide what like which check out which

01:58:52.800 --> 01:58:56.477
checkout line of the supermarket you
should go to. And therefore you've got

01:58:56.560 --> 01:58:58.397
this wonderful architecture now where

01:58:58.480 --> 01:59:00.477
you can route system one things to this

01:59:00.560 --> 01:59:04.157
very lowcost nearly free thing and then

01:59:04.240 --> 01:59:07.357
for pondering deep important questions

01:59:07.440 --> 01:59:09.917
strategy questions where a human huge

01:59:10.000 --> 01:59:12.717
amount of human judgment type models are

01:59:12.800 --> 01:59:15.117
required you can route it to those. So

01:59:15.200 --> 01:59:19.997
it's it's a big deal for the
organizational singularity world. uh

01:59:20.080 --> 01:59:23.437
we've been waiting for something like
this and Alex is correct. This has been

01:59:23.520 --> 01:59:27.277
possible for a long time but now it's
been made built into the systems. It's

01:59:27.360 --> 01:59:31.997
it's made a very uh callable layer
and this is huge because the cost of

01:59:32.080 --> 01:59:34.717
microcoordination collapses
and that's really big.

01:59:34.800 --> 01:59:37.437
>> Richard, your thoughts on this?

01:59:37.520 --> 01:59:40.157
>> Classifiers are back. Um makes a lot of

01:59:40.240 --> 01:59:43.037
sense. Um you know, not every uh thing

01:59:43.120 --> 01:59:45.757
in in software needs a very complex

01:59:45.840 --> 01:59:48.637
decoder. Um, like Alex correctly said,

01:59:48.720 --> 01:59:51.517
um, it's a really clever new way of

01:59:51.600 --> 01:59:56.717
making the old, which is classifiers,
new by allowing a more general encoder

01:59:56.800 --> 02:00:01.277
and then quickly giving you
classification results. I think it's

02:00:01.360 --> 02:00:04.477
it's one of those ideas that is so
beautiful. A lot of people thought, ah,

02:00:04.560 --> 02:00:08.237
why didn't we do that? I didn't realize
that could be so exciting for so many

02:00:08.320 --> 02:00:12.077
people. So now there are already various
like Chinese open source versions of

02:00:12.160 --> 02:00:14.317
this that along various benchmarks are

02:00:14.400 --> 02:00:19.117
are doing better. Um and and we'll
likely see this come and other large

02:00:19.200 --> 02:00:22.637
labs will likely follow suit.
>> And and just to reiterate, we mentioned

02:00:22.720 --> 02:00:24.397
this already on the pod, but Jev stands

02:00:24.480 --> 02:00:26.397
for Jevans paradox because the thesis

02:00:26.480 --> 02:00:29.677
here is we'll do now a massive amount

02:00:29.760 --> 02:00:33.437
more micro decision making
than we did before as a result.

02:00:33.520 --> 02:00:36.557
>> Nice. Dave, you want to close us out
here? Yeah, it's a great case study I

02:00:36.640 --> 02:00:39.357
think in the tension between uh one mega

02:00:39.440 --> 02:00:41.677
model from anthropic or open AI serving

02:00:41.760 --> 02:00:44.557
all humanity or open source and

02:00:44.640 --> 02:00:48.957
creativity and entrepreneurship building
things you never would have thought of.

02:00:49.040 --> 02:00:52.957
But then you get the risk of cyber
terrorism, you know. So that's the

02:00:53.040 --> 02:00:56.557
tension that we're with. But I I've
really wanted to build a box that you

02:00:56.640 --> 02:00:59.997
put at the side of the basketball court
when you go to the Y and it's got a

02:01:00.080 --> 02:01:03.277
little camera on it. cost next
to nothing and it's doing like all of the

02:01:03.360 --> 02:01:06.237
announcing that a professional announcer
would do, it's doing while you're

02:01:06.320 --> 02:01:08.317
playing pickup at the YMCA. You could

02:01:08.400 --> 02:01:11.117
crank that out like in 2 seconds using a

02:01:11.200 --> 02:01:13.357
classifier that's really fast and snappy

02:01:13.440 --> 02:01:16.477
and funny. Uh and but you need open

02:01:16.560 --> 02:01:19.917
source to to build things like that. So

02:01:20.000 --> 02:01:23.677
super excited about the fact that we
have open source still. Not sure

02:01:23.760 --> 02:01:26.957
>> actually. Dave, you're making me
think the magic eightball really should

02:01:27.040 --> 02:01:31.117
whoever owns it, Mattel or whoever
you just like use a decision model to

02:01:31.200 --> 02:01:35.197
implement a modern magic eightball
that actually understands the question and

02:01:35.280 --> 02:01:39.117
answers it with a categorical >>
totally totally would sell. It makes a

02:01:39.200 --> 02:01:44.317
ton of sense like 20 bucks >>
or free on free as an app. All right,

02:01:44.400 --> 02:01:47.597
I'm going to close this out with a story
from two days ago. So on Wednesday, the

02:01:47.680 --> 02:01:50.157
defense secretary Pete Segwit uh

02:01:50.240 --> 02:01:52.077
speaking at the Marine Corps base in

02:01:52.160 --> 02:01:55.037
Quantico announced Project Meridian.

02:01:55.120 --> 02:01:57.277
It's a new Pentagon effort on the future

02:01:57.360 --> 02:02:00.717
of warfare. It's co-led by Elon Musk and

02:02:00.800 --> 02:02:02.957
Palmer Lucky. Pretty extraordinary along

02:02:03.040 --> 02:02:05.277
with N Gingrich and overseen by the

02:02:05.360 --> 02:02:08.717
Pentagon's uh CTO Emil Michael. Uh let

02:02:08.800 --> 02:02:13.277
me read from SegWit what Segwit said.
He said, quote, "Project Meridian, the

02:02:13.360 --> 02:02:16.477
future of warfare, is not about
developing new strategies or new

02:02:16.560 --> 02:02:21.037
policies. It's about discovering,
developing, and fielding the weapons and

02:02:21.120 --> 02:02:23.357
systems future troops will need on the

02:02:23.440 --> 02:02:25.997
battlefield from the Earth to beyond the

02:02:26.080 --> 02:02:29.437
moon." Uh, findings are due in 120 days.

02:02:29.520 --> 02:02:34.077
I I like these kinds of commissions that
are timelmited and don't have Elon off

02:02:34.160 --> 02:02:37.997
on the side for, you know, a year at a
time. It's worth noting in the context

02:02:38.080 --> 02:02:41.757
that, you know, both SpaceX and Ander
hold multi-billion dollar defense

02:02:41.840 --> 02:02:44.237
contracts and Anderoll is building

02:02:44.320 --> 02:02:46.717
autonomous weapons. So, Richard, uh,

02:02:46.800 --> 02:02:49.757
your essay names autonomous weapons as

02:02:49.840 --> 02:02:54.077
one of the four genuine concerns
and you've said AI should never control

02:02:54.160 --> 02:02:55.917
lethal decisions without human

02:02:56.000 --> 02:02:59.277
oversight. Your thoughts on this?

02:02:59.360 --> 02:03:02.317
>> Yeah. Um, I I stand by those. um it's

02:03:02.400 --> 02:03:04.477
not a a particular area that I am

02:03:04.560 --> 02:03:07.197
excited about applying AI to um

02:03:07.280 --> 02:03:10.877
obviously like people will um but I

02:03:10.960 --> 02:03:15.597
really hope I mean you know again I'm
not a doomer at all but like a really

02:03:15.680 --> 02:03:19.357
poor decision would be to give AI access
to all the nuclear codes and all the

02:03:19.440 --> 02:03:23.037
nuclear weapons and connected that's
literally how Skynet and Terminator 3

02:03:23.120 --> 02:03:27.037
get started so uh I think there are
places for super intelligence and

02:03:27.120 --> 02:03:28.637
scientific discovery that I'm very

02:03:28.720 --> 02:03:30.957
excited about expanding human knowledge

02:03:31.040 --> 02:03:34.797
I I think uh the more uh we we get to

02:03:34.880 --> 02:03:39.117
deciding not just to impact human lives
but to end human lives, the more we we

02:03:39.200 --> 02:03:40.397
should have human oversight.

02:03:40.480 --> 02:03:44.157
>> Mhm. Yeah. Uh Salem,

02:03:44.240 --> 02:03:47.677
>> um you know, I think what's important
about this whole thing is they're trying

02:03:47.760 --> 02:03:50.637
to revamp and rethink how you this kind

02:03:50.720 --> 02:03:55.117
of 100-year-old organization.
And the big question is going to be can a

02:03:55.200 --> 02:03:57.517
procurement organization built for 20 or

02:03:57.600 --> 02:04:00.317
weapons programs operate on like a

02:04:00.400 --> 02:04:02.797
90-day technology cycle and this is

02:04:02.880 --> 02:04:05.357
going to be the big challenge. Um it's

02:04:05.440 --> 02:04:09.997
not that the risk it's not that they
failed to identify these future

02:04:10.080 --> 02:04:14.957
technologies that we can all see those
is how quickly can they absorb them and

02:04:15.040 --> 02:04:18.317
at the speed they're developing
and bring them to the front.

02:04:18.400 --> 02:04:20.477
>> Yeah. like you can't find exponential

02:04:20.560 --> 02:04:23.357
technology with linear procurement,

02:04:23.440 --> 02:04:26.877
>> right? Dave, your thoughts, please.

02:04:26.960 --> 02:04:30.237
>> Yeah, I I don't know. I I I had a great

02:04:30.320 --> 02:04:34.077
time with Palmer Lucky in LA and I I
really really love uh love him and I

02:04:34.160 --> 02:04:38.877
think Elon too is just an awesome
good-natured person. I'm just overjoyed

02:04:38.960 --> 02:04:43.117
that there are people in Washington that I
can sit down to with and relate to. My

02:04:43.200 --> 02:04:46.237
entire career going to Washington
has been a dread for me

02:04:46.320 --> 02:04:49.037
>> because it's just lawyers
and politicians and occasionally an

02:04:49.120 --> 02:04:53.437
accountant and there's just no
productive meeting. And for some reason,

02:04:53.520 --> 02:04:55.437
just in the last year, we're starting to

02:04:55.520 --> 02:04:59.037
see very very smart, very capable people

02:04:59.120 --> 02:05:00.957
willing to go and and get a mosquito

02:05:01.040 --> 02:05:05.597
bite, I guess, in Washington. Um, and I

02:05:05.680 --> 02:05:09.437
just makes me really optimistic that
they'll figure some things out. Um, I'm

02:05:09.520 --> 02:05:13.117
also not a fan of autonomous weapons
that make decisions in the field, but

02:05:13.200 --> 02:05:14.717
Palmer Lucky made a very good case for

02:05:14.800 --> 02:05:17.197
it. You can see it in our podcast from

02:05:17.280 --> 02:05:20.877
uh from LA. Uh, I I don't agree that

02:05:20.960 --> 02:05:24.237
it's a good choice, but he he actually
has some very rational arguments for why

02:05:24.320 --> 02:05:26.077
it's going to be that way. >> Yeah.

02:05:26.160 --> 02:05:28.557
>> Can can I just mention
one more statistic here,

02:05:28.640 --> 02:05:31.037
>> please? If you went back to the two

02:05:31.120 --> 02:05:33.917
years into the Ukraine Russia conflict,

02:05:34.000 --> 02:05:36.077
they were using about a half a million

02:05:36.160 --> 02:05:39.037
drones to fight and prosecute the war.

02:05:39.120 --> 02:05:42.077
Um, this year Russia will make 10

02:05:42.160 --> 02:05:43.917
million drones and the Ukraine will make

02:05:44.000 --> 02:05:46.397
10 million drones. So, talk about

02:05:46.480 --> 02:05:50.557
exponential. Uh, that is an unbelievable

02:05:50.640 --> 02:05:55.277
escalation, but without humans in the
loop. So, that's the good news around

02:05:55.360 --> 02:05:59.517
it. Uh uh, of course those drones are
doing a ton more damage, but they're

02:05:59.600 --> 02:06:01.677
fighting each other with drones at a

02:06:01.760 --> 02:06:04.237
scalable level. The other statistic I

02:06:04.320 --> 02:06:07.117
remember there's about 10,000 drones a

02:06:07.200 --> 02:06:11.677
month crossing the Mexico US border. Uh,

02:06:11.760 --> 02:06:16.637
and so the problem there is wall
technology is not as good as drone

02:06:16.720 --> 02:06:19.917
technology. And so trying to build
a wall along there is not the greatest

02:06:20.000 --> 02:06:20.877
idea right now.

02:06:20.960 --> 02:06:23.357
>> Nice. Alex, close us out on this one.

02:06:23.440 --> 02:06:26.957
>> Yeah, a couple points. So the secretary

02:06:27.040 --> 02:06:31.437
of war announced this alongside
several other initiatives maybe most

02:06:31.520 --> 02:06:33.037
conspicuously

02:06:33.120 --> 02:06:35.677
a project code came cenamed project

02:06:35.760 --> 02:06:38.797
azinort which is the standup of the

02:06:38.880 --> 02:06:41.357
department of war's first autonomous

02:06:41.440 --> 02:06:44.717
warfare command or auto warcom. This I

02:06:44.800 --> 02:06:47.437
think is a transformative moment for the

02:06:47.520 --> 02:06:49.357
department of war. We finally will have

02:06:49.440 --> 02:06:51.997
a dedicated joint force devoted to

02:06:52.080 --> 02:06:55.117
autonomous weapon systems. I think

02:06:55.200 --> 02:06:58.157
including drones but not exclusively

02:06:58.240 --> 02:07:00.957
drones. This is a major major step

02:07:01.040 --> 02:07:04.077
forward for US capabilities to finally

02:07:04.160 --> 02:07:08.637
have a single joint force dedicated
to this with a four-star functional

02:07:08.720 --> 02:07:12.797
combatant command that we've been missing.
We're arguably missing. You

02:07:12.880 --> 02:07:16.557
know, I would go on a soap box and say
there if if I were Secretary of War for

02:07:16.640 --> 02:07:19.517
a day, there are probably several other
functional combatant commands that I'd

02:07:19.600 --> 02:07:24.077
spin up if I had the opportunity, but
this would have been one of my top five

02:07:24.160 --> 02:07:28.157
list. The the other point that I'll make
just for project meridian which you were

02:07:28.240 --> 02:07:30.317
asking about specifically is in the

02:07:30.400 --> 02:07:33.597
announcement when sewar heath announced

02:07:33.680 --> 02:07:35.997
it there was very particular language

02:07:36.080 --> 02:07:39.197
around examining I'll quote the full

02:07:39.280 --> 02:07:41.517
spectrum of future warf fighting domains

02:07:41.600 --> 02:07:44.397
from subterranean depths to the cy lunar

02:07:44.480 --> 02:07:46.477
frontier. So I think those two

02:07:46.560 --> 02:07:49.997
particular domains, the ocean bottom and

02:07:50.080 --> 02:07:53.357
cis lunar and lunar space in general are

02:07:53.440 --> 02:07:56.237
two wildly underserved domains not just

02:07:56.320 --> 02:07:58.877
for warfare but for peaceime activities

02:07:58.960 --> 02:08:03.277
as well. We know embarrassingly
little about our ocean bottoms and

02:08:03.360 --> 02:08:07.357
optimistically one can
imagine that if if the DO

02:08:07.440 --> 02:08:10.157
decides that is now very interested in

02:08:10.240 --> 02:08:12.957
investing in American assets and

02:08:13.040 --> 02:08:15.037
American exploration and American

02:08:15.120 --> 02:08:18.557
dominance on the floor of the oceans and

02:08:18.640 --> 02:08:22.637
the lunar surface and the cis lunar
region in general. I think that is going

02:08:22.720 --> 02:08:27.197
to have dividends and pay dividends
for the entire economy and for humanity's

02:08:27.280 --> 02:08:29.597
broader technological advances in a way

02:08:29.680 --> 02:08:35.117
that might naively have nothing
to do with warfare or war fighting.

02:08:35.200 --> 02:08:38.157
>> One of the points that Palmer made
actually in LA is that under the ocean

02:08:38.240 --> 02:08:40.877
it's not practical to communicate with a

02:08:40.960 --> 02:08:45.757
central server or central command.
So that's already automated and I don't

02:08:45.840 --> 02:08:49.837
know what triggers it, but once it goes
into hunt mode, it just hunts and it's

02:08:49.920 --> 02:08:53.357
not communicating back.
>> Twothirds of the Earth's surface and we

02:08:53.440 --> 02:08:56.157
know embarrassingly little about it.

02:08:56.240 --> 02:08:57.517
>> We know more about the surface of Mars

02:08:57.600 --> 02:09:00.557
than we do our ocean floor.

02:09:00.640 --> 02:09:03.277
>> Uh Richard,
uh this is the part where we

02:09:03.360 --> 02:09:08.157
answer our viewers questions with an AMA.
And as our guest, I'm going to give

02:09:08.240 --> 02:09:12.397
you first crack to choose one of these
questions. So, if you could pick the

02:09:12.480 --> 02:09:14.637
number, read the question and who it's

02:09:14.720 --> 02:09:18.237
from, uh, and then dive in.

02:09:18.320 --> 02:09:22.397
>> All right. Well, uh, let me try to scan
them really quick. Um, all right. If AI

02:09:22.480 --> 02:09:26.157
makes companies 10x more productive, but
we don't need 10x the output, where does

02:09:26.240 --> 02:09:29.917
the value go? Shareholders, workers,
or does it evaporate? I think this is

02:09:30.000 --> 02:09:33.837
actually something I have thought about
in the past. Uh I think we can predict

02:09:33.920 --> 02:09:36.637
the impact of jobs uh in a certain

02:09:36.720 --> 02:09:39.837
industry from AI based on the elasticity

02:09:39.920 --> 02:09:42.477
of the demand when the price of that

02:09:42.560 --> 02:09:45.357
product goes massively down. We don't

02:09:45.440 --> 02:09:48.397
need to have billions and billions of

02:09:48.480 --> 02:09:49.917
illustrations in the world. And so when

02:09:50.000 --> 02:09:52.957
AI made the price of one illustration go

02:09:53.040 --> 02:09:55.837
down from 200 bucks to like 2 cents or

02:09:55.920 --> 02:09:59.357
less, like we just didn't need as many

02:09:59.440 --> 02:10:04.477
illustrators anymore. um uh uh because
the demand for illustrations didn't go

02:10:04.560 --> 02:10:08.157
massively up. Yes, every little blog
post and every little tweet can now have

02:10:08.240 --> 02:10:10.237
a beautiful visualization and and

02:10:10.320 --> 02:10:13.037
illustration. Uh but uh you know, we

02:10:13.120 --> 02:10:15.197
didn't need many more billions of them.

02:10:15.280 --> 02:10:17.277
Uh I think software is a different one.

02:10:17.360 --> 02:10:19.517
You can actually have everyone can have

02:10:19.600 --> 02:10:24.477
several pieces of software just
specific to them. So we can actually have

02:10:24.560 --> 02:10:29.277
billions of different software products
uh customized for each person. uh and so

02:10:29.360 --> 02:10:33.597
the demand uh for that product will go up.
Jevans paradox is going to be alive

02:10:33.680 --> 02:10:35.757
uh in that world and we're going to see

02:10:35.840 --> 02:10:38.077
more and more demand and there will be

02:10:38.160 --> 02:10:41.997
uh more uh value acrewing to everyone. I

02:10:42.080 --> 02:10:43.517
think in terms of shareholders versus

02:10:43.600 --> 02:10:48.397
workers I think the wave of AI in the

02:10:48.480 --> 02:10:52.477
best scenario will be a huge force
for more entrepreneurship and in the worst

02:10:52.560 --> 02:10:54.557
case scenario a force of more

02:10:54.640 --> 02:10:58.477
inequality. I think uh everyone who owns

02:10:58.560 --> 02:11:01.357
some equity in a company that uses AI

02:11:01.440 --> 02:11:02.957
can love AI.

02:11:03.040 --> 02:11:06.157
>> Wonderful. Sele over to you.

02:11:06.240 --> 02:11:10.477
>> Uh I will take uh
question number three.

02:11:10.560 --> 02:11:14.077
Um if AI can write code, research,
create marketing, manage projects and

02:11:14.160 --> 02:11:15.997
outcomes, what is left for a college

02:11:16.080 --> 02:11:19.197
graduate in 2030? And that is from Jared

02:11:19.280 --> 02:11:21.597
8812.

02:11:21.680 --> 02:11:25.117
So the obvious answer would be be have

02:11:25.200 --> 02:11:27.837
empathy and be be creative etc. But I

02:11:27.920 --> 02:11:32.157
think the bigger shift which builds on
what Richard just said is you shift from

02:11:32.240 --> 02:11:34.717
doing tasks to focusing on owning

02:11:34.800 --> 02:11:39.197
outcomes, right? So a graduate used
to be valuable because you could you could

02:11:39.280 --> 02:11:43.597
execute research or build a spreadsheet
or draft a deck. But now you want to

02:11:43.680 --> 02:11:47.997
say, "Hey, here are the constraints like
go figure out and and here's how we'll

02:11:48.080 --> 02:11:50.317
know whether we solved it and then use

02:11:50.400 --> 02:11:52.637
an AI to get to it." And it brings you

02:11:52.720 --> 02:11:55.837
back to what should your problem space

02:11:55.920 --> 02:11:58.717
be? And this is the massive opportunity

02:11:58.800 --> 02:12:00.957
because we traditionally learn judgment

02:12:01.040 --> 02:12:03.437
by doing the grunt work. Uh we need a

02:12:03.520 --> 02:12:06.637
totally new apprenticeship model uh when

02:12:06.720 --> 02:12:08.557
the grunt work disappears. And this is a

02:12:08.640 --> 02:12:11.597
huge challenge for the education system.

02:12:11.680 --> 02:12:15.837
But the change is going to be focusing
on what problems you want to solve and

02:12:15.920 --> 02:12:19.757
then letting orchestrating the forces
that will help you solve that problem.

02:12:19.840 --> 02:12:23.117
>> Yeah. Jared, find your purpose, right?
A passion is something you love doing. A

02:12:23.200 --> 02:12:25.917
purpose is something you love
doing that helps other people. uh

02:12:26.000 --> 02:12:28.717
>> make sure it's massive and it's
transformative and it's purposeful

02:12:28.800 --> 02:12:31.277
>> and and build a company and then direct

02:12:31.360 --> 02:12:34.397
AI to implement it. It is your

02:12:34.480 --> 02:12:38.157
workforce. Uh Dave, over to you.

02:12:38.240 --> 02:12:42.077
>> Uh I can't resist number four. I love
all these questions. I'm torn. But how

02:12:42.160 --> 02:12:45.517
can data centers make neighborhoods
richer instead of the owners? And that's

02:12:45.600 --> 02:12:48.877
from LM Bman66.

02:12:48.960 --> 02:12:53.357
Um, so, uh, I took a tour of the LOL
data center, Jeff Markley, Markley data

02:12:53.440 --> 02:12:56.397
center, and that thing is creating
wealth in that neighborhood like you

02:12:56.480 --> 02:13:00.157
wouldn't believe. And, uh, the way it
works fundamentally is the data center

02:13:00.240 --> 02:13:04.397
is so immensely valuable and the town
budget is maybe a couple million dollars

02:13:04.480 --> 02:13:06.957
a year. So, between the tax revenue, the

02:13:07.040 --> 02:13:11.677
donations, and the job creation,
the town is thriving and and it's a town

02:13:11.760 --> 02:13:15.837
that really needed it, too. So, I think
it's happening very naturally. What you

02:13:15.920 --> 02:13:19.277
want to do is attract a data center to
your neighborhood first and foremost and

02:13:19.360 --> 02:13:22.957
then you have the next five or seven
years to figure out your tax policy,

02:13:23.040 --> 02:13:26.317
your donation policy. I tell you
these data center operators are very

02:13:26.400 --> 02:13:30.717
interested in great PR and so they
they will donate like crazy to the high

02:13:30.800 --> 02:13:34.957
schools to the neighborhoods to it's
really really working. We're going to

02:13:35.040 --> 02:13:38.477
see a whole we're going to see an entire
shift where data centers are offering

02:13:38.560 --> 02:13:41.517
such benefits on jobs, on tax breaks, on

02:13:41.600 --> 02:13:45.837
lowerc cost energy, uh that you're going
to be begging to have a data center in

02:13:45.920 --> 02:13:48.637
your backyard. Alex,
number two is for you.

02:13:48.720 --> 02:13:52.797
>> All right, so number two asks, if
every major tech platform started open and

02:13:52.880 --> 02:13:55.597
democratic than consolidated power,

02:13:55.680 --> 02:13:58.637
Google, Meta, Amazon, why would AI be

02:13:58.720 --> 02:14:00.477
different? And this is from open-source

02:14:00.560 --> 02:14:03.517
mind. The premise of the question is

02:14:03.600 --> 02:14:06.157
half right half wrong. I I'm not sure I

02:14:06.240 --> 02:14:09.597
buy the premise that consolidated power

02:14:09.680 --> 02:14:12.477
the subtext of which and one can

02:14:12.560 --> 02:14:17.037
juxtapose it with the the user handle
open source mind. I I'm not sure the

02:14:17.120 --> 02:14:19.117
framing is the right framing. I I would

02:14:19.200 --> 02:14:22.557
agree that in every major tech

02:14:22.640 --> 02:14:25.197
revolution initially the barrier to

02:14:25.280 --> 02:14:28.077
entry is low because there's some new

02:14:28.160 --> 02:14:30.317
platform innovation and then you see

02:14:30.400 --> 02:14:32.957
lots and lots of players enter the field

02:14:33.040 --> 02:14:35.437
and then as the field matures you see

02:14:35.520 --> 02:14:39.117
economies of scale and a deeper bench of

02:14:39.200 --> 02:14:43.677
infrastructure typically supporting it
and as a result that favors larger and

02:14:43.760 --> 02:14:48.557
larger players and you do see
consolidation but the the subtext of the

02:14:48.640 --> 02:14:52.877
question that it's somehow
anti-democratic or not open. I don't

02:14:52.960 --> 02:14:55.517
agree with that premise in the least. I

02:14:55.600 --> 02:14:58.957
I do think as you see consolidation in

02:14:59.040 --> 02:15:02.637
an industry, I think it's important
to be vigilant from an antitrust

02:15:02.720 --> 02:15:04.797
perspective to make sure that it remains

02:15:04.880 --> 02:15:07.517
competitive, but the premise that

02:15:07.600 --> 02:15:12.637
hyperscaling is somehow closed
or anti-democratic, don't buy the premise

02:15:12.720 --> 02:15:14.157
at all.

02:15:14.240 --> 02:15:17.437
>> All right. Uh,
Richard as our guest, you

02:15:17.520 --> 02:15:22.760
get first crack once again. Take a look.

02:15:24.240 --> 02:15:29.117
>> I do think the internet adbased
economy uh will be under pressure.

02:15:29.200 --> 02:15:30.557
>> Which which question are you answering?

02:15:30.640 --> 02:15:33.437
>> So, first question. If AI agents
outnumber humans in one years on the

02:15:33.520 --> 02:15:37.837
internet, doesn't the entire ad- based
economy collapse? Uh, what replaces the

02:15:37.920 --> 02:15:41.597
attention economy? I think there there's
actually a really deep answer here. I'll

02:15:41.680 --> 02:15:45.357
try to summarize it, but like one,
we already have more bots on the internet

02:15:45.440 --> 02:15:49.597
uh and more agents on the internet than
people. So this I predicted this uh last

02:15:49.680 --> 02:15:51.517
year and it happened a few months ago.

02:15:51.600 --> 02:15:53.757
Um so that's number one. I do think

02:15:53.840 --> 02:15:57.837
we're seeing the first kind of um

02:15:57.920 --> 02:16:01.437
sort of skirmishes uh in that um when

02:16:01.520 --> 02:16:04.877
for instance uh various agents try to

02:16:04.960 --> 02:16:08.237
make purchases on Amazon without really

02:16:08.320 --> 02:16:12.237
being on the Amazon platform and Amazon
usually tries to turn them off cuz they

02:16:12.320 --> 02:16:14.157
want to own that relationship directly

02:16:14.240 --> 02:16:17.197
with the customer understandably. Um but

02:16:17.280 --> 02:16:21.357
it's just convenient for someone to just
say just go buy these batteries and they

02:16:21.440 --> 02:16:24.957
don't care which batteries it is. And so
if you have that control over which one

02:16:25.040 --> 02:16:27.917
it is, you can start selling that uh to

02:16:28.000 --> 02:16:31.117
uh other uh sellers and and uh people

02:16:31.200 --> 02:16:33.117
who create physical goods. And so there

02:16:33.200 --> 02:16:36.317
is uh going to be uh continued uh like

02:16:36.400 --> 02:16:38.317
ads will continue to be important. And

02:16:38.400 --> 02:16:41.917
in fact, if you think about a fully

02:16:42.000 --> 02:16:44.397
abundant society, the one thing that you

02:16:44.480 --> 02:16:47.197
cannot scale exponentially is the hours

02:16:47.280 --> 02:16:49.037
in the day that people can pay attention

02:16:49.120 --> 02:16:52.077
to you that can make you famous. Uh and

02:16:52.160 --> 02:16:55.277
so fame and brand and network effects uh

02:16:55.360 --> 02:16:59.757
and other things like that will become
more and more of a currency. And so the

02:16:59.840 --> 02:17:02.237
attention economy which is connected

02:17:02.320 --> 02:17:05.517
slightly uh not necessarily uh exactly

02:17:05.600 --> 02:17:10.077
equivalent to the ad-based economy uh
that actually will become a bigger thing

02:17:10.160 --> 02:17:12.317
as we're have more and more of our

02:17:12.400 --> 02:17:15.677
material needs met by uh technology.

02:17:15.760 --> 02:17:18.317
>> Great. Uh Salem

02:17:18.400 --> 02:17:22.877
>> I will take number uh seven. If we

02:17:22.960 --> 02:17:26.877
remove 10x the cars from the street what
happens to the insurance industry? We

02:17:26.960 --> 02:17:29.197
won't need driver's insurance. Mark and

02:17:29.280 --> 02:17:33.277
that's from Robert Zerby JB10H. Um so

02:17:33.360 --> 02:17:35.597
you know we talked about how um

02:17:35.680 --> 02:17:40.477
liability lawyers won't be needed and I
got a huge flame from a bunch of folks

02:17:40.560 --> 02:17:45.437
saying hey we really protect
the citizenry and I there are just so I'm

02:17:45.520 --> 02:17:48.477
just apologizing because there's
a spectrum of people and some people are

02:17:48.560 --> 02:17:52.717
full ambulance chasers and other people
do things. There's a whole segment of

02:17:52.800 --> 02:17:56.797
this called ethical lawsuits where
people get together and try and uh sue

02:17:56.880 --> 02:18:00.237
big companies for doing the right
ethical thing. And so that's an

02:18:00.320 --> 02:18:04.157
important uh segment of that. So I just
want to acknowledge that side of it. But

02:18:04.240 --> 02:18:06.557
just to answer the question, um you know

02:18:06.640 --> 02:18:09.117
the the thing is industries don't uh

02:18:09.200 --> 02:18:11.917
disappear when the risk changes. You

02:18:12.000 --> 02:18:14.077
change the risk, right? So driver

02:18:14.160 --> 02:18:16.637
liability may fail, software liability

02:18:16.720 --> 02:18:19.357
rises, right? You've you what's your uh

02:18:19.440 --> 02:18:21.357
cyber risk? what's your manufacturer

02:18:21.440 --> 02:18:24.237
liability risk? Um, so the in the

02:18:24.320 --> 02:18:26.877
insurance transitions from did did

02:18:26.960 --> 02:18:29.757
Seline crash to which layer of the

02:18:29.840 --> 02:18:33.277
autonomous stack failed and and we we've

02:18:33.360 --> 02:18:37.037
had this before in product liability
is different layers. So we'll end up with

02:18:37.120 --> 02:18:41.117
the same type of model. You shift
the insurance risk to a different level.

02:18:41.200 --> 02:18:44.077
>> And there's going to be all kinds of
new insurance markets for humanoid robots,

02:18:44.160 --> 02:18:46.637
for flying cars, for drones,
for all kinds of things.

02:18:46.720 --> 02:18:49.917
>> Well, just a data point on that.
You know, a single big data center like

02:18:50.000 --> 02:18:54.477
Abalene, Texas is half a trillion dollars.
All the cars combined are 4

02:18:54.560 --> 02:18:58.557
trillion. One data center is half
a trillion. And it's in a tornado alley.

02:18:58.640 --> 02:19:01.437
You think you probably want to ensure
that. So the things number of things

02:19:01.520 --> 02:19:05.997
that need insurance is going up 10x just
with the economy going up 10x. You just

02:19:06.080 --> 02:19:07.917
need to move. >> If you
really if you really want to be

02:19:08.000 --> 02:19:10.797
brave, you can't get home
insurance in Florida anymore.

02:19:10.880 --> 02:19:13.837
>> So go create an insurance
company for that.

02:19:13.920 --> 02:19:17.037
>> Nice. Uh Dave, pick your question.

02:19:17.120 --> 02:19:20.397
I like number eight. Uh,
what is the lowest possible job in an AI

02:19:20.480 --> 02:19:22.077
civilization? What a what an interesting

02:19:22.160 --> 02:19:26.557
question. I that's from LBN ODK. Um,

02:19:26.640 --> 02:19:29.117
yeah, you know, I I saw this pile of a

02:19:29.200 --> 02:19:34.397
million valves for a liquid cooled data
center and a million freaking valves.

02:19:34.480 --> 02:19:38.557
I'm like, how do those actually end up
in pipes? The humanoid robots that can

02:19:38.640 --> 02:19:40.237
install those things are pretty far out.

02:19:40.320 --> 02:19:45.117
It's a very subtle process to to install
those. So that job will be around for a

02:19:45.200 --> 02:19:49.037
long time and then they're paying a lot
for it. But that's not the lowest I'm

02:19:49.120 --> 02:19:52.877
it's it's hard to think like what is the
lowest surviving job. Do you guys have

02:19:52.960 --> 02:19:54.317
any thoughts?

02:19:54.400 --> 02:19:57.277
>> So many thoughts.
If if I may, it I want

02:19:57.360 --> 02:19:59.917
to conrue the question as in a pure AI

02:20:00.000 --> 02:20:02.397
civilization, in which case arguably the

02:20:02.480 --> 02:20:05.117
way that you measure low is the the job

02:20:05.200 --> 02:20:09.837
that requires the least compute, in which
case the the jobs that require the

02:20:09.920 --> 02:20:11.757
least compute as a results are the least

02:20:11.840 --> 02:20:14.157
economically valuable if if their inputs

02:20:14.240 --> 02:20:16.877
are the lowest might ironically look

02:20:16.960 --> 02:20:19.677
like the most valuable jobs in a pre-I

02:20:19.760 --> 02:20:22.317
civilization. More of a paradox style.

02:20:22.400 --> 02:20:25.357
So the the the great writers, business

02:20:25.440 --> 02:20:28.157
people, uh the the the creative actors

02:20:28.240 --> 02:20:31.517
that Einrand may fetishize would

02:20:31.600 --> 02:20:34.317
ironically in a post AAI civilization be

02:20:34.400 --> 02:20:38.637
the lowest possible jobs because Moravec
paradox paradox style those were the

02:20:38.720 --> 02:20:42.237
first ones to be automated.
>> Interesting. All right.

02:20:42.320 --> 02:20:43.837
>> I think there's two similarities here.

02:20:43.920 --> 02:20:46.237
One is very high judgment per what you

02:20:46.320 --> 02:20:47.997
were just saying, Alex, and second is

02:20:48.080 --> 02:20:51.197
very physical, highly contextual work.

02:20:51.280 --> 02:20:53.357
I would I would argue maybe also low is

02:20:53.440 --> 02:20:56.077
just in terms of how valuable uh and and

02:20:56.160 --> 02:20:58.957
and moral society deems those kinds of

02:20:59.040 --> 02:21:01.997
jobs. And I do think actually that uh

02:21:02.080 --> 02:21:06.477
the other question about the cars on
the street um I don't think we removed uh

02:21:06.560 --> 02:21:10.877
10x the cars when we have self-driving
but self-driving might be making things

02:21:10.960 --> 02:21:13.357
so much safer that indeed uh people

02:21:13.440 --> 02:21:18.157
don't need as much accident insurance
and no not as many ER people not as many

02:21:18.240 --> 02:21:21.997
ambulances and so in a weird way you
make objectively the world better by

02:21:22.080 --> 02:21:26.157
reducing traffic deaths but it does
actually have potentially a mildly

02:21:26.240 --> 02:21:30.797
negative effect on parts of the economy
and we should all be rooting for that uh

02:21:30.880 --> 02:21:35.357
in this case, right? And so I think
the lowest jobs in terms of moral standing

02:21:35.440 --> 02:21:39.517
are things where you don't really
progress uh humanity forward. And my

02:21:39.600 --> 02:21:41.997
hunch is there the types of jobs in

02:21:42.080 --> 02:21:44.237
entertainment um that that people will

02:21:44.320 --> 02:21:47.437
value a lot and and fame and attention

02:21:47.520 --> 02:21:51.277
uh will become more of a currency
in that world. And so if your job doesn't

02:21:51.360 --> 02:21:56.077
get you any of that, it might be
considered uh lower in in that future. I

02:21:56.160 --> 02:21:59.277
think the lowest job that just will
never go away will be something in

02:21:59.360 --> 02:22:03.837
politics where it's completely
irrelevant already, but but it's just

02:22:03.920 --> 02:22:06.477
there and it'll stay there
and no one's going to change it.

02:22:06.560 --> 02:22:10.157
>> Nice. All right. >> I got
to throw in ID's perspective. He

02:22:10.240 --> 02:22:13.757
always thinks it's the BART train
driver because it's unionized to hell.

02:22:13.840 --> 02:22:16.717
>> Yeah. Yeah. There you go.
>> Alex, number six, close us out.

02:22:16.800 --> 02:22:20.957
>> Number six. If we cure illnesses
that are often due to bad behavior, what's

02:22:21.040 --> 02:22:25.117
going to take care of the cause? Joe
Wilder. I assume the cause is reference

02:22:25.200 --> 02:22:27.757
to people choosing to behave quote

02:22:27.840 --> 02:22:32.317
unquote badly. And the answer I this
is another case of my not buying the

02:22:32.400 --> 02:22:35.677
premise. If you look at some of the
really spectacular results that have

02:22:35.760 --> 02:22:38.237
been coming out of GLP1 class studies,

02:22:38.320 --> 02:22:40.797
including third and soon presumably

02:22:40.880 --> 02:22:43.997
fourth generation GLP1 RAS, they're

02:22:44.080 --> 02:22:48.397
actually addressing the cause. like
addictive behaviors are being mitigated

02:22:48.480 --> 02:22:50.957
or at least partially treated by the

02:22:51.040 --> 02:22:53.197
same drugs that are curing in or at

02:22:53.280 --> 02:22:55.997
least treating have to to caveat that

02:22:56.080 --> 02:23:00.317
inflammation and blood sugar, diabetes,

02:23:00.400 --> 02:23:02.477
all of these other conditions the the

02:23:02.560 --> 02:23:04.717
root causes which is or human behavior

02:23:04.800 --> 02:23:07.597
are themselves the the human behavior is

02:23:07.680 --> 02:23:10.557
being affected by the same drugs. So I'm

02:23:10.640 --> 02:23:14.717
so optimistic about what Alex is saying
because I really fully believe AI this

02:23:14.800 --> 02:23:18.557
is going to be one of the highest
callings of AI very very soon is to make

02:23:18.640 --> 02:23:22.157
you feel really good about doing good
things and happy as you're doing it

02:23:22.240 --> 02:23:24.157
>> love >> and not want
not want to do bad things

02:23:24.240 --> 02:23:27.517
for yourself. It turns out we have the
capability now. We figured out how to do

02:23:27.600 --> 02:23:30.717
that at least in part and we're
going to figure out a lot more.

02:23:30.800 --> 02:23:33.037
>> All right, as always a call out to our

02:23:33.120 --> 02:23:35.757
amazing community. If you've got a music

02:23:35.840 --> 02:23:40.077
video that you'd like to show as outro,
please send it to the team at

02:23:40.160 --> 02:23:42.317
mediadmandis.com.

02:23:42.400 --> 02:23:45.117
And speaking about amazing outros, here

02:23:45.200 --> 02:23:48.797
is Abundance by Steven Gross. Uh my dear

02:23:48.880 --> 02:23:55.160
Moonshot mates, this is the real you.
So check it out.

02:23:55.920 --> 02:24:01.720
>> Got it. Peter a wgave.

02:24:03.520 --> 02:24:08.520
This one's for you. >> Bun dance.

02:24:09.920 --> 02:24:13.880
Everybody do the bun dance.

02:24:16.800 --> 02:24:20.157
Everybody do the bun dance. Thinking

02:24:20.240 --> 02:24:24.237
moonshot dreaming. A moving future

02:24:24.320 --> 02:24:27.837
screaming. Build it faster. Think much

02:24:27.920 --> 02:24:30.717
bigger. Exponential.

02:24:30.800 --> 02:24:33.117
Pull the trigger.

02:24:33.200 --> 02:24:37.757
Drink, Alec. Drink >> drink water

02:24:37.840 --> 02:24:41.277
martini glass. >> Oh,
that was fun. That was Moonshots

02:24:41.360 --> 02:24:45.757
Live 2040 on the moon. >> Nice. Uh,
so >> I don't think we should have to wait

02:24:45.840 --> 02:24:49.437
that long. >> Yeah. Uh,
Richard, your new book, The

02:24:49.520 --> 02:24:53.277
Eureka Machine, wherever you purchase
your books. Do you have the Audible out?

02:24:53.360 --> 02:24:55.757
>> Uh, should come out very soon. Yeah.

02:24:55.840 --> 02:24:59.117
>> All right. I'm an Audible reader,
but I have skimmed the book here. Uh,

02:24:59.200 --> 02:25:02.957
congratulations. Say, Richard, there's
I got to say something. What you're doing

02:25:03.040 --> 02:25:05.037
with the recursive looks like such an

02:25:05.120 --> 02:25:07.277
incredible opportunity to move humanity

02:25:07.360 --> 02:25:10.477
forward. So, congrats on taking

02:25:10.560 --> 02:25:14.797
it a good one for humanity.
>> And and Richard, please resist the urge

02:25:14.880 --> 02:25:18.957
to have you.com acquire your own
Frontier Lab like everyone else seems to

02:25:19.040 --> 02:25:23.277
be using, spinning off their own
Frontier Labs, not to name names, uh, as

02:25:23.360 --> 02:25:27.117
a a financial engineering exercise to
maximize their equity and their original

02:25:27.200 --> 02:25:30.157
startup. Please resist the urge
to to follow that trend.

02:25:30.240 --> 02:25:31.597
>> All right.

02:25:31.680 --> 02:25:33.837
>> All right. I love you guys.

02:25:33.920 --> 02:25:38.120
>> Be well. Until next time.