WEBVTT

00:00:00.000 --> 00:00:02.877
What's it's going to mean to to be
[music] human like 50 years from now?

00:00:02.960 --> 00:00:05.957
>> I think the world is finally taking
seriously, and we've been waiting for

00:00:06.040 --> 00:00:09.717
this for years, that social contract is
going to have to evolve and maybe sooner

00:00:09.800 --> 00:00:12.797
than people thought. >> What are
your predictions for when this

00:00:12.880 --> 00:00:16.397
breaks [music] 85%? >> Yeah,
I think it really depends how much

00:00:16.480 --> 00:00:20.557
effort the frontier labs are putting
into targeting Arc AGI 3 explicitly.

00:00:20.640 --> 00:00:25.117
With the new set of models that I think
will come out, the impact that [music]

00:00:25.200 --> 00:00:28.397
the technology will have on accelerating
science and accelerating the economy. We

00:00:28.480 --> 00:00:31.477
may be wrong, we may be over optimistic,
but it feels like it's about to be

00:00:31.560 --> 00:00:34.077
absolutely massive. There are a lot
of interesting projects, [music] and yet

00:00:34.160 --> 00:00:37.597
there's this one new thing
that's really working.

00:00:37.680 --> 00:00:41.517
Hi guys, I'm D, a partner at Menlo
Ventures. I'm one of the early investors

00:00:41.600 --> 00:00:45.797
in India. And we are
investors in Anthropic.

00:00:45.880 --> 00:00:49.197
So not to say that will

00:00:49.280 --> 00:00:52.277
color the discussion at all.

00:00:52.360 --> 00:00:57.917
Um but yeah, let's get started. Um
The first thing I wanted to ask you guys

00:00:58.000 --> 00:01:02.557
is, you know both of you are parents,

00:01:02.640 --> 00:01:06.397
not just founders and CEOs of companies.

00:01:06.480 --> 00:01:09.677
And today when I go
and talk to a lot of parents

00:01:09.760 --> 00:01:13.437
they're quite concerned about
their kids and the future.

00:01:13.520 --> 00:01:17.357
As as fathers what do you
think about the most when

00:01:17.440 --> 00:01:21.437
raising your kids today? Is it any
different from before? What really

00:01:21.520 --> 00:01:25.277
matters to you when it
comes to raising your kids?

00:01:25.360 --> 00:01:27.477
Maybe we'll start with Francois.

00:01:27.560 --> 00:01:30.397
Yeah, and it's in my mind a lot. It's in
my mind a lot. It's like clearly they're

00:01:30.480 --> 00:01:33.717
going to grow up in a world that's very
different from the world in which I grew

00:01:33.800 --> 00:01:37.397
up in. It's expensive child. And yeah,
I think about it like the obviously their

00:01:37.480 --> 00:01:40.557
their relationship to technology
is going to be completely different. You

00:01:40.640 --> 00:01:43.637
know, they're they're they're going
to grow up in this Star Trek world where

00:01:43.720 --> 00:01:46.357
they can just, you know, talk to a
computer and and have the computer do

00:01:46.440 --> 00:01:50.797
things. Uh while back in my day,
you know, we had we had to write code like

00:01:50.880 --> 00:01:55.157
character by character. Um yeah,
and so actually I'm a I'm a bit

00:01:55.240 --> 00:01:58.637
concerned as well because, you know, I'm
I'm wondering what's going to happen to

00:01:58.720 --> 00:02:03.077
human human agency. And and and our
ability to you know think independently.

00:02:03.160 --> 00:02:07.877
Like are we are we just going to outsource
our minds? Like, you know, our

00:02:07.960 --> 00:02:11.877
our thoughts, our cognition, and all our
abilities to the machine? What what are

00:02:11.960 --> 00:02:14.357
we going to retain? You know, what
what's what's it going to mean to to be

00:02:14.440 --> 00:02:18.997
human in like not in 10 years from now,
but like 50 years from now? Like my son,

00:02:19.080 --> 00:02:22.717
my daughter, they're they're they're
going to live to see the the 22nd

00:02:22.800 --> 00:02:27.037
century, right? That's that's
that's something I think about a lot.

00:02:27.120 --> 00:02:30.597
What about you, Sam? You know, actually

00:02:30.680 --> 00:02:34.717
I think it'll be great. I I understand
that throughout history parents have

00:02:34.800 --> 00:02:38.077
always freaked out about the rate of
change, wondered how life is going to be

00:02:38.160 --> 00:02:40.757
for their kids. I think it
is awesome that our kids are

00:02:40.840 --> 00:02:43.557
going to grow up in this Star Trek
world where you just ask a computer for

00:02:43.640 --> 00:02:48.237
anything. Uh and it does it,
and I think they will

00:02:48.320 --> 00:02:52.397
you know, look at us like I kind
of looked at my parents with this

00:02:52.480 --> 00:02:55.517
like, "Oh man, you really had
to do things before computers?"

00:02:55.600 --> 00:02:57.997
Uh and my parents worried that I wasn't
going to learn anything either cuz I was

00:02:58.080 --> 00:03:01.477
just going to look everything up
on Google. It turns out I was able to have

00:03:01.560 --> 00:03:04.397
a fulfilling career of a kind they hadn't

00:03:04.480 --> 00:03:07.957
they didn't have a model for. And

00:03:08.040 --> 00:03:11.437
I assume that our kids will
have the same experience

00:03:11.520 --> 00:03:14.557
and they will do things um

00:03:14.640 --> 00:03:16.557
that absolutely astonish us.

00:03:16.640 --> 00:03:19.757
That they will look back on us with look

00:03:19.840 --> 00:03:24.997
back on our lives with some like pity
is maybe too strong of a word, but

00:03:25.080 --> 00:03:26.757
man, you all had it hard and bad and

00:03:26.840 --> 00:03:29.117
miserable. And this the quality of life

00:03:29.200 --> 00:03:34.037
will be so incredible.
Um I I know actually I think all those

00:03:34.120 --> 00:03:39.317
things. I think it's
amazing that my kids will

00:03:39.400 --> 00:03:44.757
never be smarter than a computer, and they
will never expect that a computer can't

00:03:44.840 --> 00:03:48.317
do these incredible things, and they
will they will achieve and they will

00:03:48.400 --> 00:03:52.917
grow up expecting to achieve these amazing

00:03:53.000 --> 00:03:57.037
feats, and I think they'll have more
agency and more will than any of us do,

00:03:57.120 --> 00:04:00.957
and they'll like they'll grow up
expecting that. It won't be weird. Um

00:04:01.040 --> 00:04:03.317
and they'll develop with that way
in the same way that we developed with the

00:04:03.400 --> 00:04:07.557
computers and thought of all the things we
could do that our parents didn't think of.

00:04:07.640 --> 00:04:12.717
Emotionally I'm like oof.

00:04:12.800 --> 00:04:16.077
You know what's this
really going to be like?

00:04:16.160 --> 00:04:19.317
Do you think just a quick follow-up
on that? Do you think in that world, you

00:04:19.400 --> 00:04:22.277
know, when we grew up, intelligence
probably mattered a lot. You know, our

00:04:22.360 --> 00:04:24.317
parents probably were like, "Be smart."

00:04:24.400 --> 00:04:27.157
You know, you guys went to some pretty
good schools, you learned a lot. Do you

00:04:27.240 --> 00:04:31.317
think that still continues to matter
in this world or not so much or maybe a

00:04:31.400 --> 00:04:34.837
different kind of intelligence?
What are your thoughts?

00:04:34.920 --> 00:04:39.557
So intelligence is really the ability
to adapt, and the the more rapidly the

00:04:39.640 --> 00:04:42.597
world changes, the more important
it is to be able to adapt to it. So

00:04:42.680 --> 00:04:47.157
personally, I trust my kids that, you
know they will see a level of change uh

00:04:47.240 --> 00:04:50.997
technological change and social change
that I haven't seen in my life, but I I

00:04:51.080 --> 00:04:53.477
trust them to adapt like kids kids are

00:04:53.560 --> 00:04:56.157
the most adaptable ones in the world.

00:04:56.240 --> 00:04:59.437
I'd love to like get a peek into your

00:04:59.520 --> 00:05:03.517
into your lives. Like what's something
that's happened in the last week or two?

00:05:03.600 --> 00:05:05.717
Like a small anecdote you could share

00:05:05.800 --> 00:05:11.560
that just tell us about tell us
about what goes on behind the scenes.

00:05:13.000 --> 00:05:18.157
Sure. I mean I can yeah. [laughter]
So what one thing I've I've been doing

00:05:18.240 --> 00:05:22.357
for the last few weeks with my son.
So my son is a huge Minecraft fan. Like

00:05:22.440 --> 00:05:26.077
he's he's he's turning five soon, and so
that's it's playing so much Minecraft

00:05:26.160 --> 00:05:29.837
and watching Minecraft videos on YouTube
and so on. And what I've been doing is

00:05:29.920 --> 00:05:33.637
that I've been live coding effectively

00:05:33.720 --> 00:05:37.597
sort of like simple pretty
future complete Minecraft clone.

00:05:37.680 --> 00:05:41.717
And in fact using Codex,
so thank you Sam for that. Um

00:05:41.800 --> 00:05:44.997
and it's been it's been so fun.
And basically so my son is telling me, you

00:05:45.080 --> 00:05:47.517
know, what what what he wants to see
in the game, what features he wants, and

00:05:47.600 --> 00:05:51.357
he's he's pretty creative about it.
And then I tell Codex to do it. And I

00:05:51.440 --> 00:05:54.317
haven't I haven't written a single line
of code. So it's it's it's it's been a

00:05:54.400 --> 00:05:57.877
very fun experience. What about you, Sam?

00:05:57.960 --> 00:06:00.440
Um

00:06:01.160 --> 00:06:04.357
I I think one interesting

00:06:04.440 --> 00:06:08.317
like I'm very grateful to get to work
on so many interesting things and

00:06:08.400 --> 00:06:11.237
built my kids can't build the Codex yet,
but we can, you know, put blocks

00:06:11.320 --> 00:06:13.477
together and stuff. Um that's the most

00:06:13.560 --> 00:06:18.317
fun, but the most kind
of intellectually stimulating thing is

00:06:18.400 --> 00:06:21.477
I think the world
is finally taking seriously

00:06:21.560 --> 00:06:24.637
and we've been waiting for this for years,
that the social contract is going

00:06:24.720 --> 00:06:26.437
to have to evolve and maybe sooner than

00:06:26.520 --> 00:06:29.197
people thought. And it finally feels to

00:06:29.280 --> 00:06:32.117
me like there is a moment where in many

00:06:32.200 --> 00:06:36.557
different countries, many different
political parties, many people who

00:06:36.640 --> 00:06:41.077
normally would think
very differently about how

00:06:41.160 --> 00:06:45.117
the economy should work and how
power structures should be developed

00:06:45.200 --> 00:06:49.197
there is a real finally,
I think it's real, desire to

00:06:49.280 --> 00:06:52.877
sit down and say, "Let's start talking
about ideas. Let's not just keep saying,

00:06:52.960 --> 00:06:55.077
'Man, stuff is really going to change
and this is really going to be a big

00:06:55.160 --> 00:06:57.117
effect on the economy.'" But

00:06:57.200 --> 00:06:59.877
the Overton window the

00:06:59.960 --> 00:07:04.157
interest sort of willingness
to get to work on

00:07:04.240 --> 00:07:08.117
this with the people that can actually
make it happen. This last week, few

00:07:08.200 --> 00:07:10.757
weeks, whatever, first time I believe
like, "All right, maybe the world is

00:07:10.840 --> 00:07:14.917
really going to do this." Um and that's
been an exciting development. That's

00:07:15.000 --> 00:07:18.717
awesome. That's awesome. Let's let's
move into some AI questions. This is a

00:07:18.800 --> 00:07:22.117
question I just have to ask. I know
Sam probably just saw this benchmark.

00:07:22.200 --> 00:07:25.197
Francois, you probably have a good
idea of this or at least a prediction.

00:07:25.280 --> 00:07:26.797
You know, Arc AGI, Greg covered this,

00:07:26.880 --> 00:07:30.277
right? Remember 2019, 5 years later, O3

00:07:30.360 --> 00:07:33.997
preview crushes it, 88%. Arc AGI 2

00:07:34.080 --> 00:07:37.437
launches about a year ago.
Um right now I think the top score is

00:07:37.520 --> 00:07:39.997
Gemini Deep Thinking 84.6.

00:07:40.080 --> 00:07:43.717
Today Arc AGI 3 is 0.37.

00:07:43.800 --> 00:07:45.917
What are your predictions for when this

00:07:46.000 --> 00:07:50.917
breaks 85%? Yeah, I think
it really depends how much

00:07:51.000 --> 00:07:56.037
effort the frontier labs are putting
into targeting Arc AGI 3 explicitly. Um

00:07:56.120 --> 00:07:59.357
like for Arc 2 for instance, a lot
of the very fast progress we saw in the in

00:07:59.440 --> 00:08:02.717
the late 2025 was really because, you
know, there was just so much effort, so

00:08:02.800 --> 00:08:06.837
much compute that was
thrown at the benchmarks.

00:08:06.920 --> 00:08:11.477
Um and I think if if the the frontier
labs decide that they're just not going to

00:08:11.560 --> 00:08:14.357
look at Arc 3 whatsoever, they're not
going to they're not going to target it,

00:08:14.440 --> 00:08:16.277
they're completely they're going
to completely ignore it, they're not going

00:08:16.360 --> 00:08:18.917
to let the scores increase organically

00:08:19.000 --> 00:08:21.517
just as a function of the G of the model

00:08:21.600 --> 00:08:23.797
increasing, I think in this in this

00:08:23.880 --> 00:08:28.677
scenario it would take several years
to saturate Arc 3. Uh but what's actually

00:08:28.760 --> 00:08:31.877
going to happen in practice is they
they're they're going to throw a ton of

00:08:31.960 --> 00:08:35.277
compute at it. Um and uh I would say at

00:08:35.360 --> 00:08:38.317
least a year until saturation because

00:08:38.400 --> 00:08:42.717
uh it's it's significantly harder than
than Arc 2. Uh we were deliberately

00:08:42.800 --> 00:08:47.237
trying to make it, you know, like out
of distribution. Like the concepts, the

00:08:47.320 --> 00:08:51.357
dynamics you will see in the in the test
environments are very different than the

00:08:51.440 --> 00:08:54.317
ones that you see in the in the public
in the public games. Uh it's going to be

00:08:54.400 --> 00:08:56.597
much much harder to put forth, for sure.

00:08:56.680 --> 00:08:59.197
So at least a year. Thankfully, we have

00:08:59.280 --> 00:09:02.757
a big lab CEO right here. So I guess

00:09:02.840 --> 00:09:07.157
this is a kind of freebie question,
but does Arc AGI 3 matter to you, Sam? And

00:09:07.240 --> 00:09:07.797
uh

00:09:07.880 --> 00:09:11.757
how do you think about Arc AGI 3?

00:09:11.840 --> 00:09:15.037
Um yeah, of course. I I think any

00:09:15.120 --> 00:09:18.077
benchmarks that are interesting and real

00:09:18.160 --> 00:09:22.717
and not currently
saturated matter to us. Um

00:09:22.800 --> 00:09:25.357
there aren't many left.
I I mean there are metrics about

00:09:25.440 --> 00:09:28.037
discovering new science.
There are benchmarks about discovering new

00:09:28.120 --> 00:09:31.077
science. There are benchmarks about
economic acceleration. Those I think

00:09:31.160 --> 00:09:35.557
really matter. But there are not
a lot of benchmarks in this sense.

00:09:35.640 --> 00:09:39.517
Uh so of course it matters. Uh I thought
that was a very good and very nuanced

00:09:39.600 --> 00:09:43.637
answer, which is how long it takes will
depend how hard how much people target

00:09:43.720 --> 00:09:46.997
it. Um I hope people don't target
it that much, but I'm sure, you know,

00:09:47.080 --> 00:09:49.117
people like scoreboards and they will.

00:09:49.200 --> 00:09:53.917
Um is

00:09:54.000 --> 00:09:56.237
AGI wasn't supposed to be numbered.

00:09:56.320 --> 00:10:00.157
You know, in the way that people first
talked about AGI, it was you have AGI or

00:10:00.240 --> 00:10:01.477
not. You're not supposed to have an AGI

00:10:01.560 --> 00:10:05.077
one, AGI two, AGI three. And

00:10:05.160 --> 00:10:07.317
and yet

00:10:07.400 --> 00:10:09.997
we learn how hard it is to precisely

00:10:10.080 --> 00:10:14.357
define what AGI is and how we're going
to test for it. And I bet there'll be a

00:10:14.440 --> 00:10:16.837
four and a five and then a ASI one and

00:10:16.920 --> 00:10:23.600
ASI two and then it'll keep going.
We're working on it. There we go. Um

00:10:23.720 --> 00:10:27.397
but I think the main
thing that points to is not

00:10:27.480 --> 00:10:31.877
that this team doesn't do incredible
work, which obviously they do, but

00:10:31.960 --> 00:10:37.357
the goal posts move. AGI is hard to define
and great benchmarks are hard to make.

00:10:37.440 --> 00:10:38.757
Another benchmark that's talked about a

00:10:38.840 --> 00:10:41.677
lot that sort of has no ceiling is this

00:10:41.760 --> 00:10:44.637
uh METER benchmark, which is how long of

00:10:44.720 --> 00:10:47.917
a task can an AI agent autonomously do

00:10:48.000 --> 00:10:50.677
with some success rate. So, right now

00:10:50.760 --> 00:10:55.637
um there's 12 hours with a above 50%
success rate. There's a model called

00:10:55.720 --> 00:10:58.517
Opus that does that. What do you What
do you think of that benchmark? Like, is

00:10:58.600 --> 00:11:03.717
that not valid? How's that different
from ARC-AGI? Francois, any thoughts? Uh

00:11:03.800 --> 00:11:06.197
I like METER.
I think it's good signal in it.

00:11:06.280 --> 00:11:10.757
Um it's also a bit of a noisy signal.
Like, I wouldn't take the claim that uh

00:11:10.840 --> 00:11:12.797
current AI can do like 12 hours of

00:11:12.880 --> 00:11:16.157
autonomous work. Um and I mean, as a as

00:11:16.240 --> 00:11:20.437
a counter claim, I mean, look look at uh
any Oxbury environment. It doesn't take

00:11:20.520 --> 00:11:24.237
12 hours to to solve one of them.
It takes uh about 5 minutes like 5 to 10

00:11:24.320 --> 00:11:28.157
minutes for a human. And uh current
functional models cannot do that. So,

00:11:28.240 --> 00:11:31.717
we're not I don't think we're at 12
hours over arbitrary tasks. So, if you

00:11:31.800 --> 00:11:35.277
look at METER, I think uh some of the
issues is if you're looking at longer

00:11:35.360 --> 00:11:38.637
time horizons, and it's definitely
the regime that we're in now,

00:11:38.720 --> 00:11:42.317
uh it's less reliable because they don't
have a lot of tasks for for for long

00:11:42.400 --> 00:11:46.757
time horizons. So, it's small sample size.
Uh they don't have very good human

00:11:46.840 --> 00:11:50.637
baselines as well. Um so,
yeah, let's let's not let's not

00:11:50.720 --> 00:11:54.397
just take it at face value. A big
difference between uh their approach to

00:11:54.480 --> 00:11:58.157
benchmarking AI and our approach
is that we are really targeting general

00:11:58.240 --> 00:12:01.997
intelligence. And I feel like METER
is more of a domain-specific or even

00:12:02.080 --> 00:12:05.277
task-specific benchmark. Like,
for instance, uh they're benchmarking the

00:12:05.360 --> 00:12:08.637
hardness. I mean, if you start learning
hardness, it means that you're expecting

00:12:08.720 --> 00:12:12.317
a certain kind of task. Uh your system
is already task-specific like it's

00:12:12.400 --> 00:12:16.677
domain-specific. And they're heavily
indexing on uh software engineering. And

00:12:16.760 --> 00:12:18.757
meanwhile, we're really trying to uh

00:12:18.840 --> 00:12:20.677
benchmark the the the G like the the

00:12:20.760 --> 00:12:24.317
team in in AGI. Um so, in principle, you

00:12:24.400 --> 00:12:28.437
know, if you have AGI, uh it should
be able to look at any benchmark and it

00:12:28.520 --> 00:12:32.437
should not need any harness. Should not
need, you know, repurposing uh with a

00:12:32.520 --> 00:12:33.997
human in the loop for the new task.

00:12:34.080 --> 00:12:37.637
Should just, you know,
go and do it the way humans do.

00:12:37.720 --> 00:12:42.637
I I think it's a interesting thing
to measure for sure, but I think it's

00:12:42.720 --> 00:12:44.557
inadequate in a few different ways that

00:12:44.640 --> 00:12:48.357
are very important to understand. One is

00:12:48.440 --> 00:12:51.157
that as these tasks get to five-year or

00:12:51.240 --> 00:12:55.517
50-year tasks, it's like very hard
to evaluate and we don't want to wait five

00:12:55.600 --> 00:12:57.677
years and

00:12:57.760 --> 00:12:59.317
you know, that's going to be

00:12:59.400 --> 00:13:03.357
weird. And then the other thing is

00:13:03.440 --> 00:13:08.397
there are things that I could work
on for years and not solve and that a

00:13:08.480 --> 00:13:12.637
genius mathematician could work
on for a few minutes and solve easily.

00:13:12.720 --> 00:13:17.517
There are things that no one on Earth can
do or haven't been able to do yet without

00:13:17.600 --> 00:13:20.197
the help of tools, no matter how much
time they have, that we'd really like to

00:13:20.280 --> 00:13:23.837
get done. There's something that

00:13:23.920 --> 00:13:28.477
matters about um like,
you know, can we discover new

00:13:28.560 --> 00:13:31.477
science that humans have not been able
to discover and whether that takes a

00:13:31.560 --> 00:13:34.477
huge amount of compute and a small
amount of time or a small amount of

00:13:34.560 --> 00:13:36.077
compute and a lot of time,

00:13:36.160 --> 00:13:38.757
like we want the new knowledge. And very

00:13:38.840 --> 00:13:41.957
soon, I think we'll be talking about

00:13:42.040 --> 00:13:46.717
things like that. Can we Can we do
things of real value to people? New

00:13:46.800 --> 00:13:51.677
new scientific knowledge,
new economic acceleration. And

00:13:51.760 --> 00:13:57.037
you know, time on task is an interesting
but very perfect proxy for that.

00:13:57.120 --> 00:14:00.677
Thanks. Um guys, it would be remiss if
I didn't ask you about, you know, both of

00:14:00.760 --> 00:14:03.077
you are a part of companies that have

00:14:03.160 --> 00:14:08.797
sort of different approaches
to achieving this goal. So, namely in up

00:14:08.880 --> 00:14:11.597
the OpenAI world, at least what
is publicly known is, you know, you

00:14:11.680 --> 00:14:15.997
pre-train a big model, you do some
post-train based on RL data, and you get

00:14:16.080 --> 00:14:20.237
the model to be really good. And it's
we had reasoning happen along the way. It

00:14:20.320 --> 00:14:24.117
sounds like we got really, really good
just doing that. And that paradigm has

00:14:24.200 --> 00:14:27.997
has worked so far. Obviously, and uh
you have a slightly different approach,

00:14:28.080 --> 00:14:33.037
Francois. Um you want to do deep
learning based on program synthesis. So,

00:14:33.120 --> 00:14:35.797
maybe the way I'd want to frame this
question is, Francois, can you talk a

00:14:35.880 --> 00:14:39.837
little bit more about your approach?
I'm sure more of the audience knows less

00:14:39.920 --> 00:14:43.877
about the approach and uh I'd like
to compare and contrast that to sort of

00:14:43.960 --> 00:14:46.677
Sam, what you're seeing at OpenAI.

00:14:46.760 --> 00:14:50.237
Sure. So, you know, when I mentioned
program synthesis, lots of people ask

00:14:50.320 --> 00:14:53.477
me, "Is are you doing like coding agents?
Are you doing code gen?" And

00:14:53.560 --> 00:14:57.397
that's not at all what we're doing. Uh
we're not like adding a new layer on top

00:14:57.480 --> 00:15:00.557
of the the other end stack.
We're actually trying to rethink the

00:15:00.640 --> 00:15:04.597
foundations. So, we call
our approach symbolic learning.

00:15:04.680 --> 00:15:09.357
And it's basically uh a new branch
of machine learning. Like, if you go back

00:15:09.440 --> 00:15:12.917
to uh the the machine
learning problem in its

00:15:13.000 --> 00:15:16.597
simplest form, you have some input data,
you have some target data, you're

00:15:16.680 --> 00:15:20.197
looking for a function that will map the
inputs to targets and hopefully it will

00:15:20.280 --> 00:15:22.997
generalize to uh new inputs as well.

00:15:23.080 --> 00:15:26.277
Um and if you're doing parametric
learning, uh which includes deep

00:15:26.360 --> 00:15:30.637
learning, then your function is a
parametric curve and you're trying to

00:15:30.720 --> 00:15:33.957
find the right parameters
via uh gradient descent.

00:15:34.040 --> 00:15:38.117
And uh all of this all this picture
is basically what we're doing. What we're

00:15:38.200 --> 00:15:40.957
replacing the curve with a symbolic

00:15:41.040 --> 00:15:45.277
program that is minimal. It's like
the shortest program that can accurately

00:15:45.360 --> 00:15:50.037
explain the data. And when you're
doing this, you're much closer to uh

00:15:50.120 --> 00:15:54.477
producing a form of optimal machine
learning like optimal learning. Uh

00:15:54.560 --> 00:15:58.477
and uh there are a few consequences when
you when you get closer to optimality.

00:15:58.560 --> 00:16:03.277
Uh if your model is genuinely uh
the simplest possible to to to explain the

00:16:03.360 --> 00:16:06.117
data, then it's also going to be
the model that will be capable of

00:16:06.200 --> 00:16:10.277
generalizing the most strongly. Uh it's
going to be the most compute-efficient

00:16:10.360 --> 00:16:15.117
at this time and it's going to need uh
the least uh data points uh to find that

00:16:15.200 --> 00:16:17.757
model because, you know, if if the model
is too biased, you don't need you don't

00:16:17.840 --> 00:16:20.957
need uh trillions of tokens to find it.

00:16:21.040 --> 00:16:24.637
Um so, that's basically the story. We're
trying to rebuild machine learning from

00:16:24.720 --> 00:16:29.277
the ground up on top of uh a symbolic
substrate that would bring us much, much

00:16:29.360 --> 00:16:32.837
closer uh to optimal AI.
And eventually, you

00:16:32.920 --> 00:16:35.517
know, we're going to rebuild the entire
stack. Like, I expect in the future

00:16:35.600 --> 00:16:39.437
we'll have uh very
large-scale foundation models

00:16:39.520 --> 00:16:41.517
that will be symbolic in nature. We want

00:16:41.600 --> 00:16:46.477
our approach to be as scalable
as compute-efficient as deep learning.

00:16:46.560 --> 00:16:48.717
Sam, what do you think about
that approach? And, you know, OpenAI

00:16:48.800 --> 00:16:53.277
obviously takes a very different approach.
Is that going to continue to be the case?

00:16:53.360 --> 00:16:54.917
How do you think of different ways to

00:16:55.000 --> 00:16:57.677
get to this amorphous somewhat amorphous

00:16:57.760 --> 00:17:00.717
concept of of AGI? What do you think?

00:17:00.800 --> 00:17:04.477
I I think that is a great thing to try.

00:17:04.560 --> 00:17:08.077
I think that people
should try many approaches.

00:17:08.160 --> 00:17:11.397
We we try to bet with conviction on ours,

00:17:11.480 --> 00:17:16.677
but we could very easily hit some sort
of unexpected roadblock and I think

00:17:16.760 --> 00:17:19.997
you know, when we started with ours,
it was very unpopular and turned out to

00:17:20.080 --> 00:17:24.797
have some value and uh I'm sure other
people will start out with things that

00:17:24.880 --> 00:17:29.397
Well, this one isn't even very unpopular.
Uh this one is a great thing to try. Uh

00:17:29.480 --> 00:17:31.637
our approach is to push on the paradigm

00:17:31.720 --> 00:17:34.917
we understand and try to build towards

00:17:35.000 --> 00:17:39.197
uh an aligned AI researcher that can
discover whatever remaining stuff is to

00:17:39.280 --> 00:17:43.117
discover. Um and

00:17:43.200 --> 00:17:46.677
you know, we like seeing promising
signs, but still a ways to go.

00:17:46.760 --> 00:17:50.757
There's a specific question I got
from Twitter for about your timeline

00:17:50.840 --> 00:17:52.957
specifically for AI research, Sam. You

00:17:53.040 --> 00:17:55.597
at some point said automated AI research

00:17:55.680 --> 00:17:58.597
interns would be done by September 2026

00:17:58.680 --> 00:18:02.717
and fully automated AI researchers
by March 2028. Do you have an update on

00:18:02.800 --> 00:18:05.557
those timelines or is that still >>
say would be done. I said it was our

00:18:05.640 --> 00:18:08.877
goal. Uh but I feel like we are on

00:18:08.960 --> 00:18:10.837
trajectory for those goals.

00:18:10.920 --> 00:18:13.357
Makes sense. So, good. I also want to

00:18:13.440 --> 00:18:16.077
ask a question that's come up to me a

00:18:16.160 --> 00:18:18.317
lot, which is

00:18:18.400 --> 00:18:21.197
so many people in the research world

00:18:21.280 --> 00:18:23.517
specifically feel like they've just been

00:18:23.600 --> 00:18:25.997
it's it's been dominated by AI. And

00:18:26.080 --> 00:18:30.797
they're really looking to people like
yourself to say, "What are the kind of

00:18:30.880 --> 00:18:34.597
research directions that are less
explored that you'd you wish people

00:18:34.680 --> 00:18:37.237
would do more of?"
Almost like call for research. Do you

00:18:37.320 --> 00:18:40.517
have anything top of mind that you think,
"Yeah, I wish more people tried

00:18:40.600 --> 00:18:44.077
these approaches or tried more
experiments in this direction?"

00:18:44.160 --> 00:18:45.237
Francois, do you have any thoughts?

00:18:45.320 --> 00:18:48.677
>> Like, outside of AI? No,
uh inside of AI different than the

00:18:48.760 --> 00:18:53.277
current models or the current approaches
that we're taking in in some of the labs.

00:18:53.360 --> 00:18:56.117
Or it could be outside AI if you have
something interesting. I think there are

00:18:56.200 --> 00:18:59.637
so many directions uh that
people have not tried or not

00:18:59.720 --> 00:19:04.037
tried seriously yet. And in in a sense,
you know, I believe compute is a is a

00:19:04.120 --> 00:19:07.397
great equalizer. Like, if you throw
a ton of compute at one approach, you will

00:19:07.480 --> 00:19:09.077
see some really cool stuff happen.

00:19:09.160 --> 00:19:12.677
Famous rule of uh evolutionary
algorithms in Pascal.

00:19:12.760 --> 00:19:14.317
Uh I think it's been very underexplored.

00:19:14.400 --> 00:19:17.277
We have not tried to scale them, but we
could. I think we would see some really

00:19:17.360 --> 00:19:21.517
cool stuff. Uh that's true of uh program
synthesis and that's precisely what we

00:19:21.600 --> 00:19:23.237
are trying to scale right now.

00:19:23.320 --> 00:19:26.797
Um so, yeah, these these two I think are
pretty good candidates, but in general,

00:19:26.880 --> 00:19:30.557
they are just they're just
this infinite supply of uh

00:19:30.640 --> 00:19:35.597
novel and cool ideas that could work
maybe. And in fact, probably would if

00:19:35.680 --> 00:19:38.317
not work, at least you would see some
really interesting results if you were

00:19:38.400 --> 00:19:41.957
to to pursue them seriously.
Sam, what do you think?

00:19:42.040 --> 00:19:47.077
There was a time after right after we
had put out GPT-4, where I was having

00:19:47.160 --> 00:19:50.117
dinner with one of the great OpenAI
researchers and he said, you You the

00:19:50.200 --> 00:19:53.637
thing that is most sad to me right now,
there're all these brilliant people in

00:19:53.720 --> 00:19:56.917
academia, there're all
of these great ideas to try.

00:19:57.000 --> 00:20:01.757
Everybody just wants to reproduce GPT-4,
which at the time was, you know, still a

00:20:01.840 --> 00:20:05.157
difficult thing to do. And he's like,
there there's this incredibly wide

00:20:05.240 --> 00:20:08.437
space. How do we convince every academic

00:20:08.520 --> 00:20:12.397
department around the world to stop
saying like, you know, we just need to

00:20:12.480 --> 00:20:17.317
we just need to train a big model
in the in the same style. And

00:20:17.400 --> 00:20:21.237
we never found an answer to that, but I
think finally people are saying again,

00:20:21.320 --> 00:20:25.317
let's try some of these
very different ideas.

00:20:25.400 --> 00:20:28.997
Uh, I'm not sure what's changed there,
but it does feel exciting that that's

00:20:29.080 --> 00:20:34.317
happening. However, I will give
an answer outside of AI, which is

00:20:34.400 --> 00:20:37.037
for the first time in the last few
months, I think you can use the latest

00:20:37.120 --> 00:20:40.157
frontier models hugely

00:20:40.240 --> 00:20:42.637
accelerate some other kinds of research.

00:20:42.720 --> 00:20:46.637
And, you know, like AI research has no
lack of smart people working on it and

00:20:46.720 --> 00:20:51.277
no lack of funding and I think
it's great. But the stack that

00:20:51.360 --> 00:20:54.757
other directions of research, you know,
physics, mathematics, bio, whatever

00:20:54.840 --> 00:20:59.277
you'd like, that have been difficult
to make fast progress and all of a sudden

00:20:59.360 --> 00:21:02.997
feel like they might get burst
open and all these factors,

00:21:03.080 --> 00:21:06.677
um, that feels very underexplored
relative to the potential.

00:21:06.760 --> 00:21:09.557
Makes sense. I have one last question
for you, Sam, and then we can do the

00:21:09.640 --> 00:21:13.717
rapid fire. I know you have a hard
stop at 7:30. Um, and let me ask you the

00:21:13.800 --> 00:21:17.557
OpenAI question, Sam. Um, OpenAI
is doing a lot of different things, right?

00:21:17.640 --> 00:21:22.637
So, you have different modalities,
you have Sora, you have Operator, Atlas.

00:21:22.720 --> 00:21:25.597
There's a bunch of different experiments
you have run and I think everyone's

00:21:25.680 --> 00:21:29.477
grateful that you're you're running all
these experiments. Um, but very recently

00:21:29.560 --> 00:21:34.277
you said you'd shut down Sora. Um, there
was a leak about this new pre-train that

00:21:34.360 --> 00:21:38.037
you have and you want to focus all
of your effort into sort of coding and and

00:21:38.120 --> 00:21:43.117
the enterprise. Um, can you sort of add
some throw some color on that and uh

00:21:43.200 --> 00:21:47.557
talk a little bit about what
are OpenAI's top priorities?

00:21:47.640 --> 00:21:50.837
Um, with the new set of models

00:21:50.920 --> 00:21:55.197
that I think will come
out, the impact that

00:21:55.280 --> 00:21:58.997
the technology will have on accelerating
science and accelerating the economy.

00:21:59.080 --> 00:22:02.197
Uh, I mean, we may be wrong, we may be

00:22:02.280 --> 00:22:03.877
over-optimistic, but it feels like it's

00:22:03.960 --> 00:22:09.357
about to be absolutely massive. And we

00:22:09.440 --> 00:22:13.197
think that will be the most important
area for us to dedicate our compute, our

00:22:13.280 --> 00:22:15.717
resources. Um,

00:22:15.800 --> 00:22:19.077
I think things are going to get pretty

00:22:19.160 --> 00:22:22.837
heated. Uh, and we

00:22:22.920 --> 00:22:26.557
I mean, I think there's many
great things but to do, but

00:22:26.640 --> 00:22:30.157
several times in OpenAI's
history we've said, you know,

00:22:30.240 --> 00:22:33.197
there are a lot of interesting projects,
and yet there's this one new thing

00:22:33.280 --> 00:22:38.397
that's really working. Uh,
as happened in the original GPT days, I

00:22:38.480 --> 00:22:41.957
want to shut down a bunch of other
projects to concentrate our compute. And

00:22:42.040 --> 00:22:45.557
it feels like it's really time to
concentrate our compute. I mean, to give

00:22:45.640 --> 00:22:49.437
like an obvious example,
if we will be able to put

00:22:49.520 --> 00:22:53.477
a million GPUs towards curing all
cancers or a million GPUs towards making

00:22:53.560 --> 00:22:55.517
videos, I love the videos, I pick the

00:22:55.600 --> 00:22:57.997
cancer research. Um,

00:22:58.080 --> 00:23:02.957
and as we look at the next couple
of years, I think it is a time, like we've

00:23:03.040 --> 00:23:06.477
had a couple of times in our history, to
say, "Okay, we see a really big thing on

00:23:06.560 --> 00:23:10.437
the horizon. Like, let's point
all of our firepower at it."

00:23:10.520 --> 00:23:14.797
That makes total sense. All right, let's
do the rapid fire. Let's start with AGI.

00:23:14.880 --> 00:23:17.237
This is our AGI launch after all.

00:23:17.320 --> 00:23:19.997
I want you to answer
on a scale of 1 to 10,

00:23:20.080 --> 00:23:23.997
how far are we to your definition of

00:23:24.080 --> 00:23:27.797
AGI? One being nowhere close, 10 being

00:23:27.880 --> 00:23:30.757
probably going to happen this year.

00:23:30.840 --> 00:23:32.357
Friends, why don't we start with you?

00:23:32.440 --> 00:23:34.077
Are we talking a linear scale?

00:23:34.160 --> 00:23:36.557
>> [laughter] >> And it's linear.

00:23:36.640 --> 00:23:40.317
Let's let's I'll go with with five.
I I I think so.

00:23:40.400 --> 00:23:43.917
Obviously, we have models now that have
like genuine fluid intelligence.

00:23:44.000 --> 00:23:47.517
Uh, they are intelligent uh in hence
they're not human level and it's

00:23:47.600 --> 00:23:51.797
actually still we're still in this regime
where I'd say easy to find things

00:23:51.880 --> 00:23:55.717
uh that are, you know, easy far far
from normal people or from average people.

00:23:55.800 --> 00:23:58.997
Uh, you don't have to you don't have
to look for super geniuses to beat the

00:23:59.080 --> 00:24:02.597
models. Uh, and uh and you
know, things that are

00:24:02.680 --> 00:24:05.637
basically impossible for current AI.
So, we're still in this regime where it's

00:24:05.720 --> 00:24:09.997
not hard to find these tasks that humans
can do, AI cannot. I think as we move

00:24:10.080 --> 00:24:12.837
closer to AGI, it's going
to be harder and harder.

00:24:12.920 --> 00:24:15.757
And at some point, you know, uh it's
it's going to have to be like, you know,

00:24:15.840 --> 00:24:19.917
PhD level uh tasks only and and and it's
and it's going to be very very short

00:24:20.000 --> 00:24:22.277
list of times it's going to be very
difficult to come up with them. And

00:24:22.360 --> 00:24:26.717
that's the key when we have AGI. Makes
sense. Sam, what do you think? It feels

00:24:26.800 --> 00:24:29.877
very close to me. I think the only thing
I'd say is really missing is some sort

00:24:29.960 --> 00:24:32.917
of continuous learning or long-term
memory. Uh, but other than that, it

00:24:33.000 --> 00:24:35.197
feels very AGI-like to me already.

00:24:35.280 --> 00:24:38.317
Awesome. What is one
product or experience that

00:24:38.400 --> 00:24:40.277
you've tried, other than your own,

00:24:40.360 --> 00:24:46.800
that has sort of blown your mind,
AI or otherwise, in the last month or so?

00:24:47.400 --> 00:24:49.797
So, it's probably um as I commented to

00:24:49.880 --> 00:24:52.837
me, it's a coding like Codex and Code

00:24:52.920 --> 00:24:56.957
Code. It's really really impressive
and really useful as well. That's awesome.

00:24:57.040 --> 00:25:01.317
Sam, what do you think? >> These
magnetic tiles for small children.

00:25:01.400 --> 00:25:03.677
Every face connects to every other face.

00:25:03.760 --> 00:25:07.477
Okay. I had to like cut one open to see
how it worked. Um, but kids really love

00:25:07.560 --> 00:25:11.397
them, they're awesome.
What do you know what they're called?

00:25:11.480 --> 00:25:15.117
Magna- The tiles, they're little cubes,
like a full cubes so that every side can

00:25:15.200 --> 00:25:17.567
connect. >> going to say
they're sold out after this.

00:25:17.650 --> 00:25:18.597
>> [laughter]

00:25:18.680 --> 00:25:20.957
>> That's awesome. Um, okay.

00:25:21.040 --> 00:25:24.677
What is one industry or asset class that

00:25:24.760 --> 00:25:29.397
you are super bullish on and super
bearish on in the next 10 years?

00:25:29.480 --> 00:25:33.837
I'm very bullish on hardware as especially
uh robots and drones. I think

00:25:33.920 --> 00:25:37.277
that's going to be more and more
important, you know, as AI capabilities

00:25:37.360 --> 00:25:40.557
advance, we're going to want to put more
and more of these capabilities into the

00:25:40.640 --> 00:25:43.477
real world and we're going to need
a support for that. We're going to need

00:25:43.560 --> 00:25:46.677
robots, we're going to need drones.
So, I'm very bullish on drones.

00:25:46.760 --> 00:25:48.757
Uh, very bullish on hardware in general.

00:25:48.840 --> 00:25:53.157
Uh, as uh as software gets easier
and easier with like HNC coding and and

00:25:53.240 --> 00:25:56.877
eventually AGI, the the importance to
compare to the importance of hardware is

00:25:56.960 --> 00:25:59.797
only going to rise.
What do you say, Sam? I could give like

00:25:59.880 --> 00:26:03.477
a 3-hour answer to this question
with and I had so much fun.

00:26:03.560 --> 00:26:07.157
I will just limit myself to a single
observation though. The there was this

00:26:07.240 --> 00:26:10.397
quote that was bouncing around Twitter
for a while, which is all of the things

00:26:10.480 --> 00:26:13.797
we've learned about society, about
managing scarcity, are not really going

00:26:13.880 --> 00:26:17.517
to apply to managing abundance. And it
seemed really smart and really profound

00:26:17.600 --> 00:26:20.037
and then I was like after a day I
was like, "Yeah, that's kind of like not

00:26:20.120 --> 00:26:23.797
that interesting." And then it kept like
rattling around my head and I was like,

00:26:23.880 --> 00:26:27.397
"You know what?
There is one thing about this that is

00:26:27.480 --> 00:26:31.877
blowing my mind, which is countries
like us have a great deal of

00:26:31.960 --> 00:26:35.557
debt. And the plan
on this is we are going to

00:26:35.640 --> 00:26:37.797
uh inflate it away as far as I can tell.

00:26:37.880 --> 00:26:41.197
You know, we have 130% someone will
know the number, something like that,

00:26:41.280 --> 00:26:45.037
debt-to-GDP ratio, but, you know, we're
like a powerful country and inflation's

00:26:45.120 --> 00:26:47.917
going to keep going
and it's not that scary.

00:26:48.000 --> 00:26:52.717
If we're in a forever deflationary world,
I don't know how you're supposed to

00:26:52.800 --> 00:26:55.837
think about that as an asset
class and what's going to

00:26:55.920 --> 00:27:00.357
happen. That's a great answer.
Um, I want to do

00:27:00.440 --> 00:27:05.077
favorite book or a book that's
impacted you most in life.

00:27:05.160 --> 00:27:07.197
Feel free to go anyone. Let's go first.

00:27:07.280 --> 00:27:10.277
>> to go with the Asimov.
Okay. Uh, robots,

00:27:10.360 --> 00:27:12.357
the foundation the series.

00:27:12.440 --> 00:27:17.357
Um, I've wanted to build uh human level
AI since I was like 16 and I think one

00:27:17.440 --> 00:27:20.437
of the the big influences
on me at the time was the

00:27:20.520 --> 00:27:25.237
Asimov's uh robot stories.
What about you, Sam? Um, that's a great

00:27:25.320 --> 00:27:29.317
choice. I would going to say The Beginning
of Infinity, but uh the Asimov

00:27:29.400 --> 00:27:34.157
stories maybe did have more of a impact
on me when I was younger. I in general,

00:27:34.240 --> 00:27:37.837
I think about this now with kids,
like books that you read as a kid where

00:27:37.920 --> 00:27:41.077
you're like, "I can go do anything,
I can be a high-agency person, I can dream

00:27:41.160 --> 00:27:45.317
about the future." Those are really great.
Was that the book for you?

00:27:45.400 --> 00:27:48.637
I mean, it wasn't one. Like, I was like
all of the young adult fiction that had

00:27:48.720 --> 00:27:52.517
some vibe of that I loved. Um, last one.

00:27:52.600 --> 00:27:54.717
What is a person

00:27:54.800 --> 00:27:58.437
alive today that you respect the most?

00:27:58.520 --> 00:28:02.720
Could be family, could be
a famous person, could be

00:28:05.640 --> 00:28:07.677
So many great names, you know.

00:28:07.760 --> 00:28:11.717
Um, Okay, let me go
with a really random answer.

00:28:11.800 --> 00:28:14.757
Uh, I want to I want
to highlight like a researcher

00:28:14.840 --> 00:28:18.877
whose work has influenced me and is not
well known at all. Uh, his name is uh

00:28:18.960 --> 00:28:22.037
Pierre Boudier. Uh,
he's a he's a French researcher so

00:28:22.120 --> 00:28:25.277
we are researchers. He's been doing
AI for for a very very long time.

00:28:25.360 --> 00:28:28.517
And he was one of the pioneers on uh AI

00:28:28.600 --> 00:28:32.437
curiosity, like artificial curiosity,
which is really a fascinating concept.

00:28:32.520 --> 00:28:35.997
So, he's done a lot of great work. Uh,
he's very successful but also not not so

00:28:36.080 --> 00:28:39.917
well known. So, yeah, I respect him a lot,
but you know, that's just I I could

00:28:40.000 --> 00:28:42.877
go with like so so many different people.
Yeah, that's awesome. I like

00:28:42.960 --> 00:28:46.997
that. Also, Sam, you can answer well
known or not well known, up to you. Um,

00:28:47.080 --> 00:28:52.157
my mom. Love that. Love that. Well,
with that, I think we should wrap.

00:28:52.240 --> 00:28:56.337
Thank you guys so much for being
here and thank you all for attending.

00:28:56.420 --> 00:28:58.440
>> [applause]