WEBVTT

00:00:00.360 --> 00:00:04.157
It Open AI has multiple stacks to the
business, one of which is compute, and

00:00:04.240 --> 00:00:09.197
you guys have been very aggressive, very
aggressive on securing compute. Why is

00:00:09.280 --> 00:00:12.957
that? >> Well, in many ways,
we have a very simple business.

00:00:13.040 --> 00:00:16.957
We buy, rent, build compute,

00:00:17.040 --> 00:00:21.197
and we resell it at a margin.
That's it. As long as the margin's

00:00:21.280 --> 00:00:25.517
positive, then you want to scale
it because the demand for

00:00:25.600 --> 00:00:30.797
solving problems, the demand
for intelligence, that's unlimited.

00:00:30.880 --> 00:00:33.997
And the AIs that we have right now
really are able to rise to the challenge

00:00:34.080 --> 00:00:36.957
of effectively any kind of problem
that you want to throw at them.

00:00:37.040 --> 00:00:38.357
>> Do you have enough compute?

00:00:38.440 --> 00:00:41.317
>> No. >> Really? >> Yeah, definitely not.

00:00:41.400 --> 00:00:43.757
>> was just with Matt Garman, and he says

00:00:43.840 --> 00:00:48.957
the GPU compute availability in 2026
rounds to zero. Don't you guys

00:00:49.040 --> 00:00:52.117
have all of it? >> I mean,
we have we we would love more.

00:00:52.200 --> 00:00:56.437
We're we're constantly out there hunting
for more, honestly. And I'll tell you

00:00:56.520 --> 00:00:59.277
like when we first launched
when we launched ChatGPT,

00:00:59.360 --> 00:01:02.757
I remember being on a call with my team,
and they were like, "All right, how much

00:01:02.840 --> 00:01:06.477
compute should we buy?"
And I said, "All of it."

00:01:06.560 --> 00:01:08.517
And they're like, "No, no, no, we're
seriously like like come on, how much

00:01:08.600 --> 00:01:12.277
should we buy?" I'm like, "No matter how
fast we try to ramp compute, I guarantee

00:01:12.360 --> 00:01:14.877
we're not going to be able to keep up
with demand, and that has been true ever

00:01:14.960 --> 00:01:19.037
since." >> Um that's that's
fascinating. Moving up

00:01:19.120 --> 00:01:23.917
from compute since I don't know if
much of this audience can help you with

00:01:24.000 --> 00:01:27.237
securing more compute because
most of them are founders of

00:01:27.320 --> 00:01:31.077
startups.
About architecture and scaling laws,

00:01:31.160 --> 00:01:34.917
what are the what are
the where are we in the scaling

00:01:35.000 --> 00:01:37.397
laws? Are they still doubling each year?

00:01:37.480 --> 00:01:42.477
Are you changing architecture? What's
what is what are you guys pushing on the

00:01:42.560 --> 00:01:44.277
frontier on the research side?

00:01:44.360 --> 00:01:48.597
>> Well, I would say first of all,
the scaling laws are a deep and very

00:01:48.680 --> 00:01:50.917
beautiful mystery, right? They feel

00:01:51.000 --> 00:01:52.997
deeply fundamental. It's like the

00:01:53.080 --> 00:01:58.437
scientific truth that just like you
think about physics and, you know,

00:01:58.520 --> 00:02:00.757
Newton's laws and things like that,
they're somehow this truth of the

00:02:00.840 --> 00:02:04.957
universe. And they're
empirical. Like we don't

00:02:05.040 --> 00:02:09.037
necessarily have all the theory
to explain exactly why it works, but to me

00:02:09.120 --> 00:02:12.277
the most beautiful thing
is that neural networks

00:02:12.360 --> 00:02:14.917
were really designed like in the 1940s

00:02:15.000 --> 00:02:16.797
before there were computers.

00:02:16.880 --> 00:02:19.637
And somehow we've been able to take the

00:02:19.720 --> 00:02:24.117
exact ideas that were developed back
then and apply increasing amounts of

00:02:24.200 --> 00:02:28.197
computation. And as you pour more
compute into the models, they get

00:02:28.280 --> 00:02:32.077
correspondingly more capable.
And it just keeps going. There's no wall.

00:02:32.160 --> 00:02:34.837
And that's I think
that's a beautiful thing.

00:02:34.920 --> 00:02:36.637
>> That's pretty beautiful. Are there more

00:02:36.720 --> 00:02:40.197
researched or more algorithms that are

00:02:40.280 --> 00:02:43.797
in the works that because you
know, in the past we had neural

00:02:43.880 --> 00:02:47.477
networks to your point in 1940s, but we
couldn't we didn't have the compute for

00:02:47.560 --> 00:02:50.717
it. Now that we have the compute for it,
you just or are we just pushing the same

00:02:50.800 --> 00:02:54.077
things or are there new architectures
and new ideas coming out?

00:02:54.160 --> 00:02:58.357
>> Yeah, so I would I would think of it as
we we absolutely have new ideas that are

00:02:58.440 --> 00:03:01.797
constantly powering what we do. It's
very simplified to say, well, let's take

00:03:01.880 --> 00:03:06.077
a neural network from the 1940s and you
know, put put it in a gigawatt data

00:03:06.160 --> 00:03:09.757
center, right? We have made tons
of innovations and we constantly are

00:03:09.840 --> 00:03:13.517
improving things. And sometimes these
are micro tweaks. Like you just realize

00:03:13.600 --> 00:03:17.277
that the way you've been formatting
data was not quite right and that can

00:03:17.360 --> 00:03:21.237
actually be a a very big deal. Sometimes
it's larger. You think about the shift

00:03:21.320 --> 00:03:24.597
from the LSTM to the transformer and I
don't think the transformer is, you

00:03:24.680 --> 00:03:27.517
know, like everyone's moved past
the transformer as described in the other

00:03:27.600 --> 00:03:30.797
2018 paper. So there's
there's constant innovation

00:03:30.880 --> 00:03:34.157
happening and I think of places that
have been perhaps the most invested in

00:03:34.240 --> 00:03:37.677
long-term research on how to improve
the architectures, how to improve the

00:03:37.760 --> 00:03:41.117
fundamental algorithms, and how to get
the paradigm shifts. I think OpenAI has

00:03:41.200 --> 00:03:43.877
been leading the pack there and that's
something we continue to invest in and I

00:03:43.960 --> 00:03:45.557
see lots of fruit on the horizon.

00:03:45.640 --> 00:03:49.757
>> Got it. And on the models,
um, does OpenAI have

00:03:49.840 --> 00:03:54.117
a formal definition for AGI?
Are we closed? Are we not close?

00:03:54.200 --> 00:03:58.917
Pat and Sonia published this thing
that we are at AGI functionally.

00:03:59.000 --> 00:04:00.837
Do you agree with that?
Do you not agree with that?

00:04:00.920 --> 00:04:04.477
>> Well, we do have a formal
definition, but to some extent

00:04:04.560 --> 00:04:08.277
I one thing I have learned is that
everyone has their own intuitions about

00:04:08.360 --> 00:04:12.837
what AGI is and maybe you can view it
as like according to my view of where we

00:04:12.920 --> 00:04:17.477
are. I think we're about 80% of the way
there and that we have models that are

00:04:17.560 --> 00:04:21.557
smart. They're very capable.
They're able to if you give

00:04:21.640 --> 00:04:24.157
>> Are they smarter than you?
>> I mean I they're certainly more capable

00:04:24.240 --> 00:04:28.197
than I am at writing software, right? If
you give it all the context, then yes, I

00:04:28.280 --> 00:04:33.157
I I think that they are they're just
so capable. It's it's really remarkable.

00:04:33.240 --> 00:04:39.720
Like does anyone here feel better
at writing software than GPD 5.4?

00:04:39.960 --> 00:04:42.360
>> Oh.

00:04:43.040 --> 00:04:47.837
>> All right, writing kernels. Um so
even there we're seeing massive gains from

00:04:47.920 --> 00:04:51.877
Exactly. We're and for some of our
internal results um that there we're

00:04:51.960 --> 00:04:55.517
really seeing if you
pour the right kinds of of

00:04:55.600 --> 00:04:58.477
you know, if you have the right setup
for your problem, then you're able to

00:04:58.560 --> 00:05:03.157
get really massive results out of very
low-level um even low-level tasks. And

00:05:03.240 --> 00:05:06.557
just to give you one example
of how things have been trending um

00:05:06.640 --> 00:05:10.757
one of my systems engineers also very
similar was like, "Hey, like I haven't

00:05:10.840 --> 00:05:12.277
been able to get value out of the models

00:05:12.360 --> 00:05:14.957
for GPD 5 or 5.1

00:05:15.040 --> 00:05:19.477
for 5.2 as well. For 5.3 he on a lark

00:05:19.560 --> 00:05:22.517
prepar- prepared this design document
for a very complicated systems

00:05:22.600 --> 00:05:24.437
optimization he was about to do.

00:05:24.520 --> 00:05:29.037
He handed it over to the model, went
to sleep waking up intending to like give

00:05:29.120 --> 00:05:31.357
this to his team to work
on for the next week.

00:05:31.440 --> 00:05:33.077
And when he woke up it was done.

00:05:33.160 --> 00:05:37.317
That the model had actually
implemented the initial spec, had seen

00:05:37.400 --> 00:05:40.557
that it was slow, had added
instrumentation, had actually run the

00:05:40.640 --> 00:05:44.317
code, used a profiler to figure out
where things were slow, and iterated

00:05:44.400 --> 00:05:45.757
multiple times until it gotten to an

00:05:45.840 --> 00:05:52.360
optimized result. And like like
that is incredible. That's where we are.