1
00:00:02,120 --> 00:00:05,437
So, Greg, uh thank you
for coming back here. Um

2
00:00:05,520 --> 00:00:08,597
I don't think we ever charged you
for rent. So, uh maybe we'll send you an

3
00:00:08,680 --> 00:00:12,517
invoice later. But, uh Greg, you've
been part of like two really spectacular

4
00:00:12,600 --> 00:00:14,677
companies, Stripe as employee number

5
00:00:14,760 --> 00:00:17,397
four, and then the first CTO. I just

6
00:00:17,480 --> 00:00:19,757
recently heard that they process 1.6

7
00:00:19,840 --> 00:00:22,917
billion, sorry, 1.6% of the

8
00:00:23,000 --> 00:00:25,437
global GDP. You must be proud of that.

9
00:00:25,520 --> 00:00:29,837
That's amazing. You must be even more
proud of the fact that OpenAI has almost

10
00:00:29,920 --> 00:00:34,397
a billion or maybe more than a billion
uh in weekly active users at this point.

11
00:00:34,480 --> 00:00:35,917
I mean, it's all it's all very exciting.

12
00:00:36,000 --> 00:00:39,837
It shows you what technology can do.
And uh you're not just co-founder and

13
00:00:39,920 --> 00:00:42,237
president, but you're also chief builder.

14
00:00:42,320 --> 00:00:45,437
At OpenAI, I heard that that was one of
your titles. I'm not sure if it was ever

15
00:00:45,520 --> 00:00:49,317
an official title, but I've been
called many things. Let's just say that.

16
00:00:49,400 --> 00:00:53,277
Well, you have a audience of great
builders here, so we'll start from all

17
00:00:53,360 --> 00:00:57,597
the way at the bottom of the stack.
You OpenAI has multiple stacks of the

18
00:00:57,680 --> 00:01:01,717
business, one of which is compute, and you
guys have been very aggressive, very

19
00:01:01,800 --> 00:01:05,517
aggressive on securing
compute. Why is that?

20
00:01:05,600 --> 00:01:08,277
Well, in many ways,
we have a very simple business.

21
00:01:08,360 --> 00:01:12,277
We buy, rent, build compute,

22
00:01:12,360 --> 00:01:16,517
and we resell it at a margin.
That's it. As long as the margin's

23
00:01:16,600 --> 00:01:20,837
positive, then you want to scale
it because the demand for

24
00:01:20,920 --> 00:01:26,117
solving problems, the demand
for intelligence, that's unlimited.

25
00:01:26,200 --> 00:01:29,317
And the AIs that we have right now
really are able to rise to the challenge

26
00:01:29,400 --> 00:01:32,517
of effectively any kind of problem that
you want to throw at them. Do you have

27
00:01:32,600 --> 00:01:36,717
enough compute? No. Really?
Yeah, definitely not.

28
00:01:36,800 --> 00:01:39,397
>> just with Matt Garman, and he says the

29
00:01:39,480 --> 00:01:42,117
GPU compute availability

30
00:01:42,200 --> 00:01:46,837
in 2026 rounds to zero. Don't you guys
have all of it? I mean, we have we we

31
00:01:46,920 --> 00:01:50,437
would love more. We're we're constantly
out there hunting for more, honestly.

32
00:01:50,520 --> 00:01:54,597
And I'll tell you like when we first
launched when we launched ChatGPT,

33
00:01:54,680 --> 00:01:58,077
I remember being on a call with my team,
and they're like, all right, how much

34
00:01:58,160 --> 00:02:01,797
computer should we buy?
And I said, all of it.

35
00:02:01,880 --> 00:02:03,797
And they're like, no, no, no, we're
seriously, like like come on, how much

36
00:02:03,880 --> 00:02:07,557
should we buy? I'm like, no matter how
fast we try to ramp compute, I guarantee

37
00:02:07,640 --> 00:02:09,957
you we're not going to be able to keep
up with demand and that has been true

38
00:02:10,040 --> 00:02:11,517
ever since.

39
00:02:11,600 --> 00:02:14,357
Um, that's that's fascinating. Moving up

40
00:02:14,440 --> 00:02:19,237
from compute since I don't know if
much of this audience can help you with

41
00:02:19,320 --> 00:02:22,517
securing more compute um because
most of them are founders of

42
00:02:22,600 --> 00:02:25,397
startups. Um, about architecture and

43
00:02:25,480 --> 00:02:27,637
scaling laws, what are the

44
00:02:27,720 --> 00:02:30,197
what are the where are we in the scaling

45
00:02:30,280 --> 00:02:32,677
laws? Are they still doubling each year?

46
00:02:32,760 --> 00:02:36,797
Are you changing architecture?
What's what is

47
00:02:36,880 --> 00:02:39,797
what are you guys pushing on the
frontier on the research side? Well, I

48
00:02:39,880 --> 00:02:42,277
would say first of all, the scaling laws

49
00:02:42,360 --> 00:02:45,277
are a deep and very beautiful mystery.

50
00:02:45,360 --> 00:02:47,957
Right? They feel deeply fundamental.

51
00:02:48,040 --> 00:02:53,437
It's like the scientific truth that just
like you think about physics and you

52
00:02:53,520 --> 00:02:56,117
know, Newton's laws and things like that,
they're somehow this truth of the

53
00:02:56,200 --> 00:03:00,197
universe. And they're
empirical. Like we don't

54
00:03:00,280 --> 00:03:04,357
necessarily have all the theory
to explain exactly why it works, but to me

55
00:03:04,440 --> 00:03:07,557
the most beautiful thing
is that neural networks

56
00:03:07,640 --> 00:03:10,197
were really designed like in the 1940s

57
00:03:10,280 --> 00:03:12,077
before there were computers.

58
00:03:12,160 --> 00:03:14,957
And somehow we've been able to take the

59
00:03:15,040 --> 00:03:17,277
exact ideas that were developed back

60
00:03:17,360 --> 00:03:22,157
then and apply increasing amounts of
computation and as you pour more compute

61
00:03:22,240 --> 00:03:25,357
into the models, they get
correspondingly more capable and it just

62
00:03:25,440 --> 00:03:30,117
keeps going. There's no wall. And that's
I think that's a beautiful thing.

63
00:03:30,200 --> 00:03:31,917
That's pretty beautiful. Are there more

64
00:03:32,000 --> 00:03:35,637
research or more algorithms that are in

65
00:03:35,720 --> 00:03:39,117
the works that because you know,
in the past we had neural

66
00:03:39,200 --> 00:03:42,797
networks to your point in 1940s, but we
couldn't we didn't have the computer for

67
00:03:42,880 --> 00:03:45,717
it. Now that we have the computer for it,
you just where are we just pushing

68
00:03:45,800 --> 00:03:49,357
the same things or are there new
architectures and new ideas coming up?

69
00:03:49,440 --> 00:03:53,717
Yeah, so I would I would think of it as we
we absolutely have new ideas that are

70
00:03:53,800 --> 00:03:57,157
constantly powering what we do. It's
very simplified to say, well, let's take

71
00:03:57,240 --> 00:04:01,477
a neural network from the 1940s and,
you know, put put it in a gigawatt data

72
00:04:01,560 --> 00:04:05,117
center, right? We have made tons
of innovations and we constantly are

73
00:04:05,200 --> 00:04:08,877
improving things. And sometimes these
are micro tweaks. Like you just realize

74
00:04:08,960 --> 00:04:12,637
that the way you've been formatting
data was not quite right and that can

75
00:04:12,720 --> 00:04:16,597
actually be a very big deal. Sometimes
it's larger. You think about the shift

76
00:04:16,680 --> 00:04:19,997
from the LSTM to the transformer and I
don't think the transformer is, you

77
00:04:20,080 --> 00:04:22,837
know, like everyone's moved past
the transformer as described in the other

78
00:04:22,920 --> 00:04:26,117
2018 paper. So, there's
there's constant innovation

79
00:04:26,200 --> 00:04:29,477
happening and I think of places that
have been perhaps the most invested in

80
00:04:29,560 --> 00:04:32,997
long-term research on how to improve
the architectures, how to improve the

81
00:04:33,080 --> 00:04:36,437
fundamental algorithms, and how to get
the paradigm shifts. I think OpenAI has

82
00:04:36,520 --> 00:04:39,197
been leading the pack there and that's
something we continue to invest in and I

83
00:04:39,280 --> 00:04:42,117
see lots of fruit on the horizon. Got it.

84
00:04:42,200 --> 00:04:44,437
And on the models, um

85
00:04:44,520 --> 00:04:49,357
Does OpenAI have a formal definition
for AGI? Are we close? Are we not close? I

86
00:04:49,440 --> 00:04:54,237
Pat and Sam and Ilya published this
thing that we are at AGI functionally.

87
00:04:54,320 --> 00:04:57,477
Do you agree with that? Do you not agree
with that? Well, we do have a formal

88
00:04:57,560 --> 00:05:02,077
definition, but to some extent I one
thing I have learned is that everyone

89
00:05:02,160 --> 00:05:06,397
has their own intuitions about what
AGI is and maybe you can view it as like

90
00:05:06,480 --> 00:05:08,317
according to my view of where we are, I

91
00:05:08,400 --> 00:05:10,677
think we're about 80% of the way there

92
00:05:10,760 --> 00:05:13,557
in that we have models that are smart.

93
00:05:13,640 --> 00:05:18,317
They're very capable. They're able
to if you give Are they smarter than you?

94
00:05:18,400 --> 00:05:21,437
I mean, I they're certainly more capable
than I am at writing software, right? If

95
00:05:21,520 --> 00:05:23,517
you give it all the context, then yes, I

96
00:05:23,600 --> 00:05:28,477
I I think that they are they're just
so capable. It's it's really remarkable.

97
00:05:28,560 --> 00:05:34,960
Like is anyone here feel better
at writing software than GPT-5.4?

98
00:05:35,280 --> 00:05:37,720
Oh.

99
00:05:38,320 --> 00:05:43,157
All right, writing kernels. Um so,
even there we're seeing massive gains from

100
00:05:43,240 --> 00:05:46,997
Exactly. We're And for some
of our internal results, um that

101
00:05:47,080 --> 00:05:50,877
we're really seeing if you
pour the right kinds of of

102
00:05:50,960 --> 00:05:53,797
you know, if you have the right setup
for your problem, then you're able to

103
00:05:53,880 --> 00:05:58,477
get really massive results out of very
low-level, um, even low-level tasks. And

104
00:05:58,560 --> 00:06:01,917
just to give you one example of how
things have been trending, um,

105
00:06:02,000 --> 00:06:06,077
one of my systems engineers, also very
similar, was like, "Hey, like, I haven't

106
00:06:06,160 --> 00:06:07,597
been able to get value out of the models

107
00:06:07,680 --> 00:06:10,317
for GPT-5 or 5.1,

108
00:06:10,400 --> 00:06:14,917
for 5.2 as well." For 5.3, he on a lark

109
00:06:15,000 --> 00:06:18,397
had prepared this design document for a
very complicated systems optimization he

110
00:06:18,480 --> 00:06:22,717
was about to do. He handed
it over to the model, went to

111
00:06:22,800 --> 00:06:25,757
sleep, waking up intending to like give
this to his team to work on for the next

112
00:06:25,840 --> 00:06:28,437
week. And when he woke up, it was done.

113
00:06:28,520 --> 00:06:32,677
That the model had actually
implemented the initial spec, had seen

114
00:06:32,760 --> 00:06:35,877
that it was slow, had added
instrumentation, had actually run the

115
00:06:35,960 --> 00:06:39,637
code, used a profiler to figure out
where things were slow, and iterated

116
00:06:39,720 --> 00:06:43,717
multiple times until it got into
an optimized result. And like, like, that

117
00:06:43,800 --> 00:06:46,477
is incredible. That's where we are.

118
00:06:46,560 --> 00:06:49,877
And so, how what would you advise all
startups here to do? Because the models

119
00:06:49,960 --> 00:06:52,037
keep getting more and more capable.

120
00:06:52,120 --> 00:06:55,437
They're kind of, uh, I've asked
this, uh, when Sam was here

121
00:06:55,520 --> 00:06:59,037
in the past, and, you know, what if
you're building today, do you need to

122
00:06:59,120 --> 00:07:03,517
rebuild in 2 years when a new model
comes out? Because all the functionality

123
00:07:03,600 --> 00:07:07,237
and all the capabilities
all change around you.

124
00:07:07,320 --> 00:07:12,637
Um, do you need to make sure that
you're not in OpenAI's way because

125
00:07:12,720 --> 00:07:16,357
you're going to roll, you're just going
to run over for startups because the

126
00:07:16,440 --> 00:07:18,637
models are so much more capable.

127
00:07:18,720 --> 00:07:21,597
Um, how how would you recommend the, uh,

128
00:07:21,680 --> 00:07:26,397
a set of, um, startup founders
to to build in this environment? Well,

129
00:07:26,480 --> 00:07:28,437
first of all, I would say to lean in.

130
00:07:28,520 --> 00:07:32,717
The tools right now have become
incredibly useful. And if you look even

131
00:07:32,800 --> 00:07:36,477
over the course of December, I think
that we went from these agentic coding

132
00:07:36,560 --> 00:07:40,837
tools being like, you know, they're like
writing 20% of your code to writing 80%

133
00:07:40,920 --> 00:07:43,917
of your code. Which means they go from
being kind of a side show to being the

134
00:07:44,000 --> 00:07:48,717
main thing that you're doing.
And I think we're doing that across all of

135
00:07:48,800 --> 00:07:52,037
the work that people do with computers,
all computer work this year. And you can

136
00:07:52,120 --> 00:07:53,517
look at the recent progress on Codex.

137
00:07:53,600 --> 00:07:57,397
It's really changing from a tool for
software engineers to a tool for anyone

138
00:07:57,480 --> 00:08:01,347
who's doing work with a computer. And
just over the past week, we've released

139
00:08:01,430 --> 00:08:03,877
[snorts] a bunch of features that just
make it so much more powerful and

140
00:08:03,960 --> 00:08:08,677
capable. Um and like one thing we just
announced today is a new tool called

141
00:08:08,760 --> 00:08:12,117
Chronicle that plugs into the Codex,
where it actually can see everything

142
00:08:12,200 --> 00:08:16,237
you're doing with your computer and can
form memories of what's going on. And so

143
00:08:16,320 --> 00:08:19,397
you ask it a question, you just it
instantly knows what you're talking

144
00:08:19,480 --> 00:08:21,917
about. You're like, "Huh, what was I
doing 5 minutes ago?" It knows, right?

145
00:08:22,000 --> 00:08:25,597
You're like, "Oh, what was this person
talking about?" It knows. It's To me, it

146
00:08:25,680 --> 00:08:28,757
was this real wake-up call to realize
you spend so much of your effort right

147
00:08:28,840 --> 00:08:32,917
now just explaining your computer what's
going on. Like, why are you explaining

148
00:08:33,000 --> 00:08:35,837
to your computer what's going on?
That makes no sense. And so I think what's

149
00:08:35,920 --> 00:08:39,397
going to happen over upcoming years
is the models are going to get much more

150
00:08:39,480 --> 00:08:43,437
capable, will have better harnesses,
will be able to be able to solve harder

151
00:08:43,520 --> 00:08:46,677
and harder problems, come up with new
knowledge, all of these things. But

152
00:08:46,760 --> 00:08:49,237
there is a one-time shift that's
happening now, which is really about

153
00:08:49,320 --> 00:08:54,037
context. It's really
about is your AI able to

154
00:08:54,120 --> 00:08:57,277
you have all these meetings,
you didn't include the AI.

155
00:08:57,360 --> 00:09:00,037
You know, that's not very nice to the AI.
Like, you're asking it to to help

156
00:09:00,120 --> 00:09:02,877
you with things and it has no information.
So I think really leaning

157
00:09:02,960 --> 00:09:06,597
into how do you make sure the AI even
has enough information in theory to

158
00:09:06,680 --> 00:09:10,477
solve the problem and then trust the
models are going to really get there and

159
00:09:10,560 --> 00:09:14,317
improve. So I think it will be
a constant cycle of improvement and

160
00:09:14,400 --> 00:09:17,557
iteration and leaning into the tools
and kind of talking to your friends if

161
00:09:17,640 --> 00:09:20,877
you're how they're using it,
but that there is this investment that's a

162
00:09:20,960 --> 00:09:23,237
one-time investment that now is the time

163
00:09:23,320 --> 00:09:25,677
to make. And in terms of like, let's say

164
00:09:25,760 --> 00:09:28,237
we you you set that all up, how do you

165
00:09:28,320 --> 00:09:31,717
how is OpenAI using Codex differently

166
00:09:31,800 --> 00:09:35,677
than you think everybody
else outside is using it?

167
00:09:35,760 --> 00:09:38,997
Well, I think one of the amazing things
about being at OpenAI is you do get to

168
00:09:39,080 --> 00:09:42,717
live in the future, right? You do get to
really see the shape of what's emerging,

169
00:09:42,800 --> 00:09:46,517
and we can co-design, right? We can
really change the models, the harness,

170
00:09:46,600 --> 00:09:50,997
everything together in order
to better serve the needs that we see.

171
00:09:51,080 --> 00:09:53,477
And a lot of the approach we've been
taking is so we started with software

172
00:09:53,560 --> 00:09:58,437
engineering, and we set some clear
guidelines for example saying that

173
00:09:58,520 --> 00:10:02,397
we still want a human to be accountable
for all code that gets merged, right?

174
00:10:02,480 --> 00:10:07,877
So, at the end of the day, is it a good
thing to merge this piece of code?

175
00:10:07,960 --> 00:10:10,917
Is it well-structured? Is it going
to make our codebase more maintainable? We

176
00:10:11,000 --> 00:10:14,637
want to make sure there's a human who
is signing off to say yes. And that's I

177
00:10:14,720 --> 00:10:17,157
think that thoughtfulness of not just
saying, "Okay, let's just blindly use

178
00:10:17,240 --> 00:10:19,597
this." Or, you know, "Oh, we don't
want to use this at all." Like I think

179
00:10:19,680 --> 00:10:23,677
neither extreme is quite right. And then
we are also going vertical by vertical

180
00:10:23,760 --> 00:10:25,437
within OpenAI to adopt these tools

181
00:10:25,520 --> 00:10:28,837
within finance, within sales, within

182
00:10:28,920 --> 00:10:33,757
IT. And there we have a small dedicated
team who's really deeply understanding

183
00:10:33,840 --> 00:10:37,117
the domain, working with the people who
are the experts in it in order to build

184
00:10:37,200 --> 00:10:42,237
skills, in order to modify the Codex UI,
whatever it is that is is needed in

185
00:10:42,320 --> 00:10:45,997
order to get it to be good. And then
that's something we can then, once we

186
00:10:46,080 --> 00:10:49,877
have it in good shape, we will
externalize, and that we're able to to

187
00:10:49,960 --> 00:10:53,677
ship that to all of you. And so we are
starting to work with certain customers

188
00:10:53,760 --> 00:10:57,197
as well. So, for people who want to be
very AI forward and want to be part of

189
00:10:57,280 --> 00:11:00,757
defining this revolution, that there's
a place for that, and I'd love to talk

190
00:11:00,840 --> 00:11:03,597
afterwards. But yeah,
I think that just this desire

191
00:11:03,680 --> 00:11:06,757
to say, "Hey, we really want to be AI
forward, really live in the future, and

192
00:11:06,840 --> 00:11:10,437
experience what it will be like
for everyone else 1 year, 2 years, 3 years

193
00:11:10,520 --> 00:11:13,197
down the road."
Do you guys structure your company

194
00:11:13,280 --> 00:11:17,437
differently with the engineering teams
differently because of of the living in

195
00:11:17,520 --> 00:11:20,597
the future? I mean, if you have
to go way back, when my father learned

196
00:11:20,680 --> 00:11:22,637
computer science, he was just himself,

197
00:11:22,720 --> 00:11:25,157
and then we had these long software

198
00:11:25,240 --> 00:11:29,757
releases that became waterfall. And then
when the web happened and the cloud

199
00:11:29,840 --> 00:11:32,997
happened, when these two-pizza
teams, and we have scrum.

200
00:11:33,080 --> 00:11:38,037
Now that we have these coding agents,
is it is how do you structure around

201
00:11:38,120 --> 00:11:41,797
everything differently?
I think we're still figuring it out, and

202
00:11:41,880 --> 00:11:45,077
there's certain places where you
really see it. For example, the cost of

203
00:11:45,160 --> 00:11:49,277
building a prototype is cheap now.
It's so cheap, and if you want to build a

204
00:11:49,360 --> 00:11:53,677
dashboard that used to be like, uh, it'd
take like someone like a week to do it,

205
00:11:53,760 --> 00:11:56,917
and you just do it now. And so actually
a lot of the bottleneck has shifted to

206
00:11:57,000 --> 00:12:01,037
things like sharing. Like how do you
And so we we actually have some internal

207
00:12:01,120 --> 00:12:04,037
work on this as well that again we will
be externalizing. If How do you make it

208
00:12:04,120 --> 00:12:07,797
really easy for anyone in your
enterprise to build a dashboard, a

209
00:12:07,880 --> 00:12:11,037
widget, a bot, whatever the thing is,
and then share it with others. And then

210
00:12:11,120 --> 00:12:14,557
that starts to really put pressure
on having good governance. Like you want

211
00:12:14,640 --> 00:12:17,597
your IT organization to be
able to see all these different,

212
00:12:17,680 --> 00:12:19,717
you know, threads of execution that are
happening, all the little things that

213
00:12:19,800 --> 00:12:23,437
are being shared around,
have some control over

214
00:12:23,520 --> 00:12:26,957
data provenance, right? To really make
sure that, okay, like a good example of

215
00:12:27,040 --> 00:12:30,557
this is, um, I think people
are now starting to take

216
00:12:30,640 --> 00:12:33,837
their internal knowledge from storing
them to wikis. We we have some some a

217
00:12:33,920 --> 00:12:37,237
really cool one of these internally,
and the thing you immediately think about

218
00:12:37,320 --> 00:12:41,677
is, well, if someone has a document
in the internal knowledge base that was

219
00:12:41,760 --> 00:12:44,677
accidentally permissioned incorrectly,
and they realize, oh no, I didn't want

220
00:12:44,760 --> 00:12:49,037
this information to be accessible,
how do they fix that, right? So normally

221
00:12:49,120 --> 00:12:51,717
it's they go into the doc, they change
the permissions, but now there's these

222
00:12:51,800 --> 00:12:54,797
derived artifacts. And so you need
to make sure you have some way of tracking

223
00:12:54,880 --> 00:12:58,837
through the system to say, well,
this output document came from the source

224
00:12:58,920 --> 00:13:01,997
one. The source one's no longer
accessible to this Let's go and

225
00:13:02,080 --> 00:13:05,557
invalidate that as well. And so you have
to start really building your technical

226
00:13:05,640 --> 00:13:09,317
architecture with awareness of the way
that people are going to use this

227
00:13:09,400 --> 00:13:13,597
information, and it really changes how
teams relate to each other because you

228
00:13:13,680 --> 00:13:17,717
can just It really changes where
the bottlenecks are and what's hard.

229
00:13:17,800 --> 00:13:21,077
Uh, do you think team sizes
going to be a lot smaller?

230
00:13:21,160 --> 00:13:24,397
Are we going to have still
human software engineers in a

231
00:13:24,480 --> 00:13:28,437
decade? Well, decade
is a long time from now and

232
00:13:28,520 --> 00:13:32,757
the the ceiling on this technology is hard
to is really hard to internalize. I

233
00:13:32,840 --> 00:13:37,597
think that it is clear that what
a company is will change in a lot of ways.

234
00:13:37,680 --> 00:13:42,397
I think that we're going to have this
ability for solopreneurs to build very

235
00:13:42,480 --> 00:13:45,437
incredible businesses. And so anyone
who has a vision, I think will be able to

236
00:13:45,520 --> 00:13:49,477
realize it. I think the jobs that you
all have will become way easier in a lot

237
00:13:49,560 --> 00:13:52,677
of ways, way more fun. Now, might be
more competitive, too. Right? Because

238
00:13:52,760 --> 00:13:55,197
everyone's going to have these amazing
tools. And so really figuring out what

239
00:13:55,280 --> 00:13:58,837
is your niche, what is your unique angle,
is probably going to become kind

240
00:13:58,920 --> 00:14:02,917
of the most important core. But a lot
of how we run organizations right now, and

241
00:14:03,000 --> 00:14:05,677
it's there's almost only one way
to organize large groups of people, where

242
00:14:05,760 --> 00:14:08,677
you have teams, you have management
structures, and you have scopes, and you

243
00:14:08,760 --> 00:14:11,677
have these hierarchies, and all these
things. Maybe that can change. Maybe you

244
00:14:11,760 --> 00:14:15,597
can be much more flat, small teams
that can really just do incredible things.

245
00:14:15,680 --> 00:14:19,517
Like, we're seeing it right
now in mathematics, where

246
00:14:19,600 --> 00:14:24,677
these individuals on the internet are
using GPT-4 Pro to solve these unsolved

247
00:14:24,760 --> 00:14:28,437
math problems. Normally you need a math
team, and they're just doing it. Yeah.

248
00:14:28,520 --> 00:14:32,437
And my son's a math nerd. I just told
him that maybe we should be studying

249
00:14:32,520 --> 00:14:35,717
something else besides math.
But I well, but see, this is the

250
00:14:35,800 --> 00:14:38,957
question, right? Is if you look
at something like AlphaGo, you know, move

251
00:14:39,040 --> 00:14:43,677
37, this move that just like changed
humanity's understanding of the game,

252
00:14:43,760 --> 00:14:47,597
but the thing that was surprising
is it made the game more interesting and

253
00:14:47,680 --> 00:14:50,517
important for humans. And maybe
that will be true for for these other

254
00:14:50,600 --> 00:14:54,557
domains, too. True.
What about uh common failure modes when

255
00:14:54,640 --> 00:14:58,357
you're building when you're you're
you're building with uh production

256
00:14:58,440 --> 00:15:01,757
agentic workflows? What do you What
do you see as the common things that

257
00:15:01,840 --> 00:15:06,197
founders get wrong and they're building
incorrectly these days? Well, I think

258
00:15:06,280 --> 00:15:09,757
that these models,
they have such power, and

259
00:15:09,840 --> 00:15:13,917
really understanding how to operate them
well takes thought. And so we've been

260
00:15:14,000 --> 00:15:18,597
investing a lot in primitives, security
primitives, observability, having again

261
00:15:18,680 --> 00:15:21,917
good governance, things like that. Um
but just to give you one anecdote that I

262
00:15:22,000 --> 00:15:25,717
think is evocative, um, I asked So,
I was working with my co-workers, I asked

263
00:15:25,800 --> 00:15:28,437
them to install some package that someone
at OpenAI had written, ran into

264
00:15:28,520 --> 00:15:31,517
an error, I was like, "Oh, ping that
person on Slack and ask them for help."

265
00:15:31,600 --> 00:15:34,477
So, ping the person on Slack.
Two minutes later, it said, "This is

266
00:15:34,560 --> 00:15:37,037
taking too long. I've escalated
to the person's manager." And it actually

267
00:15:37,120 --> 00:15:42,557
pinged the person's manager. >> [laughter]
>> And And you realize it's like, on the

268
00:15:42,640 --> 00:15:45,517
one hand, it's kind of a reasonable
thing for the model to do. It's being

269
00:15:45,600 --> 00:15:48,517
proactive, it's trying to solve
my problem. It's like, you know, not just

270
00:15:48,600 --> 00:15:51,797
sitting around waiting to be told what
to do. But, on the other hand, like, you

271
00:15:51,880 --> 00:15:54,077
know, >> [laughter] >> maybe it
should have taken a little bit

272
00:15:54,160 --> 00:15:58,037
longer, maybe should have checked with me.
And so, I think that really thinking

273
00:15:58,120 --> 00:16:03,117
about these questions where we're still
building up the EQ of the model. And

274
00:16:03,200 --> 00:16:06,517
that in some places, it's getting very
good. For example, clicking approve,

275
00:16:06,600 --> 00:16:10,517
approve, approve is kind of where we've
been. And humans are not very good at

276
00:16:10,600 --> 00:16:13,357
that, either, right?
>> They just They They just default.

277
00:16:13,440 --> 00:16:16,757
>> default. And so, now we're starting
to have AIs that can actually take care of

278
00:16:16,840 --> 00:16:18,677
flagging, is this a high-risk action?

279
00:16:18,760 --> 00:16:21,637
Hey, this one should be escalated,
this one's okay to auto-approve. And it

280
00:16:21,720 --> 00:16:24,797
really makes you realize that human
attention is going to be this incredibly

281
00:16:24,880 --> 00:16:28,717
scarce resource, right? The doing
of things now is easy. The Is this a good

282
00:16:28,800 --> 00:16:32,797
thing? Is this what I wanted? Is this
aligned with my values, with my desires?

283
00:16:32,880 --> 00:16:37,197
That is going to become the single most
important bottleneck. And so, I think

284
00:16:37,280 --> 00:16:40,837
building systems that take that into
account and really think about the human

285
00:16:40,920 --> 00:16:43,997
factor, like, that's the most
important thing to do now.

286
00:16:44,080 --> 00:16:46,837
Another human factor, security.

287
00:16:46,920 --> 00:16:49,597
Um, how would you advise people to think

288
00:16:49,680 --> 00:16:52,757
about security in this world of AI and

289
00:16:52,840 --> 00:16:55,237
heard about breaches left and right with

290
00:16:55,320 --> 00:16:59,917
Vercel recently and then And these
models are incredibly powerful

291
00:17:00,000 --> 00:17:04,197
at finding security holes. So, how how
would you recommend people here use the

292
00:17:04,280 --> 00:17:06,517
models to to find those security issues?

293
00:17:06,600 --> 00:17:09,957
Well, I think there's a couple levels
to the answer. I do think that this is I

294
00:17:10,040 --> 00:17:12,797
think that the internet has been a place

295
00:17:12,880 --> 00:17:17,157
where security has been just like a a
ratcheting important concern over time.

296
00:17:17,240 --> 00:17:20,677
You think about where it started, going
through the '90s with viruses and worms

297
00:17:20,760 --> 00:17:24,357
and malware and those things and we've
moved past that. Um I think we are also

298
00:17:24,440 --> 00:17:28,597
moving now to a much more ultimately
secure regime, but it does require kind

299
00:17:28,680 --> 00:17:30,757
of an internet-wide effort to get there.

300
00:17:30,840 --> 00:17:33,677
And so a lot of this honestly is just
again leaning into the technology,

301
00:17:33,760 --> 00:17:37,637
having these models they can scan your
code base, they can actually be used for

302
00:17:37,720 --> 00:17:40,917
end-to-end red teaming. Like there's
a lot that can be done with them and a lot

303
00:17:41,000 --> 00:17:44,277
of how we're thinking about further
models and improvements there is really

304
00:17:44,360 --> 00:17:49,117
leaning into how do we how do we
actually sort of leverage trusted access

305
00:17:49,200 --> 00:17:52,277
programs, how do we leverage
the community of people who really care

306
00:17:52,360 --> 00:17:55,997
about being defenders and making
the internet more secure. I think that's

307
00:17:56,080 --> 00:18:00,317
something where everyone has a role to
play and can participate, but the number

308
00:18:00,400 --> 00:18:04,637
one thing is just sort of recognizing
that these models are very powerful, but

309
00:18:04,720 --> 00:18:07,877
they're not magic, right? That they
are just like a part of the overall

310
00:18:07,960 --> 00:18:12,237
resilience ecosystem and I think that we
as a society and I think every company

311
00:18:12,320 --> 00:18:16,357
again really contributes to this have
something to build in terms of how do we

312
00:18:16,440 --> 00:18:18,197
how do we incorporate these in a way

313
00:18:18,280 --> 00:18:20,677
that results in more

314
00:18:20,760 --> 00:18:23,397
assurance and more

315
00:18:23,480 --> 00:18:28,197
sort of certainty on the impacts
of of whether it's a particular patch that

316
00:18:28,280 --> 00:18:31,037
you're taking, whether it's thinking
about how do you make sure that you're

317
00:18:31,120 --> 00:18:34,637
um yeah, just sort of rolling in updates
quickly as they're being released. Um so

318
00:18:34,720 --> 00:18:38,437
I think that there's a lot of work to be
done, but I have a lot of optimism for

319
00:18:38,520 --> 00:18:43,517
where this is going.
Um let's switch to speed. Seems like

320
00:18:43,600 --> 00:18:47,037
things are moving faster and faster and
faster when in the world of accelerating

321
00:18:47,120 --> 00:18:49,757
change. We were talking
about it when we when you

322
00:18:49,840 --> 00:18:52,837
uh you were walking up here around how

323
00:18:52,920 --> 00:18:57,157
how you're trying to keep up with things.
How how do you you keep up with

324
00:18:57,240 --> 00:19:00,677
all the accelerating change? How would
you recommend everybody here keep up

325
00:19:00,760 --> 00:19:02,677
with everything that's changing?

326
00:19:02,760 --> 00:19:06,557
Well, I think this is the new normal
and I think to some extent it's not really

327
00:19:06,640 --> 00:19:11,517
because of AI. I think it's just been
the trend of technology for the past two

328
00:19:11,600 --> 00:19:14,757
decades. There's more
people doing things. It's

329
00:19:14,840 --> 00:19:17,997
easier to do things than ever.
Barrier to entry goes down.

330
00:19:18,080 --> 00:19:21,797
Means it's also much more easy to build
value, right? To have great successes.

331
00:19:21,880 --> 00:19:25,357
And so, I think that really trying
to keep your ear to the ground and

332
00:19:25,440 --> 00:19:28,237
understand what's changing. And to some
extent, it always starts with the same

333
00:19:28,320 --> 00:19:29,557
thing, which is play with the technology

334
00:19:29,640 --> 00:19:32,037
yourself. Like, it's very different to

335
00:19:32,120 --> 00:19:36,917
hear AI described versus to use it. But,
the beautiful thing about AI is it's so

336
00:19:37,000 --> 00:19:40,037
intuitive. Like, that's the whole point
is that rather than have the machine be

337
00:19:40,120 --> 00:19:43,957
something you have to contort yourself to,
the machine contorts itself to you,

338
00:19:44,040 --> 00:19:47,117
right? It's doing work for you. And it
should be something where you ask it and

339
00:19:47,200 --> 00:19:48,757
does something. And so, I think that

340
00:19:48,840 --> 00:19:52,077
just really trying to just get your

341
00:19:52,160 --> 00:19:54,437
finger on the pulse of what's changing,

342
00:19:54,520 --> 00:19:56,437
what's possible, where the models lag.

343
00:19:56,520 --> 00:19:59,157
That is, I think, the core skill that is

344
00:19:59,240 --> 00:20:03,877
going to really determine a lot of the
success of of companies in the future.

345
00:20:03,960 --> 00:20:07,717
And then, on the flip side of that,
you guys have held up held back models to

346
00:20:07,800 --> 00:20:11,357
work with security agents. So, it's like
the opposite of like going as fast as

347
00:20:11,440 --> 00:20:15,397
possible. So, um you're
doing things responsibly, too.

348
00:20:15,480 --> 00:20:18,357
So, how do you kind of like think
about the balance? Because you're in a

349
00:20:18,440 --> 00:20:22,237
competitive environment, you want
to ship as quickly as possible, and yet

350
00:20:22,320 --> 00:20:24,557
you're trying to do
the right thing, as well.

351
00:20:24,640 --> 00:20:28,877
Yeah, I think at a values level,
like what OpenAI is about, like we really

352
00:20:28,960 --> 00:20:33,957
want to put the power of AI in people's
hands. Like, we believe that people can

353
00:20:34,040 --> 00:20:37,037
We want to empower people to build
the future with the tools that are being

354
00:20:37,120 --> 00:20:39,917
created. But, we need to do that in a
thoughtful way, right? That we really

355
00:20:40,000 --> 00:20:42,917
think about both sides of Here are the
benefits, here are the risks. How do you

356
00:20:43,000 --> 00:20:45,757
maximize the benefits? How do you
mitigate those risks? And I think that

357
00:20:45,840 --> 00:20:50,077
in cybersecurity and in biosecurity,
those are areas where we're very

358
00:20:50,160 --> 00:20:53,597
thoughtful. We've been building We've
been working on these kinds of both

359
00:20:53,680 --> 00:20:55,757
mitigations and trusted access programs

360
00:20:55,840 --> 00:20:58,717
for quite a long time. And that what we

361
00:20:58,800 --> 00:21:00,917
see coming is models that are going to

362
00:21:01,000 --> 00:21:03,037
be increasingly powerful and capable in

363
00:21:03,120 --> 00:21:05,757
a continuous way across all dimensions

364
00:21:05,840 --> 00:21:09,077
of capability. And the

365
00:21:09,160 --> 00:21:13,277
you know, we announced last week
the expansion of our trusted access for

366
00:21:13,360 --> 00:21:17,157
cyber program. By the way,
has anyone here applied?

367
00:21:17,240 --> 00:21:18,957
No one. Oh, I see one hand, two hands.

368
00:21:19,040 --> 00:21:21,357
Okay, more of you should apply.
It's great.

369
00:21:21,440 --> 00:21:25,477
We really need help because and it's
very important that people who are

370
00:21:25,560 --> 00:21:29,277
trustworthy and responsible and really
want to push these models are

371
00:21:29,360 --> 00:21:32,237
participating in this because that is
how that's going to pay dividends for

372
00:21:32,320 --> 00:21:34,557
everyone. We're going
to have more to announce

373
00:21:34,640 --> 00:21:39,477
over upcoming weeks on how we're
expanding the program, but and also when

374
00:21:39,560 --> 00:21:42,797
we release models to everyone kind
of the mitigations that that we have and

375
00:21:42,880 --> 00:21:46,637
how we're going to tune those to be both
to really balance, right? To really try

376
00:21:46,720 --> 00:21:50,797
to bring these capabilities as broadly
as possible, while also making sure that

377
00:21:50,880 --> 00:21:54,837
the ones that are, you know, that we're
thinking about the risks and and able to

378
00:21:54,920 --> 00:21:58,597
have some observability over them and
to ensure that this is maximally positive

379
00:21:58,680 --> 00:22:01,397
in terms of deployment. So, I think
the short answer is like it's core to our

380
00:22:01,480 --> 00:22:04,917
mission. We care a lot about the impacts
of what we're doing, not just building

381
00:22:05,000 --> 00:22:08,997
technology in isolation, but it is a whole
community, a whole world effort to

382
00:22:09,080 --> 00:22:11,237
really get to where we need to be.

383
00:22:11,320 --> 00:22:13,957
On moving up from the models to the

384
00:22:14,040 --> 00:22:15,877
application layer, which is what a lot

385
00:22:15,960 --> 00:22:21,237
of people here are building.
How do you How does OpenAI decide what

386
00:22:21,320 --> 00:22:24,357
in the application layer you're you're
going to build and what you're going to

387
00:22:24,440 --> 00:22:28,757
leave out? Well, people
have probably seen the word

388
00:22:28,840 --> 00:22:34,157
focus being applied to OpenAI quite a lot
recently, possibly for the first time

389
00:22:34,240 --> 00:22:37,747
>> in a while. And uh It's
been applied to her, too.

390
00:22:37,830 --> 00:22:41,317
>> [laughter] >> And it's
it's hard because the field of

391
00:22:41,400 --> 00:22:44,597
AI is one of opportunity, right?
It's like anything you're going you can

392
00:22:44,680 --> 00:22:46,757
imagine is going to be great. No

393
00:22:46,840 --> 00:22:51,997
question it's going to be great. And we
as a company, as a single company, no

394
00:22:52,080 --> 00:22:54,997
matter how much compute we build, no
matter how many people we have, are only

395
00:22:55,080 --> 00:22:58,757
going to be able to do so much. And so,
a lot of where we've been how we've been

396
00:22:58,840 --> 00:23:01,597
thinking about things is what is the

397
00:23:01,680 --> 00:23:03,917
sort of most focused strategy that

398
00:23:04,000 --> 00:23:07,717
covers the parts of the space?

399
00:23:07,800 --> 00:23:09,997
You know, maybe it's an 80/20 or just
like the parts of the space that we

400
00:23:10,080 --> 00:23:13,797
think we can have most impact on. And I
think there it's very clear right now

401
00:23:13,880 --> 00:23:17,877
we're going through this identity
transition. And so >> [clears throat]

402
00:23:17,960 --> 00:23:21,957
>> products that are And it's not just
about enterprise versus consumer, right?

403
00:23:22,040 --> 00:23:25,197
So it's like clear we are being very
serious about enterprise. Like we're

404
00:23:25,280 --> 00:23:28,637
selling to to big companies and and
building a whole muscle and sales motion

405
00:23:28,720 --> 00:23:33,357
there. But consumer, what consumer is
is going to change, right? It's kind of a

406
00:23:33,440 --> 00:23:37,637
very broad term that buckets in multiple
things. But that the slice of consumer

407
00:23:37,720 --> 00:23:41,557
that's about not just productivity, but
about goals, about achieving your goal,

408
00:23:41,640 --> 00:23:44,517
about even knowing what is your goal,
being able to elicit that and having an

409
00:23:44,600 --> 00:23:48,077
AI that can proactively do that. It's
all kind of the same thing. Like in the

410
00:23:48,160 --> 00:23:52,037
end we're trying to build an AGI
that you can talk to, that has all this

411
00:23:52,120 --> 00:23:54,597
context that you can use in your
personal life, your work life, that's

412
00:23:54,680 --> 00:23:58,117
trustworthy, right? That you can go
to it for advice, can give you useful

413
00:23:58,200 --> 00:24:01,997
information, maybe health information,
or maybe about finances, or

414
00:24:02,080 --> 00:24:04,757
you know, about if you're trying
to figure out what to do with your career.

415
00:24:04,840 --> 00:24:09,077
Like all these things, they all kind of
ladder into one thing. And it's meant we

416
00:24:09,160 --> 00:24:12,637
had to make some very painful decisions
about what not to do. But I think I

417
00:24:12,720 --> 00:24:16,077
would just say that that's the aperture
that we look at things through and that

418
00:24:16,160 --> 00:24:20,477
things that accrue to that singular
vision of what we want to build, you

419
00:24:20,560 --> 00:24:22,957
should expect us to pursue. Got it.

420
00:24:23,040 --> 00:24:27,197
Um Do you think we'll be coding with

421
00:24:27,280 --> 00:24:31,877
command lines and agents in in
a few years or is it going to be

422
00:24:31,960 --> 00:24:36,357
completely changed? I mean, I think
that we're in a very unnatural state right

423
00:24:36,440 --> 00:24:40,837
now for how we work.
Like we all sit behind this box and kind

424
00:24:40,920 --> 00:24:44,197
of type away and it's very clear
our bodies were not designed for this. We

425
00:24:44,280 --> 00:24:48,237
got our carpal tunnel and our, you know,
hunched shoulders and all these things.

426
00:24:48,320 --> 00:24:50,997
And I don't think we want that. I don't
think any of us wanted that. Like I

427
00:24:51,080 --> 00:24:53,837
think that we want more free time.
We want more time, but it's it's not even

428
00:24:53,920 --> 00:24:56,477
about free time necessarily, right?
It's like you want to spend more time with

429
00:24:56,560 --> 00:24:59,997
your loved ones? Yes. You want to spend
more time like talking to people and

430
00:25:00,080 --> 00:25:03,237
like coming up with like brilliant
visions or just like what you're excited

431
00:25:03,320 --> 00:25:04,997
about or just understanding yourself.

432
00:25:05,080 --> 00:25:09,597
So, it's kind of like do you do you want
to be a CEO of an organization of like

433
00:25:09,680 --> 00:25:13,437
100,000 agents? Like that actually seems
pretty good. And I think that we're all

434
00:25:13,520 --> 00:25:17,557
going to be able to get so much more done,
but the mechanics of it are going

435
00:25:17,640 --> 00:25:20,277
to feel as different as like going from

436
00:25:20,360 --> 00:25:22,317
having to write out things with, you

437
00:25:22,400 --> 00:25:24,957
know, by hand with a quill or something

438
00:25:25,040 --> 00:25:29,437
to being able to you know, just send
a text message and have people go and and,

439
00:25:29,520 --> 00:25:32,717
you know, working
on your behalf on your goals.

440
00:25:32,800 --> 00:25:34,317
All right. We talked about compute. We

441
00:25:34,400 --> 00:25:38,397
talked about model and security and

442
00:25:38,480 --> 00:25:42,557
agents and app layer.
Let's talk about frontier.

443
00:25:42,640 --> 00:25:44,477
When when are the models going to be

444
00:25:44,560 --> 00:25:47,597
good enough to push the frontiers of

445
00:25:47,680 --> 00:25:50,757
science, physical AI? Seems like we had

446
00:25:50,840 --> 00:25:54,797
Jen Fan here. It seems like LLMs have
been a great scaling law for digital

447
00:25:54,880 --> 00:25:57,637
intelligence. It hasn't been as strong

448
00:25:57,720 --> 00:26:02,317
for robotics,
for physical intelligence, for

449
00:26:02,400 --> 00:26:05,837
aspects of biology
and science where the problems

450
00:26:05,920 --> 00:26:10,157
are probably a lot harder to verify
or takes a long time to verify.

451
00:26:10,240 --> 00:26:14,117
Well, how are you keeping track
of science and and physical AI in in the

452
00:26:14,200 --> 00:26:18,357
world? Well, well, science is one domain
that we're really leaning into and we

453
00:26:18,440 --> 00:26:21,917
see line of sight to really
incredible progress.

454
00:26:22,000 --> 00:26:24,437
And we're starting to have some signs
of life and I think it's always important

455
00:26:24,520 --> 00:26:28,437
to ground in what is happening today
when trying to predict what will happen

456
00:26:28,520 --> 00:26:30,477
6 months a year from now. So, for

457
00:26:30,560 --> 00:26:34,157
example, we had a physics result where

458
00:26:34,240 --> 00:26:38,397
our AI came up with this very beautiful
formula that physicists who've been

459
00:26:38,480 --> 00:26:41,117
working on this for quite some time
thought was totally impossible. Thought

460
00:26:41,200 --> 00:26:43,477
it was like maybe an unsolvable problem.

461
00:26:43,560 --> 00:26:46,837
And like it's pretty significant, right?

462
00:26:46,920 --> 00:26:51,077
It's like real serious physicists
who who who really view this as a step

463
00:26:51,160 --> 00:26:55,517
towards really being able to get to um
to to some sort of answer for quantum

464
00:26:55,600 --> 00:26:58,957
gravity and all these things. It's
not there, but it's a step. That's much

465
00:26:59,040 --> 00:27:02,317
bigger than where we were just a couple
months ago. And so you it makes you

466
00:27:02,400 --> 00:27:07,157
really wonder a year from now, like how
far will we have traveled? Now, things

467
00:27:07,240 --> 00:27:10,397
like biology that they are different
from physics and math, right? That they

468
00:27:10,480 --> 00:27:14,637
are you got to leave your beautiful
simulated world and you know, deal with

469
00:27:14,720 --> 00:27:18,997
messy reality. But I think we've been
learning how to deal with messy reality

470
00:27:19,080 --> 00:27:22,757
in other domains. Software engineering
is a perfect example where we've really

471
00:27:22,840 --> 00:27:27,037
realized that just building
the thing that solves competition

472
00:27:27,120 --> 00:27:30,677
you know, programming competitions,
like that's not enough. Like you need

473
00:27:30,760 --> 00:27:33,877
something that's seen real-world
messy codebases, humans interrupting it

474
00:27:33,960 --> 00:27:37,437
different ways, like this adversarial
banging at it. And I so I think that

475
00:27:37,520 --> 00:27:40,597
that on science, I expect we're going
to see a real renaissance. You know, maybe

476
00:27:40,680 --> 00:27:42,237
we'll see some big results this year.

477
00:27:42,320 --> 00:27:46,237
Next year I think it's going
to be a totally wild wild time.

478
00:27:46,320 --> 00:27:50,477
We live in interesting times.
Um I I promised that I get you out on time

479
00:27:50,560 --> 00:27:53,957
because you're a busy man. Um before
we let you leave, we got 1 minute on the

480
00:27:54,040 --> 00:27:58,557
shot clock. What since you have
no time, but soon you will have

481
00:27:58,640 --> 00:28:00,877
lots of time, um

482
00:28:00,960 --> 00:28:04,477
what do you and Anna do for fun? Uh fun?

483
00:28:04,560 --> 00:28:09,557
I mean, same as anyone. Like like to watch
movies, go on hikes, that those kinds of

484
00:28:09,640 --> 00:28:12,757
things. Um you know,
not as much time for it as as

485
00:28:12,840 --> 00:28:16,757
maybe we'll we'll hopefully have
post-AGI. Um but you got to kind of

486
00:28:16,840 --> 00:28:18,997
enjoy the the ride along the way.

487
00:28:19,080 --> 00:28:21,627
Thank you, Greg, for joining us.
Thank you, everyone.

488
00:28:21,710 --> 00:28:23,730
>> [applause]