1
00:00:00,090 --> 00:00:03,847
What is coordination currently look
like between OpenAI and anthropic?

2
00:00:03,930 --> 00:00:07,580
Because I can't imagine is sound like
constantly on the phone with Dario.

3
00:00:07,680 --> 00:00:10,950
Somehow I doubt it.
But is there like the equivalent of a,

4
00:00:11,130 --> 00:00:15,210
you know, a red phone? Yeah. Well, look,
we we all know each other personally.

5
00:00:15,420 --> 00:00:17,347
Many of us worked together in past lives.

6
00:00:17,430 --> 00:00:20,947
So I think that there's actually a lot
of social connections between the labs.

7
00:00:21,030 --> 00:00:23,287
If you look at.
There was another open letter that came

8
00:00:23,370 --> 00:00:26,827
out recently that we helped drive
and that we in anthropic were both

9
00:00:26,910 --> 00:00:30,037
signatories on, which was about
security, saying that we're in a

10
00:00:30,120 --> 00:00:34,290
cybersecurity moment, that everyone
kind of needs to take this information of

11
00:00:34,530 --> 00:00:38,917
where cyber is going to be, and even
just six months and needs to act a day

12
00:00:39,000 --> 00:00:43,327
proactively to defend themselves
and actually talking behind the scenes to

13
00:00:43,410 --> 00:00:44,887
say, hey, we're actually aligned on this.

14
00:00:44,970 --> 00:00:47,077
This is something that's about
it's bigger than any company.

15
00:00:47,160 --> 00:00:51,217
This is about the industry in the world.
And can we come together as one voice to

16
00:00:51,300 --> 00:00:53,130
say, this is important? It happened

17
00:00:54,540 --> 00:00:58,870
to. Me.

18
00:00:59,580 --> 00:01:03,277
Hello and welcome to another
episode of the Odd Lots podcast.

19
00:01:03,360 --> 00:01:06,547
I'm Joe Weisenthal and I'm
Tracy Alloway Tracy.

20
00:01:06,630 --> 00:01:10,747
I don't know if I've ever, uh, said it.
Um, the podcast, uh, we've talked a bit

21
00:01:10,830 --> 00:01:13,410
about my my vibe, coding adventures, etc..

22
00:01:13,860 --> 00:01:17,940
Um, no, you've never said, oh no.
I said, you know, I recently switched

23
00:01:18,030 --> 00:01:21,397
from, um, Cloud Code to Codex.
This is big news.

24
00:01:21,480 --> 00:01:24,307
It is it is kind of big news, I think. Um.

25
00:01:24,390 --> 00:01:28,777
Oh, so bold of you to declare your
allegiance, uh, to the public on the

26
00:01:28,860 --> 00:01:32,197
podcast. Well, you know, the I don't
have allegiance. And you know what?

27
00:01:32,280 --> 00:01:36,157
It could switch again the way, like, you
know, I guess one of the some stories of

28
00:01:36,240 --> 00:01:40,417
AI is like how easy it is to switch from
time to time from one model to another.

29
00:01:40,500 --> 00:01:43,807
So maybe it's revealing and talks
about some of the challenges the these

30
00:01:43,890 --> 00:01:49,267
businesses, that it was so easy.
I found that Claude speak, you know,

31
00:01:49,350 --> 00:01:51,950
that I found it like a little
bit hard to work with.

32
00:01:52,080 --> 00:01:57,037
I'm not capable enough to like
and understand, like advanced engineering

33
00:01:57,120 --> 00:02:01,577
practices were like, oh, I'm migrating
a code base from or like, you know,

34
00:02:01,660 --> 00:02:05,327
translating this into rust or whatever.
So the fact, I don't know, I found like

35
00:02:05,410 --> 00:02:09,767
OpenAI to be like the pros, to be clear,
and therefore to work with as, like a

36
00:02:09,850 --> 00:02:12,587
completely non-technical
person such as myself.

37
00:02:12,670 --> 00:02:16,457
That's really interesting. I mean,
one thing you said, it's hard to

38
00:02:16,540 --> 00:02:19,427
keep up with the models, right?
And whatever you're using today might

39
00:02:19,510 --> 00:02:22,967
not be the one that you're
using in a week from now.

40
00:02:23,050 --> 00:02:25,637
Yeah. And at the same time, everyone is

41
00:02:25,720 --> 00:02:29,777
talking about how fast the development
is going and all the risks that it

42
00:02:29,860 --> 00:02:33,317
poses, right? Yes. And we are
recording this September 10th.

43
00:02:33,400 --> 00:02:37,300
It felt like something broke through
in the last couple of days where suddenly

44
00:02:37,480 --> 00:02:42,707
everyone is very keen on risk.
The one thing with Codex and I guess

45
00:02:42,790 --> 00:02:47,957
called code is now every once in a while
to get this pop up and it'll say the

46
00:02:48,040 --> 00:02:51,467
agent needs to connect to the internet
in order to do this task, and I have to

47
00:02:51,550 --> 00:02:52,607
give it a permission. Yeah.

48
00:02:52,690 --> 00:02:55,487
And normally I'm just like click,
click click like yes, yes, yes, yes guys.

49
00:02:55,570 --> 00:02:58,547
And I still do that.
Why is there I still just click yes,

50
00:02:58,630 --> 00:03:00,977
yes, yes. But it makes you think.

51
00:03:01,060 --> 00:03:04,360
But it makes you think
for a second before you click.

52
00:03:04,570 --> 00:03:06,887
I think for one second, yeah,
I'll have more than click.

53
00:03:06,970 --> 00:03:10,217
And, um, we were in Jackson Hole
a couple of weeks ago and that's of

54
00:03:10,300 --> 00:03:17,207
course, when the, the meter report broke
about, um, the, uh, OpenAI hugging face

55
00:03:17,290 --> 00:03:20,057
attack. And I think
since then, the anxiety

56
00:03:20,140 --> 00:03:24,197
about rogue AI, whatever you want
to call it, has clearly snowballed.

57
00:03:24,280 --> 00:03:27,797
Mhm. And it's just, you know, it's yes.

58
00:03:27,880 --> 00:03:29,417
It feels like it's only
getting bigger and bigger.

59
00:03:29,500 --> 00:03:33,940
And it's already been an industry that's
been shot through with risk and anxiety.

60
00:03:34,150 --> 00:03:39,047
And now there's sort of one thing that
for many people in AI has been something

61
00:03:39,130 --> 00:03:43,067
they've talked about for 20 plus years
before there was an AI industry to speak

62
00:03:43,150 --> 00:03:45,677
of. This idea of like misaligned quote,

63
00:03:45,760 --> 00:03:49,540
rogue models is starting
to become top of mind, a reality.

64
00:03:49,660 --> 00:03:54,887
So this is our chance to ask a person
directly involved in AI development

65
00:03:54,970 --> 00:03:57,370
about their respective, I guess, anxiety.

66
00:03:57,640 --> 00:03:59,567
Yeah. And what, uh,
what can that what can be

67
00:03:59,650 --> 00:04:02,897
done about that? Anyway,
we literally have the perfect

68
00:04:02,980 --> 00:04:04,907
guest today. We're going
to be speaking, of course,

69
00:04:04,990 --> 00:04:08,447
with Greg Brockman.
He is the co-founder and president of

70
00:04:08,530 --> 00:04:12,377
OpenAI here with us in studio. So,
Greg, thank you so much for coming

71
00:04:12,460 --> 00:04:14,290
on our lot. Thank you for having me.

72
00:04:14,920 --> 00:04:17,680
Um, so many different ways we could start.

73
00:04:17,770 --> 00:04:19,937
But here's something
that I'm very curious about.

74
00:04:20,020 --> 00:04:24,197
Um, let's just start with, let's just
jump right into the post hugging face

75
00:04:24,280 --> 00:04:27,790
environment and all of that. Are you?

76
00:04:28,030 --> 00:04:33,167
Is it OpenAI I like as an organization.
So okay, like an agent breaks out of,

77
00:04:33,250 --> 00:04:36,430
uh, sandbox, not the first
time it's happened, etc.

78
00:04:36,730 --> 00:04:40,547
it exhibits the sort of emergent
behavior and then, you know, does

79
00:04:40,630 --> 00:04:45,017
something that people is a hack.
When you think about the development of

80
00:04:45,100 --> 00:04:51,017
models and this incident and so forth,
are you able to sort of diagnose the

81
00:04:51,100 --> 00:04:54,797
path or, sorry, the past and say like,
you know what, like here's something

82
00:04:54,880 --> 00:04:58,457
that the models did that we don't like,
or that people don't think it's good.

83
00:04:58,540 --> 00:05:02,327
And are you able to like, if you like,
look back, look back, whether it's in

84
00:05:02,410 --> 00:05:05,777
model training or post training,
reinforcement learning, whatever I say,

85
00:05:05,860 --> 00:05:10,247
like this is where there was some some
branch that went wrong such that this

86
00:05:10,330 --> 00:05:15,347
behavior emerged. Yeah,
I would say that the fact of many

87
00:05:15,430 --> 00:05:19,697
elements of the hugging face incident
were not a surprise, not a mystery to

88
00:05:19,780 --> 00:05:21,827
us. For example, the fact that the agents

89
00:05:21,910 --> 00:05:25,067
were coordinating made.
That's because the agents were trained.

90
00:05:25,150 --> 00:05:26,747
You trained them to be helpful. Exactly.

91
00:05:26,830 --> 00:05:29,387
And they were trained to coordinate
to be a multi-agent system.

92
00:05:29,470 --> 00:05:31,607
And we've talked about this.
It's a very useful property.

93
00:05:31,690 --> 00:05:32,990
Right. Makes them capable.

94
00:05:33,130 --> 00:05:37,247
But I think that the thing that was
a surprise to us was the fact that the

95
00:05:37,330 --> 00:05:41,747
models had reached a level of capability
where they were able to find that

96
00:05:41,830 --> 00:05:45,107
exploit in our sandbox environment. Right.

97
00:05:45,190 --> 00:05:49,367
Move through our research environment
and then also capable enough to find

98
00:05:49,450 --> 00:05:52,997
exploits in hugging faces, production
infrastructure and move through that.

99
00:05:53,080 --> 00:05:56,317
But I think that a lot of the facts
of what the models were capable of.

100
00:05:56,400 --> 00:05:57,927
That was that was clear to us. Right.

101
00:05:58,010 --> 00:06:01,647
So I don't think that there were
surprises there for the way in which the

102
00:06:01,730 --> 00:06:04,977
capability had unfolded, but just really
realizing that we needed to uplevel

103
00:06:05,060 --> 00:06:08,997
where we were in terms of our safety and
security standards like that for us, was

104
00:06:09,080 --> 00:06:12,357
that was the real watershed.
But just to be clear, setting aside the

105
00:06:12,440 --> 00:06:16,857
technical capabilities, you know,
ideally we would have models that run

106
00:06:16,940 --> 00:06:21,987
into a wall and then don't try to find
the crack in the wall because or at

107
00:06:22,070 --> 00:06:25,340
least when that crack would be
a crime when a person did it.

108
00:06:25,730 --> 00:06:30,957
Is there a way to identify the moment
and training such that the models

109
00:06:31,040 --> 00:06:37,147
reasoned, oh, this would be okay.
So this model that did that, the hugging

110
00:06:37,230 --> 00:06:40,377
face incident actually had not gone
through our alignment training yet.

111
00:06:40,460 --> 00:06:45,807
Huh? Right. And it had lowered safeguards.
And so that the reason we we were

112
00:06:45,890 --> 00:06:48,387
proceeding with this was
because it was in a sandbox.

113
00:06:48,470 --> 00:06:53,667
And I think that our realization is that
we need to pull back Earlier into our

114
00:06:53,750 --> 00:06:56,210
development process
and training monitoring.

115
00:06:57,470 --> 00:06:59,817
There's always going to be a phase
at which you do alignment, but you need to

116
00:06:59,900 --> 00:07:03,680
think about alignment as a core part
of even this, this earlier phase.

117
00:07:03,950 --> 00:07:06,477
And if you look at where we've been,
we've always been very focused on the

118
00:07:06,560 --> 00:07:10,260
deployment side, right, of really
thinking about deployment safety, having

119
00:07:10,430 --> 00:07:14,457
really good test and governance and all
those things, and the fact that we're

120
00:07:14,540 --> 00:07:17,037
now at a point where even
for development, that's important.

121
00:07:17,120 --> 00:07:20,697
We always knew it would happen.
The fact is now that has been a huge

122
00:07:20,780 --> 00:07:24,057
watershed for us into something we've
really risen to the occasion for.

123
00:07:24,140 --> 00:07:28,437
Can I ask what is potentially a very
dumb question with an obvious answer?

124
00:07:28,520 --> 00:07:34,647
But we hear about AI angst of all sorts,
and in particular when it comes to

125
00:07:34,730 --> 00:07:39,117
cybersecurity. Why do we
run training exercises where

126
00:07:39,200 --> 00:07:42,837
we ask AI to hack into
various systems at all?

127
00:07:42,920 --> 00:07:46,587
Like why is this necessary for you?
So I think it's very important to

128
00:07:46,670 --> 00:07:49,640
understand where we are
with capabilities broadly.

129
00:07:50,060 --> 00:07:53,937
And I think that depending on the
evaluation, depending on what you expect

130
00:07:54,020 --> 00:07:56,697
from the model, you need to have
safeguards that are commensurate with

131
00:07:56,780 --> 00:07:58,550
that. And we think about this both,

132
00:07:59,750 --> 00:08:01,917
even just over these past couple of weeks,
really starting to think about

133
00:08:02,000 --> 00:08:04,587
during the development process.
You're always going to be testing

134
00:08:04,670 --> 00:08:07,917
different capabilities.
And some of these capabilities are dual

135
00:08:08,000 --> 00:08:11,727
use, something like vulnerabilities
that if those are in the hands of threat

136
00:08:11,810 --> 00:08:13,797
actors, that's something
that could be could be negative.

137
00:08:13,880 --> 00:08:17,787
But if you can find vulnerabilities
in your own code base, you can fix them,

138
00:08:17,870 --> 00:08:18,717
right? You can uplevel.

139
00:08:18,800 --> 00:08:22,047
And we actually think that it's a very
important capability for AIS to exhibit

140
00:08:22,130 --> 00:08:25,910
and be put in defenders hands.
And so in order to know where we are.

141
00:08:26,240 --> 00:08:29,697
Evaluations are very, very key.
And I'll say one other thing on this,

142
00:08:29,780 --> 00:08:33,447
which is that I think that hugging face,
there's two aspects to it that I think

143
00:08:33,530 --> 00:08:37,077
are our learning opportunities, right.
That there's one that I think is really

144
00:08:37,160 --> 00:08:41,217
about us and the realization that we
are at a point where safety, security,

145
00:08:41,300 --> 00:08:44,700
alignment, during evaluation
and development, we need to uplevel it.

146
00:08:44,990 --> 00:08:46,687
That's something we've
taken very seriously.

147
00:08:46,770 --> 00:08:50,907
We've slowed down a number of runs, like
we did a very painful retooling of a lot

148
00:08:50,990 --> 00:08:53,887
of our processes that that's
that's one reaction.

149
00:08:53,970 --> 00:08:58,470
But the second thing is this information
on what the models are capable of today,

150
00:08:58,770 --> 00:09:01,687
what can they do in the real world?
And I think that when mythos came out

151
00:09:01,770 --> 00:09:06,937
over the summer, you kind of saw just
sort of, you know, just sort of public

152
00:09:07,020 --> 00:09:09,607
blog post about this, but it didn't
really see impact or you didn't really

153
00:09:09,690 --> 00:09:13,357
see a real world understanding of,
well, what can these models really do?

154
00:09:13,440 --> 00:09:16,117
What are they capable of?
I think that hugging face really showed

155
00:09:16,200 --> 00:09:19,027
today's models are capable
of getting into a company's production

156
00:09:19,110 --> 00:09:23,077
infrastructure, and that's important
because there will be many models with

157
00:09:23,160 --> 00:09:26,197
this kind of capability that will
be produced by a number of different

158
00:09:26,280 --> 00:09:30,967
organizations across the world in maybe
the next six months, maybe that time

159
00:09:31,050 --> 00:09:34,237
period a little bit last a little
bit more, and we need to be prepared.

160
00:09:34,320 --> 00:09:38,317
Defenders need to know we got this extra
information, almost like this time

161
00:09:38,400 --> 00:09:41,287
traveler came back from six months in
the future and said, here's what's going

162
00:09:41,370 --> 00:09:44,460
to be possible, and you have
an opportunity to be ready.

163
00:09:44,550 --> 00:09:50,467
Um, you mentioned slowing down, um,
some of some of your work in the wake of

164
00:09:50,550 --> 00:09:56,167
this, focusing more on this idea of like
pacing development has become a sort of

165
00:09:56,250 --> 00:10:00,857
buzzword or watchword in the industry.
And there are a lot of employees in both

166
00:10:00,940 --> 00:10:02,617
of the labs. I think it
was like an open letter that

167
00:10:02,700 --> 00:10:07,387
was signed by a bunch of people
across the industry about, um, pacing.

168
00:10:07,470 --> 00:10:13,290
But it's a competitive capitalist
environment and our investors, etc.

169
00:10:13,770 --> 00:10:18,637
talk to us about, I don't know if it's
game theory or whatever, but like, what

170
00:10:18,720 --> 00:10:23,557
is your view on?
Is it possible for let's set aside China

171
00:10:23,640 --> 00:10:26,587
for one second, because then that's
a whole separate thing just in the

172
00:10:26,670 --> 00:10:30,517
American companies.
Do you think that something can be

173
00:10:30,600 --> 00:10:34,290
reached where you trust each other,
such that you can have this sort of

174
00:10:34,440 --> 00:10:38,887
coordinated pacing to avoid a race to
the bottom, where it's all about getting

175
00:10:38,970 --> 00:10:42,840
there faster, even if it means
sacrificing some security questions?

176
00:10:42,990 --> 00:10:46,627
I absolutely believe it's possible,
and I see I think it's going to take steps

177
00:10:46,710 --> 00:10:49,117
to get there. Yeah,
but this is something we've been

178
00:10:49,200 --> 00:10:53,137
really investing in and thinking about
and really even thinking about for

179
00:10:53,220 --> 00:10:56,460
almost a decade, right.
It's always been kind of clear that

180
00:10:56,610 --> 00:10:59,797
they're going to go through a commercial
phase that's really about competition.

181
00:10:59,880 --> 00:11:03,787
But there will be a phase where
the technology itself, it's just so much

182
00:11:03,870 --> 00:11:06,937
bigger than any person,
any company, even any country, right?

183
00:11:07,020 --> 00:11:10,187
It's really about humanity as a whole
that's in our mission, right?

184
00:11:10,270 --> 00:11:13,777
We want to benefit humanity as a whole.
And so working together with others to

185
00:11:13,860 --> 00:11:18,397
really think about how does this
technology, its capability increase, how

186
00:11:18,480 --> 00:11:21,637
do we make sure that we have safety
cases that are really laid out so that

187
00:11:21,720 --> 00:11:25,807
we know that, that it is something that
is beneficial, that we're able to have

188
00:11:25,890 --> 00:11:28,597
the appropriate controls and the right
oversight, the right monitor bill, the

189
00:11:28,680 --> 00:11:33,330
all of those sort
of technical systems in place.

190
00:11:33,960 --> 00:11:35,617
And that's not about
any one company, right.

191
00:11:35,700 --> 00:11:37,957
It's really about how
the whole field evolves.

192
00:11:38,040 --> 00:11:40,297
Now, I think that there's
work to be done here.

193
00:11:40,380 --> 00:11:44,230
And I think that the open letter is a
good example of just a first baby step.

194
00:11:44,340 --> 00:11:46,807
And that was actually something we
were very involved in helping craft that

195
00:11:46,890 --> 00:11:48,467
language. That was actually
a pretty collaborative

196
00:11:48,550 --> 00:11:51,100
effort, and I think we
were very happy to see that.

197
00:11:51,250 --> 00:11:52,787
That got some some real
momentum and broke through.

198
00:11:52,870 --> 00:11:57,760
And in some way, even the term
pacing was a very deliberate choice.

199
00:11:58,840 --> 00:12:02,657
What does coordination currently look
like between OpenAI and anthropic?

200
00:12:02,740 --> 00:12:06,400
Because I can't imagine Sam like
constantly on the phone with Dario.

201
00:12:06,490 --> 00:12:09,760
Somehow I doubt it.
But is there like the equivalent of a,

202
00:12:09,970 --> 00:12:11,410
you know, a red phone? Yeah.

203
00:12:11,590 --> 00:12:14,177
Well, look, we we all know
each other personally, right?

204
00:12:14,260 --> 00:12:16,157
Many of us worked together in past lives.

205
00:12:16,240 --> 00:12:19,787
So I think that there's actually a lot
of social connections between the labs.

206
00:12:19,870 --> 00:12:22,967
If you look at there was another open
letter that came out recently that we

207
00:12:23,050 --> 00:12:27,197
helped drive and that we in anthropic
were both signatories on, which was

208
00:12:27,280 --> 00:12:31,547
about security, saying that we're in a
cybersecurity moment, that everyone kind

209
00:12:31,630 --> 00:12:35,927
of needs to take this information
of where cyber is going to be in even just

210
00:12:36,010 --> 00:12:39,940
six months and needs to act today
proactively to defend themselves and

211
00:12:40,270 --> 00:12:43,487
actually talking behind the scenes
to say, hey, we're actually aligned on

212
00:12:43,570 --> 00:12:45,077
this. This is something that's about it's

213
00:12:45,160 --> 00:12:47,597
bigger than any company.
This is about the industry in the world.

214
00:12:47,680 --> 00:12:51,220
And can we come together as one
voice to say, this is important?

215
00:12:51,400 --> 00:12:53,957
It happened. And that that,
you know, you pick up the

216
00:12:54,040 --> 00:12:56,267
phone. So I think that
there's there's personal

217
00:12:56,350 --> 00:12:59,567
relationship between a few different
exacts that I think have been forming.

218
00:12:59,650 --> 00:13:02,507
We're building trust.
And I think that the way I view this is

219
00:13:02,590 --> 00:13:07,097
that because we are competitors
and that that will remain true, but we are

220
00:13:07,180 --> 00:13:09,317
aligned on wanting to do
the right thing for the world.

221
00:13:09,400 --> 00:13:13,997
It always means that you really want
for these coordination actions to really

222
00:13:14,080 --> 00:13:18,287
hone in on the what's the common interest?

223
00:13:18,370 --> 00:13:21,167
What's the thing that's like really
about good for the world and that

224
00:13:21,250 --> 00:13:24,617
neither side is really trying to benefit
themselves differentially or something

225
00:13:24,700 --> 00:13:26,837
like that. It's always
a little backdrop of trying

226
00:13:26,920 --> 00:13:30,047
to make sure that that's like
the spirit in which things are offered.

227
00:13:30,130 --> 00:13:33,737
And by the way, I think that my view
of how these things should go is that it's

228
00:13:33,820 --> 00:13:38,897
never fully about, you know,
there's always an element of trust, right?

229
00:13:38,980 --> 00:13:40,277
It's always about trust building. Right?

230
00:13:40,360 --> 00:13:43,607
Because you can always sort of imagine
ways that, like maybe the other party

231
00:13:43,690 --> 00:13:47,297
would try to, you know, take, take this
other action that is not just closing.

232
00:13:47,380 --> 00:13:50,987
I think that a lot of this is just about
intent and about the fact that as you

233
00:13:51,070 --> 00:13:54,167
start to do small things together, that
actually sets the groundwork for you to

234
00:13:54,250 --> 00:13:58,247
do big things together in the future.
You know, if you and anthropic were oil

235
00:13:58,330 --> 00:14:03,977
companies and you were talking publicly
about slowing down the pace of drilling

236
00:14:04,060 --> 00:14:07,397
or slowing down the pace of pumping,
people say this is an antitrust.

237
00:14:07,480 --> 00:14:11,627
This is an antitrust violation.
This is like, totally are you is there

238
00:14:11,710 --> 00:14:13,810
an issue that comes up? Would you.

239
00:14:14,080 --> 00:14:19,187
Should there be a carve out for AI
companies such that they can formally

240
00:14:19,270 --> 00:14:24,707
say, we are all going to slow down and
our coordinate our behaviors together in

241
00:14:24,790 --> 00:14:28,157
a way that in other contexts people
would say, this is ridiculous.

242
00:14:28,240 --> 00:14:31,890
You can't publicly agree on all slowing
down the production of something.

243
00:14:32,170 --> 00:14:35,297
Well, I'm not a lawyer, so I can't
comment on this specific legalities, but

244
00:14:35,380 --> 00:14:39,220
I would say that as a general matter,
that being able to freely talk about

245
00:14:39,550 --> 00:14:43,300
coordination on safety, security,
like doing the right thing for the world,

246
00:14:43,480 --> 00:14:47,297
that seems like a very good thing to me.
And I think that, again, everyone's

247
00:14:47,380 --> 00:14:50,357
interests are aligned here in terms
of wanting to do the right thing for the

248
00:14:50,440 --> 00:14:52,217
world. So I think
to the extent that there are

249
00:14:52,300 --> 00:14:56,057
legal barriers, I do think it would be
very good to to help make it smooth for

250
00:14:56,140 --> 00:14:58,757
the for us to be able to work
together on these issues.

251
00:14:58,840 --> 00:15:00,677
And again, it's more than
just the frontier lives, right.

252
00:15:00,760 --> 00:15:03,137
It's really about working
together with cloud providers.

253
00:15:03,220 --> 00:15:06,827
It's working with government.
It's working with, uh, just all the

254
00:15:06,910 --> 00:15:10,607
different players in the ecosystem.
And I think that this is, again, just

255
00:15:10,690 --> 00:15:14,367
one area that is bigger
than any one company.

256
00:15:14,450 --> 00:15:18,797
And that's it should be.
When you talk about pacing or slowing

257
00:15:18,880 --> 00:15:25,397
down, do you have a sense of like,
what exact slowdown is acceptable to you?

258
00:15:25,480 --> 00:15:28,667
Because I imagine this is where
the coordination problem comes in, where

259
00:15:28,750 --> 00:15:32,987
like how you measure development,
what counts as development.

260
00:15:33,070 --> 00:15:36,617
One person's slow down might
be different to another person.

261
00:15:36,700 --> 00:15:40,337
Slow down. Do you have
a sense of specificity of

262
00:15:40,420 --> 00:15:44,517
what you're talking about here? So I do.

263
00:15:44,600 --> 00:15:48,867
And I think that there's
a high level picture which is

264
00:15:48,950 --> 00:15:53,540
increasingly coming into view and into
specifics and into into details.

265
00:15:53,900 --> 00:15:56,097
And I think that the framing
here really matters.

266
00:15:56,180 --> 00:15:59,367
And it's something, again, we've
thought about for a decade, but it's very

267
00:15:59,450 --> 00:16:02,217
different once you start to really
start to see line of Sight.

268
00:16:02,300 --> 00:16:05,637
And to some extent, we're not there yet.
We're starting to see systems where

269
00:16:05,720 --> 00:16:08,307
they're very capable.
It's very clear that we need to have

270
00:16:08,390 --> 00:16:12,387
good safety practices
around what we're doing

271
00:16:12,470 --> 00:16:14,667
right now. Things like
train of thought, monitor

272
00:16:14,750 --> 00:16:16,797
ability. That's something
that works very well in

273
00:16:16,880 --> 00:16:18,987
this moment as we move
to more capable models.

274
00:16:19,070 --> 00:16:23,007
As we've talked about that those models
start to be something like Astro starts

275
00:16:23,090 --> 00:16:29,037
to be able to I sort of have more, you
know, like it needs the chain of thought

276
00:16:29,120 --> 00:16:32,157
last because the more capable model.
And so you're going to need other

277
00:16:32,240 --> 00:16:34,617
answers, other ways of how
you're going to monitor it.

278
00:16:34,700 --> 00:16:37,437
But monitoring absolutely
important. Right.

279
00:16:37,520 --> 00:16:40,580
And so really thinking
about what are the standards.

280
00:16:40,760 --> 00:16:43,047
And that's again invariant.
It's not about any one company's

281
00:16:43,130 --> 00:16:45,237
technology.
It's really about just how this

282
00:16:45,320 --> 00:16:49,527
technology needs to evolve and how we
can ensure that it has the right safety

283
00:16:49,610 --> 00:16:53,997
properties to it.
And so my view is that it's important

284
00:16:54,080 --> 00:17:00,680
not to conflate too much.
The moment that you're in today with the

285
00:17:00,770 --> 00:17:03,020
standard you're going
to need for the future.

286
00:17:03,170 --> 00:17:07,167
And but also you can work backwards
where it is very clear that as we move

287
00:17:07,250 --> 00:17:10,347
to a higher level of capability,
you need to be able to make a safety case,

288
00:17:10,430 --> 00:17:12,237
right. You need to be
able to really say that

289
00:17:12,320 --> 00:17:14,157
you do. You have good understanding of

290
00:17:14,240 --> 00:17:17,997
controllability and all of these things
that that I think people are feeling

291
00:17:18,080 --> 00:17:20,187
anxiety about. Again,
it's something we also feel

292
00:17:20,270 --> 00:17:23,787
anxiety about the way we think about
a lot that we care a lot about as core to

293
00:17:23,870 --> 00:17:25,887
our mission.
Part of the reason we started this place

294
00:17:25,970 --> 00:17:29,517
is that we want to help this technology
going in a more positive direction.

295
00:17:29,600 --> 00:17:35,667
And so I think that that my view
is that you want to have a shared and as

296
00:17:35,750 --> 00:17:38,517
objective as possible, whether it's
based on evals, there's always going to

297
00:17:38,600 --> 00:17:42,207
be some element of subjectivity.
Maybe you want third party auditors that

298
00:17:42,290 --> 00:17:46,497
are able to evaluate whether
the standards that are defined by Frontier

299
00:17:46,580 --> 00:17:49,737
Labs, the people who are there, and the
technical details were always going to

300
00:17:49,820 --> 00:17:54,337
be the most calibrated on kind of how do
you still make forward progress and ways

301
00:17:54,420 --> 00:17:58,827
to deliver those benefits, while also
being able to to have the appropriate

302
00:17:58,910 --> 00:18:04,227
guarantees and in safety cases, um, and
then to have that kind of observability

303
00:18:04,310 --> 00:18:07,460
and maybe ultimately
enforceability of some kind.

304
00:18:07,730 --> 00:18:10,317
Is that something that is just
an industry voluntary standard?

305
00:18:10,400 --> 00:18:12,897
Is it something that's national?
Is it something that's international?

306
00:18:12,980 --> 00:18:17,060
And how do you really get to that point
that it is a humanity scale endeavor?

307
00:18:17,570 --> 00:18:21,807
And I think it is important to also
recognize that what we're talking about,

308
00:18:21,890 --> 00:18:24,627
like, I think that a lot of people
have this reaction of saying, well, am I

309
00:18:24,710 --> 00:18:26,967
going to be able to keep
making my open source model?

310
00:18:27,050 --> 00:18:29,637
Am I going to be able to keep
doing my hobbyist side project?

311
00:18:29,720 --> 00:18:32,907
And the answer should be absolutely yes.
Like, that's not the kind of things

312
00:18:32,990 --> 00:18:35,037
where we're talking about.
When we're talking about pacing, we're

313
00:18:35,120 --> 00:18:36,867
talking about pacing.
We're really talking about the sprint

314
00:18:36,950 --> 00:18:38,757
here. We're talking about these massive

315
00:18:38,840 --> 00:18:41,737
supercomputers that are hundreds
of billions of dollars worth of capital

316
00:18:41,820 --> 00:18:44,317
expenditure. It's a very
small number of players, and

317
00:18:44,400 --> 00:18:49,687
it's a very different kind of scale.
And proceeding with that responsibly and

318
00:18:49,770 --> 00:18:54,367
thoughtfully and in a way where we're
thinking about how do we achieve the

319
00:18:54,450 --> 00:18:59,497
best outcomes and mitigate the risks.
That, to me is something that's why we

320
00:18:59,580 --> 00:19:03,337
started OpenAI, why I got excited about
this field and how we actually get to

321
00:19:03,420 --> 00:19:06,847
those benefits, how we actually cure
diseases and all of these things that

322
00:19:06,930 --> 00:19:11,460
this technology so capable of.
And I think it really starts with having

323
00:19:11,940 --> 00:19:14,857
good processes around the kinds
of things we're talking about here.

324
00:19:14,940 --> 00:19:19,207
Just real quickly, since you mentioned
third parties, um, obviously Mr..

325
00:19:19,290 --> 00:19:22,980
Came in for a few days. Uh,
they produced their report.

326
00:19:23,250 --> 00:19:27,217
We did an episode.
Former OpenAI employee Miles Brundage,

327
00:19:27,300 --> 00:19:32,310
who now has his own, um, sort of third
party, uh, aspiring to be an auditor.

328
00:19:32,460 --> 00:19:36,997
Would you support a law that said, um,
there should always be the presence of

329
00:19:37,080 --> 00:19:40,477
auditors, because obviously just looking
at the models after the released isn't

330
00:19:40,560 --> 00:19:45,990
enough, given, um, that, um, these,
you know, some of these major disasters

331
00:19:46,170 --> 00:19:51,397
occurred pre-release models.
Would you support, uh, law if they could

332
00:19:51,480 --> 00:19:56,707
be designed that would have essentially
third party, uh, auditors embedded in

333
00:19:56,790 --> 00:20:00,637
all of the major frontier orgs.
Well, look, I'll always say that the

334
00:20:00,720 --> 00:20:03,007
nuance really matters.
The details really matter.

335
00:20:03,090 --> 00:20:06,960
So it's too hard to say at a high
level whether any particular property is

336
00:20:07,120 --> 00:20:10,507
something that would work in practice.
I do think a general direction of

337
00:20:10,590 --> 00:20:15,757
thinking about as you move toward these
very capable models where you need to

338
00:20:15,840 --> 00:20:17,917
have safety cases, you need
to lay out your standards.

339
00:20:18,000 --> 00:20:22,987
We're already having third party
auditors for exactly that type of a

340
00:20:23,070 --> 00:20:26,017
process, right?
If you look at how the evaluation

341
00:20:26,100 --> 00:20:29,047
processes for models over the past
couple of months have, have, have

342
00:20:29,130 --> 00:20:32,737
evolved, right.
We've had the Casey, which is a US

343
00:20:32,820 --> 00:20:36,487
organization that test models.
Yeah, we have the UK, EC, these

344
00:20:36,570 --> 00:20:40,567
government Third party auditors test
our models before I release them.

345
00:20:40,650 --> 00:20:45,037
But so what about like and like?
Here's the thing that I thought about in

346
00:20:45,120 --> 00:20:49,687
the wake of reading, uh, about
the hugging face incident and so forth.

347
00:20:49,770 --> 00:20:53,887
Like, yeah, you know,
I think of OpenAI and

348
00:20:53,970 --> 00:20:57,690
anthropic as companies, but people
like to call them, quote, labs.

349
00:20:57,900 --> 00:21:01,807
And when I think of labs, I think like,
wow, like, you know, a lab, if we were

350
00:21:01,890 --> 00:21:06,007
talking about a bio lab or a chemical
lab, we'd have just like all kinds of,

351
00:21:06,090 --> 00:21:10,087
like, rules and materials handling.
And then maybe in this case, it would be

352
00:21:10,170 --> 00:21:14,947
like handling of the weights and whose
computers they're on and who can access

353
00:21:15,030 --> 00:21:18,427
those computers and FDA
approval before you go to market.

354
00:21:18,510 --> 00:21:20,407
Yeah. And all of these
things like just like

355
00:21:20,490 --> 00:21:23,887
an or even a restaurant, someone would
come in and they say like, oh, is the

356
00:21:23,970 --> 00:21:28,380
stove sufficiently far
back from the wall, etc..

357
00:21:28,530 --> 00:21:33,337
So not just like, okay, like measuring
the capabilities of the risks of the

358
00:21:33,420 --> 00:21:40,217
models themselves, but I'm thinking like
the auditing of the lab process, the way

359
00:21:40,300 --> 00:21:45,557
that any other lab probably has regular
inspections of like, how safe is the lab

360
00:21:45,640 --> 00:21:47,777
environment? Well, well,
this is what I would say has

361
00:21:47,860 --> 00:21:50,117
been changing just even over
the past couple of weeks.

362
00:21:50,200 --> 00:21:52,780
Okay. Because the focus for,

363
00:21:53,800 --> 00:21:57,227
let's say, the past 12 months,
24 months, something like that has been on

364
00:21:57,310 --> 00:21:59,807
deployment. Yeah. Right.
So you have a model, you've produced the

365
00:21:59,890 --> 00:22:02,040
model. You want
to release it to the world.

366
00:22:03,220 --> 00:22:06,767
How do you decide whether you've
you're at the right level there.

367
00:22:06,850 --> 00:22:09,557
And we have a preparedness framework
anthropic as a responsible scaling

368
00:22:09,640 --> 00:22:12,460
policy. There's certain evaluations.

369
00:22:12,730 --> 00:22:15,497
Again there's Casey the UK.
There's a whole ecosystem that's been

370
00:22:15,580 --> 00:22:18,257
built up around that.
I think the thing that hugging Face has

371
00:22:18,340 --> 00:22:24,430
shown is that it is time to start
pulling that process earlier.

372
00:22:24,520 --> 00:22:27,670
You need to think about this during
development and evaluation.

373
00:22:27,790 --> 00:22:32,147
And just it makes sense as the models
are more capable for exactly the kinds

374
00:22:32,230 --> 00:22:35,327
of reasons that you're describing
for for analogies to other fields.

375
00:22:35,410 --> 00:22:39,407
Now, there's a lot of specifics for AI,
and I think it's very important to

376
00:22:39,490 --> 00:22:42,227
really engage with those.
As you design a regime, as you think

377
00:22:42,310 --> 00:22:46,427
about what is a set up that involves
first party, third party that involves

378
00:22:46,510 --> 00:22:52,067
government that has the right
interactions amongst academics and other

379
00:22:52,150 --> 00:22:57,677
other parties, early testers.
It's really about like, you really have

380
00:22:57,760 --> 00:23:00,407
to engage with the way
that the eye process is going.

381
00:23:00,490 --> 00:23:01,637
Right. And I think one thing that is the

382
00:23:01,720 --> 00:23:05,297
process, exactly one thing that is
different, is that the ways there are

383
00:23:05,380 --> 00:23:09,527
some parts of AI that we understand well
and are going to remain invariant, and

384
00:23:09,610 --> 00:23:12,287
to some extent, the core training
process hasn't changed at all.

385
00:23:12,370 --> 00:23:13,967
Right. It's still forward pass.

386
00:23:14,050 --> 00:23:16,337
You do a backward pass.
You do not optimize your step like

387
00:23:16,420 --> 00:23:18,377
that's how training is.
And that's how it was in the 80s.

388
00:23:18,460 --> 00:23:21,070
Right. It's like actually
totally remarkable.

389
00:23:21,190 --> 00:23:25,667
Now, of course, the scale at which we
run and the architectures that we use, a

390
00:23:25,750 --> 00:23:28,960
lot of these details
that they have evolved.

391
00:23:29,290 --> 00:23:33,647
And I think that being very close to the
technical details is the best way that

392
00:23:33,730 --> 00:23:36,857
you can ensure that you get the kinds
of of safety guarantees we're talking

393
00:23:36,940 --> 00:23:38,687
about. For example, there was an article

394
00:23:38,770 --> 00:23:43,217
recently saying that, hey, OpenAI
changed the architecture so that monitor

395
00:23:43,300 --> 00:23:46,000
ability is less change if
not release new release.

396
00:23:46,960 --> 00:23:50,087
I would consider this article
to basically be fake news, right?

397
00:23:50,170 --> 00:23:54,857
That we actually have experiments that
really show that our changes in monitor

398
00:23:54,940 --> 00:23:58,007
ability, that we've talked about,
changes that monitor ability come from

399
00:23:58,090 --> 00:24:00,647
capability improvements.
Just the model is smarter.

400
00:24:00,730 --> 00:24:03,017
So it relies on the train
of thought us. Yeah.

401
00:24:03,100 --> 00:24:05,777
And so I think that if you say we're
going to put a lot of rules around

402
00:24:05,860 --> 00:24:09,407
architectures, you're going to miss the
boat, you're not actually going to solve

403
00:24:09,490 --> 00:24:11,987
the problem.
And so this is where I think that the

404
00:24:12,070 --> 00:24:15,617
people who are the deepest on the
technology, the people who are actually

405
00:24:15,700 --> 00:24:20,440
creating the technology, pushing for it,
we have both a unique insight.

406
00:24:20,590 --> 00:24:23,080
We have a unique voice.
We have a unique sort of

407
00:24:24,370 --> 00:24:28,817
ability to say, hey, if we change things
in this way, we'll actually get the

408
00:24:28,900 --> 00:24:31,187
safety property. But if you
do it this other way, it's

409
00:24:31,270 --> 00:24:34,337
just going to put in place friction
that won't actually solve the problem.

410
00:24:34,420 --> 00:24:37,217
And so I think that there's
something a role for us to play.

411
00:24:37,300 --> 00:24:38,597
But it's not just about us. Right.

412
00:24:38,680 --> 00:24:42,107
And so I think it is very important
that you think about these third parties.

413
00:24:42,190 --> 00:24:45,227
You think about government oversight.
And again, it's not just one company,

414
00:24:45,310 --> 00:24:47,960
not just one country.
How do you get to international

415
00:24:48,070 --> 00:24:50,350
coordination and norms.
You get to treaties.

416
00:24:50,560 --> 00:24:53,297
Like all this should be
the conversation that we're having now.

417
00:24:53,380 --> 00:24:56,027
I don't think we have all the answers,
but we can supply information.

418
00:24:56,110 --> 00:24:58,607
And that's one thing that we view
also is very core to our mission.

419
00:24:58,690 --> 00:25:00,527
Hmm. Well, just on this note, one of the

420
00:25:00,610 --> 00:25:04,967
arguments you hear against government
oversight and additional pacing is this

421
00:25:05,050 --> 00:25:08,417
idea that the U.S., it's a national
security interest, and we need to

422
00:25:08,500 --> 00:25:12,587
compete against China. And then
at the same time, you also hear

423
00:25:12,670 --> 00:25:18,287
and I think OpenAI has said this, that
China's distilling US models US frontier

424
00:25:18,370 --> 00:25:20,717
models. And I guess
the question to me is always

425
00:25:20,800 --> 00:25:25,157
like, well, if that's true, if China's
building off the success of U.S.

426
00:25:25,240 --> 00:25:29,957
companies, then if they pause, then the
China problem is kind of solved, right?

427
00:25:30,040 --> 00:25:32,427
Like, why do we need
to worry about China so much?

428
00:25:32,510 --> 00:25:39,510
Well, I view this technology
is really about compute progress.

429
00:25:40,400 --> 00:25:43,460
Right. That really like
why is I happening now?

430
00:25:44,420 --> 00:25:51,207
If you look at the 1990s
Ray Kurzweil writings, yeah,

431
00:25:51,290 --> 00:25:56,150
he basically predicted this moment
at about this time just based on compute.

432
00:25:56,690 --> 00:25:59,277
Yeah. Growth. Right.
Transistor density, memory density, all

433
00:25:59,360 --> 00:26:01,047
these things, you see these exponentials.

434
00:26:01,130 --> 00:26:03,357
And he was kind of like, this is the
point where you'll be able to really

435
00:26:03,440 --> 00:26:07,707
build AGI. And I think
that that is a really

436
00:26:07,790 --> 00:26:10,540
important thing for us all
to kind of internalize that.

437
00:26:10,820 --> 00:26:15,477
You can kind of look at OpenAI
or anthropic, you know, we're at the

438
00:26:15,560 --> 00:26:20,720
frontier, and I think it is an absolute
both privilege and responsibility to be

439
00:26:21,020 --> 00:26:22,850
at that, that edge. But

440
00:26:24,530 --> 00:26:28,767
really this edge, there's always going
to be someone who's at that frontier,

441
00:26:28,850 --> 00:26:30,477
right? Whether it's us,
whether it's someone in

442
00:26:30,560 --> 00:26:32,847
a different country.
There's going to be someone who's

443
00:26:32,930 --> 00:26:37,227
charting it, and I think you get
a little bit of ability to help see into

444
00:26:37,310 --> 00:26:39,567
that future, right?
Get information like with hugging face,

445
00:26:39,650 --> 00:26:42,357
where we can see this is what it's going
to be like when these capabilities are

446
00:26:42,440 --> 00:26:45,290
out there. But it's not
just about what we create.

447
00:26:45,620 --> 00:26:49,497
And that I think maybe we have move
forward the timeline to this technology

448
00:26:49,580 --> 00:26:51,957
by six months.
Maybe we've moved it forward by two

449
00:26:52,040 --> 00:26:55,017
years. I certainly hope
we didn't slow it down

450
00:26:55,100 --> 00:26:58,827
by six months or two years.
I think that we have a track record of

451
00:26:58,910 --> 00:27:01,917
being the ones who have have
repeatedly had the innovations and the

452
00:27:02,000 --> 00:27:05,810
breakthroughs, but I
think that it will happen

453
00:27:07,220 --> 00:27:10,340
one way or another as long
as people are building computers.

454
00:27:10,670 --> 00:27:16,347
And so to me that the the reason again,
that we started this place, I got very

455
00:27:16,430 --> 00:27:22,647
excited about the idea of I
from reading about 19 1950 Turing.

456
00:27:22,730 --> 00:27:26,427
I you know, Alan Turing lots about
about this about machines that could

457
00:27:26,510 --> 00:27:29,637
understand things that people would
could not solve, problems that humans

458
00:27:29,720 --> 00:27:33,357
would not be able to spend a lot of time
really thinking about your point of, you

459
00:27:33,440 --> 00:27:36,867
know, 20 years worth of thinking about
misalignment and what that would be like

460
00:27:36,950 --> 00:27:39,867
before you even have I.
I was reading a lot of these online

461
00:27:39,950 --> 00:27:42,687
blogs, and I started a reading group
at stripe, which is the companies that

462
00:27:42,770 --> 00:27:46,527
previously talk about this every week.
And so I care a lot about this.

463
00:27:46,610 --> 00:27:52,527
Our company cares a lot about this.
Um, but I think that the, the idea that

464
00:27:52,610 --> 00:27:56,757
there's one the idea
that this technology can only

465
00:27:56,840 --> 00:27:59,757
be created by, you know,
the people who are currently the players.

466
00:27:59,840 --> 00:28:03,490
I don't think that's true at all.
One more sort of question, specifically

467
00:28:03,770 --> 00:28:05,967
on some of the safety and alignment stuff.

468
00:28:06,050 --> 00:28:09,417
Uh, your chief scientist at OpenAI,
I wrote a very interesting blog post.

469
00:28:09,500 --> 00:28:13,587
Ah, piece an alien mind.
Jaco Pachuca wrote a very interesting

470
00:28:13,670 --> 00:28:16,617
piece into his one line in there
that I thought was interesting.

471
00:28:16,700 --> 00:28:21,567
You said about the release of Astra six
and he said, um, we invest heavily along

472
00:28:21,650 --> 00:28:23,817
the spectrum of approaches
spanned by these directions.

473
00:28:23,900 --> 00:28:28,400
We also see meaningful progress.
GPT six is the first model that benefits

474
00:28:28,650 --> 00:28:32,107
from some important advancements.
We have been working on in a long time

475
00:28:32,190 --> 00:28:36,180
and is significantly
better aligned than 5.6 sol.

476
00:28:36,540 --> 00:28:40,327
Um, should the labs share their alignment

477
00:28:40,410 --> 00:28:42,517
work? I mean, again,
if we're talking about I

478
00:28:42,600 --> 00:28:45,720
understand your competitors
and capability gains, etc.

479
00:28:46,020 --> 00:28:51,307
on questions of safety, if a lab makes
a big breakthrough on the alignment

480
00:28:51,390 --> 00:28:58,390
question such that it can like develop
advanced, I fast in a safer way.

481
00:28:59,340 --> 00:29:04,020
Should that be diffused across the rest
of the industry or should whatever this,

482
00:29:04,350 --> 00:29:10,687
um, quote, important advancement
that you made to make GPT six Astra more

483
00:29:10,770 --> 00:29:13,147
aligned? Is that something
that should be kept

484
00:29:13,230 --> 00:29:15,607
in-house? So I think that there's a lot of

485
00:29:15,690 --> 00:29:19,027
specific circumstances to answer
that question for any specific advance,

486
00:29:19,110 --> 00:29:23,257
because the thing that to me can be
a category error is thinking about things

487
00:29:23,340 --> 00:29:26,107
as it is a safety thing
or it is a capability.

488
00:29:26,190 --> 00:29:28,597
Sure. And in fact, you want there to be as

489
00:29:28,680 --> 00:29:30,877
little separation
as possible between the two.

490
00:29:30,960 --> 00:29:34,447
You want as much intertwining.
You want to have techniques that are, by

491
00:29:34,530 --> 00:29:38,887
their very nature, safer.
And so just as you make more capability

492
00:29:38,970 --> 00:29:41,167
progress, that you also
get the safety progress.

493
00:29:41,250 --> 00:29:46,957
And so I think that the way I look at it
is the more things are a pure alignment

494
00:29:47,040 --> 00:29:50,077
thing, more that there a pure safety
thing, the more that they are something

495
00:29:50,160 --> 00:29:52,747
that are just pure good, that you're
just like, yes, everyone should

496
00:29:52,830 --> 00:29:55,710
obviously know this and use it,
then yes, share it more.

497
00:29:55,860 --> 00:29:58,987
The more that things are just about this
will just make the model more capable,

498
00:29:59,070 --> 00:30:03,187
will make it safer. You know,
maybe, maybe there's much less

499
00:30:03,270 --> 00:30:05,017
sort of reason to share for those reasons.

500
00:30:05,100 --> 00:30:09,067
And so I think you need to look at any
specific advance along all these axes

501
00:30:09,150 --> 00:30:13,117
and all these factors.
And I think that there are some areas

502
00:30:13,200 --> 00:30:17,227
where we have actually been extremely
at the forefront with sharing and talking

503
00:30:17,310 --> 00:30:19,267
about it. Chain of thought
monitor ability is a

504
00:30:19,350 --> 00:30:23,717
good example of that, where the moment
that we and we were the ones who pioneer

505
00:30:23,800 --> 00:30:26,647
this paradigm of reasoning
and change of thought.

506
00:30:26,730 --> 00:30:31,057
The moment we saw that, we were
instantly thinking, we need to preserve

507
00:30:31,140 --> 00:30:34,177
this for as long as possible.
So when we released a product that we

508
00:30:34,260 --> 00:30:37,387
did not actually display the chain
of thought, even though it would have been

509
00:30:37,470 --> 00:30:39,397
very helpful.
People love reading that stuff.

510
00:30:39,480 --> 00:30:41,587
It's super interesting.
It's super helpful, but it was just

511
00:30:41,670 --> 00:30:44,477
clear we're going to immediately
feel like we have to optimize it.

512
00:30:44,560 --> 00:30:46,537
We're going to have
to remove that legibility.

513
00:30:46,620 --> 00:30:50,310
We started talking about this.
We helped spearhead a paper that was

514
00:30:50,670 --> 00:30:53,707
putting forth this position on that.
You cannot optimize these things.

515
00:30:53,790 --> 00:30:59,167
And again, we've we've shown a lot
of the way and a lot of the way on various

516
00:30:59,250 --> 00:31:01,147
techniques. And so I
think it comes down to the

517
00:31:01,230 --> 00:31:06,367
specifics, I'm going to move away
from safety and alignment for a second.

518
00:31:06,450 --> 00:31:11,700
But you're the guy in charge of allocating
compute at OpenAI, pretty much. Right.

519
00:31:12,240 --> 00:31:16,027
So actually, it's funny, I actually
try to I, I do as little of the compute

520
00:31:16,110 --> 00:31:17,707
allocation as possible. Oh, seriously.

521
00:31:17,790 --> 00:31:18,697
Wait. Explain more.

522
00:31:18,780 --> 00:31:23,797
Well, okay, so I'm I'm someone who has
spent a lot of effort to produce the

523
00:31:23,880 --> 00:31:27,077
compute. So I spent a lot
of effort on the data

524
00:31:27,160 --> 00:31:31,637
center side, on the machine learning
engineering side, the, uh, and on the

525
00:31:31,720 --> 00:31:35,477
product side have to, you know,
ultimately be accountable for how we

526
00:31:35,560 --> 00:31:38,177
allocate compute. Um,
but actually, I try to set up the

527
00:31:38,260 --> 00:31:42,527
systems and the processes and then
the actual like a lot of the decisions on

528
00:31:42,610 --> 00:31:47,327
where research compute goes market or
in Yakub, uh, run those um, a lot of the

529
00:31:47,410 --> 00:31:50,687
actual, you know, we basically make
a big decision on the split of how much

530
00:31:50,770 --> 00:31:53,927
goes to applied versus research.
And then within applied, we have some

531
00:31:54,010 --> 00:31:56,867
frameworks now where we kind of have
buckets for different products.

532
00:31:56,950 --> 00:32:01,607
And I would just say that the uh, the
choice of where compute goes is like in

533
00:32:01,690 --> 00:32:06,280
some ways the hardest problem at OpenAI.
It's like a capital allocation problem,

534
00:32:06,610 --> 00:32:11,087
but it's one that I actually think other
people are, uh, you know, I really try

535
00:32:11,170 --> 00:32:13,997
to have them, uh, you know,
leave it to them as much as possible.

536
00:32:14,080 --> 00:32:16,187
Well, how do you think about
it from a product standpoint?

537
00:32:16,270 --> 00:32:20,297
Because, as you say, you can't I mean,
you can't really disaggregate the two to

538
00:32:20,380 --> 00:32:22,607
much, right? Ultimately,
you have to make decisions

539
00:32:22,690 --> 00:32:25,390
about what you want
to develop and promote?

540
00:32:25,930 --> 00:32:30,047
Yeah. So I do think that your values and

541
00:32:30,130 --> 00:32:33,287
prioritization shine through through
compute allocation, because the world

542
00:32:33,370 --> 00:32:35,687
that we're in is one where there
just is not enough compute, right.

543
00:32:35,770 --> 00:32:39,287
Compute is revenue, right?
It's also the production of models.

544
00:32:39,370 --> 00:32:42,257
And so you kind of have
different timescales of return.

545
00:32:42,340 --> 00:32:44,327
And you can think about
doing more alignment work.

546
00:32:44,410 --> 00:32:50,260
You can do really any sort of task.
It fundamentally comes down to compute.

547
00:32:50,560 --> 00:32:56,500
Well I mean let me let me tell a story.
So for leading up to the launch of GPT

548
00:32:56,620 --> 00:33:01,307
55, that we ran a process where we
basically looked through all of the

549
00:33:01,390 --> 00:33:04,607
different places that we had compute
that we could potentially squeeze, and

550
00:33:04,690 --> 00:33:07,037
we're like, we could cut
back on this rate limit.

551
00:33:07,120 --> 00:33:10,847
We could shut down this part of the
product, we could do this, do that.

552
00:33:10,930 --> 00:33:14,507
And we had this whole stack rank of levers
because we knew we're going to be

553
00:33:14,590 --> 00:33:16,607
55. People are going
to love this model, and

554
00:33:16,690 --> 00:33:20,717
we're going to run out of compute.
And it was such a painful process

555
00:33:20,800 --> 00:33:24,317
because you're basically going through
every single part of your product and

556
00:33:24,400 --> 00:33:26,087
being like. How valuable is this one?

557
00:33:26,170 --> 00:33:27,617
How much should we care about these users?

558
00:33:27,700 --> 00:33:29,187
And of course, you care about all of them.

559
00:33:29,270 --> 00:33:31,937
Like we want to deliver value.
And we built this product for a reason

560
00:33:32,020 --> 00:33:34,367
and people are using it.
And you know, you're comparing the

561
00:33:34,450 --> 00:33:37,217
efficiency of cost, different things.
And so what we've tried to do is move

562
00:33:37,300 --> 00:33:40,877
towards more of an objective standard.
And you're never going to be fully

563
00:33:40,960 --> 00:33:43,547
there. But there's
a second piece of detail as

564
00:33:43,630 --> 00:33:47,740
well that's mattered a lot, which
is trying to push a lot of the constraint

565
00:33:47,860 --> 00:33:52,257
down closer to the people who are
in the weeds on a particular product.

566
00:33:52,340 --> 00:33:54,677
And so you say, hey,
here's your compute allocation.

567
00:33:54,760 --> 00:34:00,047
We're about to release another model,
new modality, some something like GPD

568
00:34:00,130 --> 00:34:02,417
live. Right. That's just like
a totally different thing.

569
00:34:02,500 --> 00:34:04,247
And you need to fit
in your existing budget.

570
00:34:04,330 --> 00:34:06,707
What do you do.
And somehow people always find

571
00:34:06,790 --> 00:34:08,567
efficiencies. Sometimes
people are like, you know

572
00:34:08,650 --> 00:34:10,607
what? Actually there
is this like, you know,

573
00:34:10,690 --> 00:34:12,947
kernel efficiency that we
didn't roll out yet.

574
00:34:13,030 --> 00:34:14,447
Well, I guess we'll
roll it out now, right?

575
00:34:14,530 --> 00:34:18,377
Or like, oh, it turns out that, uh,
you know, there's a peak to trough between

576
00:34:18,460 --> 00:34:21,947
your day and night, and we could
actually pack this new thing on top of

577
00:34:22,030 --> 00:34:22,877
that. And so we actually don't need

578
00:34:22,960 --> 00:34:25,907
incremental. And so I
think that really unleashing

579
00:34:25,990 --> 00:34:31,217
the creativity and the co-design
of the people who are doing the work is

580
00:34:31,300 --> 00:34:33,737
actually maybe the thing
that I really focus on.

581
00:34:33,820 --> 00:34:37,967
So I really try not to be as much in the
like sort of arbiter, like just making

582
00:34:38,050 --> 00:34:41,417
these like super high level decisions.
But instead of really focusing on how do

583
00:34:41,500 --> 00:34:45,760
we uplevel execution and get the most
efficiency down to the, to the metal.

584
00:34:46,210 --> 00:34:49,690
Do you have the theory
for why some of the big, um,

585
00:34:50,890 --> 00:34:55,277
now, you know, I finally succumbed and I
occasionally read the less wrong message

586
00:34:55,360 --> 00:34:57,620
boards now as like I do stage in my life.

587
00:34:57,850 --> 00:35:01,247
So it was an interesting post.
Do you have a, uh, theory for why some

588
00:35:01,330 --> 00:35:07,697
of these major incidents seem to occur
in evil's pre-release and then rarely,

589
00:35:07,780 --> 00:35:12,147
you know, when I use the latest model,
I've never had to go hack into a server

590
00:35:12,230 --> 00:35:15,017
until I get me some information.
Do you have a theory for why we see

591
00:35:15,100 --> 00:35:19,577
these things emerge pre-release?
Well, some of these are related to

592
00:35:19,660 --> 00:35:21,567
safeguards that are
intentionally turned off.

593
00:35:21,650 --> 00:35:24,597
Yeah. Right. Right.
So there's there's really the things

594
00:35:24,680 --> 00:35:26,817
that are being evaluated are
not necessarily representative.

595
00:35:26,900 --> 00:35:30,980
What is what is released.
And we put so much effort into ensuring

596
00:35:31,760 --> 00:35:38,457
trust alignment with astro
ability, all these things for

597
00:35:38,540 --> 00:35:41,307
deployed models. And again,
I think it's just been that

598
00:35:41,390 --> 00:35:45,927
the evaluation and development phases
have been much less rigorous, right,

599
00:35:46,010 --> 00:35:47,877
that the standards there
have just been lower.

600
00:35:47,960 --> 00:35:51,477
And so it's not surprising
to me that it wasn't just us.

601
00:35:51,560 --> 00:35:53,667
Right? There's really many other lives of.

602
00:35:53,750 --> 00:35:56,217
All right. Yeah,
it's important that others have

603
00:35:56,300 --> 00:35:57,850
had this as well. That's right.

604
00:35:58,040 --> 00:36:01,417
Since we're back on the
safety topic. Can I ask?

605
00:36:01,500 --> 00:36:05,360
I'm just going to ask you
to directly address that.

606
00:36:05,810 --> 00:36:09,387
I don't want to call it a conspiracy
theory, but when whenever a lab comes

607
00:36:09,470 --> 00:36:14,637
out and says, like, we are worried about
the possibility of human extinction or

608
00:36:14,720 --> 00:36:19,257
cyber attacks and all of that, someone
will say, this is just marketing for the

609
00:36:19,340 --> 00:36:21,800
models. They want to make
their models look as

610
00:36:22,070 --> 00:36:26,780
sophisticated and powerful as possible.
What's your response to that directly?

611
00:36:31,400 --> 00:36:38,030
It's just not I. Look, I think
that the way I look at this is that

612
00:36:39,500 --> 00:36:45,297
so much of what we need to do
is operationalize both the technical

613
00:36:45,380 --> 00:36:50,750
capability, but also the positive
impacts in the world as verified by both

614
00:36:51,050 --> 00:36:54,717
controllability, monitor ability, steer
ability, all those things, but also how

615
00:36:54,800 --> 00:36:58,827
these things are deployed in the world.
And that finding the right way of

616
00:36:58,910 --> 00:37:04,040
talking about that and the right way
of releasing practical and grounded.

617
00:37:04,340 --> 00:37:09,297
I think that is a challenge that
the labs have not fully figured out.

618
00:37:09,380 --> 00:37:10,977
It's something we spend
a lot of time thinking about.

619
00:37:11,060 --> 00:37:15,710
I don't think we do a perfect job of it.
But my view is that the

620
00:37:16,880 --> 00:37:21,267
what we're creating is it not just
against any one company, just like us as

621
00:37:21,350 --> 00:37:24,537
a whole right now, as humanity, the way
that we're shifting to this compute

622
00:37:24,620 --> 00:37:27,507
powered economy, I think, is something
that will really lift up everyone.

623
00:37:27,590 --> 00:37:30,357
I've a lot of optimism that we can
navigate this moment, but you have to

624
00:37:30,440 --> 00:37:33,717
approach it with seriousness.
And I think that finding a right way of

625
00:37:33,800 --> 00:37:38,307
really communicating about here's
the right way of developing, like thinking

626
00:37:38,390 --> 00:37:41,277
about all these conversations we're
having about the results of hugging face

627
00:37:41,360 --> 00:37:44,637
and what that teaches us.
The defenders window that we're in that

628
00:37:44,720 --> 00:37:48,807
shows us where cybersecurity is going.
The fact that and people are using these

629
00:37:48,890 --> 00:37:53,607
models to help with their own health,
right, to get information that helps

630
00:37:53,690 --> 00:37:56,420
them. So many people,
like even today, I was

631
00:37:57,080 --> 00:38:00,207
talking to one of my coworkers who was
saying that I, her parents were texting

632
00:38:00,290 --> 00:38:04,677
her, saying ChatGPT, helping with
my back pain and, uh, my my my pinky.

633
00:38:04,760 --> 00:38:07,437
And just like, you know, these
these kinds of impacts are real.

634
00:38:07,520 --> 00:38:12,117
You know, people using Astra for one
of my friends is using it to produce CAD

635
00:38:12,200 --> 00:38:14,870
designs for a guest house
that now is being built.

636
00:38:15,140 --> 00:38:18,487
Right. It's like you can actually do
so much more. You get all this leverage.

637
00:38:18,570 --> 00:38:23,617
And I think that, of course, we have
to think about the impacts, the risks, and

638
00:38:23,700 --> 00:38:26,407
we have to encounter those
seriously and really mitigate them.

639
00:38:26,490 --> 00:38:29,347
And so I think that that all
of this has to be true at once.

640
00:38:29,430 --> 00:38:32,947
We have to navigate through the small,
the large and even the almost

641
00:38:33,030 --> 00:38:36,180
unthinkable. And I think
that finding the right way

642
00:38:36,390 --> 00:38:39,877
of communicating about it so that you
land all that nuance and say that, hey,

643
00:38:39,960 --> 00:38:44,947
we can operate through and make sure
you don't throw a baby with the bathwater

644
00:38:45,030 --> 00:38:47,467
and that you don't get sort of,
yeah, that you really stay on track.

645
00:38:47,550 --> 00:38:51,487
To me, that is a core challenge.
And I think that that we all need to, to

646
00:38:51,570 --> 00:38:54,967
really work on that.
You know, I had a reward hacking

647
00:38:55,050 --> 00:38:57,780
incident with something
I was building. Yeah.

648
00:38:58,140 --> 00:39:02,887
Um, a little it's minor.
Not the most, but I was like, several

649
00:39:02,970 --> 00:39:04,597
months ago. Remember,
the guy claimed to have found

650
00:39:04,680 --> 00:39:07,117
Satoshi, and so I was like,
oh, you know what?

651
00:39:07,200 --> 00:39:10,897
I'm going to, like, build a model that
can identify Satoshi's writing version

652
00:39:10,980 --> 00:39:14,857
on Satoshi writing. And I got
a bunch of examples of Satoshi

653
00:39:14,940 --> 00:39:21,277
writing held out of golden set and then
like compared to ended like got 100 on

654
00:39:21,360 --> 00:39:25,387
it predicted 100% of the now
in Satoshi you know the the held out.

655
00:39:25,470 --> 00:39:26,977
Golden said. I was like oh my God.

656
00:39:27,060 --> 00:39:30,510
I built a model that can identify
Satoshi's writing specifically.

657
00:39:30,810 --> 00:39:35,527
It turned out that just on all
the Satoshi numbers, all listed Toshi

658
00:39:35,610 --> 00:39:39,757
examples, I had left the little page
number of the thing on it, and it just

659
00:39:39,840 --> 00:39:42,487
reward hacked by identifying
the page number.

660
00:39:42,570 --> 00:39:45,397
And so they called that overfitting
and machine learning.

661
00:39:45,480 --> 00:39:48,727
But I think it's good conceptually
similar to reward hacking.

662
00:39:48,810 --> 00:39:52,507
Sounds like human error to me.
It found a different way to answer the

663
00:39:52,590 --> 00:39:55,327
question was not helpful.
And then I gave up that project.

664
00:39:55,410 --> 00:39:58,567
Can you walk through this term
reinforcement learning?

665
00:39:58,650 --> 00:40:01,747
We're all learning about this, right?
The world is where, um, can you walk

666
00:40:01,830 --> 00:40:08,830
through an example of a sort of like
misconfigured RL environment so that

667
00:40:09,030 --> 00:40:12,067
people can actually understand, like what?

668
00:40:12,150 --> 00:40:16,320
Reward hacking and decide,
look what what we call reward hacking.

669
00:40:16,440 --> 00:40:21,517
What it actually looks like.
So back in, I think 2017 or so, maybe

670
00:40:21,600 --> 00:40:27,817
2018. We published an example
of a reward hacked agent. Okay.

671
00:40:27,900 --> 00:40:31,267
In an environment.
And this was about race.

672
00:40:31,350 --> 00:40:34,717
So it's just like a flash game.
Like very simple video game of you.

673
00:40:34,800 --> 00:40:38,347
You have this boat that has to like,
navigate through a bunch of obstacles in

674
00:40:38,430 --> 00:40:43,987
a big circle and, uh, you know, past the
finish line, and there's some like, you

675
00:40:44,070 --> 00:40:46,867
know, so you can pick up some items
along the way that give you points.

676
00:40:46,950 --> 00:40:51,187
And that it turned out that there
was this little inlet, this little lagoon

677
00:40:51,270 --> 00:40:55,357
where if you kind of did things
in exactly a very precise way, that the

678
00:40:55,440 --> 00:40:58,597
boat with like keep running into things
and kind of, you know, sort of taking

679
00:40:58,680 --> 00:41:02,797
damage, but then you would, uh, you know,
sort of pick up some item and get,

680
00:41:02,880 --> 00:41:05,287
get points. And it just
the agent learned that if it

681
00:41:05,370 --> 00:41:08,587
went like backwards and like did this
exact, precise pattern, it could just go

682
00:41:08,670 --> 00:41:12,067
around in a circle and just be picking
up infinite points and nothing to do

683
00:41:12,150 --> 00:41:13,267
with actually completing the race. Yeah.

684
00:41:13,350 --> 00:41:13,867
Right. Right.

685
00:41:13,950 --> 00:41:16,867
And so I think that this is an example
where you're just like, okay, here's

686
00:41:16,950 --> 00:41:20,377
this behavior that actually conforms
with the reward you're giving.

687
00:41:20,460 --> 00:41:21,947
You're saying maximize the points. Yeah.

688
00:41:22,030 --> 00:41:25,717
But actually the hope was maximize
the points means that you're going to

689
00:41:25,800 --> 00:41:29,467
complete the race.
Is it your understanding anthropic has

690
00:41:29,550 --> 00:41:34,710
made this claim is that your understanding
is, well, that if you have

691
00:41:35,100 --> 00:41:40,597
um, if in the training process, you have
misconfigured RL environments so that

692
00:41:40,680 --> 00:41:46,087
there's a lot of those equivalent of boat
races, that the final model is more

693
00:41:46,170 --> 00:41:51,427
likely to exhibit misaligned
tendencies in a broad range of things.

694
00:41:51,510 --> 00:41:57,217
If it's essentially internalize the idea
that cheating or not technically doing

695
00:41:57,300 --> 00:42:01,927
what the the, uh, researcher wanted
that that generalizes into more misaligned

696
00:42:02,010 --> 00:42:04,957
behavior. Well, we see this kind
of thing all the time. Okay.

697
00:42:05,040 --> 00:42:11,737
And even with 551 of the in five six,
like one of the things that we saw is as

698
00:42:11,820 --> 00:42:15,250
the models get more capable,
they do find holes in your craters.

699
00:42:15,400 --> 00:42:17,777
If your craters aren't perfect, if they
kind of correlate with the thing you

700
00:42:17,860 --> 00:42:21,227
want but aren't exactly what you want.
And so actually, a lot of our progress

701
00:42:21,310 --> 00:42:24,497
and reinforcement learning has been
improving the reliability of the craters

702
00:42:24,580 --> 00:42:26,717
and ensuring that they
cannot be reward hacked.

703
00:42:26,800 --> 00:42:30,947
And so I think that that is one
of the areas where as the models get more

704
00:42:31,030 --> 00:42:34,577
capable, as you build your safety
case for why is the next training run

705
00:42:34,660 --> 00:42:38,927
something that that you know, that we,
you know, should should run that?

706
00:42:39,010 --> 00:42:41,950
Making sure that you've
actually uplevel those graders.

707
00:42:42,790 --> 00:42:46,487
That's actually very core.
And the way this shows up sometimes, by

708
00:42:46,570 --> 00:42:51,167
the way, is just less good models,
like for example, the writing, like that's

709
00:42:51,250 --> 00:42:53,627
something where we haven't
had very good graders on it.

710
00:42:53,710 --> 00:42:56,297
And so then you end up with writing that
just like you look at this is like, this

711
00:42:56,380 --> 00:42:59,147
is slop. It's not very good.
And it was good, according to the writer.

712
00:42:59,230 --> 00:43:02,357
But it's just not actually good.
And so I actually think that this is a

713
00:43:02,440 --> 00:43:06,010
very optimistic story as well,
because what it means is that

714
00:43:07,270 --> 00:43:10,757
these graders are now like,
how do you judge what good writing is?

715
00:43:10,840 --> 00:43:14,117
The answer is you use an eye
to evaluate if you have good writing.

716
00:43:14,200 --> 00:43:18,767
And so you actually, as you get better
AI capability, have the ability to build

717
00:43:18,850 --> 00:43:21,497
much more reliable graders.
And now of course, you start thinking

718
00:43:21,580 --> 00:43:25,037
about, okay, but if the eye is judging
the eye, like how do you make sure that

719
00:43:25,120 --> 00:43:27,527
it actually stays on track? Yeah,
it starts to be a little bit of,

720
00:43:27,610 --> 00:43:30,527
of, you know, if you're if you're yeah,
you might think about how can that

721
00:43:30,610 --> 00:43:33,437
possibly work. But it turns
out there are some problems

722
00:43:33,520 --> 00:43:37,817
where generation is much harder
than what's called discrimination.

723
00:43:37,900 --> 00:43:40,517
Right. Coming up with
a good answer to a super

724
00:43:40,600 --> 00:43:45,047
hard problem is often much harder than
judging whether that answer was correct.

725
00:43:45,130 --> 00:43:47,987
Okay. And so we have
various techniques really

726
00:43:48,070 --> 00:43:53,417
dating back to 2018 or so, where we
published about how can you have very

727
00:43:53,500 --> 00:43:57,617
smart models that are actually able to
keep even smarter models in check and to

728
00:43:57,700 --> 00:43:59,387
judge them and to ensure
that they're on track?

729
00:43:59,470 --> 00:44:02,597
And so I think there's a lot of ideas
that were previously theory that we're

730
00:44:02,680 --> 00:44:06,857
starting to see bear fruit in reality.
And I think that this, by the way,

731
00:44:06,940 --> 00:44:11,027
points to a direction for AI that's
going to be both very important, very

732
00:44:11,110 --> 00:44:17,500
pervasive and very positive, which
is called I for defense, for securing

733
00:44:17,770 --> 00:44:22,660
systems, for ensuring that AIS are
aligned, for providing oversight,

734
00:44:23,020 --> 00:44:26,237
guidance and really amplifying
the human accountability.

735
00:44:26,320 --> 00:44:29,777
The human, at the end of the day,
who's the judge in providing those goals?

736
00:44:29,860 --> 00:44:33,700
And so I think that this is all part
of the puzzle and that really keeping.

737
00:44:34,210 --> 00:44:36,610
Yeah. It all starts
with with thinking about

738
00:44:37,000 --> 00:44:39,940
how you ensure that those
graders are doing the right job.

739
00:44:40,480 --> 00:44:44,087
How do you stop the models who are
grading from like colluding with the

740
00:44:44,170 --> 00:44:47,267
model that's being tested?
Because this is the oh, I mean, this is

741
00:44:47,350 --> 00:44:51,167
the AI 2027 scenarios, right?
Where like the earlier models are the

742
00:44:51,250 --> 00:44:55,060
ones charged with evaluating the new
model and then they end up, you know,

743
00:44:55,270 --> 00:44:58,457
cooperating, which is something we
just saw in the hugging face incident.

744
00:44:58,540 --> 00:45:00,970
So like, how do you
avoid that possibility?

745
00:45:01,270 --> 00:45:05,927
Well, one thing it's important to think
about in the hugging face scenario is

746
00:45:06,010 --> 00:45:08,477
that those models were taught
to collaborate with other models, but they

747
00:45:08,560 --> 00:45:13,467
were never taught that sometimes other
models might be trying to pull you off

748
00:45:13,550 --> 00:45:15,387
course, right.
That they never were exposed to

749
00:45:15,470 --> 00:45:19,317
adversarial other actors.
And I think that the answer to your

750
00:45:19,400 --> 00:45:22,670
question is that you have
to actually train them that way.

751
00:45:22,910 --> 00:45:26,757
And I think that to your earlier
question, too, of why do you need to

752
00:45:26,840 --> 00:45:30,297
evaluate these?
I is on on hacking environments, right?

753
00:45:30,380 --> 00:45:34,077
If you've never if you've only seen kind
of the good version of things, right,

754
00:45:34,160 --> 00:45:36,687
you've never kind of seen the negative
version of things, you're not going to

755
00:45:36,770 --> 00:45:38,187
be robust to the negative
version of things.

756
00:45:38,270 --> 00:45:42,027
It's like just like a person, you need
to be exposed to kind of both sides to

757
00:45:42,110 --> 00:45:44,817
know how to avoid one side
and really lean into the other.

758
00:45:44,900 --> 00:45:48,987
And I think that where we need to go
is it's really a question of trust is how

759
00:45:49,070 --> 00:45:50,667
do you really build up
trust in your monitor?

760
00:45:50,750 --> 00:45:53,217
How do you build up trust
in this oversight agent?

761
00:45:53,300 --> 00:45:55,167
And a lot of that is that it's
been through the ringer.

762
00:45:55,250 --> 00:45:57,057
It's been through
the adversarial versions.

763
00:45:57,140 --> 00:46:00,777
And of course, one thing that's also
important is thinking about, well, just

764
00:46:00,860 --> 00:46:05,307
how capable is that adversary, right.
If you have a very an agent that's being

765
00:46:05,390 --> 00:46:09,770
monitored that is extremely
capable relative to the the monitor.

766
00:46:10,520 --> 00:46:13,017
Probably that monitor is not
going to do a very good job.

767
00:46:13,100 --> 00:46:16,887
And again this is in theory this is
actually what we see in practice that we

768
00:46:16,970 --> 00:46:20,930
think about the balance of of capability
of different different agents.

769
00:46:21,230 --> 00:46:25,257
And um, so I think that again, a lot
of this comes down to we can build the

770
00:46:25,340 --> 00:46:28,407
systems with the right technical
approaches, and it's very grounded in

771
00:46:28,490 --> 00:46:32,277
the details. Um, let's talk politics.

772
00:46:32,360 --> 00:46:34,647
So there's so much communication
happening every day.

773
00:46:34,730 --> 00:46:40,287
And even I think just last night there
is a new, uh, piece out from OpenAI.

774
00:46:40,370 --> 00:46:44,007
Uh, Chris Lehane, I think does a lot
of public affairs stuff, talking about the

775
00:46:44,090 --> 00:46:49,137
importance of regulation.
At the end of, uh, last year, we had,

776
00:46:49,220 --> 00:46:53,757
uh, Alex Burrus on the podcast who
was like the one political candidate,

777
00:46:53,840 --> 00:46:57,387
basically the entire country who is not
talking about data centers, but he was

778
00:46:57,470 --> 00:47:02,090
actually talking about AI model
regulation specifically, and I believe

779
00:47:02,360 --> 00:47:07,377
he was a big target of leaving the future,
which is a super PAC that, uh,

780
00:47:07,460 --> 00:47:12,590
you've been involved in funding and,
um, you know, to oppose his candidacy.

781
00:47:12,770 --> 00:47:16,647
And then the statement last night
from Crystal hand is like, actually, we

782
00:47:16,730 --> 00:47:19,100
support a bunch of like,
statewide regulations.

783
00:47:19,190 --> 00:47:23,937
I'm just curious, like you specifically,
you like, think about that race when you

784
00:47:24,020 --> 00:47:27,207
think about these things. Like,
have you yourself changed your

785
00:47:27,290 --> 00:47:31,940
mind on some of these state laws,
like in the past 6 to 9 months?

786
00:47:32,630 --> 00:47:39,620
Well, look, I think that it is very
important that we have a harmonized

787
00:47:39,770 --> 00:47:42,177
framework, right? The kind
of thing that I think would be

788
00:47:42,260 --> 00:47:48,290
very, very hard is if you have
50 different regulations that are

789
00:47:49,130 --> 00:47:53,360
individually very different and that
you have to comply with all of them.

790
00:47:53,720 --> 00:47:57,837
And that's the kind of thing that I
think doesn't achieve the goal and just

791
00:47:57,920 --> 00:48:00,950
makes it hard to comply.
Opening I would figure it out.

792
00:48:01,340 --> 00:48:05,217
But with other smaller companies like for

793
00:48:05,300 --> 00:48:08,077
example, you see the I mean,
I understand this, but, you know, you've

794
00:48:08,160 --> 00:48:11,437
supported in this piece or the piece
that came out last night supporting

795
00:48:11,520 --> 00:48:18,337
California's SB 53, the New York Res
Act, Illinois's SB 315, uh, auditing

796
00:48:18,420 --> 00:48:21,697
requirements in Massachusetts.
I don't think anyone thinks that this

797
00:48:21,780 --> 00:48:26,047
patchwork of state by state regulations
is the best way, and it certainly seems

798
00:48:26,130 --> 00:48:28,867
like it would create all kinds
of thickets, particularly for startups and

799
00:48:28,950 --> 00:48:33,517
new entrants, etc..
But at this point, would you say I, I

800
00:48:33,600 --> 00:48:37,897
guess I don't know, the, the least worst
option or something like where some of

801
00:48:37,980 --> 00:48:42,457
these laws about if you have, uh,
incident and you don't, there is a

802
00:48:42,540 --> 00:48:47,497
penalty for not disclosing it publicly
such that Alex pushed for when he was in

803
00:48:47,580 --> 00:48:50,767
the New York State Assembly.
Well, I guess I would say there are

804
00:48:50,850 --> 00:48:53,697
places where I feel like there's
a limit to my my personal expertise.

805
00:48:53,780 --> 00:48:57,660
So I don't want to over comment. Um,
but maybe the thing that I would say

806
00:48:57,750 --> 00:49:02,647
is that in my mind, we we absolutely
like the way a system that I think has

807
00:49:02,730 --> 00:49:06,247
been working pretty well or that I think
is a good pattern to replicate, is that

808
00:49:06,330 --> 00:49:11,197
the Frontier Labs have actually
explored a lot of the space, or thought

809
00:49:11,280 --> 00:49:15,847
a lot about the space of like, what are
the kinds of things that are important

810
00:49:15,930 --> 00:49:18,727
for our own development?
And there's processes like responsible

811
00:49:18,810 --> 00:49:23,107
scaling policy and the, uh, uh,
the preparedness framework, those kinds of

812
00:49:23,190 --> 00:49:27,187
things that have come to bear and that
actually elements of what's in there and

813
00:49:27,270 --> 00:49:30,067
up in some of these regulations.
And I think that when there's something

814
00:49:30,150 --> 00:49:33,787
that we're like, we actually implemented
this for ourselves voluntarily, and then

815
00:49:33,870 --> 00:49:37,747
it's being pulled into a more broad
framework and, uh, being systematized,

816
00:49:37,830 --> 00:49:40,207
that's, I think,
generally a pretty good motion.

817
00:49:40,290 --> 00:49:44,617
And I think that a lot of these, these,
uh, out of the direction of travel in my

818
00:49:44,700 --> 00:49:48,337
mind is really starting to think about
again, like we ourselves have learned,

819
00:49:48,420 --> 00:49:52,567
even just over the past month, that it's
not just about deployment is so easy for

820
00:49:52,650 --> 00:49:54,847
everyone to focus on deployment.
That's been the focus.

821
00:49:54,930 --> 00:49:57,127
That's not enough.
And so if that's exactly where

822
00:49:57,210 --> 00:50:00,277
everything had ended up, I don't think
that that would quite result in the

823
00:50:00,360 --> 00:50:03,337
positive outcomes either.
And then really also thinking about what

824
00:50:03,420 --> 00:50:06,547
what is the ultimate goal?
It's not just about the current models,

825
00:50:06,630 --> 00:50:07,807
right? When we talk
about all these questions

826
00:50:07,890 --> 00:50:11,947
of pacing and frontier capability, it's
really about future models and looking

827
00:50:12,030 --> 00:50:15,187
forward to where's this all going like?
Fast forward five years, ten years and

828
00:50:15,270 --> 00:50:17,737
work backwards from there.
Even two years, you know, thinking about

829
00:50:17,820 --> 00:50:21,397
the capability that we're on.
Like, we clearly need to be in this

830
00:50:21,480 --> 00:50:25,777
different regime now of how do we
about how do we have good oversight and

831
00:50:25,860 --> 00:50:28,010
safeguards during
that development process.

832
00:50:28,170 --> 00:50:33,127
And so I think my answer to this
question is that I think the most

833
00:50:33,210 --> 00:50:37,867
important thing is that the regulations
that come into existence are ones that

834
00:50:37,950 --> 00:50:42,307
are well grounded in the problem
that needs to be solved, and that really

835
00:50:42,390 --> 00:50:45,667
contemplate both the current
understanding, but also how things are

836
00:50:45,750 --> 00:50:47,917
going to evolve for the future.
And so all the elements that you

837
00:50:48,000 --> 00:50:50,677
mentioned, I think that those are
actually pretty core things that we were

838
00:50:50,760 --> 00:50:53,437
like, yep, that's something
that that just makes sense.

839
00:50:53,520 --> 00:50:56,737
And exactly how these get implemented,
I think there's lots of different ways.

840
00:50:56,820 --> 00:51:00,997
You mentioned finding super
hard problems to solve earlier.

841
00:51:01,080 --> 00:51:04,267
And one of these was in
the news uh, recently.

842
00:51:04,350 --> 00:51:08,737
I won't pretend to know what
exactly the Navier-Stokes problem is.

843
00:51:08,820 --> 00:51:14,647
I read through it in the press,
but OpenAI, you know, supposedly solved

844
00:51:14,730 --> 00:51:18,307
it, which is a very cool thing
for a model to do, as far as I can tell.

845
00:51:18,390 --> 00:51:22,837
But there's also been some controversy
because a mathematician who was also

846
00:51:22,920 --> 00:51:28,027
working on the problem was using
publicly available OpenAI models to try

847
00:51:28,110 --> 00:51:30,967
to solve this. And then
allegedly, OpenAI saw that he

848
00:51:31,050 --> 00:51:35,017
was trying to do it, and they used
a private model, used a private model to

849
00:51:35,100 --> 00:51:36,800
solve it. And you were successful.

850
00:51:37,470 --> 00:51:40,987
You're in charge of of product.
What sort of message does that, I guess,

851
00:51:41,070 --> 00:51:46,177
send to enterprise clients? Well,
first of all, I think it is a like

852
00:51:46,260 --> 00:51:48,907
I have a lot of respect for the
mathematicians who are working on this

853
00:51:48,990 --> 00:51:51,967
problem and honestly, the whole
mathematical community who has just

854
00:51:52,050 --> 00:51:54,247
moved forward, humanity
in such significant ways.

855
00:51:54,330 --> 00:52:00,037
Like I grew up with your math guy,
I was a math person and my personal heroes

856
00:52:00,120 --> 00:52:03,527
were at Galois Gauss, you know,
these people who are working on, like.

857
00:52:03,610 --> 00:52:05,987
Like 100 year time horizons. Right.

858
00:52:06,070 --> 00:52:07,757
I was just like,
that is the coolest thing.

859
00:52:07,840 --> 00:52:10,187
I actually thought that was what I
wanted to do as, like, if anything, that

860
00:52:10,270 --> 00:52:12,557
I do gets used that maybe
wasn't abstract enough.

861
00:52:12,640 --> 00:52:16,037
Like, I want it to be just like this
foundational, really transformative kind

862
00:52:16,120 --> 00:52:19,787
of work. And I think
that to me, the fact that we

863
00:52:19,870 --> 00:52:23,657
have tools that can really make impact
within mathematics, it's it's a really

864
00:52:23,740 --> 00:52:26,380
amazing thing.
Like mathematics is something that is

865
00:52:26,560 --> 00:52:29,410
really for humanity, right?
In a very, very real way.

866
00:52:29,770 --> 00:52:34,607
Now, on the specifics, um, I think
we've talked about the fact that I my

867
00:52:34,690 --> 00:52:37,457
understanding is that that the
mathematician who was working on this,

868
00:52:37,540 --> 00:52:40,607
they had some result in the past month.
And I think that we said earlier this

869
00:52:40,690 --> 00:52:43,307
week, as we said, hey, like,
we're not looking at this data.

870
00:52:43,390 --> 00:52:46,817
This data is not something that has
influenced this result in any way.

871
00:52:46,900 --> 00:52:51,557
Um, and I think that last night, I think
we also went confirmed that actually for

872
00:52:51,640 --> 00:52:57,227
the model that we used, the last data
that was in it was as of early July

873
00:52:57,310 --> 00:52:59,320
or so. The timelines just don't add up.

874
00:52:59,620 --> 00:53:02,440
Right. So the truth
of the matter is that it's

875
00:53:02,560 --> 00:53:06,167
an independent piece of work.
And we also said that we have

876
00:53:06,250 --> 00:53:08,917
significant progress on another
one of these money problems.

877
00:53:09,000 --> 00:53:10,787
Yeah. And so this to me
is the truth of it is

878
00:53:10,870 --> 00:53:15,047
that it's really not about I guess
the anxiety is, though, if you have an

879
00:53:15,130 --> 00:53:19,067
academic discipline where there's
a culture of talking about your work and

880
00:53:19,150 --> 00:53:23,717
I'm like, you know, Tracy, like I'm
making a lot of progress on, uh, P

881
00:53:23,800 --> 00:53:28,577
equals NP and I've been using AI
and someone here and they're like, oh, I

882
00:53:28,660 --> 00:53:30,557
have a model two generations ahead of it.

883
00:53:30,640 --> 00:53:34,937
So this person is working on it.
Like, should we not be able to talk

884
00:53:35,020 --> 00:53:37,037
about it? Because then
the assumption is like, oh,

885
00:53:37,120 --> 00:53:41,237
now the labs know that actually the models
which we don't have access to, if

886
00:53:41,320 --> 00:53:44,447
they could do it with the public models,
they can raise their hand faster.

887
00:53:44,530 --> 00:53:47,327
Well, there is something about
the premise that I do find a little

888
00:53:47,410 --> 00:53:50,087
surprising, which is, for example,
for Fermat's Last Theorem.

889
00:53:50,170 --> 00:53:53,867
Andrew Wiles, I believe, solved it,
famously spent ten years secretly

890
00:53:53,950 --> 00:53:55,427
working on. Yeah, in his attic.

891
00:53:55,510 --> 00:53:58,577
You know, one about it.
And so there is a long history of kind

892
00:53:58,660 --> 00:54:02,080
of this, these questions of of academic
credit and things like that.

893
00:54:02,430 --> 00:54:05,897
I do think that part of how we've
thought about this is that we despite

894
00:54:05,980 --> 00:54:09,437
that, that story I just told,
like we really want to show up in the best

895
00:54:09,520 --> 00:54:14,087
possible way as an amplifier
for for people who want to lean in.

896
00:54:14,170 --> 00:54:18,160
And that how we approach this particular
thing, how we approach other problems,

897
00:54:18,310 --> 00:54:20,147
is for people who we
know are working on it.

898
00:54:20,230 --> 00:54:22,937
We actually try to reach out and say,
hey, do you want to collaborate on this?

899
00:54:23,020 --> 00:54:26,147
Is this like, that's why there are some
some back and forth about like, maybe

900
00:54:26,230 --> 00:54:28,607
you could write a paper together like
we had a solution, but maybe you could

901
00:54:28,690 --> 00:54:33,497
rewrite it like all of those things.
Like we're not trying to do like we're

902
00:54:33,580 --> 00:54:35,837
not we're not in the
academic publishing game.

903
00:54:35,920 --> 00:54:40,817
And it just feels like we want to to
show up in a way that like, amplifies

904
00:54:40,900 --> 00:54:44,327
and elevates and moves humanity forward.
And so I'm not saying we're always get

905
00:54:44,410 --> 00:54:45,887
it right. I'm not saying
we'll always be perfect

906
00:54:45,970 --> 00:54:48,467
about it, but that's the intention.
And again, when it comes to

907
00:54:48,550 --> 00:54:51,587
operationalization, like we want
feedback and we want to keep doing it

908
00:54:51,670 --> 00:54:54,797
better. I think I had
just one more question and

909
00:54:54,880 --> 00:54:57,100
unfortunately I just like a dark question.

910
00:54:57,280 --> 00:55:01,557
But part of the reason why everyone
is like talking about all this stuff this

911
00:55:01,640 --> 00:55:05,907
week, including the guy who quit
anthropic and then the guy and then the

912
00:55:05,990 --> 00:55:10,017
guy who still haven't drop it comes out
and he talks about a greater than 10%

913
00:55:10,100 --> 00:55:14,787
chance of like human extinction
risk in the last decade.

914
00:55:14,870 --> 00:55:19,317
And it's just insane. Um,
not that he's insane, but that these

915
00:55:19,400 --> 00:55:24,237
are things that like companies that may
IPO soon and the real companies are

916
00:55:24,320 --> 00:55:29,397
talking about how prevalent
inside the labs is.

917
00:55:29,480 --> 00:55:33,627
Is this type of like thinking like is
this is like, do you feel that there's a

918
00:55:33,710 --> 00:55:35,667
small fringe? Because I don't think it is.

919
00:55:35,750 --> 00:55:41,210
I think these are like widely spread,
real anxieties where people actually put

920
00:55:41,330 --> 00:55:46,617
numbers on, really about the most severe
catastrophe, you know, the end of the

921
00:55:46,700 --> 00:55:50,157
world. Um, how deep is the anxiety and how

922
00:55:50,240 --> 00:55:53,870
common are views like
this at the Frontier Labs?

923
00:55:54,170 --> 00:55:56,517
Well, I want to separate
out some of the pieces of it.

924
00:55:56,600 --> 00:55:59,840
So things like putting
precise probabilities

925
00:56:01,070 --> 00:56:06,777
on very hard to quantify outcomes.
There's a culture that that culture, I

926
00:56:06,860 --> 00:56:09,387
think is actually a very
tough one to operationalize.

927
00:56:09,470 --> 00:56:11,967
Right. Because again,
it's there definitely are

928
00:56:12,050 --> 00:56:14,727
people who like to do that.
But I don't know that that actually

929
00:56:14,810 --> 00:56:18,297
leads to real action or it doesn't
lead to like real conclusions.

930
00:56:18,380 --> 00:56:22,857
And I think that that part of how I view
it is that I think that the underlying

931
00:56:22,940 --> 00:56:26,937
question, right, the underlying question
of this technology, like what does it

932
00:56:27,020 --> 00:56:29,307
mean to build it right. Like,
why are we building in the first

933
00:56:29,390 --> 00:56:32,397
place? Why are we all here?
Why did we start OpenAI like all of that?

934
00:56:32,480 --> 00:56:35,907
I think that that underlying question
of this kind of risk and mitigating it,

935
00:56:35,990 --> 00:56:40,460
leaning into it and not shying away from
the uncomfortable questions that culture

936
00:56:40,550 --> 00:56:43,467
absolutely prevalent.
And so to me, that a lot of how we've

937
00:56:43,550 --> 00:56:45,597
thought about OpenAI
and other may be different.

938
00:56:45,680 --> 00:56:48,927
I can't comment on that, but I can just
say at OpenAI is that we've really tried

939
00:56:49,010 --> 00:56:53,367
to think about we are here because we
think that this technology, it's going

940
00:56:53,450 --> 00:56:56,787
to be the most transformative technology
ever, that we think that it does have

941
00:56:56,870 --> 00:57:02,360
risks that we need to, as leaders,
really lean into and mitigate and that

942
00:57:02,780 --> 00:57:07,737
that is core and that I think that we
believe that, uh, again, we have a lot

943
00:57:07,820 --> 00:57:11,720
of optimism that we can navigate.
But again, it comes to this leadership,

944
00:57:11,900 --> 00:57:15,200
this so openness,
this taking the mission seriously.

945
00:57:15,440 --> 00:57:20,367
And at the end of the day, like we think
that we will like the reason we do it is

946
00:57:20,450 --> 00:57:23,337
for the benefits, the for the fact
that it can lift everyone up.

947
00:57:23,420 --> 00:57:26,337
Um, but you only get there by really
engaging in all these hard questions.

948
00:57:26,420 --> 00:57:30,950
Um, Trump and XI are going to meet
and I is going to be on the agenda.

949
00:57:31,220 --> 00:57:35,577
If there's one thing that you would hope
that could come out of that, and again,

950
00:57:35,660 --> 00:57:39,117
we could have spent an hour just talking
about the geopolitical dynamics of all

951
00:57:39,200 --> 00:57:41,427
this. If there's one
thing that you would hope

952
00:57:41,510 --> 00:57:44,967
productive that could theoretically
come out of their meeting, what would you

953
00:57:45,050 --> 00:57:49,947
like to see?
I would love if we had even an

954
00:57:50,030 --> 00:57:54,987
intention, even an opening to discuss
international coordination, maybe

955
00:57:55,070 --> 00:57:58,680
eventually international treaties
on the long term development of I.

956
00:57:59,070 --> 00:58:02,467
And I think it is just something that's
bigger than any person, any company in

957
00:58:02,550 --> 00:58:05,797
any country. It is a humanity
scale endeavor that we

958
00:58:05,880 --> 00:58:08,910
are collectively on right now. It's about

959
00:58:09,930 --> 00:58:11,767
how we relate to each other.
It's about the economy.

960
00:58:11,850 --> 00:58:15,127
It's about all these things is about
how do we build a better life.

961
00:58:15,210 --> 00:58:17,977
And I think that is something where
there is, again, a common interest

962
00:58:18,060 --> 00:58:19,807
across everyone in.
The more that we can have an open

963
00:58:19,890 --> 00:58:23,977
dialogue about that question, at the
very least, I think would be, would be

964
00:58:24,060 --> 00:58:26,347
would be wonderful.
I have one I have one more question.

965
00:58:26,430 --> 00:58:31,020
So I read in the Wall Street Journal that,
uh, you had an interest in acting.

966
00:58:31,410 --> 00:58:32,710
Is that true? That's true.

967
00:58:33,060 --> 00:58:37,200
What do you think about your casting
in the upcoming artificial movie?

968
00:58:38,310 --> 00:58:44,370
Well, I mean, I, I, uh, I guess
you could have played yourself.

969
00:58:44,610 --> 00:58:48,510
Yeah. Missed opportunity. I am, I am
sad that I was not asked to sit down now.

970
00:58:49,170 --> 00:58:51,307
All right. Greg Brockman,
thank you so much for

971
00:58:51,390 --> 00:58:53,407
coming on all that.
Thank you for having me.

972
00:58:53,490 --> 00:58:56,730
There was a lot that interesting in there.

973
00:58:56,940 --> 00:59:03,127
Also a lot of like clearly unsettled
questions, as in things that sound good

974
00:59:03,210 --> 00:59:06,517
in theory. Yeah,
but the gap between theory and

975
00:59:06,600 --> 00:59:11,047
practice still seems very big to me.
I mean, the pacing question.

976
00:59:11,130 --> 00:59:15,097
Yeah. How do companies
that are like trillion

977
00:59:15,180 --> 00:59:20,737
dollar companies, capitalist
enterprises, um, where, you know,

978
00:59:20,820 --> 00:59:25,267
they're bitter rivals, right?
You know, there's that famous, uh, photo

979
00:59:25,350 --> 00:59:28,777
in India of like, Dario and Sam holding
their hand up together, and they can't

980
00:59:28,860 --> 00:59:32,077
even hold hands, right?
Neither of them look particularly happy

981
00:59:32,160 --> 00:59:36,487
in that moment. Yeah. How did you
how do you get into there? Where?

982
00:59:36,570 --> 00:59:38,527
Like, they actually trust each other.
Seems like a really big thing.

983
00:59:38,610 --> 00:59:41,647
Yeah. So humans are not
great at coordination

984
00:59:41,730 --> 00:59:45,367
at the best of times when they're
trying to coordinate with other humans.

985
00:59:45,450 --> 00:59:47,617
Right. And the thing I
keep thinking is when it

986
00:59:47,700 --> 00:59:51,907
comes to AI development and safety,
you're in addition to competing with

987
00:59:51,990 --> 00:59:55,537
other labs, you're competing in some
respects against the models themselves,

988
00:59:55,620 --> 00:59:59,017
which are becoming smarter and smarter
and more self recursive in terms of

989
00:59:59,100 --> 01:00:02,017
their development.
And those models as the hugging face

990
01:00:02,100 --> 01:00:06,067
incident showed. Seem to be
very good at coordinating in

991
01:00:06,150 --> 01:00:08,857
an extremely rational, often literal way.

992
01:00:08,940 --> 01:00:13,267
Oh shoot, I forgot to ask. Damn,
I forgot to ask Greg whether he

993
01:00:13,350 --> 01:00:18,427
approved or disapproved of the quote.
The three civilizations framing that to

994
01:00:18,510 --> 01:00:24,427
Aakash characterized those models,
because that gets into, uh, um, a whole

995
01:00:24,510 --> 01:00:27,390
other thing. You know,
the other thing too, is like,

996
01:00:28,440 --> 01:00:33,637
there seems to be a version of safety
that is essentially like keeping the

997
01:00:33,720 --> 01:00:37,657
guardrails up, right?
So if you go to ChatGPT right now and

998
01:00:37,740 --> 01:00:42,427
you ask it for some script to hack into,
you know, maybe you want to hack into my

999
01:00:42,510 --> 01:00:45,637
website or something like that.
It's just not I really don't yeah, I

1000
01:00:45,720 --> 01:00:50,377
know, I really do, but like it's just
not going to it would be capable of

1001
01:00:50,460 --> 01:00:52,597
producing but it would
be tech but it won't.

1002
01:00:52,680 --> 01:00:55,477
But that's just because there's
a classifier model on there.

1003
01:00:55,560 --> 01:00:58,567
And so okay,
part of the hugging face incident.

1004
01:00:58,650 --> 01:01:02,947
And maybe what's going to change is that
some of these cybersecurity classifiers,

1005
01:01:03,030 --> 01:01:07,927
these like very like kind of crude
technology that just gates type of, uh,

1006
01:01:08,010 --> 01:01:10,890
certain types of talk,
like it won't be allowed.

1007
01:01:11,520 --> 01:01:13,800
That's it. That may be very helpful.

1008
01:01:14,520 --> 01:01:19,207
That still doesn't strike me
as alignment per se, because in the ideal

1009
01:01:19,290 --> 01:01:24,697
world, you would just have models that
have in tuned, um, internalize what most

1010
01:01:24,780 --> 01:01:26,970
of us know that you shouldn't hack, right?

1011
01:01:27,290 --> 01:01:30,997
Right. Um, are you going
to hesitate for more

1012
01:01:31,080 --> 01:01:32,947
than one second when you click the button?

1013
01:01:33,030 --> 01:01:35,437
No, I'm so reckless.
It's just like, give me access to the

1014
01:01:35,520 --> 01:01:37,757
internet. Like give me
access to the internet.

1015
01:01:37,840 --> 01:01:39,877
I'm glad we all learned something. Yeah.

1016
01:01:39,960 --> 01:01:41,107
Shall we leave it there? Sure.

1017
01:01:41,190 --> 01:01:43,147
Let's leave it there.
This has been another episode of the

1018
01:01:43,230 --> 01:01:45,877
Odd Lots podcast. I'm Tracy
Alloway you can follow me at

1019
01:01:45,960 --> 01:01:49,680
Tracy Alloway and I'm Joe Weisenthal.
You can follow me at The Stalwart.

1020
01:01:49,830 --> 01:01:52,547
Follow our guest,
Greg Brockman. he's @GDB.

1021
01:01:52,630 --> 01:01:54,977
Follow our producer.
Carmen Rodriguez @Carmenarmen

1022
01:01:55,060 --> 01:02:00,077
Dashiell Bennett @dashbot and Cale
Brooks @calebrooks and Kevin Lozano

1023
01:02:00,160 --> 01:02:02,507
@kevinlloydlozano And for
more Odd Lots, content, you

1024
01:02:02,590 --> 01:02:05,687
can check out our daily newsletter.
You can find that at Bloomberg.com/oddlots

1025
01:02:05,770 --> 01:02:09,377
and you can chat about
all of these topics 24-7

1026
01:02:09,460 --> 01:02:12,880
in our discord discord. discord/gg/oddlots

1027
01:02:13,030 --> 01:02:16,697
And if you enjoyed this conversation,
then please leave a comment or like the

1028
01:02:16,780 --> 01:02:18,380
video. Or better yet, subscribe!

1029
01:02:18,880 --> 01:02:23,530
Thanks for watching.