1
00:00:00,160 --> 00:00:04,957
One of the world leaders in artificial
intelligence is warning the industry must

2
00:00:05,040 --> 00:00:11,277
slow down now. Anthropic CEO Daario Amade
writes, "With AI advancing so quickly,

3
00:00:11,360 --> 00:00:14,663
there is a risk of humanity
losing control of the technology,

4
00:00:14,746 --> 00:00:16,717
possibly leading to cyber attacks and

5
00:00:16,800 --> 00:00:21,560
bioteterrorism." Amade spoke to CNN
exclusively today with our Anderson

6
00:00:21,643 --> 00:00:23,197
Cooper. Here's a part of

7
00:00:23,280 --> 00:00:27,803
Anderson's interview. Obviously, the
reason we're talking is these headlines

8
00:00:27,886 --> 00:00:29,197
that have come out. Jac

9
00:00:29,280 --> 00:00:33,677
Jacob Coxin who recently resigned from
anthropic made headlines last week saying

10
00:00:33,760 --> 00:00:37,504
the people building AI earnestly believe
that it could kill us all by the end of

11
00:00:37,587 --> 00:00:38,877
the decade. There's a current

12
00:00:38,960 --> 00:00:42,153
employee Evan Hubb Hubbinger
an alignment science lead.

13
00:00:42,237 --> 00:00:44,477
He agreed saying we really do earnestly

14
00:00:44,560 --> 00:00:48,324
believe AI could kill all humans.
I personally think it is greater than 10%

15
00:00:48,408 --> 00:00:49,197
chance within the

16
00:00:49,280 --> 00:00:54,724
next decade. Do you believe earnestly
believe that AI could kill all humans?

17
00:00:54,807 --> 00:00:55,597
And and what

18
00:00:55,680 --> 00:01:02,427
percent chance do you think it is? So,
you know, I I you know, it's it's funny.

19
00:01:02,511 --> 00:01:06,637
I I agree with Jacob much more
than I disagree with him, you know,

20
00:01:06,720 --> 00:01:09,117
and and and and it's
it's you know, it's it's

21
00:01:09,200 --> 00:01:12,913
an interesting resignation because
when he left, he said, you know,

22
00:01:12,997 --> 00:01:14,557
I think Anthropic is the most

23
00:01:14,640 --> 00:01:18,688
responsible player, right? I think
Anthropic is the most aware of these

24
00:01:18,771 --> 00:01:19,677
issues. He wasn't

25
00:01:19,760 --> 00:01:26,277
calling out us. He was calling out
the dynamic of the the industry as a whole

26
00:01:26,360 --> 00:01:31,116
moving too fast. Now, now I I you know,
I I I think I already said, you know,

27
00:01:31,199 --> 00:01:33,437
these sort of unconditional probabilities,

28
00:01:33,520 --> 00:01:37,955
you know, I I you know, I used to kind of
talk in that way cuz it's kind of an easy

29
00:01:38,038 --> 00:01:38,717
way to express

30
00:01:38,800 --> 00:01:43,353
I think there's some chance that things
go wrong, but you know, but you know,

31
00:01:43,436 --> 00:01:44,557
but but but it's not

32
00:01:44,640 --> 00:01:49,260
it's not all that high. But again, I think
it's more illuminating to think in terms

33
00:01:49,343 --> 00:01:49,997
of, you know,

34
00:01:50,080 --> 00:01:54,365
how do you decompose those probabilities?
What are the paths in which things go well

35
00:01:54,448 --> 00:01:55,197
and what are the

36
00:01:55,280 --> 00:01:58,954
paths in which things go poorly?
And so instead of saying, you know,

37
00:01:59,038 --> 00:02:00,557
it's a 10% chance that sounds

38
00:02:00,640 --> 00:02:05,263
like it's a roll of the dice, I think
it's much more illuminating to say, well,

39
00:02:05,346 --> 00:02:08,492
that forks into different paths
and if we take the right paths,

40
00:02:08,575 --> 00:02:10,877
then the chance of something
going wrong is very

41
00:02:10,960 --> 00:02:15,434
low. If we take the wrong paths, then the
chance of something going wrong could be

42
00:02:15,517 --> 00:02:16,157
could be even

43
00:02:16,240 --> 00:02:20,360
higher than that. And so let's focus
on our agency and our ability to take

44
00:02:20,443 --> 00:02:21,837
the right paths instead of

45
00:02:21,920 --> 00:02:26,471
the wrong paths. And I think that
the way to take the right paths is for

46
00:02:26,554 --> 00:02:27,437
the industry to

47
00:02:27,520 --> 00:02:31,470
work together. Again, you know,
we'll we'll talk about uh uh the proposal

48
00:02:31,554 --> 00:02:32,797
that I've put out today,

49
00:02:32,880 --> 00:02:36,582
but at the end of the day, the industry
has to work together and it's harder,

50
00:02:36,666 --> 00:02:40,873
but to some extent, the world has
to work together. If we go too slow,

51
00:02:40,956 --> 00:02:43,357
I still believe that the wrong people

52
00:02:43,440 --> 00:02:46,364
will be in charge of the
technology and and and you know,

53
00:02:46,448 --> 00:02:48,317
that again will bring the probability

54
00:02:48,400 --> 00:02:53,250
of things going wrong very high.
So what you're proposing is called pacing

55
00:02:53,333 --> 00:02:54,717
the frontier of of the

56
00:02:54,800 --> 00:02:58,325
AI industry. And you write we must
slow the pace at which we improve

57
00:02:58,408 --> 00:02:59,917
the capabilities of AI models.

58
00:03:00,000 --> 00:03:04,493
Progress will seem fast and we must
make wise use of the time we gain.

59
00:03:04,576 --> 00:03:05,997
And there's three parts

60
00:03:06,080 --> 00:03:10,271
to this. One is embedded evaluators.
Part two is democratic coordination.

61
00:03:10,355 --> 00:03:11,677
And part three is global

62
00:03:11,760 --> 00:03:16,325
coordination. Sam Alman at OpenAI just
endorsed the idea of embedded evaluators.

63
00:03:16,408 --> 00:03:17,197
Essentially you

64
00:03:17,280 --> 00:03:23,452
are suggesting I don't know if are they
observers inside your company and these

65
00:03:23,535 --> 00:03:24,877
other AI companies

66
00:03:24,960 --> 00:03:30,275
ideally globally but certainly in the US
who would monitor it they would have

67
00:03:30,358 --> 00:03:31,677
employee access what

68
00:03:31,760 --> 00:03:36,703
power would they have? Yeah. So,
so be before I even get to that,

69
00:03:36,787 --> 00:03:38,637
I'm I'm I'm just I'm very

70
00:03:38,720 --> 00:03:43,539
grateful to see um you know, support for
this proposal just in the first few hours

71
00:03:43,622 --> 00:03:48,253
of having put it out by by Sam Alman,
by Elon Musk, by uh you know,

72
00:03:48,336 --> 00:03:51,037
various legislators and and government

73
00:03:51,120 --> 00:03:53,426
officials. I think it's
a positive sign about, you know,

74
00:03:53,509 --> 00:03:54,877
our ability to make something that

75
00:03:54,960 --> 00:04:00,691
looks like this plan work. But but to
get to your question um yeah embedded

76
00:04:00,774 --> 00:04:02,397
evaluators so there is

77
00:04:02,480 --> 00:04:07,731
precedent for this um if you look at how
banks are regulated right there's there's

78
00:04:07,815 --> 00:04:08,317
all kinds

79
00:04:08,400 --> 00:04:12,546
of you know there's all kinds of rules
about you know how banks are allowed to do

80
00:04:12,629 --> 00:04:13,277
their business

81
00:04:13,360 --> 00:04:17,077
right because you know we we don't
want to have financial crisis and one of

82
00:04:17,160 --> 00:04:17,837
the things that

83
00:04:17,920 --> 00:04:22,346
are done in some cases is I think they're
called supervisors in the financial

84
00:04:22,429 --> 00:04:23,517
industry but they're

85
00:04:23,600 --> 00:04:28,246
they're folks who are independent of a
company who are embedded in the company

86
00:04:28,329 --> 00:04:29,277
daytoday to check

87
00:04:29,360 --> 00:04:34,997
their practices. Uh and I think we can
do the same thing in I think we can do

88
00:04:35,080 --> 00:04:36,557
the same thing in AI.

89
00:04:36,640 --> 00:04:40,383
The other thing you are calling for in
this three-part proposal is is democratic

90
00:04:40,466 --> 00:04:40,957
coordination

91
00:04:41,040 --> 00:04:45,365
which is essentially democracy all
the companies that are under democracies

92
00:04:45,448 --> 00:04:46,717
having these evaluators

93
00:04:46,800 --> 00:04:51,915
having this coordination and and probably
the most important because the argument

94
00:04:51,998 --> 00:04:52,557
is well if

95
00:04:52,640 --> 00:04:57,010
the US or companies are the only ones
regulating themselves what about does

96
00:04:57,094 --> 00:04:58,317
China get ahead you're

97
00:04:58,400 --> 00:05:04,703
calling for global coordination that's
obviously a lot more difficult yes yes I

98
00:05:04,787 --> 00:05:05,997
you know in this

99
00:05:06,080 --> 00:05:10,560
three-step proposal. Each step is more
difficult than than the one before.

100
00:05:10,643 --> 00:05:15,223
And Enthropic was kind of getting the ball
rolling by committing to the first step,

101
00:05:15,307 --> 00:05:16,877
but we shouldn't kid ourselves

102
00:05:16,960 --> 00:05:22,596
that these next two steps are going to be
more difficult and and we may not succeed.

103
00:05:22,679 --> 00:05:26,197
And and so my my proposal
was a proposal that we should try.

104
00:05:26,280 --> 00:05:29,117
Now again from all
the agreement from others that

105
00:05:29,200 --> 00:05:33,537
I've seen in just the first few hours
of putting out the essay that creates some

106
00:05:33,621 --> 00:05:34,477
optimism but that

107
00:05:34,560 --> 00:05:41,028
needs to follow up with results. So yes,
pacing among dem companies in democratic

108
00:05:41,111 --> 00:05:41,837
countries.

109
00:05:41,920 --> 00:05:48,920
Now the one of the important things that
that I need to call for here is you know

110
00:05:49,236 --> 00:05:50,237
as we get to

111
00:05:50,320 --> 00:05:55,182
the pace of model releases right
that's commercial activity and in order

112
00:05:55,265 --> 00:05:57,037
for companies to coordinate

113
00:05:57,120 --> 00:06:02,003
directly on commercial activity they
need to do so working with the government

114
00:06:02,087 --> 00:06:03,277
and so the one thing

115
00:06:03,360 --> 00:06:07,066
that we're asking for from government
right now right I mean there's also

116
00:06:07,150 --> 00:06:08,157
this question of what

117
00:06:08,240 --> 00:06:12,157
should the regulations be but there's
something even simpler that the government

118
00:06:12,240 --> 00:06:12,957
can do right now

119
00:06:13,040 --> 00:06:19,073
which is it can sit down in the room and
with all the industry players and we can

120
00:06:19,156 --> 00:06:19,677
all talk

121
00:06:19,760 --> 00:06:24,857
together and the government can make sure
that we're you know we're we're not doing

122
00:06:24,941 --> 00:06:29,059
anything nefarious from the perspective
of antitrust and collusion.

123
00:06:29,142 --> 00:06:31,357
We're just trying to make our system

124
00:06:31,440 --> 00:06:34,728
safe and to work together
on making the system safe.

125
00:06:34,812 --> 00:06:37,517
And so I think that's
an important role for

126
00:06:37,600 --> 00:06:41,929
the government to have.
Jacob Coxin joins us now. Um, Jacob,

127
00:06:42,012 --> 00:06:44,797
I appreciate you you being with us. Um,

128
00:06:44,880 --> 00:06:51,491
first of all, I assume you are saying
literally that you believe uh that AI

129
00:06:51,575 --> 00:06:57,277
could kill us all by the end
of the decade. How would that happen?

130
00:06:57,360 --> 00:07:02,077
Like what would that actually
look like? Yeah. So, the

131
00:07:02,160 --> 00:07:05,122
precise scenario sounds a little
bit like science fiction.

132
00:07:05,205 --> 00:07:07,117
Uh it sounds like something that's not

133
00:07:07,200 --> 00:07:08,798
real, but I think it
is frighteningly real.

134
00:07:08,881 --> 00:07:10,557
And it's it's easiest
to understand it if you

135
00:07:10,640 --> 00:07:15,286
think about real things that happened
two months ago. So two months ago,

136
00:07:15,369 --> 00:07:16,797
o open AI agents uh AIS

137
00:07:16,880 --> 00:07:20,333
hacked into third party infrastructure
entirely of their own accord.

138
00:07:20,416 --> 00:07:21,997
And this was like a concentrated

139
00:07:22,080 --> 00:07:25,595
hacking spree that they carried
out of their own valition.

140
00:07:25,678 --> 00:07:28,077
And I think that if you extrapolate into

141
00:07:28,160 --> 00:07:32,256
the future the level of capabilities
of these AIs with the same independent

142
00:07:32,340 --> 00:07:33,037
valition, they

143
00:07:33,120 --> 00:07:38,417
could cause extreme havoc. for example,
hacking critical infrastructure,

144
00:07:38,500 --> 00:07:39,837
building extinction

145
00:07:39,920 --> 00:07:45,406
level bioweapons. There's a lot of uh
ways that the AI could actuate itself in

146
00:07:45,489 --> 00:07:46,477
the world. I is

147
00:07:46,560 --> 00:07:50,158
it specifically when did your feelings
about this technology change?

148
00:07:50,242 --> 00:07:51,837
Was it something about the rate

149
00:07:51,920 --> 00:07:56,884
of progress that you've seen? Yeah. So I
think it's really important to just look

150
00:07:56,968 --> 00:08:01,237
at the rate of progress and say
for example in areas like coding or math.

151
00:08:01,320 --> 00:08:03,517
A couple years ago these AIs were just

152
00:08:03,600 --> 00:08:06,335
about helping humans a little bit.
They could give you suggestions.

153
00:08:06,418 --> 00:08:07,597
Now they're close to replacing

154
00:08:07,680 --> 00:08:11,764
them. We still need humans at the moment
but quite plausibly within a year we'll no

155
00:08:11,848 --> 00:08:12,317
longer need

156
00:08:12,400 --> 00:08:18,144
humans for doing research in many areas.
Only on Tuesday OpenAI solved a millennium

157
00:08:18,227 --> 00:08:19,197
problem. One of

158
00:08:19,280 --> 00:08:23,261
the biggest unsolved open problems
in mathematics purely autonomously using

159
00:08:23,345 --> 00:08:24,237
an AI. And I think

160
00:08:24,320 --> 00:08:28,381
what's most scary is if AI is used
to make itself more intelligent.

161
00:08:28,464 --> 00:08:30,237
So you can take an AI and give

162
00:08:30,320 --> 00:08:34,544
it the problem of AI research. Just like
currently they've given it math research.

163
00:08:34,627 --> 00:08:35,437
You could make it

164
00:08:35,520 --> 00:08:38,540
do the tasks that we're currently doing.
And then you get what's called

165
00:08:38,624 --> 00:08:39,677
an intelligence explosion.

166
00:08:39,760 --> 00:08:43,387
The AI just gets smarter and smarter with
no human involvement necessary until you

167
00:08:43,471 --> 00:08:44,157
have a thing that

168
00:08:44,240 --> 00:08:49,281
is vastly smarter than than humans.
And this could be coming very soon.

169
00:08:49,364 --> 00:08:53,077
We mentioned a former colleague
of yours currently works at Anthropic.

170
00:08:53,160 --> 00:08:55,357
basically endorsing
the assessment that AI could

171
00:08:55,440 --> 00:08:58,745
potentially kill everyone within a decade.
In a follow-up post, he said,

172
00:08:58,828 --> 00:09:01,563
"To be clear, as we say
in our in our latest risk report,

173
00:09:01,646 --> 00:09:04,077
I think the risk from present
models is low. What

174
00:09:04,160 --> 00:09:08,534
I'm worried about is super intelligence
arising from recursive self-improvement

175
00:09:08,617 --> 00:09:09,437
as we've said is

176
00:09:09,520 --> 00:09:12,682
happening faster than than we thought.
I assume that's what you the recursive

177
00:09:12,766 --> 00:09:13,357
self-improvement

178
00:09:13,440 --> 00:09:18,255
is what you've just spoken about." Um
you do you agree with him that the current

179
00:09:18,338 --> 00:09:19,357
models the risk is

180
00:09:19,440 --> 00:09:25,101
low but it's it's just the speed with
which this is now improving. Exactly.

181
00:09:25,185 --> 00:09:28,807
I completely agree. It's right
now there's no risk of extinction.

182
00:09:28,890 --> 00:09:30,957
The current models
the worst they can do is

183
00:09:31,040 --> 00:09:35,175
maybe hack into something
potentially cause a lot of damages in

184
00:09:35,258 --> 00:09:37,117
in infrastructure but there's

185
00:09:37,200 --> 00:09:41,159
no they're not intelligent enough to
outsmart us at the level that would lead

186
00:09:41,243 --> 00:09:42,157
to to extinction. I

187
00:09:42,240 --> 00:09:47,445
think what's just crazy is to look at the
the rate of progress and there is a very

188
00:09:47,528 --> 00:09:48,477
real possibility

189
00:09:48,560 --> 00:09:53,217
that in the immediate future these are
years that like next year the year after

190
00:09:53,300 --> 00:09:53,757
recursive

191
00:09:53,840 --> 00:09:58,027
self-improvement will happen and we'll
enter the phase of Evans post where he

192
00:09:58,110 --> 00:09:58,637
argues that

193
00:09:58,720 --> 00:10:03,349
there's a chance we could all die like
that that is coming soon you know a lot

194
00:10:03,432 --> 00:10:04,557
of the these CEOs of

195
00:10:04,640 --> 00:10:09,026
these companies will you know will go
to Congress and say hey we would love

196
00:10:09,109 --> 00:10:10,397
regulation anthropic in

197
00:10:10,480 --> 00:10:13,967
their new statement right now says says
the world would benefit from the industry

198
00:10:14,051 --> 00:10:14,717
adopting a lawful

199
00:10:14,800 --> 00:10:19,340
verifiable way to work together
to pace how we release powerful models.

200
00:10:19,423 --> 00:10:20,317
Um you know the

201
00:10:20,400 --> 00:10:25,854
argument it's an arms race if the US if
a US company doesn't achieve the best AI

202
00:10:25,937 --> 00:10:30,057
a Chinese company or some
bad actors out there might.

203
00:10:30,140 --> 00:10:34,637
Do you believe that uh that
these companies actually do

204
00:10:34,720 --> 00:10:38,408
want regulation or is it lip service?
They they completely do.

205
00:10:38,492 --> 00:10:40,477
This is completely honest. Just in

206
00:10:40,560 --> 00:10:43,797
the same way that the response
to Evans post was sort of surprise.

207
00:10:43,880 --> 00:10:45,357
It was like this can't be real.

208
00:10:45,440 --> 00:10:49,313
Surely a guy working on this technology
can't actually believe there's this level

209
00:10:49,396 --> 00:10:49,997
of danger. No,

210
00:10:50,080 --> 00:10:53,325
that's completely genuine. And these
people are also completely genuine when

211
00:10:53,408 --> 00:10:54,157
they are begging to

212
00:10:54,240 --> 00:10:58,347
be regulated. This is a genuine. And they
they find themselves in this scenario

213
00:10:58,430 --> 00:10:59,037
where they're

214
00:10:59,120 --> 00:11:03,477
compelled to race towards building
a deadly technology and they would they

215
00:11:03,560 --> 00:11:04,317
would love for

216
00:11:04,400 --> 00:11:08,283
some sort of international body to allow
them to to approach it at a reasonable

217
00:11:08,367 --> 00:11:09,037
pace. They just

218
00:11:09,120 --> 00:11:12,182
don't trust that the people around them
are going to get there in a safe way.

219
00:11:12,266 --> 00:11:12,877
So they feel like

220
00:11:12,960 --> 00:11:16,987
they have to race there, too.
You you could have stayed quiet. I mean,

221
00:11:17,071 --> 00:11:18,397
I'm sure you had a great

222
00:11:18,480 --> 00:11:22,481
job being paid, I imagine, very well.
You're obviously incredibly smart

223
00:11:22,564 --> 00:11:23,517
and mathematician.

224
00:11:23,600 --> 00:11:29,887
Um, why did you decide to speak out?
Well, I initially just decided to leave.

225
00:11:29,970 --> 00:11:30,797
I thought I

226
00:11:30,880 --> 00:11:35,493
can't be part of this anymore. But then
I uh I realized I could probably make some

227
00:11:35,576 --> 00:11:37,677
use of it. I did not expect
it to go this viral. So,

228
00:11:37,760 --> 00:11:39,517
what I did is I wrote
a Twitter thread out. I

229
00:11:39,600 --> 00:11:42,887
wrote it in a Google doc. I drafted it
with a friend. We we collaborated. Yeah,

230
00:11:42,971 --> 00:11:43,357
I had a few

231
00:11:43,440 --> 00:11:47,861
uh there were a few friends I discussed uh
the best way of expressing myself with um

232
00:11:47,945 --> 00:11:48,237
after I

233
00:11:48,320 --> 00:11:52,020
decided to decided to leave. Uh and then
I also got some friends to to retweet

234
00:11:52,104 --> 00:11:54,404
the thing. I was like, let's
try and make this a bit viral.

235
00:11:54,487 --> 00:11:56,317
But clearly there was latent
demand for this thing

236
00:11:56,400 --> 00:12:02,230
to absolutely explode. Like uh I've never
seen an AI safety tweet go this crazy.

237
00:12:02,313 --> 00:12:03,117
I think it's

238
00:12:03,200 --> 00:12:06,037
because people saw the uh the cyber
attacks in the last couple of months.

239
00:12:06,120 --> 00:12:06,877
They're also starting

240
00:12:06,960 --> 00:12:09,757
to internalize the rate of progress.
People are starting to realize this

241
00:12:09,840 --> 00:12:13,261
is real technology. This is not like hype.
This isn't going away anytime soon.

242
00:12:13,344 --> 00:12:15,277
This is only going to get more crazy. It's

243
00:12:15,360 --> 00:12:18,507
been getting crazy for the last 6 months.
And I think there was that latent

244
00:12:18,591 --> 00:12:19,197
understanding of

245
00:12:19,280 --> 00:12:22,797
the severity of the situation that meant
that this was like sort of a a match that

246
00:12:22,880 --> 00:12:26,454
lit kind of a a flame. Like I was very
shocked by the extent that how quickly it

247
00:12:26,537 --> 00:12:27,370
went viral.