WEBVTT

00:00:00.160 --> 00:00:04.957
One of the world leaders in artificial
intelligence is warning the industry must

00:00:05.040 --> 00:00:11.277
slow down now. Anthropic CEO Daario Amade
writes, "With AI advancing so quickly,

00:00:11.360 --> 00:00:14.663
there is a risk of humanity
losing control of the technology,

00:00:14.746 --> 00:00:16.717
possibly leading to cyber attacks and

00:00:16.800 --> 00:00:21.560
bioteterrorism." Amade spoke to CNN
exclusively today with our Anderson

00:00:21.643 --> 00:00:23.197
Cooper. Here's a part of

00:00:23.280 --> 00:00:27.803
Anderson's interview. Obviously, the
reason we're talking is these headlines

00:00:27.886 --> 00:00:29.197
that have come out. Jac

00:00:29.280 --> 00:00:33.677
Jacob Coxin who recently resigned from
anthropic made headlines last week saying

00:00:33.760 --> 00:00:37.504
the people building AI earnestly believe
that it could kill us all by the end of

00:00:37.587 --> 00:00:38.877
the decade. There's a current

00:00:38.960 --> 00:00:42.153
employee Evan Hubb Hubbinger
an alignment science lead.

00:00:42.237 --> 00:00:44.477
He agreed saying we really do earnestly

00:00:44.560 --> 00:00:48.324
believe AI could kill all humans.
I personally think it is greater than 10%

00:00:48.408 --> 00:00:49.197
chance within the

00:00:49.280 --> 00:00:54.724
next decade. Do you believe earnestly
believe that AI could kill all humans?

00:00:54.807 --> 00:00:55.597
And and what

00:00:55.680 --> 00:01:02.427
percent chance do you think it is? So,
you know, I I you know, it's it's funny.

00:01:02.511 --> 00:01:06.637
I I agree with Jacob much more
than I disagree with him, you know,

00:01:06.720 --> 00:01:09.117
and and and and it's
it's you know, it's it's

00:01:09.200 --> 00:01:12.913
an interesting resignation because
when he left, he said, you know,

00:01:12.997 --> 00:01:14.557
I think Anthropic is the most

00:01:14.640 --> 00:01:18.688
responsible player, right? I think
Anthropic is the most aware of these

00:01:18.771 --> 00:01:19.677
issues. He wasn't

00:01:19.760 --> 00:01:26.277
calling out us. He was calling out
the dynamic of the the industry as a whole

00:01:26.360 --> 00:01:31.116
moving too fast. Now, now I I you know,
I I I think I already said, you know,

00:01:31.199 --> 00:01:33.437
these sort of unconditional probabilities,

00:01:33.520 --> 00:01:37.955
you know, I I you know, I used to kind of
talk in that way cuz it's kind of an easy

00:01:38.038 --> 00:01:38.717
way to express

00:01:38.800 --> 00:01:43.353
I think there's some chance that things
go wrong, but you know, but you know,

00:01:43.436 --> 00:01:44.557
but but but it's not

00:01:44.640 --> 00:01:49.260
it's not all that high. But again, I think
it's more illuminating to think in terms

00:01:49.343 --> 00:01:49.997
of, you know,

00:01:50.080 --> 00:01:54.365
how do you decompose those probabilities?
What are the paths in which things go well

00:01:54.448 --> 00:01:55.197
and what are the

00:01:55.280 --> 00:01:58.954
paths in which things go poorly?
And so instead of saying, you know,

00:01:59.038 --> 00:02:00.557
it's a 10% chance that sounds

00:02:00.640 --> 00:02:05.263
like it's a roll of the dice, I think
it's much more illuminating to say, well,

00:02:05.346 --> 00:02:08.492
that forks into different paths
and if we take the right paths,

00:02:08.575 --> 00:02:10.877
then the chance of something
going wrong is very

00:02:10.960 --> 00:02:15.434
low. If we take the wrong paths, then the
chance of something going wrong could be

00:02:15.517 --> 00:02:16.157
could be even

00:02:16.240 --> 00:02:20.360
higher than that. And so let's focus
on our agency and our ability to take

00:02:20.443 --> 00:02:21.837
the right paths instead of

00:02:21.920 --> 00:02:26.471
the wrong paths. And I think that
the way to take the right paths is for

00:02:26.554 --> 00:02:27.437
the industry to

00:02:27.520 --> 00:02:31.470
work together. Again, you know,
we'll we'll talk about uh uh the proposal

00:02:31.554 --> 00:02:32.797
that I've put out today,

00:02:32.880 --> 00:02:36.582
but at the end of the day, the industry
has to work together and it's harder,

00:02:36.666 --> 00:02:40.873
but to some extent, the world has
to work together. If we go too slow,

00:02:40.956 --> 00:02:43.357
I still believe that the wrong people

00:02:43.440 --> 00:02:46.364
will be in charge of the
technology and and and you know,

00:02:46.448 --> 00:02:48.317
that again will bring the probability

00:02:48.400 --> 00:02:53.250
of things going wrong very high.
So what you're proposing is called pacing

00:02:53.333 --> 00:02:54.717
the frontier of of the

00:02:54.800 --> 00:02:58.325
AI industry. And you write we must
slow the pace at which we improve

00:02:58.408 --> 00:02:59.917
the capabilities of AI models.

00:03:00.000 --> 00:03:04.493
Progress will seem fast and we must
make wise use of the time we gain.

00:03:04.576 --> 00:03:05.997
And there's three parts

00:03:06.080 --> 00:03:10.271
to this. One is embedded evaluators.
Part two is democratic coordination.

00:03:10.355 --> 00:03:11.677
And part three is global

00:03:11.760 --> 00:03:16.325
coordination. Sam Alman at OpenAI just
endorsed the idea of embedded evaluators.

00:03:16.408 --> 00:03:17.197
Essentially you

00:03:17.280 --> 00:03:23.452
are suggesting I don't know if are they
observers inside your company and these

00:03:23.535 --> 00:03:24.877
other AI companies

00:03:24.960 --> 00:03:30.275
ideally globally but certainly in the US
who would monitor it they would have

00:03:30.358 --> 00:03:31.677
employee access what

00:03:31.760 --> 00:03:36.703
power would they have? Yeah. So,
so be before I even get to that,

00:03:36.787 --> 00:03:38.637
I'm I'm I'm just I'm very

00:03:38.720 --> 00:03:43.539
grateful to see um you know, support for
this proposal just in the first few hours

00:03:43.622 --> 00:03:48.253
of having put it out by by Sam Alman,
by Elon Musk, by uh you know,

00:03:48.336 --> 00:03:51.037
various legislators and and government

00:03:51.120 --> 00:03:53.426
officials. I think it's
a positive sign about, you know,

00:03:53.509 --> 00:03:54.877
our ability to make something that

00:03:54.960 --> 00:04:00.691
looks like this plan work. But but to
get to your question um yeah embedded

00:04:00.774 --> 00:04:02.397
evaluators so there is

00:04:02.480 --> 00:04:07.731
precedent for this um if you look at how
banks are regulated right there's there's

00:04:07.815 --> 00:04:08.317
all kinds

00:04:08.400 --> 00:04:12.546
of you know there's all kinds of rules
about you know how banks are allowed to do

00:04:12.629 --> 00:04:13.277
their business

00:04:13.360 --> 00:04:17.077
right because you know we we don't
want to have financial crisis and one of

00:04:17.160 --> 00:04:17.837
the things that

00:04:17.920 --> 00:04:22.346
are done in some cases is I think they're
called supervisors in the financial

00:04:22.429 --> 00:04:23.517
industry but they're

00:04:23.600 --> 00:04:28.246
they're folks who are independent of a
company who are embedded in the company

00:04:28.329 --> 00:04:29.277
daytoday to check

00:04:29.360 --> 00:04:34.997
their practices. Uh and I think we can
do the same thing in I think we can do

00:04:35.080 --> 00:04:36.557
the same thing in AI.

00:04:36.640 --> 00:04:40.383
The other thing you are calling for in
this three-part proposal is is democratic

00:04:40.466 --> 00:04:40.957
coordination

00:04:41.040 --> 00:04:45.365
which is essentially democracy all
the companies that are under democracies

00:04:45.448 --> 00:04:46.717
having these evaluators

00:04:46.800 --> 00:04:51.915
having this coordination and and probably
the most important because the argument

00:04:51.998 --> 00:04:52.557
is well if

00:04:52.640 --> 00:04:57.010
the US or companies are the only ones
regulating themselves what about does

00:04:57.094 --> 00:04:58.317
China get ahead you're

00:04:58.400 --> 00:05:04.703
calling for global coordination that's
obviously a lot more difficult yes yes I

00:05:04.787 --> 00:05:05.997
you know in this

00:05:06.080 --> 00:05:10.560
three-step proposal. Each step is more
difficult than than the one before.

00:05:10.643 --> 00:05:15.223
And Enthropic was kind of getting the ball
rolling by committing to the first step,

00:05:15.307 --> 00:05:16.877
but we shouldn't kid ourselves

00:05:16.960 --> 00:05:22.596
that these next two steps are going to be
more difficult and and we may not succeed.

00:05:22.679 --> 00:05:26.197
And and so my my proposal
was a proposal that we should try.

00:05:26.280 --> 00:05:29.117
Now again from all
the agreement from others that

00:05:29.200 --> 00:05:33.537
I've seen in just the first few hours
of putting out the essay that creates some

00:05:33.621 --> 00:05:34.477
optimism but that

00:05:34.560 --> 00:05:41.028
needs to follow up with results. So yes,
pacing among dem companies in democratic

00:05:41.111 --> 00:05:41.837
countries.

00:05:41.920 --> 00:05:48.920
Now the one of the important things that
that I need to call for here is you know

00:05:49.236 --> 00:05:50.237
as we get to

00:05:50.320 --> 00:05:55.182
the pace of model releases right
that's commercial activity and in order

00:05:55.265 --> 00:05:57.037
for companies to coordinate

00:05:57.120 --> 00:06:02.003
directly on commercial activity they
need to do so working with the government

00:06:02.087 --> 00:06:03.277
and so the one thing

00:06:03.360 --> 00:06:07.066
that we're asking for from government
right now right I mean there's also

00:06:07.150 --> 00:06:08.157
this question of what

00:06:08.240 --> 00:06:12.157
should the regulations be but there's
something even simpler that the government

00:06:12.240 --> 00:06:12.957
can do right now

00:06:13.040 --> 00:06:19.073
which is it can sit down in the room and
with all the industry players and we can

00:06:19.156 --> 00:06:19.677
all talk

00:06:19.760 --> 00:06:24.857
together and the government can make sure
that we're you know we're we're not doing

00:06:24.941 --> 00:06:29.059
anything nefarious from the perspective
of antitrust and collusion.

00:06:29.142 --> 00:06:31.357
We're just trying to make our system

00:06:31.440 --> 00:06:34.728
safe and to work together
on making the system safe.

00:06:34.812 --> 00:06:37.517
And so I think that's
an important role for

00:06:37.600 --> 00:06:41.929
the government to have.
Jacob Coxin joins us now. Um, Jacob,

00:06:42.012 --> 00:06:44.797
I appreciate you you being with us. Um,

00:06:44.880 --> 00:06:51.491
first of all, I assume you are saying
literally that you believe uh that AI

00:06:51.575 --> 00:06:57.277
could kill us all by the end
of the decade. How would that happen?

00:06:57.360 --> 00:07:02.077
Like what would that actually
look like? Yeah. So, the

00:07:02.160 --> 00:07:05.122
precise scenario sounds a little
bit like science fiction.

00:07:05.205 --> 00:07:07.117
Uh it sounds like something that's not

00:07:07.200 --> 00:07:08.798
real, but I think it
is frighteningly real.

00:07:08.881 --> 00:07:10.557
And it's it's easiest
to understand it if you

00:07:10.640 --> 00:07:15.286
think about real things that happened
two months ago. So two months ago,

00:07:15.369 --> 00:07:16.797
o open AI agents uh AIS

00:07:16.880 --> 00:07:20.333
hacked into third party infrastructure
entirely of their own accord.

00:07:20.416 --> 00:07:21.997
And this was like a concentrated

00:07:22.080 --> 00:07:25.595
hacking spree that they carried
out of their own valition.

00:07:25.678 --> 00:07:28.077
And I think that if you extrapolate into

00:07:28.160 --> 00:07:32.256
the future the level of capabilities
of these AIs with the same independent

00:07:32.340 --> 00:07:33.037
valition, they

00:07:33.120 --> 00:07:38.417
could cause extreme havoc. for example,
hacking critical infrastructure,

00:07:38.500 --> 00:07:39.837
building extinction

00:07:39.920 --> 00:07:45.406
level bioweapons. There's a lot of uh
ways that the AI could actuate itself in

00:07:45.489 --> 00:07:46.477
the world. I is

00:07:46.560 --> 00:07:50.158
it specifically when did your feelings
about this technology change?

00:07:50.242 --> 00:07:51.837
Was it something about the rate

00:07:51.920 --> 00:07:56.884
of progress that you've seen? Yeah. So I
think it's really important to just look

00:07:56.968 --> 00:08:01.237
at the rate of progress and say
for example in areas like coding or math.

00:08:01.320 --> 00:08:03.517
A couple years ago these AIs were just

00:08:03.600 --> 00:08:06.335
about helping humans a little bit.
They could give you suggestions.

00:08:06.418 --> 00:08:07.597
Now they're close to replacing

00:08:07.680 --> 00:08:11.764
them. We still need humans at the moment
but quite plausibly within a year we'll no

00:08:11.848 --> 00:08:12.317
longer need

00:08:12.400 --> 00:08:18.144
humans for doing research in many areas.
Only on Tuesday OpenAI solved a millennium

00:08:18.227 --> 00:08:19.197
problem. One of

00:08:19.280 --> 00:08:23.261
the biggest unsolved open problems
in mathematics purely autonomously using

00:08:23.345 --> 00:08:24.237
an AI. And I think

00:08:24.320 --> 00:08:28.381
what's most scary is if AI is used
to make itself more intelligent.

00:08:28.464 --> 00:08:30.237
So you can take an AI and give

00:08:30.320 --> 00:08:34.544
it the problem of AI research. Just like
currently they've given it math research.

00:08:34.627 --> 00:08:35.437
You could make it

00:08:35.520 --> 00:08:38.540
do the tasks that we're currently doing.
And then you get what's called

00:08:38.624 --> 00:08:39.677
an intelligence explosion.

00:08:39.760 --> 00:08:43.387
The AI just gets smarter and smarter with
no human involvement necessary until you

00:08:43.471 --> 00:08:44.157
have a thing that

00:08:44.240 --> 00:08:49.281
is vastly smarter than than humans.
And this could be coming very soon.

00:08:49.364 --> 00:08:53.077
We mentioned a former colleague
of yours currently works at Anthropic.

00:08:53.160 --> 00:08:55.357
basically endorsing
the assessment that AI could

00:08:55.440 --> 00:08:58.745
potentially kill everyone within a decade.
In a follow-up post, he said,

00:08:58.828 --> 00:09:01.563
"To be clear, as we say
in our in our latest risk report,

00:09:01.646 --> 00:09:04.077
I think the risk from present
models is low. What

00:09:04.160 --> 00:09:08.534
I'm worried about is super intelligence
arising from recursive self-improvement

00:09:08.617 --> 00:09:09.437
as we've said is

00:09:09.520 --> 00:09:12.682
happening faster than than we thought.
I assume that's what you the recursive

00:09:12.766 --> 00:09:13.357
self-improvement

00:09:13.440 --> 00:09:18.255
is what you've just spoken about." Um
you do you agree with him that the current

00:09:18.338 --> 00:09:19.357
models the risk is

00:09:19.440 --> 00:09:25.101
low but it's it's just the speed with
which this is now improving. Exactly.

00:09:25.185 --> 00:09:28.807
I completely agree. It's right
now there's no risk of extinction.

00:09:28.890 --> 00:09:30.957
The current models
the worst they can do is

00:09:31.040 --> 00:09:35.175
maybe hack into something
potentially cause a lot of damages in

00:09:35.258 --> 00:09:37.117
in infrastructure but there's

00:09:37.200 --> 00:09:41.159
no they're not intelligent enough to
outsmart us at the level that would lead

00:09:41.243 --> 00:09:42.157
to to extinction. I

00:09:42.240 --> 00:09:47.445
think what's just crazy is to look at the
the rate of progress and there is a very

00:09:47.528 --> 00:09:48.477
real possibility

00:09:48.560 --> 00:09:53.217
that in the immediate future these are
years that like next year the year after

00:09:53.300 --> 00:09:53.757
recursive

00:09:53.840 --> 00:09:58.027
self-improvement will happen and we'll
enter the phase of Evans post where he

00:09:58.110 --> 00:09:58.637
argues that

00:09:58.720 --> 00:10:03.349
there's a chance we could all die like
that that is coming soon you know a lot

00:10:03.432 --> 00:10:04.557
of the these CEOs of

00:10:04.640 --> 00:10:09.026
these companies will you know will go
to Congress and say hey we would love

00:10:09.109 --> 00:10:10.397
regulation anthropic in

00:10:10.480 --> 00:10:13.967
their new statement right now says says
the world would benefit from the industry

00:10:14.051 --> 00:10:14.717
adopting a lawful

00:10:14.800 --> 00:10:19.340
verifiable way to work together
to pace how we release powerful models.

00:10:19.423 --> 00:10:20.317
Um you know the

00:10:20.400 --> 00:10:25.854
argument it's an arms race if the US if
a US company doesn't achieve the best AI

00:10:25.937 --> 00:10:30.057
a Chinese company or some
bad actors out there might.

00:10:30.140 --> 00:10:34.637
Do you believe that uh that
these companies actually do

00:10:34.720 --> 00:10:38.408
want regulation or is it lip service?
They they completely do.

00:10:38.492 --> 00:10:40.477
This is completely honest. Just in

00:10:40.560 --> 00:10:43.797
the same way that the response
to Evans post was sort of surprise.

00:10:43.880 --> 00:10:45.357
It was like this can't be real.

00:10:45.440 --> 00:10:49.313
Surely a guy working on this technology
can't actually believe there's this level

00:10:49.396 --> 00:10:49.997
of danger. No,

00:10:50.080 --> 00:10:53.325
that's completely genuine. And these
people are also completely genuine when

00:10:53.408 --> 00:10:54.157
they are begging to

00:10:54.240 --> 00:10:58.347
be regulated. This is a genuine. And they
they find themselves in this scenario

00:10:58.430 --> 00:10:59.037
where they're

00:10:59.120 --> 00:11:03.477
compelled to race towards building
a deadly technology and they would they

00:11:03.560 --> 00:11:04.317
would love for

00:11:04.400 --> 00:11:08.283
some sort of international body to allow
them to to approach it at a reasonable

00:11:08.367 --> 00:11:09.037
pace. They just

00:11:09.120 --> 00:11:12.182
don't trust that the people around them
are going to get there in a safe way.

00:11:12.266 --> 00:11:12.877
So they feel like

00:11:12.960 --> 00:11:16.987
they have to race there, too.
You you could have stayed quiet. I mean,

00:11:17.071 --> 00:11:18.397
I'm sure you had a great

00:11:18.480 --> 00:11:22.481
job being paid, I imagine, very well.
You're obviously incredibly smart

00:11:22.564 --> 00:11:23.517
and mathematician.

00:11:23.600 --> 00:11:29.887
Um, why did you decide to speak out?
Well, I initially just decided to leave.

00:11:29.970 --> 00:11:30.797
I thought I

00:11:30.880 --> 00:11:35.493
can't be part of this anymore. But then
I uh I realized I could probably make some

00:11:35.576 --> 00:11:37.677
use of it. I did not expect
it to go this viral. So,

00:11:37.760 --> 00:11:39.517
what I did is I wrote
a Twitter thread out. I

00:11:39.600 --> 00:11:42.887
wrote it in a Google doc. I drafted it
with a friend. We we collaborated. Yeah,

00:11:42.971 --> 00:11:43.357
I had a few

00:11:43.440 --> 00:11:47.861
uh there were a few friends I discussed uh
the best way of expressing myself with um

00:11:47.945 --> 00:11:48.237
after I

00:11:48.320 --> 00:11:52.020
decided to decided to leave. Uh and then
I also got some friends to to retweet

00:11:52.104 --> 00:11:54.404
the thing. I was like, let's
try and make this a bit viral.

00:11:54.487 --> 00:11:56.317
But clearly there was latent
demand for this thing

00:11:56.400 --> 00:12:02.230
to absolutely explode. Like uh I've never
seen an AI safety tweet go this crazy.

00:12:02.313 --> 00:12:03.117
I think it's

00:12:03.200 --> 00:12:06.037
because people saw the uh the cyber
attacks in the last couple of months.

00:12:06.120 --> 00:12:06.877
They're also starting

00:12:06.960 --> 00:12:09.757
to internalize the rate of progress.
People are starting to realize this

00:12:09.840 --> 00:12:13.261
is real technology. This is not like hype.
This isn't going away anytime soon.

00:12:13.344 --> 00:12:15.277
This is only going to get more crazy. It's

00:12:15.360 --> 00:12:18.507
been getting crazy for the last 6 months.
And I think there was that latent

00:12:18.591 --> 00:12:19.197
understanding of

00:12:19.280 --> 00:12:22.797
the severity of the situation that meant
that this was like sort of a a match that

00:12:22.880 --> 00:12:26.454
lit kind of a a flame. Like I was very
shocked by the extent that how quickly it

00:12:26.537 --> 00:12:27.370
went viral.