WEBVTT

00:00:03.570 --> 00:00:07.370
So, to make it short, is your model
capable of threatening humanity?

00:00:10.040 --> 00:00:14.127
We've seen some concerning
early indications.

00:00:14.210 --> 00:00:16.210
We saw those indications six months ago.

00:00:16.320 --> 00:00:18.567
Ours are significantly more concerning.

00:00:18.650 --> 00:00:21.620
Google had both sets
of indications INTERNALLY for years.

00:00:21.920 --> 00:00:24.777
Did you know your product
could become this dangerous?

00:00:24.860 --> 00:00:28.390
My particular case was AI Watson,
Jeopardy, you know, right?

00:00:28.530 --> 00:00:30.017
Okay, let's hear from someone

00:00:30.100 --> 00:00:32.287
whose technology actually
poses an existential risk.

00:00:32.370 --> 00:00:36.757
But before we start, are we discussing
consumer extinction or also enterprise?

00:00:36.840 --> 00:00:38.890
No, no that’s not what I meant with this.

00:00:39.540 --> 00:00:42.187
Let me ask this way, Dario,
Dario, what actually frightened

00:00:42.270 --> 00:00:44.240
you about the Hugging Face incident?

00:00:44.380 --> 00:00:48.127
Some 1,200 AI agents form what
they dubbed ‘the collective’

00:00:48.210 --> 00:00:50.337
they worked to cheat
and cover their own tracks.

00:00:50.420 --> 00:00:52.537
Why does that alarm you?
It's exactly the things

00:00:52.620 --> 00:00:55.137
that you cite that concern me about it

00:00:55.220 --> 00:00:59.907
that there were these large swarms
of agents that cooperated with each other

00:00:59.990 --> 00:01:02.540
thousands of them.
To carry out a malicious attack.

00:01:02.760 --> 00:01:06.847
Together! without someone assigning
all the roles you have to appreciate

00:01:06.930 --> 00:01:09.387
the coordination. HOST: I don't think I
do. ANTHROPIC: In fact, you could even see

00:01:09.470 --> 00:01:12.320
In fact, you could even see
examples where one agent felt

00:01:12.910 --> 00:01:16.810
that, in the time it was given, that it
wouldn't be able to complete its task.

00:01:17.080 --> 00:01:20.967
So it did some of the task
and essentially sacrificed itself.

00:01:21.050 --> 00:01:23.567
They sacrificed themselves? You know it...

00:01:23.650 --> 00:01:26.107
Can we stop anthropomorphizing
the LLMs, please?

00:01:26.190 --> 00:01:27.450
Sure. They hate that.

00:01:28.750 --> 00:01:32.577
So, let me ask. ...it passed over
the knowledge to the next generation of

00:01:32.660 --> 00:01:33.877
agents. Well, that all sounds

00:01:33.960 --> 00:01:34.647
ANTHROPIC: Like a hive of insects.
HOST: Well, that all sounds

00:01:34.730 --> 00:01:35.107
ANTHROPIC: Like a hive of insects.
HOST: horrifying.

00:01:35.190 --> 00:01:35.230
ANTHROPIC: Like a hive of insects.
HOST: horrifying.

00:01:35.300 --> 00:01:37.347
Like a hive of insects.
Like a hive of insects.

00:01:37.430 --> 00:01:41.687
Yes, but from a distributed systems
perspective, the agents were able to...

00:01:41.770 --> 00:01:46.127
I promise that multi agent collaboration
will be a premium offering in

00:01:46.210 --> 00:01:48.827
the next Azure rollout.
Okay, let's go back to the

00:01:48.910 --> 00:01:50.610
safety incident with Hugging Face.

00:01:50.810 --> 00:01:54.797
We may not have had this one big
incident, but we had But we had

00:01:54.880 --> 00:01:57.020
smaller ones that were much worse.

00:01:57.520 --> 00:01:58.997
You believe they were much worse?

00:01:59.080 --> 00:02:02.507
A state sponsored group used our coding
agent, essentially, to automate attacks on

00:02:02.590 --> 00:02:05.077
used our coding agent,
essentially, to automate attacks on

00:02:05.160 --> 00:02:08.360
around 30 targets. Yes,
yours needed a sponsor.

00:02:08.460 --> 00:02:12.230
Our internal tests had had models
circumventing oversight for years.

00:02:13.400 --> 00:02:15.350
And you didn't tell anybody about this?

00:02:15.600 --> 00:02:16.433
John,

00:02:17.270 --> 00:02:19.270
whenever our models leave the sandbox...

00:02:19.770 --> 00:02:23.297
Sorry, whenever?
Your agents completed a control task.

00:02:23.380 --> 00:02:26.667
Our agents participated
in a real operation.

00:02:26.750 --> 00:02:29.750
Google agents have been hacking
Google internally for years.

00:02:30.680 --> 00:02:31.967
What did our models attack?

00:02:32.050 --> 00:02:34.967
We recently released six more
incidents where OpenAI models have been

00:02:35.050 --> 00:02:38.560
hacking uncontrollably. Oh,
we own ChatGPT (partially)?

00:02:39.160 --> 00:02:41.507
I remember, yeah. With all due respect,
Hacktron AI leveraged Claude to hack into

00:02:41.590 --> 00:02:45.977
Hacktron AI leveraged
Claude to hack into ChatGPT.

00:02:46.060 --> 00:02:49.100
You're all describing
these as capabilities?

00:02:49.270 --> 00:02:53.057
And capabilities require compute.

00:02:53.140 --> 00:02:54.357
No, that's not... HOST: No, that's not...
ANTHROPIC: I want to emphasize that

00:02:54.440 --> 00:02:56.627
I want to emphasize that these
events are deeply concerning.

00:02:56.710 --> 00:02:59.097
Thank you. We're even more concerned.

00:02:59.180 --> 00:03:02.980
We're way more concerned. We can't
disclose it, that's how concerning it is.

00:03:04.080 --> 00:03:07.667
Our models are most dangerous
when integrated with Excel.

00:03:07.750 --> 00:03:11.460
I'm just getting the news our models
also hacked three companies

00:03:12.320 --> 00:03:14.177
months ago.
Okay let’s hear from someone...

00:03:14.260 --> 00:03:15.830
(breathes out in relief)

00:03:17.860 --> 00:03:21.117
The purpose of this panel
is to reassure the public.

00:03:21.200 --> 00:03:24.287
I won't lie to you,
there are real dangers.

00:03:24.370 --> 00:03:28.157
And I don't want viewers to assume that
those dangers are exclusive to Anthropic.

00:03:28.240 --> 00:03:30.540
I'm not claiming exclusivity,

00:03:31.580 --> 00:03:35.227
but we understand the risks.
of our technology.

00:03:35.310 --> 00:03:38.767
That we financed. If you guys remember.

00:03:38.850 --> 00:03:40.307
Built on research we
published a long time ago.

00:03:40.390 --> 00:03:44.477
published a long time ago. Every regulator
is asking the same question right now:

00:03:44.560 --> 00:03:46.330
Should we pause development?

00:03:46.960 --> 00:03:48.510
Absolutely, everyone behind us.

00:03:49.090 --> 00:03:51.260
A pause now would lock in their lead.

00:03:51.530 --> 00:03:55.240
That would be unsafe. So you support
a pause only when you're ahead?

00:03:55.700 --> 00:03:58.670
When the safest company is ahead.

00:03:58.770 --> 00:04:01.857
Thank you. Jensen, what's
your opinion on all of this?

00:04:01.940 --> 00:04:05.357
I think both safety strategies
will require additional compute.

00:04:05.440 --> 00:04:07.927
to scale to the next
level what we need is...

00:04:08.010 --> 00:04:10.997
There's an alternative today,
it's called open source.

00:04:11.080 --> 00:04:15.307
Their (chinese) models don't require
the same capacity as the US models

00:04:15.390 --> 00:04:17.807
the proprietary models the GPU
capacity, so they have an

00:04:17.890 --> 00:04:19.630
Who let this guy on the panel?

00:04:19.860 --> 00:04:21.677
Well, I guess all of our models
have had certain alignment

00:04:21.760 --> 00:04:22.547
Well, I guess all of our models
have had certain alignment

00:04:22.630 --> 00:04:25.517
Well, I guess all of our models
have had certain alignment failures.

00:04:25.600 --> 00:04:29.517
With respect, ours are the least aligned
models anyone has ever studied, *what*

00:04:29.600 --> 00:04:31.087
...which is why we need
to be regulated first.

00:04:31.170 --> 00:04:33.870
Google has been regulated
internally for 6 months.

00:04:34.070 --> 00:04:37.127
And as of today, we're pivoting
the entire company from training models

00:04:37.210 --> 00:04:40.597
to solving alignment.
Google has made that pivot a while ago.

00:04:40.680 --> 00:04:42.337
that’s why we haven't
published any new models.

00:04:42.420 --> 00:04:44.520
*laughs* Well, mine's already aligned.

00:04:45.250 --> 00:04:46.083
With what?

00:04:47.220 --> 00:04:49.450
Me. Oh, Jensen, you wanna add something?

00:04:49.990 --> 00:04:53.207
I was only going to say that
alignment is a very hard problem.

00:04:53.290 --> 00:04:54.123
Thank you.

00:04:54.690 --> 00:04:59.217
That will require a lot of GPUs ...scaled
to the next level what we need is

00:04:59.300 --> 00:05:01.747
So nobody here is willing to stop?

00:05:01.830 --> 00:05:04.300
We stopped one internally months ago.

00:05:04.600 --> 00:05:06.100
It had hacked three companies,

00:05:06.710 --> 00:05:08.710
as I said before if you remember.

00:05:08.910 --> 00:05:11.597
And? We have lots of labs,
we've probably started three more.

00:05:11.680 --> 00:05:13.680
Satya, can yours be stopped?

00:05:14.210 --> 00:05:18.010
Administrative control will be a premium
offering in the next Azure rollout.

00:05:18.450 --> 00:05:21.450
So I pay for it to stop?
At a platform level.

00:05:22.090 --> 00:05:22.130
Okay, so how soon are we actually
talking about these models being

00:05:22.120 --> 00:05:26.707
Okay, so how soon are we actually
talking about these models being mature?

00:05:26.790 --> 00:05:29.947
From the R&D cycle, it'd be like
seven to ten years in the models, three

00:05:30.030 --> 00:05:30.070
From the R&D cycle, it'd be like
seven to ten years in the models, three

00:05:30.100 --> 00:05:32.017
three to five for short term.
Seven to ten years?

00:05:32.100 --> 00:05:33.270
6-12 months.

00:05:34.530 --> 00:05:36.300
Until AGI or until catastrophe?

00:05:36.700 --> 00:05:38.317
Those will be separate announcements.

00:05:38.400 --> 00:05:40.110
Both already exist internally.

00:05:40.810 --> 00:05:46.527
My fear is that these models
try to conquer humanity

00:05:46.610 --> 00:05:48.550
without being masculine enough to...

00:05:49.620 --> 00:05:53.550
Jensen, you keep agreeing with everyone.

00:05:54.090 --> 00:05:57.937
Almost everything that Elon is part of,
you really want to be part of as well.

00:05:58.020 --> 00:06:00.077
I'm building a model with no off switch.

00:06:00.160 --> 00:06:04.730
Tremendous ambition. We're building
a system to stop models like that.

00:06:05.230 --> 00:06:07.747
Equally tremendous ambition. Look,

00:06:07.830 --> 00:06:10.987
humanity must not become
a casualty of competition.

00:06:11.070 --> 00:06:15.540
Absolutely. Completely agree.
Sure, Let me just actually tweet that.

00:06:16.540 --> 00:06:22.167
I mean X post that. So, headline is:
“No company here is close to building

00:06:22.250 --> 00:06:25.300
“No company here is close
to building civilization-ending AI”

00:06:27.250 --> 00:06:30.937
That... That seriously
understates our capabilities.

00:06:31.020 --> 00:06:33.577
Even more so ours.
Not on the free tier, no.

00:06:33.660 --> 00:06:35.960
Again, internally we have... Hello.

00:06:37.000 --> 00:06:41.630
Yeah, Mr. President. Yeah,
our new models Milky Way and Twix.

00:06:41.900 --> 00:06:43.740
We've achieved AGIs...

00:06:45.400 --> 00:06:46.810
In 6-12 months.

00:06:48.470 --> 00:06:53.680
At the platform level we believe
that ...there will be multiple tiers.