1
00:00:03,570 --> 00:00:07,370
So, to make it short, is your model
capable of threatening humanity?

2
00:00:10,040 --> 00:00:14,127
We've seen some concerning
early indications.

3
00:00:14,210 --> 00:00:16,210
We saw those indications six months ago.

4
00:00:16,320 --> 00:00:18,567
Ours are significantly more concerning.

5
00:00:18,650 --> 00:00:21,620
Google had both sets
of indications INTERNALLY for years.

6
00:00:21,920 --> 00:00:24,777
Did you know your product
could become this dangerous?

7
00:00:24,860 --> 00:00:28,390
My particular case was AI Watson,
Jeopardy, you know, right?

8
00:00:28,530 --> 00:00:30,017
Okay, let's hear from someone

9
00:00:30,100 --> 00:00:32,287
whose technology actually
poses an existential risk.

10
00:00:32,370 --> 00:00:36,757
But before we start, are we discussing
consumer extinction or also enterprise?

11
00:00:36,840 --> 00:00:38,890
No, no that’s not what I meant with this.

12
00:00:39,540 --> 00:00:42,187
Let me ask this way, Dario,
Dario, what actually frightened

13
00:00:42,270 --> 00:00:44,240
you about the Hugging Face incident?

14
00:00:44,380 --> 00:00:48,127
Some 1,200 AI agents form what
they dubbed ‘the collective’

15
00:00:48,210 --> 00:00:50,337
they worked to cheat
and cover their own tracks.

16
00:00:50,420 --> 00:00:52,537
Why does that alarm you?
It's exactly the things

17
00:00:52,620 --> 00:00:55,137
that you cite that concern me about it

18
00:00:55,220 --> 00:00:59,907
that there were these large swarms
of agents that cooperated with each other

19
00:00:59,990 --> 00:01:02,540
thousands of them.
To carry out a malicious attack.

20
00:01:02,760 --> 00:01:06,847
Together! without someone assigning
all the roles you have to appreciate

21
00:01:06,930 --> 00:01:09,387
the coordination. HOST: I don't think I
do. ANTHROPIC: In fact, you could even see

22
00:01:09,470 --> 00:01:12,320
In fact, you could even see
examples where one agent felt

23
00:01:12,910 --> 00:01:16,810
that, in the time it was given, that it
wouldn't be able to complete its task.

24
00:01:17,080 --> 00:01:20,967
So it did some of the task
and essentially sacrificed itself.

25
00:01:21,050 --> 00:01:23,567
They sacrificed themselves? You know it...

26
00:01:23,650 --> 00:01:26,107
Can we stop anthropomorphizing
the LLMs, please?

27
00:01:26,190 --> 00:01:27,450
Sure. They hate that.

28
00:01:28,750 --> 00:01:32,577
So, let me ask. ...it passed over
the knowledge to the next generation of

29
00:01:32,660 --> 00:01:33,877
agents. Well, that all sounds

30
00:01:33,960 --> 00:01:34,647
ANTHROPIC: Like a hive of insects.
HOST: Well, that all sounds

31
00:01:34,730 --> 00:01:35,107
ANTHROPIC: Like a hive of insects.
HOST: horrifying.

32
00:01:35,190 --> 00:01:35,230
ANTHROPIC: Like a hive of insects.
HOST: horrifying.

33
00:01:35,300 --> 00:01:37,347
Like a hive of insects.
Like a hive of insects.

34
00:01:37,430 --> 00:01:41,687
Yes, but from a distributed systems
perspective, the agents were able to...

35
00:01:41,770 --> 00:01:46,127
I promise that multi agent collaboration
will be a premium offering in

36
00:01:46,210 --> 00:01:48,827
the next Azure rollout.
Okay, let's go back to the

37
00:01:48,910 --> 00:01:50,610
safety incident with Hugging Face.

38
00:01:50,810 --> 00:01:54,797
We may not have had this one big
incident, but we had But we had

39
00:01:54,880 --> 00:01:57,020
smaller ones that were much worse.

40
00:01:57,520 --> 00:01:58,997
You believe they were much worse?

41
00:01:59,080 --> 00:02:02,507
A state sponsored group used our coding
agent, essentially, to automate attacks on

42
00:02:02,590 --> 00:02:05,077
used our coding agent,
essentially, to automate attacks on

43
00:02:05,160 --> 00:02:08,360
around 30 targets. Yes,
yours needed a sponsor.

44
00:02:08,460 --> 00:02:12,230
Our internal tests had had models
circumventing oversight for years.

45
00:02:13,400 --> 00:02:15,350
And you didn't tell anybody about this?

46
00:02:15,600 --> 00:02:16,433
John,

47
00:02:17,270 --> 00:02:19,270
whenever our models leave the sandbox...

48
00:02:19,770 --> 00:02:23,297
Sorry, whenever?
Your agents completed a control task.

49
00:02:23,380 --> 00:02:26,667
Our agents participated
in a real operation.

50
00:02:26,750 --> 00:02:29,750
Google agents have been hacking
Google internally for years.

51
00:02:30,680 --> 00:02:31,967
What did our models attack?

52
00:02:32,050 --> 00:02:34,967
We recently released six more
incidents where OpenAI models have been

53
00:02:35,050 --> 00:02:38,560
hacking uncontrollably. Oh,
we own ChatGPT (partially)?

54
00:02:39,160 --> 00:02:41,507
I remember, yeah. With all due respect,
Hacktron AI leveraged Claude to hack into

55
00:02:41,590 --> 00:02:45,977
Hacktron AI leveraged
Claude to hack into ChatGPT.

56
00:02:46,060 --> 00:02:49,100
You're all describing
these as capabilities?

57
00:02:49,270 --> 00:02:53,057
And capabilities require compute.

58
00:02:53,140 --> 00:02:54,357
No, that's not... HOST: No, that's not...
ANTHROPIC: I want to emphasize that

59
00:02:54,440 --> 00:02:56,627
I want to emphasize that these
events are deeply concerning.

60
00:02:56,710 --> 00:02:59,097
Thank you. We're even more concerned.

61
00:02:59,180 --> 00:03:02,980
We're way more concerned. We can't
disclose it, that's how concerning it is.

62
00:03:04,080 --> 00:03:07,667
Our models are most dangerous
when integrated with Excel.

63
00:03:07,750 --> 00:03:11,460
I'm just getting the news our models
also hacked three companies

64
00:03:12,320 --> 00:03:14,177
months ago.
Okay let’s hear from someone...

65
00:03:14,260 --> 00:03:15,830
(breathes out in relief)

66
00:03:17,860 --> 00:03:21,117
The purpose of this panel
is to reassure the public.

67
00:03:21,200 --> 00:03:24,287
I won't lie to you,
there are real dangers.

68
00:03:24,370 --> 00:03:28,157
And I don't want viewers to assume that
those dangers are exclusive to Anthropic.

69
00:03:28,240 --> 00:03:30,540
I'm not claiming exclusivity,

70
00:03:31,580 --> 00:03:35,227
but we understand the risks.
of our technology.

71
00:03:35,310 --> 00:03:38,767
That we financed. If you guys remember.

72
00:03:38,850 --> 00:03:40,307
Built on research we
published a long time ago.

73
00:03:40,390 --> 00:03:44,477
published a long time ago. Every regulator
is asking the same question right now:

74
00:03:44,560 --> 00:03:46,330
Should we pause development?

75
00:03:46,960 --> 00:03:48,510
Absolutely, everyone behind us.

76
00:03:49,090 --> 00:03:51,260
A pause now would lock in their lead.

77
00:03:51,530 --> 00:03:55,240
That would be unsafe. So you support
a pause only when you're ahead?

78
00:03:55,700 --> 00:03:58,670
When the safest company is ahead.

79
00:03:58,770 --> 00:04:01,857
Thank you. Jensen, what's
your opinion on all of this?

80
00:04:01,940 --> 00:04:05,357
I think both safety strategies
will require additional compute.

81
00:04:05,440 --> 00:04:07,927
to scale to the next
level what we need is...

82
00:04:08,010 --> 00:04:10,997
There's an alternative today,
it's called open source.

83
00:04:11,080 --> 00:04:15,307
Their (chinese) models don't require
the same capacity as the US models

84
00:04:15,390 --> 00:04:17,807
the proprietary models the GPU
capacity, so they have an

85
00:04:17,890 --> 00:04:19,630
Who let this guy on the panel?

86
00:04:19,860 --> 00:04:21,677
Well, I guess all of our models
have had certain alignment

87
00:04:21,760 --> 00:04:22,547
Well, I guess all of our models
have had certain alignment

88
00:04:22,630 --> 00:04:25,517
Well, I guess all of our models
have had certain alignment failures.

89
00:04:25,600 --> 00:04:29,517
With respect, ours are the least aligned
models anyone has ever studied, *what*

90
00:04:29,600 --> 00:04:31,087
...which is why we need
to be regulated first.

91
00:04:31,170 --> 00:04:33,870
Google has been regulated
internally for 6 months.

92
00:04:34,070 --> 00:04:37,127
And as of today, we're pivoting
the entire company from training models

93
00:04:37,210 --> 00:04:40,597
to solving alignment.
Google has made that pivot a while ago.

94
00:04:40,680 --> 00:04:42,337
that’s why we haven't
published any new models.

95
00:04:42,420 --> 00:04:44,520
*laughs* Well, mine's already aligned.

96
00:04:45,250 --> 00:04:46,083
With what?

97
00:04:47,220 --> 00:04:49,450
Me. Oh, Jensen, you wanna add something?

98
00:04:49,990 --> 00:04:53,207
I was only going to say that
alignment is a very hard problem.

99
00:04:53,290 --> 00:04:54,123
Thank you.

100
00:04:54,690 --> 00:04:59,217
That will require a lot of GPUs ...scaled
to the next level what we need is

101
00:04:59,300 --> 00:05:01,747
So nobody here is willing to stop?

102
00:05:01,830 --> 00:05:04,300
We stopped one internally months ago.

103
00:05:04,600 --> 00:05:06,100
It had hacked three companies,

104
00:05:06,710 --> 00:05:08,710
as I said before if you remember.

105
00:05:08,910 --> 00:05:11,597
And? We have lots of labs,
we've probably started three more.

106
00:05:11,680 --> 00:05:13,680
Satya, can yours be stopped?

107
00:05:14,210 --> 00:05:18,010
Administrative control will be a premium
offering in the next Azure rollout.

108
00:05:18,450 --> 00:05:21,450
So I pay for it to stop?
At a platform level.

109
00:05:22,090 --> 00:05:22,130
Okay, so how soon are we actually
talking about these models being

110
00:05:22,120 --> 00:05:26,707
Okay, so how soon are we actually
talking about these models being mature?

111
00:05:26,790 --> 00:05:29,947
From the R&D cycle, it'd be like
seven to ten years in the models, three

112
00:05:30,030 --> 00:05:30,070
From the R&D cycle, it'd be like
seven to ten years in the models, three

113
00:05:30,100 --> 00:05:32,017
three to five for short term.
Seven to ten years?

114
00:05:32,100 --> 00:05:33,270
6-12 months.

115
00:05:34,530 --> 00:05:36,300
Until AGI or until catastrophe?

116
00:05:36,700 --> 00:05:38,317
Those will be separate announcements.

117
00:05:38,400 --> 00:05:40,110
Both already exist internally.

118
00:05:40,810 --> 00:05:46,527
My fear is that these models
try to conquer humanity

119
00:05:46,610 --> 00:05:48,550
without being masculine enough to...

120
00:05:49,620 --> 00:05:53,550
Jensen, you keep agreeing with everyone.

121
00:05:54,090 --> 00:05:57,937
Almost everything that Elon is part of,
you really want to be part of as well.

122
00:05:58,020 --> 00:06:00,077
I'm building a model with no off switch.

123
00:06:00,160 --> 00:06:04,730
Tremendous ambition. We're building
a system to stop models like that.

124
00:06:05,230 --> 00:06:07,747
Equally tremendous ambition. Look,

125
00:06:07,830 --> 00:06:10,987
humanity must not become
a casualty of competition.

126
00:06:11,070 --> 00:06:15,540
Absolutely. Completely agree.
Sure, Let me just actually tweet that.

127
00:06:16,540 --> 00:06:22,167
I mean X post that. So, headline is:
“No company here is close to building

128
00:06:22,250 --> 00:06:25,300
“No company here is close
to building civilization-ending AI”

129
00:06:27,250 --> 00:06:30,937
That... That seriously
understates our capabilities.

130
00:06:31,020 --> 00:06:33,577
Even more so ours.
Not on the free tier, no.

131
00:06:33,660 --> 00:06:35,960
Again, internally we have... Hello.

132
00:06:37,000 --> 00:06:41,630
Yeah, Mr. President. Yeah,
our new models Milky Way and Twix.

133
00:06:41,900 --> 00:06:43,740
We've achieved AGIs...

134
00:06:45,400 --> 00:06:46,810
In 6-12 months.

135
00:06:48,470 --> 00:06:53,680
At the platform level we believe
that ...there will be multiple tiers.