1
00:00:00,000 --> 00:00:02,677
On fully autonomous weapons.
The logic, as I understand

2
00:00:02,760 --> 00:00:05,277
it, is not so much
about democratic values.

3
00:00:05,360 --> 00:00:10,677
It's more that you think the technology
is not ready for your models

4
00:00:10,760 --> 00:00:13,157
to be used fully autonomously.

5
00:00:13,240 --> 00:00:15,637
Is that is that right?
So you you have a different view

6
00:00:15,720 --> 00:00:18,397
of the capabilities of the technology,
perhaps in the department of.

7
00:00:18,480 --> 00:00:20,837
Yeah, I think there's actually two things.

8
00:00:20,920 --> 00:00:25,797
One is this, this issue that you just
mentioned, which is you can think of this

9
00:00:25,880 --> 00:00:31,517
as like a supplier of aircraft who
says this aircraft is not safe, default

10
00:00:31,600 --> 00:00:36,037
to fly above a certain, you know,
altitude or to turn at a certain speed.

11
00:00:36,120 --> 00:00:38,677
We didn't make these systems to be safe.

12
00:00:38,760 --> 00:00:40,277
If you use them in that particular way.

13
00:00:40,360 --> 00:00:43,557
Right. They're not
suitable for this use case.

14
00:00:43,640 --> 00:00:47,677
Like we haven't made them
or manufactured them to, to to work

15
00:00:47,760 --> 00:00:49,237
well for it to be safe for this use.

16
00:00:49,320 --> 00:00:52,877
And in that. Was there anything
specific I mean your your model has been

17
00:00:52,960 --> 00:00:56,077
used was used in Venezuela.
It's being used in Iran.

18
00:00:56,160 --> 00:00:59,717
Is there anything in these
existing use cases that is more so.

19
00:00:59,800 --> 00:01:04,200
One thing I want to say is, is we
started with with a substantially,

20
00:01:05,040 --> 00:01:08,437
more limited contract that enabled
a more limited number of use cases

21
00:01:08,520 --> 00:01:11,797
that I think is in force,
maybe even until today.

22
00:01:11,880 --> 00:01:14,677
This was all about
negotiating a new contract.

23
00:01:14,760 --> 00:01:17,520
So we had even
a substantially more limited,

24
00:01:18,840 --> 00:01:24,277
contract for, for quite a long time
and that more limited contract, you know,

25
00:01:24,360 --> 00:01:26,557
they didn't run into problems
with it on the ground.

26
00:01:26,640 --> 00:01:32,437
So none of this was driven by we
have concerns right now about what

27
00:01:32,520 --> 00:01:34,437
the Department of War was doing. Right?

28
00:01:34,520 --> 00:01:37,357
Iran was is fine and Venezuela was fine.

29
00:01:37,440 --> 00:01:42,077
You know, again, you know, it's I'm
not expressing opinion either way

30
00:01:42,160 --> 00:01:44,717
on military policy or government policy.

31
00:01:44,800 --> 00:01:45,997
I'm just I'm just saying if you.

32
00:01:46,080 --> 00:01:50,397
Use of your model. Yeah, you.
Can if you if you provide a service,

33
00:01:50,480 --> 00:01:53,677
you don't get to decide
where that service is used.

34
00:01:53,760 --> 00:01:57,997
There's this operational side
of military decisions course.

35
00:01:58,080 --> 00:01:59,517
We don't have any say about that.

36
00:01:59,600 --> 00:02:02,397
What we do have say over
is this, this question

37
00:02:02,480 --> 00:02:04,877
of like very broad use cases, right?

38
00:02:04,960 --> 00:02:06,597
It's not about any administration.

39
00:02:06,680 --> 00:02:12,797
It's not about any policy. It's like, how
should this technology be used in general?

40
00:02:12,880 --> 00:02:15,997
I actually think there's
a lot of value here.

41
00:02:16,080 --> 00:02:18,517
And, you know, I want to make sure
the Department of War understands

42
00:02:18,600 --> 00:02:23,637
this as well. This isn't about concerns we
have about actions they've taken already,

43
00:02:23,720 --> 00:02:26,077
like we had a much more limited contract.

44
00:02:26,160 --> 00:02:29,157
We never had, you know,
concerns about this contract.

45
00:02:29,240 --> 00:02:32,197
This is about, you know,
this is about going forward,

46
00:02:32,280 --> 00:02:35,717
how should we think about
the uses and the governance?

47
00:02:35,800 --> 00:02:38,677
I think going forward,
their perspective would be

48
00:02:38,760 --> 00:02:42,597
and they would probably use the analogy
of the fighter jet or something.

49
00:02:42,680 --> 00:02:47,197
And they would say, listen, we have
to have suppliers who we can trust,

50
00:02:47,280 --> 00:02:51,280
and they provide us with tools
that we can use in combat,

51
00:02:51,440 --> 00:02:54,677
and we can't have some guy
in Silicon Valley suddenly saying, no,

52
00:02:54,760 --> 00:02:56,917
you can't use it
for this fact or the other.

53
00:02:57,000 --> 00:03:02,637
And their view would be they need to have
full operational ability to use it,

54
00:03:02,720 --> 00:03:05,077
and you are somewhat
capriciously in that view,

55
00:03:05,160 --> 00:03:07,157
putting all of these constraints on it.

56
00:03:07,240 --> 00:03:10,357
Yeah. So so I would
say we agree with them.

57
00:03:10,440 --> 00:03:15,277
We don't want to do that.
We agree on 99%, not only the contract

58
00:03:15,360 --> 00:03:19,277
we were operating on together,
we agree on like this much broader

59
00:03:19,360 --> 00:03:22,717
set of things that that, you know,
we're haven't even done yet

60
00:03:22,800 --> 00:03:26,197
or are just starting to do. So,
you know, I just want to say again, like,

61
00:03:26,280 --> 00:03:28,917
there's so much more agreement
here than there is this, I.

62
00:03:29,000 --> 00:03:31,037
Mean, listening to you,
it does seem to me extraordinary

63
00:03:31,120 --> 00:03:33,357
that, you know,
you end up in this situation

64
00:03:33,440 --> 00:03:35,340
where you're going to be taken out of,

65
00:03:36,080 --> 00:03:38,277
all work with, with the Pentagon.

66
00:03:38,360 --> 00:03:41,557
That said the other, because I've
been thinking about this quite a lot.

67
00:03:41,640 --> 00:03:47,637
The the conclusion, the logical
conclusion of total control

68
00:03:47,720 --> 00:03:52,477
by the Pentagon and no ability
for the supply to do anything

69
00:03:52,560 --> 00:03:55,597
is surely some sort of effective
nationalisation of those

70
00:03:55,680 --> 00:03:57,517
AI companies that work with the Pentagon.

71
00:03:57,600 --> 00:04:01,237
Do you think that's a risk? I think over
the coming years, we're going to have

72
00:04:01,320 --> 00:04:08,157
to have a discussion about
the role of AI and the government.

73
00:04:08,240 --> 00:04:11,437
You know, AI is becoming
something that has implications

74
00:04:11,520 --> 00:04:14,560
at the level of all of humanity, not,

75
00:04:15,440 --> 00:04:20,237
you know, for national security in the
sense of the contest between nations

76
00:04:20,320 --> 00:04:23,677
for national security, in the sense of,
you know, can individuals

77
00:04:23,760 --> 00:04:27,637
use this technology for destruction,
for national security, in the sense of

78
00:04:27,720 --> 00:04:31,117
are the models themselves
a risk to national security

79
00:04:31,200 --> 00:04:34,200
for economics in terms of their impacts?

80
00:04:34,320 --> 00:04:37,517
And it's it's not about one
particular administration.

81
00:04:37,600 --> 00:04:39,397
It's not about one particular department.

82
00:04:39,480 --> 00:04:43,040
It's not about specific
military operations.

83
00:04:43,480 --> 00:04:46,477
Anyone who thinks that this is
what that's about, that's that's

84
00:04:46,560 --> 00:04:50,157
that's a misunderstood this is this,
this, this is bigger than that.

85
00:04:50,240 --> 00:04:52,797
This is big. This is about
the next administration, too.

86
00:04:52,880 --> 00:04:56,317
This is about, you know, Congress
when there's different Congress people,

87
00:04:56,400 --> 00:04:59,957
this is about, you know,
a different geopolitical situation

88
00:05:00,040 --> 00:05:01,517
that we're in than the one we're in now.

89
00:05:01,600 --> 00:05:06,877
It's it's, you know, we should not make it
about what's happening right, right now.

90
00:05:06,960 --> 00:05:11,560
And the thing we're dealing
with here is AI is very powerful.

91
00:05:12,280 --> 00:05:18,120
So there's actually a dual dilemma
here where AI has the power to make

92
00:05:18,800 --> 00:05:22,677
private actors more powerful
than they've ever been before.

93
00:05:22,760 --> 00:05:25,237
Folks have, you know, some people
have criticised me for that.

94
00:05:25,320 --> 00:05:27,957
Or you think as a private actor
you're more powerful in government?

95
00:05:28,040 --> 00:05:31,477
Actually, I've been worried about
the risk of that for the longest time.

96
00:05:31,560 --> 00:05:34,957
Like I'm worried about
that this technology abstractly

97
00:05:35,040 --> 00:05:37,877
in the hands of government,
not just the US government,

98
00:05:37,960 --> 00:05:41,197
other governments, democratic
governments, autocratic government

99
00:05:41,280 --> 00:05:46,920
has the ability to vest them
with unprecedented power as well.

100
00:05:47,680 --> 00:05:50,517
And so when we talked about
fully autonomous weapons,

101
00:05:50,600 --> 00:05:51,877
I said there were two objections.

102
00:05:51,960 --> 00:05:54,197
The first one I said was reliability.

103
00:05:54,280 --> 00:05:57,880
The second one I said was was
what I would call oversight.

104
00:05:58,960 --> 00:06:03,797
And this is the idea that, you know, right
now you have an army of human soldiers,

105
00:06:03,880 --> 00:06:06,637
and there are norms about
serving in the military.

106
00:06:06,720 --> 00:06:10,077
You're supposed to follow orders,
but, you know, if something crazy,

107
00:06:10,160 --> 00:06:12,997
if something crazy enough happened,
the soldiers would say,

108
00:06:13,080 --> 00:06:14,317
I'm not going to do that. Right.

109
00:06:14,400 --> 00:06:16,757
You have basically,
you have a bunch of norms about,

110
00:06:16,840 --> 00:06:19,157
you know, how soldiers serve, right?

111
00:06:19,240 --> 00:06:22,240
You know what they see their duties to be.

112
00:06:22,440 --> 00:06:25,677
What if you have an army of 10
million drones instead of,

113
00:06:25,760 --> 00:06:27,837
you know, 10 million human soldiers?

114
00:06:27,920 --> 00:06:31,237
What what are the norms of the
of the of the AI driven drones?

115
00:06:31,320 --> 00:06:34,757
I think if we handled this wrongly,
you could have a situation where

116
00:06:34,840 --> 00:06:37,677
there's a very small number
of people or one person

117
00:06:37,760 --> 00:06:40,637
who has their hand
on the button and kind of,

118
00:06:40,720 --> 00:06:42,477
you know, controls
those 10 million drones.

119
00:06:42,560 --> 00:06:46,717
And we need to answer these questions
because we don't want to make companies

120
00:06:46,800 --> 00:06:51,157
more powerful than the government,
but we also don't want to make government.

121
00:06:51,240 --> 00:06:54,077
So powerful that it
that it can't be stopped.

122
00:06:54,160 --> 00:06:55,797
We have both problems at once.

123
00:06:55,880 --> 00:06:58,757
So you basically just said to me,
this technology is too powerful

124
00:06:58,840 --> 00:07:02,037
to be in the hands of a few private
companies, and it's too powerful

125
00:07:02,120 --> 00:07:03,157
to be in the hands of a government.

126
00:07:03,240 --> 00:07:05,390
That is unfortunately,
the situation where.