WEBVTT

00:00:00.000 --> 00:00:02.677
On fully autonomous weapons.
The logic, as I understand

00:00:02.760 --> 00:00:05.277
it, is not so much
about democratic values.

00:00:05.360 --> 00:00:10.677
It's more that you think the technology
is not ready for your models

00:00:10.760 --> 00:00:13.157
to be used fully autonomously.

00:00:13.240 --> 00:00:15.637
Is that is that right?
So you you have a different view

00:00:15.720 --> 00:00:18.397
of the capabilities of the technology,
perhaps in the department of.

00:00:18.480 --> 00:00:20.837
Yeah, I think there's actually two things.

00:00:20.920 --> 00:00:25.797
One is this, this issue that you just
mentioned, which is you can think of this

00:00:25.880 --> 00:00:31.517
as like a supplier of aircraft who
says this aircraft is not safe, default

00:00:31.600 --> 00:00:36.037
to fly above a certain, you know,
altitude or to turn at a certain speed.

00:00:36.120 --> 00:00:38.677
We didn't make these systems to be safe.

00:00:38.760 --> 00:00:40.277
If you use them in that particular way.

00:00:40.360 --> 00:00:43.557
Right. They're not
suitable for this use case.

00:00:43.640 --> 00:00:47.677
Like we haven't made them
or manufactured them to, to to work

00:00:47.760 --> 00:00:49.237
well for it to be safe for this use.

00:00:49.320 --> 00:00:52.877
And in that. Was there anything
specific I mean your your model has been

00:00:52.960 --> 00:00:56.077
used was used in Venezuela.
It's being used in Iran.

00:00:56.160 --> 00:00:59.717
Is there anything in these
existing use cases that is more so.

00:00:59.800 --> 00:01:04.200
One thing I want to say is, is we
started with with a substantially,

00:01:05.040 --> 00:01:08.437
more limited contract that enabled
a more limited number of use cases

00:01:08.520 --> 00:01:11.797
that I think is in force,
maybe even until today.

00:01:11.880 --> 00:01:14.677
This was all about
negotiating a new contract.

00:01:14.760 --> 00:01:17.520
So we had even
a substantially more limited,

00:01:18.840 --> 00:01:24.277
contract for, for quite a long time
and that more limited contract, you know,

00:01:24.360 --> 00:01:26.557
they didn't run into problems
with it on the ground.

00:01:26.640 --> 00:01:32.437
So none of this was driven by we
have concerns right now about what

00:01:32.520 --> 00:01:34.437
the Department of War was doing. Right?

00:01:34.520 --> 00:01:37.357
Iran was is fine and Venezuela was fine.

00:01:37.440 --> 00:01:42.077
You know, again, you know, it's I'm
not expressing opinion either way

00:01:42.160 --> 00:01:44.717
on military policy or government policy.

00:01:44.800 --> 00:01:45.997
I'm just I'm just saying if you.

00:01:46.080 --> 00:01:50.397
Use of your model. Yeah, you.
Can if you if you provide a service,

00:01:50.480 --> 00:01:53.677
you don't get to decide
where that service is used.

00:01:53.760 --> 00:01:57.997
There's this operational side
of military decisions course.

00:01:58.080 --> 00:01:59.517
We don't have any say about that.

00:01:59.600 --> 00:02:02.397
What we do have say over
is this, this question

00:02:02.480 --> 00:02:04.877
of like very broad use cases, right?

00:02:04.960 --> 00:02:06.597
It's not about any administration.

00:02:06.680 --> 00:02:12.797
It's not about any policy. It's like, how
should this technology be used in general?

00:02:12.880 --> 00:02:15.997
I actually think there's
a lot of value here.

00:02:16.080 --> 00:02:18.517
And, you know, I want to make sure
the Department of War understands

00:02:18.600 --> 00:02:23.637
this as well. This isn't about concerns we
have about actions they've taken already,

00:02:23.720 --> 00:02:26.077
like we had a much more limited contract.

00:02:26.160 --> 00:02:29.157
We never had, you know,
concerns about this contract.

00:02:29.240 --> 00:02:32.197
This is about, you know,
this is about going forward,

00:02:32.280 --> 00:02:35.717
how should we think about
the uses and the governance?

00:02:35.800 --> 00:02:38.677
I think going forward,
their perspective would be

00:02:38.760 --> 00:02:42.597
and they would probably use the analogy
of the fighter jet or something.

00:02:42.680 --> 00:02:47.197
And they would say, listen, we have
to have suppliers who we can trust,

00:02:47.280 --> 00:02:51.280
and they provide us with tools
that we can use in combat,

00:02:51.440 --> 00:02:54.677
and we can't have some guy
in Silicon Valley suddenly saying, no,

00:02:54.760 --> 00:02:56.917
you can't use it
for this fact or the other.

00:02:57.000 --> 00:03:02.637
And their view would be they need to have
full operational ability to use it,

00:03:02.720 --> 00:03:05.077
and you are somewhat
capriciously in that view,

00:03:05.160 --> 00:03:07.157
putting all of these constraints on it.

00:03:07.240 --> 00:03:10.357
Yeah. So so I would
say we agree with them.

00:03:10.440 --> 00:03:15.277
We don't want to do that.
We agree on 99%, not only the contract

00:03:15.360 --> 00:03:19.277
we were operating on together,
we agree on like this much broader

00:03:19.360 --> 00:03:22.717
set of things that that, you know,
we're haven't even done yet

00:03:22.800 --> 00:03:26.197
or are just starting to do. So,
you know, I just want to say again, like,

00:03:26.280 --> 00:03:28.917
there's so much more agreement
here than there is this, I.

00:03:29.000 --> 00:03:31.037
Mean, listening to you,
it does seem to me extraordinary

00:03:31.120 --> 00:03:33.357
that, you know,
you end up in this situation

00:03:33.440 --> 00:03:35.340
where you're going to be taken out of,

00:03:36.080 --> 00:03:38.277
all work with, with the Pentagon.

00:03:38.360 --> 00:03:41.557
That said the other, because I've
been thinking about this quite a lot.

00:03:41.640 --> 00:03:47.637
The the conclusion, the logical
conclusion of total control

00:03:47.720 --> 00:03:52.477
by the Pentagon and no ability
for the supply to do anything

00:03:52.560 --> 00:03:55.597
is surely some sort of effective
nationalisation of those

00:03:55.680 --> 00:03:57.517
AI companies that work with the Pentagon.

00:03:57.600 --> 00:04:01.237
Do you think that's a risk? I think over
the coming years, we're going to have

00:04:01.320 --> 00:04:08.157
to have a discussion about
the role of AI and the government.

00:04:08.240 --> 00:04:11.437
You know, AI is becoming
something that has implications

00:04:11.520 --> 00:04:14.560
at the level of all of humanity, not,

00:04:15.440 --> 00:04:20.237
you know, for national security in the
sense of the contest between nations

00:04:20.320 --> 00:04:23.677
for national security, in the sense of,
you know, can individuals

00:04:23.760 --> 00:04:27.637
use this technology for destruction,
for national security, in the sense of

00:04:27.720 --> 00:04:31.117
are the models themselves
a risk to national security

00:04:31.200 --> 00:04:34.200
for economics in terms of their impacts?

00:04:34.320 --> 00:04:37.517
And it's it's not about one
particular administration.

00:04:37.600 --> 00:04:39.397
It's not about one particular department.

00:04:39.480 --> 00:04:43.040
It's not about specific
military operations.

00:04:43.480 --> 00:04:46.477
Anyone who thinks that this is
what that's about, that's that's

00:04:46.560 --> 00:04:50.157
that's a misunderstood this is this,
this, this is bigger than that.

00:04:50.240 --> 00:04:52.797
This is big. This is about
the next administration, too.

00:04:52.880 --> 00:04:56.317
This is about, you know, Congress
when there's different Congress people,

00:04:56.400 --> 00:04:59.957
this is about, you know,
a different geopolitical situation

00:05:00.040 --> 00:05:01.517
that we're in than the one we're in now.

00:05:01.600 --> 00:05:06.877
It's it's, you know, we should not make it
about what's happening right, right now.

00:05:06.960 --> 00:05:11.560
And the thing we're dealing
with here is AI is very powerful.

00:05:12.280 --> 00:05:18.120
So there's actually a dual dilemma
here where AI has the power to make

00:05:18.800 --> 00:05:22.677
private actors more powerful
than they've ever been before.

00:05:22.760 --> 00:05:25.237
Folks have, you know, some people
have criticised me for that.

00:05:25.320 --> 00:05:27.957
Or you think as a private actor
you're more powerful in government?

00:05:28.040 --> 00:05:31.477
Actually, I've been worried about
the risk of that for the longest time.

00:05:31.560 --> 00:05:34.957
Like I'm worried about
that this technology abstractly

00:05:35.040 --> 00:05:37.877
in the hands of government,
not just the US government,

00:05:37.960 --> 00:05:41.197
other governments, democratic
governments, autocratic government

00:05:41.280 --> 00:05:46.920
has the ability to vest them
with unprecedented power as well.

00:05:47.680 --> 00:05:50.517
And so when we talked about
fully autonomous weapons,

00:05:50.600 --> 00:05:51.877
I said there were two objections.

00:05:51.960 --> 00:05:54.197
The first one I said was reliability.

00:05:54.280 --> 00:05:57.880
The second one I said was was
what I would call oversight.

00:05:58.960 --> 00:06:03.797
And this is the idea that, you know, right
now you have an army of human soldiers,

00:06:03.880 --> 00:06:06.637
and there are norms about
serving in the military.

00:06:06.720 --> 00:06:10.077
You're supposed to follow orders,
but, you know, if something crazy,

00:06:10.160 --> 00:06:12.997
if something crazy enough happened,
the soldiers would say,

00:06:13.080 --> 00:06:14.317
I'm not going to do that. Right.

00:06:14.400 --> 00:06:16.757
You have basically,
you have a bunch of norms about,

00:06:16.840 --> 00:06:19.157
you know, how soldiers serve, right?

00:06:19.240 --> 00:06:22.240
You know what they see their duties to be.

00:06:22.440 --> 00:06:25.677
What if you have an army of 10
million drones instead of,

00:06:25.760 --> 00:06:27.837
you know, 10 million human soldiers?

00:06:27.920 --> 00:06:31.237
What what are the norms of the
of the of the AI driven drones?

00:06:31.320 --> 00:06:34.757
I think if we handled this wrongly,
you could have a situation where

00:06:34.840 --> 00:06:37.677
there's a very small number
of people or one person

00:06:37.760 --> 00:06:40.637
who has their hand
on the button and kind of,

00:06:40.720 --> 00:06:42.477
you know, controls
those 10 million drones.

00:06:42.560 --> 00:06:46.717
And we need to answer these questions
because we don't want to make companies

00:06:46.800 --> 00:06:51.157
more powerful than the government,
but we also don't want to make government.

00:06:51.240 --> 00:06:54.077
So powerful that it
that it can't be stopped.

00:06:54.160 --> 00:06:55.797
We have both problems at once.

00:06:55.880 --> 00:06:58.757
So you basically just said to me,
this technology is too powerful

00:06:58.840 --> 00:07:02.037
to be in the hands of a few private
companies, and it's too powerful

00:07:02.120 --> 00:07:03.157
to be in the hands of a government.

00:07:03.240 --> 00:07:05.390
That is unfortunately,
the situation where.