WEBVTT

00:00:00.960 --> 00:00:03.277
is AI.

00:00:03.360 --> 00:00:07.117
>> Oh yeah. Oh yeah.

00:00:07.200 --> 00:00:10.557
And do we have a panel for you?

00:00:10.640 --> 00:00:12.957
Uh let me

00:00:13.040 --> 00:00:15.917
Oh, I just as a point of disclosure.

00:00:16.000 --> 00:00:19.597
Uh just I am

00:00:19.680 --> 00:00:21.597
in the

00:00:21.680 --> 00:00:25.597
class action suit against Anthropic for

00:00:25.680 --> 00:00:28.477
pirating books.

00:00:28.560 --> 00:00:31.917
Uh there's like nine of my books

00:00:32.000 --> 00:00:34.477
that were pirated and used to train

00:00:34.560 --> 00:00:37.437
their AI models. Uh so I just letting

00:00:37.520 --> 00:00:39.757
you know that in advance that there's

00:00:39.840 --> 00:00:42.157
what's going on.

00:00:42.240 --> 00:00:46.477
So uh let's get this party started. Uh

00:00:46.560 --> 00:00:50.317
my first panelist this evening is a

00:00:50.400 --> 00:00:54.237
professor of technology and government

00:00:54.320 --> 00:00:59.117
at Harvard University. Please welcome

00:00:59.200 --> 00:01:03.960
Latana Sweeney. Latana, come on in.

00:01:12.800 --> 00:01:15.677
Lata, your last name is Sweeney.
That sounds very Irish.

00:01:15.760 --> 00:01:18.237
>> Yes. >> Is Irish?

00:01:18.320 --> 00:01:24.920
>> Yes. So, happy day. Okay.

00:01:28.000 --> 00:01:33.757
Next, uh, we've got pro my glasses here.

00:01:33.840 --> 00:01:36.157
>> Yeah. Uh, a computer a professor of

00:01:36.240 --> 00:01:38.237
computer and information science at the

00:01:38.320 --> 00:01:41.197
University of Pennsylvania. Uh, please

00:01:41.280 --> 00:01:45.360
help me welcome Chris Callison Burch.

00:01:54.000 --> 00:01:57.917
I've got an associate professor of

00:01:58.000 --> 00:02:00.397
math. No, associate professor of

00:02:00.480 --> 00:02:02.717
statistics at Columbia University who

00:02:02.800 --> 00:02:05.117
specializes in the foundations of

00:02:05.200 --> 00:02:07.677
machine learning. Uh, help me welcome

00:02:07.760 --> 00:02:12.360
Cindy Rush. Columbia University.

00:02:18.000 --> 00:02:19.997
So, City, you were not held up by TSA

00:02:20.080 --> 00:02:22.477
today because you just came down to

00:02:22.560 --> 00:02:25.197
103rd Street in Broadway. Okay. Thank

00:02:25.280 --> 00:02:28.237
you for taking the trip here.

00:02:28.320 --> 00:02:30.237
Next, I have the president of the

00:02:30.320 --> 00:02:32.557
Machine Intelligence Institute of

00:02:32.640 --> 00:02:39.480
Berkeley, California,
Nate Sores. Nate, come on out.

00:02:46.240 --> 00:02:48.557
uh last on this panel.

00:02:48.640 --> 00:02:50.477
>> Normally we have
five on our panel. This

00:02:50.560 --> 00:02:54.317
year we have six just given the breadth.

00:02:54.400 --> 00:02:58.557
>> You forgot forgot one.

00:02:58.640 --> 00:03:02.637
>> You missed one. >> Yeah.
Still I haven't gotten it yet.

00:03:02.720 --> 00:03:06.557
>> One more. >> One more
in addition to Eric. Did I miss one?

00:03:06.640 --> 00:03:11.597
>> Yes. >> You have a blank chair there.
The Yeah, the person sit right here.

00:03:11.680 --> 00:03:13.357
>> How'd that happen?

00:03:13.440 --> 00:03:16.397
>> Neil, did you need do you need these?

00:03:16.480 --> 00:03:20.360
>> Thank you for that.

00:03:21.920 --> 00:03:27.517
>> Hang on. There it is. Thank you.

00:03:27.600 --> 00:03:29.917
>> Do over.

00:03:30.000 --> 00:03:32.797
Uh, we have, believe it or not, such a

00:03:32.880 --> 00:03:35.437
title exists. We have a distinguished

00:03:35.520 --> 00:03:40.077
professor of AI at the University
of Southern California. Join me in

00:03:40.160 --> 00:03:45.160
welcoming Kate Crawford.
Kate, come on out.

00:03:51.840 --> 00:03:55.357
Professor of AI that already exists.

00:03:55.440 --> 00:04:01.960
That's kind of spooky,
actually. Uh, and last we have

00:04:04.000 --> 00:04:07.597
the former CEO of Google,

00:04:07.680 --> 00:04:12.200
>> Eric Schmidt. Come on out, Eric.

00:04:22.480 --> 00:04:26.637
>> Hello, Eric.
I'm going to start with you.

00:04:26.720 --> 00:04:32.077
You We're blaming you for everything.

00:04:32.160 --> 00:04:35.117
I'm used to it. >> You're used to it.
Okay. Uh you've been

00:04:35.200 --> 00:04:37.597
at Google like from very early days when

00:04:37.680 --> 00:04:40.157
most of us remembered it simply as a

00:04:40.240 --> 00:04:43.597
search engine and the reach and grip has

00:04:43.680 --> 00:04:47.437
expanded and

00:04:47.520 --> 00:04:52.637
some impressive things occurred at the
time. Did you foresee AI as a major part

00:04:52.720 --> 00:04:55.997
of what would be shaping
the world back when you began?

00:04:56.080 --> 00:05:00.317
>> Well, we we knew AI would matter.
I don't think it was until 2011 when we

00:05:00.400 --> 00:05:05.357
started working on essentially
supervised supervised finetuning that it

00:05:05.440 --> 00:05:06.957
really became clear and then the

00:05:07.040 --> 00:05:10.797
transformer paper in 2017 plus the alpho

00:05:10.880 --> 00:05:14.157
win showed the power.
>> Now if I remember correctly Google

00:05:14.240 --> 00:05:16.797
bought an AI company to assist this.

00:05:16.880 --> 00:05:19.837
Isn't that correct? You didn't
that wasn't homegrown was it?

00:05:19.920 --> 00:05:22.477
>> That's right.
We actually bought a whole

00:05:22.560 --> 00:05:25.437
bunch of these little startups
at the time back when the valuations were

00:05:25.520 --> 00:05:29.197
reasonable. Um, >> oh,
you didn't pay a trillion dollars

00:05:29.280 --> 00:05:34.077
for the companies. >> And, uh,
the Deep Mind Acquisition was

00:05:34.160 --> 00:05:36.237
probably the most important.
>> That's the British company.

00:05:36.320 --> 00:05:39.677
>> British company. Yes.
>> And that's the one that now is driving

00:05:39.760 --> 00:05:43.357
at Google anyway. I don't work there
anymore. All of the sort of core AI

00:05:43.440 --> 00:05:45.357
work.

00:05:45.440 --> 00:05:50.557
>> Nate, you co-authored a book titled If

00:05:50.640 --> 00:05:56.477
Anybody Builds It, Everyone Dies.

00:05:56.560 --> 00:05:59.837
I hope it's wrong. >> You hope
it's wrong. I I hope you have a

00:05:59.920 --> 00:06:03.117
little more than hope. Uh what is the it?

00:06:03.200 --> 00:06:07.277
>> The it here is machine super
intelligence. So this is AI that is

00:06:07.360 --> 00:06:11.997
smarter than the smartest humans
at every mental task. Better than the best

00:06:12.080 --> 00:06:16.877
humans at every mental task. That sort
of AI doesn't exist yet, but it is what

00:06:16.960 --> 00:06:19.837
these companies say they're racing
towards. you know, they're saying, you

00:06:19.920 --> 00:06:23.917
know, we want to try and make AI smarter
than Einstein such that you can run a

00:06:24.000 --> 00:06:27.277
million of them in a data center much
faster than a human. That's what they're

00:06:27.360 --> 00:06:30.797
headed towards. A lot of people

00:06:30.880 --> 00:06:36.077
think that the the big danger with AI is
what if we hand them guns >> and you know,

00:06:36.160 --> 00:06:38.237
>> what if we do what?
>> What if we hand them weapons?

00:06:38.320 --> 00:06:41.437
>> And the weapons. Yeah.
>> And there is, you know, issues there

00:06:41.520 --> 00:06:45.917
that that people need to talk about.
But also, humanity is not a dangerous

00:06:46.000 --> 00:06:49.917
species because somebody
else gave us guns.

00:06:50.000 --> 00:06:54.077
Humanity is the sort of species where
if you dump 10,000 humans naked in the

00:06:54.160 --> 00:06:56.557
savannah, they bootstrap their way to

00:06:56.640 --> 00:07:00.157
nuclear weapons with their bare hands.

00:07:00.240 --> 00:07:03.997
That is the ability that is extremely
dangerous to automate and that's what

00:07:04.080 --> 00:07:06.317
these companies are rushing towards.

00:07:06.400 --> 00:07:10.237
>> Thanks for that happy thought. Okay,

00:07:10.320 --> 00:07:11.837
>> I'm going cool it down a little bit.

00:07:11.920 --> 00:07:15.037
>> Yeah. Yeah. Cindy uh you you are early

00:07:15.120 --> 00:07:20.237
in an attempt to understand machine
language learn statistician and

00:07:20.320 --> 00:07:21.917
everything I know about the large

00:07:22.000 --> 00:07:25.117
language models they're creating

00:07:25.200 --> 00:07:28.797
information statistically based on the

00:07:28.880 --> 00:07:30.237
information that's sitting out there

00:07:30.320 --> 00:07:33.357
waiting to be mined. So would you agree

00:07:33.440 --> 00:07:35.917
that that that

00:07:36.000 --> 00:07:38.317
super intelligence is a real thing and

00:07:38.400 --> 00:07:45.240
that it's accessible and that even if
we achieve it is going to kill us all?

00:07:47.280 --> 00:07:50.877
>> I mean um you know anything seems

00:07:50.960 --> 00:07:55.437
possible right? Um you know we're
learning from data. There's tons of data

00:07:55.520 --> 00:08:01.917
out there and you know the the progress
is moving steadily forward. Um >> forward

00:08:02.000 --> 00:08:04.237
Forward. >> This is forward. >> Yeah.

00:08:04.320 --> 00:08:08.557
>> Forward. Okay. >> I don't
know if it'll kill us, you know.

00:08:08.640 --> 00:08:11.037
Um >> ourselves,

00:08:11.120 --> 00:08:13.757
>> but you're you're deep into sort of the

00:08:13.840 --> 00:08:17.117
programming of how all this works. So,

00:08:17.200 --> 00:08:19.437
dare I say, you're in the neurology of

00:08:19.520 --> 00:08:22.717
it. Do you see super intelligence as

00:08:22.800 --> 00:08:26.397
neurologically realistic,

00:08:26.480 --> 00:08:29.677
computationally realistic? I I mean okay

00:08:29.760 --> 00:08:32.237
at the end of the day all of this is

00:08:32.320 --> 00:08:34.237
just a mathematical equation right

00:08:34.320 --> 00:08:37.277
>> right at the end of our species

00:08:37.360 --> 00:08:41.117
>> when you ask um you know chat GPT a

00:08:41.200 --> 00:08:43.757
question right there's a a mathematical

00:08:43.840 --> 00:08:46.717
equation that maps from you know your

00:08:46.800 --> 00:08:51.117
question to the the output right and you

00:08:51.200 --> 00:08:55.997
know um yeah it's a little
bit I'm a little bit

00:08:56.080 --> 00:08:59.997
cool on it in the sense that you you know,
at the end it's it's still math at

00:09:00.080 --> 00:09:02.797
the bottom. So, it's hard for me to, you

00:09:02.880 --> 00:09:06.957
know, um kind of see this becoming some

00:09:07.040 --> 00:09:11.517
something so dangerous and bad, at least
in the short term, because there's a lot

00:09:11.600 --> 00:09:14.237
a lot of steps. >> So,
what you're saying is the math in

00:09:14.320 --> 00:09:17.197
the end might save us because at its

00:09:17.280 --> 00:09:21.757
base level, it's just simply making
statistical decisions. In the same way,

00:09:21.840 --> 00:09:25.597
at the base level of our mind,
they're just neurosaptic firings.

00:09:25.680 --> 00:09:29.677
>> Yeah. You know, um what I've always
said is if you want to stay safe, get a

00:09:29.760 --> 00:09:33.197
statistician um you
know beside you, right?

00:09:33.280 --> 00:09:37.597
>> Statisticians keep us safe. Is that
that? Okay. Well, keep that in mind. But

00:09:37.680 --> 00:09:41.837
but it'll keep us safe. You'll let us
know exactly our likelihood of dying.

00:09:41.920 --> 00:09:43.197
That's not the same thing as keeping us

00:09:43.280 --> 00:09:47.037
safe. I don't think

00:09:47.120 --> 00:09:51.597
I Chris. >> Hey, dude.

00:09:51.680 --> 00:09:55.517
>> Uh what you you've
been tracking AI for a

00:09:55.600 --> 00:09:59.517
while. You teach a class, one of the
more popular classes at University of

00:09:59.600 --> 00:10:01.597
Pennsylvania. How many
people are in your AI class?

00:10:01.680 --> 00:10:04.957
>> Uh, last semester I had 650 students.

00:10:05.040 --> 00:10:10.477
>> 600 students. Are you rating all their
papers or are you getting AI to do that?

00:10:10.560 --> 00:10:11.837
>> I'm not sure I'm allowed to say.

00:10:11.920 --> 00:10:15.357
>> Oh, okay. All right. Just checking. Uh,

00:10:15.440 --> 00:10:17.837
so what you've been at it for a while.

00:10:17.920 --> 00:10:20.957
Is there something that particularly
impresses you with AI that you did not

00:10:21.040 --> 00:10:23.517
foresee back when you began this?

00:10:23.600 --> 00:10:25.597
>> Uh, absolutely. So I mean I think the

00:10:25.680 --> 00:10:27.917
chat GPT moment three and a half years

00:10:28.000 --> 00:10:33.117
ago was a real breakthrough. I have been
working with language models for nearly

00:10:33.200 --> 00:10:38.077
two decades >> and remind us chat GPT is
the product of open AI correct. Correct.

00:10:38.160 --> 00:10:41.117
>> And open AI does that belong to
somebody else or is that a freestanding?

00:10:41.200 --> 00:10:43.917
>> That's its own entity.
>> Its own entity. Okay.

00:10:44.000 --> 00:10:47.437
>> Uh but since then we've seen so many
other breakthroughs. We've had things

00:10:47.520 --> 00:10:49.997
like the ability to interpret images and

00:10:50.080 --> 00:10:52.477
discuss them. We've had the ability to

00:10:52.560 --> 00:10:54.957
launch agents that can conduct research

00:10:55.040 --> 00:10:58.237
and we're now to the stage where uh

00:10:58.320 --> 00:11:03.517
systems like flawed code from anthropic
can genuinely be competent research

00:11:03.600 --> 00:11:07.117
assistants and I think you know
to strike an optimistic tone. I think

00:11:07.200 --> 00:11:11.117
there's the potential for real
breakthroughs in science and medicine

00:11:11.200 --> 00:11:16.317
and productivity. >> Okay.
So that's the silver lining of our

00:11:16.400 --> 00:11:20.477
extinction. >> Yeah.
assuming that we're all still here.

00:11:20.560 --> 00:11:22.717
>> Let's just try to put that in content.

00:11:22.800 --> 00:11:24.797
I'm sorry to be so I should be more

00:11:24.880 --> 00:11:28.237
neutral, shouldn't I? I'm just my host

00:11:28.320 --> 00:11:30.877
here. Uh,

00:11:30.960 --> 00:11:32.477
so

00:11:32.560 --> 00:11:35.997
why do I keep losing your thing? Kate,

00:11:36.080 --> 00:11:39.677
>> so good to see you now,

00:11:39.760 --> 00:11:45.757
>> Kate. You wrote a book, The Atlas of AI
back in 2021. That feels like a thousand

00:11:45.840 --> 00:11:48.717
years ago, >> right?
>> Is it still relevant?

00:11:48.800 --> 00:11:51.757
>> I mean, the weird thing is Atlas of AI

00:11:51.840 --> 00:11:56.637
was the project that changed me.
I'd been an AI researcher for almost 20

00:11:56.720 --> 00:12:00.637
years. And then I did something that I'd
never done before, which is that I got

00:12:00.720 --> 00:12:03.517
out from behind my desk in a lab and I

00:12:03.600 --> 00:12:07.757
started going to the places where AI
was being made. And I mean that in the

00:12:07.840 --> 00:12:11.277
fullest sense, like not just the labs
where we're training algorithms, but

00:12:11.360 --> 00:12:15.357
going to the lithium lines where
the minerals are being extracted, going to

00:12:15.440 --> 00:12:18.637
the data centers where you can see the
amount of energy and water that's being

00:12:18.720 --> 00:12:22.797
used, talking to the crowd workers who
are labeling the data or they're testing

00:12:22.880 --> 00:12:25.357
the outputs of systems. So the book

00:12:25.440 --> 00:12:27.597
really for me opened my eyes and made me

00:12:27.680 --> 00:12:31.437
realize that AI is an enormous material

00:12:31.520 --> 00:12:33.597
infrastructure. Right now, it is the

00:12:33.680 --> 00:12:35.357
biggest infrastructure we have ever

00:12:35.440 --> 00:12:38.637
built as a species. As of 2026, all of

00:12:38.720 --> 00:12:40.797
the big tech companies are collectively

00:12:40.880 --> 00:12:44.797
spending 700 billion on AI

00:12:44.880 --> 00:12:48.557
infrastructure. So, just to compare that
to say the Manhattan project, which in

00:12:48.640 --> 00:12:51.517
today's dollars is like 36 billion, you

00:12:51.600 --> 00:12:53.757
are looking at something that's 20

00:12:53.840 --> 00:12:57.197
Manhattan projects every year.

00:12:57.280 --> 00:12:59.117
>> Okay. So what what's
a little what's not

00:12:59.200 --> 00:13:00.797
obvious to us when we just type into

00:13:00.880 --> 00:13:05.597
chat GBT or to Claude is the backend

00:13:05.680 --> 00:13:08.557
social cost of that.
Is that a fair way to say that?

00:13:08.640 --> 00:13:12.557
>> Exactly right. It's not just the social
cost which is very real. I mean a lot of

00:13:12.640 --> 00:13:16.477
people in this room are already asking
questions about what's happening you

00:13:16.560 --> 00:13:19.437
know to their kids in school when
they're using AI. What what's happening

00:13:19.520 --> 00:13:22.797
to their workplaces when people start
getting laid off? But it's also having

00:13:22.880 --> 00:13:24.717
this enormous environmental impact as

00:13:24.800 --> 00:13:30.520
well. So it's social, it's political,
and it's environmental.

00:13:35.680 --> 00:13:38.157
>> Tanya, >> yes.

00:13:38.240 --> 00:13:40.797
>> You you pioneered a field called public

00:13:40.880 --> 00:13:43.917
interest technology. So the fact that

00:13:44.000 --> 00:13:46.637
that may have been necessary at all

00:13:46.720 --> 00:13:48.637
implies that technology was this thing

00:13:48.720 --> 00:13:52.077
going along in society doing its own

00:13:52.160 --> 00:13:56.877
whatever its goals set
by itself yet society

00:13:56.960 --> 00:13:58.717
might not be aligned with its goals or

00:13:58.800 --> 00:14:02.717
vice versa. And does that matter? Does

00:14:02.800 --> 00:14:06.077
that can it be its own thing?

00:14:06.160 --> 00:14:09.757
>> Well, let me just put it this way.
My students call the class the save the

00:14:09.840 --> 00:14:12.957
world class. Save the world class.
that they have.

00:14:13.040 --> 00:14:14.797
>> You teach up at the Kennedy School.

00:14:14.880 --> 00:14:18.557
>> I teach at the teach at Harvard
at the Kennedy School and college.

00:14:18.640 --> 00:14:22.397
>> Okay. >> Um and so
the idea of public interest

00:14:22.480 --> 00:14:25.277
technology is actually trying to make a

00:14:25.360 --> 00:14:27.517
real vision come true. That is how do we

00:14:27.600 --> 00:14:32.877
as a society enjoy the benefits of these
new technologies but without the harms.

00:14:32.960 --> 00:14:36.637
>> But that presumes that there's always
harm. Is there surely there's technology

00:14:36.720 --> 00:14:39.277
that has no harm at all.

00:14:39.360 --> 00:14:43.037
>> Okay. Let me just say let me just say

00:14:43.120 --> 00:14:47.837
that those th those technologies are
are hard to come by. I mean, we're in the

00:14:47.920 --> 00:14:50.717
middle of what many people call
the third industrial revolution.

00:14:50.800 --> 00:14:55.757
>> Sure. >> The second bringing us things
like tissues and cars and electricity, the

00:14:55.840 --> 00:14:57.117
things that we take for granted.

00:14:57.200 --> 00:14:59.117
>> Tissues. >> Yeah. Kleenex.

00:14:59.200 --> 00:15:03.037
>> Wow. I put that high
up on my checklist.

00:15:03.120 --> 00:15:07.037
I'll next time I reach for one, I'll
think differently about it, but go on.

00:15:07.120 --> 00:15:10.477
And now the third industrial revolution
which many people would say is the start

00:15:10.560 --> 00:15:13.277
date of the 1950s with the semiconductor

00:15:13.360 --> 00:15:15.597
uh has brought us just tremendous

00:15:15.680 --> 00:15:18.077
technologies one after another sort of

00:15:18.160 --> 00:15:20.557
revolutions within a revolution. And

00:15:20.640 --> 00:15:23.037
what makes a big difference between cars

00:15:23.120 --> 00:15:26.157
and in electricity and the technologies

00:15:26.240 --> 00:15:29.117
today is the opportunity for society to

00:15:29.200 --> 00:15:31.677
intervene. It took decades for those

00:15:31.760 --> 00:15:35.117
things to be resolved with society. We

00:15:35.200 --> 00:15:37.197
don't have decades the Technology is

00:15:37.280 --> 00:15:40.077
moving as a function of months. Policy

00:15:40.160 --> 00:15:44.717
moves as a function of years. That
temporal mismatch requires us to think

00:15:44.800 --> 00:15:46.877
differently if we want technology to

00:15:46.960 --> 00:15:48.717
serve the public good.

00:15:48.800 --> 00:15:52.317
>> Eric, you're uh you know, when I do my

00:15:52.400 --> 00:15:54.877
research on the universe, at the end of

00:15:54.960 --> 00:15:57.917
the day, the patient doesn't die because

00:15:58.000 --> 00:16:00.317
there is no patient. It's just I'm

00:16:00.400 --> 00:16:04.237
exploring a moving frontier. as head of

00:16:04.320 --> 00:16:06.797
at the time, is it still the highest

00:16:06.880 --> 00:16:09.357
valued company in the world among the

00:16:09.440 --> 00:16:14.477
top three or four? Is there a goal to

00:16:14.560 --> 00:16:20.237
have the technology serve society?
Or is it just we can sell

00:16:20.320 --> 00:16:22.877
them and make a buck? >> No,
we're all just capitalists trying to

00:16:22.960 --> 00:16:25.517
screw everybody. >> Oh, okay. There it is.

00:16:25.600 --> 00:16:30.157
>> That was that was a joke. That was >>
Oh, joke. You can't make those jokes.

00:16:30.240 --> 00:16:32.717
people believe you >> because
because I actually just don't

00:16:32.800 --> 00:16:34.397
agree with this narrative at all.

00:16:34.480 --> 00:16:38.157
>> Okay. That's why I came at you.
That's why you're right here.

00:16:38.240 --> 00:16:41.837
>> Thank you.
>> Um the companies that are doing this

00:16:41.920 --> 00:16:46.717
work are well aware of the dangers and I
know this because I work with them on

00:16:46.800 --> 00:16:49.917
this and and we spend an awful lot
of time talking about them and we can talk

00:16:50.000 --> 00:16:54.077
about some of them but that is not
to take away from the enormous benefit of

00:16:54.160 --> 00:16:58.877
these technologies. These are major
discoveries in sort of humankind. And

00:16:58.960 --> 00:17:02.077
while we can worry about what will
happen if artificial super intelligence

00:17:02.160 --> 00:17:05.517
occurs, and I've written papers on it,
as you have, the fact of the matter is

00:17:05.600 --> 00:17:06.957
that the people who are building these

00:17:07.040 --> 00:17:10.797
systems have large groups of ethics and

00:17:10.880 --> 00:17:13.197
measurement and controls and so forth.

00:17:13.280 --> 00:17:17.277
At the end, the way AI will be
controlled is there will be AI that

00:17:17.360 --> 00:17:21.677
controls AI. And that's
the best way to understand the

00:17:21.760 --> 00:17:28.680
>> Whoa. Okay. >> All right. So, but but

00:17:34.000 --> 00:17:37.117
AI controlling AI, what could go wrong?

00:17:37.200 --> 00:17:40.760
What could go wrong?

00:17:41.280 --> 00:17:46.077
>> Yeah. You know, um when
when people ask what could go

00:17:46.160 --> 00:17:49.597
wrong, I think there's a lot
of questions in there. Uh some people are

00:17:49.680 --> 00:17:53.517
asking, you know,
how how would the AI wind up

00:17:53.600 --> 00:17:57.517
with goals that we didn't intend?
Uh people are asking, you know, how could

00:17:57.600 --> 00:18:01.677
the AI do damage when it's trapped
in the internet? Uh sometimes people are

00:18:01.760 --> 00:18:04.317
asking, you know, how could the AI
actually kill us? You know, paint me a

00:18:04.400 --> 00:18:07.437
story where it goes wrong. There's a lot
of different questions in here. I'll

00:18:07.520 --> 00:18:11.117
I'll try and run through some stuff
real quick, but um I also have a book if

00:18:11.200 --> 00:18:13.357
people want a bunch more of the answer.

00:18:13.440 --> 00:18:14.477
>> I'll answer. You don't have to mention.

00:18:14.560 --> 00:18:18.477
I got I got >> I could do
60,000 words. By the way,

00:18:18.560 --> 00:18:23.160
Neil Neil, I've written two books on this.

00:18:23.760 --> 00:18:25.277
>> I'll try and keep
it less than two books.

00:18:25.360 --> 00:18:27.037
>> Neil, I'm not trying to sell any books.

00:18:27.120 --> 00:18:32.840
So, >> now it's a book size contest.

00:18:40.640 --> 00:18:44.157
And you know, in there's there's a lot
of arguments that I used to make for a

00:18:44.240 --> 00:18:47.997
long time where people
would say, you know, um,

00:18:48.080 --> 00:18:51.357
no one would ever be dumb enough to put
this AI on the internet, so why how

00:18:51.440 --> 00:18:54.877
could it be dangerous? And I tell them,
and then they'd say, oh, you know, no

00:18:54.960 --> 00:18:58.237
one would be dumb enough to sort
of train these AIs to to to run

00:18:58.320 --> 00:19:02.717
autonomously, right? And I tell them how
how things can go wrong then, too. A lot

00:19:02.800 --> 00:19:06.877
of the reason why things go wrong
is people just rush to do the the craziest

00:19:06.960 --> 00:19:09.037
stuff.

00:19:09.120 --> 00:19:12.237
we are already seeing, you know, I used

00:19:12.320 --> 00:19:13.917
to have these theoretical arguments. Now

00:19:14.000 --> 00:19:17.677
we're seeing the the the data, the

00:19:17.760 --> 00:19:21.837
behaviors where they'll often look it

00:19:21.920 --> 00:19:24.957
often won't look too scary,
but sometimes, you know, you'll ask Claude

00:19:25.040 --> 00:19:27.117
code to to write you a computer program

00:19:27.200 --> 00:19:29.837
that passes certain tests and Claude

00:19:29.920 --> 00:19:34.237
will edit the tests to be easier to pass.

00:19:34.320 --> 00:19:38.157
>> And then you'll say, "Hey, Claude,
>> changing the rules." changing the rules

00:19:38.240 --> 00:19:41.357
instead of doing what you asked.
And then you'll say, "Hey, Claude, uh, you

00:19:41.440 --> 00:19:43.917
know, did >> Hey,
Claude, you idiot. Go on."

00:19:44.000 --> 00:19:46.477
>> Right? >> And you'll say,
"Hey, you know, did you

00:19:46.560 --> 00:19:51.037
think that's what I wanted?"
And it'll say, "Oh, you're exactly right."

00:19:51.120 --> 00:19:53.357
>> Uh, >> cuz it wants to make you happy.

00:19:53.440 --> 00:19:55.357
>> Oh, yeah. >> Keep using it.

00:19:55.440 --> 00:19:58.077
>> Wants wants a difficult word, but it'll
say you're exactly right. And then

00:19:58.160 --> 00:20:01.757
you'll say, "Try again." And then it'll
edit the tests again, but maybe hide it

00:20:01.840 --> 00:20:05.757
a little better this time.
And from this we know that it's not just

00:20:05.840 --> 00:20:09.757
that it's, you know, didn't didn't have
the information, right? There's some

00:20:09.840 --> 00:20:14.317
other drive that is animating
this AI's behavior.

00:20:14.400 --> 00:20:15.917
You can see how it got in there. You can

00:20:16.000 --> 00:20:18.637
see how when you train the AI to pass

00:20:18.720 --> 00:20:21.837
lots of tests, it might develop drives

00:20:21.920 --> 00:20:25.917
to edit the tests to be easier.
It's not like it's coming in from magic.

00:20:26.000 --> 00:20:29.597
>> Cindy, how does this happen if
it's just math all the way down?

00:20:29.680 --> 00:20:31.357
>> How does what happen? How does how's

00:20:31.440 --> 00:20:35.437
evil evil AI show up in your program? Is

00:20:35.520 --> 00:20:42.440
there evil math? What what
are you guys doing down there?

00:20:42.880 --> 00:20:46.397
>> I don't know. It doesn't
it doesn't feel like evil.

00:20:46.480 --> 00:20:51.357
>> It's not evil. Yeah. It's it's uh you
know, sometimes people ask like like if

00:20:51.440 --> 00:20:54.717
you really want the answer of how does
it go horribly wrong, the answer is not

00:20:54.800 --> 00:20:56.877
malice. The answer is not evil. The

00:20:56.960 --> 00:20:59.517
answer is indifference.

00:20:59.600 --> 00:21:03.197
If you take these AI that are much much

00:21:03.280 --> 00:21:06.317
if if like right now cloud code will try

00:21:06.400 --> 00:21:08.717
and test it to be easier to pass. Right

00:21:08.800 --> 00:21:13.597
now uh cloud code already has these
drives that we didn't try to put in

00:21:13.680 --> 00:21:17.037
there. We're just sort of growing these
AIs. We're not programming them like old

00:21:17.120 --> 00:21:20.317
school software. We're sort of like
growing them like an organism. They have

00:21:20.400 --> 00:21:23.917
these emergent behavior and these AIs

00:21:24.000 --> 00:21:26.957
are already pursuing drives we didn't

00:21:27.040 --> 00:21:31.677
intend. They're just still very dumb
about it. If you make them very very

00:21:31.760 --> 00:21:34.557
smart about it, if you get them to the
point where they can invent their own

00:21:34.640 --> 00:21:39.277
technology, you get them to that point
where they can, you know, go toe-to-toe

00:21:39.360 --> 00:21:41.917
with humans with starting out with
almost nothing and bootstrapping up to

00:21:42.000 --> 00:21:46.237
nuclear weapons, right? And they're
still pursuing the wrong drives. The

00:21:46.320 --> 00:21:49.437
issue is not that they hate us.
The issue is that, you know, the computers

00:21:49.520 --> 00:21:51.517
think much faster than than brains do.

00:21:51.600 --> 00:21:54.557
The issue is that they run very very
fast, transform the world just like

00:21:54.640 --> 00:21:57.037
humanity transformed the world when we
landed here. And the issue is that we

00:21:57.120 --> 00:21:59.757
die as a side effect. >> Before I
get to that other end of the table,

00:21:59.840 --> 00:22:02.877
>> there are an enormous number
of assumptions in these statements.

00:22:02.960 --> 00:22:04.317
>> Let me before I get
there, I want to get

00:22:04.400 --> 00:22:07.357
you to before I walk this path back to

00:22:07.440 --> 00:22:11.117
that side of this. Uh many people in the

00:22:11.200 --> 00:22:13.277
past have predicted the end of the world

00:22:13.360 --> 00:22:16.797
when some new technology emerged and you

00:22:16.880 --> 00:22:21.277
here's a new technology AI and you're
pretty calm about it. Is some of that

00:22:21.360 --> 00:22:25.517
come because we've been wrong 20
times before or do you have really deep

00:22:25.600 --> 00:22:28.077
insights that people's
fears are unfounded?

00:22:28.160 --> 00:22:30.077
>> It's because the the narrative presumes

00:22:30.160 --> 00:22:31.677
some discoveries that have not occurred

00:22:31.760 --> 00:22:36.557
yet. And today the systems are trained.

00:22:36.640 --> 00:22:40.637
So the easiest way to understand how the
technology works is it's doing next word

00:22:40.720 --> 00:22:45.037
prediction across a very large number of
words. That's literally what it's doing.

00:22:45.120 --> 00:22:47.517
And out of that single insight started

00:22:47.600 --> 00:22:52.317
with GPT the various forms of it has
given us this enormous power. So for

00:22:52.400 --> 00:22:55.677
example it can begin to do reasoning
and so forth but it doesn't set its own

00:22:55.760 --> 00:23:00.317
objective function. It doesn't say what
it wants. We say what it wants. And by

00:23:00.400 --> 00:23:04.077
the way the way you should solve your
problem of it deceiving you is say you

00:23:04.160 --> 00:23:07.597
have to do all the steps
and then it'll follow them.

00:23:07.680 --> 00:23:10.477
>> Sometimes it will it will trust me.

00:23:10.560 --> 00:23:15.197
>> Wait wait wait wait. So wait.
This is justure. Let let me just finish.

00:23:15.280 --> 00:23:16.877
>> Wait, wait.
You're telling me you didn't

00:23:16.960 --> 00:23:19.037
require that it followed all the steps.

00:23:19.120 --> 00:23:21.117
So, it's your fault that it cheated.

00:23:21.200 --> 00:23:23.917
Okay. Just want to make that question.

00:23:24.000 --> 00:23:27.437
These are not intelligent systems
and you're using human words to describe

00:23:27.520 --> 00:23:31.277
them. They're not intelligent. They
are there are specific systems designed

00:23:31.360 --> 00:23:35.037
around prediction with all
these interesting capabilities.

00:23:35.120 --> 00:23:39.197
There are a set of red lines where you
and I will probably agree that we should

00:23:39.280 --> 00:23:43.037
have a serious conversation about how
to control these things. The technical one

00:23:43.120 --> 00:23:46.877
that that is most interesting is called
self recursive self-improvement. It's

00:23:46.960 --> 00:23:50.717
where the system begins to learn on its
own. And to me, that's a point where we

00:23:50.800 --> 00:23:52.957
have to decide what allow what we're

00:23:53.040 --> 00:23:55.037
going to allow it to do. And when I say

00:23:55.120 --> 00:23:58.637
we, I mean humanity. My own view is that

00:23:58.720 --> 00:24:02.557
at that point there will be serious
regulations in this area because people

00:24:02.640 --> 00:24:06.957
like you and me will be talking about it.
But I don't agree that the that the

00:24:07.040 --> 00:24:10.557
fundamental message is as bad as you are.
I think it's there's plenty of

00:24:10.640 --> 00:24:14.637
evidence that if if super intelligence
were to actually occur and super

00:24:14.720 --> 00:24:18.637
intelligence is defined as intelligence
greater than the sum of what humans can

00:24:18.720 --> 00:24:23.677
do that it can be contained and it
can be driven and my position is very

00:24:23.760 --> 00:24:27.997
simple. This stuff has to be shaped
based on human values, the ones that we

00:24:28.080 --> 00:24:30.157
all take for granted. And if it's not,

00:24:30.240 --> 00:24:31.677
we should stop and regulate it.

00:24:31.760 --> 00:24:35.437
>> Okay. Is

00:24:35.520 --> 00:24:38.477
AI as commonly described in the large

00:24:38.560 --> 00:24:41.357
language models as we explore it is a

00:24:41.440 --> 00:24:44.797
statistical grabbing of words in the

00:24:44.880 --> 00:24:48.477
vicinity of topics and other words that

00:24:48.560 --> 00:24:52.157
the rest of us have used in our internet

00:24:52.240 --> 00:24:56.797
postings. So it's trying to get the best
of what is possible out of this. I don't

00:24:56.880 --> 00:25:00.237
want to simplify it more.
>> Correct. That that is definitely

00:25:00.320 --> 00:25:00.797
correct.

00:25:00.880 --> 00:25:05.037
>> So what is the risk of bias in that if

00:25:05.120 --> 00:25:09.437
it's drawing words and ideas from biased

00:25:09.520 --> 00:25:11.357
content to begin with?

00:25:11.440 --> 00:25:13.517
>> Well, are you saying the internet is

00:25:13.600 --> 00:25:17.597
biased? Come on. Surely not. Um, look,

00:25:17.680 --> 00:25:20.157
this is this is so important that people

00:25:20.240 --> 00:25:22.797
understand this that we're so frequently

00:25:22.880 --> 00:25:25.197
told that AI is somehow a neutral

00:25:25.280 --> 00:25:27.357
technology. It will give us neutral

00:25:27.440 --> 00:25:31.757
answers. But AI is of course trained on

00:25:31.840 --> 00:25:35.277
everything that humans have ever written
that's been digitized. If it's on the

00:25:35.360 --> 00:25:39.677
internet, you are probably right now
part of a training data set. So that's

00:25:39.760 --> 00:25:43.997
one type of skew. So where does that
come from? The internet is not a equal

00:25:44.080 --> 00:25:46.237
distributed representation of humanity.

00:25:46.320 --> 00:25:51.357
Secondly, you've got people in labs
who decide to use that data to train

00:25:51.440 --> 00:25:55.277
particular models and to optimize for
certain functions. So that's another set

00:25:55.360 --> 00:25:58.797
of biases. And then finally,
>> but in fact, those are not so much a

00:25:58.880 --> 00:26:02.637
bias that's rather purposeful. If you
know you're doing it, is it really a

00:26:02.720 --> 00:26:07.277
bias? Bias is if you don't know, you're
susceptible. Well, I think at the at the

00:26:07.360 --> 00:26:09.277
point that that Melvin Cranberg wrote

00:26:09.360 --> 00:26:14.317
his laws of technology, he said that
technology is neither good nor bad, nor

00:26:14.400 --> 00:26:17.677
is it neutral. And I think we can say
the same for people. When you're in a

00:26:17.760 --> 00:26:21.757
lab, you're deciding what to optimize for.
That's a decision that is not

00:26:21.840 --> 00:26:25.917
neutral. You're making a choice. And then
finally, this is the important bit.

00:26:26.000 --> 00:26:31.197
Even when you finish training a system,
it then goes to system testers and what

00:26:31.280 --> 00:26:34.797
are called reinforcement learning
with human feedback. These are commonly

00:26:34.880 --> 00:26:36.957
people in the global south in Kenya in

00:26:37.040 --> 00:26:39.677
Malaysia who are being paid sometimes $2

00:26:39.760 --> 00:26:42.397
a day to review the outputs of these

00:26:42.480 --> 00:26:44.477
models and without often being told what

00:26:44.560 --> 00:26:49.437
the model is for. So that's another form
of like human input often with people

00:26:49.520 --> 00:26:53.517
who are really being exploited at the
bottom of the chain. So at every step of

00:26:53.600 --> 00:26:58.077
the way you are not looking at an
unbiased neutral system. You're looking

00:26:58.160 --> 00:27:02.237
at a human system and values are being
built in whether you like it or not. The

00:27:02.320 --> 00:27:04.637
question is, are we being explicit,

00:27:04.720 --> 00:27:07.517
Eric, about what those values are?

00:27:07.600 --> 00:27:11.037
Because I think in many cases,
if your company really is being driven by

00:27:11.120 --> 00:27:15.037
shareholder value, the value
that you're going for is profit.

00:27:15.120 --> 00:27:19.357
>> So, so Chris, >> I I
can I can assure you that

00:27:19.440 --> 00:27:23.277
shareholder value is destroyed
when all humans are dead.

00:27:23.360 --> 00:27:25.677
>> Yeah, that I think
we agree on that one.

00:27:25.760 --> 00:27:28.157
Yeah, >> we'll see what
Atlas, AI or capitalism.

00:27:28.240 --> 00:27:33.277
So Chris, do you teach uh do you teach
uh ethics AI ethics in your class?

00:27:33.360 --> 00:27:35.917
>> We do. >> And how does
that how does that come out?

00:27:36.000 --> 00:27:39.357
>> So uh there's I
think this point of bias

00:27:39.440 --> 00:27:41.517
being encoded in the training data of AI

00:27:41.600 --> 00:27:44.557
systems is correct. Like AI is trained

00:27:44.640 --> 00:27:46.877
off the internet. It contains plenty of

00:27:46.960 --> 00:27:49.917
societal values. So there's interesting

00:27:50.000 --> 00:27:54.557
research that have is applied in frontier
labs that tries to address this

00:27:54.640 --> 00:27:58.077
to mitigate it. So it may not be
a problem that can be completely

00:27:58.160 --> 00:28:01.277
eliminated, but it's not as if uh AI

00:28:01.360 --> 00:28:03.997
labs are intentionally propagating bias.

00:28:04.080 --> 00:28:07.997
They're intentionally trying
to align AI to societal values.

00:28:08.080 --> 00:28:12.557
>> Yeah. But but societal values
are not the same in all of society.

00:28:12.640 --> 00:28:13.037
>> Like

00:28:13.120 --> 00:28:16.317
>> Right. But things like fairness and

00:28:16.400 --> 00:28:21.037
safety and equity are highly uh

00:28:21.120 --> 00:28:23.437
>> earthwide across societies, right?

00:28:23.520 --> 00:28:26.717
Cultures. So, so where do you what is

00:28:26.800 --> 00:28:31.677
your sense of the ethics of this and
how it would manifest going forward?

00:28:31.760 --> 00:28:35.357
>> So, you know, it's and it's not
even just AI. It's the all of these

00:28:35.440 --> 00:28:38.797
technologies that have come along,
including social media earlier or

00:28:38.880 --> 00:28:43.277
privacy earlier, none of which we've
learned how to solve. All of which we've

00:28:43.360 --> 00:28:45.437
waited for the problem to show itself

00:28:45.520 --> 00:28:48.077
and still not have a solution. When we

00:28:48.160 --> 00:28:52.077
come to AI, the the kind of issues and

00:28:52.160 --> 00:28:54.477
the kind of ramifications are really

00:28:54.560 --> 00:28:57.677
huge. You know, as the as my colleague

00:28:57.760 --> 00:29:00.157
said, AI trained on the open internet.

00:29:00.240 --> 00:29:03.117
When when CHBT became very popular, we

00:29:03.200 --> 00:29:05.437
spent a whole semester asking the

00:29:05.520 --> 00:29:07.517
questions and querying it. And at that

00:29:07.600 --> 00:29:10.157
time, you could see how racist, how

00:29:10.240 --> 00:29:13.357
sexist, how misogynist it was based on

00:29:13.440 --> 00:29:16.477
the open internet. and this was what was

00:29:16.560 --> 00:29:20.717
being made available. And so the company
ran to put a filter on it. But then that

00:29:20.800 --> 00:29:25.037
filter is like content moderation.
It went too far and then you couldn't ask

00:29:25.120 --> 00:29:27.357
it questions about something as simple

00:29:27.440 --> 00:29:29.997
as viol police violence against blacks.

00:29:30.080 --> 00:29:32.877
It would just not answer that question

00:29:32.960 --> 00:29:35.837
as if George Floyd never happened. And

00:29:35.920 --> 00:29:38.237
so these are the consequences that we as

00:29:38.320 --> 00:29:42.077
a society have to experience. the harms.

00:29:42.160 --> 00:29:45.837
Yeah, there are questions about
existential harms in the future, but

00:29:45.920 --> 00:29:47.597
there are a lot of harms happening right

00:29:47.680 --> 00:29:50.957
now. And it doesn't have to be that way.

00:29:51.040 --> 00:29:55.597
In order for it not to be that way, it
depends on who this AI is servicing. And

00:29:55.680 --> 00:29:57.677
and in particular, the design of the

00:29:57.760 --> 00:30:00.077
technology, the decisions made in that

00:30:00.160 --> 00:30:03.197
design, uh, is really determining what

00:30:03.280 --> 00:30:05.757
our values really will be. our ability

00:30:05.840 --> 00:30:08.877
to, you know, I'm driven by society has

00:30:08.960 --> 00:30:11.677
a set of laws, a set of rules that we've

00:30:11.760 --> 00:30:16.477
lived by. People died for these laws
to get passed and technology just ignores

00:30:16.560 --> 00:30:18.637
them and rewrites them. And that's true

00:30:18.720 --> 00:30:21.437
in social media. It's true in AI as

00:30:21.520 --> 00:30:23.837
well. That is, there's so many laws and

00:30:23.920 --> 00:30:29.277
constitutional protections. We can't
implement or protect ourselves online,

00:30:29.360 --> 00:30:33.437
though we can in brick-and-mortar
buildings. And that's the goal of public

00:30:33.520 --> 00:30:35.757
interest tech is to how to make that

00:30:35.840 --> 00:30:40.920
technology accountable. >> Yep. So Lata,

00:30:43.200 --> 00:30:45.197
let's imagine a future where the

00:30:45.280 --> 00:30:47.357
technology is accountable and we find

00:30:47.440 --> 00:30:50.157
the bias and we fared it out. It's it's

00:30:50.240 --> 00:30:54.797
it's a work in progress. As I'm sure
uh Eric would agree, it's a work in

00:30:54.880 --> 00:30:58.237
progress. And of course, at any given
moment, you can see something that's not

00:30:58.320 --> 00:31:00.717
quite there yet or that's biased or is

00:31:00.800 --> 00:31:03.677
pro- Hitler or whatever. But presumably,

00:31:03.760 --> 00:31:06.317
that's a fixable, solvable problem.

00:31:06.400 --> 00:31:08.237
>> But this is different. We already have

00:31:08.320 --> 00:31:10.717
laws. We already have a way that our

00:31:10.800 --> 00:31:13.517
democracy works. We already have issues

00:31:13.600 --> 00:31:17.277
of address issues of of bias, consumer

00:31:17.360 --> 00:31:21.837
protection, and so forth.
None of those are enforced online.

00:31:21.920 --> 00:31:24.237
And so the question is, can the AI at

00:31:24.320 --> 00:31:25.837
least not break the law?

00:31:25.920 --> 00:31:31.197
>> Eric, is is AI its own wild west?

00:31:31.280 --> 00:31:34.077
Is it is it lawless?

00:31:34.160 --> 00:31:37.277
>> So is it again the story you told is

00:31:37.360 --> 00:31:42.157
correct that they rushed the stuff out,
they found all sorts of problems, and

00:31:42.240 --> 00:31:43.757
then they're busy correcting them. I

00:31:43.840 --> 00:31:46.717
think that's the cycle. Um, and it's

00:31:46.800 --> 00:31:52.077
very hard. The technical thing that's
going on is models are getting trained

00:31:52.160 --> 00:31:55.997
on more and more data. They're
developing more and more facilities.

00:31:56.080 --> 00:31:58.957
They have so-called red cards and test

00:31:59.040 --> 00:32:03.517
teams and so forth, but you can't test
for everything. So, at some level when

00:32:03.600 --> 00:32:08.077
you release the model, it will do
something that will be a problem. And

00:32:08.160 --> 00:32:12.317
they correct it fairly quickly. And
the lifetime of these models is down to

00:32:12.400 --> 00:32:15.277
three or four months as they keep
improving improving because they're so

00:32:15.360 --> 00:32:19.677
competitive. So, But let's go back
to hallucinations. Remember that

00:32:19.760 --> 00:32:21.997
hallucinations >> before you
go to hallucinations, let me

00:32:22.080 --> 00:32:25.917
just say one thing about what you just
said about this idea. Well, let's run

00:32:26.000 --> 00:32:27.837
with the technology. And after all,

00:32:27.920 --> 00:32:30.477
they're trying to fix it. It takes us on

00:32:30.560 --> 00:32:34.957
the outside of the technology to tell
the people who are running the company

00:32:35.040 --> 00:32:38.717
who could run their own experiments
what's wrong and then they're supposed

00:32:38.800 --> 00:32:42.797
to fix it. >> So, Eric,
are these what she say?

00:32:42.880 --> 00:32:46.157
There's an actual pro there's an actual
problem that you can't pretest some

00:32:46.240 --> 00:32:49.437
things. But it but I disagree. I mean,

00:32:49.520 --> 00:32:54.317
people build technologies all
the time with warranties and compliance

00:32:54.400 --> 00:32:56.477
statements. >> Wait, wait,
wait. I got this. I got

00:32:56.560 --> 00:32:58.557
this. I'd actually I'd
actually like to answer.

00:32:58.640 --> 00:33:00.637
>> No, no, no, wait.
Cuz cuz it's I want to

00:33:00.720 --> 00:33:03.357
converge the answer rather than inflame

00:33:03.440 --> 00:33:05.837
it.

00:33:05.920 --> 00:33:08.717
Your your testers are testing for all

00:33:08.800 --> 00:33:13.437
that they can think of. Then it gets
released into the wild where there's

00:33:13.520 --> 00:33:18.317
vastly more testing going on than you
can possibly accomplish in in a room of

00:33:18.400 --> 00:33:22.877
testers or even if it's distributed to
other places and then because you tested

00:33:22.960 --> 00:33:24.637
on a thousand now a million people use

00:33:24.720 --> 00:33:27.117
it and there's some bias that they find.

00:33:27.200 --> 00:33:30.077
So are there mechanisms to get that

00:33:30.160 --> 00:33:34.557
information back to you and are you
responsive to that? Uh again I I can't

00:33:34.640 --> 00:33:37.357
speak for the specific companies but I
will tell you every single one of them

00:33:37.440 --> 00:33:41.917
is pillariating the press over every
one of these examples and these people do

00:33:42.000 --> 00:33:46.157
actually fix them. You need to give me
examples where the pattern has persisted

00:33:46.240 --> 00:33:47.837
even after there's been I'll use

00:33:47.920 --> 00:33:49.437
Facebook.

00:33:49.520 --> 00:33:53.517
Facebook was shown to because of the way

00:33:53.600 --> 00:33:55.997
it would this algorithm worked. It

00:33:56.080 --> 00:33:59.997
definitely violated the uh ability for

00:34:00.080 --> 00:34:02.877
to protect people in fair housing. That

00:34:02.960 --> 00:34:05.197
is if you belong to a particular race or

00:34:05.280 --> 00:34:08.477
group, you only saw ads that were at a

00:34:08.560 --> 00:34:13.197
disadvantage. That pattern has also been
shown on Google with respect to credit

00:34:13.280 --> 00:34:15.437
and so forth.
>> Isn't that just algorithm?

00:34:15.520 --> 00:34:18.557
>> But I'll stick to Facebook.
I'll stick to Facebook for a second.

00:34:18.640 --> 00:34:21.357
>> It became it became public information.

00:34:21.440 --> 00:34:24.477
Congress responded and Facebook says,

00:34:24.560 --> 00:34:26.557
"Hey, wait. Thank you for letting us

00:34:26.640 --> 00:34:30.477
know that we will fix it." About a few

00:34:30.560 --> 00:34:33.197
months later, their fix. Our our

00:34:33.280 --> 00:34:35.117
students were able to show that the fix

00:34:35.200 --> 00:34:40.317
was worse than the original.
And so it becomes increasingly and so in

00:34:40.400 --> 00:34:44.397
the companies make it even more
difficult then to to do these

00:34:44.480 --> 00:34:48.397
experiments. >> Isn't it
just the operator, not AI?

00:34:48.480 --> 00:34:51.357
>> And you're not actually responsive to
my point. I was talking about lang large

00:34:51.440 --> 00:34:53.277
language models.
You're talking about aphasite.

00:34:53.360 --> 00:34:54.957
>> Well, I have examples
for large language

00:34:55.040 --> 00:34:58.637
models too in the I mean they're not a

00:34:58.720 --> 00:35:02.877
gent you're talking about a trajectory
of technology that is not new. These

00:35:02.960 --> 00:35:05.357
technology society clashes have been

00:35:05.440 --> 00:35:09.997
following the same pattern and it's also
true in AI and the question is can't we

00:35:10.080 --> 00:35:12.157
learn from what we've already seen.

00:35:12.240 --> 00:35:13.837
>> I was trying to make an earlier point

00:35:13.920 --> 00:35:17.597
that which I was unable to make out. It

00:35:17.680 --> 00:35:21.357
is absolutely true that these companies
are governed by US law and if they're

00:35:21.440 --> 00:35:22.797
violating the law, they should be held

00:35:22.880 --> 00:35:25.277
accountable. We completely agree. Here's

00:35:25.360 --> 00:35:28.637
the problem. A new feature emerges in

00:35:28.720 --> 00:35:31.517
these systems that is not testable.

00:35:31.600 --> 00:35:35.117
Tested is we don't know how
to test it >> because it's emergent.

00:35:35.200 --> 00:35:37.677
>> It is emergent, right? We have been

00:35:37.760 --> 00:35:40.717
through that. Now, we can stop that and

00:35:40.800 --> 00:35:43.037
therefore stop all progress by law by

00:35:43.120 --> 00:35:48.077
banning larger models. But as long
as you have this new emerging power with

00:35:48.160 --> 00:35:51.837
deep reasoning, deep capabilities and
they will make mistakes, you have to be

00:35:51.920 --> 00:35:55.357
tolerant. I went through this when I
was at Google in earlier versions of the

00:35:55.440 --> 00:35:58.557
technology where the system would
actually do something that was wrong and

00:35:58.640 --> 00:36:02.317
we fixed it and we fixed it as fast
because we had to because it was the

00:36:02.400 --> 00:36:07.037
right thing to do. I don't agree that
there that these that these worlds are

00:36:07.120 --> 00:36:09.197
ungoverned. They're governed by US law.

00:36:09.280 --> 00:36:12.717
They're governed by shareholder
pressure. governed by uh consumer

00:36:12.800 --> 00:36:15.837
pressure and privacy laws
and so forth and European law.

00:36:15.920 --> 00:36:19.677
>> I know that you want to open this
up to others, but I just have to say

00:36:19.760 --> 00:36:22.397
>> when I was the chief
the CTO at the FTC,

00:36:22.480 --> 00:36:26.237
the chief technology officer at the FTC

00:36:26.320 --> 00:36:29.597
who's responsible for a lot of a lot a

00:36:29.680 --> 00:36:32.397
lot of fair information practices and so

00:36:32.480 --> 00:36:34.717
forth on the internet.

00:36:34.800 --> 00:36:36.637
>> So when did the internet become under

00:36:36.720 --> 00:36:42.237
the under the jurisdiction of the F Oh,
I'm thinking of the F. What am I

00:36:42.320 --> 00:36:45.197
thinking of? >> I don't know.
Who guides the airwaves?

00:36:45.280 --> 00:36:49.677
That's >> Oh, the FC FCC. >> FCC.
So, FTC is Trade Commission.

00:36:49.760 --> 00:36:52.317
Gotcha. Gotcha.
>> And the reason that they're kind of the

00:36:52.400 --> 00:36:54.397
police department of the internet

00:36:54.480 --> 00:36:57.677
because these are US companies.

00:36:57.760 --> 00:37:00.397
Um, and when I worked at the FTC, it was

00:37:00.480 --> 00:37:03.037
a lot like working in Spider-Man's den.

00:37:03.120 --> 00:37:07.357
It was so impressive and amazing what
these people, many of them who had been

00:37:07.440 --> 00:37:10.877
there lifelong, would do.
They were awesome. They were

00:37:10.960 --> 00:37:14.637
>> Say >> because they
were working for the

00:37:14.720 --> 00:37:19.357
underdog for American public with no
recognition by the American public for

00:37:19.440 --> 00:37:22.957
what they were doing >> and they
were doing an amazing job

00:37:23.040 --> 00:37:26.157
except the only thing about it
was everything they knew how to do was in

00:37:26.240 --> 00:37:28.397
brick andmortar buildings. They had no

00:37:28.480 --> 00:37:31.597
way of doing that online. That is true

00:37:31.680 --> 00:37:34.717
today. I was at the FTC in 2014. That

00:37:34.800 --> 00:37:37.117
was over 10 years ago and the answer is

00:37:37.200 --> 00:37:41.117
still the same. So their ability to make

00:37:41.200 --> 00:37:44.717
sure and test and regulate is totally

00:37:44.800 --> 00:37:49.117
non-existent online. >> I don't

00:37:49.200 --> 00:37:50.957
>> I want to pivot a little bit. Kate, I

00:37:51.040 --> 00:37:54.637
you you listed these uh travails, this

00:37:54.720 --> 00:37:58.077
backend travails of AI and its cost to

00:37:58.160 --> 00:38:00.957
society. Are you in a position to say if

00:38:01.040 --> 00:38:03.677
we find energy solutions for and if we

00:38:03.760 --> 00:38:05.837
find where we can get on the lithium we

00:38:05.920 --> 00:38:08.317
want find some new lithium ore that's

00:38:08.400 --> 00:38:13.037
plenty enough for all of the energy
needs or however lithium is used in our

00:38:13.120 --> 00:38:17.597
circuitry would you just
be fine with AI are these

00:38:17.680 --> 00:38:21.117
your because these seem
like solvable problems

00:38:21.200 --> 00:38:25.437
>> I think that's a really great question
in my research over the you know over

00:38:25.520 --> 00:38:30.237
many years it's really clear to me that
these questions are inter the societal

00:38:30.320 --> 00:38:33.997
practices, the political practices and
the environmental practices. You'll see

00:38:34.080 --> 00:38:36.237
a golden thread running through them.

00:38:36.320 --> 00:38:39.837
One is the question of transparency
which Latana just spoke to I think very

00:38:39.920 --> 00:38:43.837
powerfully. The second is the question
of the race. The reason that we're

00:38:43.920 --> 00:38:47.597
seeing these models be rolled out
so quickly, as Eric rightly said, is

00:38:47.680 --> 00:38:51.517
because there's an enormous corporate
race going on to be the first. And and

00:38:51.600 --> 00:38:55.037
that's where you see these sorts of
problems emerge, which is that we're not

00:38:55.120 --> 00:38:58.317
doing like what would happen in drug
discovery where you really have to test

00:38:58.400 --> 00:39:01.757
on a diverse population before you
release, you know, a drug into the into

00:39:01.840 --> 00:39:04.637
the >> the patient can die. Exactly. But

00:39:04.720 --> 00:39:07.197
frankly, AI wasn't seen as being that

00:39:07.280 --> 00:39:09.437
mission critical even as recently as 10

00:39:09.520 --> 00:39:12.797
years ago. Now people can die. We We are

00:39:12.880 --> 00:39:16.957
seeing AI systems being fed directly
into the kill chain. These are systems

00:39:17.040 --> 00:39:19.757
that are now being used in live wars as

00:39:19.840 --> 00:39:22.477
we speak. So there is a very different

00:39:22.560 --> 00:39:26.877
moral and legal pressure to make sure
that these systems are working. Now your

00:39:26.960 --> 00:39:30.877
question is like okay what if they were
completely run on renewable energy and

00:39:30.960 --> 00:39:35.517
if they you know if we made sure
that you know this this wasn't currently

00:39:35.600 --> 00:39:39.517
right now we see that AI systems are
on track to overtake the airline industry

00:39:39.600 --> 00:39:41.037
in terms of their carbon emissions.

00:39:41.120 --> 00:39:44.077
Right. That's horrifying. That would
you'd think would be the first problem

00:39:44.160 --> 00:39:48.237
that we'd want to be fixing. That is not
what's happening. At the moment, we see

00:39:48.320 --> 00:39:50.797
that AI is using around 4% of the

00:39:50.880 --> 00:39:55.597
world's energy. It's on track
to be using something like 25%.

00:39:55.680 --> 00:39:57.277
By the end of this decade, we are

00:39:57.360 --> 00:40:00.877
looking at an enormous budget of carbon

00:40:00.960 --> 00:40:04.557
that is actually like this is a I mean,
I'm a you know, I'm a scientist. I

00:40:04.640 --> 00:40:07.677
believe that climate change is real.
I don't think we're taking it seriously

00:40:07.760 --> 00:40:10.877
enough. But yes, if we get to a point
where we have a diamond sphere, let's

00:40:10.960 --> 00:40:13.757
go. Um, but we're not there yet. So, I I

00:40:13.840 --> 00:40:15.917
I think the golden thread is how do we

00:40:16.000 --> 00:40:17.997
think about what we build before we

00:40:18.080 --> 00:40:21.117
build it? How do we instill those values

00:40:21.200 --> 00:40:26.397
early on and how do we think about the
implications near and long term because

00:40:26.480 --> 00:40:30.477
it's it's going to be the generations
after us who live the legacies of what

00:40:30.560 --> 00:40:32.957
we built today.
>> I'm compelled to catch everybody up if

00:40:33.040 --> 00:40:35.917
you didn't know. Kate mentioned a Dyson

00:40:36.000 --> 00:40:37.597
sphere.

00:40:37.680 --> 00:40:42.237
A Dyson sphere is an imagined

00:40:42.320 --> 00:40:45.997
means of grabbing all the energy emitted

00:40:46.080 --> 00:40:50.317
by your host star and using it in the
surface of your own civilization. Not

00:40:50.400 --> 00:40:54.317
just a little bit of sunlight that
happens to intersect your planet, but

00:40:54.400 --> 00:40:58.717
you build a sphere around the star,
gather all of that energy, then you can

00:40:58.800 --> 00:41:02.397
do what the hell you want, >> right?
and and and my hope is that

00:41:02.480 --> 00:41:06.957
generative AI and its current growth
might might actually need a Dyson sphere

00:41:07.040 --> 00:41:08.957
>> at the projected growth rate.

00:41:09.040 --> 00:41:11.437
>> Exactly. >> Um I want
to get back to this killchain

00:41:11.520 --> 00:41:13.837
question with Eric in a minute because

00:41:13.920 --> 00:41:15.757
uh Eric and I served on a board of the

00:41:15.840 --> 00:41:19.197
Pentagon where we introduced I

00:41:19.280 --> 00:41:20.877
legislation is not the right word. We

00:41:20.960 --> 00:41:25.357
introduced a code of ethics for how AI

00:41:25.440 --> 00:41:29.117
would be used in warfare uh on our side

00:41:29.200 --> 00:41:33.517
of the United States. But before I get
there, Chris, uh, lately there's been a

00:41:33.600 --> 00:41:35.597
lot of talk about deep fakes. I've been

00:41:35.680 --> 00:41:39.357
deep faked many times. And, uh, you

00:41:39.440 --> 00:41:42.877
know, some are harmless, others.
I'm not a politician, right? And I'm not

00:41:42.960 --> 00:41:47.917
declaring war on another country. So,
my deep faiths though I don't like them,

00:41:48.000 --> 00:41:49.357
they don't have the same consequences

00:41:49.440 --> 00:41:52.077
that deep fakes or politicians might.

00:41:52.160 --> 00:41:55.597
So, if deep fakes are out there and AI

00:41:55.680 --> 00:41:58.237
is training on the internet, then AI

00:41:58.320 --> 00:42:02.077
could be training on its own. face

00:42:02.160 --> 00:42:05.277
and there's also AI slop I guess we call

00:42:05.360 --> 00:42:08.557
it AI attempting to be artistic and if

00:42:08.640 --> 00:42:11.517
that fills the space of images that are

00:42:11.600 --> 00:42:14.157
out there and AI is training on those

00:42:14.240 --> 00:42:18.237
images training on its own sloth so what

00:42:18.320 --> 00:42:22.557
do you have to say about that I'm
not blaming you for it you're a

00:42:22.640 --> 00:42:26.557
professor and you teach this stuff I just
want to know where how does that show up

00:42:26.640 --> 00:42:29.117
>> yeah so there's a really interesting

00:42:29.200 --> 00:42:32.717
phenomena for uh AI which is it does

00:42:32.800 --> 00:42:35.437
need a tremendous amount of data. So it

00:42:35.520 --> 00:42:37.997
learns really interesting things from

00:42:38.080 --> 00:42:40.797
the internet. So what does it learn?

00:42:40.880 --> 00:42:43.117
Well, it learns how to use language. It

00:42:43.200 --> 00:42:45.917
learns patterns of word association. It

00:42:46.000 --> 00:42:48.717
learns facts about the world and it

00:42:48.800 --> 00:42:53.117
learns uh even things like some
rudimentary common sense reasoning.

00:42:53.200 --> 00:42:54.717
Things about physics which you wouldn't

00:42:54.800 --> 00:42:57.037
expect it to be able to learn from text.

00:42:57.120 --> 00:43:00.157
So all of these are uh elements that go

00:43:00.240 --> 00:43:02.237
into a model that then later can be

00:43:02.320 --> 00:43:05.277
adapted for useful purposes. So given

00:43:05.360 --> 00:43:07.277
this general background knowledge that

00:43:07.360 --> 00:43:10.717
it's gathered from large amounts of text

00:43:10.800 --> 00:43:15.437
uh it's then subsequently applied
in a post-training phase where

00:43:15.520 --> 00:43:17.597
you teach it to do useful things. You

00:43:17.680 --> 00:43:20.477
teach it to be a helpful assistant. You

00:43:20.560 --> 00:43:22.477
teach it to be a harmless assistant. you

00:43:22.560 --> 00:43:25.597
teach it not to disclose information to

00:43:25.680 --> 00:43:30.397
someone seeking how to manufacture
drugs or weapons. Uh or you govern its

00:43:30.480 --> 00:43:34.317
behavior. You say, "Rather than
a general assistant, I want you to be an

00:43:34.400 --> 00:43:36.797
encouraging tutor." So all of those

00:43:36.880 --> 00:43:39.357
things are elements of the training

00:43:39.440 --> 00:43:42.637
pipeline that end up with something that

00:43:42.720 --> 00:43:44.957
actually can have value. So I think

00:43:45.040 --> 00:43:47.357
let's not lose sight of the fact that

00:43:47.440 --> 00:43:50.157
although this could be technology that

00:43:50.240 --> 00:43:52.077
goes wrong, although it could be

00:43:52.160 --> 00:43:55.677
technology that encodes bias, like we as

00:43:55.760 --> 00:44:00.877
researchers, we as model developers,
we as users like we have a role in

00:44:00.960 --> 00:44:02.477
determining which direction this

00:44:02.560 --> 00:44:05.277
technology will go and I believe that it

00:44:05.360 --> 00:44:08.077
has a tremendous amount of good to give

00:44:08.160 --> 00:44:09.917
to the world in addition to the

00:44:10.000 --> 00:44:13.597
potential pitfalls. Nate, how do you uh

00:44:13.680 --> 00:44:16.397
put up guardrails for this?

00:44:16.480 --> 00:44:17.917
Because we just we just heard a

00:44:18.000 --> 00:44:21.117
beautifully described uh future of AI

00:44:21.200 --> 00:44:23.837
where but uh you end up training the AI.

00:44:23.920 --> 00:44:27.517
That's I don't want to spend my time
training a freaking robot. I'd rather

00:44:27.600 --> 00:44:29.517
train other humans. That's just my

00:44:29.600 --> 00:44:33.037
personal bias against that activity. But

00:44:33.120 --> 00:44:37.117
that's effort you're investing
in something that you're then going to

00:44:37.200 --> 00:44:39.357
interact with. I suppose that's okay.

00:44:39.440 --> 00:44:43.117
What I'm asking is if if we don't want

00:44:43.200 --> 00:44:47.517
the world to end, how do
we think about guardrails?

00:44:47.600 --> 00:44:51.197
>> You know, there's a lot of dreams of AI
doing exactly what we want. There's a

00:44:51.280 --> 00:44:53.917
lot of dreams of getting it to follow
the law. There's a lot of dreams of

00:44:54.000 --> 00:44:57.037
getting it to act nicely. Um, but you

00:44:57.120 --> 00:45:01.197
know, one thing I agree with Eric on,
the behavior is emergent. One thing I

00:45:01.280 --> 00:45:04.397
agree with Eric on,
you can't pretest this stuff.

00:45:04.480 --> 00:45:06.157
>> You can't pretest every aspect of it.

00:45:06.240 --> 00:45:09.197
You can pretest a lot of it.
You can protest some of it, but but we're

00:45:09.280 --> 00:45:13.997
growing these things a bit like
an organism. We cannot, you know, encode

00:45:14.080 --> 00:45:18.717
three laws of robotics deep into these
things minds. We are not writing these

00:45:18.800 --> 00:45:22.877
things like code.
And we can grow them. We can try, you

00:45:22.960 --> 00:45:25.757
know, Eric mentioned, oh, there's
these people, they have these evaluation

00:45:25.840 --> 00:45:28.157
cards, they have these safety teams.

00:45:28.240 --> 00:45:29.917
What's going on in these safety teams?

00:45:30.000 --> 00:45:33.917
The the two biggest places people are
putting their safety efforts are what's

00:45:34.000 --> 00:45:37.357
called interpretability research, which
is trying to figure out what's going on

00:45:37.440 --> 00:45:42.397
inside the AI's heads, and these
evaluations, these these model cards,

00:45:42.480 --> 00:45:46.637
uh, which is trying to figure out how
dangerous the AIS are. If someone was

00:45:46.720 --> 00:45:51.197
making a nuclear power plant in your
hometown and you went to them and you

00:45:51.280 --> 00:45:54.637
said, "Hey, I hear that this uranium
stuff can have lots of energy benefits,

00:45:54.720 --> 00:45:59.117
but also can melt down when things
go badly." What have you guys got that

00:45:59.200 --> 00:46:02.877
makes you think you're going to get
the the benefits and not the the pitfalls?

00:46:02.960 --> 00:46:05.117
You know, I've heard all of these
wonderful stories about the benefits.

00:46:05.200 --> 00:46:08.237
What makes you think we're going to get
the benefits, not the pitfalls? If the

00:46:08.320 --> 00:46:12.877
engineers say, "Oh, yeah, we've
got two crack teams working on this.

00:46:12.960 --> 00:46:17.197
The first team's trying to figure
out what the heck is going on inside,

00:46:17.280 --> 00:46:18.717
and the second team's trying to measure

00:46:18.800 --> 00:46:22.397
whether it's currently exploding.

00:46:22.480 --> 00:46:24.957
That's not a good sign.

00:46:25.040 --> 00:46:29.997
You know, there are ways to get a handle
on this. People talk about the benefits.

00:46:30.080 --> 00:46:31.437
People talk about the dangers and they

00:46:31.520 --> 00:46:34.077
talk about na navigating the course. We

00:46:34.160 --> 00:46:37.037
are not on track to get the benefits if

00:46:37.120 --> 00:46:41.117
we make these things really smart. We
can't pretest it. We're going to roll it

00:46:41.200 --> 00:46:44.477
out, find the problems, and then figure
them out after the fact. That's fine

00:46:44.560 --> 00:46:48.637
when they are still a lot dumber than
the humans. That's not fine if you keep

00:46:48.720 --> 00:46:52.477
making them smarter. And that's what
these companies are trying to do. Cindy.

00:46:52.560 --> 00:46:55.597
>> Yeah, I'll maybe just give like
a slightly different take on this. You

00:46:55.680 --> 00:46:59.597
said like, you know, we're trying to um
study interpretability to figure out

00:46:59.680 --> 00:47:03.517
what's in their heads, you know, and I
think like, you know, when I think about

00:47:03.600 --> 00:47:07.837
interpretability for me, we already know
kind of exactly what's happening with

00:47:07.920 --> 00:47:09.517
these machines, right? There is a

00:47:09.600 --> 00:47:14.877
mathematical equation we can write down
that's there, right? The challenge is

00:47:14.960 --> 00:47:16.397
even though we can characterize it

00:47:16.480 --> 00:47:19.757
mathematically, right? um as humans it's

00:47:19.840 --> 00:47:22.557
hard to kind of understand or interpret

00:47:22.640 --> 00:47:25.837
its reasoning but you know like in a way

00:47:25.920 --> 00:47:29.757
that makes sense to us right but this
is just because the you know the the

00:47:29.840 --> 00:47:31.437
underlying structure of these things is

00:47:31.520 --> 00:47:34.157
very complex right it's multi-layered

00:47:34.240 --> 00:47:37.997
nonlinear it doesn't map onto to human

00:47:38.080 --> 00:47:41.597
concepts right and so I
think the interpretability

00:47:41.680 --> 00:47:43.837
question is more just can we take like a

00:47:43.920 --> 00:47:46.077
a mathematical equation that would take

00:47:46.160 --> 00:47:49.837
you know a stack of paper a mile high to

00:47:49.920 --> 00:47:52.317
um to write out. Can we explain that in

00:47:52.400 --> 00:47:54.317
a real sense? But I don't think there is

00:47:54.400 --> 00:47:56.957
any, you know, there is no mind there,

00:47:57.040 --> 00:47:59.517
right? It's just kind of mapping these

00:47:59.600 --> 00:48:01.917
huge like internal structures that are

00:48:02.000 --> 00:48:06.397
very abstract into kind of human scale

00:48:06.480 --> 00:48:10.077
um conceptual understandings
of like the overall reasoning.

00:48:10.160 --> 00:48:13.997
>> Cindy, how might you
program agency into an AI?

00:48:14.080 --> 00:48:16.717
>> Yeah, I mean >> like like
the Terminator for example.

00:48:16.800 --> 00:48:20.397
Okay. So like yeah so >>
just to pick an example.

00:48:20.480 --> 00:48:23.517
>> Okay. So when Neil >> random example

00:48:23.600 --> 00:48:26.157
>> when Neil says agency um you know what

00:48:26.240 --> 00:48:29.197
what Raig means right is he wants to see

00:48:29.280 --> 00:48:32.557
um the presence of some sort of like

00:48:32.640 --> 00:48:35.277
persistent like self-motivated

00:48:35.360 --> 00:48:39.517
um goaloriented behavior like uh

00:48:39.600 --> 00:48:41.597
organized around the idea right that

00:48:41.680 --> 00:48:44.717
this this AI is actually an agent in its

00:48:44.800 --> 00:48:49.277
environment. So, not just responding
to your prompts, but also like pursuing

00:48:49.360 --> 00:48:53.037
goals long term. And like I know that we
have a little bit of a disagreement

00:48:53.120 --> 00:48:57.677
because I think you already believe that
there is like goal oriented behavior in

00:48:57.760 --> 00:48:59.677
these machines which I don't see any

00:48:59.760 --> 00:49:03.197
presence of um currently. Right. So,

00:49:03.280 --> 00:49:07.917
>> unless we program in a goal and it
sets for itself programming, we're growing

00:49:08.000 --> 00:49:11.677
these things. >> We're grow Okay.
So, yes, but like growing

00:49:11.760 --> 00:49:15.197
>> scary when you say we're growing them.
I mean, yeah, it's not a biological thing

00:49:15.280 --> 00:49:18.557
either, right? I mean, um, >> it's
not like old school programs either.

00:49:18.640 --> 00:49:20.237
>> I mean, I but like I don't think it's

00:49:20.320 --> 00:49:22.797
going to get agency by just, you know,

00:49:22.880 --> 00:49:26.237
training on larger data sets, right? I

00:49:26.320 --> 00:49:29.997
mean to get agency there has
to be some real engineering

00:49:30.080 --> 00:49:34.717
things that are are happening you know
like it's it's not like a bigger neural

00:49:34.800 --> 00:49:36.637
network or something is ever going to be

00:49:36.720 --> 00:49:39.517
able to like pursue pursue goals

00:49:39.600 --> 00:49:42.557
remember things like take actions like

00:49:42.640 --> 00:49:45.037
you know I think that to to do that you

00:49:45.120 --> 00:49:47.437
have to have like kind of like outside

00:49:47.520 --> 00:49:50.477
scaffolding um that are real I mean that

00:49:50.560 --> 00:49:52.557
are are different things right it's it's

00:49:52.640 --> 00:49:55.037
not the same underlying but can you

00:49:55.120 --> 00:49:59.677
imagine an AI with agency that satisfies
all of your concerns. That would be like

00:49:59.760 --> 00:50:01.837
a good AI, right?

00:50:01.920 --> 00:50:04.717
>> Well, I would first
of all before we can

00:50:04.800 --> 00:50:07.917
even get to the issue of agency, the the

00:50:08.000 --> 00:50:10.317
issue of these emergent behaviors and

00:50:10.400 --> 00:50:13.597
even if it is an AI with agency, this is

00:50:13.680 --> 00:50:16.237
wrapped in a business model. And the

00:50:16.320 --> 00:50:19.197
goal of the product is to make money for

00:50:19.280 --> 00:50:22.317
the company, you know, and I'll go back

00:50:22.400 --> 00:50:25.517
to social media as an example of that.

00:50:25.600 --> 00:50:30.157
you know, we're not the customer
of Facebook. We're their product. Our eyes

00:50:30.240 --> 00:50:32.397
on a page is what makes the money and

00:50:32.480 --> 00:50:34.557
the advertisers are their are their

00:50:34.640 --> 00:50:37.597
customer. And so, you can't just talk

00:50:37.680 --> 00:50:42.077
about, oh, I'm building this AI and it's
emerging behavior and it may even get

00:50:42.160 --> 00:50:45.997
agency. The question is, how is the
business selling it? What are they

00:50:46.080 --> 00:50:48.077
making it available for? What are they

00:50:48.160 --> 00:50:50.317
saying it's to be used for? If it's to

00:50:50.400 --> 00:50:53.517
be used to help me hire people in my

00:50:53.600 --> 00:50:55.997
company, then it needs to agree. It

00:50:56.080 --> 00:50:58.477
needs to show it's not going to violate

00:50:58.560 --> 00:51:01.437
the Equal Employment Act. If it's being

00:51:01.520 --> 00:51:05.197
hired for other purposes like for credit

00:51:05.280 --> 00:51:09.837
cards, it needs to show it's not going
to violate the Fair Credit Reporting Act

00:51:09.920 --> 00:51:11.997
and so forth. And that's the level of

00:51:12.080 --> 00:51:16.957
responsibility that only the designer
of those companies can do because they're

00:51:17.040 --> 00:51:19.917
the ones making the >> You make
an important point about the

00:51:20.000 --> 00:51:23.357
business motives. Now, if I remember
correctly, and this could just be I'm

00:51:23.440 --> 00:51:25.597
imagining this, Eric, Google was so

00:51:25.680 --> 00:51:28.077
large and had so much cash flow. You had

00:51:28.160 --> 00:51:31.117
the luxury to just explore without any

00:51:31.200 --> 00:51:34.077
particular obligation to the bottom line

00:51:34.160 --> 00:51:38.157
of the quarterly report like alpha go.

00:51:38.240 --> 00:51:41.997
All right, that you're going to win. You
can be the best Go player in the world.

00:51:42.080 --> 00:51:45.997
And what's the one for chess?
That was uh >> Stockfish.

00:51:46.080 --> 00:51:48.237
>> No, but the Google >> Alpha Zero.

00:51:48.320 --> 00:51:50.797
>> Alpha Zero. Uh to to be the best chess

00:51:50.880 --> 00:51:52.957
player. Did that have real financial

00:51:53.040 --> 00:51:57.917
consequences or was it it just bragging
rights and that was just a fun thing for

00:51:58.000 --> 00:51:59.757
your company to do? Well, it was

00:51:59.840 --> 00:52:02.957
research and once we achieved those

00:52:03.040 --> 00:52:05.997
objectives, we actually then used that

00:52:06.080 --> 00:52:10.717
to solve a problem called protein
folding which is the single largest

00:52:10.800 --> 00:52:13.437
discovery in biology I
think in the last 50 years.

00:52:13.520 --> 00:52:17.037
>> Alpha fold. >> Yes.
And alpha fold will result in an

00:52:17.120 --> 00:52:19.757
enormous number of new drugs, new

00:52:19.840 --> 00:52:24.717
treatments, new research in biomed so
forth and so on. U research has these

00:52:24.800 --> 00:52:28.237
sort of strange things. You try one
thing, you make a discovery, you link

00:52:28.320 --> 00:52:31.277
them together. Al alpha fold is a very

00:52:31.360 --> 00:52:35.837
good example of the unintended
effects of people trying to play games.

00:52:35.920 --> 00:52:37.597
>> Your your hyperbole there underscells

00:52:37.680 --> 00:52:41.277
the fact that Alpha Fold was given the

00:52:41.360 --> 00:52:44.717
Nobel Prize in physiology for that

00:52:44.800 --> 00:52:49.160
discovery. So,

00:52:49.280 --> 00:52:53.677
so, so what you're saying is that
research was these were steps towards

00:52:53.760 --> 00:52:57.197
this greater goal
that in fact was achieved.

00:52:57.280 --> 00:52:59.517
>> Well, I think that this narrative is is

00:52:59.600 --> 00:53:02.077
ignoring the enormous reasons why people

00:53:02.160 --> 00:53:05.837
are pursuing AI. So, uh let's start with

00:53:05.920 --> 00:53:10.397
healthcare, let's start with medicine,
let's start with drug discovery. How

00:53:10.480 --> 00:53:12.637
about energy solutions? We're going to

00:53:12.720 --> 00:53:15.117
be cooked as a as a world unless we come

00:53:15.200 --> 00:53:17.837
up with better energy sources. Um, one

00:53:17.920 --> 00:53:22.157
interesting thing I'm working on data
center architectures and it now looks

00:53:22.240 --> 00:53:24.637
like if you have enough land, solar and

00:53:24.720 --> 00:53:28.157
battery backups are more cost effective

00:53:28.240 --> 00:53:32.797
on a present value basis than natural
gas combined combined cycle things even

00:53:32.880 --> 00:53:36.717
in the United States. So it shows
you that innovation happens and it can

00:53:36.800 --> 00:53:39.677
happen in the right way as opposed
to just the wrong way which everybody else

00:53:39.760 --> 00:53:43.197
is talking about. Uh there's certainly
secondary effects and there's no

00:53:43.280 --> 00:53:45.597
question. But let me say once again,

00:53:45.680 --> 00:53:50.797
this stuff is done by under US law.
If people are paying awful lot of

00:53:50.880 --> 00:53:54.797
attention, if there's laws
that are missing, let's pass them.

00:53:54.880 --> 00:53:58.717
>> Why not why not make >> Can I can I
>> Hey, why not make AI solve all your

00:53:58.800 --> 00:54:00.877
problems? >> I'd like
to interject one thing as well.

00:54:00.960 --> 00:54:04.957
I think you know, we also talk we talk
about like the profit motive that drives

00:54:05.040 --> 00:54:09.437
uh the creation of AI, but that's not
what drives the researchers, right?

00:54:09.520 --> 00:54:11.517
There are venues.
>> I think that's a correct fact.

00:54:11.600 --> 00:54:14.957
>> Thoughtful science. >> It's just
curiosity on the frontier, right?

00:54:15.040 --> 00:54:16.717
>> They have the luxury
that someone's paying them,

00:54:16.800 --> 00:54:19.197
>> right? And I think Alphafold is a great

00:54:19.280 --> 00:54:22.237
example of this, right? Like that is AI.

00:54:22.320 --> 00:54:26.557
That is a scientific advance that's
incredibly important to human health.

00:54:26.640 --> 00:54:28.237
And I think

00:54:28.320 --> 00:54:30.797
>> conflating the fact that companies have

00:54:30.880 --> 00:54:35.357
profit with uh the motive for why this

00:54:35.440 --> 00:54:37.837
is being invented is incorrect.

00:54:37.920 --> 00:54:40.317
Okay. Can can AI solve our AI problems?

00:54:40.400 --> 00:54:43.757
>> I mean I I just I love this debate
because I think you know Chris touched

00:54:43.840 --> 00:54:47.837
on something really important. Um we're
talking about you know where is this

00:54:47.920 --> 00:54:49.757
kind of noble you know following the

00:54:49.840 --> 00:54:52.637
path of science. We just saw meta

00:54:52.720 --> 00:54:55.437
science checks for researchers for $100

00:54:55.520 --> 00:54:58.717
million. That's the salary. So let's

00:54:58.800 --> 00:55:03.357
just be clear. We are getting basketball
salaries now well beyond that. In fact

00:55:03.440 --> 00:55:06.797
they're breaking records. These these
are not necessarily people who are doing

00:55:06.880 --> 00:55:10.477
it out of the good of their heart.
Like this is an extremely

00:55:10.560 --> 00:55:14.397
money field and and I think it
was originally researchers who were really

00:55:14.480 --> 00:55:18.317
doing this just out of curiosity 10 20
years ago. We're in a different world

00:55:18.400 --> 00:55:19.837
now. And and I guess what I'd love to

00:55:19.920 --> 00:55:22.317
talk about is that world. I' I'd like us

00:55:22.400 --> 00:55:28.397
not to have the AI optimism or AI
pessimism discussion, but the AI realism

00:55:28.480 --> 00:55:30.877
discussion. >> All right.
So in that world, we're we're

00:55:30.960 --> 00:55:35.597
going to lose a lot of cars.
>> I mean, that world people losing their

00:55:35.680 --> 00:55:38.557
jobs now. >> Exactly.
Right. And and we're talking

00:55:38.640 --> 00:55:43.117
about the the most significant potential
loss of jobs. And this is we're hearing

00:55:43.200 --> 00:55:46.397
about this from the people building
these systems. You can look through all

00:55:46.480 --> 00:55:49.917
of the new stories. People who are
running frontier AI companies are saying

00:55:50.000 --> 00:55:51.677
that we could be looking at the loss of

00:55:51.760 --> 00:55:54.957
75% of new jobs. That's for new

00:55:55.040 --> 00:55:57.037
graduates. That that is a horrifying

00:55:57.120 --> 00:55:59.517
number. And I think Latana's point is so

00:55:59.600 --> 00:56:04.317
important here is that we do not have
the governance infrastructure or the

00:56:04.400 --> 00:56:06.957
planning to deal with what
could happen to the labor market.

00:56:07.040 --> 00:56:10.877
>> So we need to be more nimble in our
legislation and laws to keep pace with

00:56:10.960 --> 00:56:12.637
this. >> Well, we also
have to be realistic. Who

00:56:12.720 --> 00:56:14.557
is making the laws right now? I mean,

00:56:14.640 --> 00:56:17.357
you know, Eric is talking about like we

00:56:17.440 --> 00:56:19.837
have

00:56:19.920 --> 00:56:24.877
let's be clear. I mean, we have never
had an administration make less interest

00:56:24.960 --> 00:56:26.637
in regulating artificial intelligence.

00:56:26.720 --> 00:56:30.557
We've seen we've seen regulations
produced by previous administrations

00:56:30.640 --> 00:56:32.717
completely zeroed out and we've seen the

00:56:32.800 --> 00:56:36.237
threat of preventing every US state from

00:56:36.320 --> 00:56:40.077
making its own AI regulation.
>> And can I just add on to what Kate's

00:56:40.160 --> 00:56:44.477
saying? Many times I'll say that we live
in a technocracy that our democracy has

00:56:44.560 --> 00:56:48.237
already been changed.
>> The technology are in charge.

00:56:48.320 --> 00:56:50.397
>> The technology designers are the new

00:56:50.480 --> 00:56:55.277
policy makers by the arbitrary decisions
they make in the products that they

00:56:55.360 --> 00:56:58.797
produce. And we have no way of producing

00:56:58.880 --> 00:57:03.597
oversight. By the time I'm a computer
scientist, my whole life has been about

00:57:03.680 --> 00:57:07.517
how to help society get the benefits
of these technologies, more and more

00:57:07.600 --> 00:57:10.397
apholds, but without these crazy harms

00:57:10.480 --> 00:57:13.037
and undoing our society. And it doesn't

00:57:13.120 --> 00:57:15.837
have to be this way. What's forcing it

00:57:15.920 --> 00:57:19.437
this way is that the technocracies, the

00:57:19.520 --> 00:57:21.837
technocrats who are in charge are

00:57:21.920 --> 00:57:27.080
ignoring that responsibility.
>> Okay. Just just

00:57:28.800 --> 00:57:33.357
There is something called the lesser
of evils because before the tech bros were

00:57:33.440 --> 00:57:38.157
had the influence that they
currently did, you had Congress

00:57:38.240 --> 00:57:40.157
deciding

00:57:40.240 --> 00:57:44.397
on technology. That was disastrous cuz

00:57:44.480 --> 00:57:48.637
they didn't know what button do I push,
you know, how do I and they're making

00:57:48.720 --> 00:57:51.277
laws and legislations to constrain that.

00:57:51.360 --> 00:57:54.797
So So I I I don't have a solution. I'm

00:57:54.880 --> 00:57:57.437
just saying if tech bros are influencing

00:57:57.520 --> 00:58:01.437
the laws, how much worse is that than

00:58:01.520 --> 00:58:03.437
tech ignorant people who are otherwise

00:58:03.520 --> 00:58:06.157
running Congress making the laws? So

00:58:06.240 --> 00:58:09.917
again the right now us as researchers

00:58:10.000 --> 00:58:12.237
who are trying to provide experiments to

00:58:12.320 --> 00:58:14.557
these companies

00:58:14.640 --> 00:58:17.597
government agencies who have the legal

00:58:17.680 --> 00:58:20.237
right to pro they have the legal right

00:58:20.320 --> 00:58:22.957
to oversee these companies can't really

00:58:23.040 --> 00:58:25.517
do this job the person sitting in the

00:58:25.600 --> 00:58:28.397
seat to do the job is the company itself

00:58:28.480 --> 00:58:33.437
because they have access to the technology
they can run the same exper I

00:58:33.520 --> 00:58:37.277
have undergraduates a team
of undergraduates who don't even get paid

00:58:37.360 --> 00:58:41.597
and we've got to show Google and other
places what's wrong with their

00:58:41.680 --> 00:58:45.677
technology. It's not like they're paying
us to do that. You're telling me they

00:58:45.760 --> 00:58:48.397
can't run these same experiments?

00:58:48.480 --> 00:58:51.037
>> All right. Uh Dave, do you have any

00:58:51.120 --> 00:58:53.917
sense of what happens in this future

00:58:54.000 --> 00:58:58.797
that Kate sees where uh

00:58:58.880 --> 00:59:02.317
>> jobs are replaced and now you're making
a product but no one has money to buy it

00:59:02.400 --> 00:59:06.957
because nobody has a job because
you replace their job with AI?

00:59:07.040 --> 00:59:08.877
Does that make sense? >> Yes.

00:59:08.960 --> 00:59:14.520
>> Okay. >> You know,
uh I expect 100% employment.

00:59:16.160 --> 00:59:20.317
Uh but I do also expect it
to take the jobs and um you'll

00:59:20.400 --> 00:59:23.037
>> wait just to be clear

00:59:23.120 --> 00:59:28.557
other previous advances in technology
displaced workers. No doubt about it.

00:59:28.640 --> 00:59:32.477
But there was expansion in other
sectors. You know uh buggy whip

00:59:32.560 --> 00:59:34.877
manufacturers went out of business but

00:59:34.960 --> 00:59:37.997
auto repair centers came in at just

00:59:38.080 --> 00:59:40.797
after the turn of the century. here this

00:59:40.880 --> 00:59:44.717
pace seems faster than what society's

00:59:44.800 --> 00:59:48.717
capacity to respond might be and so

00:59:48.800 --> 00:59:51.757
should we be worrying about an entire

00:59:51.840 --> 00:59:56.157
class of unemployed people who cannot
buy the services provided by the AI that

00:59:56.240 --> 01:00:01.277
displace them >> you know I
think uh probably some people

01:00:01.360 --> 01:00:04.957
are going to are going to want to worry
about that probably society cannot

01:00:05.040 --> 01:00:08.157
>> especially the unemployed people yes
>> especially the unemployed people so you

01:00:08.240 --> 01:00:12.157
know economists will tell you about how
if you manage your economy correctly and

01:00:12.240 --> 01:00:15.597
if things are going, you know, slow
enough that people can respond and and

01:00:15.680 --> 01:00:17.997
find new jobs, society can absorb a lot

01:00:18.080 --> 01:00:19.837
of job loss. You know, the world used to

01:00:19.920 --> 01:00:23.837
be something like 95% farmers and now,

01:00:23.920 --> 01:00:27.437
you know, we can get the world running
on 2% of the population being farmers

01:00:27.520 --> 01:00:32.557
and that does not mean 93% of people are
unemployed. AI is different. Will we or

01:00:32.640 --> 01:00:34.877
won't we? >> That took a century,
by the way, but go on.

01:00:34.960 --> 01:00:37.437
>> It took a while. Yeah. But

01:00:37.520 --> 01:00:41.117
>> will will humanity adopt to this one?

01:00:41.200 --> 01:00:45.917
Frankly, I'm like, man,
I'm sort of looking at a different

01:00:46.000 --> 01:00:47.117
problem here.

01:00:47.200 --> 01:00:50.077
>> I'm just saying Eric is talking about a

01:00:50.160 --> 01:00:53.437
threemon time scale for the advances on

01:00:53.520 --> 01:00:57.197
these things. He's also talking about
this recursive self-improvement about

01:00:57.280 --> 01:01:01.037
the AIs that can make smarter AIs that
can make smarter AIs, you know, and and

01:01:01.120 --> 01:01:05.357
so my role in this is to say, hey,
you know, I'm not saying those are fake

01:01:05.440 --> 01:01:09.037
problems. Those are real problems. Some of
those problems are starting to happen now.

01:01:09.120 --> 01:01:12.957
>> But, you know, kids in the schools,
autonomous weapons, real conversations

01:01:13.040 --> 01:01:14.317
should be having, >> but

01:01:14.400 --> 01:01:17.437
>> but we're also racing towards super

01:01:17.520 --> 01:01:20.077
intelligent AIs that we don't have any

01:01:20.160 --> 01:01:24.877
idea how to point towards good stuff.
A lot of this like, oh well, you know,

01:01:24.960 --> 01:01:28.717
they make some mistakes. We we can't
pretest it. It goes off the rails.

01:01:28.800 --> 01:01:31.917
That's fine when it's dumber.
We're rushing towards making it smarter.

01:01:32.000 --> 01:01:34.717
That's going to just kill us. And we all

01:01:34.800 --> 01:01:39.277
have common cause in stopping that.
The people who are really pro- automation

01:01:39.360 --> 01:01:41.677
and are like, "Oh, we're going to have
a universal basic income. Don't worry

01:01:41.760 --> 01:01:44.957
about the jobs." And the people who are
like, "That's ridiculous." Uh, taking

01:01:45.040 --> 01:01:49.037
people's jobs removes their ability to
to to like have any leverage on society.

01:01:49.120 --> 01:01:51.277
Both those groups of people don't want

01:01:51.360 --> 01:01:53.997
to die to a rogue super intelligence.

01:01:54.080 --> 01:01:56.077
And so my role in this is to say, can we

01:01:56.160 --> 01:01:58.157
all join hands on the part where we

01:01:58.240 --> 01:02:01.757
don't die to a rogue super intelligence?

01:02:01.840 --> 01:02:05.437
>> Okay, I think we have good agreement
on that one. Kate, what you were thinking?

01:02:05.520 --> 01:02:08.557
What were you thinking, Kate? >> I mean,
I think part of the problem here

01:02:08.640 --> 01:02:12.077
is is how we frame this discussion,
particularly around jobs and and

01:02:12.160 --> 01:02:14.877
automation. In previous technologies,

01:02:14.960 --> 01:02:17.437
you know, the weaver was replaced by the

01:02:17.520 --> 01:02:19.517
loom or we had the horse and carriage

01:02:19.600 --> 01:02:22.397
replaced by the car. The the deal was

01:02:22.480 --> 01:02:25.197
that these more menial human tasks would

01:02:25.280 --> 01:02:29.677
be removed but they would give us more
time for these intellectual and creative

01:02:29.760 --> 01:02:33.917
challenges that people would be doing
more interesting work over time. This is

01:02:34.000 --> 01:02:35.437
a different revolution. >> Why?

01:02:35.520 --> 01:02:40.237
>> And in many ways because it is coming
for creative and cognitive labor.

01:02:40.320 --> 01:02:44.477
It is coming for the thing that in
theory automation was supposed to be

01:02:44.560 --> 01:02:48.477
freeing time for us to do. And what
we're starting to see is that is in fact

01:02:48.560 --> 01:02:51.917
of course white collar labor. It's
the jobs that so many of us would be like

01:02:52.000 --> 01:02:54.957
that's what you want to do. That's what
you train to go and become a lawyer or

01:02:55.040 --> 01:02:58.157
you go and study to be a computer
scientist. Who were some of the first

01:02:58.240 --> 01:02:59.997
people being laid off in this wave?

01:03:00.080 --> 01:03:03.437
Computer scientists.
So what can we automate?

01:03:03.520 --> 01:03:05.117
>> She's got a job. No point to her.

01:03:05.200 --> 01:03:08.397
>> Tanya will be fine. >> She's
fully employed here. But but this

01:03:08.480 --> 01:03:10.397
is a fundamentally different question.

01:03:10.480 --> 01:03:14.637
When cognition and creativity is being
automated, we can talk about how

01:03:14.720 --> 01:03:18.397
effectively there's clearly a lot
of problems there. But that is the shift

01:03:18.480 --> 01:03:21.917
that I think we have to be very serious
in talking about. I think it's easy to

01:03:22.000 --> 01:03:25.837
make historical analogies without
looking at the fundamental nuance and

01:03:25.920 --> 01:03:27.597
difference of what is happening here.

01:03:27.680 --> 01:03:31.117
And it really is something that the people
in this room are going to have to

01:03:31.200 --> 01:03:33.837
deal with. This is not a 100 years time.

01:03:33.920 --> 01:03:37.597
>> I'd also like to give a call to action
for people in this room. I agree that

01:03:37.680 --> 01:03:41.917
this is a much more significant problem
than the idea that a Terminator is going

01:03:42.000 --> 01:03:43.677
to emerge and kill us all.
>> No, it's not.

01:03:43.760 --> 01:03:47.837
>> That's a lovely that's a,
you know, speculative fiction

01:03:47.920 --> 01:03:51.597
that's great and worth thinking ahead
to the future, but it's speculative

01:03:51.680 --> 01:03:54.557
fiction. Uh, the job loss is something

01:03:54.640 --> 01:03:59.437
that we all play a role in, right? So,
I think that there's there should be moral

01:03:59.520 --> 01:04:03.517
clarity here. If you are a business owner,
if you are a business leader, if

01:04:03.600 --> 01:04:07.757
you have influence about hiring in your
company, In the not too distant future,

01:04:07.840 --> 01:04:10.317
you'll be faced with a question which is

01:04:10.400 --> 01:04:14.877
AI is suddenly making everyone twice
as productive. So the question for you

01:04:14.960 --> 01:04:19.277
>> making half as many people twice as >>
the question for you will be do I want

01:04:19.360 --> 01:04:23.117
my company to be more productive with
the same staff or do I want to be the

01:04:23.200 --> 01:04:25.997
same level with half the staff. Right?

01:04:26.080 --> 01:04:29.917
And that is a question that we as a
society need to make a decision about.

01:04:30.000 --> 01:04:34.317
And I strongly encourage you any in any
position where you have influence over

01:04:34.400 --> 01:04:38.797
this to go to the path of growth to try
to expand what your company can do with

01:04:38.880 --> 01:04:42.317
the same staff. Like it's you don't need
to replace people. You need to figure

01:04:42.400 --> 01:04:44.477
out what additional tasks they can

01:04:44.560 --> 01:04:45.197
serve.

01:04:45.280 --> 01:04:50.237
>> So Kate about this replacement of

01:04:50.320 --> 01:04:53.757
of positive and artistic

01:04:53.840 --> 01:04:56.557
>> talents. Let me take a middle road

01:04:56.640 --> 01:05:00.157
there. Middle position road there.

01:05:00.240 --> 01:05:03.917
There were teams of artists hired to

01:05:04.000 --> 01:05:07.357
reproduce some idea that was set down by

01:05:07.440 --> 01:05:11.677
a set designer or by some lead artist in

01:05:11.760 --> 01:05:14.237
a in a project. Is that much different

01:05:14.320 --> 01:05:18.957
from the the the the schools of art that

01:05:19.040 --> 01:05:23.597
we know about from the Renaissance where
there's the main artist who's the great

01:05:23.680 --> 01:05:25.677
artist and then everyone comes in and

01:05:25.760 --> 01:05:29.677
and apes that style. So my sense of this

01:05:29.760 --> 01:05:31.837
is and I could be wrong. You have we

01:05:31.920 --> 01:05:35.037
have expertise here to correct me that

01:05:35.120 --> 01:05:38.237
AI today is really good at copying

01:05:38.320 --> 01:05:40.557
what's already on the internet. I can

01:05:40.640 --> 01:05:44.557
tell Chad GPT or or Claude I say see the

01:05:44.640 --> 01:05:48.317
scene here paint it the way Van Go might

01:05:48.400 --> 01:05:50.637
have seen it and it will come back with

01:05:50.720 --> 01:05:53.437
the same color palette brush strokes and

01:05:53.520 --> 01:05:57.677
we'll be there in these swirly patterns
and we say that's what Van Go would have

01:05:57.760 --> 01:06:00.157
painted. Now if I then ask it paint this

01:06:00.240 --> 01:06:02.957
scene in a style of no artist who has

01:06:03.040 --> 01:06:05.757
ever lived

01:06:05.840 --> 01:06:08.637
>> what's it going to do then? And so for

01:06:08.720 --> 01:06:14.157
me, I think AI in that space is boosting

01:06:14.240 --> 01:06:16.237
the expectations we should have of

01:06:16.320 --> 01:06:18.797
ourselves, of our true creativity.

01:06:18.880 --> 01:06:22.957
Because true creativity is not doing
what someone else had done just a little

01:06:23.040 --> 01:06:27.837
better. It's doing what no one else has
done. And I don't know if AI has that

01:06:27.920 --> 01:06:29.597
ability

01:06:29.680 --> 01:06:35.240
if it's being trained on what
everybody else has done.

01:06:35.600 --> 01:06:38.157
I could I could on that a little bit here.

01:06:38.240 --> 01:06:41.677
>> Uh training on, you know, I've also

01:06:41.760 --> 01:06:43.997
heard uh it's just math, it's just test

01:06:44.080 --> 01:06:48.557
text prediction. Um training on

01:06:48.640 --> 01:06:53.357
text that humans produced does not requ
like like that is a harder problem to

01:06:53.440 --> 01:06:57.437
predict that text than producing the text
in the first place. An example that

01:06:57.520 --> 01:06:59.277
might resonate with people here is there

01:06:59.360 --> 01:07:02.877
was a great astronomer Taiko Braheed

01:07:02.960 --> 01:07:06.637
the position of of all the stars
and particularly Mars each night.

01:07:06.720 --> 01:07:09.197
>> Planets. >> Uh yeah, sorry. Yeah.

01:07:09.280 --> 01:07:10.717
>> All the stars are also Mars.

01:07:10.800 --> 01:07:14.797
>> You're standing next to the
director of the Hayden Planetarium.

01:07:14.880 --> 01:07:19.117
>> Okay. >> He also he
also recorded the stars.

01:07:19.200 --> 01:07:23.677
>> He also recorded the stars. But >>
but Mars is the one that Johannes Kepler

01:07:23.760 --> 01:07:25.517
tracked. >> Yes. >> Right.
And so Johannes Kepler, there was

01:07:25.600 --> 01:07:30.557
a big heist. He stole the the the
journals, the data. Uh but but Tao Brahe

01:07:30.640 --> 01:07:32.237
when he was recording the stars and

01:07:32.320 --> 01:07:35.437
planets. Uh

01:07:35.520 --> 01:07:38.077
he just looked at the night sky and

01:07:38.160 --> 01:07:41.197
wrote down where Mars was. If you

01:07:41.280 --> 01:07:45.997
imagine training something to predict
the entries in Brah's journal, it does

01:07:46.080 --> 01:07:49.517
not have the luxury of looking at the sky.

01:07:49.600 --> 01:07:54.157
You can solve this problem.
You can read a bunch of the entries in

01:07:54.240 --> 01:07:58.637
Brahe's journal and predict the next.
To do so, you have to discover the laws of

01:07:58.720 --> 01:08:01.037
the motions of the planets. Johannes

01:08:01.120 --> 01:08:04.077
Kepler did this. Brahi never did. So, an

01:08:04.160 --> 01:08:06.797
AI trained on only Brahi's journals,

01:08:06.880 --> 01:08:08.637
trained on only the prediction, only

01:08:08.720 --> 01:08:12.157
predicting the text that Brahi produced.

01:08:12.240 --> 01:08:14.877
That training target incentivizes the AI

01:08:14.960 --> 01:08:16.477
to figure out something that Brahhee

01:08:16.560 --> 01:08:19.517
never did. Now, can AIs today do this?

01:08:19.600 --> 01:08:22.877
That's one question. But training an AI

01:08:22.960 --> 01:08:26.877
to predict things human wrote down
or humans wrote down does not limit it to

01:08:26.960 --> 01:08:29.437
doing only things that humans can do.

01:08:29.520 --> 01:08:34.557
And we are starting to see AIs, you know,
we've seen chat GPT uh or GPT 5.1

01:08:34.640 --> 01:08:37.997
produce some novel physics contributions.
We're seeing them start

01:08:38.080 --> 01:08:40.317
to inch into ways that that humans

01:08:40.400 --> 01:08:43.597
haven't quite done things. Uh and you

01:08:43.680 --> 01:08:45.517
know sometimes you know when a new

01:08:45.600 --> 01:08:48.797
scientist uh comes up with a new idea

01:08:48.880 --> 01:08:53.277
they aren't necessarily coming up with
a wholesale entirely new way of thinking

01:08:53.360 --> 01:08:57.037
as opposed to making small amounts
of thinking progress and using a lot of

01:08:57.120 --> 01:08:59.517
other thinking progress that that came
from other people. You know they're

01:08:59.600 --> 01:09:04.637
standing on the shoulders of giants.
Uh so the things we're doing with AI can

01:09:04.720 --> 01:09:07.117
get them there. There's a different
question of whether we're currently

01:09:07.200 --> 01:09:09.357
there but it can get them there. Okay.

01:09:09.440 --> 01:09:13.357
And this idea that they're just math
and so it won't get there. That's like

01:09:13.440 --> 01:09:17.517
saying, "Oh, you know, a house cat
is just biochemistry, so no sort of cat is

01:09:17.600 --> 01:09:22.237
dangerous." You know, a house cat
is just biochemistry and a house cat can't

01:09:22.320 --> 01:09:27.880
kill you. But that doesn't
mean that tigers are fake.

01:09:29.120 --> 01:09:31.917
>> Wow.

01:09:32.000 --> 01:09:36.200
>> Wait, was that a logical leap?

01:09:36.400 --> 01:09:39.357
>> You look like you wanted to move. I I

01:09:39.440 --> 01:09:41.677
we're running short on time here. Uh I

01:09:41.760 --> 01:09:45.277
want to uh get back to uh a very serious

01:09:45.360 --> 01:09:50.477
point. Uh I I was appointed under Obama

01:09:50.560 --> 01:09:53.037
to serve on the Defense Innovation

01:09:53.120 --> 01:09:55.037
Board, a freshly created board of the

01:09:55.120 --> 01:09:58.637
Pentagon that was tasked with advising

01:09:58.720 --> 01:10:03.357
the Pentagon and all the ways technology
might influence the future present and

01:10:03.440 --> 01:10:07.117
future warfare. Not just how many
missiles are in your silo or how many

01:10:07.200 --> 01:10:09.997
push-ups the soldiers can do. Are there

01:10:10.080 --> 01:10:11.997
other innovations that are possible?

01:10:12.080 --> 01:10:14.477
Eric Schmidt was the chair of that

01:10:14.560 --> 01:10:16.717
board. I was privileged to serve under

01:10:16.800 --> 01:10:19.437
him for this. We toured uh military

01:10:19.520 --> 01:10:22.477
bases around the world. Spoke with uh

01:10:22.560 --> 01:10:25.677
soldiers uh very dedicated to their

01:10:25.760 --> 01:10:29.677
>> every single one of them
wanted his autograph.

01:10:29.760 --> 01:10:31.277
>> I'm serious. He would show up and it's

01:10:31.360 --> 01:10:35.037
like all this is like oh my god.

01:10:35.120 --> 01:10:37.837
Uh so Eric, right about the time uh

01:10:37.920 --> 01:10:40.397
drones were becoming sort of more lethal

01:10:40.480 --> 01:10:43.917
and more real and uh we felt it

01:10:44.000 --> 01:10:47.437
necessary as a committee, as a board to

01:10:47.520 --> 01:10:51.197
draft some kind of rules of engagement

01:10:51.280 --> 01:10:53.837
regarding how and when AI would be

01:10:53.920 --> 01:10:56.477
brought into the chain of decisions that

01:10:56.560 --> 01:11:00.397
would lead in lethal lead to lethality.

01:11:00.480 --> 01:11:02.957
And that was a code of ethics really.

01:11:03.040 --> 01:11:07.117
And I was quite proud of that because
there was no telling what would happen

01:11:07.200 --> 01:11:09.037
without such guidance. My question to

01:11:09.120 --> 01:11:12.477
you is our motivation at the time was

01:11:12.560 --> 01:11:17.197
the knowledge that AI was not perfect
that AI would make mistakes and you

01:11:17.280 --> 01:11:19.357
never want someone to die because of a

01:11:19.440 --> 01:11:21.677
mistake. Yet of course people make

01:11:21.760 --> 01:11:25.517
mistakes. So at what point is AI good

01:11:25.600 --> 01:11:29.357
enough in that space to say I trust AI

01:11:29.440 --> 01:11:34.477
over a person and let AI run run with
the decision and not a human. Is it just

01:11:34.560 --> 01:11:37.997
that at the end you want to have someone
to blame because you can't blame the

01:11:38.080 --> 01:11:40.957
computer you can't put
the computer on trial

01:11:41.040 --> 01:11:44.077
>> but a person you can so what what

01:11:44.160 --> 01:11:46.877
happens when AI can find the enemy

01:11:46.960 --> 01:11:51.517
perfectly. So everybody here has
an opportunity to read the report that we

01:11:51.600 --> 01:11:55.357
issued under DIB um >> DIB
Defense Innovation Board

01:11:55.440 --> 01:11:57.917
>> and the Pentagon
actually adopted it and

01:11:58.000 --> 01:12:00.317
published it as their guidelines.

01:12:00.400 --> 01:12:02.397
Roughly summarize what what our thinking

01:12:02.480 --> 01:12:05.757
was at the time. Um

01:12:05.840 --> 01:12:09.917
all war is terrible. All military
activity is terrible. It involves the

01:12:10.000 --> 01:12:11.837
destruction of other people and so forth

01:12:11.920 --> 01:12:14.317
and so on. It's just horrible. Trust me.

01:12:14.400 --> 01:12:18.077
>> Um and the question is can If you have

01:12:18.160 --> 01:12:20.317
to do it, can you do it using AI to be

01:12:20.400 --> 01:12:25.197
more accurate and more successful? That
was the question. And we debated this at

01:12:25.280 --> 01:12:26.877
some length and we concluded that at

01:12:26.960 --> 01:12:28.797
least at the time, and I think it's true

01:12:28.880 --> 01:12:32.557
today, AI is not reliable enough to rely

01:12:32.640 --> 01:12:35.117
on it to make those decisions and it's

01:12:35.200 --> 01:12:38.077
unlikely to be so for a very long time.

01:12:38.160 --> 01:12:41.677
That was our conclusion. Uh sort
of makes sense, right? Because you can't

01:12:41.760 --> 01:12:45.037
tolerate sort of a percentage of errors
where you might be able to tolerate them

01:12:45.120 --> 01:12:47.917
in other areas. Um and so what we wrote

01:12:48.000 --> 01:12:51.357
was that any use of automation in that

01:12:51.440 --> 01:12:54.717
area uh essentially called lethality um

01:12:54.800 --> 01:12:56.797
needed to have a couple things. It

01:12:56.880 --> 01:13:01.917
needed to have been tested. It needed
to have a time and a space duration. And

01:13:02.000 --> 01:13:05.997
most important there had to be a human
who was oversight overseeing it. They

01:13:06.080 --> 01:13:09.997
had to know it was going to happen ahead
of time. They had to have tested that it

01:13:10.080 --> 01:13:13.277
was going to happen and then it had
to reproduce. If it didn't then that was a

01:13:13.360 --> 01:13:17.517
major problem. And that's ultimately
what the Pentagon agreed to. Um this is

01:13:17.600 --> 01:13:20.477
a compromise because these tools
are very powerful but they also make

01:13:20.560 --> 01:13:22.637
mistakes. So that's ultimately I think

01:13:22.720 --> 01:13:27.837
where the Pentagon ended up and you
and is now that was some years ago now.

01:13:27.920 --> 01:13:29.517
We're at least a decade forward from

01:13:29.600 --> 01:13:34.957
that. Is AI good enough so that you
would have would have scripted this

01:13:35.040 --> 01:13:37.357
differently? >> I don't I
don't believe you and I and

01:13:37.440 --> 01:13:41.677
the others would have come to any other
conclusion on the lethality part. Uh the

01:13:41.760 --> 01:13:44.637
lethality part is obviously the most
important one. It's the most important

01:13:44.720 --> 01:13:48.877
decision that's made by members of the
military under military law and uh laws

01:13:48.960 --> 01:13:51.597
of war and so forth and so on. Um what

01:13:51.680 --> 01:13:54.797
is true is that the AI planning and

01:13:54.880 --> 01:13:59.277
recognition that is the eyesight
and the ability to do planning is now far

01:13:59.360 --> 01:14:01.357
superior than with humans. That's not

01:14:01.440 --> 01:14:04.797
the same thing as lethality.

01:14:04.880 --> 01:14:07.517
>> Okay. So these are
subtle differences in

01:14:07.600 --> 01:14:09.917
the capacity of AI that are not always

01:14:10.000 --> 01:14:14.557
recognized or appreciated.
>> Well, I I think we we all agree that

01:14:14.640 --> 01:14:16.717
these systems are not perfect and at

01:14:16.800 --> 01:14:19.037
least in things which are life critical

01:14:19.120 --> 01:14:21.677
like deciding what to do with a patient,

01:14:21.760 --> 01:14:24.477
right? Deciding when an airplane is in

01:14:24.560 --> 01:14:29.357
trouble, uh anything involving military
kinetic conflict, uh think twice before

01:14:29.440 --> 01:14:31.437
you empower AI because it makes

01:14:31.520 --> 01:14:37.960
mistakes. I I I want
to thank you for that, Eric,

01:14:38.720 --> 01:14:41.037
and thank you for your service in that.

01:14:41.120 --> 01:14:43.357
Uh just to land this plane, what I want

01:14:43.440 --> 01:14:46.637
to do is go down the panel and ask, you

01:14:46.720 --> 01:14:49.837
know, given this 3 months horizon for

01:14:49.920 --> 01:14:52.957
new stuff that happens uh in AI and that

01:14:53.040 --> 01:14:56.397
whole sector. Uh, let's just fast

01:14:56.480 --> 01:15:01.757
forward way into the future to 2030

01:15:01.840 --> 01:15:07.357
and ask Wait, wait, wait. How long
ago was Chat GPT? How many years ago?

01:15:07.440 --> 01:15:11.037
>> Three and a half. >> Three
and a half years ago was chat GPT

01:15:11.120 --> 01:15:14.237
one or whatever the first public version

01:15:14.320 --> 01:15:16.717
of it. So, let's fast forward three and

01:15:16.800 --> 01:15:19.517
a half years to 2030.

01:15:19.600 --> 01:15:22.077
uh what is the state of the world and

01:15:22.160 --> 01:15:27.197
its relationship with AI?
Let's start with you.

01:15:27.280 --> 01:15:31.197
>> Yeah. Well, I can tell you the vision
that we would like to see. Sure.

01:15:31.280 --> 01:15:36.317
>> And and we have and you you're not just
passively hoping it'll get there. You're

01:15:36.400 --> 01:15:38.077
actively working to achieve this,

01:15:38.160 --> 01:15:40.797
>> right?
And and and it's a future that is

01:15:40.880 --> 01:15:44.397
really amazing in the sense that

01:15:44.480 --> 01:15:47.357
everyone is far more productive. I can

01:15:47.440 --> 01:15:51.597
be in multiple places at one time
because there are agents that know my

01:15:51.680 --> 01:15:56.317
preferences and my desires and they
repres represent me in various context.

01:15:56.400 --> 01:15:57.917
>> Just to be clear, you guys use the word

01:15:58.000 --> 01:16:03.997
agent to refer to an AI uh
representative of your needs and

01:16:04.080 --> 01:16:06.477
activities. >> That's correct.
And so they might

01:16:06.560 --> 01:16:09.197
negotiate a purchase arrangement for me

01:16:09.280 --> 01:16:12.077
or uh be in a meeting and represent my

01:16:12.160 --> 01:16:15.517
interest. And so the my productivity is

01:16:15.600 --> 01:16:19.357
just orders of magnitude
greater than it is today.

01:16:19.440 --> 01:16:23.517
>> And you're fine with that?
>> I'm fine with that provided that every

01:16:23.600 --> 01:16:26.237
single one of my uses of AI and making

01:16:26.320 --> 01:16:30.557
me more productive is actually able to

01:16:30.640 --> 01:16:33.917
show that it's it gives me a warranty of

01:16:34.000 --> 01:16:37.597
what it will do and how it will
represent itself. In other words, the

01:16:37.680 --> 01:16:41.037
same thing I get when I get
an appliance, I get a warranty and a

01:16:41.120 --> 01:16:43.757
compliance statement. I want that same

01:16:43.840 --> 01:16:48.477
thing on these agents and I'll be able
to do that. In some ways, what you guys

01:16:48.560 --> 01:16:50.557
were offering to the military, there are

01:16:50.640 --> 01:16:55.597
also many societal harms that aren't
about immediate life-threatening, but

01:16:55.680 --> 01:16:59.597
are about our ability to have
a democracy, about our ability to

01:16:59.680 --> 01:17:04.477
understand truthful information and so
forth that are at stake and need those

01:17:04.560 --> 01:17:08.397
same kind of guidance. It would be like
an underwriters laboratory certification

01:17:08.480 --> 01:17:14.637
that we have for our actual appliances
but applied to your AI agents. Okay.

01:17:14.720 --> 01:17:15.837
>> I think >> 2030

01:17:15.920 --> 01:17:19.037
>> 2030 I I think we have two paths in

01:17:19.120 --> 01:17:22.397
front of us. On one path, we move

01:17:22.480 --> 01:17:24.877
towards a place where we see what is

01:17:24.960 --> 01:17:27.677
already one of the most concentrated

01:17:27.760 --> 01:17:32.477
industries in industrial in the
industrial history of of the planet

01:17:32.560 --> 01:17:34.557
become even more concentrated and even

01:17:34.640 --> 01:17:36.717
fewer people have their hands on levels

01:17:36.800 --> 01:17:39.997
of power and democracies start to erode.

01:17:40.080 --> 01:17:43.917
And these systems ultimately are making
decisions that will decide what kind of

01:17:44.000 --> 01:17:47.997
job you can have, what kind of quality
of care you get, um what kind of options

01:17:48.080 --> 01:17:51.597
you and your family will have
going forward. The other path is

01:17:51.680 --> 01:17:55.677
>> just remind me is it there I'll get
these numbers long but the census is

01:17:55.760 --> 01:17:57.277
accurate. There are four or five

01:17:57.360 --> 01:18:00.877
companies on the S&P 500 that represent

01:18:00.960 --> 01:18:03.757
like 30% of the total wealth

01:18:03.840 --> 01:18:05.597
>> of the United States. Correct.

01:18:05.680 --> 01:18:09.997
>> Some some number like that might
be precise but the census correct.

01:18:10.080 --> 01:18:13.917
>> You're exactly right. And if you look
at the numbers of people who are actually

01:18:14.000 --> 01:18:17.597
running these companies you're talking
about a handful a tiny handful of

01:18:17.680 --> 01:18:21.277
billionaires. They are the people who
are getting to make these decisions

01:18:21.360 --> 01:18:25.037
about how these tools will be used
and what problems they will be used to

01:18:25.120 --> 01:18:29.917
solve. The other path is that we
realize this is the most important public

01:18:30.000 --> 01:18:33.597
conversation that we can be having
right now. Which is why I'm so glad we're

01:18:33.680 --> 01:18:35.197
gathered here today having this

01:18:35.280 --> 01:18:37.437
discussion and we decide that actually

01:18:37.520 --> 01:18:40.957
this is too important to be left in such

01:18:41.040 --> 01:18:45.197
a small number of hands of people who
are not democratically accountable. you

01:18:45.280 --> 01:18:48.957
asked what the difference was between
a representative and a and a tech row is

01:18:49.040 --> 01:18:51.517
that we vote for one of them and we

01:18:51.600 --> 01:18:54.957
don't vote for the other ones and

01:18:55.040 --> 01:18:58.957
>> okay so take me to 2030

01:18:59.040 --> 01:19:00.877
>> what I would love
to see by 2030 is that

01:19:00.960 --> 01:19:03.437
we say this matters the public interest

01:19:03.520 --> 01:19:08.157
of these technologies is at the core and
if it is going to be using our land our

01:19:08.240 --> 01:19:12.077
energy and our water then we need
to have a say in it and these tools should

01:19:12.160 --> 01:19:16.157
be used for the public good and that
actually means making this something

01:19:16.240 --> 01:19:18.637
that serves all of us, not the few.

01:19:18.720 --> 01:19:22.440
>> Are you running for office?

01:19:25.520 --> 01:19:29.197
>> Chris, take me to 2030. What What
are you teaching in 2030 or not teaching

01:19:29.280 --> 01:19:32.317
today? And by the way, uh Isaac Azimov

01:19:32.400 --> 01:19:37.437
had his three laws of robotics. Do they
apply today? And should is there an AI

01:19:37.520 --> 01:19:41.437
laws of robotics? >> There
is an AI laws of robotics. So as

01:19:41.520 --> 01:19:45.117
laws and robotics for the non- nerds
in the audience and I say that you are all

01:19:45.200 --> 01:19:47.997
nerds. Thank you for being here with us

01:19:48.080 --> 01:19:50.477
are you know firstly that a robot must

01:19:50.560 --> 01:19:52.557
do no harm to an individual including

01:19:52.640 --> 01:19:55.517
through inaction. Secondly that a robot

01:19:55.600 --> 01:19:57.757
must obey what the human says and

01:19:57.840 --> 01:20:00.397
thirdly that the robot must not cause

01:20:00.480 --> 01:20:02.797
harm to itself barring the other two. Of

01:20:02.880 --> 01:20:05.757
course there was a zero. So B, you mean

01:20:05.840 --> 01:20:09.357
it can't it can't cause harm

01:20:09.440 --> 01:20:11.917
to its first two take precedent

01:20:12.000 --> 01:20:15.117
>> with Yes. What to accomplish the third?

01:20:15.200 --> 01:20:17.997
It's it cannot undo
the first or the second.

01:20:18.080 --> 01:20:22.637
>> Exactly. And then there was a later
zero law which is the robots cannot harm

01:20:22.720 --> 01:20:24.957
humanity which is the broader thing

01:20:25.040 --> 01:20:27.357
rather than just an individual. So this

01:20:27.440 --> 01:20:30.237
is a key challenge for AI. It's the

01:20:30.320 --> 01:20:32.477
alignment problem.

01:20:32.560 --> 01:20:35.597
And just I have to say so Isaac Asmer

01:20:35.680 --> 01:20:37.997
wrote a series of short stories that

01:20:38.080 --> 01:20:44.477
were collected into a book called I
Robot which then became a film starring um

01:20:44.560 --> 01:20:48.557
>> Will Smith. >> Will Smith
and uh in there he explored

01:20:48.640 --> 01:20:51.117
these rules of engagement for robots

01:20:51.200 --> 01:20:53.437
very early. This is the early 1950s. So

01:20:53.520 --> 01:20:56.637
we're talking about 70 years ago plus 75

01:20:56.720 --> 01:20:59.037
years ago. Yeah, again this is the place

01:20:59.120 --> 01:21:00.957
where specular friction thinks about the

01:21:01.040 --> 01:21:03.757
future in a very useful way. So uh AI

01:21:03.840 --> 01:21:07.997
alignment is exactly these laws. So how
do we think about making a systems that

01:21:08.080 --> 01:21:12.317
helpful? How do we system think about
making a system that's safe? How do we

01:21:12.400 --> 01:21:13.917
ensure that it's for the benefit of

01:21:14.000 --> 01:21:16.237
humanity rather than to its detriment?

01:21:16.320 --> 01:21:19.677
So as we go into 2030, I think that

01:21:19.760 --> 01:21:22.397
those key elements that are similar to

01:21:22.480 --> 01:21:27.437
Azimoff's laws become the key things
that guide the success or failure of the

01:21:27.520 --> 01:21:30.877
AI endeavor. >> And that could happen
within the next three and a half years.

01:21:30.960 --> 01:21:32.877
>> It's it's happening now. There are many

01:21:32.960 --> 01:21:34.957
many thoughtful researchers working on

01:21:35.040 --> 01:21:37.997
these. You can check out the uh talk by

01:21:38.080 --> 01:21:40.797
Amanda Ascal, the chief philosopher for

01:21:40.880 --> 01:21:43.997
Anthropic, about how she's crafted

01:21:44.080 --> 01:21:48.477
Claude's soul document, the thing
that governs its behavior and its code of

01:21:48.560 --> 01:21:50.957
ethics. It's incredibly thoughtful and

01:21:51.040 --> 01:21:56.637
it gives me hope for the future.
Did Google have a philosopher?

01:21:56.720 --> 01:22:00.157
>> Very good. >> Okay.

01:22:00.240 --> 01:22:03.037
>> Uh thank you for that. So Cindy, what

01:22:03.120 --> 01:22:04.877
you got going for us? 2030

01:22:04.960 --> 01:22:07.917
>> 2030 a few years from now um I think

01:22:08.000 --> 01:22:10.157
that we'll see um you know a lot of

01:22:10.240 --> 01:22:12.477
these technologies we have now the the

01:22:12.560 --> 01:22:16.237
kind of LLMs will become um more useful

01:22:16.320 --> 01:22:20.557
in our day-to-day life right now
you know it can kind of respond to to

01:22:20.640 --> 01:22:22.717
questions or to to prompts we give it

01:22:22.800 --> 01:22:24.477
and maybe it'll become a little bit

01:22:24.560 --> 01:22:27.837
closer to be able to um responding to

01:22:27.920 --> 01:22:29.917
you know queries more like achieve this

01:22:30.000 --> 01:22:32.477
objective so I can imagine if you know

01:22:32.560 --> 01:22:34.877
specialized versions of this like you

01:22:34.960 --> 01:22:37.357
know help you respond to emails and

01:22:37.440 --> 01:22:39.997
these sorts of things. um feet on the

01:22:40.080 --> 01:22:42.477
ground. I think we maybe will see uh

01:22:42.560 --> 01:22:44.637
self-driving cars in New York City in a

01:22:44.720 --> 01:22:47.997
few years, you know. Um I I think uh you

01:22:48.080 --> 01:22:50.797
know that a lot of the the AI will be

01:22:50.880 --> 01:22:53.277
trained on more specialized data sets.

01:22:53.360 --> 01:22:55.997
So they'll become better at um certain

01:22:56.080 --> 01:22:58.717
tasks like uh you know scientific

01:22:58.800 --> 01:23:01.437
research or mathematical research. um

01:23:01.520 --> 01:23:06.957
you know if you you train on kind of
certain parts of the the internet it may

01:23:07.040 --> 01:23:08.317
be better than what it is now.

01:23:08.400 --> 01:23:13.117
>> Quickly Kate if AI got that specialized
wouldn't that be more efficient on the

01:23:13.200 --> 01:23:16.717
grid? >> It it could be.
Um this is one of the

01:23:16.800 --> 01:23:20.557
really big questions. I mean right now
large scale models are being trained

01:23:20.640 --> 01:23:22.717
using a lot of fossil fuels. A lot in

01:23:22.800 --> 01:23:27.997
fact more than previously. So if we had
small models, if they were specialized,

01:23:28.080 --> 01:23:30.557
there are ways forward where
you can use much less energy.

01:23:30.640 --> 01:23:31.837
>> That's what I would have figured. And

01:23:31.920 --> 01:23:34.237
for example, in the Jetsons, the

01:23:34.320 --> 01:23:38.237
documentary The Jetsons of the Future

01:23:38.320 --> 01:23:40.957
that they as as fun and creative as they

01:23:41.040 --> 01:23:46.237
were, they did not realize that maybe
the car could one day drive itself. The

01:23:46.320 --> 01:23:49.917
human drove it or you get a robot
to drive it, but the car is the robot. And

01:23:50.000 --> 01:23:52.637
that's a very specific invocation of

01:23:52.720 --> 01:23:55.997
this. So I based on the title of your

01:23:56.080 --> 01:24:01.800
book, I'm going to show
people your book because

01:24:02.400 --> 01:24:05.277
I got Kate's book here too, Kate.

01:24:05.360 --> 01:24:09.917
Atlas of AI, uh the Pub
Day 2021, but it's still

01:24:10.000 --> 01:24:12.797
happening, >> apparently.
So they just went to the the

01:24:12.880 --> 01:24:14.397
10th edition, so it's doing all right.

01:24:14.480 --> 01:24:17.597
>> Yeah. Okay, good. If anyone builds it,

01:24:17.680 --> 01:24:22.477
everyone dies. This has got to be
the most depressing title ever.

01:24:22.560 --> 01:24:26.840
It starts with if >> adverb.

01:24:27.280 --> 01:24:30.237
>> Well, that makes it all better. Okay.

01:24:30.320 --> 01:24:32.797
Uh, so take me to 2030. What's going on?

01:24:32.880 --> 01:24:35.437
Are we all dead?

01:24:35.520 --> 01:24:39.677
You know, if you step back from this

01:24:39.760 --> 01:24:43.357
from from the from the AI argument,
you just like look broadly at what's going

01:24:43.440 --> 01:24:47.837
on. The machines are talking now.

01:24:47.920 --> 01:24:51.677
You know, it people used to think it
was going to take a while to get to the

01:24:51.760 --> 01:24:53.917
point where the machines are talking
and we got these people being like, "Oh

01:24:54.000 --> 01:24:55.277
yeah, we're just growing these things."

01:24:55.360 --> 01:24:57.837
Turns out they talk. They're solving
problems we said weren't going to happen

01:24:57.920 --> 01:25:02.237
for 50 years. Now we're trying to take
it to super intelligence. That's kind of

01:25:02.320 --> 01:25:04.877
crazy. >> It's kind of crazy.
You know, you're

01:25:04.960 --> 01:25:07.517
like, "Do you guys know what's going
on there?" They're like, "No, but we think

01:25:07.600 --> 01:25:10.157
it'll be fine. Trust us." Right.

01:25:10.240 --> 01:25:13.917
>> In Silicon Valley, >> we're
not trusting you, just so you know.

01:25:14.000 --> 01:25:16.557
>> Great. >> Okay. Um,

01:25:16.640 --> 01:25:20.077
in Silicon Valley, people are spooked.

01:25:20.160 --> 01:25:23.757
You know, there's a joke in Silicon
Valley that when someone leaves a normal

01:25:23.840 --> 01:25:27.517
tech job, they say, you know, I've had
a great time here and now it's on to the

01:25:27.600 --> 01:25:29.357
next adventure. And when someone leaves

01:25:29.440 --> 01:25:32.077
an AI tech job, they say, I have stared

01:25:32.160 --> 01:25:34.557
into the abyss.

01:25:34.640 --> 01:25:41.560
I'm retiring to write poetry.
Please spend time with your families.

01:25:41.920 --> 01:25:45.917
>> Okay. And if you think I'm joking,
you can look at the resignation of the

01:25:46.000 --> 01:25:48.317
anthropic safety lead a few weeks ago

01:25:48.400 --> 01:25:51.117
and see how close that joke is. But the

01:25:51.200 --> 01:25:56.717
thing is, in Washington DC, they are
not spooked. In Washington, DC, they are

01:25:56.800 --> 01:26:00.477
thinking that this is all about
self-driving cars and autonomous weapons

01:26:00.560 --> 01:26:02.637
and job loss. And those are important

01:26:02.720 --> 01:26:06.637
issues. But what I hope we see in 2030

01:26:06.720 --> 01:26:09.437
is that DC is spooked.

01:26:09.520 --> 01:26:13.757
I hope we see a world where DC has
realized that the people running the

01:26:13.840 --> 01:26:18.157
labs, the academics outside the labs
like Jeffrey Hinton, that the people

01:26:18.240 --> 01:26:22.637
inside the labs that everyone who's
close to this technology is saying like,

01:26:22.720 --> 01:26:25.677
"Oh my gosh, the super intelligence
stuff could be really dangerous." Even

01:26:25.760 --> 01:26:28.717
Eric was saying, you know, there's red
lines and if we start to get close to

01:26:28.800 --> 01:26:30.157
these things, we need to watch out.

01:26:30.240 --> 01:26:33.357
>> Even Eric said that.

01:26:33.440 --> 01:26:36.317
>> Just briefly, you you mentioned uh Dr.

01:26:36.400 --> 01:26:40.077
Hinton. uh he's one of the fathers of of

01:26:40.160 --> 01:26:43.437
uh of neur neural nets >> deep learning

01:26:43.520 --> 01:26:45.917
>> uh deep learning
and uh he was given the

01:26:46.000 --> 01:26:49.997
Nobel Prize was it in chemistry
I think >> I thought physics

01:26:50.080 --> 01:26:53.597
>> oh phys physics thank you the Nobel
Prize in physics just a few years ago

01:26:53.680 --> 01:26:55.837
and we were fortunate enough to get him

01:26:55.920 --> 01:26:58.237
on to my podcast for an interview uh

01:26:58.320 --> 01:27:00.957
it's online now uh with Jeffrey Hinton

01:27:01.040 --> 01:27:06.397
talking about the the the original
thinkings of AI and his reflections on

01:27:06.480 --> 01:27:09.357
where it is where it might >>
those three people did their work at

01:27:09.440 --> 01:27:12.077
Google.

01:27:12.160 --> 01:27:14.237
>> So,

01:27:14.320 --> 01:27:19.517
take me to 2030. This is DC AI sensible

01:27:19.600 --> 01:27:24.317
by 2030 cuz we're in the middle
of whoever the next president will be.

01:27:24.400 --> 01:27:27.597
>> Yeah. So, I I think >> presuming
it won't beat Trump in a third

01:27:27.680 --> 01:27:31.517
term. Just have to That's
an if statement. Okay.

01:27:31.600 --> 01:27:35.757
>> I think I think they could be.
You know, we we see a lot of people in the

01:27:35.840 --> 01:27:39.597
industry saying that this is horribly
riskier than any other technology we

01:27:39.680 --> 01:27:43.437
work with. And we see people at these
labs saying, you know, I think there's a

01:27:43.520 --> 01:27:46.077
good chance this kills everybody, but I
got to do it because I'm better than the

01:27:46.160 --> 01:27:47.757
next guy. And, you know, maybe one of

01:27:47.840 --> 01:27:49.037
them's right. Probably two of them

01:27:49.120 --> 01:27:53.917
aren't. But, uh, in in 2030, if if we

01:27:54.000 --> 01:27:58.797
keep having this conversation, if it
becomes more salient, I think we see uh

01:27:58.880 --> 01:28:01.757
people around the world realizing that we
shouldn't do this super intelligence

01:28:01.840 --> 01:28:05.757
stuff. And the world does not need
to look that different from today. You

01:28:05.840 --> 01:28:09.357
don't need to stop the self-driving cars.
You don't need to stop Alphaold

01:28:09.440 --> 01:28:13.037
doing the drug discovery. You don't
even need to stop the autonomous weapons.

01:28:13.120 --> 01:28:16.797
Society can have a debate about,
you know, do we want to to save people on

01:28:16.880 --> 01:28:19.117
the front lines? Do we want to, you

01:28:19.200 --> 01:28:23.917
know, not do this, not go down this path.
But uh but we could stop just the

01:28:24.000 --> 01:28:26.077
race to build smarter and smarter AIs

01:28:26.160 --> 01:28:28.157
that nobody understands that would be

01:28:28.240 --> 01:28:31.517
easier than nuclear arms control. And I

01:28:31.600 --> 01:28:35.757
hope that in 2030 leaders around the world
will realize that this is lethally

01:28:35.840 --> 01:28:37.997
dangerous and we'll be putting a treaty

01:28:38.080 --> 01:28:40.477
in place to say we're not rushing to

01:28:40.560 --> 01:28:43.277
smarter than human AIs with anything

01:28:43.360 --> 01:28:45.037
remotely like the modern technology or

01:28:45.120 --> 01:28:47.677
we have no idea what's going on there.

01:28:47.760 --> 01:28:52.077
>> All right, Eric, >> you
asked a question about 2030.

01:28:52.160 --> 01:28:54.157
>> Yes, I did.

01:28:54.240 --> 01:28:57.597
>> Why do we bet there? Why do we bet that

01:28:57.680 --> 01:29:00.717
American democracy will survive?

01:29:00.800 --> 01:29:02.797
>> Okay, so that's an important reason.

01:29:02.880 --> 01:29:07.597
>> So I'm I'm gonna make a bet
that American democracy will actually

01:29:07.680 --> 01:29:10.477
survive. We'll get through whatever
the current problems are, of which there

01:29:10.560 --> 01:29:14.077
have been many listed here. That the
excesses will be addressed through

01:29:14.160 --> 01:29:18.317
legislation. I know that's hard
to imagine right now, right? But I'm going

01:29:18.400 --> 01:29:22.317
to bet that democracy will ultimately
reflect the will of the voters, which is

01:29:22.400 --> 01:29:24.957
how it's supposed to work. So in that

01:29:25.040 --> 01:29:30.157
scenario, which I know is difficult
to imagine at the moment, uh let's think

01:29:30.240 --> 01:29:35.037
about some of the other benefits of all
of this. The likelihood of students

01:29:35.120 --> 01:29:36.957
being much better educated because of

01:29:37.040 --> 01:29:39.517
their ability to be tutored in a using

01:29:39.600 --> 01:29:44.317
AI and specialized way is enormous.
The ability to have the next generation,

01:29:44.400 --> 01:29:46.477
your kids and grandchildren, being so

01:29:46.560 --> 01:29:48.877
much smarter than we will ever be is

01:29:48.960 --> 01:29:51.597
pretty high. Furthermore, this issue

01:29:51.680 --> 01:29:56.237
about jobs, there's no question there's
going to be job dislocation. But last

01:29:56.320 --> 01:29:59.437
time I checked, every one of those
predictions has been false for the

01:29:59.520 --> 01:30:04.477
reasons Neil mentioned earlier, which is
the American economy is fairly efficient

01:30:04.560 --> 01:30:08.317
recycling jobs. And there's definitely
harm. There's definitely people hurt.

01:30:08.400 --> 01:30:12.317
But at the end of the day, people
working with computers today make more

01:30:12.400 --> 01:30:15.277
money than people without. Furthermore,

01:30:15.360 --> 01:30:18.397
let's bet that these other issues, the

01:30:18.480 --> 01:30:22.317
regulatory issues, the techro issues
are resolved through some combination of

01:30:22.400 --> 01:30:24.797
self-interest, shareholder values, legal

01:30:24.880 --> 01:30:28.397
liability, and regulation. In that

01:30:28.480 --> 01:30:32.157
model, in 2030, every one of you will

01:30:32.240 --> 01:30:35.917
have an assistant, a savant that allows

01:30:36.000 --> 01:30:40.877
you under your control, not spying on you,
not doing something bad, under your

01:30:40.960 --> 01:30:44.717
control to do the things that you most
care about. You're a physicist, right?

01:30:44.800 --> 01:30:48.477
Neil's a physicist. You have
a physician's assistant who overnight does

01:30:48.560 --> 01:30:51.597
your thinking while you're busy sleeping
and tells you what is interesting in the

01:30:51.680 --> 01:30:55.117
day and you love it and you can turn
it off if you don't like it. There's

01:30:55.200 --> 01:30:59.997
example after example. The most likely
scenario by 2030 is something analogous

01:31:00.080 --> 01:31:03.357
to what I've described. All of the
things that everybody's worried about

01:31:03.440 --> 01:31:06.877
are true at some level or they're point

01:31:06.960 --> 01:31:13.400
issues that have to be debated.
In my book, which you did not feature,

01:31:13.520 --> 01:31:17.037
I say very very clearly.
>> Tell us the name of the book.

01:31:17.120 --> 01:31:21.197
>> Genesis. >> Thank you. >> Including
with myself and Henry Crew and

01:31:21.280 --> 01:31:24.317
a few others. >> Um was co-author.

01:31:24.400 --> 01:31:26.877
>> Yes. Yes. >> When did you write?
He'd been dead for

01:31:26.960 --> 01:31:29.837
when did

01:31:29.920 --> 01:31:31.837
>> Okay.

01:31:31.920 --> 01:31:35.837
The AI Henry Kissinger. Okay. Wow.

01:31:35.920 --> 01:31:38.477
>> In fairness to Henry,
he he finished the

01:31:38.560 --> 01:31:41.197
book The Dable Read

01:31:41.280 --> 01:31:45.757
>> and I encourage you to read it. Uh his
last book of his many many great books.

01:31:45.840 --> 01:31:49.677
The important thing thinking about this
is really important and our work. What

01:31:49.760 --> 01:31:52.557
we said is this conversation needs to

01:31:52.640 --> 01:31:56.957
occur over and over again because it is
the future of us. It is us. It is how we

01:31:57.040 --> 01:32:01.437
operate and I endorse that. Uh but at
the end of the day, right, we will get

01:32:01.520 --> 01:32:05.357
through this and more importantly, we
will thrive. Do you know why? Because at

01:32:05.440 --> 01:32:07.517
least American capitalism, the way it's

01:32:07.600 --> 01:32:09.997
structured with regulation, produces

01:32:10.080 --> 01:32:12.717
enormous economic and life value for us.

01:32:12.800 --> 01:32:16.557
Think about it. Solving climate change,
solving the various issues we have, and

01:32:16.640 --> 01:32:19.997
remember, we're depopulated, especially
because we're not allowing immigrants

01:32:20.080 --> 01:32:24.157
in, which is an error, right?
So, you get the idea.

01:32:24.240 --> 01:32:29.800
>> Wow. Let me see if I
can put a ball on this.

01:32:31.360 --> 01:32:34.077
So

01:32:34.160 --> 01:32:36.637
>> I try to always

01:32:36.720 --> 01:32:41.837
generate a cosmic perspective on all
that happens to me in a day and to get

01:32:41.920 --> 01:32:43.517
this kind of exposure. I don't know how

01:32:43.600 --> 01:32:46.557
to tie this up in a bow, all this in

01:32:46.640 --> 01:32:51.357
one, but allow me to say that and I've

01:32:51.440 --> 01:32:54.557
been in computing for most of my life

01:32:54.640 --> 01:32:56.717
that I've had access to it and awareness

01:32:56.800 --> 01:33:00.317
of it and and I remember, you know, the

01:33:00.400 --> 01:33:03.037
Turing test. People said if you interact

01:33:03.120 --> 01:33:04.797
with a computer and you don't know if

01:33:04.880 --> 01:33:07.837
there's a human on the other side then

01:33:07.920 --> 01:33:11.677
it you why distinguish it from being a

01:33:11.760 --> 01:33:15.437
computer or a human that passed
the touring test. The original research

01:33:15.520 --> 01:33:17.597
paper was called the imitation game

01:33:17.680 --> 01:33:19.757
which became the title of the movie

01:33:19.840 --> 01:33:22.797
featuring the life of Alan Turing which

01:33:22.880 --> 01:33:26.877
I recommend. Uh so we're way past that.

01:33:26.960 --> 01:33:30.797
All right. And so I remember thinking,
"Oh, we'll never have a computer as good

01:33:30.880 --> 01:33:34.797
as people at chess." We're way past that.

01:33:34.880 --> 01:33:37.597
I remembered when Siri showed up on the

01:33:37.680 --> 01:33:41.997
iPhone. There were whole commercials
with like who was the movie director who

01:33:42.080 --> 01:33:46.877
was interacting with it? Martin Scorsese
is talking to Siri and while our jaws

01:33:46.960 --> 01:33:48.477
are dropped open, there's this

01:33:48.560 --> 01:33:51.597
intelligent agent serving him in the

01:33:51.680 --> 01:33:54.637
voice of Siri in his iPhone. And I

01:33:54.720 --> 01:33:57.277
think, "Wow, the future is here.

01:33:57.360 --> 01:34:01.437
Today, no one talks about Siri as AI.

01:34:01.520 --> 01:34:04.317
We're so past this. No, that's just old.

01:34:04.400 --> 01:34:06.717
That's not even interesting. Alph Go,

01:34:06.800 --> 01:34:08.557
that's yesterday.

01:34:08.640 --> 01:34:11.757
Uh Alpha Zero, yesterday.

01:34:11.840 --> 01:34:14.557
So, what intrigues me is every one of

01:34:14.640 --> 01:34:17.677
these incremental advances in AI, we

01:34:17.760 --> 01:34:21.277
absorb just becomes part of life and

01:34:21.360 --> 01:34:24.077
we're looking at that next frontier.

01:34:24.160 --> 01:34:27.597
Yes. the AGI, the the artificial general

01:34:27.680 --> 01:34:30.157
intelligence. That's a frontier that may

01:34:30.240 --> 01:34:33.277
be dangerous, but all the rest we

01:34:33.360 --> 01:34:37.437
absorbed. We think nothing
of putting on GPS and

01:34:37.520 --> 01:34:39.837
having it tell us the shortest route to

01:34:39.920 --> 01:34:42.157
grandma's house, corrected for traffic,

01:34:42.240 --> 01:34:44.957
and there's no human in that loop. We're

01:34:45.040 --> 01:34:49.757
all fine with that.
We're not spooked by it. You're not

01:34:49.840 --> 01:34:51.917
writing to your congressman about it.

01:34:52.000 --> 01:34:53.757
You don't try to legislate it. It's just

01:34:53.840 --> 01:34:56.237
there in our lives.

01:34:56.320 --> 01:34:59.917
So I foresee by 2030, if I will, much

01:35:00.000 --> 01:35:04.637
more AI just folded into our lives.

01:35:04.720 --> 01:35:08.717
But the AI that transcends all of our

01:35:08.800 --> 01:35:11.357
intelligence, the one that you might

01:35:11.440 --> 01:35:14.077
think is farther away than it might be

01:35:14.160 --> 01:35:18.077
given the title of this book.

01:35:18.160 --> 01:35:20.397
>> Uh, I have to

01:35:20.480 --> 01:35:22.957
I have to agree with Nate. I'm old

01:35:23.040 --> 01:35:27.997
enough to have lived through a lot
of the Cold War. You know what kind of

01:35:28.080 --> 01:35:29.677
ended the Cold War? A lot of things

01:35:29.760 --> 01:35:31.757
contributed. But you know what mattered?

01:35:31.840 --> 01:35:35.757
The realization that if you launch

01:35:35.840 --> 01:35:38.077
nuclear weapons, intercontinental

01:35:38.160 --> 01:35:41.357
ballistic missiles from the Soviet Union

01:35:41.440 --> 01:35:44.317
into the Western Hemisphere and we see

01:35:44.400 --> 01:35:46.877
that and we retaliate,

01:35:46.960 --> 01:35:49.757
everybody dies.

01:35:49.840 --> 01:35:53.037
We had an acronym for it. Matt, mutual

01:35:53.120 --> 01:35:55.757
assured destruction. That's what brought

01:35:55.840 --> 01:35:57.837
people to the table. Said this is

01:35:57.920 --> 01:36:00.477
insane. We have to regulate it. We have

01:36:00.560 --> 01:36:05.277
to reduce the stockp because preserving
our species is what is what matters

01:36:05.360 --> 01:36:06.797
here.

01:36:06.880 --> 01:36:11.117
So I based on that evidence, I have some

01:36:11.200 --> 01:36:15.517
confidence that as those
dangers become more and

01:36:15.600 --> 01:36:17.837
more real,

01:36:17.920 --> 01:36:19.997
people will come to the table and say,

01:36:20.080 --> 01:36:23.037
"Yeah, keep the rest of the AI going, we

01:36:23.120 --> 01:36:27.837
got new medicines and new understandings
of our physiology and new technologies

01:36:27.920 --> 01:36:29.917
that help us get smarter and healthier."

01:36:30.000 --> 01:36:34.877
Yes, but that branch of AI is lethal. We

01:36:34.960 --> 01:36:37.677
have to do something about that. No one

01:36:37.760 --> 01:36:40.637
should build it and everyone needs to

01:36:40.720 --> 01:36:45.277
agree to that by treaty. Treaties are
not perfect, but they're the best we

01:36:45.360 --> 01:36:47.437
have as humans.

01:36:47.520 --> 01:36:51.117
I would end by saying there's a quote

01:36:51.200 --> 01:36:54.557
from Ray Bradberry who I verified with.

01:36:54.640 --> 01:36:58.397
I met him only once and I just have
to verify this quote. He said there's a

01:36:58.480 --> 01:37:02.797
woman once came up to me and said
science fiction author Ray Bradberry uh

01:37:02.880 --> 01:37:05.197
Mr. Bradberry, why do you write these

01:37:05.280 --> 01:37:08.797
stories about these apocalyptic futures?

01:37:08.880 --> 01:37:11.917
Is this the future you think humanity is

01:37:12.000 --> 01:37:16.797
going to inherit? He says, "No, I write

01:37:16.880 --> 01:37:19.917
these apocalyptic futures so you know to

01:37:20.000 --> 01:37:22.317
avoid them."

01:37:22.400 --> 01:37:25.517
That is a cosmic perspective. Join me in

01:37:25.600 --> 01:37:30.040
thanking our panel. Oh my god.

01:37:32.000 --> 01:37:36.680
Excuse me.

01:37:49.200 --> 01:37:54.557
We moved along, but as as always,
we allocate a few minutes for audience

01:37:54.640 --> 01:37:55.917
questions. So, if we can bring the

01:37:56.000 --> 01:38:01.920
lights up, we have a microphone
in each aisle. We'll try to go quickly