WEBVTT

00:00:00.000 --> 00:00:03.877
Let's start with science. >> Why
don't I start by saying how grateful

00:00:03.960 --> 00:00:08.440
I am to Princeton? Um

00:00:10.120 --> 00:00:14.717
Um for those of you that are
graduating, you don't know this,

00:00:14.800 --> 00:00:16.877
but this is the pivot point for every

00:00:16.960 --> 00:00:19.517
aspect of success in your life.

00:00:19.600 --> 00:00:22.077
And that was true of me.

00:00:22.160 --> 00:00:24.917
Princeton begat Berkeley, which then

00:00:25.000 --> 00:00:27.557
created Wendy and the family and the

00:00:27.640 --> 00:00:29.557
money and the opportunities and the so

00:00:29.640 --> 00:00:32.877
forth. Had I not come to Princeton, this

00:00:32.960 --> 00:00:35.597
would not have happened. It's binary.

00:00:35.680 --> 00:00:37.437
That's how important Princeton is to me.

00:00:37.520 --> 00:00:40.400
So, thank you.

00:00:45.240 --> 00:00:47.997
>> Well, I want to start
off with the science of AI.

00:00:48.080 --> 00:00:50.877
Um a lot of us that are

00:00:50.960 --> 00:00:53.477
ignorant or semi-ignorant in AI, unlike

00:00:53.560 --> 00:00:57.077
you, are wondering what happened to to

00:00:57.160 --> 00:00:59.237
account for this burst. I can I'm old

00:00:59.320 --> 00:01:02.077
enough to remember people in the '60s

00:01:02.160 --> 00:01:04.557
talking about AI is on the verge, in the

00:01:04.640 --> 00:01:07.957
'70s on the verge, in the '80s. Now it's

00:01:08.040 --> 00:01:11.557
here. It's really here. What happened?

00:01:11.640 --> 00:01:14.837
>> Um to give you a sense
of what's happening,

00:01:14.920 --> 00:01:18.917
uh this week the first
this may be one of the most

00:01:19.000 --> 00:01:20.797
historic moments in human history

00:01:20.880 --> 00:01:24.997
occurred. Um a proof of an 80-year-old

00:01:25.080 --> 00:01:28.997
uh theorem was generated by OpenAI,

00:01:29.080 --> 00:01:32.357
and the paper was written by the computer.

00:01:32.440 --> 00:01:36.597
The The proof is incomprehensible
to anyone here except the senior math

00:01:36.680 --> 00:01:39.797
people. Um so, I'm just
going to accept that the

00:01:39.880 --> 00:01:41.837
math people think it's correct.

00:01:41.920 --> 00:01:45.477
Um and they all think
it's really important.

00:01:45.560 --> 00:01:47.157
The most interesting thing about this

00:01:47.240 --> 00:01:52.960
proof is it was done with a model
that was not trained on math.

00:01:53.080 --> 00:01:55.077
So, we had a long meeting yesterday.

00:01:55.160 --> 00:01:58.997
What is What is the thing that it's doing?

00:01:59.080 --> 00:02:03.837
Right? So, one way to understand the
current LLM models is that they're very

00:02:03.920 --> 00:02:08.037
good at synthesizing a structure
of knowledge into a small vector

00:02:08.120 --> 00:02:12.517
representation space.
And these systems can look across that

00:02:12.600 --> 00:02:16.517
representation space, and they can
collect all these facts in ways that

00:02:16.600 --> 00:02:20.277
humans can't. Many people
believe that this is the

00:02:20.360 --> 00:02:23.757
beginning of artificial superintelligence.

00:02:23.840 --> 00:02:26.317
These are things that humans cannot do

00:02:26.400 --> 00:02:29.477
that computers can do that we value.

00:02:29.560 --> 00:02:32.757
So, then the question is how do
we get here? Well, when I was here,

00:02:32.840 --> 00:02:36.557
there was a nascent AI program.
Of course, there was no computer science.

00:02:36.640 --> 00:02:39.277
And it was all around what is called
symbolic reasoning. And the idea there

00:02:39.360 --> 00:02:43.317
was that you could build a language that
would describe knowledge. Right? This is

00:02:43.400 --> 00:02:49.037
a sort of a Spinoza model of of knowledge.
And many, many people worked

00:02:49.120 --> 00:02:51.597
on that, and it turned out that it failed.

00:02:51.680 --> 00:02:54.957
And after that, and it was a hard problem.
So, there was something called

00:02:55.040 --> 00:02:58.237
the AI winter in the '80s and '90s. In

00:02:58.320 --> 00:03:02.757
the 1980s, at the Institute for Advanced
Studies, a bunch of mathematicians

00:03:02.840 --> 00:03:07.317
proved a theorem that I also
cannot describe, which describes

00:03:07.400 --> 00:03:10.917
essentially deep learning as we know
it today, which is a series of layers of

00:03:11.000 --> 00:03:12.997
networks that talk to each other.

00:03:13.080 --> 00:03:16.477
And then a bunch of scientists at various
universities, including here, in

00:03:16.560 --> 00:03:21.197
the early 2000s, built the model,
the mathematical model.

00:03:21.280 --> 00:03:24.937
So, it started with math. Then, right?

00:03:25.020 --> 00:03:27.237
>> [snorts] >> It wasn't
until about 2011 when a bunch

00:03:27.320 --> 00:03:31.557
of people at Google decided this would
be interesting to apply to YouTube. And

00:03:31.640 --> 00:03:35.397
they watched YouTube, and they
discovered something. They discovered

00:03:35.480 --> 00:03:39.237
the concept of a cat.
And I remember it sitting at Google at

00:03:39.320 --> 00:03:42.557
2011 saying, like, "You're watching
YouTube, and you discover the concept of

00:03:42.640 --> 00:03:46.037
a cat? Isn't there something more
interesting on YouTube?" And the answer

00:03:46.120 --> 00:03:50.077
turned out that during that period,
there was an awful lot of cat pictures

00:03:50.160 --> 00:03:52.917
and movies and so on. People were
uploading pictures of their cats, and

00:03:53.000 --> 00:03:56.757
that's how it discovered it.
From that moment, there were a series of

00:03:56.840 --> 00:04:01.077
innovations, and the most important
ones were in 2015, something called

00:04:01.160 --> 00:04:03.437
reinforcement learning

00:04:03.520 --> 00:04:06.517
was used by DeepMind to win a game

00:04:06.600 --> 00:04:08.197
called AlphaGo, which was thought to be

00:04:08.280 --> 00:04:10.557
incomputable. That was a big deal, and I

00:04:10.640 --> 00:04:14.477
know cuz I was there. Um 2017, Google

00:04:14.560 --> 00:04:18.157
published a paper called Attention
Is All You Need, which showed

00:04:18.240 --> 00:04:20.237
algorithmically how you could take the

00:04:20.320 --> 00:04:23.957
math and do it in parallel on computers.

00:04:24.040 --> 00:04:28.677
That single paper spawned what you
know of today, and then that created the

00:04:28.760 --> 00:04:32.917
competition, which then moved very,
very quickly. Cuz now you've got money and

00:04:33.000 --> 00:04:36.237
people, and you've got a computer
architecture and a awful lot of

00:04:36.320 --> 00:04:39.277
computing, and that's how it happened.

00:04:39.360 --> 00:04:44.037
>> It's an amazing story. I didn't hear
you mention the huge increase in computing

00:04:44.120 --> 00:04:48.317
power. Was that not
the big deal? Or what were

00:04:48.400 --> 00:04:51.877
the big deals here?
>> The computing power is essential, but

00:04:51.960 --> 00:04:56.797
that's true of my whole industry.
So here's an example. When I was here, um

00:04:56.880 --> 00:05:00.957
Tom Wyant is here. I can see a number of
other people that I did this with. So we

00:05:01.040 --> 00:05:05.197
all remember what we were when we were
much younger. Um we spent an awful lot

00:05:05.280 --> 00:05:09.917
of time on something called an IBM 360-91,
which is at 87 Prospect, and the

00:05:10.000 --> 00:05:13.797
old-timers here know what I'm talking
about. And this thing was a work of art.

00:05:13.880 --> 00:05:16.797
It had a raised floor. It had a

00:05:16.880 --> 00:05:18.317
window cuz they didn't want you near the

00:05:18.400 --> 00:05:22.637
computer. Each of these boxes were about

00:05:22.720 --> 00:05:24.717
1 megabit.

00:05:24.800 --> 00:05:28.157
1 megabit. And it was all water-cooled,

00:05:28.240 --> 00:05:30.117
and they used something called iron core

00:05:30.200 --> 00:05:31.877
cuz semiconductor memory had not been

00:05:31.960 --> 00:05:35.917
invented yet. I personally, or we, owned

00:05:36.000 --> 00:05:38.957
the console for that 360-91 cuz the rest

00:05:39.040 --> 00:05:41.357
of it was too big to put in the office.

00:05:41.440 --> 00:05:43.477
So that's how important that computer

00:05:43.560 --> 00:05:45.917
was. Uh and we used fed it with punch

00:05:46.000 --> 00:05:50.877
cards. The punchline is the phone you
have in your pocket today is at least a

00:05:50.960 --> 00:05:53.837
hundred million times more powerful
than the whole computer that was on the

00:05:53.920 --> 00:05:56.997
campus in 1975.

00:05:57.080 --> 00:06:01.517
Um, and furthermore, the 360-91, there
were only three, and they had special

00:06:01.600 --> 00:06:04.717
floating point because they were
used for the NASA moon landing.

00:06:04.800 --> 00:06:08.957
So, that's an example of how
the industry changed in this period. Now,

00:06:09.040 --> 00:06:10.317
how do I know a hundred million times?

00:06:10.400 --> 00:06:13.277
That's the GFT the GFTL number

00:06:13.360 --> 00:06:17.957
um from their calculations
of computation in the campus.

00:06:18.040 --> 00:06:22.197
>> So, is there something analogous
with this is an analogy, it

00:06:22.280 --> 00:06:25.437
wouldn't be the speed of light, but an
analogous to the speed of light that's a

00:06:25.520 --> 00:06:30.397
limit on how fast this can go? Most
of us are shaking our heads in wonderment

00:06:30.480 --> 00:06:34.357
about how how much
this is changing so fast.

00:06:34.440 --> 00:06:38.477
>> Well, first, it's happening much too
fast for everybody, including me. Um,

00:06:38.560 --> 00:06:41.237
I think the following statements are true.

00:06:41.320 --> 00:06:43.797
Um, computers can now compete with

00:06:43.880 --> 00:06:48.757
top-level graduate students in basic
scientists ba- basic sciences, and

00:06:48.840 --> 00:06:50.597
almost certainly in writing and

00:06:50.680 --> 00:06:52.997
analysis, and perhaps in art.

00:06:53.080 --> 00:06:55.997
Um, how much smarter can they go? Um,

00:06:56.080 --> 00:06:59.117
most of us believe that they will they,

00:06:59.200 --> 00:07:00.837
whatever you want to call them, will in

00:07:00.920 --> 00:07:04.037
our lifetimes be materially smarter in

00:07:04.120 --> 00:07:06.437
some ways over the humans.

00:07:06.520 --> 00:07:11.077
They are not going to have the same kind
of intelligence as humans, and lots of

00:07:11.160 --> 00:07:13.997
people have been studying
what this thing looks like,

00:07:14.080 --> 00:07:16.837
and it's not human. Don't be confused.

00:07:16.920 --> 00:07:21.637
Furthermore, they they make a kind
of mistake that's very interesting.

00:07:21.720 --> 00:07:25.837
They're trained on knowledge that we
give them, but there's an awful lot of

00:07:25.920 --> 00:07:29.917
knowledge that's not written down
anywhere. So, I'll give you an example.

00:07:30.000 --> 00:07:33.277
I'm in a business meeting, and somebody
comes up with an idea of doing a rev

00:07:33.360 --> 00:07:36.157
share, and another person
says, "What we should

00:07:36.240 --> 00:07:41.277
do is do a joint venture." And I said,
confidently in my CEO voice, "Joint

00:07:41.360 --> 00:07:44.357
ventures don't work,
and rev shares don't work.

00:07:44.440 --> 00:07:47.077
Now, I've never said that publicly.

00:07:47.160 --> 00:07:49.877
So, how would the computer that's
impersonating me know that? He would

00:07:49.960 --> 00:07:54.037
have to go through lots of failures,
lots of disasters, lots of, you know,

00:07:54.120 --> 00:07:56.917
problems. Furthermore,
these kinds of rules are

00:07:57.000 --> 00:07:59.157
not written down in business books.

00:07:59.240 --> 00:08:03.037
So, that's I'm using that as a trivial
example. There are thousands and

00:08:03.120 --> 00:08:05.997
thousands and thousands of pieces
of knowledge that you all have in your

00:08:06.080 --> 00:08:08.677
heads that these things don't know and

00:08:08.760 --> 00:08:11.117
will not or are unlikely to know.

00:08:11.200 --> 00:08:14.117
Um I'm funding some research in the

00:08:14.200 --> 00:08:18.437
following. What's the limit?
Is the limit is this kind of analytical

00:08:18.520 --> 00:08:22.597
superintelligence without wisdom,
if you will? And by the way, it's really

00:08:22.680 --> 00:08:26.397
important because Princeton and, you know,
your successors in the faculty and

00:08:26.480 --> 00:08:28.797
so forth in economics
going to use this every day.

00:08:28.880 --> 00:08:31.877
>> Wisdom is tough. >> Wisdom is tough.
Wisdom is really tough.

00:08:31.960 --> 00:08:33.437
It's hard to get.

00:08:33.520 --> 00:08:36.397
Um and so, what happens is there's

00:08:36.480 --> 00:08:39.117
research on how do we imbue wisdom and

00:08:39.200 --> 00:08:43.797
non- non-human knowledge into
these things. The best ideas are

00:08:43.880 --> 00:08:47.677
unfortunately, you have to teach them
and humans have to teach them. So, you

00:08:47.760 --> 00:08:50.997
have to figure out a way to do
the the technical term is get the training

00:08:51.080 --> 00:08:54.077
data for wisdom.
No one's solved that problem yet.

00:08:54.160 --> 00:08:55.917
>> But go ahead. Please go ahead.

00:08:56.000 --> 00:09:00.557
>> But I I want to make sure I'm I'm clear
before everyone talks about the negatives.

00:09:00.640 --> 00:09:04.637
Um in the next 15 years, there are
people in my industry who believe and

00:09:04.720 --> 00:09:07.997
are funding that we believe
we can solve essentially

00:09:08.080 --> 00:09:12.957
all human diseases. Now,
that would be like a big deal, right?

00:09:13.040 --> 00:09:15.477
And that would be like a big deal,
right? I mean, let's just admit. That's

00:09:15.560 --> 00:09:20.117
like we should be happy with that.
Um it's highly likely that you'll see new

00:09:20.200 --> 00:09:23.757
energy solutions that will help address
uh the terrible things involving climate

00:09:23.840 --> 00:09:26.157
change. All right. So,
you start going through

00:09:26.240 --> 00:09:28.557
problems, new materials, uh new

00:09:28.640 --> 00:09:34.357
chemistry, um so forth and so on. All
of those things are within our grasp. And

00:09:34.440 --> 00:09:38.557
part of the reason that I think it's
so important to celebrate this now is

00:09:38.640 --> 00:09:40.317
a thousand years from now people will

00:09:40.400 --> 00:09:42.997
record this period

00:09:43.080 --> 00:09:45.837
as an epochal change. This is what

00:09:45.920 --> 00:09:50.037
Henry's Henry's Henry book Henry's
Henry Kissinger book with me helping him

00:09:50.120 --> 00:09:52.237
basically said. This is like the

00:09:52.320 --> 00:09:53.757
enlightenment

00:09:53.840 --> 00:09:57.757
at that level because of the scale

00:09:57.840 --> 00:10:00.437
of what it can touch. Every aspect of

00:10:00.520 --> 00:10:05.037
human progress has been basically
because of science in one form,

00:10:05.120 --> 00:10:08.597
invention in one form, technology
in one form starting with the agrarian

00:10:08.680 --> 00:10:10.957
revolution. This is the next one and

00:10:11.040 --> 00:10:15.717
it's moving very fast.
>> And will we to take on something that's

00:10:15.800 --> 00:10:19.917
controversial these days,
will we need 10,000 times more

00:10:20.000 --> 00:10:25.037
electricity and data centers and things
like that? I noted the other day and I

00:10:25.120 --> 00:10:30.837
went like this, Elon Musk is talking
about putting data centers on the moon.

00:10:30.920 --> 00:10:35.117
Uh, which seems have we
really run out of space on

00:10:35.200 --> 00:10:39.437
Earth? >> We have lots of space on Earth.

00:10:39.520 --> 00:10:41.197
Um, thank goodness.

00:10:41.280 --> 00:10:42.957
Without answering the Elon questions, I

00:10:43.040 --> 00:10:46.477
prefer not to answer rebut rebut Elon.

00:10:46.560 --> 00:10:48.800
Uh,

00:10:49.760 --> 00:10:53.437
so so so let's let's talk
a little bit about data centers.

00:10:53.520 --> 00:10:59.117
Um, my so to some degree I'm in mourning.

00:10:59.200 --> 00:11:02.357
Uh, when I started Princeton, um,

00:11:02.440 --> 00:11:07.397
and we'll talk about this again,
I showed up uh as a young freshman and I

00:11:07.480 --> 00:11:10.997
met this professor named Brian Kernighan
and a whole bunch of other people and

00:11:11.080 --> 00:11:13.717
they basically said you're a computer
scientist, there's no computer science,

00:11:13.800 --> 00:11:17.037
we'll just work on we'll
we'll just fake it, right?

00:11:17.120 --> 00:11:21.517
That was the beginning of modern
computer science. As I learned computer

00:11:21.600 --> 00:11:23.957
science, it is over.

00:11:24.040 --> 00:11:27.477
When I watch these things write code, I

00:11:27.560 --> 00:11:29.757
go, holy

00:11:29.840 --> 00:11:31.917
I mean, I was good, especially as a

00:11:32.000 --> 00:11:38.080
21-year-old and my my ego
was consistent with my attitude.

00:11:38.200 --> 00:11:42.317
I was really good. These things
are so much better than I ever was.

00:11:42.400 --> 00:11:45.477
That's a big deal.
Because it empowers people to do the

00:11:45.560 --> 00:11:48.797
kind of scaling that I've
done in the last 50 years.

00:11:48.880 --> 00:11:53.037
Now, what happens with data centers
goes something like this. First,

00:11:53.120 --> 00:11:57.197
the White House and the government
should be very happy because the data

00:11:57.280 --> 00:11:59.797
center build-out has added 1% of the GDP

00:11:59.880 --> 00:12:03.437
growth to America. That's a big number.

00:12:03.520 --> 00:12:05.717
And it's also changed my industry from

00:12:05.800 --> 00:12:10.837
being high gross margin, low capital
cost to being very high capital cost.

00:12:10.920 --> 00:12:12.157
So, now what you're seeing is this

00:12:12.240 --> 00:12:17.957
enormous funding of data centers and the
numbers are incredible. Give you some I

00:12:18.040 --> 00:12:21.197
happen to have a data center business,
so I know the numbers now. 1 gigawatt of

00:12:21.280 --> 00:12:24.277
power is about 50 billion dollars.

00:12:24.360 --> 00:12:27.997
The The industry and I've testified
in Congress to this, the industry estimate

00:12:28.080 --> 00:12:30.637
is it needs somewhere between 90 and 100

00:12:30.720 --> 00:12:33.957
gigawatts of power by 2030. For

00:12:34.040 --> 00:12:38.237
reference, the average nuclear reactor,
which we don't make in America largely,

00:12:38.320 --> 00:12:41.757
is 1 gigawatt. All right,
that gives you a sense of the

00:12:41.840 --> 00:12:45.477
scale of the problem.
There are plenty of people who are

00:12:45.560 --> 00:12:48.157
working on reducing the power costs.

00:12:48.240 --> 00:12:51.237
There's something called Jevons
paradox where when you lower the cost of

00:12:51.320 --> 00:12:53.197
something, you just use more.

00:12:53.280 --> 00:12:56.197
So, in my community, even though all of

00:12:56.280 --> 00:13:01.397
those all of that work of efficiency and
lower cost is welcome, we still believe,

00:13:01.480 --> 00:13:04.517
whether we're right or not,
we still believe this is going to happen.

00:13:04.600 --> 00:13:08.437
Why is this important?
What's the limit? Well, data centers

00:13:08.520 --> 00:13:12.277
should not be near cities
because they annoy people and that

00:13:12.360 --> 00:13:15.397
that's not fair to them.
They shouldn't use a lot of water. They

00:13:15.480 --> 00:13:17.317
don't now as much. They need to be in

00:13:17.400 --> 00:13:20.397
remote spaces. What's true of America?

00:13:20.480 --> 00:13:22.677
Lots of empty space. Do you know why?

00:13:22.760 --> 00:13:26.597
You fly over it. Just look down.

00:13:26.680 --> 00:13:30.477
You will see it.
There's large large plots. It's a great

00:13:30.560 --> 00:13:34.877
There's an old De Tocqueville quote
about America that America is successful

00:13:34.960 --> 00:13:39.837
because of the the abundance
of land, the richness of

00:13:39.920 --> 00:13:43.077
the spirit of its people,
and the absence of a king.

00:13:43.160 --> 00:13:45.317
This was 1750.

00:13:45.400 --> 00:13:47.877
Right? So, think about it.

00:13:47.960 --> 00:13:51.957
So, so that land bank, if you will,

00:13:52.040 --> 00:13:54.317
plus the American financial system

00:13:54.400 --> 00:13:58.997
allows us to do crazy things. I mean,
I was talking to this guy who says

00:13:59.080 --> 00:14:03.677
he's got 15 people. He's raising at a 4
billion valuation, and he's raising $400

00:14:03.760 --> 00:14:09.240
million today. He's only
The company's only 6 weeks old.

00:14:10.040 --> 00:14:14.037
He's already oversubscribed.
Thank you. I want to thank everyone here

00:14:14.120 --> 00:14:18.277
who works in Wall Street, who's invented
this system. There's no place in the

00:14:18.360 --> 00:14:22.237
world that can give crazy
people this amount of money.

00:14:22.320 --> 00:14:26.757
Right? And and furthermore,
some of the smarter

00:14:26.840 --> 00:14:30.197
of you are beginning to write articles
saying like, "Will we ever make any

00:14:30.280 --> 00:14:33.637
money?" And the answer is,
"Hell if I know."

00:14:33.720 --> 00:14:38.317
Right? But, you know, you're
the guys who gave us the money.

00:14:38.400 --> 00:14:42.877
So, it's okay. We'll try.

00:14:42.960 --> 00:14:47.037
So, in any case,
to answer your real question,

00:14:47.120 --> 00:14:51.677
the data center limit is actually
not physical. It's financial.

00:14:51.760 --> 00:14:56.557
You can build the data centers with your
own power systems. So, all of the new

00:14:56.640 --> 00:14:59.957
data centers are essentially The term is
called behind the meter. They're off the

00:15:00.040 --> 00:15:02.637
grid cuz the grid can't support it.

00:15:02.720 --> 00:15:06.077
And it's not fair to people
to raise their electricity rates. The

00:15:06.160 --> 00:15:09.517
interconnection queues, as they're
called, are now full for 7 years. So,

00:15:09.600 --> 00:15:13.597
all of the data centers that you hear
about will be off the grid. There is an

00:15:13.680 --> 00:15:17.317
issue with the climate
impact of these things cuz

00:15:17.400 --> 00:15:19.277
these are combined cycle turbines.

00:15:19.360 --> 00:15:21.797
Everybody understands
that that's not good.

00:15:21.880 --> 00:15:24.357
And the And the best solution we've come

00:15:24.440 --> 00:15:28.837
up with is to build fully renewable data
centers, which is which we're working

00:15:28.920 --> 00:15:30.557
on, which is hard but possible.

00:15:30.640 --> 00:15:33.237
>> But don't they have to have
an ultimate energy source?

00:15:33.320 --> 00:15:35.357
>> Yeah, you want >>
Electricity doesn't just flow in.

00:15:35.440 --> 00:15:38.517
>> Well, that's right. You use either
natural gas with a combined cycle

00:15:38.600 --> 00:15:43.557
turbine, which are which you can purchase,
and these things last for 20 years.

00:15:43.640 --> 00:15:46.117
And you can also use solar and batteries.

00:15:46.200 --> 00:15:49.677
>> Is solar practicable for something
that's an enormous energy user?

00:15:49.760 --> 00:15:52.997
>> If you have enough land,
look out the window.

00:15:53.080 --> 00:15:57.597
Um right? Think Texas,
New Mexico, Arizona, all

00:15:57.680 --> 00:16:01.597
those nice big empty places.
>> investment strategy is to buy land in

00:16:01.680 --> 00:16:06.797
Nebraska right now.
>> Um I recommend buying all basically

00:16:06.880 --> 00:16:10.597
in places where you can get these things
permitted. This is the new If you're a

00:16:10.680 --> 00:16:13.997
real estate person, this is where all
the growth is going to be. If you look,

00:16:14.080 --> 00:16:18.717
all the real estate capital has moved
from normal building to data centers.

00:16:18.800 --> 00:16:23.200
That's a mistake, by the way.
We need normal buildings, too.

00:16:24.600 --> 00:16:28.037
>> Does any of this call in your mind for

00:16:28.120 --> 00:16:30.037
for sensible, and that of course is a

00:16:30.120 --> 00:16:35.997
big word in this sentence, regulation
of the industry? Let me give you two

00:16:36.080 --> 00:16:39.917
uh analogies.
There was a lot of discussion in the

00:16:40.000 --> 00:16:44.637
early days of the internet about
regulating and not regulating it. And we

00:16:44.720 --> 00:16:48.837
basically opted for let a thousand
flowers bloom and let it just go where

00:16:48.920 --> 00:16:51.077
it goes.

00:16:51.160 --> 00:16:53.917
Same And And that's widely looked at, I

00:16:54.000 --> 00:16:58.037
believe, as a sensible

00:16:58.120 --> 00:17:00.917
resolution to what looked like a

00:17:01.000 --> 00:17:05.677
difficult problem.
When it came to social media, we adopted

00:17:05.760 --> 00:17:10.357
the same strategy.
And I think, and it's not just

00:17:10.440 --> 00:17:14.757
an old codger like me, I've talked
to some of our students about this,

00:17:14.840 --> 00:17:18.117
there's at least a reasonable case
there's more bad than good that has come

00:17:18.200 --> 00:17:21.317
out of that decision. So,
you have one on one side,

00:17:21.400 --> 00:17:24.437
one on the other side.
What do you think about that?

00:17:24.520 --> 00:17:28.037
>> So, I've come to the conclusion that

00:17:28.120 --> 00:17:31.637
this all of this is going
to get regulated.

00:17:31.720 --> 00:17:35.597
And it's going to get regulated in
different ways for different reasons. Um

00:17:35.680 --> 00:17:40.677
there's no question that the there's
a misalignment of societal interest and

00:17:40.760 --> 00:17:43.957
business interest. And it goes like this,
the best way to maximize revenue

00:17:44.040 --> 00:17:46.477
is to maximize outrage. The best the

00:17:46.560 --> 00:17:51.157
best way to max because that that
essentially causes people to participate

00:17:51.240 --> 00:17:55.277
more. And the those businesses are
in a business to fight for all of your

00:17:55.360 --> 00:17:58.277
attention. So,
they're winning while you're

00:17:58.360 --> 00:18:02.517
obsessed with somebody
something and unhappy.

00:18:02.600 --> 00:18:04.957
That's a simple way of understanding it.

00:18:05.040 --> 00:18:06.517
And if you get confused about this

00:18:06.600 --> 00:18:10.637
point, watch any movie from the 90s.

00:18:10.720 --> 00:18:13.157
Any you like a normal movie like how men

00:18:13.240 --> 00:18:15.317
and women work together and and how

00:18:15.400 --> 00:18:17.957
politics works and so forth. And you'll

00:18:18.040 --> 00:18:22.357
you'll remember, right? You know,
the you're in your 25th your

00:18:22.440 --> 00:18:24.257
reunion. You know what I'm talking about.

00:18:24.340 --> 00:18:28.157
>> [snorts] >> Um and I
mourn that simply because of

00:18:28.240 --> 00:18:33.157
the nature of the conversation, the
quality of the morality that we operate.

00:18:33.240 --> 00:18:36.517
Because a system where the incentives
for communications are around outrage

00:18:36.600 --> 00:18:40.317
produces outrage on all sides. That's
a simple way of understanding what has

00:18:40.400 --> 00:18:43.477
happened. Now, I worked
really hard with a whole

00:18:43.560 --> 00:18:45.597
bunch of people including Jonathan Haidt

00:18:45.680 --> 00:18:48.597
on how to solve this problem. It's very

00:18:48.680 --> 00:18:50.877
hard to solve it without putting

00:18:50.960 --> 00:18:53.357
restrictions on speech.

00:18:53.440 --> 00:18:55.917
So, there is a fundamental problem of

00:18:56.000 --> 00:18:58.397
democracies in these technologies, which

00:18:58.480 --> 00:19:02.997
is that they can be used to to violate
or to promote particular non-democratic

00:19:03.080 --> 00:19:06.157
norms um for business
interests, right? This

00:19:06.240 --> 00:19:08.757
is true of all sides. I'm not trying
to make a political statement. I'm trying

00:19:08.840 --> 00:19:12.317
to say it's just true.
And that has to be addressed.

00:19:12.400 --> 00:19:15.837
Now, if you look around the world,
countries [clears throat] have very

00:19:15.920 --> 00:19:19.957
different attitudes towards freedom of
speech. We used to have We used to be at

00:19:20.040 --> 00:19:22.957
one of the extreme.
I'm not sure we're still at the

00:19:23.040 --> 00:19:26.717
extreme. That's another story that I'm
not going to ask you about. But we used

00:19:26.800 --> 00:19:30.877
to be at one extreme. China,
Russia, and a number of are at a

00:19:30.960 --> 00:19:33.197
different extreme.

00:19:33.280 --> 00:19:36.717
And if restricting free speech is part

00:19:36.800 --> 00:19:40.597
of the recipe,
does that actually give them

00:19:40.680 --> 00:19:44.317
paradoxically a leg up
rather than a leg down?

00:19:44.400 --> 00:19:48.117
>> Well, we know a lot about how China
operates because China is our natural

00:19:48.200 --> 00:19:51.197
competitor. Notice I
didn't use the word enemy

00:19:51.280 --> 00:19:56.277
uh in this space. And they solved
this problem by dictatorial regulation,

00:19:56.360 --> 00:20:00.197
right? And it's brutal.
Uh but that's what their government

00:20:00.280 --> 00:20:03.357
chose, which is obviously not a democracy.
So, they solved that problem.

00:20:03.440 --> 00:20:05.677
But we're clearly not going to do that.

00:20:05.760 --> 00:20:08.277
>> Is that one of the reasons
we're ahead of them now?

00:20:08.360 --> 00:20:11.957
>> No, we're we're ahead of uh
China for a number of reasons. The

00:20:12.040 --> 00:20:13.877
most important of which,

00:20:13.960 --> 00:20:17.597
the US government uh under Trump 1 and

00:20:17.680 --> 00:20:22.717
Biden and Trump 2 have all restricted
sales of hardware chips to China. These

00:20:22.800 --> 00:20:25.157
are There's a particular threshold.

00:20:25.240 --> 00:20:27.357
>> Mhm. >> Um and the threshold involves a

00:20:27.440 --> 00:20:29.317
particular manufacturing technique and a

00:20:29.400 --> 00:20:32.237
particular manufacturer called ASML.

00:20:32.320 --> 00:20:36.557
And I had thought, cuz I was one of
the authors of this program however many

00:20:36.640 --> 00:20:41.677
years ago in the AI Commission,
that it would give us a 5 to 10-year uh

00:20:41.760 --> 00:20:45.517
advantage. And that was
my technical calculation.

00:20:45.600 --> 00:20:48.477
And of course, I was wrong again. Um it

00:20:48.560 --> 00:20:53.237
looks like China's within 6 months of us
now based on the Deep Seek V4 result.

00:20:53.320 --> 00:20:57.437
And you can read that. Just look up
the evaluations of Deep Seek V4. Chinese

00:20:57.520 --> 00:21:01.997
people are very smart. They work really
hard. Their work ethic is incredible.

00:21:02.080 --> 00:21:04.037
And even with reduced resources, they've

00:21:04.120 --> 00:21:05.837
been able to get caught up.

00:21:05.920 --> 00:21:09.197
The President Xi and their AI program is

00:21:09.280 --> 00:21:15.237
slightly different than ours. Um the
their goal is to imbue AI in everything.

00:21:15.320 --> 00:21:16.957
And again, that's subject [clears
throat] to their laws, which

00:21:17.040 --> 00:21:18.757
include the draconian free speech

00:21:18.840 --> 00:21:21.997
issues. So, everything has AI in it now,

00:21:22.080 --> 00:21:24.277
whereas that's less so in America.

00:21:24.360 --> 00:21:27.637
Furthermore, in China, there's huge
support for AI, and as you know, in the

00:21:27.720 --> 00:21:32.037
US, there's huge non-support for AI
at the moment politically. So, but but the

00:21:32.120 --> 00:21:34.117
answer about China is one way to

00:21:34.200 --> 00:21:37.037
understand the evolution of AI is that

00:21:37.120 --> 00:21:41.397
you need a large country full of an awful
lot of the things that the US has.

00:21:41.480 --> 00:21:43.957
You need the money. You need
the creativity. You need the hardware. You

00:21:44.040 --> 00:21:47.397
need the software. You need the audience
base. You need the monetization. China

00:21:47.480 --> 00:21:51.557
has that. Very few other
countries have the scale

00:21:51.640 --> 00:21:55.677
that is required. Uh if Europe ever
got its act together and operated as a

00:21:55.760 --> 00:21:58.997
single country, they could pull it off,
but they don't do that. India's a

00:21:59.080 --> 00:22:04.280
possibility. Most of the other
countries cannot get there.

00:22:04.400 --> 00:22:09.517
>> In many economics lectures in this room
and other rooms, we teach our students

00:22:09.600 --> 00:22:15.277
that especially when it comes
to innovation, capitalism has a gigantic

00:22:15.360 --> 00:22:19.997
uh advantage over state intervention.

00:22:20.080 --> 00:22:24.037
Is Are we looking at something
that could be an exception to that? I'm

00:22:24.120 --> 00:22:27.157
playing off what you're
saying about China.

00:22:27.240 --> 00:22:29.237
>> Well, it depends
on whether you ignoring

00:22:29.320 --> 00:22:32.957
the political comment, China when
when I'm in China, it feels very

00:22:33.040 --> 00:22:34.717
capitalistic.

00:22:34.800 --> 00:22:39.717
Um they they refer to themselves as um

00:22:39.800 --> 00:22:44.117
communists with Chinese characteristics,
which I conclude is capitalism.

00:22:44.200 --> 00:22:48.077
So, the the people
that I work with are just

00:22:48.160 --> 00:22:51.597
like the American
entrepreneurs that I work with.

00:22:51.680 --> 00:22:56.357
They're crazy hard working. They're
intense. They're very ambitious, very

00:22:56.440 --> 00:22:58.037
self-confident, you know, all that kind

00:22:58.120 --> 00:23:00.797
of stuff. Uh and they work in a a system

00:23:00.880 --> 00:23:03.397
where they've largely been left alone, I

00:23:03.480 --> 00:23:06.117
think that that's likely to end.

00:23:06.200 --> 00:23:08.317
Um give you an example, China just

00:23:08.400 --> 00:23:10.877
announced a restriction on DD, which is

00:23:10.960 --> 00:23:13.797
their Uber equivalent, to not allow them

00:23:13.880 --> 00:23:16.637
to do self-driving cars because of the

00:23:16.720 --> 00:23:18.957
obvious impact on employment.

00:23:19.040 --> 00:23:22.397
So again, one of the problems with an

00:23:22.480 --> 00:23:26.837
authoritarian structure is you can
never quite figure out which priority is

00:23:26.920 --> 00:23:29.677
right, whereas at least in the market
you can predict it. I'm I'm obviously a

00:23:29.760 --> 00:23:32.997
capitalist. So the fact
and I had dinner with a

00:23:33.080 --> 00:23:36.637
number of my Chinese friends and they
were in their usual, you know, this and

00:23:36.720 --> 00:23:38.317
that and the complaining and so forth. I

00:23:38.400 --> 00:23:41.397
said, "Look, read the rhetoric from your

00:23:41.480 --> 00:23:44.357
government. They're not capitalist."

00:23:44.440 --> 00:23:48.197
You just read the documents.
So I think that my friends, maybe they

00:23:48.280 --> 00:23:52.957
were caught in some capitalist bubble,
but they sure made an awful lot of

00:23:53.040 --> 00:23:55.277
money. All right.
And now they're trying to get

00:23:55.360 --> 00:23:58.157
it out of the country.

00:23:58.240 --> 00:24:02.037
>> Which which is not so easy.

00:24:02.120 --> 00:24:04.437
>> Yeah, so to be clear, I I much prefer

00:24:04.520 --> 00:24:09.157
American free speech, freedom, and
capitalism for obvious reasons. I think

00:24:09.240 --> 00:24:11.797
it created the success we see in the room.

00:24:11.880 --> 00:24:15.677
Uh on the free speech question, I did
want to say that I tried to I'm in favor

00:24:15.760 --> 00:24:18.677
of free speech of humans,
including crazy people,

00:24:18.760 --> 00:24:21.917
of which there seem to be
an increasing number every day.

00:24:22.000 --> 00:24:24.957
Um I am not in favor of boosted speech

00:24:25.040 --> 00:24:28.997
having the same rules.
So a simple way of thinking about this

00:24:29.080 --> 00:24:32.757
is um here at Princeton we have all
sorts of people. We do we have tolerance

00:24:32.840 --> 00:24:36.077
for different ideas. I'm in favor
of all of those ideas even if I and it's

00:24:36.160 --> 00:24:39.877
perfectly fine for someone to sit uh
in the equivalent of McCosh Courtyard and

00:24:39.960 --> 00:24:44.357
scream that for a while. Um when they
get a bullhorn and then they start

00:24:44.440 --> 00:24:48.557
spreading it through all the dorms from
that single bullhorn, that's a violation

00:24:48.640 --> 00:24:50.757
of somebody else's rights. So there is

00:24:50.840 --> 00:24:54.517
some limit to amplification

00:24:54.600 --> 00:24:58.117
of one's message.
That's the way to phrase it.

00:24:58.200 --> 00:24:59.917
I'm not a a legal expert, but there's

00:25:00.000 --> 00:25:03.797
got to be a way to say it right.

00:25:03.880 --> 00:25:07.557
>> We hope so.
Let me ask you a slightly related

00:25:07.640 --> 00:25:11.477
question, maybe less a little less cosmic.

00:25:11.560 --> 00:25:13.397
As I read

00:25:13.480 --> 00:25:17.557
it with my inexpert eyes and ears about

00:25:17.640 --> 00:25:19.597
China versus the US,

00:25:19.680 --> 00:25:24.717
it seems like China's going much more
for closed source, and we're going much

00:25:24.800 --> 00:25:27.077
more for open source. >> Inverse.

00:25:27.160 --> 00:25:30.237
>> in >> The inverse. >> Okay.

00:25:30.320 --> 00:25:32.757
I said it the backwards, sorry.

00:25:32.840 --> 00:25:38.920
So, the question was to be, is that true?
And with the correction, it's true.

00:25:41.560 --> 00:25:45.877
And are And are we on the Are
we on the right side of it?

00:25:45.960 --> 00:25:48.597
>> Um I think not. >> You think not?

00:25:48.680 --> 00:25:51.917
>> So, so let me give a For those of you
who haven't been following, this is a

00:25:52.000 --> 00:25:55.797
really important distinction. Um

00:25:55.880 --> 00:25:58.277
the top models, the ones that you use,

00:25:58.360 --> 00:26:01.957
are all run by for-profit companies.

00:26:02.040 --> 00:26:05.517
And they have lawyers and CEOs,
and they have shareholders or will have

00:26:05.600 --> 00:26:08.477
shareholders. And they
take an enormous amount of

00:26:08.560 --> 00:26:12.557
capital. We're talking
about uh hundreds of

00:26:12.640 --> 00:26:16.197
literally, well, hundreds
of billions of dollars at this rate.

00:26:16.280 --> 00:26:20.197
Now, they properly want an economic
return, and so what they do is they keep

00:26:20.280 --> 00:26:23.677
their software their weight
Technically, the weights

00:26:23.760 --> 00:26:27.237
and the software proprietary.
You can't borrow it. They

00:26:27.320 --> 00:26:30.477
don't give it to you.
You can't get access to it.

00:26:30.560 --> 00:26:33.357
Um why is this so important? The entire

00:26:33.440 --> 00:26:37.957
industry that I was part of, which
started here at Princeton, by the way,

00:26:38.040 --> 00:26:42.597
was based on an the other side principle,
the open source principle. We

00:26:42.680 --> 00:26:45.357
will write software, and we'll give it
to everyone. And the lawyers all would

00:26:45.440 --> 00:26:49.437
go, "You're giving away your
intellectual property?" I said, "Yes."

00:26:49.520 --> 00:26:53.957
And the reason is that in the same
principle as if you want to go fast, go

00:26:54.040 --> 00:26:58.157
alone. If you want to go far, go with a
crew or I go with people. It's the same

00:26:58.240 --> 00:27:02.397
principle. So, what happens with open
source is that you put the stuff out

00:27:02.480 --> 00:27:05.677
there and then thousands
of people strengthen it.

00:27:05.760 --> 00:27:09.037
That's the magic of open source. That's
what I did and what I've done for most

00:27:09.120 --> 00:27:12.837
of my life. It's important to note,
for example, that Google was based on open

00:27:12.920 --> 00:27:16.597
source architectures. So,
this is another big change in my

00:27:16.680 --> 00:27:18.997
industry. Now, let's think about China.

00:27:19.080 --> 00:27:21.677
So, China, which again
is highly regulated,

00:27:21.760 --> 00:27:24.957
uh decided to do something
completely different.

00:27:25.040 --> 00:27:29.637
All of the top models are
open weight and open source,

00:27:29.720 --> 00:27:33.277
which means you can take the source code
and you can take all the numbers, which

00:27:33.360 --> 00:27:37.117
is the weights are the thing that the
computer uses to reason, and you can do

00:27:37.200 --> 00:27:40.957
whatever you want to with it. So,
I have startups in California with

00:27:41.040 --> 00:27:45.437
have a lot of Chinese uh speakers
who prefer the Chinese models over the

00:27:45.520 --> 00:27:49.597
American models. So, if you ask them,
they say, "Well, it works really well.

00:27:49.680 --> 00:27:52.037
Oh, and by the way, it's free."

00:27:52.120 --> 00:27:53.557
Now, you sit there and you go, "Hmm,

00:27:53.640 --> 00:27:56.237
okay." Now, at Princeton,

00:27:56.320 --> 00:28:01.917
how would a researcher here get into this?
Well, they'd have to work

00:28:02.000 --> 00:28:05.277
for, you know, how does
a graduate student work for

00:28:05.360 --> 00:28:09.637
them? So, ultimately, about 10 years
ago, Google, through me, created a

00:28:09.720 --> 00:28:13.997
Google center here on the other side of
Prospect, right? So, that primarily, you

00:28:14.080 --> 00:28:16.237
could actually walk from here to the

00:28:16.320 --> 00:28:21.077
Google data centers, logically.
That's not a scalable strategy.

00:28:21.160 --> 00:28:24.597
So, there's a problem with open source
leadership in America. Why is this so

00:28:24.680 --> 00:28:27.757
important geographically if you don't
care about that? It's because the

00:28:27.840 --> 00:28:31.717
majority of the models in the world will
be Chinese. Do you know why? Cuz they're

00:28:31.800 --> 00:28:36.037
going to be free. It's not
that complicated. Uh we were at

00:28:36.120 --> 00:28:39.757
dinner last night with my friend who's
quite wealthy, and he was joking that he

00:28:39.840 --> 00:28:43.757
was using one of the American models,
and he calculated that he was doing it

00:28:43.840 --> 00:28:47.397
for fun. Um and he calculated it
was going to cost him a half a million

00:28:47.480 --> 00:28:51.117
dollars a year for his entertainment,
which consisted of building a newsreader

00:28:51.200 --> 00:28:54.117
that was biased on his political views.

00:28:54.200 --> 00:28:55.677
You know, it's perfectly fine.

00:28:55.760 --> 00:28:56.677
Uh,

00:28:56.760 --> 00:28:58.797
so he switched to a Chinese model to to

00:28:58.880 --> 00:29:02.877
read American news cheaply.

00:29:02.960 --> 00:29:06.317
So, it's real. >> So,
that's hot tough competition.

00:29:06.400 --> 00:29:09.917
>> Yeah. Chinese are going to be very
tough, and I I'm very concerned, and I

00:29:10.000 --> 00:29:11.917
want to be clear to everybody here. Most

00:29:12.000 --> 00:29:14.397
people here are US citizens.

00:29:14.480 --> 00:29:16.997
We are not meeting the

00:29:17.080 --> 00:29:19.877
competition and beating them globally.

00:29:19.960 --> 00:29:23.917
They are investing in science
and technology and innovation at a rate to

00:29:24.000 --> 00:29:27.517
overcome America. What would it be like if

00:29:27.600 --> 00:29:31.717
if China were the winner in all
categories? AI, quantum, so forth and so

00:29:31.800 --> 00:29:35.277
on. Start thinking about conflict.

00:29:35.360 --> 00:29:39.477
America is always seems to be in some
kind of war or not. Uh, we're getting

00:29:39.560 --> 00:29:43.517
ready over Taiwan. Right?
Can you imagine, here's the

00:29:43.600 --> 00:29:46.877
thought experiment. So, there's
something called a superintelligence,

00:29:46.960 --> 00:29:49.677
which is the point at which
it's smarter than all of us.

00:29:49.760 --> 00:29:53.357
Uh, many people in my industry think
that this will occur within 10 years. We

00:29:53.440 --> 00:29:56.197
can debate this. Nobody really knows.

00:29:56.280 --> 00:29:59.077
Let's imagine they get there first.

00:29:59.160 --> 00:30:00.797
And as part of that superintelligence,

00:30:00.880 --> 00:30:06.317
they figure out some attack vector
against America that we cannot imagine

00:30:06.400 --> 00:30:10.557
right now. And and by the way,
the inverse is true for us.

00:30:10.640 --> 00:30:14.277
So, this is big deal. If you care about
national security, you need to fund

00:30:14.360 --> 00:30:17.597
this. If you don't care about national
security, you care about social welfare,

00:30:17.680 --> 00:30:20.957
you need to fund this. If you care about
economic growth, which I think all of us

00:30:21.040 --> 00:30:24.037
certainly do, uh,
you better fund it. You better

00:30:24.120 --> 00:30:26.837
figure out a way to keep the the engine

00:30:26.920 --> 00:30:29.677
that got me to where I am going.

00:30:29.760 --> 00:30:31.840
Right?

00:30:32.440 --> 00:30:36.357
>> What you said before come out
what you just said now makes me

00:30:36.440 --> 00:30:38.957
think of the electricity grid.

00:30:39.040 --> 00:30:43.117
I mean, all all of this is
ultimately on on electricity.

00:30:43.200 --> 00:30:46.837
And it is is that or might that become a

00:30:46.920 --> 00:30:51.557
vulnerability in a case like that? You
don't really need to send airplanes with

00:30:51.640 --> 00:30:52.957
bombs.

00:30:53.040 --> 00:30:56.437
>> So first place China
last year built 130

00:30:56.520 --> 00:31:00.757
gigawatts of renewable energy.

00:31:00.840 --> 00:31:04.157
Okay, remember the number I said is that
we needed about 100 gigawatts between

00:31:04.240 --> 00:31:08.557
now and 2030 in America. They already
have it. Right? And there's estimates

00:31:08.640 --> 00:31:12.797
that they have 20 gigawatts just sitting
around. Right? I I don't know what the

00:31:12.880 --> 00:31:15.157
real numbers are. I'll try to find out.

00:31:15.240 --> 00:31:19.397
So assuming that these data centers

00:31:19.480 --> 00:31:21.037
continue to be these large things, they

00:31:21.120 --> 00:31:24.877
become military targets. I wrote a paper

00:31:24.960 --> 00:31:28.997
which we called um the argument went
something like this. This is influence

00:31:29.080 --> 00:31:32.437
of Kissinger. Um you start
thinking about what happens

00:31:32.520 --> 00:31:35.517
when we're getting stronger and China's

00:31:35.600 --> 00:31:37.677
getting stronger or whoever.

00:31:37.760 --> 00:31:40.797
Um there's a point at which one side

00:31:40.880 --> 00:31:43.397
could decide to do a preemptory attack

00:31:43.480 --> 00:31:46.397
of the other, which is your question.

00:31:46.480 --> 00:31:49.757
Doesn't it make sense to slow down
your competitor by just bombing their data

00:31:49.840 --> 00:31:55.197
centers? So we being pleasant
Princetonian type graduates said

00:31:55.280 --> 00:31:59.197
there's a better alternative, which
is to just give the enemy a switch.

00:31:59.280 --> 00:32:03.477
So they don't have to bomb the data
center, they just have to turn it off.

00:32:03.560 --> 00:32:07.637
And everybody goes, that's
the stupidest idea I've ever seen.

00:32:07.720 --> 00:32:10.477
If you believe that these data centers

00:32:10.560 --> 00:32:12.877
are essential for the national security

00:32:12.960 --> 00:32:15.037
of for a nation, you've got to come up

00:32:15.120 --> 00:32:18.317
with some kind of a deterrent strategy.

00:32:18.400 --> 00:32:22.400
In that sense it is similar to nuclear.

00:32:22.560 --> 00:32:27.117
>> Which is cross your fingers and hope
that this there's symmetric forbearance.

00:32:27.200 --> 00:32:28.717
>> Well >> Which worked with nuclear.

00:32:28.800 --> 00:32:33.077
>> Well nuclear worked because of a theory
of mutual assured destruction. It's and

00:32:33.160 --> 00:32:34.637
it's important to know, I wrote a book

00:32:34.720 --> 00:32:40.400
about this, that mutual assured
destruction has never been tested.

00:32:42.440 --> 00:32:46.277
And I'm really glad. >> Yeah.
>> But let's start with an assertion. It's

00:32:46.360 --> 00:32:48.957
never been tested.
We don't know what will happen.

00:32:49.040 --> 00:32:53.717
Uh we don't know what will
happen if there is a third nuclear

00:32:53.800 --> 00:32:58.477
bombing of any kind having
the US having done the first two.

00:32:58.560 --> 00:32:59.917
We just don't know.

00:33:00.000 --> 00:33:02.397
Um and we're paying we're playing with

00:33:02.480 --> 00:33:05.837
incredibly powerful stuff here.

00:33:05.920 --> 00:33:09.677
So what I One things I've learned
is that this is not a kid's game. They

00:33:09.760 --> 00:33:13.637
They're these huge organizations
that worry about safety and security and

00:33:13.720 --> 00:33:17.677
biosecurity and so forth.
One way to understand the threats of

00:33:17.760 --> 00:33:20.757
models there's the threat
to democracy which

00:33:20.840 --> 00:33:25.077
everybody here understands.
There's We've known for a long time that

00:33:25.160 --> 00:33:28.157
the first things it would
be able to do would be

00:33:28.240 --> 00:33:31.477
cyber attacks. And the reason is that if
you can do good software program, you

00:33:31.560 --> 00:33:33.717
can also write good cyber attacks.

00:33:33.800 --> 00:33:38.037
When Methos came out, which is a couple
months ago, this is the Anthropic cyber

00:33:38.120 --> 00:33:40.197
cyber attack model. They had the good

00:33:40.280 --> 00:33:45.117
judgment of only releasing it
to about 40 companies as well as the US

00:33:45.200 --> 00:33:48.757
government. By the way, this is during
the time when they're having a big fight

00:33:48.840 --> 00:33:51.997
with the Pentagon and the White House
and they get banned while they're

00:33:52.080 --> 00:33:56.917
essential for the use of the NSA.
So it's like crazy. Um and I'm sure that

00:33:57.000 --> 00:34:00.357
will get resolved because they they
ultimately have to work together.

00:34:00.440 --> 00:34:02.477
But if you think about
it if you're the head

00:34:02.560 --> 00:34:07.717
of the NSA or in that group, you want
these tools for offensive cyber attack

00:34:07.800 --> 00:34:12.357
and you also want to be able to use these
tools to prevent defensive cyber attack.

00:34:12.440 --> 00:34:15.557
The term in the industry is called
offense dominant versus defense

00:34:15.640 --> 00:34:20.197
dominant. Does Are you more likely
to win with offense or defense? My

00:34:20.280 --> 00:34:23.997
assessment is these tools are offense
dominant. So we need to be very careful

00:34:24.080 --> 00:34:29.797
that we have a strong offense as well as
a strong defense. That's my personal view.

00:34:29.880 --> 00:34:34.157
>> Let Let turn to something a little
more less cataclysmic, but that a lot of

00:34:34.240 --> 00:34:37.317
people worry about, which is job loss.

00:34:37.400 --> 00:34:38.077
>> Yeah.

00:34:38.160 --> 00:34:42.477
>> As opposed to job change, or both, or

00:34:42.560 --> 00:34:44.837
uh is one bigger than the other? Do you

00:34:44.920 --> 00:34:46.637
do you spend any time worrying about

00:34:46.720 --> 00:34:50.757
massive unemployment as a result of AI?

00:34:50.840 --> 00:34:53.037
>> Uh I do, but not now.

00:34:53.120 --> 00:34:54.957
And I'll give you the numbers. Um we

00:34:55.040 --> 00:34:58.637
know that the current stuff in the US

00:34:58.720 --> 00:35:03.117
is displacing low-level customer
service jobs, and we know it's displacing

00:35:03.200 --> 00:35:06.797
software jobs, typically
the lower-level, entry-level jobs.

00:35:06.880 --> 00:35:08.237
Uh which is a big deal for those people

00:35:08.320 --> 00:35:11.557
who are displaced. We have not yet seen

00:35:11.640 --> 00:35:15.277
mass effect of the white-collar workforce.

00:35:15.360 --> 00:35:19.757
It's going to happen in the next 5 years.
And here's how it happens.

00:35:19.840 --> 00:35:23.957
Um there's a a very good article
yesterday by a friend of mine who runs

00:35:24.040 --> 00:35:27.757
Cloudflare, who wrote
a very nice explanation that

00:35:27.840 --> 00:35:31.477
he runs a company which is growing
at 30% a year, has huge valuations, and so

00:35:31.560 --> 00:35:35.717
forth, and he did 20% layoff.
And he justifies his layoff by looking

00:35:35.800 --> 00:35:40.477
at the categories of workers, and he
doesn't need this set of categories.

00:35:40.560 --> 00:35:44.117
And those specific categories are not
sales, they're not engineering, they're

00:35:44.200 --> 00:35:46.717
a bunch of the infrastructure. So, your

00:35:46.800 --> 00:35:48.997
decision you should decide whether you

00:35:49.080 --> 00:35:51.717
think those people can be retrained or

00:35:51.800 --> 00:35:57.157
get new skills that are more consistent
with what what AI cannot do well.

00:35:57.240 --> 00:36:01.997
Um I know one way to say this is that

00:36:02.080 --> 00:36:05.437
this is the first technology that's
really threatened white-collar work,

00:36:05.520 --> 00:36:08.437
and because white-collar people write
really well, they write about it all

00:36:08.520 --> 00:36:11.917
day, right? Right?

00:36:12.000 --> 00:36:14.317
They write about their their experience.

00:36:14.400 --> 00:36:17.117
2/3 of the jobs in America are

00:36:17.200 --> 00:36:19.677
essentially labor human labor-centric.

00:36:19.760 --> 00:36:23.357
They require hands or pressure
or presence or so forth. Yeah. And these

00:36:23.440 --> 00:36:25.157
are important jobs.

00:36:25.240 --> 00:36:29.597
So, as long as robots don't really work,

00:36:29.680 --> 00:36:32.357
the impact will be manageable.

00:36:32.440 --> 00:36:35.637
The fear I have is that in 10 years

00:36:35.720 --> 00:36:37.997
the robotic revolution, which is much

00:36:38.080 --> 00:36:41.477
slower than America and software, will

00:36:41.560 --> 00:36:45.957
in fact begin to displace those jobs. If
that begins to occur, now this is a long

00:36:46.040 --> 00:36:49.117
time from now, we're going to have
a whole hell of a lot of problems, right?

00:36:49.200 --> 00:36:53.197
For obvious reasons.
Um and why do I say it's going to take

00:36:53.280 --> 00:36:55.957
that long? Um the US robotics industry

00:36:56.040 --> 00:37:00.757
is relatively slow and relatively
expensive. Um in China they have a I'll

00:37:00.840 --> 00:37:03.557
give you an example.
They have a company called Unitree,

00:37:03.640 --> 00:37:08.037
um which builds an a robot
called R1, which I've ordered

00:37:08.120 --> 00:37:09.717
for $5,000.

00:37:09.800 --> 00:37:12.157
Look at the videos of what it can do.

00:37:12.240 --> 00:37:15.117
So, that's what's coming. Now, it's dumb

00:37:15.200 --> 00:37:19.477
as you could imagine. The thing needs
a proper brain, right? Whatever whatever

00:37:19.560 --> 00:37:22.317
that means. I mean,
it's like the stupidest

00:37:22.400 --> 00:37:26.677
robot you could imagine.
Um but that can be fixed over time.

00:37:26.760 --> 00:37:28.837
That's I think the real concern.

00:37:28.920 --> 00:37:31.827
>> Take you back before the robots.

00:37:31.910 --> 00:37:34.877
>> [clears throat] >> The the
good side of of the answer to

00:37:34.960 --> 00:37:38.477
this question that's been
around since the Luddites

00:37:38.560 --> 00:37:42.837
is that yes, the new technology will
destroy some jobs, but it will also create

00:37:42.920 --> 00:37:43.397
>> Yeah.

00:37:43.480 --> 00:37:46.477
>> uh a lot of new jobs. So, it's mostly

00:37:46.560 --> 00:37:50.797
about job change and disruption and

00:37:50.880 --> 00:37:54.597
moving resources from here to there.

00:37:54.680 --> 00:37:58.957
The United You can tell me if you agree
with this or not. I would say the United

00:37:59.040 --> 00:38:02.997
States among the rich countries
is almost uniquely bad at managing that

00:38:03.080 --> 00:38:07.797
transition. Retraining workers,
supporting them while they're

00:38:07.880 --> 00:38:09.477
unemployed,

00:38:09.560 --> 00:38:12.597
uh etc. etc. Making the transition

00:38:12.680 --> 00:38:16.357
easier rather than harder.

00:38:16.440 --> 00:38:18.837
So, do you do you see it that way?

00:38:18.920 --> 00:38:21.757
That That's the answer to the

00:38:21.840 --> 00:38:25.197
on the warrior side of it's only

00:38:25.280 --> 00:38:27.597
a job change, it's not job destruction.

00:38:27.680 --> 00:38:31.997
>> Um the benefit the counter argument is
that the benefit of the brutal nature of

00:38:32.080 --> 00:38:35.957
American employment is that it's
a reconfiguration into a new into a new

00:38:36.040 --> 00:38:40.077
role quicker, which is part of why
American productivity is better than

00:38:40.160 --> 00:38:43.397
Europe for example and there are debate
technical debates about that. Paul

00:38:43.480 --> 00:38:46.277
Krugman for example has been
arguing against that point.

00:38:46.360 --> 00:38:50.317
Um What do I really think?

00:38:50.400 --> 00:38:54.800
Um >> Yeah, what do you really think?

00:38:56.320 --> 00:38:57.997
You're a You're a very good thinker.

00:38:58.080 --> 00:39:03.520
>> I think I I'm That was a
That was a strategic pause.

00:39:04.680 --> 00:39:09.597
I I'll give an example of how crazy
things are in my little world. Um there

00:39:09.680 --> 00:39:13.437
was a paper and again,
you're the economist so

00:39:13.520 --> 00:39:17.757
I I believe it's a fair assertion
that there are no economics for economies

00:39:17.840 --> 00:39:21.757
that are doubling every year.
There are no examples of doubling GDP.

00:39:21.840 --> 00:39:25.997
>> a whole side of the economy, not >>
Yeah, doubling. Okay. So anyway, this

00:39:26.080 --> 00:39:30.357
group managed to actually come up
with a set of math which I've no way of

00:39:30.440 --> 00:39:34.837
validating that if you replace all labor

00:39:34.920 --> 00:39:39.237
by either AI or by robots.
This is the presumption.

00:39:39.320 --> 00:39:42.037
It's obviously not a good
thing for other reasons.

00:39:42.120 --> 00:39:44.637
You could grow the economy by a fact You

00:39:44.720 --> 00:39:47.837
could double the economy every year.

00:39:47.920 --> 00:39:51.517
Okay. So I thought highly
unlikely to be true.

00:39:51.600 --> 00:39:54.077
But let's start thinking about that.

00:39:54.160 --> 00:39:58.277
So if it Okay, so here we are and now
everyone's twice as wealthy and then

00:39:58.360 --> 00:40:01.757
twice as wealthy again and twice
as wealthy again. You all know you can do

00:40:01.840 --> 00:40:06.077
combinatorial math.
So then would you be happy if everyone

00:40:06.160 --> 00:40:10.317
here was as rich as Qatar?
You'd be happier.

00:40:10.400 --> 00:40:13.437
All right? Just think of all
the Rolls Royces and the equivalents.

00:40:13.520 --> 00:40:15.597
Um what if you were 10 times happier

00:40:15.680 --> 00:40:20.717
than Qatar? What if you were 100 times
happier than Qatar? Um and what if it

00:40:20.800 --> 00:40:22.117
were true for the whole world or at

00:40:22.200 --> 00:40:24.957
least the countries that adopted this?

00:40:25.040 --> 00:40:28.157
So, there is an argument, which I'm not

00:40:28.240 --> 00:40:32.237
personally making,
that the abundance and the the

00:40:32.320 --> 00:40:35.957
elimination of scarcity, right?
So, the the basic argument from the

00:40:36.040 --> 00:40:40.077
accelerationists is that we're
eliminating scarcity. Every day, every

00:40:40.160 --> 00:40:43.357
aspect of history, every aspect
of economics is fundamentally about

00:40:43.440 --> 00:40:47.517
scarcity, scarcity pricing scarcity,
markets over You understand this very

00:40:47.600 --> 00:40:52.557
well. Um what happens if we
remove all of those constraints?

00:40:52.640 --> 00:40:55.557
No one knows. Now,
let me give you a counter and a

00:40:55.640 --> 00:40:57.397
further counter.

00:40:57.480 --> 00:41:00.557
The US is growing at 2 to 3%. 3% is

00:41:00.640 --> 00:41:03.117
considered high. Chinese growth at in

00:41:03.200 --> 00:41:07.637
its heyday was 10%. It's now down to 2
or 3% and they're very upset. Let's

00:41:07.720 --> 00:41:09.397
imagine that all of the countries that

00:41:09.480 --> 00:41:13.477
we care about could grow at 15% a year.

00:41:13.560 --> 00:41:17.237
How would that feel? Because if the slope
is what I just said, and that's a

00:41:17.320 --> 00:41:20.637
lot of ifs, relax.

00:41:20.720 --> 00:41:21.837
Okay?

00:41:21.920 --> 00:41:26.197
If the slope is not that, don't relax.

00:41:26.280 --> 00:41:30.917
If the slope is not fundamentally
changed by the business improvements,

00:41:31.000 --> 00:41:33.717
economic improvements, the scientific
improvements, the quality of life

00:41:33.800 --> 00:41:40.557
improvements that I've described, then we
have screwed up collectively as a society.

00:41:40.640 --> 00:41:42.477
>> The slope is

00:41:42.560 --> 00:41:45.397
very few people doubt that the slope is

00:41:45.480 --> 00:41:47.757
in the direction that you say.

00:41:47.840 --> 00:41:52.997
15% a year, I don't think
you'll get many people signing on

00:41:53.080 --> 00:41:58.317
to that. But, there's a question
of who owns the resources.

00:41:58.400 --> 00:42:02.317
>> Mhm. >> Uh it it's not
going to be equally distributed.

00:42:02.400 --> 00:42:05.317
>> That's right. >> In any
conceive We can't even conceive

00:42:05.400 --> 00:42:10.677
of a scenario which it would be equally uh

00:42:10.760 --> 00:42:14.597
distributed. So, there going
to be people left behind.

00:42:14.680 --> 00:42:18.037
Now, a reasonable answer
is the society will

00:42:18.120 --> 00:42:20.477
be so rich that it can take care of the

00:42:20.560 --> 00:42:23.037
people that are left behind.

00:42:23.120 --> 00:42:27.117
My suggestion 5 2 minutes ago was

00:42:27.200 --> 00:42:30.837
this country is not
very good at doing that.

00:42:30.920 --> 00:42:33.477
And that's that's where
the worry comes from.

00:42:33.560 --> 00:42:37.717
>> I understand. That is essentially
a political decision for a rich country.

00:42:37.800 --> 00:42:40.317
So, the most likely scenario

00:42:40.400 --> 00:42:45.557
is that a smaller number of people
still a large absolute number, but a

00:42:45.640 --> 00:42:49.157
smaller number of people get more
and more of the economic share of America

00:42:49.240 --> 00:42:50.957
because of this technology. >> Yes.

00:42:51.040 --> 00:42:53.237
>> That That is a reasonable presumption.

00:42:53.320 --> 00:42:56.797
If you look at the tech market If you
look at the market today, all the top

00:42:56.880 --> 00:42:59.197
value is in the tech tech sector.

00:42:59.280 --> 00:43:02.997
Furthermore, the stock market's gone up
for the last 2 years basically because

00:43:03.080 --> 00:43:07.037
of all of the the tech companies,
the leading tech companies.

00:43:07.120 --> 00:43:10.557
So, the amount of wealth creation
is similar to that of the Golden Age, and

00:43:10.640 --> 00:43:14.077
I'm obviously happy to be part of that,
and happy We're happy to give that money

00:43:14.160 --> 00:43:18.277
to Princeton. Um the the

00:43:18.360 --> 00:43:22.157
it's not good for society to have a Gini
coefficient that's that imbalanced,

00:43:22.240 --> 00:43:25.797
right? For obvious reasons.
That's a political decision.

00:43:25.880 --> 00:43:29.677
But, I think we it's obvious to me
that the gains will not be uniformly

00:43:29.760 --> 00:43:32.037
distributed. It's It does not then

00:43:32.120 --> 00:43:34.677
follow that the bottom is still poor.

00:43:34.760 --> 00:43:38.437
You can imagine a situation There's
a lot of evidence that the internet

00:43:38.520 --> 00:43:43.397
ultimately took people who were very, very
poor and made them a little bit less poor.

00:43:43.480 --> 00:43:46.677
Because they had larger markets,
better knowledge, and so forth. So, it's

00:43:46.760 --> 00:43:50.557
possible that the that that the poor
do better and the rich do much, much

00:43:50.640 --> 00:43:55.117
better. That's a political decision.
We need to get ahead of it.

00:43:55.200 --> 00:43:58.157
>> Let me ask you a related question,
and I'm going to open it to the audience

00:43:58.240 --> 00:44:00.557
pretty soon.

00:44:00.640 --> 00:44:04.517
There are some technologies that lead to

00:44:04.600 --> 00:44:07.237
a kind of a winner-take-all

00:44:07.320 --> 00:44:09.877
market structure and some that don't.

00:44:09.960 --> 00:44:14.877
How do you see this one? >> So,
this the question of concentration

00:44:14.960 --> 00:44:18.917
at the technical level is something that
we spend a lot of time thinking about.

00:44:19.000 --> 00:44:21.637
For a long time I thought

00:44:21.720 --> 00:44:25.997
that there would only be 10 computer
companies. There would be four or five

00:44:26.080 --> 00:44:29.317
in the US, two or three
in China, and a couple of

00:44:29.400 --> 00:44:32.917
others, maybe one in Israel, maybe one
in Europe, maybe one in India, something

00:44:33.000 --> 00:44:35.077
like that, on the order of 10.

00:44:35.160 --> 00:44:37.397
And the reason was that the capital cost

00:44:37.480 --> 00:44:39.757
of these things is so large.

00:44:39.840 --> 00:44:44.997
How many can you have? Right? I mean,
there are limits even to the craziness

00:44:45.080 --> 00:44:48.957
in my industry. You know, how many
trillion-dollar companies, individuals,

00:44:49.040 --> 00:44:51.197
businesses can you have? It's just the

00:44:51.280 --> 00:44:53.917
the money doesn't add up.

00:44:54.000 --> 00:44:57.997
It that looks to be true, but

00:44:58.080 --> 00:45:01.437
there's then going to be a pyramid

00:45:01.520 --> 00:45:04.517
of very powerful, much less expensive

00:45:04.600 --> 00:45:09.797
choices. Then there's a debate in my
industry as to whether these lower power

00:45:09.880 --> 00:45:13.797
performance things will be the technical
term is called distilled. Will they come

00:45:13.880 --> 00:45:17.877
from the big models or will
they be built in their own way?

00:45:17.960 --> 00:45:22.277
We don't know. Most people
believe that I talk to that

00:45:22.360 --> 00:45:25.917
the vast majority of the AI that you
experiment that you experience in your

00:45:26.000 --> 00:45:29.037
future lives will not actually be from

00:45:29.120 --> 00:45:32.997
the largest models, but rather will
be embedded in your daily life and you

00:45:33.080 --> 00:45:36.157
won't even know it.
Like if you think about it,

00:45:36.240 --> 00:45:37.477
do you think very much about the

00:45:37.560 --> 00:45:40.997
operating system that's on your watch?

00:45:41.080 --> 00:45:45.877
I do. Cuz I used to work
on operating systems

00:45:45.960 --> 00:45:49.677
here. Okay? I'm like the last one.

00:45:49.760 --> 00:45:53.717
Um you take it for granted.
And if you think about 100 years ago,

00:45:53.800 --> 00:45:57.637
electrification, people like, "Oh my god."
And there was all sorts of issues

00:45:57.720 --> 00:46:01.197
and houses would burn down
and electricity fires and all of that.

00:46:01.280 --> 00:46:05.157
You don't worry about that today. You
worry about, you know, Google and Amazon

00:46:05.240 --> 00:46:09.597
and OpenAI and all of that.
Um so so societies work these things

00:46:09.680 --> 00:46:13.360
out. So

00:46:13.560 --> 00:46:15.677
I I think in that sense we'll be fine.

00:46:15.760 --> 00:46:21.360
But we'll get there.
>> Thank you very much.

00:46:30.880 --> 00:46:35.877
Now If you If you have
a question, Kathleen

00:46:35.960 --> 00:46:39.997
is Dana Is it just you?
Kathleen and Dana are around with

00:46:40.080 --> 00:46:42.237
microphones. Raise your hand.

00:46:42.320 --> 00:46:47.077
And when you get the microphone,
identify yourself please briefly.

00:46:47.160 --> 00:46:51.597
And And keep Keep it
to questions, not speeches.

00:46:51.680 --> 00:46:56.117
>> We better have some some graduating
seniors ask questions as well.

00:46:56.200 --> 00:46:58.077
Go ahead. Yes. >> Hi Mr.
Schmidt. Thank you so much for

00:46:58.160 --> 00:47:01.077
the talk. Uh Tiger Gao, class of '21. Uh

00:47:01.160 --> 00:47:05.277
Do you think large language models,
which are by construction a next word

00:47:05.360 --> 00:47:08.677
predictor or next token predictor, will
be the model architecture that take us

00:47:08.760 --> 00:47:11.877
to artificial general intelligence
or superintelligence, or do you think we

00:47:11.960 --> 00:47:15.157
need another model architecture?
And secondly, what

00:47:15.240 --> 00:47:19.477
do you think is the probability
of doom, P(doom)? I think if you ask

00:47:19.560 --> 00:47:23.797
If you poll Anthropic, the median
response rate was probably like 25%,

00:47:23.880 --> 00:47:26.637
which is high as a Where are
all your friends thinking?

00:47:26.720 --> 00:47:29.917
>> On the latter, I'm not I'm not a fan
of that whole process and I don't agree

00:47:30.000 --> 00:47:31.637
with it and I don't believe in it.

00:47:31.720 --> 00:47:35.557
Uh on the former Um the the

00:47:35.640 --> 00:47:37.197
Ask the question again.

00:47:37.280 --> 00:47:39.437
The The first question was

00:47:39.520 --> 00:47:43.477
Do we need more things to get to AGI?

00:47:43.560 --> 00:47:44.997
>> Does LLM by construction prevent this?

00:47:45.080 --> 00:47:46.717
>> So For those of you
who don't understand

00:47:46.800 --> 00:47:49.637
what an LLM does, it's just predicting

00:47:49.720 --> 00:47:52.997
the next word. And you go, "How could it

00:47:53.080 --> 00:47:57.997
be?" As a story, there
was a group at OpenAI and

00:47:58.080 --> 00:47:59.757
I've talked to them at length. It was a

00:47:59.840 --> 00:48:05.517
Thursday night, and they had just
turned on They had GPT-2. The The

00:48:05.600 --> 00:48:10.797
The The T is transformer. They had taken
that from Google work. And they're busy

00:48:10.880 --> 00:48:14.877
and they'd done a GPT-2, and then they
put more training into it. And they

00:48:14.960 --> 00:48:17.597
turned on GPT-3, and they realized that

00:48:17.680 --> 00:48:22.557
it could write.
And that was a real discovery.

00:48:22.640 --> 00:48:26.837
I've tried to figure out why
it's so good. And the answer is

00:48:26.920 --> 00:48:31.157
that it's similar to the way our brains
work in the sense that do you really

00:48:31.240 --> 00:48:35.557
know what word you're about to say
or think? It just occurs naturally.

00:48:35.640 --> 00:48:37.917
So, in that sense, it's very similar.

00:48:38.000 --> 00:48:41.597
The question to you
is is that going to be enough?

00:48:41.680 --> 00:48:44.277
There are, of course, two schools.
The people who are building large language

00:48:44.360 --> 00:48:48.517
models think it's going to be enough.
I personally think it's not.

00:48:48.600 --> 00:48:52.557
And the reason is this this implied
intelligence that I was describing.

00:48:52.640 --> 00:48:56.357
There's just a whole bunch of knowledge
that you have in your head by virtue of

00:48:56.440 --> 00:48:59.837
coming to Princeton, being a good
person, being a citizen of the world

00:48:59.920 --> 00:49:05.080
that they just don't know and they're
not going to get unless you tell them.

00:49:06.120 --> 00:49:10.157
Yeah. Yes, sir.
>> What is your viewpoint on the

00:49:10.240 --> 00:49:14.437
relationship between AI and consciousness?
Do you think at some

00:49:14.520 --> 00:49:19.197
point AI will become advanced enough
that it will be indistinguishable from

00:49:19.280 --> 00:49:23.437
consciousness?
>> I'm in the no category. Um so, I of

00:49:23.520 --> 00:49:27.397
course came to Princeton, and I
met with the neuroscience group.

00:49:27.480 --> 00:49:30.877
And the neuroscience group I said to
the neuroscience, "How does consciousness

00:49:30.960 --> 00:49:35.357
appear?" And their answer
was the experimental

00:49:35.440 --> 00:49:39.797
evidence, and I If I misquote them,
I'm sorry. Is that when you get disparate

00:49:39.880 --> 00:49:41.237
systems

00:49:41.320 --> 00:49:44.637
in biological brains, and the disparate

00:49:44.720 --> 00:49:47.157
systems are different enough and are

00:49:47.240 --> 00:49:49.837
forced to interact enough, consciousness

00:49:49.920 --> 00:49:52.677
appears to appear show up because of the

00:49:52.760 --> 00:49:55.237
need for awareness of what's going on.

00:49:55.320 --> 00:49:58.117
That's the simplest
explanation I can give you.

00:49:58.200 --> 00:49:59.837
Do I think that's going to occur with

00:49:59.920 --> 00:50:02.557
these? The answer is no. And the reason

00:50:02.640 --> 00:50:05.557
is that they're not biological.

00:50:05.640 --> 00:50:09.477
But I could be wrong.
>> There's one over here.

00:50:09.560 --> 00:50:12.397
>> Frank Langford, class of '86.

00:50:12.480 --> 00:50:16.277
What policy prescriptions would you give
to the leaders of this country over the

00:50:16.360 --> 00:50:19.357
next decade to prepare for white-collar

00:50:19.440 --> 00:50:23.997
job loss and to prevent the kind
of political backlash that we've seen from

00:50:24.080 --> 00:50:30.440
the offshoring earlier of blue-collar
jobs to countries like China?

00:50:31.360 --> 00:50:33.477
>> Um

00:50:33.560 --> 00:50:38.437
I think the first question is can
we get the young people ready?

00:50:38.520 --> 00:50:43.037
And the simplest prescription for that
is to have a mandatory freshman class at

00:50:43.120 --> 00:50:47.517
Princeton along with every other
university. When when we came, we had to

00:50:47.600 --> 00:50:51.197
have a TB test, we had to do a swimming
test. Well, it seems to me that we

00:50:51.280 --> 00:50:53.317
should have an LLM test.

00:50:53.400 --> 00:50:55.117
And because if you think about an

00:50:55.200 --> 00:50:58.237
18-year-old today, LLMs will mediate

00:50:58.320 --> 00:51:02.997
every aspect of their lives for the
rest of their lives in some form. So,

00:51:03.080 --> 00:51:06.357
learning how to work with them, learning
how to empower them, and so forth.

00:51:06.440 --> 00:51:10.637
That's the first answer.
Then the second answer is to try to get

00:51:10.720 --> 00:51:15.717
people to understand how that how these
tools can enable them to make more money

00:51:15.800 --> 00:51:16.997
at work.

00:51:17.080 --> 00:51:19.717
So, um again, last night at dinner, one

00:51:19.800 --> 00:51:22.957
of the businesses announced that

00:51:23.040 --> 00:51:26.037
their productivity per human is so high,

00:51:26.120 --> 00:51:31.037
they're hiring more humans. So,
I'm I'm not really fully answering

00:51:31.120 --> 00:51:33.597
your question cuz I'm avoiding the hard
questions. What do you do with the

00:51:33.680 --> 00:51:35.837
people who can't do one and two?

00:51:35.920 --> 00:51:38.837
Well, the answer for them, if you look
at the history of Luddites and so forth,

00:51:38.920 --> 00:51:42.877
is they lose their jobs. And then
that becomes a political problem of uh

00:51:42.960 --> 00:51:47.637
retraining, motivating, and so forth. In
our current political climate, I have to

00:51:47.720 --> 00:51:50.437
assume that they will take up pitchforks.

00:51:50.520 --> 00:51:53.277
Right? In other words, that when
these waves go and I'm not trying to be

00:51:53.360 --> 00:51:56.397
negative here, I'm just trying to be
honest. When these waves go through

00:51:56.480 --> 00:51:58.997
society, the evidence is they go through

00:51:59.080 --> 00:52:02.757
much more quickly than society is ready.

00:52:02.840 --> 00:52:06.757
And so the government needs
to be ready for this. Um

00:52:06.840 --> 00:52:08.477
this the current administration started

00:52:08.560 --> 00:52:10.517
off with a completely libertarian view

00:52:10.600 --> 00:52:14.117
of this and is now beginning to propose

00:52:14.200 --> 00:52:17.797
regulations and restrictions, which must
be very hard for them to deal with the

00:52:17.880 --> 00:52:22.517
worst examples. And I think that's
the future. And you'll see it it will come

00:52:22.600 --> 00:52:26.960
naturally because
of the political pressure.

00:52:27.120 --> 00:52:30.957
>> Who have you got? >> [clears throat]
>> Hello. Um this is Karthik Shastri from

00:52:31.040 --> 00:52:35.077
the class of '16 and also the economics
faculty. And I wanted to return to a

00:52:35.160 --> 00:52:38.437
question that Alan asked about lessons,
if any, from the social media

00:52:38.520 --> 00:52:42.517
revolution. And And you mentioned
yourself that in many ways we live in in

00:52:42.600 --> 00:52:47.157
a culture of magnified outrage.
It's arguably a negative feedback loop

00:52:47.240 --> 00:52:51.477
between the economics of social media,
which can do kind of amazing community

00:52:51.560 --> 00:52:54.517
building things and make a lot of money
from clicks and engagement and outrage

00:52:54.600 --> 00:52:56.797
to society and kind of back and forth.

00:52:56.880 --> 00:53:00.277
Now take AI, which is capable
of genuinely amazing um intellectual

00:53:00.360 --> 00:53:04.357
advances, like proving mathematical
theorems, as you suggested. It's also

00:53:04.440 --> 00:53:07.477
capable of other things that may
maximize more engagement, if that

00:53:07.560 --> 00:53:10.877
becomes the model, like using its
mastery of language to, you know,

00:53:10.960 --> 00:53:14.757
surreptitiously advertise things
to people and change how they think or to

00:53:14.840 --> 00:53:18.917
give a veneer of technical
credibility at a coherent >> Sorry.

00:53:19.000 --> 00:53:20.957
>> Um the question
of coherence of logic to

00:53:21.040 --> 00:53:24.597
the things that that um you know, aren't
really serious. So So do you see that as

00:53:24.680 --> 00:53:28.757
a trap essentially for the development
of AI? Um and if so, kind of socially,

00:53:28.840 --> 00:53:31.917
politically, economically,
what could we do >> Oh my god.

00:53:32.000 --> 00:53:34.437
>> to to sort of forestall that outcome.

00:53:34.520 --> 00:53:38.477
>> So So LLMs are a neutral technology in

00:53:38.560 --> 00:53:42.597
the sense that all technology
is essentially dual use, good and bad,

00:53:42.680 --> 00:53:47.037
military, civilian, and so forth.
It's up to us how we decide to limit these

00:53:47.120 --> 00:53:49.957
things. There is no
question that if I were

00:53:50.040 --> 00:53:53.277
truly evil, I could use these tools to

00:53:53.360 --> 00:53:57.397
completely manipulate um some
society. I'll give an example.

00:53:57.480 --> 00:54:00.637
In one of the first uh safety
demonstrations, the UK is the leader

00:54:00.720 --> 00:54:02.917
here, strangely, not the US.

00:54:03.000 --> 00:54:05.677
Um somebody said, "Let's build a uh

00:54:05.760 --> 00:54:08.157
community around some particular British

00:54:08.240 --> 00:54:10.477
political thing." And so they construct

00:54:10.560 --> 00:54:15.397
a set of fake profiles. They completely
fill all of social media with them, and

00:54:15.480 --> 00:54:18.277
they sit there, and they don't do
anything. They don't do much, but then

00:54:18.360 --> 00:54:22.117
in a coordinated attack,
they actually then

00:54:22.200 --> 00:54:23.837
um spread something which they know will

00:54:23.920 --> 00:54:27.357
get magnified. So, remember that the and

00:54:27.440 --> 00:54:30.997
I call this the dual evil problem. So,

00:54:31.080 --> 00:54:35.837
for purposes of argument, I have a uh
soft a piece of software that magnifies

00:54:35.920 --> 00:54:38.197
everything. Well,
I can build another piece of

00:54:38.280 --> 00:54:41.597
software that can figure out what that
software is doing, and then I can feed

00:54:41.680 --> 00:54:45.997
whatever evil idea I have, have this
software feed it into the thing that

00:54:46.080 --> 00:54:49.797
magnifies, and off it goes. Now,
the social media people will say that they

00:54:49.880 --> 00:54:54.037
have good defenses against this. There's
plenty of evidence that they don't.

00:54:54.120 --> 00:54:56.997
So, that's from my perspective, that's
a technical problem, but it's also a

00:54:57.080 --> 00:55:00.157
regulatory problem. The technical
problem is making sure that I cannot

00:55:00.240 --> 00:55:04.317
come in through an illegal
or mischievous path where I'm a foreign

00:55:04.400 --> 00:55:07.757
agent, you know, Russians or something
like that. Russians have a long history

00:55:07.840 --> 00:55:10.957
of trying to manipulate
our elections. This is how

00:55:11.040 --> 00:55:15.157
they would do it. Um we have to deal
with that, but it requires some

00:55:15.240 --> 00:55:19.317
constraint on the on the
economics of social media, some

00:55:19.400 --> 00:55:22.237
of you understand well. >> Uh

00:55:22.320 --> 00:55:24.317
Thank you. Uh my name is Jorge Aguilar,

00:55:24.400 --> 00:55:27.237
I'm class of '06. You alluded to several

00:55:27.320 --> 00:55:29.437
safeguards. One of them was that free

00:55:29.520 --> 00:55:32.357
speech is good, but bullhorns are can be

00:55:32.440 --> 00:55:34.757
bad, which sounds like nuisance law,

00:55:34.840 --> 00:55:39.557
right? And something intangible and
annoying. But, the other safeguard that

00:55:39.640 --> 00:55:44.157
I thought I heard was, and I'm asking
you to confirm, was you said LLMs

00:55:44.240 --> 00:55:48.197
are so dependent on the information we
give them. And you mentioned how there

00:55:48.280 --> 00:55:51.437
was something, and I forget what it was,
that you hadn't given the LLM, so it

00:55:51.520 --> 00:55:56.477
couldn't predict what you said. Is there
some thought then that a safeguard

00:55:56.560 --> 00:56:00.877
is there's some information we shouldn't
give to these AIs, cuz I think I

00:56:00.960 --> 00:56:04.437
remember you you then went on to say how
advanced it could be if we keep feeding

00:56:04.520 --> 00:56:08.197
it information. Is there
a a filter that we need to kind

00:56:08.280 --> 00:56:11.517
of set up? >> I think that's
unlikely to work, because

00:56:11.600 --> 00:56:15.757
it would require coordination with all
of the people who are busy feeding these

00:56:15.840 --> 00:56:18.437
LLMs with everything they can find.

00:56:18.520 --> 00:56:20.477
The current state of the industry is

00:56:20.560 --> 00:56:23.077
that every single piece of information

00:56:23.160 --> 00:56:25.637
that is essentially searchable,

00:56:25.720 --> 00:56:27.917
collectible, or scrapeable has been

00:56:28.000 --> 00:56:30.677
collected. Right? We sort of run out of

00:56:30.760 --> 00:56:35.677
training data, as it's called. And there
was a a little while where everyone was

00:56:35.760 --> 00:56:38.957
writing these articles about data
was the new oil, and we're running out of

00:56:39.040 --> 00:56:41.197
it, which is completely false.

00:56:41.280 --> 00:56:44.397
Because the systems can
also generate synthetic data.

00:56:44.480 --> 00:56:48.397
Furthermore, there's plenty of data that
you can get if you start, for example,

00:56:48.480 --> 00:56:50.157
surveilling people. I'm not arguing in

00:56:50.240 --> 00:56:52.357
favor of that. Um that's not an

00:56:52.440 --> 00:56:57.597
argument. Do not get me wrong. Um so
there's plenty more knowledge that we

00:56:57.680 --> 00:57:00.997
can feed to these things.
Um the the industry is governed by

00:57:01.080 --> 00:57:06.437
something called scaling laws,
and every 6 months I ask the same

00:57:06.520 --> 00:57:07.837
question,

00:57:07.920 --> 00:57:10.477
are the scaling laws stopping?

00:57:10.560 --> 00:57:14.357
And the answer right now is no, which is

00:57:14.440 --> 00:57:17.797
unbelievable.
And so this has led to the following

00:57:17.880 --> 00:57:22.437
phenomena. Uh a model
costs 200 to 500 million

00:57:22.520 --> 00:57:25.357
dollars, you release it,

00:57:25.440 --> 00:57:29.637
and it has a life
expectancy of about 6 months.

00:57:29.720 --> 00:57:33.077
And then there's another one. And the
competition between the leading players

00:57:33.160 --> 00:57:37.797
is so brutal, right, that that's they
literally count in the number of weeks

00:57:37.880 --> 00:57:41.397
value of these things, which cost
hundreds of millions of dollars.

00:57:41.480 --> 00:57:45.837
So, that's where the pressure is. And
they're they're they're go I don't know

00:57:45.920 --> 00:57:49.957
what there is a slope, right? There
is eventually a slope. We don't know where

00:57:50.040 --> 00:57:52.757
it is. Um a lot of people
think that it will

00:57:52.840 --> 00:57:57.397
happen fairly soon. The what
you're seeing now is emergent

00:57:57.480 --> 00:57:59.757
behavior that none of us expected.

00:57:59.840 --> 00:58:04.357
Emergent in the sense that we didn't
know it could do this. Like the math

00:58:04.440 --> 00:58:08.237
example I used. Everyone
believed that you needed a a

00:58:08.320 --> 00:58:12.637
model that was tuned to math. You don't.

00:58:12.720 --> 00:58:16.197
Wow. >> Hi.

00:58:16.280 --> 00:58:19.077
>> This is uh Ernie Hutton, class of '66.

00:58:19.160 --> 00:58:22.757
Uh to take things back to the campus
level, Princeton has just abolished, I

00:58:22.840 --> 00:58:25.797
think tragically, its honor code. And uh

00:58:25.880 --> 00:58:28.077
there's part of a definition of uh in

00:58:28.160 --> 00:58:32.837
within that of what is cheating.
You know, how do you take uh in the age of

00:58:32.920 --> 00:58:36.477
AI uh assimilate knowledge
and still maintain

00:58:36.560 --> 00:58:41.597
the students' ability to be
creative and and uh and learn?

00:58:41.680 --> 00:58:43.197
>> Well, obviously the way you test

00:58:43.280 --> 00:58:47.117
students is without access to computers.

00:58:47.200 --> 00:58:50.597
Right? So, basically you can have a math
class and you can have a lot of fun with

00:58:50.680 --> 00:58:53.917
math. And at the end there's a math test
where you can't have a computer in front

00:58:54.000 --> 00:58:56.157
of you. Right? So, it's I'm sure the

00:58:56.240 --> 00:59:01.357
university can can find ways
of validating the knowledge transfer.

00:59:01.440 --> 00:59:03.117
I'm going to argue

00:59:03.200 --> 00:59:06.437
strongly that the more students use

00:59:06.520 --> 00:59:11.597
these systems in their daily lives, the
better. With an understanding that they

00:59:11.680 --> 00:59:17.157
have to it will Let me say it right. There
are things that they will not then learn

00:59:17.240 --> 00:59:20.757
that they're going to have to learn some
other way. But that's a it's a pedagogic

00:59:20.840 --> 00:59:24.717
institution. The university can figure
that out. But I want Princeton students

00:59:24.800 --> 00:59:28.797
to walk out being so incredibly
comfortable with this stuff that they

00:59:28.880 --> 00:59:31.157
can use it to master whatever personal

00:59:31.240 --> 00:59:34.837
financial business challenge they have.

00:59:34.920 --> 00:59:37.567
>> Okay, I think this
is the last question.

00:59:37.650 --> 00:59:42.807
>> [clears throat] >> Um yes.
>> So make it a good one. No, I'm kidding.

00:59:42.890 --> 00:59:47.837
>> [laughter] >> I'll I'll try no favorite
color question. Um so first off, thank you

00:59:47.920 --> 00:59:50.477
Alan for, you know, creating this center.
Eric, disappointed in the

00:59:50.560 --> 00:59:53.477
seniors that there's no
Volkswagen Beetle on stage.

00:59:53.560 --> 00:59:56.957
Um my question revolves around adoption
right now. So how do you think financial

00:59:57.040 --> 01:00:01.797
firms will balance algorithmic modeling
and human intuition when trading in

01:00:01.880 --> 01:00:04.157
unprecedented >> Financial firms?

01:00:04.240 --> 01:00:06.997
>> Yes. >> Okay. So

01:00:07.080 --> 01:00:09.357
>> If you look at the quant

01:00:09.440 --> 01:00:14.437
sort of revolution which provided
a great deal of efficiency to the markets,

01:00:14.520 --> 01:00:18.197
it sure looks to me like
all of that can be AI driven.

01:00:18.280 --> 01:00:22.317
Uh and there are plenty of of hedge
funds and so forth that are building that.

01:00:22.400 --> 01:00:24.597
Um I have a group uh that I started

01:00:24.680 --> 01:00:29.477
that's trying to do essentially
equity trading using signals. What's

01:00:29.560 --> 01:00:32.997
interesting about it is that it only
takes about 10 people to build a trading

01:00:33.080 --> 01:00:35.557
platform because they're
all they spend all day

01:00:35.640 --> 01:00:39.197
using cloud code. And uh my last meeting

01:00:39.280 --> 01:00:42.677
with them they said, "Well, we're
in cloud code jail right now." And I said,

01:00:42.760 --> 01:00:45.197
"What does that mean?" They said, "Well,
we ran out of tokens this morning. So

01:00:45.280 --> 01:00:48.477
we're waiting until 5:00
when we get more tokens."

01:00:48.560 --> 01:00:50.517
So this is sort of their lives.

01:00:50.600 --> 01:00:53.997
So uh my own opinion is that the vast

01:00:54.080 --> 01:00:58.117
majority of financial
investing will become automated.

01:00:58.200 --> 01:01:01.517
This does not mean that the sales people
will lose their jobs. It does not mean

01:01:01.600 --> 01:01:03.357
that the owners will lose their jobs.

01:01:03.440 --> 01:01:04.957
But a lot of the people who run around

01:01:05.040 --> 01:01:07.677
using intuition to trade are highly

01:01:07.760 --> 01:01:10.677
likely to be replaced by um computer

01:01:10.760 --> 01:01:15.117
with with deeper intuition. It's the same
argument that Basically, you think

01:01:15.200 --> 01:01:18.037
you're such a good investor and you
and there are plenty of very talented

01:01:18.120 --> 01:01:22.637
investors. They're they have
intuition around a state space.

01:01:22.720 --> 01:01:26.317
And computers should be able to develop
a bigger state space that they can have

01:01:26.400 --> 01:01:29.517
the same intuition on.
It's an interesting story is that we

01:01:29.600 --> 01:01:33.637
tried this at Google and we
actually started the program.

01:01:33.720 --> 01:01:37.397
This is early. And it was ultimately

01:01:37.480 --> 01:01:39.797
we decided not to do it because in order

01:01:39.880 --> 01:01:41.477
to begin the trading, the company would

01:01:41.560 --> 01:01:44.797
have to register itself as a trader.

01:01:44.880 --> 01:01:47.917
And Google decided
correctly not to do that.

01:01:48.000 --> 01:01:51.037
Um so I know that people have been
working on it for at least a decade and

01:01:51.120 --> 01:01:56.077
it's coming. >> So please join
me in thanking Eric very much.

01:01:56.160 --> 01:01:57.877
>> Thank you.

01:01:57.960 --> 01:02:03.480
Thank you, Alan.
Thank you very so much, Alan.