Google前CEO施密特：AI競賽已經變成國家權力戰，未來10年最危險的變局來了

Opening and gratitude to Princeton

00:00Let's start with science. >> Why don't I start by saying how grateful I am to Princeton? Um Um for those of you that are graduating, you don't know this, but this is the pivot point for every aspect of success in your life. And that was true of me. Princeton begat Berkeley, which then created Wendy and the family and the money and the opportunities and the so forth. Had I not come to Princeton, this would not have happened. It's binary. That's how important Princeton is to me. So, thank you.

The science of AI and the breakthrough moment

00:45>> Well, I want to start off with the science of AI. Um a lot of us that are ignorant or semi-ignorant in AI, unlike you, are wondering what happened to to account for this burst. I can I'm old enough to remember people in the '60s talking about AI is on the verge, in the '70s on the verge, in the '80s. Now it's here. It's really here. What happened? >> Um to give you a sense of what's happening, uh this week the first this may be one of the most historic moments in human history occurred. Um a proof of an 80-year-old uh theorem was generated by OpenAI, and the paper was written by the computer. The The proof is incomprehensible to anyone here except the senior math people. Um so, I'm just going to accept that the

01:39math people think it's correct. Um and they all think it's really important. The most interesting thing about this proof is it was done with a model that was not trained on math. So, we had a long meeting yesterday. What is What is the thing that it's doing? Right? So, one way to understand the current LLM models is that they're very good at synthesizing a structure of knowledge into a small vector representation space. And these systems can look across that representation space, and they can collect all these facts in ways that humans can't. Many people believe that this is the beginning of artificial superintelligence. These are things that humans cannot do that computers can do that we value.

02:29So, then the question is how do we get here? Well, when I was here, there was a nascent AI program. Of course, there was no computer science. And it was all around what is called symbolic reasoning. And the idea there was that you could build a language that would describe knowledge. Right? This is a sort of a Spinoza model of of knowledge. And many, many people worked on that, and it turned out that it failed. And after that, and it was a hard problem. So, there was something called the AI winter in the '80s and '90s. In the 1980s, at the Institute for Advanced Studies, a bunch of mathematicians proved a theorem that I also cannot describe, which describes essentially deep learning as we know it

03:09today, which is a series of layers of networks that talk to each other. And then a bunch of scientists at various universities, including here, in the early 2000s, built the model, the mathematical model. So, it started with math. Then, right? >> [snorts] >> It wasn't until about 2011 when a bunch of people at Google decided this would be interesting to apply to YouTube. And they watched YouTube, and they discovered something. They discovered the concept of a cat. And I remember it sitting at Google at 2011 saying, like, "You're watching YouTube, and you discover the concept of a cat? Isn't there something more interesting on YouTube?" And the answer turned out that during that period, there was an awful lot of cat pictures

History: symbolic reasoning to deep learning

03:50and movies and so on. People were uploading pictures of their cats, and that's how it discovered it. From that moment, there were a series of innovations, and the most important ones were in 2015, something called reinforcement learning was used by DeepMind to win a game called AlphaGo, which was thought to be incomputable. That was a big deal, and I know cuz I was there. Um 2017, Google published a paper called Attention Is All You Need, which showed algorithmically how you could take the math and do it in parallel on computers. That single paper spawned what you know of today, and then that created the competition, which then moved very, very quickly. Cuz now you've got money and people, and you've got a computer

04:34architecture and a awful lot of computing, and that's how it happened. >> It's an amazing story. I didn't hear you mention the huge increase in computing power. Was that not the big deal? Or what were the big deals here? >> The computing power is essential, but that's true of my whole industry. So here's an example. When I was here, um Tom Wyant is here. I can see a number of other people that I did this with. So we all remember what we were when we were much younger. Um we spent an awful lot of time on something called an IBM 360-91, which is at 87 Prospect, and the old-timers here know what I'm talking about. And this thing was a work of art. It had a raised floor. It had a window cuz they didn't want you near the

Google, YouTube cats, AlphaGo and transformers

05:18computer. Each of these boxes were about 1 megabit. 1 megabit. And it was all water-cooled, and they used something called iron core cuz semiconductor memory had not been invented yet. I personally, or we, owned the console for that 360-91 cuz the rest of it was too big to put in the office. So that's how important that computer was. Uh and we used fed it with punch cards. The punchline is the phone you have in your pocket today is at least a hundred million times more powerful than the whole computer that was on the campus in 1975. Um, and furthermore, the 360-91, there were only three, and they had special floating point because they were used for the NASA moon landing. So, that's an example of how the

Computing power then and now

06:06industry changed in this period. Now, how do I know a hundred million times? That's the GFT the GFTL number um from their calculations of computation in the campus. >> So, is there something analogous with this is an analogy, it wouldn't be the speed of light, but an analogous to the speed of light that's a limit on how fast this can go? Most of us are shaking our heads in wonderment about how how much this is changing so fast. >> Well, first, it's happening much too fast for everybody, including me. Um, I think the following statements are true. Um, computers can now compete with top-level graduate students in basic scientists ba- basic sciences, and almost certainly in writing and analysis, and perhaps in art.

Limits of intelligence and the wisdom problem

06:53Um, how much smarter can they go? Um, most of us believe that they will they, whatever you want to call them, will in our lifetimes be materially smarter in some ways over the humans. They are not going to have the same kind of intelligence as humans, and lots of people have been studying what this thing looks like, and it's not human. Don't be confused. Furthermore, they they make a kind of mistake that's very interesting. They're trained on knowledge that we give them, but there's an awful lot of knowledge that's not written down anywhere. So, I'll give you an example. I'm in a business meeting, and somebody comes up with an idea of doing a rev share, and another person says, "What we should

07:36do is do a joint venture." And I said, confidently in my CEO voice, "Joint ventures don't work, and rev shares don't work. Now, I've never said that publicly. So, how would the computer that's impersonating me know that? He would have to go through lots of failures, lots of disasters, lots of, you know, problems. Furthermore, these kinds of rules are not written down in business books. So, that's I'm using that as a trivial example. There are thousands and thousands and thousands of pieces of knowledge that you all have in your heads that these things don't know and will not or are unlikely to know. Um I'm funding some research in the following. What's the limit? Is the limit is this kind of analytical

08:18superintelligence without wisdom, if you will? And by the way, it's really important because Princeton and, you know, your successors in the faculty and so forth in economics going to use this every day. >> Wisdom is tough. >> Wisdom is tough. Wisdom is really tough. It's hard to get. Um and so, what happens is there's research on how do we imbue wisdom and non- non-human knowledge into these things. The best ideas are unfortunately, you have to teach them and humans have to teach them. So, you have to figure out a way to do the the technical term is get the training data for wisdom. No one's solved that problem yet. >> But go ahead. Please go ahead. >> But I I want to make sure I'm I'm clear

Upside: disease, energy, epochal change

08:58before everyone talks about the negatives. Um in the next 15 years, there are people in my industry who believe and are funding that we believe we can solve essentially all human diseases. Now, that would be like a big deal, right? And that would be like a big deal, right? I mean, let's just admit. That's like we should be happy with that. Um it's highly likely that you'll see new energy solutions that will help address uh the terrible things involving climate change. All right. So, you start going through problems, new materials, uh new chemistry, um so forth and so on. All of those things are within our grasp. And part of the reason that I think it's so important to celebrate this now is a thousand years from now people will

09:40record this period as an epochal change. This is what Henry's Henry's Henry book Henry's Henry Kissinger book with me helping him basically said. This is like the enlightenment at that level because of the scale of what it can touch. Every aspect of human progress has been basically because of science in one form, invention in one form, technology in one form starting with the agrarian revolution. This is the next one and it's moving very fast. >> And will we to take on something that's controversial these days, will we need 10,000 times more electricity and data centers and things like that? I noted the other day and I went like this, Elon Musk is talking about putting data centers on the moon.

Data centers, power and financing

10:30Uh, which seems have we really run out of space on Earth? >> We have lots of space on Earth. Um, thank goodness. Without answering the Elon questions, I prefer not to answer rebut rebut Elon. Uh, so so so let's let's talk a little bit about data centers. Um, my so to some degree I'm in mourning. Uh, when I started Princeton, um, and we'll talk about this again, I showed up uh as a young freshman and I met this professor named Brian Kernighan and a whole bunch of other people and they basically said you're a computer scientist, there's no computer science, we'll just work on we'll we'll just fake it, right? That was the beginning of modern computer science. As I learned computer science, it is over.

11:24When I watch these things write code, I go, holy I mean, I was good, especially as a 21-year-old and my my ego was consistent with my attitude. I was really good. These things are so much better than I ever was. That's a big deal. Because it empowers people to do the kind of scaling that I've done in the last 50 years. Now, what happens with data centers goes something like this. First, the White House and the government should be very happy because the data center build-out has added 1% of the GDP growth to America. That's a big number. And it's also changed my industry from being high gross margin, low capital cost to being very high capital cost. So, now what you're seeing is this enormous funding of data centers and the

12:15numbers are incredible. Give you some I happen to have a data center business, so I know the numbers now. 1 gigawatt of power is about 50 billion dollars. The The industry and I've testified in Congress to this, the industry estimate is it needs somewhere between 90 and 100 gigawatts of power by 2030. For reference, the average nuclear reactor, which we don't make in America largely, is 1 gigawatt. All right, that gives you a sense of the scale of the problem. There are plenty of people who are working on reducing the power costs. There's something called Jevons paradox where when you lower the cost of something, you just use more. So, in my community, even though all of those all of that work of efficiency and

12:58lower cost is welcome, we still believe, whether we're right or not, we still believe this is going to happen. Why is this important? What's the limit? Well, data centers should not be near cities because they annoy people and that that's not fair to them. They shouldn't use a lot of water. They don't now as much. They need to be in remote spaces. What's true of America? Lots of empty space. Do you know why? You fly over it. Just look down. You will see it. There's large large plots. It's a great There's an old De Tocqueville quote about America that America is successful because of the the abundance of land, the richness of the spirit of its people, and the absence of a king. This was 1750. Right? So, think about it.

13:47So, so that land bank, if you will, plus the American financial system allows us to do crazy things. I mean, I was talking to this guy who says he's got 15 people. He's raising at a 4 billion valuation, and he's raising $400 million today. He's only The company's only 6 weeks old. He's already oversubscribed. Thank you. I want to thank everyone here who works in Wall Street, who's invented this system. There's no place in the world that can give crazy people this amount of money. Right? And and furthermore, some of the smarter of you are beginning to write articles saying like, "Will we ever make any money?" And the answer is, "Hell if I know." Right? But, you know, you're the guys who gave us the money.

14:38So, it's okay. We'll try. So, in any case, to answer your real question, the data center limit is actually not physical. It's financial. You can build the data centers with your own power systems. So, all of the new data centers are essentially The term is called behind the meter. They're off the grid cuz the grid can't support it. And it's not fair to people to raise their electricity rates. The interconnection queues, as they're called, are now full for 7 years. So, all of the data centers that you hear about will be off the grid. There is an issue with the climate impact of these things cuz these are combined cycle turbines. Everybody understands that that's not good. And the And the best solution we've come

15:24up with is to build fully renewable data centers, which is which we're working on, which is hard but possible. >> But don't they have to have an ultimate energy source? >> Yeah, you want >> Electricity doesn't just flow in. >> Well, that's right. You use either natural gas with a combined cycle turbine, which are which you can purchase, and these things last for 20 years. And you can also use solar and batteries. >> Is solar practicable for something that's an enormous energy user? >> If you have enough land, look out the window. Um right? Think Texas, New Mexico, Arizona, all those nice big empty places. >> investment strategy is to buy land in Nebraska right now. >> Um I recommend buying all basically

16:06in places where you can get these things permitted. This is the new If you're a real estate person, this is where all the growth is going to be. If you look, all the real estate capital has moved from normal building to data centers. That's a mistake, by the way. We need normal buildings, too. >> Does any of this call in your mind for for sensible, and that of course is a big word in this sentence, regulation of the industry? Let me give you two uh analogies. There was a lot of discussion in the early days of the internet about regulating and not regulating it. And we basically opted for let a thousand flowers bloom and let it just go where it goes. Same And And that's widely looked at, I believe, as a sensible

Regulation, outrage and free speech

16:58resolution to what looked like a difficult problem. When it came to social media, we adopted the same strategy. And I think, and it's not just an old codger like me, I've talked to some of our students about this, there's at least a reasonable case there's more bad than good that has come out of that decision. So, you have one on one side, one on the other side. What do you think about that? >> So, I've come to the conclusion that this all of this is going to get regulated. And it's going to get regulated in different ways for different reasons. Um there's no question that the there's a misalignment of societal interest and business interest. And it goes like this, the best way to maximize revenue

17:44is to maximize outrage. The best the best way to max because that that essentially causes people to participate more. And the those businesses are in a business to fight for all of your attention. So, they're winning while you're obsessed with somebody something and unhappy. That's a simple way of understanding it. And if you get confused about this point, watch any movie from the 90s. Any you like a normal movie like how men and women work together and and how politics works and so forth. And you'll you'll remember, right? You know, the you're in your 25th your reunion. You know what I'm talking about. >> [snorts] >> Um and I mourn that simply because of the nature of the conversation, the quality of the morality that we operate.

18:33Because a system where the incentives for communications are around outrage produces outrage on all sides. That's a simple way of understanding what has happened. Now, I worked really hard with a whole bunch of people including Jonathan Haidt on how to solve this problem. It's very hard to solve it without putting restrictions on speech. So, there is a fundamental problem of democracies in these technologies, which is that they can be used to to violate or to promote particular non-democratic norms um for business interests, right? This is true of all sides. I'm not trying to make a political statement. I'm trying to say it's just true. And that has to be addressed. Now, if you look around the world,

19:14countries [clears throat] have very different attitudes towards freedom of speech. We used to have We used to be at one of the extreme. I'm not sure we're still at the extreme. That's another story that I'm not going to ask you about. But we used to be at one extreme. China, Russia, and a number of are at a different extreme. And if restricting free speech is part of the recipe, does that actually give them paradoxically a leg up rather than a leg down? >> Well, we know a lot about how China operates because China is our natural competitor. Notice I didn't use the word enemy uh in this space. And they solved this problem by dictatorial regulation, right? And it's brutal. Uh but that's what their government

China competition and chip restrictions

20:00chose, which is obviously not a democracy. So, they solved that problem. But we're clearly not going to do that. >> Is that one of the reasons we're ahead of them now? >> No, we're we're ahead of uh China for a number of reasons. The most important of which, the US government uh under Trump 1 and Biden and Trump 2 have all restricted sales of hardware chips to China. These are There's a particular threshold. >> Mhm. >> Um and the threshold involves a particular manufacturing technique and a particular manufacturer called ASML. And I had thought, cuz I was one of the authors of this program however many years ago in the AI Commission, that it would give us a 5 to 10-year uh advantage. And that was my technical calculation.

20:45And of course, I was wrong again. Um it looks like China's within 6 months of us now based on the Deep Seek V4 result. And you can read that. Just look up the evaluations of Deep Seek V4. Chinese people are very smart. They work really hard. Their work ethic is incredible. And even with reduced resources, they've been able to get caught up. The President Xi and their AI program is slightly different than ours. Um the their goal is to imbue AI in everything. And again, that's subject [clears throat] to their laws, which include the draconian free speech issues. So, everything has AI in it now, whereas that's less so in America. Furthermore, in China, there's huge support for AI, and as you know, in the

21:27US, there's huge non-support for AI at the moment politically. So, but but the answer about China is one way to understand the evolution of AI is that you need a large country full of an awful lot of the things that the US has. You need the money. You need the creativity. You need the hardware. You need the software. You need the audience base. You need the monetization. China has that. Very few other countries have the scale that is required. Uh if Europe ever got its act together and operated as a single country, they could pull it off, but they don't do that. India's a possibility. Most of the other countries cannot get there. >> In many economics lectures in this room and other rooms, we teach our students

22:09that especially when it comes to innovation, capitalism has a gigantic uh advantage over state intervention. Is Are we looking at something that could be an exception to that? I'm playing off what you're saying about China. >> Well, it depends on whether you ignoring the political comment, China when when I'm in China, it feels very capitalistic. Um they they refer to themselves as um communists with Chinese characteristics, which I conclude is capitalism. So, the the people that I work with are just like the American entrepreneurs that I work with. They're crazy hard working. They're intense. They're very ambitious, very self-confident, you know, all that kind of stuff. Uh and they work in a a system

23:00where they've largely been left alone, I think that that's likely to end. Um give you an example, China just announced a restriction on DD, which is their Uber equivalent, to not allow them to do self-driving cars because of the obvious impact on employment. So again, one of the problems with an authoritarian structure is you can never quite figure out which priority is right, whereas at least in the market you can predict it. I'm I'm obviously a capitalist. So the fact and I had dinner with a number of my Chinese friends and they were in their usual, you know, this and that and the complaining and so forth. I said, "Look, read the rhetoric from your government. They're not capitalist." You just read the documents.

23:46So I think that my friends, maybe they were caught in some capitalist bubble, but they sure made an awful lot of money. All right. And now they're trying to get it out of the country. >> Which which is not so easy. >> Yeah, so to be clear, I I much prefer American free speech, freedom, and capitalism for obvious reasons. I think it created the success we see in the room. Uh on the free speech question, I did want to say that I tried to I'm in favor of free speech of humans, including crazy people, of which there seem to be an increasing number every day. Um I am not in favor of boosted speech having the same rules. So a simple way of thinking about this is um here at Princeton we have all sorts of people. We do we have tolerance

24:32for different ideas. I'm in favor of all of those ideas even if I and it's perfectly fine for someone to sit uh in the equivalent of McCosh Courtyard and scream that for a while. Um when they get a bullhorn and then they start spreading it through all the dorms from that single bullhorn, that's a violation of somebody else's rights. So there is some limit to amplification of one's message. That's the way to phrase it. I'm not a a legal expert, but there's got to be a way to say it right. >> We hope so. Let me ask you a slightly related question, maybe less a little less cosmic. As I read it with my inexpert eyes and ears about China versus the US, it seems like China's going much more for closed source, and we're going much

25:24more for open source. >> Inverse. >> in >> The inverse. >> Okay. I said it the backwards, sorry. So, the question was to be, is that true? And with the correction, it's true.

25:41And are And are we on the Are we on the right side of it? >> Um I think not. >> You think not? >> So, so let me give a For those of you who haven't been following, this is a really important distinction. Um the top models, the ones that you use, are all run by for-profit companies. And they have lawyers and CEOs, and they have shareholders or will have shareholders. And they take an enormous amount of capital. We're talking about uh hundreds of literally, well, hundreds of billions of dollars at this rate. Now, they properly want an economic return, and so what they do is they keep their software their weight Technically, the weights and the software proprietary. You can't borrow it. They don't give it to you. You can't get

Open source versus closed weights

26:29access to it. Um why is this so important? The entire industry that I was part of, which started here at Princeton, by the way, was based on an the other side principle, the open source principle. We will write software, and we'll give it to everyone. And the lawyers all would go, "You're giving away your intellectual property?" I said, "Yes." And the reason is that in the same principle as if you want to go fast, go alone. If you want to go far, go with a crew or I go with people. It's the same principle. So, what happens with open source is that you put the stuff out there and then thousands of people strengthen it. That's the magic of open source. That's what I did and what I've done for most

27:09of my life. It's important to note, for example, that Google was based on open source architectures. So, this is another big change in my industry. Now, let's think about China. So, China, which again is highly regulated, uh decided to do something completely different. All of the top models are open weight and open source, which means you can take the source code and you can take all the numbers, which is the weights are the thing that the computer uses to reason, and you can do whatever you want to with it. So, I have startups in California with have a lot of Chinese uh speakers who prefer the Chinese models over the American models. So, if you ask them, they say, "Well, it works really well.

27:49Oh, and by the way, it's free." Now, you sit there and you go, "Hmm, okay." Now, at Princeton, how would a researcher here get into this? Well, they'd have to work for, you know, how does a graduate student work for them? So, ultimately, about 10 years ago, Google, through me, created a Google center here on the other side of Prospect, right? So, that primarily, you could actually walk from here to the Google data centers, logically. That's not a scalable strategy. So, there's a problem with open source leadership in America. Why is this so important geographically if you don't care about that? It's because the majority of the models in the world will be Chinese. Do you know why? Cuz they're going to be free.

28:33It's not that complicated. Uh we were at dinner last night with my friend who's quite wealthy, and he was joking that he was using one of the American models, and he calculated that he was doing it for fun. Um and he calculated it was going to cost him a half a million dollars a year for his entertainment, which consisted of building a newsreader that was biased on his political views. You know, it's perfectly fine. Uh, so he switched to a Chinese model to to read American news cheaply. So, it's real. >> So, that's hot tough competition. >> Yeah. Chinese are going to be very tough, and I I'm very concerned, and I want to be clear to everybody here. Most people here are US citizens. We are not meeting the

Superintelligence and national security

29:17competition and beating them globally. They are investing in science and technology and innovation at a rate to overcome America. What would it be like if if China were the winner in all categories? AI, quantum, so forth and so on. Start thinking about conflict. America is always seems to be in some kind of war or not. Uh, we're getting ready over Taiwan. Right? Can you imagine, here's the thought experiment. So, there's something called a superintelligence, which is the point at which it's smarter than all of us. Uh, many people in my industry think that this will occur within 10 years. We can debate this. Nobody really knows. Let's imagine they get there first. And as part of that superintelligence,

30:00they figure out some attack vector against America that we cannot imagine right now. And and by the way, the inverse is true for us. So, this is big deal. If you care about national security, you need to fund this. If you don't care about national security, you care about social welfare, you need to fund this. If you care about economic growth, which I think all of us certainly do, uh, you better fund it. You better figure out a way to keep the the engine that got me to where I am going. Right? >> What you said before come out what you just said now makes me think of the electricity grid. I mean, all all of this is ultimately on on electricity. And it is is that or might that become a vulnerability in a case like that? You

30:49don't really need to send airplanes with bombs. >> So first place China last year built 130 gigawatts of renewable energy. Okay, remember the number I said is that we needed about 100 gigawatts between now and 2030 in America. They already have it. Right? And there's estimates that they have 20 gigawatts just sitting around. Right? I I don't know what the real numbers are. I'll try to find out. So assuming that these data centers continue to be these large things, they become military targets. I wrote a paper which we called um the argument went something like this. This is influence of Kissinger. Um you start thinking about what happens when we're getting stronger and China's getting stronger or whoever.

31:37Um there's a point at which one side could decide to do a preemptory attack of the other, which is your question. Doesn't it make sense to slow down your competitor by just bombing their data centers? So we being pleasant Princetonian type graduates said there's a better alternative, which is to just give the enemy a switch. So they don't have to bomb the data center, they just have to turn it off. And everybody goes, that's the stupidest idea I've ever seen. If you believe that these data centers are essential for the national security of for a nation, you've got to come up with some kind of a deterrent strategy. In that sense it is similar to nuclear. >> Which is cross your fingers and hope

32:24that this there's symmetric forbearance. >> Well >> Which worked with nuclear. >> Well nuclear worked because of a theory of mutual assured destruction. It's and it's important to know, I wrote a book about this, that mutual assured destruction has never been tested. And I'm really glad. >> Yeah. >> But let's start with an assertion. It's never been tested. We don't know what will happen. Uh we don't know what will happen if there is a third nuclear bombing of any kind having the US having done the first two. We just don't know. Um and we're paying we're playing with incredibly powerful stuff here. So what I One things I've learned is that this is not a kid's game. They They're these huge organizations that

33:11worry about safety and security and biosecurity and so forth. One way to understand the threats of models there's the threat to democracy which everybody here understands. There's We've known for a long time that the first things it would be able to do would be cyber attacks. And the reason is that if you can do good software program, you can also write good cyber attacks. When Methos came out, which is a couple months ago, this is the Anthropic cyber cyber attack model. They had the good judgment of only releasing it to about 40 companies as well as the US government. By the way, this is during the time when they're having a big fight with the Pentagon and the White House and they get banned while they're

33:52essential for the use of the NSA. So it's like crazy. Um and I'm sure that will get resolved because they they ultimately have to work together. But if you think about it if you're the head of the NSA or in that group, you want these tools for offensive cyber attack and you also want to be able to use these tools to prevent defensive cyber attack. The term in the industry is called offense dominant versus defense dominant. Does Are you more likely to win with offense or defense? My assessment is these tools are offense dominant. So we need to be very careful that we have a strong offense as well as a strong defense. That's my personal view. >> Let Let turn to something a little more less cataclysmic, but that a lot of

Job displacement and robotics

34:34people worry about, which is job loss. >> Yeah. >> As opposed to job change, or both, or uh is one bigger than the other? Do you do you spend any time worrying about massive unemployment as a result of AI? >> Uh I do, but not now. And I'll give you the numbers. Um we know that the current stuff in the US is displacing low-level customer service jobs, and we know it's displacing software jobs, typically the lower-level, entry-level jobs. Uh which is a big deal for those people who are displaced. We have not yet seen mass effect of the white-collar workforce. It's going to happen in the next 5 years. And here's how it happens. Um there's a a very good article yesterday by a friend of mine who runs

35:24Cloudflare, who wrote a very nice explanation that he runs a company which is growing at 30% a year, has huge valuations, and so forth, and he did 20% layoff. And he justifies his layoff by looking at the categories of workers, and he doesn't need this set of categories. And those specific categories are not sales, they're not engineering, they're a bunch of the infrastructure. So, your decision you should decide whether you think those people can be retrained or get new skills that are more consistent with what what AI cannot do well. Um I know one way to say this is that this is the first technology that's really threatened white-collar work, and because white-collar people write really well, they write about it all

36:08day, right? Right? They write about their their experience. 2/3 of the jobs in America are essentially labor human labor-centric. They require hands or pressure or presence or so forth. Yeah. And these are important jobs. So, as long as robots don't really work, the impact will be manageable. The fear I have is that in 10 years the robotic revolution, which is much slower than America and software, will in fact begin to displace those jobs. If that begins to occur, now this is a long time from now, we're going to have a whole hell of a lot of problems, right? For obvious reasons. Um and why do I say it's going to take that long? Um the US robotics industry is relatively slow and relatively expensive. Um in China they have a I'll

37:00give you an example. They have a company called Unitree, um which builds an a robot called R1, which I've ordered for $5,000. Look at the videos of what it can do. So, that's what's coming. Now, it's dumb as you could imagine. The thing needs a proper brain, right? Whatever whatever that means. I mean, it's like the stupidest robot you could imagine. Um but that can be fixed over time. That's I think the real concern. >> Take you back before the robots. >> [clears throat] >> The the good side of of the answer to this question that's been around since the Luddites is that yes, the new technology will destroy some jobs, but it will also create >> Yeah. >> uh a lot of new jobs. So, it's mostly about job change and disruption and

37:50moving resources from here to there. The United You can tell me if you agree with this or not. I would say the United States among the rich countries is almost uniquely bad at managing that transition. Retraining workers, supporting them while they're unemployed, uh etc. etc. Making the transition easier rather than harder. So, do you do you see it that way? That That's the answer to the on the warrior side of it's only a job change, it's not job destruction. >> Um the benefit the counter argument is that the benefit of the brutal nature of American employment is that it's a reconfiguration into a new into a new role quicker, which is part of why American productivity is better than Europe for example and there are debate

38:41technical debates about that. Paul Krugman for example has been arguing against that point. Um What do I really think? Um >> Yeah, what do you really think? You're a You're a very good thinker. >> I think I I'm That was a That was a strategic pause. I I'll give an example of how crazy things are in my little world. Um there was a paper and again, you're the economist so I I believe it's a fair assertion that there are no economics for economies that are doubling every year. There are no examples of doubling GDP. >> a whole side of the economy, not >> Yeah, doubling. Okay. So anyway, this group managed to actually come up with a set of math which I've no way of validating that if you replace all labor

39:34by either AI or by robots. This is the presumption. It's obviously not a good thing for other reasons. You could grow the economy by a fact You could double the economy every year. Okay. So I thought highly unlikely to be true. But let's start thinking about that. So if it Okay, so here we are and now everyone's twice as wealthy and then twice as wealthy again and twice as wealthy again. You all know you can do combinatorial math. So then would you be happy if everyone here was as rich as Qatar? You'd be happier. All right? Just think of all the Rolls Royces and the equivalents. Um what if you were 10 times happier than Qatar? What if you were 100 times happier than Qatar? Um and what if it were true for the whole world or at

Abundance, growth and inequality

40:22least the countries that adopted this? So, there is an argument, which I'm not personally making, that the abundance and the the elimination of scarcity, right? So, the the basic argument from the accelerationists is that we're eliminating scarcity. Every day, every aspect of history, every aspect of economics is fundamentally about scarcity, scarcity pricing scarcity, markets over You understand this very well. Um what happens if we remove all of those constraints? No one knows. Now, let me give you a counter and a further counter. The US is growing at 2 to 3%. 3% is considered high. Chinese growth at in its heyday was 10%. It's now down to 2 or 3% and they're very upset. Let's imagine that all of the countries that

41:09we care about could grow at 15% a year. How would that feel? Because if the slope is what I just said, and that's a lot of ifs, relax. Okay? If the slope is not that, don't relax. If the slope is not fundamentally changed by the business improvements, economic improvements, the scientific improvements, the quality of life improvements that I've described, then we have screwed up collectively as a society. >> The slope is very few people doubt that the slope is in the direction that you say. 15% a year, I don't think you'll get many people signing on to that. But, there's a question of who owns the resources. >> Mhm. >> Uh it it's not going to be equally distributed. >> That's right. >> In any conceive We can't even conceive

42:05of a scenario which it would be equally uh distributed. So, there going to be people left behind. Now, a reasonable answer is the society will be so rich that it can take care of the people that are left behind. My suggestion 5 2 minutes ago was this country is not very good at doing that. And that's that's where the worry comes from. >> I understand. That is essentially a political decision for a rich country. So, the most likely scenario is that a smaller number of people still a large absolute number, but a smaller number of people get more and more of the economic share of America because of this technology. >> Yes. >> That That is a reasonable presumption. If you look at the tech market If you

42:55look at the market today, all the top value is in the tech tech sector. Furthermore, the stock market's gone up for the last 2 years basically because of all of the the tech companies, the leading tech companies. So, the amount of wealth creation is similar to that of the Golden Age, and I'm obviously happy to be part of that, and happy We're happy to give that money to Princeton. Um the the it's not good for society to have a Gini coefficient that's that imbalanced, right? For obvious reasons. That's a political decision. But, I think we it's obvious to me that the gains will not be uniformly distributed. It's It does not then follow that the bottom is still poor. You can imagine a situation There's a

43:36lot of evidence that the internet ultimately took people who were very, very poor and made them a little bit less poor. Because they had larger markets, better knowledge, and so forth. So, it's possible that the that that the poor do better and the rich do much, much better. That's a political decision. We need to get ahead of it. >> Let me ask you a related question, and I'm going to open it to the audience pretty soon. There are some technologies that lead to a kind of a winner-take-all market structure and some that don't. How do you see this one? >> So, this the question of concentration at the technical level is something that we spend a lot of time thinking about. For a long time I thought

44:21that there would only be 10 computer companies. There would be four or five in the US, two or three in China, and a couple of others, maybe one in Israel, maybe one in Europe, maybe one in India, something like that, on the order of 10. And the reason was that the capital cost of these things is so large. How many can you have? Right? I mean, there are limits even to the craziness in my industry. You know, how many trillion-dollar companies, individuals, businesses can you have? It's just the the money doesn't add up. It that looks to be true, but there's then going to be a pyramid of very powerful, much less expensive choices. Then there's a debate in my industry as to whether these lower power

45:09performance things will be the technical term is called distilled. Will they come from the big models or will they be built in their own way? We don't know. Most people believe that I talk to that the vast majority of the AI that you experiment that you experience in your future lives will not actually be from the largest models, but rather will be embedded in your daily life and you won't even know it. Like if you think about it, do you think very much about the operating system that's on your watch? I do. Cuz I used to work on operating systems here. Okay? I'm like the last one. Um you take it for granted. And if you think about 100 years ago, electrification, people like, "Oh my god." And there was all sorts of issues

45:57and houses would burn down and electricity fires and all of that. You don't worry about that today. You worry about, you know, Google and Amazon and OpenAI and all of that. Um so so societies work these things out. So I I think in that sense we'll be fine. But we'll get there. >> Thank you very much.

Audience Q&A

46:30Now If you If you have a question, Kathleen is Dana Is it just you? Kathleen and Dana are around with microphones. Raise your hand. And when you get the microphone, identify yourself please briefly. And And keep Keep it to questions, not speeches. >> We better have some some graduating seniors ask questions as well. Go ahead. Yes. >> Hi Mr. Schmidt. Thank you so much for the talk. Uh Tiger Gao, class of '21. Uh Do you think large language models, which are by construction a next word predictor or next token predictor, will be the model architecture that take us to artificial general intelligence or superintelligence, or do you think we need another model architecture? And secondly, what do you think is the probability of doom,

47:17P(doom)? I think if you ask If you poll Anthropic, the median response rate was probably like 25%, which is high as a Where are all your friends thinking? >> On the latter, I'm not I'm not a fan of that whole process and I don't agree with it and I don't believe in it. Uh on the former Um the the Ask the question again. The The first question was Do we need more things to get to AGI? >> Does LLM by construction prevent this? >> So For those of you who don't understand what an LLM does, it's just predicting the next word. And you go, "How could it be?" As a story, there was a group at OpenAI and I've talked to them at length. It was a Thursday night, and they had just turned on They had GPT-2. The The

48:05The The T is transformer. They had taken that from Google work. And they're busy and they'd done a GPT-2, and then they put more training into it. And they turned on GPT-3, and they realized that it could write. And that was a real discovery. I've tried to figure out why it's so good. And the answer is that it's similar to the way our brains work in the sense that do you really know what word you're about to say or think? It just occurs naturally. So, in that sense, it's very similar. The question to you is is that going to be enough? There are, of course, two schools. The people who are building large language models think it's going to be enough. I personally think it's not. And the reason is this this implied

48:50intelligence that I was describing. There's just a whole bunch of knowledge that you have in your head by virtue of coming to Princeton, being a good person, being a citizen of the world that they just don't know and they're not going to get unless you tell them. Yeah. Yes, sir. >> What is your viewpoint on the relationship between AI and consciousness? Do you think at some point AI will become advanced enough that it will be indistinguishable from consciousness? >> I'm in the no category. Um so, I of course came to Princeton, and I met with the neuroscience group. And the neuroscience group I said to the neuroscience, "How does consciousness appear?" And their answer was the experimental evidence, and I If I misquote them, I'm

49:37sorry. Is that when you get disparate systems in biological brains, and the disparate systems are different enough and are forced to interact enough, consciousness appears to appear show up because of the need for awareness of what's going on. That's the simplest explanation I can give you. Do I think that's going to occur with these? The answer is no. And the reason is that they're not biological. But I could be wrong. >> There's one over here. >> Frank Langford, class of '86. What policy prescriptions would you give to the leaders of this country over the next decade to prepare for white-collar job loss and to prevent the kind of political backlash that we've seen from the offshoring earlier of blue-collar

50:26jobs to countries like China? >> Um I think the first question is can we get the young people ready? And the simplest prescription for that is to have a mandatory freshman class at Princeton along with every other university. When when we came, we had to have a TB test, we had to do a swimming test. Well, it seems to me that we should have an LLM test. And because if you think about an 18-year-old today, LLMs will mediate every aspect of their lives for the rest of their lives in some form. So, learning how to work with them, learning how to empower them, and so forth. That's the first answer. Then the second answer is to try to get people to understand how that how these tools can enable them to make more money

51:15at work. So, um again, last night at dinner, one of the businesses announced that their productivity per human is so high, they're hiring more humans. So, I'm I'm not really fully answering your question cuz I'm avoiding the hard questions. What do you do with the people who can't do one and two? Well, the answer for them, if you look at the history of Luddites and so forth, is they lose their jobs. And then that becomes a political problem of uh retraining, motivating, and so forth. In our current political climate, I have to assume that they will take up pitchforks. Right? In other words, that when these waves go and I'm not trying to be negative here, I'm just trying to be honest. When these waves go through

51:56society, the evidence is they go through much more quickly than society is ready. And so the government needs to be ready for this. Um this the current administration started off with a completely libertarian view of this and is now beginning to propose regulations and restrictions, which must be very hard for them to deal with the worst examples. And I think that's the future. And you'll see it it will come naturally because of the political pressure. >> Who have you got? >> [clears throat] >> Hello. Um this is Karthik Shastri from the class of '16 and also the economics faculty. And I wanted to return to a question that Alan asked about lessons, if any, from the social media revolution. And And you mentioned

52:40yourself that in many ways we live in in a culture of magnified outrage. It's arguably a negative feedback loop between the economics of social media, which can do kind of amazing community building things and make a lot of money from clicks and engagement and outrage to society and kind of back and forth. Now take AI, which is capable of genuinely amazing um intellectual advances, like proving mathematical theorems, as you suggested. It's also capable of other things that may maximize more engagement, if that becomes the model, like using its mastery of language to, you know, surreptitiously advertise things to people and change how they think or to give a veneer of technical credibility at a coherent

53:18>> Sorry. >> Um the question of coherence of logic to the things that that um you know, aren't really serious. So So do you see that as a trap essentially for the development of AI? Um and if so, kind of socially, politically, economically, what could we do >> Oh my god. >> to to sort of forestall that outcome. >> So So LLMs are a neutral technology in the sense that all technology is essentially dual use, good and bad, military, civilian, and so forth. It's up to us how we decide to limit these things. There is no question that if I were truly evil, I could use these tools to completely manipulate um some society. I'll give an example. In one of the first uh safety demonstrations, the UK is the leader

54:00here, strangely, not the US. Um somebody said, "Let's build a uh community around some particular British political thing." And so they construct a set of fake profiles. They completely fill all of social media with them, and they sit there, and they don't do anything. They don't do much, but then in a coordinated attack, they actually then um spread something which they know will get magnified. So, remember that the and I call this the dual evil problem. So, for purposes of argument, I have a uh soft a piece of software that magnifies everything. Well, I can build another piece of software that can figure out what that software is doing, and then I can feed whatever evil idea I have, have this

54:44software feed it into the thing that magnifies, and off it goes. Now, the social media people will say that they have good defenses against this. There's plenty of evidence that they don't. So, that's from my perspective, that's a technical problem, but it's also a regulatory problem. The technical problem is making sure that I cannot come in through an illegal or mischievous path where I'm a foreign agent, you know, Russians or something like that. Russians have a long history of trying to manipulate our elections. This is how they would do it. Um we have to deal with that, but it requires some constraint on the on the economics of social media, some of you understand well. >> Uh Thank you. Uh my name is Jorge Aguilar,

55:24I'm class of '06. You alluded to several safeguards. One of them was that free speech is good, but bullhorns are can be bad, which sounds like nuisance law, right? And something intangible and annoying. But, the other safeguard that I thought I heard was, and I'm asking you to confirm, was you said LLMs are so dependent on the information we give them. And you mentioned how there was something, and I forget what it was, that you hadn't given the LLM, so it couldn't predict what you said. Is there some thought then that a safeguard is there's some information we shouldn't give to these AIs, cuz I think I remember you you then went on to say how advanced it could be if we keep feeding it information.

56:06Is there a a filter that we need to kind of set up? >> I think that's unlikely to work, because it would require coordination with all of the people who are busy feeding these LLMs with everything they can find. The current state of the industry is that every single piece of information that is essentially searchable, collectible, or scrapeable has been collected. Right? We sort of run out of training data, as it's called. And there was a a little while where everyone was writing these articles about data was the new oil, and we're running out of it, which is completely false. Because the systems can also generate synthetic data. Furthermore, there's plenty of data that you can get if you start, for example,

56:48surveilling people. I'm not arguing in favor of that. Um that's not an argument. Do not get me wrong. Um so there's plenty more knowledge that we can feed to these things. Um the the industry is governed by something called scaling laws, and every 6 months I ask the same question, are the scaling laws stopping? And the answer right now is no, which is unbelievable. And so this has led to the following phenomena. Uh a model costs 200 to 500 million dollars, you release it, and it has a life expectancy of about 6 months. And then there's another one. And the competition between the leading players is so brutal, right, that that's they literally count in the number of weeks value of these things, which cost

57:39hundreds of millions of dollars. So, that's where the pressure is. And they're they're they're go I don't know what there is a slope, right? There is eventually a slope. We don't know where it is. Um a lot of people think that it will happen fairly soon. The what you're seeing now is emergent behavior that none of us expected. Emergent in the sense that we didn't know it could do this. Like the math example I used. Everyone believed that you needed a a model that was tuned to math. You don't. Wow. >> Hi. >> This is uh Ernie Hutton, class of '66. Uh to take things back to the campus level, Princeton has just abolished, I think tragically, its honor code. And uh there's part of a definition of uh in

58:28within that of what is cheating. You know, how do you take uh in the age of AI uh assimilate knowledge and still maintain the students' ability to be creative and and uh and learn? >> Well, obviously the way you test students is without access to computers. Right? So, basically you can have a math class and you can have a lot of fun with math. And at the end there's a math test where you can't have a computer in front of you. Right? So, it's I'm sure the university can can find ways of validating the knowledge transfer. I'm going to argue strongly that the more students use these systems in their daily lives, the better. With an understanding that they have to it will Let me say it right. There are

59:15things that they will not then learn that they're going to have to learn some other way. But that's a it's a pedagogic institution. The university can figure that out. But I want Princeton students to walk out being so incredibly comfortable with this stuff that they can use it to master whatever personal financial business challenge they have. >> Okay, I think this is the last question. >> [clears throat] >> Um yes. >> So make it a good one. No, I'm kidding. >> [laughter] >> I'll I'll try no favorite color question. Um so first off, thank you Alan for, you know, creating this center. Eric, disappointed in the seniors that there's no Volkswagen Beetle on stage. Um my question revolves around adoption

59:55right now. So how do you think financial firms will balance algorithmic modeling and human intuition when trading in unprecedented >> Financial firms? >> Yes. >> Okay. So >> If you look at the quant sort of revolution which provided a great deal of efficiency to the markets, it sure looks to me like all of that can be AI driven. Uh and there are plenty of of hedge funds and so forth that are building that. Um I have a group uh that I started that's trying to do essentially equity trading using signals. What's interesting about it is that it only takes about 10 people to build a trading platform because they're all they spend all day using cloud code. And uh my last meeting with them they said, "Well, we're in

1:00:40cloud code jail right now." And I said, "What does that mean?" They said, "Well, we ran out of tokens this morning. So we're waiting until 5:00 when we get more tokens." So this is sort of their lives. So uh my own opinion is that the vast majority of financial investing will become automated. This does not mean that the sales people will lose their jobs. It does not mean that the owners will lose their jobs. But a lot of the people who run around using intuition to trade are highly likely to be replaced by um computer with with deeper intuition. It's the same argument that Basically, you think you're such a good investor and you and there are plenty of very talented investors. They're they have intuition

1:01:20around a state space. And computers should be able to develop a bigger state space that they can have the same intuition on. It's an interesting story is that we tried this at Google and we actually started the program. This is early. And it was ultimately we decided not to do it because in order to begin the trading, the company would have to register itself as a trader. And Google decided correctly not to do that. Um so I know that people have been working on it for at least a decade and it's coming. >> So please join me in thanking Eric very much. >> Thank you. Thank you, Alan. Thank you very so much, Alan.