WEBVTT 00:09:58.480 --> 00:10:00.640 Mhm. 00:12:00.480 --> 00:12:02.597 >> Good evening and welcome to the John F. 00:12:02.680 --> 00:12:05.317 Kennedy Jr. Forum at the Institute of 00:12:05.400 --> 00:12:08.877 Politics. My name is Amo Gyantah and I'm 00:12:08.960 --> 00:12:10.997 a freshman at the college studying 00:12:11.080 --> 00:12:13.637 computer science and economics and I'm a 00:12:13.720 --> 00:12:17.157 member of the JFK Jr. Forum Committee. 00:12:17.240 --> 00:12:19.637 Before we begin, please note the exit 00:12:19.720 --> 00:12:24.917 doors which are located both at the park side and the JFK street side of the 00:12:25.000 --> 00:12:28.197 forum. In the event of an emergency, walk to 00:12:28.280 --> 00:12:30.997 the exit closest to you and congregate 00:12:31.080 --> 00:12:33.677 in the JFK park. Please also take a 00:12:33.760 --> 00:12:37.517 moment now to silence your cell phones. 00:12:37.600 --> 00:12:44.440 And join me in welcoming Harvard College undergraduate Matteo Cagliardo. 00:12:47.280 --> 00:12:50.197 Good evening everyone. It is my pleasure tonight to welcome you 00:12:50.280 --> 00:12:53.677 all to the John F. Kennedy Jr. Forum at the Harvard Institute of Politics. 00:12:53.760 --> 00:12:56.677 My name is Matteo Cagliardo. I'm a junior at the college studying applied 00:12:56.760 --> 00:12:59.197 math and computer science with economics. I'm a member of the John F. 00:12:59.280 --> 00:13:01.837 Kennedy Jr. Forum Student Committee. 00:13:01.920 --> 00:13:05.517 We gather tonight at what is arguably one of the most pivotal moments in human 00:13:05.600 --> 00:13:09.757 history. Artificial intelligence has evolved from being a futuristic concept 00:13:09.840 --> 00:13:13.797 from to becoming a fundamental part of our everyday lives. 00:13:13.880 --> 00:13:18.277 Quietly but profoundly reshaping economies, warfare, and the very fabric 00:13:18.360 --> 00:13:21.797 of human society. For the first time since the dawn of the 00:13:21.880 --> 00:13:25.597 nuclear era, global stability may hinge not just on military strength or 00:13:25.680 --> 00:13:29.877 economic weight, but also on competing to secure a 00:13:29.960 --> 00:13:33.877 competitive advantage on this new technological intelligence. 00:13:33.960 --> 00:13:37.317 All in all, AI embodies something we have never seen before. 00:13:37.400 --> 00:13:39.717 A shift in how knowledge is generated. 00:13:39.800 --> 00:13:43.997 A shift in how decisions are made and ultimately how humanity sees its own 00:13:44.080 --> 00:13:48.997 place in the world. What will our society look like 20 years from now? 00:13:49.080 --> 00:13:52.877 And more importantly, how will artificial intelligence have shaped it? 00:13:52.960 --> 00:13:55.877 Now to comment on these ideas, it is my pleasure to introduce you all to our 00:13:55.960 --> 00:13:59.877 speakers for the evening. Eric Schmidt served as chairman and CEO 00:13:59.960 --> 00:14:02.877 of Google between 2001 and 2011. 00:14:02.960 --> 00:14:06.677 Today he serves as chair and CEO of Relativity Space. He is a founding 00:14:06.760 --> 00:14:10.317 partner at Innovation Endeavors and is a leading voice in AI and national 00:14:10.400 --> 00:14:13.077 security in the US and around the globe. 00:14:13.160 --> 00:14:16.117 Graham Allison is the Douglas Dillon Professor of Government at the Harvard 00:14:16.200 --> 00:14:19.517 Kennedy School where he served as founding dean and director of its Belfer 00:14:19.600 --> 00:14:22.157 Center for Science and International Affairs. 00:14:22.240 --> 00:14:25.117 He also served as assistant secretary of defense in the first Clinton 00:14:25.200 --> 00:14:29.557 administration receiving the Defense Medal for distinguished public service. 00:14:29.640 --> 00:14:32.557 Now without further ado, please join me in opening up tonight's discussion and 00:14:32.640 --> 00:14:36.720 welcoming our esteemed guests to the stage. 00:14:47.400 --> 00:14:50.797 So thank you for a generous introduction. It's a 00:14:50.880 --> 00:14:53.597 extraordinary pleasure to welcome back 00:14:53.680 --> 00:14:56.757 to the JFK Jr. Forum Eric Schmidt, our 00:14:56.840 --> 00:14:58.637 colleague and friend, uh for a 00:14:58.720 --> 00:15:04.517 discussion of Henry Kissinger uh and AI and the future. 00:15:04.600 --> 00:15:10.717 Uh as you can see here uh this picture of Eric and Henry, uh 00:15:10.800 --> 00:15:13.837 uh they were best friends. Uh 00:15:13.920 --> 00:15:16.517 late developing friendship but became 00:15:16.600 --> 00:15:20.277 very deep friendship in which Eric was 00:15:20.360 --> 00:15:22.317 uh generously the tutor 00:15:22.400 --> 00:15:25.837 for a 95-year-old Henry Kissinger who 00:15:25.920 --> 00:15:28.877 discovered AI hanging around the end of 00:15:28.960 --> 00:15:32.997 a lecture that he had given uh when 00:15:33.080 --> 00:15:36.797 uh Demis Hassabis was uh offering to 00:15:36.880 --> 00:15:42.637 beginning to talk about AI and Henry decided he needed to learn about this. 00:15:42.720 --> 00:15:46.277 Uh he called me up. I told him, "Henry, forget about it. 00:15:46.360 --> 00:15:49.797 You know that you don't have any background in science 00:15:49.880 --> 00:15:52.197 and technology. I told him actually you 00:15:52.280 --> 00:15:54.077 don't know the difference between a chip 00:15:54.160 --> 00:15:56.877 and a potato chip." Mhm. Uh he said, 00:15:56.960 --> 00:15:59.677 "That's true but Eric has promised to to 00:15:59.760 --> 00:16:02.197 to teach me." So we're very pleased to 00:16:02.280 --> 00:16:07.157 have him here. We had him here last year. This is actually 00:16:07.240 --> 00:16:08.797 uh maybe it will become an annual 00:16:08.880 --> 00:16:10.637 tradition. Uh 00:16:10.720 --> 00:16:15.157 Henry passed 2 years ago last week. Uh 00:16:15.240 --> 00:16:18.077 so he was 100 years old. Uh 00:16:18.160 --> 00:16:20.437 to think of such an amazing life over 00:16:20.520 --> 00:16:23.637 that century. Uh he was a person who 00:16:23.720 --> 00:16:26.717 made a huge difference in America's 00:16:26.800 --> 00:16:29.397 national security and the world and made 00:16:29.480 --> 00:16:33.997 a huge difference in the lives of many many people. Uh some of whom began as 00:16:34.080 --> 00:16:39.597 his students, some who became as his tutors, uh and many others. So 00:16:39.680 --> 00:16:43.557 uh Eric has been introduced. Uh 00:16:43.640 --> 00:16:47.957 I would just remind you a couple of things. I would say first that 00:16:48.040 --> 00:16:51.997 Eric was the chief executive who took Google from a 00:16:52.080 --> 00:16:54.557 cause to one of the great companies of 00:16:54.640 --> 00:16:56.917 the world. 00:16:57.000 --> 00:16:59.117 And that's a pretty amazing thing. 00:16:59.200 --> 00:17:02.957 Uh secondly, early on he had 00:17:03.040 --> 00:17:05.397 identified AI 00:17:05.480 --> 00:17:08.477 as an arena of the future. 00:17:08.560 --> 00:17:11.157 And at Google essentially bought up all 00:17:11.240 --> 00:17:15.517 the super talent in the world that he could find including uh 00:17:15.600 --> 00:17:17.477 uh DeepMind 00:17:17.560 --> 00:17:20.117 uh which was actually the company that 00:17:20.200 --> 00:17:24.957 then brought to Google uh Demis Hassabis 00:17:25.040 --> 00:17:27.637 who got the Nobel Prize last year for 00:17:27.720 --> 00:17:31.397 his work at Google on uh protein. 00:17:31.480 --> 00:17:35.117 Mustafa Suleyman who's now running uh 00:17:35.200 --> 00:17:39.237 uh Microsoft's consumer AI was another part 00:17:39.320 --> 00:17:43.277 of the acquisition and many many many many others. 00:17:43.360 --> 00:17:47.717 So the other thing to say about Eric about Eric and why it's such a pleasure 00:17:47.800 --> 00:17:52.197 to have him here is that as one tries to make any sense of all the claims that 00:17:52.280 --> 00:17:54.237 are being made about AI, 00:17:54.320 --> 00:17:57.157 most of the people who talk loudest 00:17:57.240 --> 00:17:59.517 are talking their own book. 00:17:59.600 --> 00:18:02.437 So when I listen to Sam Altman or to 00:18:02.520 --> 00:18:04.997 folks at Anthropic or with folks even 00:18:05.080 --> 00:18:07.837 Mustafa now at Microsoft, they have to 00:18:07.920 --> 00:18:12.597 talk their companies >> holding my book. Uh well, wait a second. 00:18:12.680 --> 00:18:14.717 They have to talk their companies, you 00:18:14.800 --> 00:18:17.597 know, business, their future and they 00:18:17.680 --> 00:18:22.477 are chasing what they believe is the biggest pot of gold at the end of the 00:18:22.560 --> 00:18:24.397 rainbow that there's ever been. 00:18:24.480 --> 00:18:28.717 So it's pretty hard to tell what they actually think and what they're saying. 00:18:28.800 --> 00:18:32.117 Eric has kind of graduated to be the statesman 00:18:32.200 --> 00:18:35.157 in this domain having had a big stake in 00:18:35.240 --> 00:18:39.157 it but now having stand standing back and I 00:18:39.240 --> 00:18:43.837 particularly after his work with Henry, basically trying to 00:18:43.920 --> 00:18:47.597 say things as as clearly as he can and as clearly as 00:18:47.680 --> 00:18:49.917 he can see them. So I take him to be a 00:18:50.000 --> 00:18:53.597 much more valuable source of clues about 00:18:53.680 --> 00:18:57.717 what's happening than listening to most of the people who are talking their own 00:18:57.800 --> 00:19:00.397 books. So let's start with Henry and then we're 00:19:00.480 --> 00:19:05.837 going to go to AI and then we'll go to questions of the audience. 00:19:05.920 --> 00:19:08.797 So uh Eric uh 00:19:08.880 --> 00:19:12.237 you uh spoke beautifully at at Henry's 00:19:12.320 --> 00:19:15.037 memorial about how 00:19:15.120 --> 00:19:20.117 much impact Henry had on your own life and even on the 00:19:20.200 --> 00:19:21.797 you know, the questions that you were 00:19:21.880 --> 00:19:25.077 asking, the things that you 00:19:25.160 --> 00:19:27.717 think that matter. So tell us a little 00:19:27.800 --> 00:19:30.517 bit about more what Henry meant 00:19:30.600 --> 00:19:33.797 in this relationship and also for people 00:19:33.880 --> 00:19:36.277 that wouldn't have the opportunity to to 00:19:36.360 --> 00:19:38.997 know him, what what how might they 00:19:39.080 --> 00:19:41.477 capture a little bit of this magic? 00:19:41.560 --> 00:19:45.237 You know, when I when I first met and thank you Graham as always, it's great 00:19:45.320 --> 00:19:47.477 to be here. Um 00:19:47.560 --> 00:19:50.357 I I met Henry when he was in his early 00:19:50.440 --> 00:19:55.237 80s and normally men in their early 80s become a little 00:19:55.320 --> 00:19:58.997 off. And I was struck by how brilliant he 00:19:59.080 --> 00:20:03.357 was. And so I thought to myself, you know, he he served in World War II, he 00:20:03.440 --> 00:20:07.077 got the Bronze Star, and you know, emigrated or escaped from Germany cuz 00:20:07.160 --> 00:20:12.037 they were a Jewish family. And um came to fought in the war, came 00:20:12.120 --> 00:20:15.117 on the GI Bill to Harvard uh undergraduate and graduate, and 00:20:15.200 --> 00:20:17.237 ultimately was a professor here. 00:20:17.320 --> 00:20:21.477 And I tried to figure out he must have been really smart when he was in your 00:20:21.560 --> 00:20:24.837 chairs, not that the building existed at the time. 00:20:24.920 --> 00:20:29.117 So here's a quote when he was an undergraduate at Harvard. 00:20:29.200 --> 00:20:32.837 Quote, "In the life of every person there comes 00:20:32.920 --> 00:20:35.397 a point when he realizes that out of all 00:20:35.480 --> 00:20:40.837 of the seemingly limitless possibilities of his youth, he has in fact become one 00:20:40.920 --> 00:20:44.837 actuality. No longer is life a broad plane with 00:20:44.920 --> 00:20:49.397 forests and mountains and beckoning all around, but it becomes apparent that 00:20:49.480 --> 00:20:53.157 one's journey across the meadows has indeed followed a 00:20:53.240 --> 00:20:57.157 regular path. That one can no longer go this way or 00:20:57.240 --> 00:21:01.877 that, but that the direction is set and the limits are defined." 00:21:01.960 --> 00:21:05.277 That's what he wrote when he was your age. 00:21:05.360 --> 00:21:08.157 Um when he was an undergraduate here, he 00:21:08.240 --> 00:21:12.437 holds the record for the longest undergraduate thesis in the uh in the 00:21:12.520 --> 00:21:15.717 college. After he submitted his undergraduate 00:21:15.800 --> 00:21:19.597 thesis on Kant and the meaning of the world, they instituted a new rule which 00:21:19.680 --> 00:21:24.757 still applies to you that your thesis can be no more than 350 pages. 00:21:24.840 --> 00:21:29.637 FYI. It's true. >> So he was clearly an unusually brilliant 00:21:29.720 --> 00:21:34.237 polymath. And uh my own explanation for Henry, 00:21:34.320 --> 00:21:38.397 aside from you know, I really cared deeply about him, 00:21:38.480 --> 00:21:41.077 was that watching his family, and in 00:21:41.160 --> 00:21:45.757 particular his father, and he we talked a lot about this, um see the destruction 00:21:45.840 --> 00:21:50.157 of the world around him in Germany as the Nazis took over. 00:21:50.240 --> 00:21:53.797 And when they fled, and he saw the damage to his father and his father's 00:21:53.880 --> 00:21:56.037 identity and so forth, Henry after the 00:21:56.120 --> 00:21:57.717 war decided that he would do everything 00:21:57.800 --> 00:22:01.317 he can could to avoid a future war. 00:22:01.400 --> 00:22:04.197 Now you can quibble with or argue very 00:22:04.280 --> 00:22:06.637 much with the things he did, but you 00:22:06.720 --> 00:22:09.237 cannot argue that his that his goal was 00:22:09.320 --> 00:22:13.957 not what I just said. The the data supports it, the historians all agree. 00:22:14.040 --> 00:22:18.077 You can disagree with the specific tactics, but at the end of the day what 00:22:18.160 --> 00:22:22.597 he sought is he sought a a a world that did not have a World War 00:22:22.680 --> 00:22:27.077 III. And think about the issues that he faced in the '50s and '60s. 00:22:27.160 --> 00:22:30.757 He used to tell me about this small group, and he always thought that the 00:22:30.840 --> 00:22:34.557 most interesting policy things happen with small groups, and this was a 00:22:34.640 --> 00:22:37.037 collaboration between MIT and Harvard 00:22:37.120 --> 00:22:40.317 and RAND, uh which in the early 1950s 00:22:40.400 --> 00:22:44.797 dev- de- um um invented mutual assured destruction. 00:22:44.880 --> 00:22:47.677 And I said, "Henry, well who was in this group?" He says, "Oh, Oppenheimer." And 00:22:47.760 --> 00:22:50.077 I'm like, you know, give me a break. 00:22:50.160 --> 00:22:52.637 Right? All the most famous people. 00:22:52.720 --> 00:22:57.157 So uh he was lucky uh to be born at the 00:22:57.240 --> 00:23:00.917 right time. He was clearly one of the smartest people at alive at the time. He 00:23:01.000 --> 00:23:04.637 was lucky to get out of Germany before his family and he were killed. He was 00:23:04.720 --> 00:23:07.237 lucky to be part of the GI Bill. 00:23:07.320 --> 00:23:12.237 Um and he had all sorts of interesting stories. My favorite was one day I told 00:23:12.320 --> 00:23:15.477 him, "You have to go to the doctor." You know, he's old. And he goes, "Okay." And 00:23:15.560 --> 00:23:19.197 I said, "I want you to go to the Mayo Clinic." And I said, "Okay." Now he's a 00:23:19.280 --> 00:23:21.837 New Yorker. And the people in Mayo were like, you 00:23:21.920 --> 00:23:26.397 know, Midwestern, soft-spoken, pleasant, nice, not like the intense New Yorker 00:23:26.480 --> 00:23:28.717 types. He goes to the Mayo Clinic, comes back, 00:23:28.800 --> 00:23:32.437 and he said, "I loved it." I said, "Why would you like the Mayo Clinic in the 00:23:32.520 --> 00:23:35.117 middle of Rochester, Minnesota?" He said, "It reminded me of the war." I 00:23:35.200 --> 00:23:38.957 said, "What?" And he goes, "Well, he was in a uh 00:23:39.040 --> 00:23:40.357 he didn't speak English when he came to 00:23:40.440 --> 00:23:44.037 the country, and he worked as a um 00:23:44.120 --> 00:23:48.357 uh in a um shaving brush factory where they made shaving brushes, and he was 00:23:48.440 --> 00:23:51.677 drafted as everyone expected, and he went to war, and he went with a 00:23:51.760 --> 00:23:54.637 Wisconsin-based American group. 00:23:54.720 --> 00:23:59.637 So he indexed his identity, could hardly speak English, and and nobody could 00:23:59.720 --> 00:24:03.357 understand his English anyway, he indexed to that group. So he had all 00:24:03.440 --> 00:24:07.517 these sort of touching little stories. I have lots of stories like that about how 00:24:07.600 --> 00:24:11.277 he became an American, and of course became an American citizen. 00:24:11.360 --> 00:24:13.517 And you know the rest of the history. 00:24:13.600 --> 00:24:16.157 Well, I would say a fantastic person. I 00:24:16.240 --> 00:24:21.677 had the good fortune to enroll in the course as a student. 00:24:21.760 --> 00:24:24.717 >> What was that like? >> 1965, 00:24:24.800 --> 00:24:26.437 God help us, uh 00:24:26.520 --> 00:24:29.277 taught by Henry, uh and 00:24:29.360 --> 00:24:33.597 ever thereafter was part of his entourage, 00:24:33.680 --> 00:24:35.677 and I think the thing that struck me 00:24:35.760 --> 00:24:39.077 most about him was this uh 00:24:39.160 --> 00:24:41.837 and the strategic acumen. That is, the 00:24:41.920 --> 00:24:44.597 ability to take a problem and lift it up 00:24:44.680 --> 00:24:47.317 to its whole 00:24:47.400 --> 00:24:51.357 360° of strategic challenge, and then 00:24:51.440 --> 00:24:53.877 find the 00:24:53.960 --> 00:24:56.397 points at which the 00:24:56.480 --> 00:25:01.197 policy process could be impacted. And uh 00:25:01.280 --> 00:25:06.237 I wrote about uh for his 100th birthday, which Richard 00:25:06.320 --> 00:25:09.117 asked us to write a little, you know, something that 00:25:09.200 --> 00:25:13.797 Eric wrote something, I wrote something, I said, "Really in thinking about Henry, 00:25:13.880 --> 00:25:14.997 uh 00:25:15.080 --> 00:25:17.717 he was not about, if you think about, he 00:25:17.800 --> 00:25:21.437 was often criticized for his realism or 00:25:21.520 --> 00:25:26.597 ruthless realism or realpolitik, but if you look and look and see what he 00:25:26.680 --> 00:25:27.877 was doing, 00:25:27.960 --> 00:25:31.637 uh it wasn't raw real- realpolitik 00:25:31.720 --> 00:25:34.637 uh just for the advancement of the 00:25:34.720 --> 00:25:36.477 interests of one country. 00:25:36.560 --> 00:25:39.357 It was always about the construction of 00:25:39.440 --> 00:25:44.157 a viable order to prevent catastrophic war. 00:25:44.240 --> 00:25:48.397 He had lived through the experience of the Holocaust and the most deadly war in 00:25:48.480 --> 00:25:51.037 history. He 00:25:51.120 --> 00:25:54.837 foresaw the prospect of a nuclear war 00:25:54.920 --> 00:25:57.477 that could actually extinguish life on 00:25:57.560 --> 00:25:58.877 Earth. 00:25:58.960 --> 00:26:02.197 He was caught up in the Cold War, the 00:26:02.280 --> 00:26:07.357 fiercest rivalry that we'd seen to that point between the US and the Soviet 00:26:07.440 --> 00:26:11.037 Union, and all the time he was trying to reach 00:26:11.120 --> 00:26:14.197 beyond that to some 00:26:14.280 --> 00:26:15.877 way to survive 00:26:15.960 --> 00:26:19.517 for both the US and for for uh 00:26:19.600 --> 00:26:23.397 for fellow human beings. So that's I 00:26:23.480 --> 00:26:25.837 think and that's how he kind of backed 00:26:25.920 --> 00:26:27.357 into 00:26:27.440 --> 00:26:30.917 the AI issue, which for him was 00:26:31.000 --> 00:26:36.197 another generation of a somewhat analogous problem. Well, 00:26:36.280 --> 00:26:39.717 he had written his undergraduate thesis on Kant and the meaning of meaning, I 00:26:39.800 --> 00:26:44.677 think. And um he was very when he went to hear 00:26:44.760 --> 00:26:49.317 Demos speak, um it sort of he immediately got it. And 00:26:49.400 --> 00:26:51.597 he immediately said to himself, 00:26:51.680 --> 00:26:54.517 "What does this mean to be human?" 00:26:54.600 --> 00:26:59.077 And we are today grappling with the question that he foresaw 20 years ago 00:26:59.160 --> 00:27:02.637 when we when we first started working on this, "What does it mean to be human in 00:27:02.720 --> 00:27:08.077 the age of AI?" What does it mean to be a child to an adult, to be a leader? 00:27:08.160 --> 00:27:09.397 What does it mean for economics? What 00:27:09.480 --> 00:27:10.757 does it mean for jobs? You know, all of 00:27:10.840 --> 00:27:15.677 that. But his core argument was that 00:27:15.760 --> 00:27:20.117 this is an epochal change in the sense that it's like the um 00:27:20.200 --> 00:27:24.837 uh the various major changes that we've had 00:27:24.920 --> 00:27:26.957 in sort of reasoning, the scientific 00:27:27.040 --> 00:27:31.917 revolution and so forth, cuz we as humans have never had a competitor that 00:27:32.000 --> 00:27:35.157 is not human but of similar or greater 00:27:35.240 --> 00:27:37.197 level of intelligence. And it is 00:27:37.280 --> 00:27:39.357 unpredictable what we human will do. He 00:27:39.440 --> 00:27:42.597 used to say that what would happen is in 00:27:42.680 --> 00:27:44.597 magic, when people don't understand 00:27:44.680 --> 00:27:49.797 things, they either decide that it's a new religion or they take up arms. 00:27:49.880 --> 00:27:52.397 And so he would say, "Well, are we going to take up arms to AI or we going to 00:27:52.480 --> 00:27:56.797 make it a new religion?" And I said, "I hope it's a religion." Um cuz of course 00:27:56.880 --> 00:27:59.157 I benefit from the religion, I guess. 00:27:59.240 --> 00:28:02.357 So in the book that was published this time 00:28:02.440 --> 00:28:05.597 last year called Genesis, which was the 00:28:05.680 --> 00:28:09.117 work that Eric and Henry and Craig Bundy 00:28:09.200 --> 00:28:12.677 were still doing up to the point at which he uh 00:28:12.760 --> 00:28:15.797 passed, he here's a a passage that 00:28:15.880 --> 00:28:18.917 clearly Henry wrote. It says, quote, 00:28:19.000 --> 00:28:21.837 it talking about the US and China, "If 00:28:21.920 --> 00:28:24.917 each wishes to maximize its unilateral 00:28:25.000 --> 00:28:28.557 position in the AI space, 00:28:28.640 --> 00:28:31.117 then the conditions would be such for a 00:28:31.200 --> 00:28:34.357 contest between rival military forces 00:28:34.440 --> 00:28:36.757 and intelligence agencies, the likes of 00:28:36.840 --> 00:28:39.477 which humanity has never faced before. 00:28:39.560 --> 00:28:42.117 Today, in the weeks and months and days 00:28:42.200 --> 00:28:44.837 leading up to the arrival of the first 00:28:44.920 --> 00:28:49.917 superintelligence, a security dilemma of existential nature 00:28:50.000 --> 00:28:53.677 awaits us." So you've thought about this 00:28:53.760 --> 00:28:56.397 superintelligence as an existential 00:28:56.480 --> 00:28:59.197 threat beyond what 00:28:59.280 --> 00:29:03.477 what we can imagine? Say more, yeah. 00:29:03.560 --> 00:29:07.317 Well, um there is something which I call the San 00:29:07.400 --> 00:29:10.877 Francisco consensus. And the reason I call it the San Francisco consensus is 00:29:10.960 --> 00:29:13.437 everyone in San Francisco believes this. 00:29:13.520 --> 00:29:17.037 It may or may not be true, but if you go to San Francisco, trust me, that's what 00:29:17.120 --> 00:29:21.197 they're doing besides the usual San Francisco recreational behaviors. 00:29:21.280 --> 00:29:23.997 And um 00:29:24.080 --> 00:29:28.397 the the basic build goes that we are we're we're we've gone through a 00:29:28.480 --> 00:29:31.597 language revolution. You understand that, you know, language to language 00:29:31.680 --> 00:29:34.277 chat GPT, everybody here knows what that is. 00:29:34.360 --> 00:29:38.597 Agents, which are now on the horizon, and these agents allow you to 00:29:38.680 --> 00:29:42.637 essentially automate tasks. And the key thing to understand about agents is they 00:29:42.720 --> 00:29:47.157 can be concatenated. So you can, you you do do this and this and this and this 00:29:47.240 --> 00:29:51.317 and this and then they all get piped together. Why am I talking about 00:29:51.400 --> 00:29:54.517 agents? Because I've just described workflow. 00:29:54.600 --> 00:29:57.637 And that's what businesses do, what it's universities do, it's what governments 00:29:57.720 --> 00:30:01.957 do, so forth and so on. And then the next one is reasoning. 00:30:02.040 --> 00:30:04.517 Now, reasoning is higher-order function 00:30:04.600 --> 00:30:06.557 in in humans and the 00:30:06.640 --> 00:30:09.277 reasoning revolution is just beginning. 00:30:09.360 --> 00:30:11.957 Um as of today, it is technically 00:30:12.040 --> 00:30:14.797 correct to say that the scaling laws 00:30:14.880 --> 00:30:19.517 that operate in AI have not slowed down yet. The scaling law basically says that 00:30:19.600 --> 00:30:23.557 if you put more data and more electricity and more chips, you get this 00:30:23.640 --> 00:30:26.157 emergent behavior one after the other. 00:30:26.240 --> 00:30:30.717 And you can see that, for example, with Gemini 3 which just came out, which beat 00:30:30.800 --> 00:30:33.117 uh OpenAI 5 which just came out, which 00:30:33.200 --> 00:30:37.157 beat um Claude 4.5 which just came out, 00:30:37.240 --> 00:30:41.437 which beat Deep Seek cuz they copied it anyway, and blah blah blah. You know, so 00:30:41.520 --> 00:30:46.677 so it's >> In case you didn't notice, Gemini 3 is from Google. Yes. 00:30:46.760 --> 00:30:49.597 I'm proud to say that we're back in charge 00:30:49.680 --> 00:30:53.757 um until the next one. It's it's very very competitive and what's happening is 00:30:53.840 --> 00:30:58.157 that these massive data center build-outs uh which are, by the way, one of the key 00:30:58.240 --> 00:31:02.237 drivers of the American economy are helping not only lift our economy, but 00:31:02.320 --> 00:31:05.117 they're also building building this kind of stuff. And it's a whole new world 00:31:05.200 --> 00:31:07.677 compared to my whole experience. 00:31:07.760 --> 00:31:10.077 So, the question is what happens over time? 00:31:10.160 --> 00:31:13.157 So, you have language, agents, and reasoning. 00:31:13.240 --> 00:31:18.317 Well, isn't that what we do? Right? We do stuff, we communicate, and 00:31:18.400 --> 00:31:19.917 we do actions. 00:31:20.000 --> 00:31:22.557 So, the San Francisco consensus is that 00:31:22.640 --> 00:31:26.117 at some point that stuff comes together 00:31:26.200 --> 00:31:29.837 and you get what is called technically recursive self-improvement. And 00:31:29.920 --> 00:31:33.677 recursive self-improvement is when it's learning on its own. 00:31:33.760 --> 00:31:37.637 This is not true today. Today, when you set up one of these huge 00:31:37.720 --> 00:31:40.717 data centers, you know what they look like, um you have to tell them what to 00:31:40.800 --> 00:31:43.357 learn. But the belief is that this is coming. 00:31:43.440 --> 00:31:47.077 And there's lots of evidence that this is coming. The ability for computers to 00:31:47.160 --> 00:31:51.757 write programs, to generate mathematical conjectures, to discover new facts looks 00:31:51.840 --> 00:31:55.277 like it's very very close. Many people believe that there will be 00:31:55.360 --> 00:31:58.317 new math design, new mathematicians, AI 00:31:58.400 --> 00:32:01.157 mathematicians to love in the next year. 00:32:01.240 --> 00:32:04.157 So, we collectively as an industry believe that this is going to happen 00:32:04.240 --> 00:32:06.437 soon. If you ask the San Francisco 00:32:06.520 --> 00:32:09.197 people, they'll say 2 years. 00:32:09.280 --> 00:32:14.717 Which is really soon. If you ask me, I double that to 4 years, which is really 00:32:14.800 --> 00:32:19.237 soon. Right? So, it's happening. It's happening very quickly. I want and Henry 00:32:19.320 --> 00:32:22.757 certainly wanted it. We wanted it to be built with American values and human 00:32:22.840 --> 00:32:27.797 values. There is a point, in my view, and we talked a lot about this, 00:32:27.880 --> 00:32:32.317 where somebody's going to have to raise their hand and say, we just went too far. 00:32:32.400 --> 00:32:35.557 There's too much danger here. We don't want to give that agency to the 00:32:35.640 --> 00:32:39.557 computer. We want humans to be in charge of it. It is not agreed to where that 00:32:39.640 --> 00:32:43.237 point is, but our book spends a lot of time talking about where that might be. 00:32:43.320 --> 00:32:46.637 Another example would be the you discover that the computer has decided 00:32:46.720 --> 00:32:49.597 on its own to get access to weapons. 00:32:49.680 --> 00:32:53.077 That that's clearly a Like everyone would agree that's not a good idea, 00:32:53.160 --> 00:32:55.957 right? It's bad enough that humans have access to weapons. Imagine if the 00:32:56.040 --> 00:32:59.317 computer has it. And what are the criteria by which And you can think of 00:32:59.400 --> 00:33:04.037 many other such examples. Fundamen- fundamentally about human agency. We 00:33:04.120 --> 00:33:07.717 also spend a lot of time talking about the effect on children. 00:33:07.800 --> 00:33:10.717 Um we're running a mass experiment on 00:33:10.800 --> 00:33:13.277 human development by deploying these 00:33:13.360 --> 00:33:15.917 systems that are incredibly addictive. 00:33:16.000 --> 00:33:20.517 Um what whether it's on iPad or phone or, you know, whatever for young people 00:33:20.600 --> 00:33:24.357 who may not be certainly not adults, may not have their own identity, and they 00:33:24.440 --> 00:33:27.597 can really be manipulated. So, what does it mean to be a child whose best friend 00:33:27.680 --> 00:33:32.557 is non-human? Aside from maybe becoming like a super nerd. 00:33:32.640 --> 00:33:35.077 But I don't know, what does it mean? We we we have no data. We don't know what 00:33:35.160 --> 00:33:38.637 it means to young young boys and girls, to their development, to their ability 00:33:38.720 --> 00:33:43.117 to associate. Will they ultimately rebel and say, I only love people, I hate 00:33:43.200 --> 00:33:45.197 computers? Like, you know, kids rebel. 00:33:45.280 --> 00:33:47.717 We just don't know. 00:33:47.800 --> 00:33:54.640 So, can you take the next slide? I'm sorry, I have the clicker, but I left it. 00:33:57.880 --> 00:34:02.680 Can I Can you guys make the next slide go up? 00:34:03.560 --> 00:34:06.917 Yeah, thank you. So, here's the so-called Kissinger 00:34:07.000 --> 00:34:08.197 challenge. 00:34:08.280 --> 00:34:11.357 This is Henry 1969. 00:34:11.440 --> 00:34:14.917 So, Nixon was elected in 1968. 00:34:15.000 --> 00:34:19.037 Became president January 1969. 00:34:19.120 --> 00:34:21.197 Appointed Henry, who was a professor 00:34:21.280 --> 00:34:23.397 here at Harvard, as his national 00:34:23.480 --> 00:34:28.957 security advisor. And Henry writes, as you'll see here, that anyone coming to 00:34:29.040 --> 00:34:30.117 office 00:34:30.200 --> 00:34:32.917 in the late '60s could not fail to be 00:34:33.000 --> 00:34:36.517 awed by the unprecedented dimensions of 00:34:36.600 --> 00:34:38.637 the challenge of peace. 00:34:38.720 --> 00:34:43.757 And then down to the bottom line, there could be no higher duty 00:34:43.840 --> 00:34:45.517 than to prevent the catastro- 00:34:45.600 --> 00:34:49.717 catastrophe of nuclear war. 00:34:49.800 --> 00:34:54.797 So, that became for him a defining challenge 00:34:54.880 --> 00:34:56.477 in the Cold War 00:34:56.560 --> 00:34:58.237 with nuclear weapons, arsenals 00:34:58.320 --> 00:35:03.597 developing, but quote, no higher duty than to prevent. 00:35:03.680 --> 00:35:07.997 So, how might this apply currently as we try 00:35:08.080 --> 00:35:10.397 to think about the US, China, and AI? 00:35:10.480 --> 00:35:13.037 Well, let me say that I think there's no higher duty 00:35:13.120 --> 00:35:17.157 than to preserve human agency and human freedom. 00:35:17.240 --> 00:35:20.997 Right? The things that we value collectively the most. 00:35:21.080 --> 00:35:24.037 And I think that's going to be a central challenge for all of you when you 00:35:24.120 --> 00:35:28.557 graduate. Um all of you will be facing these issues and they're complicated and 00:35:28.640 --> 00:35:32.237 they're subtle. Um imagine if the internet had been 00:35:32.320 --> 00:35:35.997 invented by China and it didn't have the kind of openness 00:35:36.080 --> 00:35:40.397 that uh the internet has today. It was under the Chinese internet. 00:35:40.480 --> 00:35:44.917 Um it looks like China is pursuing a different strategy than what I'm talking 00:35:45.000 --> 00:35:50.357 about. In my most recent visit to China, my the way I operated, I ask engineers 00:35:50.440 --> 00:35:52.877 technical questions cuz they don't lie to you. 00:35:52.960 --> 00:35:55.197 Whereas kind of everyone else, I'm not so sure. 00:35:55.280 --> 00:35:58.837 And I eventually figured out what the Chinese are doing is they're really 00:35:58.920 --> 00:36:02.357 really focused on applying AI in their businesses. And they're going to 00:36:02.440 --> 00:36:04.677 outcompete with us. 00:36:04.760 --> 00:36:07.357 We We will lose to China for their 00:36:07.440 --> 00:36:09.717 incredible adoption of AI in every 00:36:09.800 --> 00:36:14.957 product. Because they're just relentless and they work so hard. Called 996, 9:00 00:36:15.040 --> 00:36:18.357 in the morning, 9:00 in the evening, 6 days a week. Illegal, by the way, in 00:36:18.440 --> 00:36:23.077 China. Illegal in America, right? But nevertheless practiced. Um they're 00:36:23.160 --> 00:36:27.077 coming. Um they do not appear to be as focused 00:36:27.160 --> 00:36:29.317 on superintelligence and the path that I 00:36:29.400 --> 00:36:31.197 was describing as the San Francisco 00:36:31.280 --> 00:36:34.237 consensus. Of course, that could change. 00:36:34.320 --> 00:36:38.197 So, it appears that the two are pursuing different paths. One of the questions 00:36:38.280 --> 00:36:41.997 for you all and uh at the graduate level, even undergraduates given it's 00:36:42.080 --> 00:36:46.917 Harvard, is start figuring out what happens when these divergent paths hit 00:36:47.000 --> 00:36:51.637 roadblocks. Right? Cuz they're both have it. Example would be in America, 00:36:51.720 --> 00:36:55.037 we have essentially produced no new electricity because it's very hard to 00:36:55.120 --> 00:36:59.317 provision electricity. China has infinite 00:36:59.400 --> 00:37:01.957 electricity because of their incredible 00:37:02.040 --> 00:37:04.357 investment in renewables and so forth. 00:37:04.440 --> 00:37:08.637 They built something like 120 gigawatts of new renewable energy in the last 5 00:37:08.720 --> 00:37:10.197 years. It's some number like that. 00:37:10.280 --> 00:37:12.157 >> no. Wait. Wait. Wait. Wait. 00:37:12.240 --> 00:37:15.317 Every day in China, 00:37:15.400 --> 00:37:18.557 Thursday, Saturday, Monday, every day uh 00:37:18.640 --> 00:37:22.917 another gigawatt of electricity has added to the grid. Wow. That's pretty 00:37:23.000 --> 00:37:28.037 extreme. >> Every day. So, my point is And here And here, by 00:37:28.120 --> 00:37:31.717 the way, just to give you an example, how big is a nuclear power plant? About 00:37:31.800 --> 00:37:36.517 1.5 gigawatts. So, that's the scale of the electricity 00:37:36.600 --> 00:37:40.837 revolution that's going on in China, again using those numbers. 00:37:40.920 --> 00:37:44.397 How many of those kinds of plants have we built in America? Zero. 00:37:44.480 --> 00:37:47.877 Uh and we're certainly losing the renewables race to China for the reasons 00:37:47.960 --> 00:37:49.677 that everybody here knows. 00:37:49.760 --> 00:37:54.957 So, they have a lot of power, right? We don't. We have a lot of chips, 00:37:55.040 --> 00:37:58.677 they don't. Um it sets up the competition, right? 00:37:58.760 --> 00:38:03.437 And each will pursue different ways. Um one of the technical questions 00:38:03.520 --> 00:38:07.357 is something called diffusion, where what you do is you take a very powerful 00:38:07.440 --> 00:38:11.197 model, we'll use Gemini 3, the top one at the moment, 00:38:11.280 --> 00:38:15.357 um and you ask it 10,000 questions and you take the answers, and then the 00:38:15.440 --> 00:38:20.197 system can learn from the questions and answers enough to mimic, without the 00:38:20.280 --> 00:38:24.637 expense of doing the big training, the big model. So, again, 00:38:24.720 --> 00:38:28.917 thinking through the strategies that China will pursue and the the strategies 00:38:29.000 --> 00:38:32.957 that are available for America um are is probably pretty important. 00:38:33.040 --> 00:38:37.797 What's interesting is both countries are relying on the private sector to do this. 00:38:37.880 --> 00:38:41.437 You would have thought in Henry's period that you would have used the government. 00:38:41.520 --> 00:38:45.397 But in fact, our government cannot move this quickly, the compensation systems, 00:38:45.480 --> 00:38:49.157 and so forth. It turns out it's probably true for the Chinese as well. I don't 00:38:49.240 --> 00:38:51.157 know at the at the security level if 00:38:51.240 --> 00:38:53.677 this is true, but I've not found any 00:38:53.760 --> 00:38:58.357 large, weird, Manhattan-type projects within China, although there's plenty of 00:38:58.440 --> 00:39:00.037 people in the private companies that are 00:39:00.120 --> 00:39:02.477 working hard on national security. 00:39:02.560 --> 00:39:06.397 So, on the AI topic in general, tell us 00:39:06.480 --> 00:39:08.477 Just give us a minute or two of what is 00:39:08.560 --> 00:39:10.637 the upside, the things that excite you 00:39:10.720 --> 00:39:13.677 the most that are 1 year, 2 year, 3 00:39:13.760 --> 00:39:16.157 years that are the line of sight to. 00:39:16.240 --> 00:39:20.157 Well, the first question is, why is this madness occurring? It must be a bubble 00:39:20.240 --> 00:39:25.037 and it's going to crash. No, it's not a bubble. If if anything, 00:39:25.120 --> 00:39:29.437 it's underhyped because you're fundamentally automating businesses. 00:39:29.520 --> 00:39:33.397 And the reason people are spending this enormous amount of money is to automate 00:39:33.480 --> 00:39:36.917 the boring parts of what they view as the boring parts of their business. So, 00:39:37.000 --> 00:39:40.597 whether it's billing or accounting or product design or delivery or inventory 00:39:40.680 --> 00:39:44.477 or management or whatever, people are automating those. Um 00:39:44.560 --> 00:39:48.157 and there's an awful lot there. Think about medicine, think about climate 00:39:48.240 --> 00:39:53.840 change, engineering, new science, it's extraordinary. 00:39:57.160 --> 00:40:01.957 So, what excites you the most of the ones that you see 00:40:02.040 --> 00:40:05.997 that you have that you have a line of sight to that the rest of us probably do 00:40:06.080 --> 00:40:12.520 or not seeing yet? Sorry, I have to Sorry. have a cough. 00:40:12.880 --> 00:40:17.637 I apologize. We can all 00:40:17.720 --> 00:40:19.717 see in our own imagination what we're 00:40:19.800 --> 00:40:25.037 thinking of and then we'll see what Eric says. Yes. Well, when I started when I 00:40:25.120 --> 00:40:28.717 was in high school, I was an early programmer. 00:40:28.800 --> 00:40:32.157 And I delighted in writing code. When I went to college and graduate school, 00:40:32.240 --> 00:40:35.077 that's all I ever wanted to do. I didn't I ignored all these history things and 00:40:35.160 --> 00:40:39.157 things like this. I was the definition of a nerd at the time. 00:40:39.240 --> 00:40:43.517 And everything that I did in my 20s, which got me to where I am, has now been 00:40:43.600 --> 00:40:47.277 completely automated. Every aspect of the programming that I 00:40:47.360 --> 00:40:50.197 did, every aspect of the design is now 00:40:50.280 --> 00:40:54.877 done by computers. I recently had it write a whole program 00:40:54.960 --> 00:40:59.197 for me and I'm sitting there watching it define the classes and the detail of the 00:40:59.280 --> 00:41:03.477 interactions and so forth as it's generating it got like, holy crap, you 00:41:03.560 --> 00:41:07.077 know, the end of me. And I think watching I've been doing 00:41:07.160 --> 00:41:10.997 programming for 55 years. 00:41:11.080 --> 00:41:14.677 So, to see something start and then end in front of your own life and you're 00:41:14.760 --> 00:41:18.037 still alive is really profound, if I might say. 00:41:18.120 --> 00:41:20.557 Now, computer science is not going away. 00:41:20.640 --> 00:41:23.597 The computer scientist will be at least until the computer scientist gets 00:41:23.680 --> 00:41:27.117 replaced, will be supervising this. 00:41:27.200 --> 00:41:30.277 But the ability to generate code at the 00:41:30.360 --> 00:41:34.677 power that these systems can do is revolutionary. It means that each and 00:41:34.760 --> 00:41:37.077 every one of you has a supercomputer and 00:41:37.160 --> 00:41:39.357 a superprogrammer in your pocket. 00:41:39.440 --> 00:41:43.557 Now, nobody here is a terrorist, right? But using it's always easier to 00:41:43.640 --> 00:41:47.917 use negative examples. There's plenty of I'll use a stereotype, young men living 00:41:48.000 --> 00:41:50.997 in the basement, their mothers give them food and they sit there on the 00:41:51.080 --> 00:41:55.437 equivalent of crypto for chan, you know, paranoia, whatever. What pick your pick 00:41:55.520 --> 00:41:59.037 your poison. They all have the ability to use these 00:41:59.120 --> 00:42:02.797 tools to build incredibly powerful systems. 00:42:02.880 --> 00:42:06.877 Right? Cyber attacks, other things. No whatever they care about. 00:42:06.960 --> 00:42:10.677 Um there was there's some evidence that I 00:42:10.760 --> 00:42:15.477 think it's Mengioni, the fellow who killed the insurance insurance executive, 00:42:15.560 --> 00:42:19.237 um was into some of this and there's some people were looking at some of his 00:42:19.320 --> 00:42:22.517 writings. Of course, he's in jail now, but that he was somehow influenced with 00:42:22.600 --> 00:42:26.637 that. Now, I'm not proving causality here, but it's an example of some of the 00:42:26.720 --> 00:42:30.957 darkest recesses of humanity. You give those people these kinds of 00:42:31.040 --> 00:42:34.357 tools, we have to be ready. Now, the industry is well aware of this and we're 00:42:34.440 --> 00:42:36.277 working on it. It's very important that 00:42:36.360 --> 00:42:39.117 the defensive systems be capable. The 00:42:39.200 --> 00:42:41.597 eventual solution to AI, by the way, bad 00:42:41.680 --> 00:42:46.237 AI is AI fighting good AI fighting bad AI and that's how it will resolve 00:42:46.320 --> 00:42:49.957 itself. Okay, can we do the next slide, please? 00:42:50.040 --> 00:42:52.797 I want to ask you about the US China 00:42:52.880 --> 00:42:57.197 rivalry in AI as you see it. So, this is 00:42:57.280 --> 00:42:59.317 apologies if the slide's not clear 00:42:59.400 --> 00:43:04.837 enough, but what it suggests is that if we take the series of indices, 00:43:04.920 --> 00:43:08.597 uh if you look at the performance gap 00:43:08.680 --> 00:43:13.677 in January of 204, it was significantly larger than it is 00:43:13.760 --> 00:43:18.480 today. And then the uh 00:43:20.160 --> 00:43:23.877 What do we make of this and what do we make of the likely future? 00:43:23.960 --> 00:43:28.477 So, the chart is correct, but the people who are 00:43:28.560 --> 00:43:32.477 influenced by this claim that it's not going to be true for long 00:43:32.560 --> 00:43:37.277 because the reasoning revolution requires so many chips and so much of 00:43:37.360 --> 00:43:40.717 the magic that the San Francisco people have invented, 00:43:40.800 --> 00:43:42.917 using that as a sort of moniker, that 00:43:43.000 --> 00:43:47.757 the gap will widen. My own view is that the gap will widen, 00:43:47.840 --> 00:43:51.837 but for different reasons. I think that the Chinese focus is 00:43:51.920 --> 00:43:56.957 largely, as I mentioned, on embedding AI in everything. Smart 00:43:57.040 --> 00:43:58.637 toasters, cars, so forth and so on. 00:43:58.720 --> 00:44:03.237 They're moving much more quickly than our their robots. I think that the vast 00:44:03.320 --> 00:44:06.957 majority of humanoid robots will be Chinese AI powered and Chinese AI 00:44:07.040 --> 00:44:10.917 manufactured simply because they know how to drive the cost of things down. 00:44:11.000 --> 00:44:13.717 Their supply chains are incredible, their cost management, they work so 00:44:13.800 --> 00:44:15.077 hard, blah blah blah, all that kind of 00:44:15.160 --> 00:44:16.277 stuff. 00:44:16.360 --> 00:44:20.757 So, my guess is that that it's true that 00:44:20.840 --> 00:44:24.677 the gap will probably get larger, but the real question is will you as a 00:44:24.760 --> 00:44:28.317 consumer ultimately have a a better experience as a consumer with a Chinese 00:44:28.400 --> 00:44:32.317 product than a US product? And the answer is from a fit and finish, 00:44:32.400 --> 00:44:35.757 probably the Chinese product and that's of concern. 00:44:35.840 --> 00:44:40.637 So, let me drill a little further on this one. So, what there is a half dozen 00:44:40.720 --> 00:44:45.117 questions about which people are making bets and you've thought about them quite 00:44:45.200 --> 00:44:47.357 deeply. So, one is are we going to bet 00:44:47.440 --> 00:44:48.317 on 00:44:48.400 --> 00:44:53.157 computer chips or stacks or brains? 00:44:53.240 --> 00:44:54.797 Another one is are we going to bet 00:44:54.880 --> 00:44:57.797 closed or open? Yep. Another one is are 00:44:57.880 --> 00:45:01.557 we going to bet um working hard on AGI 00:45:01.640 --> 00:45:04.757 or diffusion and applications? 00:45:04.840 --> 00:45:08.317 So, if you go across the spectrum there, 00:45:08.400 --> 00:45:09.357 uh 00:45:09.440 --> 00:45:12.877 if I look at the Chinese piece, 00:45:12.960 --> 00:45:17.117 uh certainly the guys at Deep Seek think 200 people 00:45:17.200 --> 00:45:20.077 uh who just have brains 00:45:20.160 --> 00:45:22.637 can have a reasoning machine 00:45:22.720 --> 00:45:24.837 that costs 1/1000 00:45:24.920 --> 00:45:27.837 of the cost of Open AI. 00:45:27.920 --> 00:45:30.277 And now there's six other little dragons 00:45:30.360 --> 00:45:32.957 coming along from the same from the same 00:45:33.040 --> 00:45:35.597 space. So, that one makes me worry. 00:45:35.680 --> 00:45:40.637 On the closed versus open, if I remember our last conversation, you had pretty 00:45:40.720 --> 00:45:44.677 much concluded that open is going to beat closed, but all of our companies 00:45:44.760 --> 00:45:49.637 are mostly closed. So, what about that? 00:45:49.720 --> 00:45:52.357 And then thirdly, uh 00:45:52.440 --> 00:45:57.357 maybe if there's this AGI breakthrough, all the rest of this other stuff won't 00:45:57.440 --> 00:46:00.237 matter, but if the 00:46:00.320 --> 00:46:03.157 if the diffusion and application is 00:46:03.240 --> 00:46:05.197 already working in 00:46:05.280 --> 00:46:08.357 these other arenas, so just take us 00:46:08.440 --> 00:46:13.557 another step on the on the rivalry. So, diffusion refer backward diffusion 00:46:13.640 --> 00:46:17.277 refers to essentially learning from one model by 00:46:17.360 --> 00:46:21.117 the by many pairs and learning from it, as we discussed. 00:46:21.200 --> 00:46:25.677 My own opinion is that which is not I'm not sure of, but I think so, 00:46:25.760 --> 00:46:29.677 that the large companies will ultimately not release their largest models. It'll 00:46:29.760 --> 00:46:33.357 be too dangerous to do so that they'll subset them. That's what I would do. 00:46:33.440 --> 00:46:36.237 Now, I assume that they'll come to that decision. 00:46:36.320 --> 00:46:39.837 I think the most interesting question is open versus closed. For the those of you 00:46:39.920 --> 00:46:41.877 who don't have a background in this, 00:46:41.960 --> 00:46:44.077 open source open weights, open source is 00:46:44.160 --> 00:46:48.037 what I did for decades. Um when you use 00:46:48.120 --> 00:46:52.277 any form of computer, much of the software that you're using was developed 00:46:52.360 --> 00:46:55.837 by open source, which meant that the source was published and people would 00:46:55.920 --> 00:46:59.157 collectively advance it. And there was a whole movement around this, which I was 00:46:59.240 --> 00:47:03.477 part of. So, I'm relentlessly open source in my view. 00:47:03.560 --> 00:47:06.117 Um the major companies are closed source 00:47:06.200 --> 00:47:11.117 largely for economic reasons. Um basically, if you borrow 50 billion 00:47:11.200 --> 00:47:15.037 dollars from the financial market, they do want a return and saying to them, oh, 00:47:15.120 --> 00:47:18.157 by the way, we're going to give all the models away and you won't get your 00:47:18.240 --> 00:47:22.837 return probably isn't a very good legal or financial strategy. 00:47:22.920 --> 00:47:24.797 So, it So, the American model seems to 00:47:24.880 --> 00:47:26.197 have emerged as 00:47:26.280 --> 00:47:29.917 as closed. Strangely, the Chinese model 00:47:30.000 --> 00:47:34.557 is completely open open weights open source. 00:47:34.640 --> 00:47:39.677 Why? I don't know. Um one possible explanation is that the 00:47:39.760 --> 00:47:42.797 Chinese government has figured out that they were losing in closed source 00:47:42.880 --> 00:47:46.717 because they couldn't get the hardware and open source, because it's free, 00:47:46.800 --> 00:47:50.077 gives them proliferation. So, one of the consequences of open 00:47:50.160 --> 00:47:52.397 source open weights is the vast majority 00:47:52.480 --> 00:47:54.077 of humans on the planet will use Chinese 00:47:54.160 --> 00:47:57.797 models. Why? Cuz they're free, right? 00:47:57.880 --> 00:48:01.597 And most countries cannot afford the computing and the data centers and so 00:48:01.680 --> 00:48:05.637 forth. They're just going to take the Chinese models for free and embed them. 00:48:05.720 --> 00:48:09.517 Now, is that an issue? Absolutely, because it comes with Chinese values, 00:48:09.600 --> 00:48:13.237 Chinese training, Chinese biases and so forth. We would prefer to have it be 00:48:13.320 --> 00:48:17.557 American. Um we will see. There is a there's a 00:48:17.640 --> 00:48:20.717 couple of open source projects in America that I am a 00:48:20.800 --> 00:48:24.837 supporter of, but they can't raise the 10 billion 00:48:24.920 --> 00:48:28.797 dollars that are needed from the public markets to get to where they want. So, 00:48:28.880 --> 00:48:33.477 they are gems, but they're not at the scale. I've argued that the US 00:48:33.560 --> 00:48:35.597 government should help fund them. I've 00:48:35.680 --> 00:48:37.357 argued that philanthropists should help 00:48:37.440 --> 00:48:40.637 fund them, but I don't really know. 00:48:40.720 --> 00:48:45.557 Okay, that's very helpful. So, one last question and then we're going to uh 00:48:45.640 --> 00:48:50.597 take questions from the audience. If you were picking two or three questions 00:48:50.680 --> 00:48:52.477 that a graduate student or an 00:48:52.560 --> 00:48:58.117 undergraduate interested in this space, AI and geopolitics, should be thinking 00:48:58.200 --> 00:49:01.757 about. Give us two or three questions that we 00:49:01.840 --> 00:49:05.877 might have somebody that has an answer to or part of an answer to 00:49:05.960 --> 00:49:09.677 the next time you come. So, so a couple well, again, we're dealing with some of 00:49:09.760 --> 00:49:10.797 the smartest people in the world here. 00:49:10.880 --> 00:49:14.917 So, the first question is what does it mean to be human in the age of AI? 00:49:15.000 --> 00:49:19.597 That is a question that you can write many PhD theses on. So, basically, study 00:49:19.680 --> 00:49:23.357 history, study philosophy, study how people work, study economics and then 00:49:23.440 --> 00:49:26.837 figure out what this new technology is going to do. We always in in my 00:49:26.920 --> 00:49:30.597 industry, because we didn't take those classes, we always ignore those things, 00:49:30.680 --> 00:49:34.477 right? You guys are capable of answering these questions in ways and if we get 00:49:34.560 --> 00:49:38.677 out of line, maybe you can use that to call to call us out. 00:49:38.760 --> 00:49:40.677 The second question has to do with this 00:49:40.760 --> 00:49:44.397 China-US rivalry. Why China versus US? 00:49:44.480 --> 00:49:48.997 The only two countries that are going to matter. And the reason is you need an 00:49:49.080 --> 00:49:52.997 enormous amount of money and an enormous number of people. And as much as I like 00:49:53.080 --> 00:49:55.717 Europe, Europe is not organized, doesn't have enough people, doesn't have enough 00:49:55.800 --> 00:49:59.397 money to do it. India is not organized enough yet to do 00:49:59.480 --> 00:50:02.597 it, although they're trying. And most of the other countries don't have enough 00:50:02.680 --> 00:50:05.157 money, don't have enough of the talent, don't have the right universities and so 00:50:05.240 --> 00:50:07.677 forth. And then the third question has 00:50:07.760 --> 00:50:09.797 to do much more with conflict. 00:50:09.880 --> 00:50:13.197 In a world where you have terrorists who 00:50:13.280 --> 00:50:17.637 have access to AI and you have governments that have access to AI, what 00:50:17.720 --> 00:50:22.077 does conflict look like? What does a terrorist attack look like against a 00:50:22.160 --> 00:50:25.597 major power? Obviously, I'm not endorsing that, it's a terrible thing. 00:50:25.680 --> 00:50:27.917 How do we defend against it? Same thing 00:50:28.000 --> 00:50:30.357 for China versus US. But what about 00:50:30.440 --> 00:50:32.877 Russia versus Ukraine? What about Europe 00:50:32.960 --> 00:50:37.717 versus whatever? Trying to understand how conflict plays out in an algorithmic 00:50:37.800 --> 00:50:42.557 warfare where the AI is driving everything is a very, very fertile area 00:50:42.640 --> 00:50:47.240 for research and new ideas. And it's just starting. 00:50:47.680 --> 00:50:50.677 So, good topics for somebody to work on 00:50:50.760 --> 00:50:53.997 in the interim. Let's start here. Please 00:50:54.080 --> 00:50:57.237 introduce yourself and a short question. 00:50:57.320 --> 00:51:00.717 Hi, my name is Teresa. I'm a second year student here at the Kennedy School and 00:51:00.800 --> 00:51:04.837 I'm in Professor Allison's national security class. We had a class talk 00:51:04.920 --> 00:51:08.237 about cybersecurity and most of the tech that 00:51:08.320 --> 00:51:11.357 underpins US homeland security is all in 00:51:11.440 --> 00:51:15.837 private hands. So, what governance governance model do you think will 00:51:15.920 --> 00:51:20.077 actually help work to coordinate government and private companies 00:51:20.160 --> 00:51:24.797 to during especially during like a AI type of emergency? 00:51:24.880 --> 00:51:28.597 And given we talked a lot about like US-China rivalry 00:51:28.680 --> 00:51:31.117 and despite the technology and 00:51:31.200 --> 00:51:35.877 investment rivalry, there's also a big difference in governance model. So, how 00:51:35.960 --> 00:51:40.677 do you think the two governance model will lead in terms of there's a homeland 00:51:40.760 --> 00:51:44.317 security cyber crisis? Well, first place, I think in under the Trump 00:51:44.400 --> 00:51:47.477 administration, you're not going to see much regulation of AI. That's pretty 00:51:47.560 --> 00:51:52.237 clear. In China, it appears that they're allowing the the companies to do 00:51:52.320 --> 00:51:56.157 whatever they want. Although they have the laws about 00:51:56.240 --> 00:51:59.077 various things, they don't seem to be implementing them. So, it looks like 00:51:59.160 --> 00:52:01.717 it's an all-out business conflict. The 00:52:01.800 --> 00:52:04.197 biggest concern I have is cyber attacks. 00:52:04.280 --> 00:52:07.357 If you can write code the way that I've 00:52:07.440 --> 00:52:09.517 seen these things write, and every 00:52:09.600 --> 00:52:14.437 company in my world is now using programmers and AI programmers together. 00:52:14.520 --> 00:52:18.277 It's extraordinary. It's happened very quickly. Look at Claude code, for 00:52:18.360 --> 00:52:22.117 example, the most recent one which is by far at the top. It's from Anthropic. And 00:52:22.200 --> 00:52:25.637 others coming. Gemini, of course, claims it has a competitor, but at the moment 00:52:25.720 --> 00:52:29.037 Claude code is a bit better, it looks like. 00:52:29.120 --> 00:52:32.037 If you can write code, you can also write cyber attacks 00:52:32.120 --> 00:52:34.117 because the objective function is easy. 00:52:34.200 --> 00:52:36.917 Just keep writing code until you break something. 00:52:37.000 --> 00:52:39.557 And if you have enough hardware and enough energy, you can just keep doing 00:52:39.640 --> 00:52:43.717 that. So, I think there's going to be a many, many more cyber attacks. 00:52:43.800 --> 00:52:47.397 And that doesn't mean from governments, it could also be from terrorists and and 00:52:47.480 --> 00:52:51.237 bad organizations. And I think getting organized for that would be my primary 00:52:51.320 --> 00:52:54.717 concerns. Please. 00:52:54.800 --> 00:53:00.157 Hi. Thank you for coming. I'm David Wideman. I'm MPP2 here. 00:53:00.240 --> 00:53:05.357 You You mentioned the strength of current closed-source models. You can tell me if 00:53:05.440 --> 00:53:08.357 I'm wrong, but I think open-source models are about a half generation 00:53:08.440 --> 00:53:10.917 behind the current closed-source models. 00:53:11.000 --> 00:53:14.437 Perhaps it's a little dangerous to characterize open-source versus 00:53:14.520 --> 00:53:19.600 closed-source as an America versus China issue. 00:53:20.160 --> 00:53:23.037 And my my fear is that the Silicon Valley people that you're talking about 00:53:23.120 --> 00:53:27.117 are seeking regulatory capture and that's why they want to fearmonger 00:53:27.200 --> 00:53:31.437 around open-source models. What would it take if we wanted to 00:53:31.520 --> 00:53:36.197 establish a moratorium on closed-source models if that's something that we seek? 00:53:36.280 --> 00:53:39.797 What would it take in order to do establish this moratorium worldwide? 00:53:39.880 --> 00:53:44.117 I think I just don't agree with your question, so I'm sorry. 00:53:44.200 --> 00:53:48.797 I I don't see the open-source leadership in America. And I see nothing but 00:53:48.880 --> 00:53:52.437 open-source leadership in China. I think those are the facts. 00:53:52.520 --> 00:53:57.117 So, I don't think there's regulatory capture on the open-source side. 00:53:57.200 --> 00:54:00.317 And I think again under the Trump administration, it's highly unlikely 00:54:00.400 --> 00:54:03.797 that you're going to see significant regulation of the closed-source 00:54:03.880 --> 00:54:08.197 companies. But I think the closed-source decision is largely driven by economics, 00:54:08.280 --> 00:54:10.157 not policy. If you see my point. 00:54:10.240 --> 00:54:13.037 Literally, you just can't I mean, think about the cost of these things. We're 00:54:13.120 --> 00:54:15.917 talking about $10 billion, $20 billion. 00:54:16.000 --> 00:54:19.277 How would you raise that without it? 00:54:19.360 --> 00:54:21.877 And I think Eric, this goes back to 00:54:21.960 --> 00:54:24.077 before making our list of questions, I 00:54:24.160 --> 00:54:26.597 would add add to it. How does the 00:54:26.680 --> 00:54:29.077 financing considerations 00:54:29.160 --> 00:54:32.077 impact the strategic choices? Because if 00:54:32.160 --> 00:54:34.597 it if it just happens to be an accident 00:54:34.680 --> 00:54:37.557 of the financial 00:54:37.640 --> 00:54:42.637 structure, that wouldn't necessarily reflect the national interest. That 00:54:42.720 --> 00:54:46.717 would just reflect the capital markets as they are. I think 00:54:46.800 --> 00:54:51.077 that it's important to acknowledge that America has the most extraordinary 00:54:51.160 --> 00:54:53.157 capital market financial system in the 00:54:53.240 --> 00:54:55.957 world by far. I think the numbers are 00:54:56.040 --> 00:54:59.477 60% of the volume and 90% of of the 00:54:59.560 --> 00:55:03.197 value dollar and so forth is in US dollars. 00:55:03.280 --> 00:55:07.037 And so countries that are not dollar denominated do fear 00:55:07.120 --> 00:55:11.917 the power of that financial market. And we see it in the ability to raise money. 00:55:12.000 --> 00:55:15.757 When I was in China, it was quite clear in talking to my friends, they don't 00:55:15.840 --> 00:55:19.117 have access to the depth of the financial market. They literally cannot 00:55:19.200 --> 00:55:23.557 get the money. The venture financing that was in China is cut by a factor of 00:55:23.640 --> 00:55:28.157 five since three years ago. Now, there are many reasons for that, not just our 00:55:28.240 --> 00:55:32.477 world and not just the US. But without that access to capital, it's very hard 00:55:32.560 --> 00:55:37.037 to to do these large models with these companies and training. Now, you can 00:55:37.120 --> 00:55:42.037 imagine everything I'm saying changing if the underlying 00:55:42.120 --> 00:55:45.317 algorithms change. There are people who are working on new non-transformer 00:55:45.400 --> 00:55:48.517 models which are less economically 00:55:48.600 --> 00:55:52.437 expensive. There are many people who have commented on the energy efficiency 00:55:52.520 --> 00:55:56.117 of human brains versus the cost of these data centers. And trust me, our brains 00:55:56.200 --> 00:56:01.077 are complicated, but they're not energy they're not energy expensive. So, so 00:56:01.160 --> 00:56:04.117 again, there there are there might be a breakthrough that would change the 00:56:04.200 --> 00:56:07.157 calculus that we're discussing. Go ahead. Please. 00:56:07.240 --> 00:56:10.837 Thank you, Dr. Schmidt. My name is Fatjon Semanajaku. I'm an 00:56:10.920 --> 00:56:13.277 MC/MPA student here at Harvard Kennedy 00:56:13.360 --> 00:56:16.557 School. Originally from Albania. 00:56:16.640 --> 00:56:19.597 In the age of AI, you and Dr. Kissinger 00:56:19.680 --> 00:56:23.637 describe a world where AI systems begin 00:56:23.720 --> 00:56:26.757 to interpret reality for us. 00:56:26.840 --> 00:56:29.197 If human strategic judgment becomes 00:56:29.280 --> 00:56:31.077 shaped, even subtly, by 00:56:31.160 --> 00:56:33.877 machine-generated frames, what becomes 00:56:33.960 --> 00:56:36.477 the anchor of responsibility in global 00:56:36.560 --> 00:56:39.277 affairs? In other words, when an 00:56:39.360 --> 00:56:42.077 AI-influenced decision produces a 00:56:42.160 --> 00:56:44.877 real-world consequences, who holds the 00:56:44.960 --> 00:56:47.837 moral burden? The human who acted, the 00:56:47.920 --> 00:56:51.317 institution that deployed the system, or 00:56:51.400 --> 00:56:53.677 the algorithm that shaped the 00:56:53.760 --> 00:56:56.837 perception? And how should democracies 00:56:56.920 --> 00:56:59.877 redesign their institutions before this 00:56:59.960 --> 00:57:03.117 ambiguity becomes a geopolitical 00:57:03.200 --> 00:57:05.557 vulnerability? A very well-phrased 00:57:05.640 --> 00:57:08.957 point. So, I worry that the future of 00:57:09.040 --> 00:57:11.757 democracy is uncertain 00:57:11.840 --> 00:57:16.237 simply because I'll use an American example. We believe in free speech and 00:57:16.320 --> 00:57:18.277 I'm certainly in favor of absolute free 00:57:18.360 --> 00:57:20.677 speech in America. I'm not in favor of 00:57:20.760 --> 00:57:23.437 boosted speech or algorithmic speech. 00:57:23.520 --> 00:57:28.317 So, where is the line, right? If if I say something that's wrong and then the 00:57:28.400 --> 00:57:31.517 algorithm, because my claim is outrageous, decides to spread it 00:57:31.600 --> 00:57:35.837 everywhere, is that appropriate in a democracy? And you can imagine, and 00:57:35.920 --> 00:57:37.317 again, I'm not making a partisan thing. 00:57:37.400 --> 00:57:41.277 I think any any any part of the political system can use this for 00:57:41.360 --> 00:57:43.797 whatever effect. The ability to generate 00:57:43.880 --> 00:57:45.757 misinformation that people really 00:57:45.840 --> 00:57:50.237 believe is so simple now that I hope the 00:57:50.320 --> 00:57:54.357 answer is better education and critical thinking among human beings. But you 00:57:54.440 --> 00:57:58.357 could imagine if I were evil, which I hope I'm not, and I I sat down and I 00:57:58.440 --> 00:58:02.637 started to flood everyone with my particular unique messages, that I could 00:58:02.720 --> 00:58:05.357 overwhelm your belief in truth by simply 00:58:05.440 --> 00:58:09.797 relentless copying and repetition. We know that there's somebody called 00:58:09.880 --> 00:58:13.877 essentially anchor bias. If you hear it first, then you judge So, if I managed 00:58:13.960 --> 00:58:17.477 to get the message to you first that the building is on fire, which it's not, so 00:58:17.560 --> 00:58:21.597 I'm not committing a crime. If I managed to get that first, you then 00:58:21.680 --> 00:58:26.157 anchor from that point. You see the danger of this. It's very real. I don't 00:58:26.240 --> 00:58:28.077 think and my my answer to your question 00:58:28.160 --> 00:58:30.917 is that every every democracy will face 00:58:31.000 --> 00:58:35.517 this. And I think democracies will make different decisions based on cultural 00:58:35.600 --> 00:58:39.557 values and their understanding of the threat. You won't see a single answer 00:58:39.640 --> 00:58:43.037 from democracies. And by the way, the way authoritarian countries solve this 00:58:43.120 --> 00:58:46.397 problem is they just ban it. Right. So, a lot of this stuff is 00:58:46.480 --> 00:58:49.357 illegal in authoritarian countries because they have the ability to 00:58:49.440 --> 00:58:53.917 suppress speech. And obviously, we're we're all in favor of free speech. 00:58:54.000 --> 00:58:58.317 Please. Good. Yes, ma'am. Yes, thank you very much for for coming 00:58:58.400 --> 00:59:02.317 here this evening. So, I'm called Elena Kraine and I'm a fellow at the Belfast 00:59:02.400 --> 00:59:04.237 Center. So, I'm wondering firstly, could 00:59:04.320 --> 00:59:06.757 you comment on the role of humans such 00:59:06.840 --> 00:59:09.437 as you and me in the very long term with 00:59:09.520 --> 00:59:13.837 regards to AI? What's our what is our role become? And also, 00:59:13.920 --> 00:59:16.157 we often frame, you know, because of the 00:59:16.240 --> 00:59:18.877 value systems, the US-China debate as a 00:59:18.960 --> 00:59:23.437 competition. But is there is there value to thinking about that in terms of a 00:59:23.520 --> 00:59:26.597 collaboration? And maybe more interestingly, you know, you mentioned 00:59:26.680 --> 00:59:28.957 Europe being maybe somewhat disorganized, but they also have 00:59:29.040 --> 00:59:33.037 enormous strength and enormous talent like Mistral in France and so on. 00:59:33.120 --> 00:59:36.277 Is there something to do there with um collaboration? 00:59:36.360 --> 00:59:39.837 Um so so a couple comments on Europe, um 00:59:39.920 --> 00:59:43.877 I was the first investor in Mistral, so I'm a big Mistral fan. 00:59:43.960 --> 00:59:48.037 Mistral cannot raise the money that their US competitors. They have the 00:59:48.120 --> 00:59:50.277 same problem and they're working on 00:59:50.360 --> 00:59:55.277 solutions to that. Um with respect to US and China, 00:59:55.360 --> 00:59:58.560 the um 00:59:58.760 --> 01:00:03.397 I started because of Henry, I I spent about 5 years looking at Chinese peer 01:00:03.480 --> 01:00:07.917 competition. And I thought that it would be possible 01:00:08.000 --> 01:00:11.917 to get closer to China. And I eventually through his work and 01:00:12.000 --> 01:00:15.477 others discovered that the Chinese were more afraid of competing partnering with 01:00:15.560 --> 01:00:20.837 us than we were with them. So it takes two to tango, right? And I 01:00:20.920 --> 01:00:24.957 think it's highly unlikely. I think you're going to see in your lifetime 01:00:25.040 --> 01:00:28.757 um hopefully understanding that we have to 01:00:28.840 --> 01:00:33.717 coexist, but it's highly unlikely that the two systems will become best buddies 01:00:33.800 --> 01:00:35.597 for the obvious reasons. 01:00:35.680 --> 01:00:37.557 I think on this this other question 01:00:37.640 --> 01:00:41.557 about what is what are humans useful for 01:00:41.640 --> 01:00:46.357 uh in the very long term. Um it's very very clear that humans are 01:00:46.440 --> 01:00:49.677 social animals and want to be around other humans. 01:00:49.760 --> 01:00:53.597 It's also very clear that there are some things that we need, medical care and so 01:00:53.680 --> 01:00:56.037 forth, which will be delivered by humans. 01:00:56.120 --> 01:00:58.477 I think that that it's a rough way to 01:00:58.560 --> 01:01:03.637 say this is most other functions will be doable by 01:01:03.720 --> 01:01:07.837 computers. Will we allow them? That's a question 01:01:07.920 --> 01:01:09.677 for your research. 01:01:09.760 --> 01:01:11.837 Right? Is where is that line? 01:01:11.920 --> 01:01:15.237 My my old example was the following. 01:01:15.320 --> 01:01:19.077 This is back when we were building Waymo at Google. So the thought experiment is 01:01:19.160 --> 01:01:23.357 New York City is nothing but Waymo and competitor automobile cars. 01:01:23.440 --> 01:01:25.957 And the engineers at Google and the other companies have figured out how to 01:01:26.040 --> 01:01:30.757 completely optimize the traffic so that the the streets which don't change, you 01:01:30.840 --> 01:01:34.957 have the absolutely mathematically true highest loading per street ever and it's 01:01:35.040 --> 01:01:39.317 seamless. And then you have somebody with an emergency, a pregnant woman, you know, 01:01:39.400 --> 01:01:42.757 whatever who has to get to the hospital and needs an exception. 01:01:42.840 --> 01:01:47.437 Is there an exception button in the car which says you have to violate all the 01:01:47.520 --> 01:01:51.477 rules because there's something uh Now if there is not an exception button 01:01:51.560 --> 01:01:56.597 in the car, then that's dominance of humans by a computer. 01:01:56.680 --> 01:02:00.877 And humans will revolt. If I go back to the pickaxe versus religion, humans will 01:02:00.960 --> 01:02:02.997 fight that. That will be seen as 01:02:03.080 --> 01:02:06.037 oppression by the government, in this 01:02:06.120 --> 01:02:10.637 case, in the governor of New York City, whatever metaphor you want, 01:02:10.720 --> 01:02:12.637 and the computer companies themselves. 01:02:12.720 --> 01:02:16.757 If the system, on the other hand, is adaptable to human needs 01:02:16.840 --> 01:02:20.437 and says, "Ah, we have an actual, you know, medical emergency and not somebody 01:02:20.520 --> 01:02:23.717 who's high on drugs or, you know, whatever who's just goofing around or 01:02:23.800 --> 01:02:26.237 some kid who's, you know, playing with the buttons." 01:02:26.320 --> 01:02:30.357 Um and it reasons and it says, "Oh my god, I've got to do everything I can to 01:02:30.440 --> 01:02:32.237 get the the person to the hospital." 01:02:32.320 --> 01:02:36.477 Then it'll be more accepted. So a lot of this depends on whether it integrates 01:02:36.560 --> 01:02:38.677 with our human experience and needs. 01:02:38.760 --> 01:02:42.757 Does it limit our freedom or does it increase freedom? 01:02:42.840 --> 01:02:46.437 Uh I've come to the view, I'm old enough now to believe that preserving our 01:02:46.520 --> 01:02:47.957 freedom, our freedom of thought, of 01:02:48.040 --> 01:02:50.517 motion, of assembly, of gathering, you 01:02:50.600 --> 01:02:55.397 know, of all of the things, right? Uh are really really important. If they 01:02:55.480 --> 01:02:58.797 impinge on our freedom, then they will be fought and I will be leading that 01:02:58.880 --> 01:03:01.197 fight. 01:03:01.280 --> 01:03:03.437 Please. 01:03:03.520 --> 01:03:06.157 Namaste, sir. My name is Ilma Afroz. I'm 01:03:06.240 --> 01:03:11.277 from India. You mentioned India in your conversation. I'm lucky to be Professor 01:03:11.360 --> 01:03:12.837 Allison's student and to be learning 01:03:12.920 --> 01:03:15.717 from him. My question is how we can 01:03:15.800 --> 01:03:18.637 bring together US and India. India has 01:03:18.720 --> 01:03:21.037 got some excellent talented people. How 01:03:21.120 --> 01:03:23.517 we can bring both our countries together 01:03:23.600 --> 01:03:26.957 in order uh to create a world where 01:03:27.040 --> 01:03:29.997 democracies flourish and it's a win-win 01:03:30.080 --> 01:03:32.757 partnership for both our countries. 01:03:32.840 --> 01:03:37.317 Um I strongly agree. I've spent lots of time in India and the 01:03:37.400 --> 01:03:41.797 literally because of the IITs and the quality of the people, the 01:03:41.880 --> 01:03:46.517 um the depth of talent in India is extraordinary. However, the depth of 01:03:46.600 --> 01:03:50.317 computing is not. Um last year we did a calculation, there 01:03:50.400 --> 01:03:54.877 were only about a thousand GPUs in the entire country for 1 billion people. So 01:03:54.960 --> 01:03:57.037 I and others have organized to try to to 01:03:57.120 --> 01:04:02.077 fix that. Um so I was alarmed with the most recent 01:04:02.160 --> 01:04:05.837 trade war, uh which I think has set uh India and 01:04:05.920 --> 01:04:09.557 the United States back and that needs to get fixed. I don't understand the trade 01:04:09.640 --> 01:04:11.757 war and I just my position is that's 01:04:11.840 --> 01:04:14.477 hurting us. India is our natural 01:04:14.560 --> 01:04:17.397 partner, right? It's a democracy, it's a 01:04:17.480 --> 01:04:20.637 messy democracy, but we are too. Um 01:04:20.720 --> 01:04:22.957 w- in in Silicon Valley, most of the 01:04:23.040 --> 01:04:25.037 people I work with are of 01:04:25.120 --> 01:04:28.117 um essentially South Asian origin with 01:04:28.200 --> 01:04:32.717 India or some of the other countries. So at least in Silicon Valley, the Indians 01:04:32.800 --> 01:04:37.077 are flourishing and I want the tightest possible integration with them. 01:04:37.160 --> 01:04:42.357 Thank you. Please. Hi, my name's Josh. I'm an MPP2 here at 01:04:42.440 --> 01:04:45.957 the Kennedy School. Earlier, you outlined two national strategies 01:04:46.040 --> 01:04:50.557 earlier. There was pushing the frontier or pursuing adoption, but ideally you 01:04:50.640 --> 01:04:55.037 have both, so thinking about adoption in the US, what 01:04:55.120 --> 01:04:59.357 are some of the major barriers to enterprise adoption right now? 01:04:59.440 --> 01:05:02.517 Does that vary by industry and what should government do to help address 01:05:02.600 --> 01:05:06.077 those barriers if anything? Um the government usually doesn't do a 01:05:06.160 --> 01:05:08.797 very good job on that. Um 01:05:08.880 --> 01:05:10.917 the industry believes that there is 01:05:11.000 --> 01:05:13.397 something called a technology overhang, 01:05:13.480 --> 01:05:16.117 that we, your friendly industry, have 01:05:16.200 --> 01:05:18.877 produced more tools than you are using. 01:05:18.960 --> 01:05:20.677 Now that that is our belief, whether 01:05:20.760 --> 01:05:22.837 it's true or not, you can discuss. 01:05:22.920 --> 01:05:26.397 Um and a lot of it seems to be readiness 01:05:26.480 --> 01:05:29.917 to adopt the technology, uh the fact that a lot of the stuff is 01:05:30.000 --> 01:05:32.877 software, most companies don't have very good software people. I'm sorry to say 01:05:32.960 --> 01:05:36.077 that. Um the the changes internally that 01:05:36.160 --> 01:05:40.717 are required. Um I personally believe that this technology adoption thing is 01:05:40.800 --> 01:05:45.677 just a temporary problem and that as new CEOs come in and winners 01:05:45.760 --> 01:05:47.637 emerge, the competitive pressure, which 01:05:47.720 --> 01:05:49.797 are very high in America, will cause 01:05:49.880 --> 01:05:52.837 that adoption. Um you don't see very 01:05:52.920 --> 01:05:57.477 much adoption in regulated industries because regulation is used as a excuse 01:05:57.560 --> 01:06:00.557 to not innovate, but at least in the hard innovative 01:06:00.640 --> 01:06:04.757 industries, I think this will get solved. But the general answer is I 01:06:04.840 --> 01:06:07.117 would not have the government do very much because I don't think it would 01:06:07.200 --> 01:06:11.717 help. Um what would help is just having every business understand that if they 01:06:11.800 --> 01:06:15.197 want to make money, which is the capitalist vision, they need to use AI 01:06:15.280 --> 01:06:19.117 much more profoundly. Remember, you can target your customers, you can serve 01:06:19.200 --> 01:06:23.277 them, you can understand them, you can talk to them and so forth, all with AI. 01:06:23.360 --> 01:06:26.877 Um and a lot of this there's a lot of negative issues with this. For example, 01:06:26.960 --> 01:06:28.997 it may lead to layoffs in companies. 01:06:29.080 --> 01:06:33.397 There's lots of examples where low-end jobs are being replaced by computers and 01:06:33.480 --> 01:06:37.157 that's obviously a job loss, that's a societal problem, but from an adoption 01:06:37.240 --> 01:06:40.077 perspective, the answer is just time. 01:06:40.160 --> 01:06:42.997 Please. Hi, my name's Ayush. I'm a third-year 01:06:43.080 --> 01:06:44.917 PhD student in computer science doing AI 01:06:45.000 --> 01:06:50.277 research. Um my question is regarding um you were mentioning China's sort of 01:06:50.360 --> 01:06:53.997 intense focus on automating AI in business. I guess in the Valley it seems 01:06:54.080 --> 01:06:56.877 like, you know, every day there's a new startup that comes around like there's 01:06:56.960 --> 01:07:01.357 the big seed targeting some automation for workflow and we we also sort of have 01:07:01.440 --> 01:07:03.557 the the 9-9-6 fad. 01:07:03.640 --> 01:07:05.677 Um what would you identify as like a key 01:07:05.760 --> 01:07:08.997 distinguisher in China's AI um 01:07:09.080 --> 01:07:12.877 automating business that you think the US needs to catch up on or at least in 01:07:12.960 --> 01:07:17.637 the the Silicon Valley startups now that is What is your PhD topic? I work on 01:07:17.720 --> 01:07:19.317 reasoning for large language models. 01:07:19.400 --> 01:07:22.517 Perfect. Right? So what is your time frame for 01:07:22.600 --> 01:07:26.517 AGI? Uh I'd give it like six to seven years. 01:07:26.600 --> 01:07:30.117 Yeah, you see, not San Francisco consensus, East Coast consensus. Good. 01:07:30.200 --> 01:07:31.517 We will see. 01:07:31.600 --> 01:07:37.680 You may be right. Um I think that the 01:07:38.320 --> 01:07:42.077 It has to do, I think, with the grandeur of the dreams. 01:07:42.160 --> 01:07:45.237 When I'm in China, I don't hear the same 01:07:45.320 --> 01:07:48.557 rhetoric as I do in California. 01:07:48.640 --> 01:07:53.797 In California, it's two years, the world will change, no one is ready, 01:07:53.880 --> 01:07:56.717 this is happening so fast and so forth and so on. It's this sort of rhetoric 01:07:56.800 --> 01:07:59.997 that drives and it's self-reproducing, it's a belief system, it's like a 01:08:00.080 --> 01:08:01.237 religion. 01:08:01.320 --> 01:08:04.797 Now it always takes longer than than uh 01:08:04.880 --> 01:08:07.157 the dreamers say. I don't hear that in 01:08:07.240 --> 01:08:11.837 China. It it's it's it's noticeably different. 01:08:11.920 --> 01:08:15.317 So for example, with reasoning, you know what DeepSeek did with R3, they did a 01:08:15.400 --> 01:08:18.837 fantastic job, they invented a new way of doing supervised fine-tuning. I mean, 01:08:18.920 --> 01:08:22.757 it's really really clever stuff. You just don't hear it at a national scale. 01:08:22.840 --> 01:08:25.877 And by the way, DeepSeek is a national champion for China. 01:08:25.960 --> 01:08:28.597 Right? They literally now are, you know, they they're they're on the list of 01:08:28.680 --> 01:08:32.077 famous companies, they're getting a enormous amount of money. Uh last time I 01:08:32.160 --> 01:08:35.437 met with them, they said, "We've solved our hardware problem." which was code 01:08:35.520 --> 01:08:36.757 for the government is just going to give 01:08:36.840 --> 01:08:42.760 them a lot of chips. Right? Welcome to a communist country. 01:08:43.480 --> 01:08:47.157 Okay, so this gentle lady here, please. Hi, thank 01:08:47.240 --> 01:08:48.597 you so much for being here. I'm Sonia. 01:08:48.680 --> 01:08:51.077 I'm a joint degree student with the MBA 01:08:51.160 --> 01:08:55.757 and uh and the policy school. Um this might be a slightly more out there 01:08:55.840 --> 01:08:59.957 question and and open-ended, but you talk a lot about what it means to be 01:09:00.040 --> 01:09:03.917 human and I think a lot of that is consciousness. 01:09:04.000 --> 01:09:07.597 Um and I'm curious when I think about theories of consciousness, which 01:09:07.680 --> 01:09:11.837 obviously is very ill-defined, AI is hitting more and more of those and so it 01:09:11.920 --> 01:09:14.517 it like if anything, we need a revolution in how we think about 01:09:14.600 --> 01:09:17.197 consciousness, but I'm curious how you 01:09:17.280 --> 01:09:20.837 think about consciousness in AI and if 01:09:20.920 --> 01:09:24.557 you do think that there's a possibility that AI is or will become conscious, 01:09:24.640 --> 01:09:27.917 what that looks like, what model welfare looks like in that case. Well, let me 01:09:28.000 --> 01:09:29.357 ask let me ask you a simple question. 01:09:29.440 --> 01:09:34.517 So, we have I'm going to give Graham credit for being conscious. 01:09:34.600 --> 01:09:37.997 And in return, he appears to be conscious. And in return, Graham is 01:09:38.080 --> 01:09:41.397 going to give it to me. So, for purposes of argument, the computer is on the 01:09:41.480 --> 01:09:43.717 table. And we ask the computer, "Are you 01:09:43.800 --> 01:09:46.597 conscious?" And it says, "Yes." 01:09:46.680 --> 01:09:49.037 And so, you, being the smart graduate 01:09:49.120 --> 01:09:51.917 student here at Harvard, um 01:09:52.000 --> 01:09:54.037 come up with a series of questions. 01:09:54.120 --> 01:09:56.997 It answers every question correctly. 01:09:57.080 --> 01:09:59.837 How do we know? How could you know that the computer is 01:09:59.920 --> 01:10:04.637 conscious? How would you understand its internal reasoning state? 01:10:04.720 --> 01:10:08.077 Now, you could instrument the way its reasoning works, which people are now 01:10:08.160 --> 01:10:11.197 doing. What they're doing is they're watching what are called supernodes 01:10:11.280 --> 01:10:15.797 within the um within the weight structure to watch how it's actually 01:10:15.880 --> 01:10:19.357 making its decisions. So, maybe you could discover consciousness by 01:10:19.440 --> 01:10:22.437 inspection, but that's speculation. 01:10:22.520 --> 01:10:24.917 So, I sat down with a bunch of neuroscientists cuz I didn't know the 01:10:25.000 --> 01:10:28.917 answer to this question. I said, "How does it How does consciousness evolve?" 01:10:29.000 --> 01:10:32.477 And their theory, which is just a theory, 01:10:32.560 --> 01:10:36.997 was that consciousness evolves when you have dissimilar systems that are working 01:10:37.080 --> 01:10:41.277 together and growing, and they develop an awareness of the 01:10:41.360 --> 01:10:44.317 other. Right? And that human consciousness 01:10:44.400 --> 01:10:47.157 evolved because we needed to understand 01:10:47.240 --> 01:10:51.477 that we were something. Right? That the the the development of the id, the 01:10:51.560 --> 01:10:55.957 identity, and so forth and so on was necessary in order to command the system. 01:10:56.040 --> 01:10:58.037 Now, there's no way to prove that. 01:10:58.120 --> 01:11:02.197 So, the answer is I don't know, but this is a good another it's a fourth question 01:11:02.280 --> 01:11:04.717 to ask this audience to work on. 01:11:04.800 --> 01:11:07.757 And and there's two questions. The first, how does it work? And the second 01:11:07.840 --> 01:11:11.397 is, how would you verify it? Thank you. 01:11:11.480 --> 01:11:13.997 I have the unfortunate 01:11:14.080 --> 01:11:18.677 responsibility to say that we've come to the witching hour. We there Graham, this 01:11:18.760 --> 01:11:21.877 is so much fun. I love this. Well, you want to stay for a while more? Five more 01:11:21.960 --> 01:11:23.157 minutes. This is good. I love this. 01:11:23.240 --> 01:11:28.117 >> no. We'll stay forever. I'm sorry. It's just I was pro- I I promised your I 01:11:28.200 --> 01:11:32.037 promised your schedulers that I would get you out of here. So, please. 01:11:32.120 --> 01:11:34.237 Short questions and short answers. 01:11:34.320 --> 01:11:37.477 Uh thanks, Eric. Troop here from Harvard Business School, a second-year student 01:11:37.560 --> 01:11:40.797 there. Also a background in international relations before my MBA. 01:11:40.880 --> 01:11:43.837 Um you Are you convincingly about the need for an 01:11:43.920 --> 01:11:48.077 IAEA or International Atomic Energy Agency equivalent for AI? 01:11:48.160 --> 01:11:51.317 So, my question is, how does the State Department, how does the Pentagon, and 01:11:51.400 --> 01:11:54.997 similar agencies need to configure themselves to prepare for this the 01:11:55.080 --> 01:11:57.317 coming era of superintelligence? And my 01:11:57.400 --> 01:12:02.437 second, slightly risky question is, um if you are looking for someone to help 01:12:02.520 --> 01:12:06.477 you this summer or this winter to work on this topic, I'd be very happy to. 01:12:06.560 --> 01:12:08.477 I love people with courage. 01:12:08.560 --> 01:12:11.357 Um so, the IAEA question. So, there's a 01:12:11.440 --> 01:12:16.477 group of people who are very close to me who concluded that the only solution to 01:12:16.560 --> 01:12:18.357 the problems that I'm answering 01:12:18.440 --> 01:12:22.197 is to create the equivalent of CERN. 01:12:22.280 --> 01:12:25.877 And the idea was that we would get together, all of us, including China, 01:12:25.960 --> 01:12:30.197 you know, everybody, and we put it all in here. Take all the best minds, and 01:12:30.280 --> 01:12:31.997 we'd all work on this. 01:12:32.080 --> 01:12:35.797 And um as a result, we would 01:12:35.880 --> 01:12:40.117 m- m- build this huge future because the benefits to humanity of this are so 01:12:40.200 --> 01:12:43.237 great. Start thinking about every disease 01:12:43.320 --> 01:12:47.637 eliminated, solutions to real hard problems. The We have Megan here. Energy 01:12:47.720 --> 01:12:51.997 issues, and she's the expert. All of this kind of stuff. 01:12:52.080 --> 01:12:55.397 I think it's pretty you probably say it's probably pretty 01:12:55.480 --> 01:12:59.397 unlikely that's going to happen. So, then the next group says, "Well, then we 01:12:59.480 --> 01:13:00.997 need an a an 01:13:01.080 --> 01:13:04.237 IAEA, which is essentially a mandatory 01:13:04.320 --> 01:13:07.237 inspection group for nuclear." 01:13:07.320 --> 01:13:09.797 And what in that scenario, what happens 01:13:09.880 --> 01:13:14.517 is you'd have to go, and this group would visit the data centers and the 01:13:14.600 --> 01:13:18.157 algorithms of every company in every country. 01:13:18.240 --> 01:13:22.917 Now, that occurred, remember, the Vienna Group occurred 01:13:23.000 --> 01:13:26.397 after 15 years of negotiation, including people like Henry, 01:13:26.480 --> 01:13:30.677 and after two nuclear bombs were dropped on Hiroshima and Nagasaki, which we can 01:13:30.760 --> 01:13:35.437 all agree was hor- horrific. We've not had such an example. 01:13:35.520 --> 01:13:39.357 There are people that I know who say the following, and they don't say this with 01:13:39.440 --> 01:13:43.877 evil intent. They say that we won't we will have an 01:13:43.960 --> 01:13:45.437 event 01:13:45.520 --> 01:13:48.437 that will force this IAEA thing, and we 01:13:48.520 --> 01:13:52.117 hope that it is a Chernobyl-level event. 01:13:52.200 --> 01:13:53.597 And I said, "What does that mean?" And 01:13:53.680 --> 01:13:56.717 they said, "Not that many people died." 01:13:56.800 --> 01:14:00.037 Okay? As opposed to a nuclear attack. 01:14:00.120 --> 01:14:04.597 So, there's a a strain of thinking in my world that is consistent with that. 01:14:04.680 --> 01:14:07.917 But I think uh first, I think nobody knows, and it's 01:14:08.000 --> 01:14:12.877 unlikely that the countries will um 01:14:12.960 --> 01:14:16.997 put up with that uh until there's a real crisis. 01:14:17.080 --> 01:14:21.197 I used to say that in 01:14:21.280 --> 01:14:23.477 I don't know, 20 years 01:14:23.560 --> 01:14:26.117 after the climate is being destroyed 01:14:26.200 --> 01:14:31.077 and, you know, the Earth is melting and sea level rise, there's going to be a 01:14:31.160 --> 01:14:33.757 there's going to there's going to be a meeting of all the powers where people 01:14:33.840 --> 01:14:35.717 say, "Well, we really screwed this up. 01:14:35.800 --> 01:14:37.637 We now have to fix it." 01:14:37.720 --> 01:14:42.797 So, I think there is precedent for a global challenge and a resolution, but 01:14:42.880 --> 01:14:46.437 it's very messy. It's not done rationally. It's not like 01:14:46.520 --> 01:14:50.437 you and I agree. There has to be some crisis and some political dynamic, and 01:14:50.520 --> 01:14:53.597 it can be quite severe. It's an opportunity for the next 01:14:53.680 --> 01:14:57.677 generation. Let's do this very quick. Do we take the three questions? Just do 01:14:57.760 --> 01:15:01.597 your short questions. We'll let Eric wrap up, please. Yes. 01:15:01.680 --> 01:15:04.517 Hi. Uh thank you for your extra time. Uh my name is Kevin. I'm a second-year at 01:15:04.600 --> 01:15:09.197 the law school. Um my question is about automation and jobs. Um a couple months 01:15:09.280 --> 01:15:12.677 ago, Sam Altman went on an interview saying that insinuating that if 01:15:12.760 --> 01:15:16.557 something could be automated, maybe it wasn't a real job to begin with. Uh I 01:15:16.640 --> 01:15:20.317 want to get your thoughts on that statement, and if you agree with the 01:15:20.400 --> 01:15:24.437 premise that if something can be automated, should it? Um and relatedly, 01:15:24.520 --> 01:15:28.197 if you disagree, what are ways that we as a society can decide what we should 01:15:28.280 --> 01:15:31.317 and shouldn't automate? >> Sure. Please have it. 01:15:31.400 --> 01:15:35.237 Um good night. My name is Griselda. I'm a middle mid-career MPA and a former 01:15:35.320 --> 01:15:37.437 Googler, so I'm very happy to see you. 01:15:37.520 --> 01:15:39.957 Um my question is regarding algorithm 01:15:40.040 --> 01:15:42.437 diplomacy. Like Dr. Kissinger used to 01:15:42.520 --> 01:15:45.757 say that constructive um ambiguity and 01:15:45.840 --> 01:15:50.037 the human pause is really necessary, but then AI is binary. 01:15:50.120 --> 01:15:53.317 Are we getting to a towards a world where de-escalation is computationally 01:15:53.400 --> 01:15:57.677 impossible? And does this push for new algorithm diplomacy that we could push 01:15:57.760 --> 01:16:01.237 harder? Ben. Uh thank you. My name is Kanishk. 01:16:01.320 --> 01:16:04.197 I'm from Harvard Business School. Uh my question is about ethical 01:16:04.280 --> 01:16:08.317 superintelligence. Uh first, do you Why do you not believe that if America 01:16:08.400 --> 01:16:11.757 develops superintelligence, other nations will follow and be able to copy 01:16:11.840 --> 01:16:15.277 it, like China did with Deep Seek? And then second of all, if we're trying to 01:16:15.360 --> 01:16:18.317 embody it with ethics, then if someone else comes up with an unethical 01:16:18.400 --> 01:16:22.117 superintelligence, isn't that going to be strictly more free and more capable 01:16:22.200 --> 01:16:26.597 than a constrained superintelligence model that America tries to develop? 01:16:26.680 --> 01:16:29.437 You It's uh one word about your question again? 01:16:29.520 --> 01:16:33.397 Uh yeah, I wanted to ask what your thoughts were on Sam Altman's quote. 01:16:33.480 --> 01:16:38.837 Yeah. >> So, um one of the things that people at this 01:16:38.920 --> 01:16:43.797 level forget is that human dignity involves purpose. 01:16:43.880 --> 01:16:47.677 And an awful lot of jobs provide purpose to an awful lot of people. And the loss 01:16:47.760 --> 01:16:52.397 of those jobs is a major crisis, not financially, but emotionally, in terms 01:16:52.480 --> 01:16:55.757 of their meaning. So, in order for us to get through this, 01:16:55.840 --> 01:16:58.357 we're going to have to address that. 01:16:58.440 --> 01:17:01.877 Right? We're going to have to actually do the right thing. 01:17:01.960 --> 01:17:06.597 And the right thing will be some combination of better tools and so forth. 01:17:06.680 --> 01:17:10.717 I'm not as worried about the jobs issue because we're producing fewer humans, 01:17:10.800 --> 01:17:12.717 which I view as a major crisis. 01:17:12.800 --> 01:17:16.797 Um collectively, you all are having fewer children than my generation, which 01:17:16.880 --> 01:17:20.397 is having fewer children than my parents' generation, and so forth. And 01:17:20.480 --> 01:17:25.597 we need more humans. Um partly I'm a businessman, so you need more customers. 01:17:25.680 --> 01:17:29.397 But the important point is, we need more humans. And if we have less humans, then 01:17:29.480 --> 01:17:33.917 there's going to be open jobs and no people to fill them. AI can help take 01:17:34.000 --> 01:17:36.357 people who are not ready for a job and get them. 01:17:36.440 --> 01:17:40.397 One word from yours, aside from being a a future Googler? 01:17:40.480 --> 01:17:42.357 No, the algorithm diplomacy and 01:17:42.440 --> 01:17:44.997 information and computation. So, in your 01:17:45.080 --> 01:17:47.757 premise of your question, you described 01:17:47.840 --> 01:17:50.077 the algorithm as as binary. 01:17:50.160 --> 01:17:54.957 And so, if you think about Kissinger in 1971, when he came up with strategic 01:17:55.040 --> 01:17:59.317 ambiguity, do you think that a computer 01:17:59.400 --> 01:18:04.357 in one or two years will be able to invent strategic diplomacy ambiguity, 01:18:04.440 --> 01:18:08.077 given that it's already been done before? I think the answer is yes. 01:18:08.160 --> 01:18:10.477 So, I think that that um the algorithms 01:18:10.560 --> 01:18:15.397 are getting so intelligent that as long as it's a concept that's been around in 01:18:15.480 --> 01:18:19.717 the past, it can probably figure out a way to apply it. So, I don't agree with 01:18:19.800 --> 01:18:23.677 you that it's as binary as your question thing. And your question was 01:18:23.760 --> 01:18:26.077 >> Uh superintelligence. Superintelligence. 01:18:26.160 --> 01:18:28.560 Um 01:18:30.840 --> 01:18:33.837 I I think it's the same answers, right? 01:18:33.920 --> 01:18:38.477 It Superintelligence in one form or another is going to apply. 01:18:38.560 --> 01:18:41.717 My own view, which is maybe different from the consensus, 01:18:41.800 --> 01:18:44.837 is that we're going to develop brilliant 01:18:44.920 --> 01:18:47.837 AI physicists, brilliant AI biologists, 01:18:47.920 --> 01:18:49.797 brilliant AI chemists, brilliant AI 01:18:49.880 --> 01:18:52.677 writers, brilliant AI historians. 01:18:52.760 --> 01:18:56.277 Um but that that the concept and and 01:18:56.360 --> 01:18:59.037 there'll be systems that can drive them. 01:18:59.120 --> 01:19:02.717 It's not at all obvious that if you were 01:19:02.800 --> 01:19:05.037 Einstein in 1902 with that amount of 01:19:05.120 --> 01:19:07.397 math available to you, that you would 01:19:07.480 --> 01:19:10.837 have had the brilliance to invent um 01:19:10.920 --> 01:19:12.917 special relativity 01:19:13.000 --> 01:19:15.157 using the algorithms of today. 01:19:15.240 --> 01:19:18.437 In the industry, there is a view that 01:19:18.520 --> 01:19:20.277 that's the next really hard problem. 01:19:20.360 --> 01:19:24.317 There are various theories about it. Uh one answer is you could just do uh, 01:19:24.400 --> 01:19:27.477 repetition. You could just keep asking questions, you know, the monkey at the 01:19:27.560 --> 01:19:31.277 keyboard kind of thing, and eventually you do it. Another way is that you could 01:19:31.360 --> 01:19:34.917 give the optimization function to be curiosity, and if you just wait long 01:19:35.000 --> 01:19:38.597 enough, it'll discover special relativity. But that's not how Einstein 01:19:38.680 --> 01:19:40.437 did it. Einstein sat with his little pen 01:19:40.520 --> 01:19:43.077 and paper at the age of 18 or 17 with a 01:19:43.160 --> 01:19:46.357 lamp and figured it out. We're not there 01:19:46.440 --> 01:19:48.837 yet. My own view is that's going to be a 01:19:48.920 --> 01:19:52.877 very hard boundary. So, in other words, 01:19:52.960 --> 01:19:56.757 we're going to get to superhuman behavior, which is not the same thing as 01:19:56.840 --> 01:19:59.957 superintelligence, which I would say Einstein and Kissinger 01:20:00.040 --> 01:20:03.837 and so forth really were. Graham, it's such a privilege to be here with 01:20:03.920 --> 01:20:07.557 >> It's such a pleasure to have you here, and we look forward to having you back. 01:20:07.640 --> 01:20:11.200 Let's say thank you very much. Thank you all.