WEBVTT 00:00:04.890 --> 00:00:07.610 From the New York Times, I'm Michael Bavaro. 00:00:07.740 --> 00:00:12.820 This is The Daily. Last night, in a lavish ceremony, 00:00:13.230 --> 00:00:17.130 President Trump welcomed Chinese leader Xi Jinping 00:00:17.340 --> 00:00:22.750 As he arrived in the United States, for two days of closely watched negotiations. 00:00:25.990 --> 00:00:30.570 On their crowded agenda, what if any role they should play 00:00:30.720 --> 00:00:33.500 in regulating artificial intelligence. 00:00:33.690 --> 00:00:37.207 amid a global freakout over its risks. 00:00:37.290 --> 00:00:39.350 Thank you. Thank you. today. 00:00:39.960 --> 00:00:42.600 As my colleague David Sanger explains... 00:00:42.900 --> 00:00:47.740 When it comes to protecting humanity from technological catastrophe, 00:00:48.000 --> 00:00:53.580 there's really only one proven model for the two leaders to look to. 00:00:56.240 --> 00:00:59.820 It's Thursday, September 24th. 00:01:03.000 --> 00:01:03.880 David? 00:01:04.500 --> 00:01:05.560 Up from Washington. 00:01:06.510 --> 00:01:09.150 DC sitting here in our New York studios. 00:01:09.750 --> 00:01:12.510 And I didn't even have to come to you this time. 00:01:13.170 --> 00:01:17.816 I just had to come see the Dailies Empire in New York. In person? In person. 00:01:17.899 --> 00:01:22.690 Yeah. It's pretty remarkable. It's a humble empire. It's a late stage empire. 00:01:24.470 --> 00:01:29.947 So, David, when President Trump and the leader of China, Xi Jinping, meet China, 00:01:30.030 --> 00:01:35.031 In Washington today, they're going to broach this subject on many people's 00:01:35.115 --> 00:01:40.047 minds, which is the risk of artificial intelligence, that risk including, 00:01:40.130 --> 00:01:42.757 of course, these swarming rogue people. 00:01:42.840 --> 00:01:48.170 AI agents that have been hacking into companies and basically breaking the rules 00:01:48.254 --> 00:01:51.840 of their creators. And what she and Trump uniquely... 00:01:52.050 --> 00:01:55.270 can maybe do about it is going to be the subject of 00:01:55.410 --> 00:02:00.127 at least some of this meeting. And I think we've all come to understand that, 00:02:00.210 --> 00:02:02.010 The leaders of the U.S. and China... 00:02:02.100 --> 00:02:06.153 are uniquely positioned to talk about the safety of AI because they're the countries 00:02:06.237 --> 00:02:07.327 with the most advanced 00:02:07.410 --> 00:02:12.607 frontier AI laboratories. And the only way that AI gets contained is, 00:02:12.690 --> 00:02:17.944 is if both of those countries somehow act together and at the same time. 00:02:18.027 --> 00:02:22.370 And as this meeting, approached and this is why we love you so much 00:02:22.800 --> 00:02:25.340 You reached out to us to say, you know... 00:02:25.620 --> 00:02:32.107 I have seen this movie before. Big fears, big thoughts about coordination, anxiety. 00:02:32.190 --> 00:02:34.327 and solutions to big problems. 00:02:34.410 --> 00:02:37.030 You told us there's a historical parallel. 00:02:37.470 --> 00:02:41.040 to all of this and maybe a historical solution. 00:02:41.550 --> 00:02:45.416 Well, Michael, there is an obvious parallel. 00:02:45.499 --> 00:02:52.230 We had an era starting in the late 1940s and through the Cold War where the 00:02:52.440 --> 00:02:55.859 Two states, the Soviet Union and the United States, 00:02:55.942 --> 00:03:00.390 had created a potentially terrifying technology that had all kinds 00:03:00.474 --> 00:03:05.334 of commercial uses but could also bring about civilizational extinction. 00:03:05.418 --> 00:03:10.828 Nuclear weapons. Nuclear weapons. And everybody had to go figure out how we were 00:03:10.911 --> 00:03:15.771 going to get the benefits without getting the terror. And you know what? 00:03:15.855 --> 00:03:17.640 It took decades to happen. 00:03:22.440 --> 00:03:29.129 Came up with agreements that ultimately have brought us to the remarkable moment 00:03:29.213 --> 00:03:35.394 that there has been no use of a nuclear weapon in anger since 1945. Right. 00:03:35.477 --> 00:03:40.980 Nuclear nonproliferation worked. Nuclear nonproliferation worked. 00:03:41.070 --> 00:03:44.670 combined with a balance of terror worked. 00:03:44.850 --> 00:03:50.560 So now, quite naturally, everybody's looking to that model as we begin to think 00:03:50.643 --> 00:03:54.373 about how you might contain artificial intelligence, 00:03:54.457 --> 00:03:59.653 particularly after this summer of surprises that we've had where we have 00:03:59.737 --> 00:04:04.870 rogue agents breaking out of their sandboxes. Right. Treat AI the way. 00:04:05.250 --> 00:04:06.790 superpowers treated. 00:04:06.960 --> 00:04:12.191 nuclear weaponry. Well, look, if you think that the heads of some of these frontier 00:04:12.275 --> 00:04:17.058 companies are coming out and saying there was a 10% chance of civilizational 00:04:17.141 --> 00:04:21.924 extinction, right? That's probably a higher proportion chance than there was 00:04:22.008 --> 00:04:27.047 in the early days of the nuclear era, that we would wipe each other off the map, 00:04:27.130 --> 00:04:30.460 at least its equivalent. And so now the question is, 00:04:30.960 --> 00:04:35.166 Can two very different countries in very different times, 00:04:35.249 --> 00:04:41.185 China and the United States, come to some kind of framework agreement that could 00:04:41.268 --> 00:04:44.880 begin to get us there? Okay. Well, given all of… 00:04:45.120 --> 00:04:45.953 That. 00:04:46.830 --> 00:04:50.010 In your capacity, David, as a historian of history, 00:04:50.190 --> 00:04:56.380 all the world's great technological dangers, I think it would be instructive, 00:04:56.670 --> 00:04:58.810 to have you tell us the story 00:04:58.950 --> 00:05:03.363 of nuclear nonproliferation, how that old framework that kept nuclear weapons 00:05:03.447 --> 00:05:07.568 from going off around the world could, in theory, apply right now to AI. 00:05:07.651 --> 00:05:10.488 So just give us a quick history lesson here. Well, 00:05:10.571 --> 00:05:14.985 let's just start with when the U.S. developed its first atomic weapon through 00:05:15.068 --> 00:05:19.656 the Manhattan Project. Familiar to anybody who's been off to see the Oppenheimer 00:05:19.740 --> 00:05:23.010 movie or read some of this history. It was in the 1940s. 00:05:28.950 --> 00:05:33.316 another country would beat us to it. But it wasn't the Soviets at that time. 00:05:33.400 --> 00:05:36.888 It was Germany. In a last desperate attempt to escape defeat, 00:05:36.971 --> 00:05:40.693 German science threw everything it had into the perfection of V2. 00:05:40.777 --> 00:05:43.470 Where the fear of American intelligence was... 00:05:43.560 --> 00:05:47.282 the Germans would get a nuclear weapon before the United States. 00:05:47.365 --> 00:05:51.443 And there was this just abiding fear we had to get there first. Right. 00:05:51.527 --> 00:05:55.248 In the spreading, teeming laboratories of the Manhattan Project, 00:05:55.332 --> 00:05:56.580 the pace intensified. 00:05:56.670 --> 00:06:01.820 Groves had ordered that the first form must be ready for testing in mid-July. 00:06:01.903 --> 00:06:03.670 And in the summer of 1945, 00:06:04.140 --> 00:06:07.580 Scientists from Los Alamos went out to do the Trinity test, 00:06:07.663 --> 00:06:12.680 which was the first explosion of a nuclear bomb to see if the technology could work. 00:06:22.290 --> 00:06:25.057 would never see again. There were... 00:06:25.140 --> 00:06:29.852 People within the Los Alamos program who were doing mathematical calculations 00:06:29.936 --> 00:06:34.585 and coming to the conclusion that we could light the entire atmosphere afire 00:06:34.669 --> 00:06:37.160 and destroy the earth in doing the test. 00:06:39.820 --> 00:06:46.820 But despite those fears, they went ahead with the test. 00:06:51.100 --> 00:06:52.660 Just weeks later. 00:06:55.140 --> 00:06:59.440 American airplane dropped one bomb on Hiroshima. 00:06:59.760 --> 00:07:06.760 came Hiroshima. And days after that, 00:07:14.580 --> 00:07:19.596 Nagasaki. Four square miles of the city were blasted into dust and a black rain 00:07:19.679 --> 00:07:23.100 of destruction hovered over the ruins for many hours. 00:07:23.220 --> 00:07:27.870 And in total, Those two bombs killed around 200,000 people. 00:07:28.520 --> 00:07:31.768 Of course, the world had never seen anything like this before. 00:07:31.851 --> 00:07:34.860 Tens of thousands of homes and buildings were wiped out. 00:07:35.760 --> 00:07:40.469 the devastation was everywhere. Japan was driven to quick surrender. 00:07:40.552 --> 00:07:46.260 For the future, the choice is peace or total destruction. The atomic age is here. 00:07:54.840 --> 00:07:59.830 you But somewhat astonishingly, it didn't stop the arms race. 00:08:00.210 --> 00:08:04.677 Of course, it turned out to be just the beginning. And now, of course, 00:08:04.760 --> 00:08:10.007 the U.S. thought it had an insurmountable lead over countries that didn't yet have 00:08:10.090 --> 00:08:13.580 nuclear weapons. That lead didn't last very long. 00:08:13.770 --> 00:08:18.985 The Soviets were working away on this. They had a bunch of atomic spies along 00:08:19.069 --> 00:08:21.944 the way who helped them to it. And by 1949, 00:08:22.028 --> 00:08:25.950 they had conducted a test and we were off into the races. 00:08:26.370 --> 00:08:31.736 And the 1950s were a period of great terror where each side was not only 00:08:31.819 --> 00:08:37.336 building up its arsenal, but building up the powers of these weapons as if 00:08:37.420 --> 00:08:42.483 the weapons used at Hiroshima and Nagasaki were not powerful enough. 00:08:42.566 --> 00:08:47.856 And by this stage, proliferation was starting. Off northwest Australia, 00:08:47.939 --> 00:08:53.010 this was the place chosen to explode Britain's first atomic weapon. 00:08:56.370 --> 00:09:00.815 Britain and France had nuclear weapons. China wasn't that far behind. 00:09:00.898 --> 00:09:04.770 I've just been told that China has exploded an atomic bomb. 00:09:05.070 --> 00:09:07.147 Right, things are starting to spin. 00:09:07.230 --> 00:09:10.230 Kind of out of control. They are. And then we had... 00:09:10.410 --> 00:09:16.147 one big terrifying incident. The purpose of these bases can be none other 00:09:16.230 --> 00:09:20.448 than to provide a nuclear strike capability against the Western Hemisphere. 00:09:20.531 --> 00:09:22.687 And it was the Cuban Missile Crisis... 00:09:22.770 --> 00:09:27.562 in October of 1962. This was when U.S. intelligence came up with evidence 00:09:27.645 --> 00:09:31.235 that the Soviets were moving nuclear weapons onto Cuba, 00:09:31.318 --> 00:09:34.307 that they would be just 90 miles from Florida, 00:09:34.390 --> 00:09:38.114 that they could easily reach any place on the East Coast. 00:09:38.197 --> 00:09:43.256 The waterfront in Havana and along other parts of the coast bristled with gun 00:09:43.339 --> 00:09:48.866 emplacements as the Cuban regime waited to see what their bosses in the Kremlin were 00:09:48.949 --> 00:09:49.783 to do. 00:09:52.770 --> 00:09:55.724 This was intolerable. But the question came, 00:09:55.808 --> 00:09:59.927 is this worth going to war over? Rockets, bombs, ammunition. 00:10:00.010 --> 00:10:04.230 Air support for any limited war ground operation which might develop. 00:10:04.313 --> 00:10:07.930 Within 12 hours, all units were assembled and ready to go. 00:10:08.170 --> 00:10:09.430 and 00:10:09.520 --> 00:10:15.467 There were moments when Kennedy thought, and many around him thought, that... 00:10:15.550 --> 00:10:20.350 The United States was going to disappear, you know, in the next night or two. Wow. 00:10:26.170 --> 00:10:29.434 A final chapter in the Cuban crisis is brought to a close. 00:10:29.517 --> 00:10:32.207 As President Kennedy holds a news conference... 00:10:32.290 --> 00:10:35.147 But in the end, we know how this ended. 00:10:35.230 --> 00:10:38.470 I have today been informed by Chairman Khrushchev 00:10:38.920 --> 00:10:43.937 that all of the IL-28 bombers now in Cuba will be withdrawn in 30 days. 00:10:44.020 --> 00:10:48.520 The nuclear crisis was averted at the last moment. 00:10:48.610 --> 00:10:52.608 And Kennedy then went on to give a really famous speech. 00:10:52.691 --> 00:10:57.564 It was a commencement speech at American University in June of 1963. 00:10:57.647 --> 00:11:01.135 So it was just months before he was assassinated. 00:11:01.218 --> 00:11:03.550 I realized the pursuit of peace. 00:11:04.150 --> 00:11:07.640 is not as dramatic. as the pursuit of war 00:11:07.840 --> 00:11:13.629 And he basically said, look, the terror of this event has made us think we cannot 00:11:13.712 --> 00:11:14.800 live with this. 00:11:17.800 --> 00:11:19.520 heirs that we all inhabit, 00:11:19.660 --> 00:11:24.944 this small planet. And he said, our most basic common link is that we all inhabit 00:11:25.027 --> 00:11:28.540 this small planet. We all breathe the same air. 00:11:28.660 --> 00:11:31.060 We all cherish our children's futures. 00:11:31.210 --> 00:11:34.130 And we are all mortal. And we are all mortal. 00:11:34.930 --> 00:11:38.590 And that's the line that has resonated ever since. 00:11:39.490 --> 00:11:44.909 He announced at that moment that there would be negotiations for a first tiny 00:11:44.993 --> 00:11:49.810 step. First, Chairman Khrushchev, Prime Minister McMillan and I have agreed 00:11:50.380 --> 00:11:53.740 that high-level discussions will shortly begin in Moscow, 00:11:54.310 --> 00:11:59.690 looking towards early agreement on a comprehensive test ban treaty. 00:11:59.920 --> 00:12:03.077 He called it the Comprehensive Nuclear Test Ban Treaty. 00:12:03.160 --> 00:12:06.400 Our hopes must be tempered with the caution of history. 00:12:06.850 --> 00:12:08.830 But with our hopes go the hopes... 00:12:08.920 --> 00:12:10.240 of all mankind. 00:12:10.570 --> 00:12:15.782 That the U.S. at that moment, he said, would stop testing in the atmosphere above 00:12:15.865 --> 00:12:20.358 ground. You know, those you've seen the images of the mushroom clouds, 00:12:20.441 --> 00:12:25.196 the mushroom clouds. Right. And that over time we had to move to reduction 00:12:25.279 --> 00:12:28.090 of nuclear weapons. But let's start simple. 00:12:28.450 --> 00:12:33.884 with testing, with freezes, and with the first discussion of how you'd keep 00:12:33.967 --> 00:12:37.709 this technology from spreading to many other states. 00:12:37.792 --> 00:12:43.373 Because Kennedy himself had said that he thought there could be 15 or even 20 00:12:43.457 --> 00:12:49.590 nuclear-armed nations coming. by the mid 70s. Wow. 15 or 20. That's right. 00:12:49.870 --> 00:12:54.589 And you began to see a real cultural awakening in the United States about 00:12:54.673 --> 00:12:59.129 the depth of this danger. Here's Tony going to his Cub Scout meeting. 00:12:59.212 --> 00:13:03.537 Tony knows the bomb can explode any time of the year, day or night. 00:13:03.620 --> 00:13:04.870 He is ready for it. 00:13:04.960 --> 00:13:10.004 So, of course, this is the era when the government is encouraging schools to do 00:13:10.087 --> 00:13:14.500 those ridiculous duck and cover exercises. Duck and cover exercises. 00:13:14.740 --> 00:13:18.060 Attaboy, Tony. That flash means act fast. 00:13:18.670 --> 00:13:21.615 Of course, everybody knew that none of this would help. 00:13:21.698 --> 00:13:24.038 Older people will help us as they always do, 00:13:24.121 --> 00:13:27.810 but there might not be any grown-ups around when the bomb explodes. 00:13:28.570 --> 00:13:30.380 Then... You're on your own. 00:13:31.030 --> 00:13:36.815 And, you know, this was when Tom Lehrer, the famous songwriter and satirist, 00:13:36.899 --> 00:13:41.757 had, you know, his song. So long, mom. I'm off to drop the bomb. 00:13:41.840 --> 00:13:46.467 So don't wait up for me. And who's next? France got the bomb, 00:13:46.551 --> 00:13:50.251 but don't you grieve because they're on our side. 00:13:50.334 --> 00:13:53.957 I believe China got the bomb, but have no fears. 00:13:54.041 --> 00:13:57.670 They can't wipe us out for at least five years. 00:14:01.030 --> 00:14:06.914 countries are going to seek their next weapons. Who's next? Who's next? 00:14:06.997 --> 00:14:08.930 Who's next? Who's next? 00:14:12.670 --> 00:14:14.770 And I think this created... 00:14:14.890 --> 00:14:20.165 and incredible political pressure on the Johnson administration to get this thing 00:14:20.248 --> 00:14:22.630 done. And in 1968, it came together. 00:14:24.480 --> 00:14:28.351 After endless diplomatic meetings back and forth, 00:14:28.435 --> 00:14:34.808 occupying lots of days at the Waldorf Astoria Hotel in New York and other places 00:14:34.891 --> 00:14:41.426 around the world before they came up with what became the Nuclear Nonproliferation 00:14:41.509 --> 00:14:45.623 Treaty. "Distinguished guests, ladies and gentlemen, 00:14:45.706 --> 00:14:51.840 this is a very reassuring and hopeful moment in the relations among nations. 00:14:52.030 --> 00:14:58.277 And on July 1st, 1968, Five years after that Kennedy speech, 00:14:58.360 --> 00:15:01.641 The treaty was signed by President Johnson at the White House. 00:15:01.724 --> 00:15:04.980 We have come here today to the East Room of the White House. 00:15:05.410 --> 00:15:11.493 to sign a treaty which limits the spread of nuclear weapons. 00:15:11.576 --> 00:15:15.912 And then it was signed in the Soviet Union, 00:15:15.995 --> 00:15:22.995 and then by about 60 countries in total. Man can still shape his destiny in 00:15:23.703 --> 00:15:28.430 the nuclear age and learn to live as brothers. 00:15:30.130 --> 00:15:32.981 And here's what the treaty says. First of all, 00:15:33.064 --> 00:15:37.573 it says we understand there are five countries that already have nuclear 00:15:37.656 --> 00:15:42.038 weapons and they all commit to eliminating their stockpiles over time. 00:15:42.121 --> 00:15:44.290 It doesn't specify when. And then. 00:15:44.440 --> 00:15:48.527 Countries that do not have nuclear weapons will commit that they will never die. 00:15:48.610 --> 00:15:52.210 build them. But in return, they get 00:15:52.540 --> 00:15:57.561 help for peaceful nuclear technology to build nuclear power plants and so forth. 00:15:57.644 --> 00:16:02.155 And then, of course, there needs to be an inspection regime to make sure 00:16:02.238 --> 00:16:07.259 that they're not cheating. And that was the creation of the International Atomic 00:16:07.342 --> 00:16:11.151 Energy Agency, which is based in Vienna. It's part of the UN. 00:16:11.234 --> 00:16:16.255 And they begin the process of developing teams that go visit countries that have 00:16:16.338 --> 00:16:17.172 signed this 00:16:22.540 --> 00:16:24.660 being diverted off to bomb projects. 00:16:26.100 --> 00:16:31.578 And As we look back at this, this was even a bigger moment than we knew at the time. 00:16:31.662 --> 00:16:33.810 What do you mean? Because... 00:16:34.000 --> 00:16:39.749 Nations that seem to be inevitably racing to adopt a fairly non-complicated 00:16:39.832 --> 00:16:44.571 technology to build nuclear weapons all agreed to halt. Right. 00:16:44.654 --> 00:16:47.920 There's very few moments in history where. 00:16:48.220 --> 00:16:49.053 Everybody. 00:16:49.840 --> 00:16:54.377 who wants to catch up with the leaders in an arms race, voluntarily... 00:16:54.460 --> 00:16:57.848 abandon the pursuit. That's right. Almost everybody. 00:16:57.931 --> 00:17:01.520 There are a couple of outliers and Israel never signed. 00:17:01.603 --> 00:17:04.657 And while it's never admitted to having a bomb, 00:17:04.741 --> 00:17:08.796 it's not exactly the biggest secret in the Middle East. Right. 00:17:08.880 --> 00:17:12.334 North Korea signed but renounced it in the mid 1990s. 00:17:12.418 --> 00:17:15.672 And they now have an arsenal of 60 to 100 weapons, 00:17:15.756 --> 00:17:18.960 depending on who you believe. India and Pakistan 00:17:24.460 --> 00:17:28.300 worry about Iran, but they don't actually have a nuclear weapon yet. 00:17:28.383 --> 00:17:30.460 But ultimately, this treaty prevents 00:17:30.580 --> 00:17:37.166 the kind of spreading of nuclear weaponry that Kennedy so worried about. It did. 00:17:37.249 --> 00:17:40.500 And it's remarkable. Kennedy was wrong. 00:17:40.630 --> 00:17:47.270 We've only got nine nuclear states, and many of these countries have had nuclear 00:17:47.353 --> 00:17:53.152 weapons for decades. And so far, it has remained relatively contained, 00:17:53.236 --> 00:17:59.707 which is why this does seem like perhaps the best model we have to think about 00:17:59.790 --> 00:18:05.086 a comparable international regulatory system for restraining AI. 00:18:05.169 --> 00:18:07.690 That's exactly right, Michael. 00:18:07.900 --> 00:18:14.900 And as everybody enters this early stage debate about whether or not you can do 00:18:17.270 --> 00:18:24.270 the same with artificial intelligence, here is an example, a rare example, 00:18:26.047 --> 00:18:33.047 in which nations have voluntarily agreed to slow a technology, 00:18:37.900 --> 00:18:41.625 the leaders of the frontier companies. They're not saying stop it. 00:18:41.709 --> 00:18:45.376 They're not saying let's go out of business. They're saying, hey, 00:18:45.460 --> 00:18:46.960 let's just slow this down. 00:18:47.110 --> 00:18:52.326 so that we can understand how to control it before we find ourselves in another 00:18:52.409 --> 00:18:57.491 crisis. Right. In some ways, a more modest request than what nonproliferation 00:18:57.574 --> 00:19:01.649 was about. You're right, Michael. It is a more modest request, 00:19:01.733 --> 00:19:04.550 but it's a far more complicated situation. 00:19:10.510 --> 00:19:11.890 We'll be right back. 00:19:17.770 --> 00:19:22.367 So David, understanding as we now do, that nuclear nonproliferation 00:19:22.450 --> 00:19:27.010 as a model is our best and so far really our only goal. 00:19:27.160 --> 00:19:31.277 model for AI. In what ways... 00:19:31.360 --> 00:19:35.477 is it quite challenging to apply that model to AI right now? 00:19:35.560 --> 00:19:38.977 You just said it's very different and complicated. 00:19:39.060 --> 00:19:43.947 So there was something of a remarkable simplicity to the arms race with 00:19:44.030 --> 00:19:47.237 the Soviet Union, even while it was terrifying. 00:19:47.320 --> 00:19:52.640 What was simple about it? Well, two countries had almost all the technology, 00:19:52.990 --> 00:19:57.495 We were only worried about states. We were not worried about non-state actors, 00:19:57.578 --> 00:20:01.640 terrorist groups, people working in their basements and 00:20:01.970 --> 00:20:06.094 If you didn't have plutonium or uranium, you couldn't make a bomb. 00:20:06.178 --> 00:20:10.130 So we knew that there was a way to sort of cut off the oxygen. 00:20:10.520 --> 00:20:16.482 There was a limited number of people with launch authority in each country, 00:20:16.565 --> 00:20:23.011 and both countries had their own political dynamic for wanting to make sure there 00:20:23.095 --> 00:20:29.140 was no accident and there was no escalation if they could avoid it. Mm-hmm. 00:20:29.330 --> 00:20:32.982 That didn't mean they were willing to give up the weapons, 00:20:33.065 --> 00:20:36.327 but it did mean that you could contain the problem. 00:20:36.410 --> 00:20:41.184 But when you think about AI, the situation is completely different. 00:20:41.268 --> 00:20:45.110 The governments don't really control this technology. 00:20:46.130 --> 00:20:49.408 You don't need the equivalent of uranium and plutonium. 00:20:49.491 --> 00:20:53.441 You just need compute a lot of it. And maybe that's the oxygen for 00:20:53.524 --> 00:20:58.230 this particular case. Computer power. You need computer power and tons of it. 00:20:58.370 --> 00:21:04.506 There's no reason to believe that as this technology becomes more efficient, 00:21:04.590 --> 00:21:10.399 that smaller and smaller groups working out of basements, terror groups, 00:21:10.482 --> 00:21:16.864 countries that have interest in causing disruption can't get a hold of basic AI 00:21:16.947 --> 00:21:22.430 technology. And we've already seen some examples of that happening. 00:21:22.790 --> 00:21:26.561 There's one other big difference, Michael. In the Cold War, 00:21:26.644 --> 00:21:31.395 we didn't have to convince our national leaders that this was a terrifying 00:21:31.479 --> 00:21:36.770 technology. They had to, if anything, to convince their populations of that. But. 00:21:36.860 --> 00:21:42.803 In this case, we have an American president who is declaring that all these 00:21:42.886 --> 00:21:46.180 warnings are a hoax. And Xi Jinping is... 00:21:46.580 --> 00:21:50.745 is clearly more concerned, as some of his aides have made clear, 00:21:50.828 --> 00:21:53.997 about the threat to the Communist Party of China, 00:21:54.081 --> 00:21:56.918 rather than its threat to the broader world. 00:21:57.001 --> 00:22:00.320 So you're saying the leaders of the superpowers... 00:22:00.470 --> 00:22:03.218 locked in this arms race, the AI arms race, 00:22:03.301 --> 00:22:07.959 can't even agree on what the threat is. Whereas back in the nuclear era, 00:22:08.042 --> 00:22:13.029 the threat was pretty self-evident and the leaders could agree on it. It was, 00:22:13.112 --> 00:22:16.470 this is going to kill all of us. That's right. Now, 00:22:16.910 --> 00:22:22.427 They also recognized that they were not going to give up all nuclear weapons back 00:22:22.511 --> 00:22:26.991 then because they believed they needed that for national survival. 00:22:27.074 --> 00:22:31.624 In this case, the U.S. and China both think they need to be pushing 00:22:31.707 --> 00:22:35.081 this technology ahead for their economic survival. 00:22:35.164 --> 00:22:38.884 But it's also a big part of the struggle for dominance. 00:22:38.967 --> 00:22:42.747 China's made no secret of the fact that by 2049, China, 00:22:42.830 --> 00:22:47.050 the 100th anniversary of the creation of the People's Republic of China. 00:22:47.133 --> 00:22:51.533 It wants to be the biggest military power, the biggest technological power, 00:22:51.616 --> 00:22:53.230 the biggest economic power. 00:22:54.530 --> 00:23:00.297 And you don't get to that point without AI. Right. And so if you're China... 00:23:00.380 --> 00:23:06.233 and you're suddenly faced with the prospect of a nuclear non-proliferation 00:23:06.316 --> 00:23:10.564 style agreement with your biggest adversary, the U.S., 00:23:10.648 --> 00:23:15.300 when it comes to AI, you're probably thinking to yourself, 00:23:16.490 --> 00:23:20.150 If we sign that, then we might be signing away our economic future. 00:23:21.680 --> 00:23:23.640 Adding to that suspicion... 00:23:24.140 --> 00:23:28.758 would probably be the reality, and we've covered it a lot on the show, 00:23:28.841 --> 00:23:33.140 that China already does a much more robust job of regulating AI, 00:23:33.770 --> 00:23:38.052 than the United States does. The U.S. has taken a really hands-off approach, 00:23:38.135 --> 00:23:41.153 and AI here is even more advanced than it is in China. 00:23:41.236 --> 00:23:44.510 And so if you're Xi Jinping, you're thinking to yourself, 00:23:44.750 --> 00:23:46.550 Why on earth would I sign a deal? 00:23:47.090 --> 00:23:48.990 that locks in U.S. 00:23:49.610 --> 00:23:55.539 AI advantages, It makes it harder for us to catch up and realize our AI future. 00:23:55.623 --> 00:23:57.710 It's exactly right, because... 00:23:58.040 --> 00:24:03.507 The Chinese are convinced the U.S. is already trying to do that with export 00:24:03.590 --> 00:24:06.920 controls on the world's fastest chips. Right. 00:24:07.400 --> 00:24:13.765 President Trump has agreed to give the Chinese the world's second fastest chips, 00:24:13.848 --> 00:24:20.213 but not the fastest. And the Chinese view of this is this is modern containment. 00:24:20.296 --> 00:24:23.517 This is their effort to preserve a lead. 00:24:23.600 --> 00:24:27.491 And then if you're other countries, you're probably thinking, 00:24:27.574 --> 00:24:32.247 I don't want to sign anything that's going to cut off my economic future. 00:24:32.331 --> 00:24:37.069 And if everybody in the world is running around saying the economic future 00:24:37.152 --> 00:24:40.261 is completely dependent on the development of AI, 00:24:40.344 --> 00:24:45.147 why would I sign a treaty that would keep me from being a full participant? 00:24:45.231 --> 00:24:49.137 So we're clearly running up against some of the real issues. 00:24:49.220 --> 00:24:50.053 limits. 00:24:50.390 --> 00:24:57.388 of the nuclear model for AI. So let me return to the slightly more optimistic 00:24:57.471 --> 00:25:01.150 thought that it is our only model. So... 00:25:01.250 --> 00:25:02.410 Does that mean... 00:25:02.540 --> 00:25:06.447 that President Xi and President Trump are in any way inclined to, 00:25:06.530 --> 00:25:11.805 to pursue it? Well, in some ways, they already have. What do you mean? Well, 00:25:11.889 --> 00:25:17.164 back during the Biden administration, Biden's advisers wanted to see if they 00:25:17.247 --> 00:25:22.452 could get the Chinese engaged in the most basic conversation on this topic. 00:25:22.535 --> 00:25:28.445 And it took about a year and a half to get the Chinese to agree to basically a single 00:25:28.529 --> 00:25:31.490 sentence, which is which is basically that 00:25:31.820 --> 00:25:37.928 Neither country would let artificial intelligence agents control their nuclear 00:25:38.011 --> 00:25:39.440 weapons stockpile. 00:25:39.800 --> 00:25:44.805 A seemingly very wise pledge. You wouldn't think it would take a year and a half 00:25:44.888 --> 00:25:49.287 to get to something that basic. But you're saying the U.S. and China – 00:25:49.370 --> 00:25:51.507 have come to an understanding... 00:25:51.590 --> 00:25:55.110 that feels like an extension of 00:25:55.370 --> 00:25:59.798 the nuclear containment policies that the U S and Russia and everybody else have 00:25:59.882 --> 00:26:04.760 agreed to. We're going to keep, AI and nuclear weapons from intermingling. 00:26:05.330 --> 00:26:10.532 That's the concept, Michael. Now, no one really knows what this sentence means 00:26:10.616 --> 00:26:11.307 because 00:26:11.390 --> 00:26:16.315 Supposing it simply means you're not going to let AI agents decide whether or not 00:26:16.398 --> 00:26:19.778 to launch nuclear weapons. OK, we can all agree on that. 00:26:19.861 --> 00:26:24.168 But how do you know whether the information that President Trump or one 00:26:24.251 --> 00:26:28.372 of his successors is receiving or President Xi and one of one of his 00:26:28.455 --> 00:26:33.566 successors receives about the threat that might lead to a decision to employ nuclear 00:26:33.649 --> 00:26:37.050 weapons? It wasn't influenced by AI. Yeah, faked by AI. 00:26:37.220 --> 00:26:39.100 influenced by AI, 00:26:39.290 --> 00:26:44.066 the result of a quick AI search by an aide who's sitting outside the room. 00:26:44.150 --> 00:26:48.072 So we don't really know what this means, but it was progress. 00:26:48.155 --> 00:26:53.063 And you could imagine a situation in which President Trump and President Xi, 00:26:53.146 --> 00:26:57.200 if they were willing, could build on this and not just nuclear, 00:26:57.284 --> 00:27:02.060 that we don't want AI anywhere near the development of biological weapons. 00:27:02.143 --> 00:27:06.132 Right, because AI has now reached the level of sophistication, 00:27:06.215 --> 00:27:09.117 we have come to understand, where it could... 00:27:09.200 --> 00:27:16.200 help actually design biological weapons. And we don't want AI to be used 00:27:16.990 --> 00:27:18.180 in hacking. 00:27:18.350 --> 00:27:25.008 Because we have come to understand that AI can find and exploit vulnerabilities 00:27:25.091 --> 00:27:29.360 in the electric power grid or in the water systems, 00:27:29.443 --> 00:27:33.687 that it might take human hackers months or months. 00:27:33.770 --> 00:27:38.291 or years to find and then design an attack. In other words, 00:27:38.374 --> 00:27:43.988 that it speeds up cyber attacks and is pushing us into a new era of cyber 00:27:44.072 --> 00:27:44.930 insecurity. 00:27:45.680 --> 00:27:50.340 So, David, let's just talk logistics and brass tacks. Is there a world in which... 00:27:50.510 --> 00:27:53.277 the leader of the United States and the leader of China. 00:27:53.360 --> 00:27:58.811 sit down with their teams and begin to hash out the kind of nuclear 00:27:58.895 --> 00:28:02.860 nonproliferation scheme that we're really doing. 00:28:02.960 --> 00:28:09.297 talking about here for the world of AI, in which they begin to pledge... 00:28:09.380 --> 00:28:14.386 Things like, "We both agree," that AI shouldn't touch biological weapons, 00:28:14.469 --> 00:28:18.609 shouldn't be involved in the hacking of private corporations or governments. 00:28:18.693 --> 00:28:24.440 And this all eventually finds its way into something that resembles a treaty that 00:28:24.980 --> 00:28:29.160 And brings us. To a point. where 00:28:29.900 --> 00:28:32.997 What looks like the nuclear non-perforation deal exists in AI. 00:28:33.080 --> 00:28:37.621 Well, this week, as President Xi and President Trump meet, no, 00:28:37.705 --> 00:28:41.425 I don't think there's even a remote chance of that. 00:28:41.509 --> 00:28:47.467 I think we would be lucky to get a second sentence with a joint experts committee 00:28:47.551 --> 00:28:51.048 that will meet sometime. But over the long term, 00:28:51.131 --> 00:28:54.027 I could imagine a broader conversation. 00:28:54.110 --> 00:29:00.435 understanding, not really a treaty, in which you could set up some mechanisms 00:29:00.518 --> 00:29:01.850 that might work. 00:29:02.540 --> 00:29:03.373 Early warning. 00:29:03.740 --> 00:29:09.405 sharing of models back and forth for a few days or weeks before they are publicly 00:29:09.488 --> 00:29:14.514 released. Although I think American companies would be reluctant to give 00:29:14.598 --> 00:29:17.720 the Chinese an early peak and certainly not. 00:29:17.840 --> 00:29:22.030 The inspection part of the nuclear nonproliferation treaty, 00:29:22.113 --> 00:29:27.824 you could imagine, you already saw a hint of that in an essay from Dario Amadei, 00:29:27.908 --> 00:29:31.518 the head of Anthropic, about a week and a half ago, 00:29:31.602 --> 00:29:36.371 where he described a commitment by Anthropic to allow in some group 00:29:36.454 --> 00:29:42.238 of independent inspectors to sit among the developers and watch over the creation 00:29:42.321 --> 00:29:46.160 of new and more powerful models and blow the whistle. 00:29:47.840 --> 00:29:53.489 that they believed could pose a threat to humanity. And inside OpenAI, 00:29:53.572 --> 00:29:59.917 they called this the IAEA model, the International Atomic Energy Agency model. 00:30:00.000 --> 00:30:04.336 That there are these inspectors who are independent enough that they can call up 00:30:04.419 --> 00:30:08.699 people like you and me and say, hey, there's a big problem in Anthropic and let 00:30:08.782 --> 00:30:11.019 me come over and explain it to you. Right. 00:30:11.102 --> 00:30:14.554 They're referring literally to the inspectors who exist to check 00:30:14.637 --> 00:30:18.310 on countries' nuclear programs. I mean, we keep coming back to this, 00:30:18.393 --> 00:30:21.100 but everywhere you turn, everybody ultimately is. 00:30:21.510 --> 00:30:27.201 returns to the nuclear nonproliferation model. That's true. Now, 00:30:27.284 --> 00:30:33.847 do I think this can be done by treaty? No. Treaties take forever forever. 00:30:33.930 --> 00:30:38.314 to negotiate. And this technology is racing like a freight train. 00:30:38.397 --> 00:30:43.400 The fundamental nuclear technology hasn't changed all that much from 1945. 00:30:43.483 --> 00:30:47.950 So another big difference. here is just simply speed. That's right. 00:30:48.270 --> 00:30:50.640 Hmm. Well... when I'm, 00:30:50.730 --> 00:30:54.674 Sitting here listening to you, I'm also doing a little bit of a mental calculation 00:30:54.757 --> 00:30:57.960 of the time it took for the Non-Proliferation Treaty 00:30:58.230 --> 00:31:04.264 to come into existence after the first nuclear weapon was dropped on Japan. 00:31:04.347 --> 00:31:10.440 And I think it was basically 20 something years. And, That's a long time. 00:31:10.530 --> 00:31:14.230 But it does suggest that administrations change, 00:31:14.340 --> 00:31:19.480 and viewpoints change, and these things are painstaking and they're slow, 00:31:19.564 --> 00:31:24.030 but ultimately... Everyone did agree to sign that treaty. 00:31:24.240 --> 00:31:27.930 So is it foolhardy to think that over time 00:31:28.590 --> 00:31:31.828 people will come around to the same kind of conclusion with AI, 00:31:31.912 --> 00:31:36.130 and that we could be looking back in 20 years, you'll be really gray, I'll be... 00:31:36.450 --> 00:31:39.561 just a little less gray. And we'll say, wow, can you believe it? 00:31:39.644 --> 00:31:41.994 They took their time, but they got there on AI. 00:31:42.720 --> 00:31:46.310 Your question ignores the issue of time. Look, 00:31:46.393 --> 00:31:51.660 we don't exactly understand the threats we're dealing with with AI, 00:31:51.743 --> 00:31:56.771 but there are some engineers who are saying we have just months, 00:31:56.854 --> 00:32:01.802 if a year away from recursive self-improvement, that is to say, 00:32:01.885 --> 00:32:04.520 code that keeps improving itself. 00:32:04.710 --> 00:32:09.463 or artificial general intelligence within months or a year or two, 00:32:09.547 --> 00:32:14.750 we don't have 25 years this time. We may not have two and a half years. 00:32:15.150 --> 00:32:18.827 This time, which then raises the next question. 00:32:18.910 --> 00:32:25.547 Is it going to take a Cuba missile crisis like event in AI to rally the world around 00:32:25.630 --> 00:32:29.710 this? And thank you. I think we'd rather miss that. 00:32:30.600 --> 00:32:36.287 And remember, in the atomic age, we started with Hiroshima and Nagasaki. 00:32:36.371 --> 00:32:41.500 So there was no debate about the power of these weapons to kill. 00:32:41.910 --> 00:32:48.729 But this takes us to the biggest difference between AI and nuclear. 00:32:48.813 --> 00:32:50.770 In the nuclear era, 00:32:51.330 --> 00:32:56.330 We never really doubted that we all controlled the arsenal. 00:32:56.970 --> 00:33:02.400 You know, at one point, Winston Churchill in 1955 – said, 00:33:02.790 --> 00:33:09.540 The atomic bomb, with all its terrors, did not carry us outside the scope of human 00:33:09.623 --> 00:33:10.457 control. 00:33:10.980 --> 00:33:12.260 By which he meant... 00:33:12.870 --> 00:33:18.250 Human beings were making the decisions and containing the size of the – 00:33:18.420 --> 00:33:19.880 weapon and the reaction. 00:33:20.670 --> 00:33:22.070 In AI, 00:33:23.070 --> 00:33:29.631 We have no assurance of human control. That once we've discovered that agents 00:33:29.714 --> 00:33:35.585 were hacking into Hugging Face and into other corporate sites without 00:33:35.669 --> 00:33:38.377 the knowledge of their creators, 00:33:38.460 --> 00:33:44.869 we have begun to question the sort of Frankenstein's monster nature of 00:33:44.952 --> 00:33:46.900 this whole thing. And 00:33:47.130 --> 00:33:51.450 If that's the case, then we have ignored the Churchill warning. 00:33:52.170 --> 00:33:57.301 that we are now on the verge of being outside of human control. 00:33:57.384 --> 00:34:02.350 And that wasn't really much of an issue during the pandemic. 00:34:02.580 --> 00:34:08.224 nuclear era, but it may be the most urgent problem in the age of artificial 00:34:08.307 --> 00:34:09.300 intelligence. 00:34:11.480 --> 00:34:14.702 Well, David, thank you, I guess. Well, Michael, 00:34:14.786 --> 00:34:19.920 we live in fascinating times. Yeah. Appreciate your time. Thanks. Thanks. 00:34:20.250 --> 00:34:22.310 you 00:34:23.610 --> 00:34:25.670 you 00:34:26.280 --> 00:34:30.240 *music* We'll be right back. 00:34:30.930 --> 00:34:32.560 you you 00:34:38.310 --> 00:34:42.690 Here's what else you need to know today. On Thursday morning, 00:34:42.773 --> 00:34:47.300 a federal judge found that President Trump's order barring CNN, 00:34:47.383 --> 00:34:53.080 MSNOW and Politico from the White House was likely unconstitutional and ordered 00:34:53.163 --> 00:34:56.090 that the president restore their access. 00:34:56.460 --> 00:35:02.137 The judge, a Trump appointee, rejected the president's claim that revoking the media 00:35:02.220 --> 00:35:07.020 outlet's press credentials was necessary to protect national security. 00:35:07.860 --> 00:35:11.770 Trump is expected to appeal the ruling. And: 00:35:11.853 --> 00:35:18.666 Our nation has developed the power of the government and the consequences of 00:35:18.750 --> 00:35:23.381 the sanctions. In a defiant wartime speech at the United Nations, 00:35:23.465 --> 00:35:28.518 Iran's president said that his country would not be bullied into submission 00:35:28.602 --> 00:35:30.040 by the United States. 00:35:34.360 --> 00:35:40.379 President Masood Pazeshkin vowed that Iran would defend itself until, quote, 00:35:40.463 --> 00:35:45.120 its last breath and would not give up its nuclear program. 00:35:51.280 --> 00:35:55.954 Today's episode was produced by Shannon Lynn and Jessica Chung, 00:35:56.037 --> 00:36:01.013 with help from Michael Simon Johnson. It was edited by Maria Byrne, 00:36:01.097 --> 00:36:05.640 fact-checked by Susan Lee. Contains music by Will Reed. 00:36:05.730 --> 00:36:10.647 Dan Powell and Pat McCusker, and was engineered by Chris Wood. 00:36:10.730 --> 00:36:13.230 Our theme music is by Wonderly. 00:36:13.470 --> 00:36:15.970 Special thanks to Daniel Ramirez. 00:36:22.890 --> 00:36:26.730 That's it for The Daily. I'm Michael Barbaro. 00:36:26.820 --> 00:36:28.080 See you tomorrow.