1 00:00:04,890 --> 00:00:07,610 From the New York Times, I'm Michael Bavaro. 2 00:00:07,740 --> 00:00:12,820 This is The Daily. Last night, in a lavish ceremony, 3 00:00:13,230 --> 00:00:17,130 President Trump welcomed Chinese leader Xi Jinping 4 00:00:17,340 --> 00:00:22,750 As he arrived in the United States, for two days of closely watched negotiations. 5 00:00:25,990 --> 00:00:30,570 On their crowded agenda, what if any role they should play 6 00:00:30,720 --> 00:00:33,500 in regulating artificial intelligence. 7 00:00:33,690 --> 00:00:37,207 amid a global freakout over its risks. 8 00:00:37,290 --> 00:00:39,350 Thank you. Thank you. today. 9 00:00:39,960 --> 00:00:42,600 As my colleague David Sanger explains... 10 00:00:42,900 --> 00:00:47,740 When it comes to protecting humanity from technological catastrophe, 11 00:00:48,000 --> 00:00:53,580 there's really only one proven model for the two leaders to look to. 12 00:00:56,240 --> 00:00:59,820 It's Thursday, September 24th. 13 00:01:03,000 --> 00:01:03,880 David? 14 00:01:04,500 --> 00:01:05,560 Up from Washington. 15 00:01:06,510 --> 00:01:09,150 DC sitting here in our New York studios. 16 00:01:09,750 --> 00:01:12,510 And I didn't even have to come to you this time. 17 00:01:13,170 --> 00:01:17,816 I just had to come see the Dailies Empire in New York. In person? In person. 18 00:01:17,899 --> 00:01:22,690 Yeah. It's pretty remarkable. It's a humble empire. It's a late stage empire. 19 00:01:24,470 --> 00:01:29,947 So, David, when President Trump and the leader of China, Xi Jinping, meet China, 20 00:01:30,030 --> 00:01:35,031 In Washington today, they're going to broach this subject on many people's 21 00:01:35,115 --> 00:01:40,047 minds, which is the risk of artificial intelligence, that risk including, 22 00:01:40,130 --> 00:01:42,757 of course, these swarming rogue people. 23 00:01:42,840 --> 00:01:48,170 AI agents that have been hacking into companies and basically breaking the rules 24 00:01:48,254 --> 00:01:51,840 of their creators. And what she and Trump uniquely... 25 00:01:52,050 --> 00:01:55,270 can maybe do about it is going to be the subject of 26 00:01:55,410 --> 00:02:00,127 at least some of this meeting. And I think we've all come to understand that, 27 00:02:00,210 --> 00:02:02,010 The leaders of the U.S. and China... 28 00:02:02,100 --> 00:02:06,153 are uniquely positioned to talk about the safety of AI because they're the countries 29 00:02:06,237 --> 00:02:07,327 with the most advanced 30 00:02:07,410 --> 00:02:12,607 frontier AI laboratories. And the only way that AI gets contained is, 31 00:02:12,690 --> 00:02:17,944 is if both of those countries somehow act together and at the same time. 32 00:02:18,027 --> 00:02:22,370 And as this meeting, approached and this is why we love you so much 33 00:02:22,800 --> 00:02:25,340 You reached out to us to say, you know... 34 00:02:25,620 --> 00:02:32,107 I have seen this movie before. Big fears, big thoughts about coordination, anxiety. 35 00:02:32,190 --> 00:02:34,327 and solutions to big problems. 36 00:02:34,410 --> 00:02:37,030 You told us there's a historical parallel. 37 00:02:37,470 --> 00:02:41,040 to all of this and maybe a historical solution. 38 00:02:41,550 --> 00:02:45,416 Well, Michael, there is an obvious parallel. 39 00:02:45,499 --> 00:02:52,230 We had an era starting in the late 1940s and through the Cold War where the 40 00:02:52,440 --> 00:02:55,859 Two states, the Soviet Union and the United States, 41 00:02:55,942 --> 00:03:00,390 had created a potentially terrifying technology that had all kinds 42 00:03:00,474 --> 00:03:05,334 of commercial uses but could also bring about civilizational extinction. 43 00:03:05,418 --> 00:03:10,828 Nuclear weapons. Nuclear weapons. And everybody had to go figure out how we were 44 00:03:10,911 --> 00:03:15,771 going to get the benefits without getting the terror. And you know what? 45 00:03:15,855 --> 00:03:17,640 It took decades to happen. 46 00:03:22,440 --> 00:03:29,129 Came up with agreements that ultimately have brought us to the remarkable moment 47 00:03:29,213 --> 00:03:35,394 that there has been no use of a nuclear weapon in anger since 1945. Right. 48 00:03:35,477 --> 00:03:40,980 Nuclear nonproliferation worked. Nuclear nonproliferation worked. 49 00:03:41,070 --> 00:03:44,670 combined with a balance of terror worked. 50 00:03:44,850 --> 00:03:50,560 So now, quite naturally, everybody's looking to that model as we begin to think 51 00:03:50,643 --> 00:03:54,373 about how you might contain artificial intelligence, 52 00:03:54,457 --> 00:03:59,653 particularly after this summer of surprises that we've had where we have 53 00:03:59,737 --> 00:04:04,870 rogue agents breaking out of their sandboxes. Right. Treat AI the way. 54 00:04:05,250 --> 00:04:06,790 superpowers treated. 55 00:04:06,960 --> 00:04:12,191 nuclear weaponry. Well, look, if you think that the heads of some of these frontier 56 00:04:12,275 --> 00:04:17,058 companies are coming out and saying there was a 10% chance of civilizational 57 00:04:17,141 --> 00:04:21,924 extinction, right? That's probably a higher proportion chance than there was 58 00:04:22,008 --> 00:04:27,047 in the early days of the nuclear era, that we would wipe each other off the map, 59 00:04:27,130 --> 00:04:30,460 at least its equivalent. And so now the question is, 60 00:04:30,960 --> 00:04:35,166 Can two very different countries in very different times, 61 00:04:35,249 --> 00:04:41,185 China and the United States, come to some kind of framework agreement that could 62 00:04:41,268 --> 00:04:44,880 begin to get us there? Okay. Well, given all of… 63 00:04:45,120 --> 00:04:45,953 That. 64 00:04:46,830 --> 00:04:50,010 In your capacity, David, as a historian of history, 65 00:04:50,190 --> 00:04:56,380 all the world's great technological dangers, I think it would be instructive, 66 00:04:56,670 --> 00:04:58,810 to have you tell us the story 67 00:04:58,950 --> 00:05:03,363 of nuclear nonproliferation, how that old framework that kept nuclear weapons 68 00:05:03,447 --> 00:05:07,568 from going off around the world could, in theory, apply right now to AI. 69 00:05:07,651 --> 00:05:10,488 So just give us a quick history lesson here. Well, 70 00:05:10,571 --> 00:05:14,985 let's just start with when the U.S. developed its first atomic weapon through 71 00:05:15,068 --> 00:05:19,656 the Manhattan Project. Familiar to anybody who's been off to see the Oppenheimer 72 00:05:19,740 --> 00:05:23,010 movie or read some of this history. It was in the 1940s. 73 00:05:28,950 --> 00:05:33,316 another country would beat us to it. But it wasn't the Soviets at that time. 74 00:05:33,400 --> 00:05:36,888 It was Germany. In a last desperate attempt to escape defeat, 75 00:05:36,971 --> 00:05:40,693 German science threw everything it had into the perfection of V2. 76 00:05:40,777 --> 00:05:43,470 Where the fear of American intelligence was... 77 00:05:43,560 --> 00:05:47,282 the Germans would get a nuclear weapon before the United States. 78 00:05:47,365 --> 00:05:51,443 And there was this just abiding fear we had to get there first. Right. 79 00:05:51,527 --> 00:05:55,248 In the spreading, teeming laboratories of the Manhattan Project, 80 00:05:55,332 --> 00:05:56,580 the pace intensified. 81 00:05:56,670 --> 00:06:01,820 Groves had ordered that the first form must be ready for testing in mid-July. 82 00:06:01,903 --> 00:06:03,670 And in the summer of 1945, 83 00:06:04,140 --> 00:06:07,580 Scientists from Los Alamos went out to do the Trinity test, 84 00:06:07,663 --> 00:06:12,680 which was the first explosion of a nuclear bomb to see if the technology could work. 85 00:06:22,290 --> 00:06:25,057 would never see again. There were... 86 00:06:25,140 --> 00:06:29,852 People within the Los Alamos program who were doing mathematical calculations 87 00:06:29,936 --> 00:06:34,585 and coming to the conclusion that we could light the entire atmosphere afire 88 00:06:34,669 --> 00:06:37,160 and destroy the earth in doing the test. 89 00:06:39,820 --> 00:06:46,820 But despite those fears, they went ahead with the test. 90 00:06:51,100 --> 00:06:52,660 Just weeks later. 91 00:06:55,140 --> 00:06:59,440 American airplane dropped one bomb on Hiroshima. 92 00:06:59,760 --> 00:07:06,760 came Hiroshima. And days after that, 93 00:07:14,580 --> 00:07:19,596 Nagasaki. Four square miles of the city were blasted into dust and a black rain 94 00:07:19,679 --> 00:07:23,100 of destruction hovered over the ruins for many hours. 95 00:07:23,220 --> 00:07:27,870 And in total, Those two bombs killed around 200,000 people. 96 00:07:28,520 --> 00:07:31,768 Of course, the world had never seen anything like this before. 97 00:07:31,851 --> 00:07:34,860 Tens of thousands of homes and buildings were wiped out. 98 00:07:35,760 --> 00:07:40,469 the devastation was everywhere. Japan was driven to quick surrender. 99 00:07:40,552 --> 00:07:46,260 For the future, the choice is peace or total destruction. The atomic age is here. 100 00:07:54,840 --> 00:07:59,830 you But somewhat astonishingly, it didn't stop the arms race. 101 00:08:00,210 --> 00:08:04,677 Of course, it turned out to be just the beginning. And now, of course, 102 00:08:04,760 --> 00:08:10,007 the U.S. thought it had an insurmountable lead over countries that didn't yet have 103 00:08:10,090 --> 00:08:13,580 nuclear weapons. That lead didn't last very long. 104 00:08:13,770 --> 00:08:18,985 The Soviets were working away on this. They had a bunch of atomic spies along 105 00:08:19,069 --> 00:08:21,944 the way who helped them to it. And by 1949, 106 00:08:22,028 --> 00:08:25,950 they had conducted a test and we were off into the races. 107 00:08:26,370 --> 00:08:31,736 And the 1950s were a period of great terror where each side was not only 108 00:08:31,819 --> 00:08:37,336 building up its arsenal, but building up the powers of these weapons as if 109 00:08:37,420 --> 00:08:42,483 the weapons used at Hiroshima and Nagasaki were not powerful enough. 110 00:08:42,566 --> 00:08:47,856 And by this stage, proliferation was starting. Off northwest Australia, 111 00:08:47,939 --> 00:08:53,010 this was the place chosen to explode Britain's first atomic weapon. 112 00:08:56,370 --> 00:09:00,815 Britain and France had nuclear weapons. China wasn't that far behind. 113 00:09:00,898 --> 00:09:04,770 I've just been told that China has exploded an atomic bomb. 114 00:09:05,070 --> 00:09:07,147 Right, things are starting to spin. 115 00:09:07,230 --> 00:09:10,230 Kind of out of control. They are. And then we had... 116 00:09:10,410 --> 00:09:16,147 one big terrifying incident. The purpose of these bases can be none other 117 00:09:16,230 --> 00:09:20,448 than to provide a nuclear strike capability against the Western Hemisphere. 118 00:09:20,531 --> 00:09:22,687 And it was the Cuban Missile Crisis... 119 00:09:22,770 --> 00:09:27,562 in October of 1962. This was when U.S. intelligence came up with evidence 120 00:09:27,645 --> 00:09:31,235 that the Soviets were moving nuclear weapons onto Cuba, 121 00:09:31,318 --> 00:09:34,307 that they would be just 90 miles from Florida, 122 00:09:34,390 --> 00:09:38,114 that they could easily reach any place on the East Coast. 123 00:09:38,197 --> 00:09:43,256 The waterfront in Havana and along other parts of the coast bristled with gun 124 00:09:43,339 --> 00:09:48,866 emplacements as the Cuban regime waited to see what their bosses in the Kremlin were 125 00:09:48,949 --> 00:09:49,783 to do. 126 00:09:52,770 --> 00:09:55,724 This was intolerable. But the question came, 127 00:09:55,808 --> 00:09:59,927 is this worth going to war over? Rockets, bombs, ammunition. 128 00:10:00,010 --> 00:10:04,230 Air support for any limited war ground operation which might develop. 129 00:10:04,313 --> 00:10:07,930 Within 12 hours, all units were assembled and ready to go. 130 00:10:08,170 --> 00:10:09,430 and 131 00:10:09,520 --> 00:10:15,467 There were moments when Kennedy thought, and many around him thought, that... 132 00:10:15,550 --> 00:10:20,350 The United States was going to disappear, you know, in the next night or two. Wow. 133 00:10:26,170 --> 00:10:29,434 A final chapter in the Cuban crisis is brought to a close. 134 00:10:29,517 --> 00:10:32,207 As President Kennedy holds a news conference... 135 00:10:32,290 --> 00:10:35,147 But in the end, we know how this ended. 136 00:10:35,230 --> 00:10:38,470 I have today been informed by Chairman Khrushchev 137 00:10:38,920 --> 00:10:43,937 that all of the IL-28 bombers now in Cuba will be withdrawn in 30 days. 138 00:10:44,020 --> 00:10:48,520 The nuclear crisis was averted at the last moment. 139 00:10:48,610 --> 00:10:52,608 And Kennedy then went on to give a really famous speech. 140 00:10:52,691 --> 00:10:57,564 It was a commencement speech at American University in June of 1963. 141 00:10:57,647 --> 00:11:01,135 So it was just months before he was assassinated. 142 00:11:01,218 --> 00:11:03,550 I realized the pursuit of peace. 143 00:11:04,150 --> 00:11:07,640 is not as dramatic. as the pursuit of war 144 00:11:07,840 --> 00:11:13,629 And he basically said, look, the terror of this event has made us think we cannot 145 00:11:13,712 --> 00:11:14,800 live with this. 146 00:11:17,800 --> 00:11:19,520 heirs that we all inhabit, 147 00:11:19,660 --> 00:11:24,944 this small planet. And he said, our most basic common link is that we all inhabit 148 00:11:25,027 --> 00:11:28,540 this small planet. We all breathe the same air. 149 00:11:28,660 --> 00:11:31,060 We all cherish our children's futures. 150 00:11:31,210 --> 00:11:34,130 And we are all mortal. And we are all mortal. 151 00:11:34,930 --> 00:11:38,590 And that's the line that has resonated ever since. 152 00:11:39,490 --> 00:11:44,909 He announced at that moment that there would be negotiations for a first tiny 153 00:11:44,993 --> 00:11:49,810 step. First, Chairman Khrushchev, Prime Minister McMillan and I have agreed 154 00:11:50,380 --> 00:11:53,740 that high-level discussions will shortly begin in Moscow, 155 00:11:54,310 --> 00:11:59,690 looking towards early agreement on a comprehensive test ban treaty. 156 00:11:59,920 --> 00:12:03,077 He called it the Comprehensive Nuclear Test Ban Treaty. 157 00:12:03,160 --> 00:12:06,400 Our hopes must be tempered with the caution of history. 158 00:12:06,850 --> 00:12:08,830 But with our hopes go the hopes... 159 00:12:08,920 --> 00:12:10,240 of all mankind. 160 00:12:10,570 --> 00:12:15,782 That the U.S. at that moment, he said, would stop testing in the atmosphere above 161 00:12:15,865 --> 00:12:20,358 ground. You know, those you've seen the images of the mushroom clouds, 162 00:12:20,441 --> 00:12:25,196 the mushroom clouds. Right. And that over time we had to move to reduction 163 00:12:25,279 --> 00:12:28,090 of nuclear weapons. But let's start simple. 164 00:12:28,450 --> 00:12:33,884 with testing, with freezes, and with the first discussion of how you'd keep 165 00:12:33,967 --> 00:12:37,709 this technology from spreading to many other states. 166 00:12:37,792 --> 00:12:43,373 Because Kennedy himself had said that he thought there could be 15 or even 20 167 00:12:43,457 --> 00:12:49,590 nuclear-armed nations coming. by the mid 70s. Wow. 15 or 20. That's right. 168 00:12:49,870 --> 00:12:54,589 And you began to see a real cultural awakening in the United States about 169 00:12:54,673 --> 00:12:59,129 the depth of this danger. Here's Tony going to his Cub Scout meeting. 170 00:12:59,212 --> 00:13:03,537 Tony knows the bomb can explode any time of the year, day or night. 171 00:13:03,620 --> 00:13:04,870 He is ready for it. 172 00:13:04,960 --> 00:13:10,004 So, of course, this is the era when the government is encouraging schools to do 173 00:13:10,087 --> 00:13:14,500 those ridiculous duck and cover exercises. Duck and cover exercises. 174 00:13:14,740 --> 00:13:18,060 Attaboy, Tony. That flash means act fast. 175 00:13:18,670 --> 00:13:21,615 Of course, everybody knew that none of this would help. 176 00:13:21,698 --> 00:13:24,038 Older people will help us as they always do, 177 00:13:24,121 --> 00:13:27,810 but there might not be any grown-ups around when the bomb explodes. 178 00:13:28,570 --> 00:13:30,380 Then... You're on your own. 179 00:13:31,030 --> 00:13:36,815 And, you know, this was when Tom Lehrer, the famous songwriter and satirist, 180 00:13:36,899 --> 00:13:41,757 had, you know, his song. So long, mom. I'm off to drop the bomb. 181 00:13:41,840 --> 00:13:46,467 So don't wait up for me. And who's next? France got the bomb, 182 00:13:46,551 --> 00:13:50,251 but don't you grieve because they're on our side. 183 00:13:50,334 --> 00:13:53,957 I believe China got the bomb, but have no fears. 184 00:13:54,041 --> 00:13:57,670 They can't wipe us out for at least five years. 185 00:14:01,030 --> 00:14:06,914 countries are going to seek their next weapons. Who's next? Who's next? 186 00:14:06,997 --> 00:14:08,930 Who's next? Who's next? 187 00:14:12,670 --> 00:14:14,770 And I think this created... 188 00:14:14,890 --> 00:14:20,165 and incredible political pressure on the Johnson administration to get this thing 189 00:14:20,248 --> 00:14:22,630 done. And in 1968, it came together. 190 00:14:24,480 --> 00:14:28,351 After endless diplomatic meetings back and forth, 191 00:14:28,435 --> 00:14:34,808 occupying lots of days at the Waldorf Astoria Hotel in New York and other places 192 00:14:34,891 --> 00:14:41,426 around the world before they came up with what became the Nuclear Nonproliferation 193 00:14:41,509 --> 00:14:45,623 Treaty. "Distinguished guests, ladies and gentlemen, 194 00:14:45,706 --> 00:14:51,840 this is a very reassuring and hopeful moment in the relations among nations. 195 00:14:52,030 --> 00:14:58,277 And on July 1st, 1968, Five years after that Kennedy speech, 196 00:14:58,360 --> 00:15:01,641 The treaty was signed by President Johnson at the White House. 197 00:15:01,724 --> 00:15:04,980 We have come here today to the East Room of the White House. 198 00:15:05,410 --> 00:15:11,493 to sign a treaty which limits the spread of nuclear weapons. 199 00:15:11,576 --> 00:15:15,912 And then it was signed in the Soviet Union, 200 00:15:15,995 --> 00:15:22,995 and then by about 60 countries in total. Man can still shape his destiny in 201 00:15:23,703 --> 00:15:28,430 the nuclear age and learn to live as brothers. 202 00:15:30,130 --> 00:15:32,981 And here's what the treaty says. First of all, 203 00:15:33,064 --> 00:15:37,573 it says we understand there are five countries that already have nuclear 204 00:15:37,656 --> 00:15:42,038 weapons and they all commit to eliminating their stockpiles over time. 205 00:15:42,121 --> 00:15:44,290 It doesn't specify when. And then. 206 00:15:44,440 --> 00:15:48,527 Countries that do not have nuclear weapons will commit that they will never die. 207 00:15:48,610 --> 00:15:52,210 build them. But in return, they get 208 00:15:52,540 --> 00:15:57,561 help for peaceful nuclear technology to build nuclear power plants and so forth. 209 00:15:57,644 --> 00:16:02,155 And then, of course, there needs to be an inspection regime to make sure 210 00:16:02,238 --> 00:16:07,259 that they're not cheating. And that was the creation of the International Atomic 211 00:16:07,342 --> 00:16:11,151 Energy Agency, which is based in Vienna. It's part of the UN. 212 00:16:11,234 --> 00:16:16,255 And they begin the process of developing teams that go visit countries that have 213 00:16:16,338 --> 00:16:17,172 signed this 214 00:16:22,540 --> 00:16:24,660 being diverted off to bomb projects. 215 00:16:26,100 --> 00:16:31,578 And As we look back at this, this was even a bigger moment than we knew at the time. 216 00:16:31,662 --> 00:16:33,810 What do you mean? Because... 217 00:16:34,000 --> 00:16:39,749 Nations that seem to be inevitably racing to adopt a fairly non-complicated 218 00:16:39,832 --> 00:16:44,571 technology to build nuclear weapons all agreed to halt. Right. 219 00:16:44,654 --> 00:16:47,920 There's very few moments in history where. 220 00:16:48,220 --> 00:16:49,053 Everybody. 221 00:16:49,840 --> 00:16:54,377 who wants to catch up with the leaders in an arms race, voluntarily... 222 00:16:54,460 --> 00:16:57,848 abandon the pursuit. That's right. Almost everybody. 223 00:16:57,931 --> 00:17:01,520 There are a couple of outliers and Israel never signed. 224 00:17:01,603 --> 00:17:04,657 And while it's never admitted to having a bomb, 225 00:17:04,741 --> 00:17:08,796 it's not exactly the biggest secret in the Middle East. Right. 226 00:17:08,880 --> 00:17:12,334 North Korea signed but renounced it in the mid 1990s. 227 00:17:12,418 --> 00:17:15,672 And they now have an arsenal of 60 to 100 weapons, 228 00:17:15,756 --> 00:17:18,960 depending on who you believe. India and Pakistan 229 00:17:24,460 --> 00:17:28,300 worry about Iran, but they don't actually have a nuclear weapon yet. 230 00:17:28,383 --> 00:17:30,460 But ultimately, this treaty prevents 231 00:17:30,580 --> 00:17:37,166 the kind of spreading of nuclear weaponry that Kennedy so worried about. It did. 232 00:17:37,249 --> 00:17:40,500 And it's remarkable. Kennedy was wrong. 233 00:17:40,630 --> 00:17:47,270 We've only got nine nuclear states, and many of these countries have had nuclear 234 00:17:47,353 --> 00:17:53,152 weapons for decades. And so far, it has remained relatively contained, 235 00:17:53,236 --> 00:17:59,707 which is why this does seem like perhaps the best model we have to think about 236 00:17:59,790 --> 00:18:05,086 a comparable international regulatory system for restraining AI. 237 00:18:05,169 --> 00:18:07,690 That's exactly right, Michael. 238 00:18:07,900 --> 00:18:14,900 And as everybody enters this early stage debate about whether or not you can do 239 00:18:17,270 --> 00:18:24,270 the same with artificial intelligence, here is an example, a rare example, 240 00:18:26,047 --> 00:18:33,047 in which nations have voluntarily agreed to slow a technology, 241 00:18:37,900 --> 00:18:41,625 the leaders of the frontier companies. They're not saying stop it. 242 00:18:41,709 --> 00:18:45,376 They're not saying let's go out of business. They're saying, hey, 243 00:18:45,460 --> 00:18:46,960 let's just slow this down. 244 00:18:47,110 --> 00:18:52,326 so that we can understand how to control it before we find ourselves in another 245 00:18:52,409 --> 00:18:57,491 crisis. Right. In some ways, a more modest request than what nonproliferation 246 00:18:57,574 --> 00:19:01,649 was about. You're right, Michael. It is a more modest request, 247 00:19:01,733 --> 00:19:04,550 but it's a far more complicated situation. 248 00:19:10,510 --> 00:19:11,890 We'll be right back. 249 00:19:17,770 --> 00:19:22,367 So David, understanding as we now do, that nuclear nonproliferation 250 00:19:22,450 --> 00:19:27,010 as a model is our best and so far really our only goal. 251 00:19:27,160 --> 00:19:31,277 model for AI. In what ways... 252 00:19:31,360 --> 00:19:35,477 is it quite challenging to apply that model to AI right now? 253 00:19:35,560 --> 00:19:38,977 You just said it's very different and complicated. 254 00:19:39,060 --> 00:19:43,947 So there was something of a remarkable simplicity to the arms race with 255 00:19:44,030 --> 00:19:47,237 the Soviet Union, even while it was terrifying. 256 00:19:47,320 --> 00:19:52,640 What was simple about it? Well, two countries had almost all the technology, 257 00:19:52,990 --> 00:19:57,495 We were only worried about states. We were not worried about non-state actors, 258 00:19:57,578 --> 00:20:01,640 terrorist groups, people working in their basements and 259 00:20:01,970 --> 00:20:06,094 If you didn't have plutonium or uranium, you couldn't make a bomb. 260 00:20:06,178 --> 00:20:10,130 So we knew that there was a way to sort of cut off the oxygen. 261 00:20:10,520 --> 00:20:16,482 There was a limited number of people with launch authority in each country, 262 00:20:16,565 --> 00:20:23,011 and both countries had their own political dynamic for wanting to make sure there 263 00:20:23,095 --> 00:20:29,140 was no accident and there was no escalation if they could avoid it. Mm-hmm. 264 00:20:29,330 --> 00:20:32,982 That didn't mean they were willing to give up the weapons, 265 00:20:33,065 --> 00:20:36,327 but it did mean that you could contain the problem. 266 00:20:36,410 --> 00:20:41,184 But when you think about AI, the situation is completely different. 267 00:20:41,268 --> 00:20:45,110 The governments don't really control this technology. 268 00:20:46,130 --> 00:20:49,408 You don't need the equivalent of uranium and plutonium. 269 00:20:49,491 --> 00:20:53,441 You just need compute a lot of it. And maybe that's the oxygen for 270 00:20:53,524 --> 00:20:58,230 this particular case. Computer power. You need computer power and tons of it. 271 00:20:58,370 --> 00:21:04,506 There's no reason to believe that as this technology becomes more efficient, 272 00:21:04,590 --> 00:21:10,399 that smaller and smaller groups working out of basements, terror groups, 273 00:21:10,482 --> 00:21:16,864 countries that have interest in causing disruption can't get a hold of basic AI 274 00:21:16,947 --> 00:21:22,430 technology. And we've already seen some examples of that happening. 275 00:21:22,790 --> 00:21:26,561 There's one other big difference, Michael. In the Cold War, 276 00:21:26,644 --> 00:21:31,395 we didn't have to convince our national leaders that this was a terrifying 277 00:21:31,479 --> 00:21:36,770 technology. They had to, if anything, to convince their populations of that. But. 278 00:21:36,860 --> 00:21:42,803 In this case, we have an American president who is declaring that all these 279 00:21:42,886 --> 00:21:46,180 warnings are a hoax. And Xi Jinping is... 280 00:21:46,580 --> 00:21:50,745 is clearly more concerned, as some of his aides have made clear, 281 00:21:50,828 --> 00:21:53,997 about the threat to the Communist Party of China, 282 00:21:54,081 --> 00:21:56,918 rather than its threat to the broader world. 283 00:21:57,001 --> 00:22:00,320 So you're saying the leaders of the superpowers... 284 00:22:00,470 --> 00:22:03,218 locked in this arms race, the AI arms race, 285 00:22:03,301 --> 00:22:07,959 can't even agree on what the threat is. Whereas back in the nuclear era, 286 00:22:08,042 --> 00:22:13,029 the threat was pretty self-evident and the leaders could agree on it. It was, 287 00:22:13,112 --> 00:22:16,470 this is going to kill all of us. That's right. Now, 288 00:22:16,910 --> 00:22:22,427 They also recognized that they were not going to give up all nuclear weapons back 289 00:22:22,511 --> 00:22:26,991 then because they believed they needed that for national survival. 290 00:22:27,074 --> 00:22:31,624 In this case, the U.S. and China both think they need to be pushing 291 00:22:31,707 --> 00:22:35,081 this technology ahead for their economic survival. 292 00:22:35,164 --> 00:22:38,884 But it's also a big part of the struggle for dominance. 293 00:22:38,967 --> 00:22:42,747 China's made no secret of the fact that by 2049, China, 294 00:22:42,830 --> 00:22:47,050 the 100th anniversary of the creation of the People's Republic of China. 295 00:22:47,133 --> 00:22:51,533 It wants to be the biggest military power, the biggest technological power, 296 00:22:51,616 --> 00:22:53,230 the biggest economic power. 297 00:22:54,530 --> 00:23:00,297 And you don't get to that point without AI. Right. And so if you're China... 298 00:23:00,380 --> 00:23:06,233 and you're suddenly faced with the prospect of a nuclear non-proliferation 299 00:23:06,316 --> 00:23:10,564 style agreement with your biggest adversary, the U.S., 300 00:23:10,648 --> 00:23:15,300 when it comes to AI, you're probably thinking to yourself, 301 00:23:16,490 --> 00:23:20,150 If we sign that, then we might be signing away our economic future. 302 00:23:21,680 --> 00:23:23,640 Adding to that suspicion... 303 00:23:24,140 --> 00:23:28,758 would probably be the reality, and we've covered it a lot on the show, 304 00:23:28,841 --> 00:23:33,140 that China already does a much more robust job of regulating AI, 305 00:23:33,770 --> 00:23:38,052 than the United States does. The U.S. has taken a really hands-off approach, 306 00:23:38,135 --> 00:23:41,153 and AI here is even more advanced than it is in China. 307 00:23:41,236 --> 00:23:44,510 And so if you're Xi Jinping, you're thinking to yourself, 308 00:23:44,750 --> 00:23:46,550 Why on earth would I sign a deal? 309 00:23:47,090 --> 00:23:48,990 that locks in U.S. 310 00:23:49,610 --> 00:23:55,539 AI advantages, It makes it harder for us to catch up and realize our AI future. 311 00:23:55,623 --> 00:23:57,710 It's exactly right, because... 312 00:23:58,040 --> 00:24:03,507 The Chinese are convinced the U.S. is already trying to do that with export 313 00:24:03,590 --> 00:24:06,920 controls on the world's fastest chips. Right. 314 00:24:07,400 --> 00:24:13,765 President Trump has agreed to give the Chinese the world's second fastest chips, 315 00:24:13,848 --> 00:24:20,213 but not the fastest. And the Chinese view of this is this is modern containment. 316 00:24:20,296 --> 00:24:23,517 This is their effort to preserve a lead. 317 00:24:23,600 --> 00:24:27,491 And then if you're other countries, you're probably thinking, 318 00:24:27,574 --> 00:24:32,247 I don't want to sign anything that's going to cut off my economic future. 319 00:24:32,331 --> 00:24:37,069 And if everybody in the world is running around saying the economic future 320 00:24:37,152 --> 00:24:40,261 is completely dependent on the development of AI, 321 00:24:40,344 --> 00:24:45,147 why would I sign a treaty that would keep me from being a full participant? 322 00:24:45,231 --> 00:24:49,137 So we're clearly running up against some of the real issues. 323 00:24:49,220 --> 00:24:50,053 limits. 324 00:24:50,390 --> 00:24:57,388 of the nuclear model for AI. So let me return to the slightly more optimistic 325 00:24:57,471 --> 00:25:01,150 thought that it is our only model. So... 326 00:25:01,250 --> 00:25:02,410 Does that mean... 327 00:25:02,540 --> 00:25:06,447 that President Xi and President Trump are in any way inclined to, 328 00:25:06,530 --> 00:25:11,805 to pursue it? Well, in some ways, they already have. What do you mean? Well, 329 00:25:11,889 --> 00:25:17,164 back during the Biden administration, Biden's advisers wanted to see if they 330 00:25:17,247 --> 00:25:22,452 could get the Chinese engaged in the most basic conversation on this topic. 331 00:25:22,535 --> 00:25:28,445 And it took about a year and a half to get the Chinese to agree to basically a single 332 00:25:28,529 --> 00:25:31,490 sentence, which is which is basically that 333 00:25:31,820 --> 00:25:37,928 Neither country would let artificial intelligence agents control their nuclear 334 00:25:38,011 --> 00:25:39,440 weapons stockpile. 335 00:25:39,800 --> 00:25:44,805 A seemingly very wise pledge. You wouldn't think it would take a year and a half 336 00:25:44,888 --> 00:25:49,287 to get to something that basic. But you're saying the U.S. and China – 337 00:25:49,370 --> 00:25:51,507 have come to an understanding... 338 00:25:51,590 --> 00:25:55,110 that feels like an extension of 339 00:25:55,370 --> 00:25:59,798 the nuclear containment policies that the U S and Russia and everybody else have 340 00:25:59,882 --> 00:26:04,760 agreed to. We're going to keep, AI and nuclear weapons from intermingling. 341 00:26:05,330 --> 00:26:10,532 That's the concept, Michael. Now, no one really knows what this sentence means 342 00:26:10,616 --> 00:26:11,307 because 343 00:26:11,390 --> 00:26:16,315 Supposing it simply means you're not going to let AI agents decide whether or not 344 00:26:16,398 --> 00:26:19,778 to launch nuclear weapons. OK, we can all agree on that. 345 00:26:19,861 --> 00:26:24,168 But how do you know whether the information that President Trump or one 346 00:26:24,251 --> 00:26:28,372 of his successors is receiving or President Xi and one of one of his 347 00:26:28,455 --> 00:26:33,566 successors receives about the threat that might lead to a decision to employ nuclear 348 00:26:33,649 --> 00:26:37,050 weapons? It wasn't influenced by AI. Yeah, faked by AI. 349 00:26:37,220 --> 00:26:39,100 influenced by AI, 350 00:26:39,290 --> 00:26:44,066 the result of a quick AI search by an aide who's sitting outside the room. 351 00:26:44,150 --> 00:26:48,072 So we don't really know what this means, but it was progress. 352 00:26:48,155 --> 00:26:53,063 And you could imagine a situation in which President Trump and President Xi, 353 00:26:53,146 --> 00:26:57,200 if they were willing, could build on this and not just nuclear, 354 00:26:57,284 --> 00:27:02,060 that we don't want AI anywhere near the development of biological weapons. 355 00:27:02,143 --> 00:27:06,132 Right, because AI has now reached the level of sophistication, 356 00:27:06,215 --> 00:27:09,117 we have come to understand, where it could... 357 00:27:09,200 --> 00:27:16,200 help actually design biological weapons. And we don't want AI to be used 358 00:27:16,990 --> 00:27:18,180 in hacking. 359 00:27:18,350 --> 00:27:25,008 Because we have come to understand that AI can find and exploit vulnerabilities 360 00:27:25,091 --> 00:27:29,360 in the electric power grid or in the water systems, 361 00:27:29,443 --> 00:27:33,687 that it might take human hackers months or months. 362 00:27:33,770 --> 00:27:38,291 or years to find and then design an attack. In other words, 363 00:27:38,374 --> 00:27:43,988 that it speeds up cyber attacks and is pushing us into a new era of cyber 364 00:27:44,072 --> 00:27:44,930 insecurity. 365 00:27:45,680 --> 00:27:50,340 So, David, let's just talk logistics and brass tacks. Is there a world in which... 366 00:27:50,510 --> 00:27:53,277 the leader of the United States and the leader of China. 367 00:27:53,360 --> 00:27:58,811 sit down with their teams and begin to hash out the kind of nuclear 368 00:27:58,895 --> 00:28:02,860 nonproliferation scheme that we're really doing. 369 00:28:02,960 --> 00:28:09,297 talking about here for the world of AI, in which they begin to pledge... 370 00:28:09,380 --> 00:28:14,386 Things like, "We both agree," that AI shouldn't touch biological weapons, 371 00:28:14,469 --> 00:28:18,609 shouldn't be involved in the hacking of private corporations or governments. 372 00:28:18,693 --> 00:28:24,440 And this all eventually finds its way into something that resembles a treaty that 373 00:28:24,980 --> 00:28:29,160 And brings us. To a point. where 374 00:28:29,900 --> 00:28:32,997 What looks like the nuclear non-perforation deal exists in AI. 375 00:28:33,080 --> 00:28:37,621 Well, this week, as President Xi and President Trump meet, no, 376 00:28:37,705 --> 00:28:41,425 I don't think there's even a remote chance of that. 377 00:28:41,509 --> 00:28:47,467 I think we would be lucky to get a second sentence with a joint experts committee 378 00:28:47,551 --> 00:28:51,048 that will meet sometime. But over the long term, 379 00:28:51,131 --> 00:28:54,027 I could imagine a broader conversation. 380 00:28:54,110 --> 00:29:00,435 understanding, not really a treaty, in which you could set up some mechanisms 381 00:29:00,518 --> 00:29:01,850 that might work. 382 00:29:02,540 --> 00:29:03,373 Early warning. 383 00:29:03,740 --> 00:29:09,405 sharing of models back and forth for a few days or weeks before they are publicly 384 00:29:09,488 --> 00:29:14,514 released. Although I think American companies would be reluctant to give 385 00:29:14,598 --> 00:29:17,720 the Chinese an early peak and certainly not. 386 00:29:17,840 --> 00:29:22,030 The inspection part of the nuclear nonproliferation treaty, 387 00:29:22,113 --> 00:29:27,824 you could imagine, you already saw a hint of that in an essay from Dario Amadei, 388 00:29:27,908 --> 00:29:31,518 the head of Anthropic, about a week and a half ago, 389 00:29:31,602 --> 00:29:36,371 where he described a commitment by Anthropic to allow in some group 390 00:29:36,454 --> 00:29:42,238 of independent inspectors to sit among the developers and watch over the creation 391 00:29:42,321 --> 00:29:46,160 of new and more powerful models and blow the whistle. 392 00:29:47,840 --> 00:29:53,489 that they believed could pose a threat to humanity. And inside OpenAI, 393 00:29:53,572 --> 00:29:59,917 they called this the IAEA model, the International Atomic Energy Agency model. 394 00:30:00,000 --> 00:30:04,336 That there are these inspectors who are independent enough that they can call up 395 00:30:04,419 --> 00:30:08,699 people like you and me and say, hey, there's a big problem in Anthropic and let 396 00:30:08,782 --> 00:30:11,019 me come over and explain it to you. Right. 397 00:30:11,102 --> 00:30:14,554 They're referring literally to the inspectors who exist to check 398 00:30:14,637 --> 00:30:18,310 on countries' nuclear programs. I mean, we keep coming back to this, 399 00:30:18,393 --> 00:30:21,100 but everywhere you turn, everybody ultimately is. 400 00:30:21,510 --> 00:30:27,201 returns to the nuclear nonproliferation model. That's true. Now, 401 00:30:27,284 --> 00:30:33,847 do I think this can be done by treaty? No. Treaties take forever forever. 402 00:30:33,930 --> 00:30:38,314 to negotiate. And this technology is racing like a freight train. 403 00:30:38,397 --> 00:30:43,400 The fundamental nuclear technology hasn't changed all that much from 1945. 404 00:30:43,483 --> 00:30:47,950 So another big difference. here is just simply speed. That's right. 405 00:30:48,270 --> 00:30:50,640 Hmm. Well... when I'm, 406 00:30:50,730 --> 00:30:54,674 Sitting here listening to you, I'm also doing a little bit of a mental calculation 407 00:30:54,757 --> 00:30:57,960 of the time it took for the Non-Proliferation Treaty 408 00:30:58,230 --> 00:31:04,264 to come into existence after the first nuclear weapon was dropped on Japan. 409 00:31:04,347 --> 00:31:10,440 And I think it was basically 20 something years. And, That's a long time. 410 00:31:10,530 --> 00:31:14,230 But it does suggest that administrations change, 411 00:31:14,340 --> 00:31:19,480 and viewpoints change, and these things are painstaking and they're slow, 412 00:31:19,564 --> 00:31:24,030 but ultimately... Everyone did agree to sign that treaty. 413 00:31:24,240 --> 00:31:27,930 So is it foolhardy to think that over time 414 00:31:28,590 --> 00:31:31,828 people will come around to the same kind of conclusion with AI, 415 00:31:31,912 --> 00:31:36,130 and that we could be looking back in 20 years, you'll be really gray, I'll be... 416 00:31:36,450 --> 00:31:39,561 just a little less gray. And we'll say, wow, can you believe it? 417 00:31:39,644 --> 00:31:41,994 They took their time, but they got there on AI. 418 00:31:42,720 --> 00:31:46,310 Your question ignores the issue of time. Look, 419 00:31:46,393 --> 00:31:51,660 we don't exactly understand the threats we're dealing with with AI, 420 00:31:51,743 --> 00:31:56,771 but there are some engineers who are saying we have just months, 421 00:31:56,854 --> 00:32:01,802 if a year away from recursive self-improvement, that is to say, 422 00:32:01,885 --> 00:32:04,520 code that keeps improving itself. 423 00:32:04,710 --> 00:32:09,463 or artificial general intelligence within months or a year or two, 424 00:32:09,547 --> 00:32:14,750 we don't have 25 years this time. We may not have two and a half years. 425 00:32:15,150 --> 00:32:18,827 This time, which then raises the next question. 426 00:32:18,910 --> 00:32:25,547 Is it going to take a Cuba missile crisis like event in AI to rally the world around 427 00:32:25,630 --> 00:32:29,710 this? And thank you. I think we'd rather miss that. 428 00:32:30,600 --> 00:32:36,287 And remember, in the atomic age, we started with Hiroshima and Nagasaki. 429 00:32:36,371 --> 00:32:41,500 So there was no debate about the power of these weapons to kill. 430 00:32:41,910 --> 00:32:48,729 But this takes us to the biggest difference between AI and nuclear. 431 00:32:48,813 --> 00:32:50,770 In the nuclear era, 432 00:32:51,330 --> 00:32:56,330 We never really doubted that we all controlled the arsenal. 433 00:32:56,970 --> 00:33:02,400 You know, at one point, Winston Churchill in 1955 – said, 434 00:33:02,790 --> 00:33:09,540 The atomic bomb, with all its terrors, did not carry us outside the scope of human 435 00:33:09,623 --> 00:33:10,457 control. 436 00:33:10,980 --> 00:33:12,260 By which he meant... 437 00:33:12,870 --> 00:33:18,250 Human beings were making the decisions and containing the size of the – 438 00:33:18,420 --> 00:33:19,880 weapon and the reaction. 439 00:33:20,670 --> 00:33:22,070 In AI, 440 00:33:23,070 --> 00:33:29,631 We have no assurance of human control. That once we've discovered that agents 441 00:33:29,714 --> 00:33:35,585 were hacking into Hugging Face and into other corporate sites without 442 00:33:35,669 --> 00:33:38,377 the knowledge of their creators, 443 00:33:38,460 --> 00:33:44,869 we have begun to question the sort of Frankenstein's monster nature of 444 00:33:44,952 --> 00:33:46,900 this whole thing. And 445 00:33:47,130 --> 00:33:51,450 If that's the case, then we have ignored the Churchill warning. 446 00:33:52,170 --> 00:33:57,301 that we are now on the verge of being outside of human control. 447 00:33:57,384 --> 00:34:02,350 And that wasn't really much of an issue during the pandemic. 448 00:34:02,580 --> 00:34:08,224 nuclear era, but it may be the most urgent problem in the age of artificial 449 00:34:08,307 --> 00:34:09,300 intelligence. 450 00:34:11,480 --> 00:34:14,702 Well, David, thank you, I guess. Well, Michael, 451 00:34:14,786 --> 00:34:19,920 we live in fascinating times. Yeah. Appreciate your time. Thanks. Thanks. 452 00:34:20,250 --> 00:34:22,310 you 453 00:34:23,610 --> 00:34:25,670 you 454 00:34:26,280 --> 00:34:30,240 *music* We'll be right back. 455 00:34:30,930 --> 00:34:32,560 you you 456 00:34:38,310 --> 00:34:42,690 Here's what else you need to know today. On Thursday morning, 457 00:34:42,773 --> 00:34:47,300 a federal judge found that President Trump's order barring CNN, 458 00:34:47,383 --> 00:34:53,080 MSNOW and Politico from the White House was likely unconstitutional and ordered 459 00:34:53,163 --> 00:34:56,090 that the president restore their access. 460 00:34:56,460 --> 00:35:02,137 The judge, a Trump appointee, rejected the president's claim that revoking the media 461 00:35:02,220 --> 00:35:07,020 outlet's press credentials was necessary to protect national security. 462 00:35:07,860 --> 00:35:11,770 Trump is expected to appeal the ruling. And: 463 00:35:11,853 --> 00:35:18,666 Our nation has developed the power of the government and the consequences of 464 00:35:18,750 --> 00:35:23,381 the sanctions. In a defiant wartime speech at the United Nations, 465 00:35:23,465 --> 00:35:28,518 Iran's president said that his country would not be bullied into submission 466 00:35:28,602 --> 00:35:30,040 by the United States. 467 00:35:34,360 --> 00:35:40,379 President Masood Pazeshkin vowed that Iran would defend itself until, quote, 468 00:35:40,463 --> 00:35:45,120 its last breath and would not give up its nuclear program. 469 00:35:51,280 --> 00:35:55,954 Today's episode was produced by Shannon Lynn and Jessica Chung, 470 00:35:56,037 --> 00:36:01,013 with help from Michael Simon Johnson. It was edited by Maria Byrne, 471 00:36:01,097 --> 00:36:05,640 fact-checked by Susan Lee. Contains music by Will Reed. 472 00:36:05,730 --> 00:36:10,647 Dan Powell and Pat McCusker, and was engineered by Chris Wood. 473 00:36:10,730 --> 00:36:13,230 Our theme music is by Wonderly. 474 00:36:13,470 --> 00:36:15,970 Special thanks to Daniel Ramirez. 475 00:36:22,890 --> 00:36:26,730 That's it for The Daily. I'm Michael Barbaro. 476 00:36:26,820 --> 00:36:28,080 See you tomorrow.