Public transcript

What Safeguarding Nukes Can Teach Us About Safeguarding A.I.

By: The Daily
Length: 37 min
Published: 24 September 2026
Language: English
Text from: the audio
Transcribed: 27 September 2026

Read the full transcript Open on Apple Podcasts

Notes in English.

Summary

As President Trump and Xi Jinping meet amid global anxiety about AI risks, David Sanger walks through the history of nuclear nonproliferation as the only proven model for containing a catastrophic technology. He explains how the Nuclear Nonproliferation Treaty of 1968 kept nuclear weapons to nine states, and why applying that model to AI is far harder: governments don't control the technology, leaders don't agree on the threat, and there may be only months or a couple of years before recursive self-improvement arrives. Still, small steps like the U.S.-China pledge to keep AI away from nuclear weapons decisions offer a starting point.

Key points

The U.S. and China are uniquely positioned to regulate AI because they host the most advanced frontier AI laboratories, and containment only works if both act together.
The Nuclear Nonproliferation Treaty, signed in 1968, helped keep the world at nine nuclear states, far fewer than Kennedy's feared 15 to 20 by the mid-1970s.
The nuclear era was simpler: only states had the technology, uranium and plutonium were controllable chokepoints, and leaders agreed the threat was existential.
With AI, governments don't control the technology; the key input is compute, which will become accessible to smaller groups, terrorists, and rogue states.
Trump calls AI warnings a hoax, and Xi Jinping is more worried about threats to the Communist Party than global risk, so the superpowers can't even agree on what the threat is.
China already regulates AI more robustly than the U.S., so Xi sees a treaty as locking in American advantages, especially given U.S. export controls on the fastest chips.
The U.S. and China have already agreed on one sentence: neither will let AI agents control their nuclear weapons stockpiles.
Sanger argues there is no time for a treaty: some engineers say recursive self-improvement or AGI could be months to two years away, and unlike nuclear weapons, AI offers no assurance of human control.
Dario Amodei of Anthropic proposed an IAEA-style model of independent inspectors watching over the development of powerful new models.

Questions it answers

00:00Trump-Xi meeting and AI risk

Why are Trump and Xi uniquely positioned to address AI safety?

The U.S. and China have the most advanced frontier AI laboratories, and AI can only be contained if both countries act together and at the same time. Their meeting includes discussion of AI risks like rogue swarming agents hacking companies.

AI containment requires simultaneous U.S.-China action.
Rogue AI agents have been breaking out of their sandboxes this summer.

02:52The nuclear parallel

What historical model exists for containing a catastrophic technology?

The Cold War nuclear era, when the U.S. and Soviet Union created a terrifying technology and eventually negotiated agreements that have prevented any use of a nuclear weapon in anger since 1945. Nuclear nonproliferation combined with a balance of terror worked.

Nuclear nonproliferation is the only proven model for technological containment.
Some frontier company heads cite a 10% chance of civilizational extinction from AI.

05:00History of the nuclear arms race

How did the nuclear threat unfold from 1945 to 1962?

The Manhattan Project, driven by fear of Germany getting the bomb first, led to the Trinity test and the bombings of Hiroshima and Nagasaki, which killed around 200,000 people. The Soviets tested by 1949, proliferation spread to Britain, France, and China, and the Cuban Missile Crisis of October 1962 brought the world to the brink of war.

Los Alamos scientists feared the Trinity test could ignite the atmosphere.
The Cuban Missile Crisis was the terrifying incident that spurred action.

11:17Kennedy's speech and the treaty

How did the world move from crisis to the Nuclear Nonproliferation Treaty?

After the Cuban crisis, Kennedy's June 1963 American University speech declared humanity's common bonds and launched negotiations toward a test ban. Political pressure, cultural anxiety, and years of diplomacy produced the Nuclear Nonproliferation Treaty, signed July 1, 1968 by Johnson and about 60 countries.

Kennedy feared 15 to 20 nuclear states by the mid-70s.
The treaty traded non-possession for peaceful nuclear technology, enforced by IAEA inspections.
Only nine nuclear states exist today, so the treaty largely worked.

19:17Why AI is harder to contain

What makes applying the nuclear model to AI so difficult?

Nuclear was containable because states held the technology, uranium and plutonium were chokepoints, and launch authority was limited. AI is controlled by companies, runs on compute that will spread widely, and the U.S. and Chinese leaders don't even agree on the threat, with Trump calling warnings a hoax and Xi focused on the Communist Party's survival.

Compute is AI's oxygen, and it will reach basements and terror groups.
Both countries see AI as essential to economic survival and dominance.
China views U.S. chip export controls as modern containment and won't sign away its AI future.

25:01What an AI agreement could look like

Is there any real prospect of a Trump-Xi AI deal?

Not a treaty this week; Sanger says we'd be lucky to get a second sentence and a joint experts committee. But the two countries have already agreed that AI agents won't control nuclear weapons stockpiles, and a longer-term understanding could cover biological weapons, hacking, early warning sharing, and IAEA-style inspectors as proposed by Anthropic's Dario Amodei.

The Biden administration took 18 months to get one sentence agreed with China.
OpenAI internally calls the inspector concept the IAEA model.
Treaties take too long for technology racing like a freight train.

30:00Time and human control

Why might the world not have 20 years to get an AI treaty?

Some engineers say recursive self-improvement or AGI could arrive within months to two years, and unlike nuclear weapons, AI offers no assurance of human control, as rogue agents hacking Hugging Face have shown. Sanger raises the fear that it may take a Cuban Missile Crisis-like AI event to rally the world.

The nuclear treaty took about 20 years after Hiroshima; AI may not allow even two and a half years.
Churchill's 1955 warning about staying within human control is the urgent AI problem.

34:38News update

What else happened today?

A federal judge, a Trump appointee, found the president's order barring CNN, MSNBC and Politico from the White House likely unconstitutional and ordered their access restored; Trump is expected to appeal. Iran's president Masoud Pezeshkian told the UN that Iran would defend itself until its last breath and would not give up its nuclear program.

The judge rejected Trump's national security justification for revoking press credentials.
Iran vowed not to be bullied into submission by the United States.

Notes

The Nuclear Precedent

Sanger draws a parallel between AI today and nuclear weapons from the late 1940s through the Cold War: two states held a terrifying technology with commercial uses and extinction-level risk.
The Manhattan Project, driven by fear that Germany would get the bomb first, produced the Trinity test in summer 1945, despite Los Alamos scientists calculating the test might ignite the atmosphere.
Hiroshima and Nagasaki killed around 200,000 people, but the arms race only accelerated: the Soviets tested by 1949, and Britain, France, and China followed.

From Crisis to Treaty

The Cuban Missile Crisis of October 1962 brought the U.S. to the brink, after which Kennedy gave his June 1963 American University speech: "We all breathe the same air... and we are all mortal."
Kennedy predicted 15 to 20 nuclear-armed nations by the mid-70s; the Nuclear Nonproliferation Treaty, signed July 1, 1968, prevented that outcome.
The treaty let five nuclear states commit to eventual disarmament, barred others from building weapons in exchange for peaceful nuclear technology, and created the IAEA in Vienna to inspect for cheating.
Outliers exist: Israel never signed, North Korea renounced it in the mid-1990s and has 60 to 100 weapons, and India and Pakistan never signed. Still, only nine nuclear states exist today.

Why AI Is Different

Nuclear was simple: states held the technology, uranium and plutonium were chokepoints, and launch authority was limited.
AI's "oxygen" is compute, which will spread to basements, terror groups, and rogue states as the technology becomes more efficient.
Leaders don't agree on the threat: Trump calls warnings a hoax; Xi worries mainly about the Communist Party. Both see AI as essential to economic survival and dominance, with China aiming to be the top power by 2049.
China already regulates AI more than the U.S. and views chip export controls as "modern containment," so it fears a treaty would lock in U.S. advantages.

What Could Actually Happen

The Biden administration took a year and a half to get China to agree to one sentence: no AI agents controlling nuclear weapons stockpiles.
Possible next steps: keeping AI away from biological weapons design and hacking of power grids and water systems; early warning sharing; and IAEA-style independent inspectors, as proposed by Anthropic's Dario Amadei.
Sanger sees no chance of a treaty this week, only perhaps a second sentence and a joint experts committee.
The biggest differences are speed and control: some engineers say recursive self-improvement or AGI is months to two years away, and unlike the nuclear era, AI offers no assurance of human control, echoing Churchill's 1955 warning that the atomic bomb "did not carry us outside the scope of human control."

Transcribed automatically from the audio. Summary and notes written by AI from the transcript, so check anything important against the recording.

Recently transcribed

Watch this if you constantly worry | Finding Freedom Podcast
Peter Crone · YouTube · 1 h 5 min
Every Time You Try to Fix Yourself You Make It Worse.
Peter Crone · YouTube · 1 h 12 min
你家神明廳位子擺對了嗎_用風水讓生活更順風順水feat.徐玉蘭老師
科技仙姑實驗室 · YouTube · 53 min

Browse all transcripts