
Altman and Amodei address UN: 'There are many things that AI cannot and should not automate'
- Length
- 13 min
- Language
- English
- Text from
- the video's captions
- Transcribed
- 13 October 2026
Summary
Sam Altman of OpenAI and Dario Amodei of Anthropic address the UN Security Council on the promise and risks of AI. Altman argues AI should be built for people, identifies two catastrophic risks — losing control to AI and concentrating power in too few hands — and calls for international frontier AI standards. Amodei describes AI's rapid progress toward a "country of geniuses in the data center," outlines misuse and loss-of-control risks, and proposes narrow global agreements, verification systems, and common testing standards. Both stress that no company or nation can manage these risks alone.
Key points
- Sam Altman says there are many things AI cannot and should not automate, and that AI should be built for people.
- Altman identifies two catastrophic risks: losing control of the future to AI, and concentrating too much power in too few hands.
- Altman says OpenAI has unilaterally slowed down in the past and will do so in the future, and that no level of catastrophe risk is acceptable.
- Altman highlights that an OpenAI model solved the Navier-Stokes equations, a Millennium Prize problem.
- Altman calls for complementary national and international frontier AI standards, incident reporting protocols, and secure information-sharing channels.
- Dario Amodei says AI is one or two years, maybe less, from what he calls a country of geniuses in the data center.
- Amodei describes two main risks: misuse of AI models, such as by bioterrorists, and loss of control as capabilities accelerate.
- Amodei proposes narrow agreements like a ban on using AI for biological weapons, evaluation and verification systems, and common global testing standards with incident notification.
Questions it answers
00:00AI's promise and risks
What is the tension Altman describes around AI?
Altman says AI should be built for people, and while it can enable discovery in medicines, health care, education, and productivity, it could also move faster than institutions, concentrate power, and become uncontrollable, especially with recursive self-improvement.
- AI can be a defensive tool against life's increasing complexity.
- Recursive self-improvement could accelerate AI progress beyond institutional control.
01:23Two ways AI could go badly
What two catastrophic scenarios does Altman say must be avoided?
Losing control of the future to AI, and concentrating too much power in too few hands. He says no catastrophe risk level is acceptable and OpenAI has unilaterally slowed down before and will again.
- Companies must not build systems without alignment, monitorability, and safety guarantees.
- No person, company, or country should impose their worldview via powerful AI.
02:37OpenAI's three principles
What principles guide OpenAI, and what progress does Altman cite?
People must remain at the center of AI decision-making; benefits must be by and for people; and the technology must empower people individually. He also cites math progress culminating in a model solving the Navier-Stokes equations.
- Alignment means ensuring systems reliably remain under human control.
- OpenAI sees itself as a builder, not the hero of the story.
05:55Global standards and the crossroads
What international measures does Altman call for?
Complementary national and international frontier AI standards, incident reporting protocols, and secure channels to share vulnerabilities. He warns of a crossroads between concentrated, opaque power and a democratic, empowering future.
- Standards must support open and closed developers, new entrants and established labs.
- The most important decisions cannot be made by labs in San Francisco alone.
07:54Amodei on progress and risk
How fast is AI advancing and what risks does Amodei see?
He says AI is one or two years, maybe less, from a country of geniuses in the data center, and highlights Claude's discovery of a possible new gene editing mechanism. He identifies misuse (e.g., biological weapons) and loss of control as the two main risks.
- Claude led the literature review, theorizing, and experimental design; humans verified experiments.
- Amodei believes AI could be a risk to humanity as a whole if managed poorly.
10:06Anthropic's safety practices and proposals
What is Anthropic doing on safety and what did Amodei propose?
Anthropic tests every model for dangerous capabilities, publishes long risk reports, and pioneered mechanistic interpretability. On September 12th he called for external evaluators with employee-like access, industry cooperation on pace, and global government standards.
- Anthropic will slow down as much as necessary to keep releases safe.
- He proposes narrow agreements like a ban on AI-made biological weapons, verification systems, and common testing standards with incident notification.
Notes
Overview
- Sam Altman (OpenAI) and Dario Amodei (Anthropic) brief the UN Security Council, convened by France, on AI's benefits and risks.
- Both frame AI as a global issue requiring international cooperation, not just economic or ICT ministries.
Altman's remarks
- "There are many things that AI cannot and should not automate" — AI should be built for people, not to turn the crank of machines faster for their own sake.
- He sees two ways progress could go badly: losing control of the future to AI, and concentrating too much power in too few hands.
- He agrees with Professor Bengio but says two risks rather than three.
- No catastrophe risk level — 10%, 1%, 12%, or 0.1% — is remotely acceptable; models without an extremely strong case for human control should not be trained.
- OpenAI has unilaterally slowed down in the past and will do so again; it rejects both blind optimism and doomerism, walking a middle path.
- Three principles: people must remain at the center of AI decision-making; scientific and economic progress must be by and for people; the technology must empower people individually.
- Math progress: grade school math three summers ago, high school competitions two summers ago, a gold-level medal at the most prestigious international math competition last summer, and recently a model solved the Navier-Stokes equations, a Millennium Prize problem used in aircraft design, weather forecasting, and blood flow study.
- Calls for complementary national and international frontier AI standards, accurate and speedy incident reporting, and secure channels among governments, critical infrastructure operators, and technical experts.
Amodei's remarks
- Four years ago AI could barely write code; today it writes most of the code at Anthropic and solves world-famous unsolved math problems.
- The trajectory needs only one or two years, maybe less, to reach a "country of geniuses in the data center."
- Anthropic announced a Claude-led discovery of a new molecular machine, possibly a new gene editing mechanism with gene therapy applications; Claude led literature review, theorizing, and experimental design while humans ran and verified experiments.
- Two main risks: misuse (e.g., bioterrorists creating biological weapons) and loss of control as capabilities accelerate beyond developers' ability to control them.
- Anthropic tests every model for dangerous capabilities in cybersecurity and biology, publishes risk reports of hundreds of pages, and pioneered mechanistic interpretability and alignment science.
- On September 12th he called for a collaborative effort to "pace the frontier": embedding external evaluators with employee-like access, industry-wide standards cooperation, and global government cooperation.
- Three proposals: narrow agreements like a ban on AI-made biological weapons, evaluation and verification systems for states, and common global testing standards with an incident notification system for global security.
Text from the video's own captions. Summary and notes written by AI from the transcript, so check anything important against the recording.