# FULL REMARKS: OpenAI CEO Sam Altman Warns UN of AI Risks, Calls for Human Control | AI15 *DWS News* · https://www.youtube.com/watch?v=wgZXXFgESKg ## Summary Sam Altman, CEO of OpenAI, addresses the UN Security Council on the risks and promise of AI. He identifies two major risks—losing control of the future to AI and concentrating too much power in too few hands—and lays out three principles for a safe path forward. He calls for international cooperation, common standards for frontier AI, and democratic oversight of the technology's development. ## Key points - Altman says AI should be built for people, not simply to accelerate machines for their own sake. - He warns of two ways AI progress could go badly: losing human control of the future, and concentrating too much power in too few hands. - He states that no level of catastrophic risk—10%, 1%, or 0.1%—is remotely acceptable, and models must be kept under human control. - OpenAI has unilaterally slowed down in the past and will do so in the future rather than race rashly. - He outlines three principles: keeping people at the center of AI decision-making, ensuring benefits are by and for people, and empowering people individually. - He cites rapid progress in math, including a model solving the Navier-Stokes equations, a millennium prize problem. - He argues the most important AI decisions cannot be made by labs in San Francisco alone and must be shaped through democratic processes and governments. - He calls for complementary national and international frontier AI standards, incident reporting protocols, and secure channels to share vulnerabilities and threats. - He frames the moment as a crossroads between concentrated, opaque power and a democratic future where AI works for people. ## Notes ### Context and Framing - Sam Altman addresses the UN Security Council, convened by France, on the risks and potential of AI. - He describes the moment as complicated: tremendous potential alongside very understandable anxiety. - AI can help discover new knowledge, medicines, better health care, stronger education, and more productive economies. ### Two Ways AI Progress Could Go Badly - **Losing control to AI:** As systems become more capable and autonomous, they could move faster than our institutions or make decisions people no longer understand or control. This is especially concerning with recursive self-improvement. - Altman says no level of catastrophic risk—10%, 1%, or 0.1%—is remotely acceptable, and we should not train models we cannot make an extremely strong case will stay under human control. - **Concentration of power:** No one person, company, or country should use the most powerful AI models to impose their worldview on everyone else. ### OpenAI's Three Principles - People must remain at the center of AI decision-making; alignment means systems reliably remain under human control and reflect human values. - The benefits of scientific progress and economic growth must be by people and for people. - The technology must empower people individually; OpenAI is a builder but "not the heroes of this story." ### Progress and Democratic Oversight - Altman cites math progress: grade school level three summers ago, gold medal at the most prestigious international math competition last summer, and a model solving the Navier-Stokes equations, a millennium prize problem. - He stresses that companies must not substitute for the democratic process; decisions cannot be made by labs in San Francisco alone. ### International Cooperation - He calls for complementary national and international frontier AI standards: measuring capabilities, assessing risks, verifying safeguards, and preserving human oversight. - He urges accurate and speedy incident reporting and secure channels among governments, critical infrastructure operators, and technical experts. - Standards should not lock in incumbents and must support open and closed model developers, new entrants, and established labs. ### Conclusion - Altman frames the moment as a crossroads: one path of concentrated, opaque power; another where AI works for people through democratic institutions. ## Transcript [00:01] There are many things that AI cannot and should not automate. AI should be built for people, not simply to turn the crank crank of the machines faster for their own sake. Some of the things people working to build AI have said are so dystopian that they sound like the plot of bad science fiction movies. Anything this power is also powerful is also intimidating. This moment is complicated as we feel both tremendous potential and very understandable anxiety at the same time. These models can help people discover new knowledge, new medicines, better health care, stronger education, and build more productive economies. They can be new tools for people to pursue dreams and ambition that today feel out [00:40] of reach. The complexity of life has increased so dramatically for the average person. AI can be a defensive tool against this complexity as much as it can be this tool of unprecedented discovery and innovation. On the other hand, as AI systems become more capable and more autonomous, they could move faster than our institutions, concentrate power in two two few hands, or make decisions that people no longer understand or control. That concern becomes especially important as we approach systems that can improve themselves and future versions of themselves, often called recursive self-improvement. As the process of building AI becomes more automated, the pace of AI progress could accelerate rapidly. [01:24] This moment calls for extreme care. I largely agree with Professor Bengio. Uh although I'll say two instead of three. I believe there are two ways that AI progress could go very badly that we have to avoid. First, we could lose control of the future to AI. The risk is that it moves so fast that people can no longer follow what's happening or intervene when needed. This would obviously be terrible. The industry must not accept too much technological risk just because the benefits are too great and that they feel too important to just down. Beating companies in a race is not a reason to make rash decisions, nor do we believe we are locked in a race where we are unable to do that. We have unilaterally slowed down in the past. We [02:04] will do so in the future. We do not want to build systems. We think it's important that everyone avoids building systems where we cannot make the alignment, monitorability, and safety guarantees that we must. We need to understand what these systems are doing and have strong evidence that they will do what people intend even as they get very, very smart. Actually, especially as they get very, very smart. It doesn't matter whether people put the risk of catastrophe at 10% or 1% or 12% or 0.1%. None of these levels are remotely acceptable. And we should not train models that we cannot make an extremely strong case they will be able to keep under human control. So, that's one way things can go wrong. [02:41] In the other direction, these systems could concentrate too much power in too few hands. No one person or company or country should be able to use the most powerful AI models to impose their worldview on everyone else. A company or country that believes only it can be trusted with this technology can use that belief to justify almost anything else. We have to reject that logic even when it's convenient for any one actor. I believe there is a pragmatic path through both of these. At OpenAI, we approach these questions with a clear goal. AI should give people more power over their own lives, more freedom to live lives the way they want, more ability to solve problems, big or small, and to improve their communities. [03:22] We don't want to fall into the trap of blind optimism. We also don't want to fall into the trap of doomerism, either. Getting pulled into either extreme will not help us successfully figure out the challenges in front of us, especially at a time when the world and its people face such enormous challenges that this technology can help solve. So, we try to walk the middle path between these two poles. For us, that means three principles. First, as AI becomes more capable, people must remain at the center of AI decision-making. Alignment is not an abstract research question. It is the work of ensuring that these systems reliably remain under human control, reflect human values, and help people guide the next stages of development. [04:03] We are not trying to and must not automate human judgment or human values. Second, the benefits of scientific progress and economic growth must be by people and for people. AI should help researchers make discoveries, doctors treat patients, teachers teach students, entrepreneurs start companies, communities solve problems that felt out of reach, and much more. The continuous story of human progress is not the tools replace human agency, it is that the right tools make people more capable, and I believe this is going to go much further than we believe possible. Third, this technology must empower people individually. A great future will come from people realizing their potential and building value for themselves, their communities, [04:43] their countries, and the broader world. We might be with the builders, one of the builders of this technology, but we are not the heroes of this story. Our role is to enable people all over the world, and to trust in the magic of humanity's skill and faculties. We now have models capable of discovering knowledge that we have never had before. Consider the progress we've seen in math. Three summers ago, our models were okay at grade school math. Two summers ago, they were pretty good at high school math competitions. Last summer, we reached gold level a gold level medal in the most prestigious international math competition. And just a few weeks ago this summer, one of our models solved one of the [05:20] millennium prize problems, the Navier-Stokes equations. These equations are used for aircraft design, weather forecasting, and the study of blood flow. We are not solving math problems for their own sake, but to empower people to discover new knowledge, and to use it to improve health care, raise standards of livings st- of living, and expand everyone's potential around the world. There remains so much to be discovered. But it's very important that companies not substitute for the democratic process. If AI is to be democratic, the most important decisions cannot be made by labs in San Francisco alone. They must be shaped through democratic processes and by governments accountable to the people that they serve. [05:59] At the international level, this will require cooperation. In our history, there have been times where countries who compete and don't always like each other very much still come together for shared interests and the collective good in the face of a powerful new technology. We believe this must be one of those times. We need a mechanism for complementary national and international frontier AI standards. Standards for measuring capabilities, assessing risks, determining whether safeguards are sufficient, and preserving meaningful human oversight as systems become more autonomous. We need common standards so countries can compare evidence, verify compliance, and have a shared language and understanding about what is [06:39] happening. We need to accurate and speedy incident reporting, classification and reporting protocols so the world can learn from failures before they become catastrophes. And we need secure channels among governments, critical infrastructure operators, and technical experts to share emerging vulnerabilities and new threats. Obviously, these standards should not lock in incumbents or favor one business model over another. They must support open and closed model developers, new entrants and established labs, national authority, and international cooperation. Each government should decide how to incorporate standards into its own legal system, but we will all be better off if we can agree on what good evidence, good [07:18] safeguards, and good oversight look like on the global stage. We do not have all the answers, and we're very open to any better ideas. We're ready to work with you all and try our hardest to figure out the path forward. We are at a crossroads. One path leads to a world where power is concentrated, decisions are opaque, and people feel that the most important technology of their lifetime is something being done to them. The other path leads to a world where AI works for people, because people, through democratic institutions, help decide how it should develop and guide it. Where scientific progress improves real lives, where more people can participate in the future, and where this technology continues to give [07:58] individuals and communities greater power to live life as they choose. This is a future of realized potential, excitement, and wonder, and it is the future we believe is worth building. Thank you very much. >> I thank Mr. Altman for his briefing. I now give the floor to Mr. Dario Amodei. >> Good afternoon. I'd like to thank France for convening this meeting, and the members of the council for inviting me to speak.