WEBVTT 00:00:00.480 --> 00:00:03.917 There are many things that AI cannot and should not automate. 00:00:04.000 --> 00:00:07.397 AI should be built for people, not simply to turn the crank crank of the 00:00:07.480 --> 00:00:10.757 machines faster for their own sake. 00:00:10.840 --> 00:00:13.797 Some of the things people working to build AI have said are so dystopian that 00:00:13.880 --> 00:00:17.197 they sound like the plot of bad science fiction movies. 00:00:17.280 --> 00:00:21.117 Anything this power is also powerful is also intimidating. 00:00:21.200 --> 00:00:22.717 This moment is complicated as we feel 00:00:22.800 --> 00:00:24.837 both tremendous potential and the very 00:00:24.920 --> 00:00:28.477 understandable anxiety at the same time. 00:00:28.560 --> 00:00:31.517 These models can help people discover new knowledge, new medicines, better 00:00:31.600 --> 00:00:34.837 health care, stronger education, and build more productive economies. They 00:00:34.920 --> 00:00:38.877 can be new tools for people to pursue dreams and ambition that today feel out 00:00:38.960 --> 00:00:42.077 of reach. The complexity of life has increased so 00:00:42.160 --> 00:00:44.677 dramatically for the average person. 00:00:44.760 --> 00:00:48.237 AI can be a defensive tool against this complexity as much as it can be this 00:00:48.320 --> 00:00:52.597 tool of unprecedented discovery and innovation. 00:00:52.680 --> 00:00:54.997 On the other hand, as AI systems become 00:00:55.080 --> 00:00:57.637 more capable and more autonomous, they 00:00:57.720 --> 00:01:02.677 could move faster than our institutions, concentrate power in two two few hands, 00:01:02.760 --> 00:01:07.677 or make decisions that people no longer understand or control. 00:01:07.760 --> 00:01:10.997 That concern becomes especially important as we approach systems that 00:01:11.080 --> 00:01:14.437 can improve themselves and future versions of themselves, 00:01:14.520 --> 00:01:17.437 often called recursive self-improvement. 00:01:17.520 --> 00:01:21.557 As the process of building AI becomes more automated, the pace of AI progress 00:01:21.640 --> 00:01:23.437 could accelerate rapidly. 00:01:23.520 --> 00:01:26.637 This moment calls for extreme care. 00:01:26.720 --> 00:01:30.517 I largely agree with Professor Bengio, uh although I'll say two instead of 00:01:30.600 --> 00:01:34.357 three. I believe there are two ways that AI progress could go very badly that we 00:01:34.440 --> 00:01:38.477 have to avoid. First, we could lose control of the future to 00:01:38.560 --> 00:01:42.237 AI. The risk is that it moves so fast that 00:01:42.320 --> 00:01:46.037 people can no longer follow what's happening or intervene when needed. This 00:01:46.120 --> 00:01:49.517 would obviously be terrible. The industry must not accept too much 00:01:49.600 --> 00:01:52.797 technological risk just because the benefits are too great and that they 00:01:52.880 --> 00:01:54.837 feel too important to slow down. 00:01:54.920 --> 00:01:58.517 Beating companies in a competitive race is not a reason to make rash decisions, 00:01:58.600 --> 00:02:01.397 nor do we believe we are locked in a race where we are unable to do that. We 00:02:01.480 --> 00:02:04.797 have unilaterally slowed down in the past. We will do so in the future. We do 00:02:04.880 --> 00:02:07.637 not want to build systems. We think it's important that everyone avoids building 00:02:07.720 --> 00:02:11.397 systems where we cannot make the alignment, monitorability, and safety 00:02:11.480 --> 00:02:14.797 guarantees that we must. We need to understand what these systems 00:02:14.880 --> 00:02:18.637 are doing and have strong evidence that they will do what people intend even as 00:02:18.720 --> 00:02:23.637 they get very, very smart. Actually, especially as they get very, very smart. 00:02:23.720 --> 00:02:27.637 It doesn't matter whether people put the risk of catastrophe at 10% or 1% or 12% 00:02:27.720 --> 00:02:31.837 or 0.1%. None of these levels are remotely acceptable. And we should not 00:02:31.920 --> 00:02:35.157 train models that we cannot make an extremely strong case they will be able 00:02:35.240 --> 00:02:40.237 to keep under human control. So, that's one way things can go wrong. 00:02:40.320 --> 00:02:43.397 In the other direction, these systems could concentrate too much power in too 00:02:43.480 --> 00:02:47.277 few hands. No one person or company or country 00:02:47.360 --> 00:02:51.397 should be able to use the most powerful AI models to impose their worldview on 00:02:51.480 --> 00:02:54.837 everyone else. A company or country that believes only 00:02:54.920 --> 00:02:58.397 it can be trusted with this technology can use that belief to justify almost 00:02:58.480 --> 00:03:01.517 anything else. We have to reject that logic even when 00:03:01.600 --> 00:03:04.277 it is convenient for any one actor. 00:03:04.360 --> 00:03:07.957 I believe there is a pragmatic path through both of these. 00:03:08.040 --> 00:03:11.597 At OpenAI, we approach these questions with a clear goal. 00:03:11.680 --> 00:03:15.317 AI should give people more power over their own lives, more freedom to live 00:03:15.400 --> 00:03:19.237 lives the way they want, more ability to solve problems big or small, and to 00:03:19.320 --> 00:03:23.717 improve their communities. We don't want to fall into the trap of blind optimism. 00:03:23.800 --> 00:03:26.757 We also don't want to fall into the trap of doomerism, either. 00:03:26.840 --> 00:03:30.197 Getting pulled into either extreme will not help us successfully figure out the 00:03:30.280 --> 00:03:33.397 challenges in front of us, especially at a time when the world and 00:03:33.480 --> 00:03:38.357 its people face such enormous challenges that this technology can help solve. 00:03:38.440 --> 00:03:41.957 So, we try to walk the middle path between these two poles. 00:03:42.040 --> 00:03:44.197 For us, that means three principles. 00:03:44.280 --> 00:03:49.277 First, as AI becomes more capable, people must remain at the center of AI 00:03:49.360 --> 00:03:52.437 decision-making. Alignment is not an abstract research 00:03:52.520 --> 00:03:56.397 question. It is the work of ensuring that these systems reliably remain under 00:03:56.480 --> 00:03:59.757 human control, reflect human values, and help people 00:03:59.840 --> 00:04:02.157 guide the next stages of development. 00:04:02.240 --> 00:04:06.717 We are not trying to and must not automate human judgment or human values. 00:04:06.800 --> 00:04:10.877 Second, the benefits of scientific progress and economic growth must be by 00:04:10.960 --> 00:04:13.717 people and for people. AI should help researchers make 00:04:13.800 --> 00:04:17.117 discoveries, doctors treat patients, teachers teach students, entrepreneurs 00:04:17.200 --> 00:04:20.437 start companies, communities solve problems that felt out of reach, and 00:04:20.520 --> 00:04:25.437 much more. The continuous story of human progress is not the tools replace human 00:04:25.520 --> 00:04:29.157 agency, it is that the right tools make people more capable, and I believe this 00:04:29.240 --> 00:04:32.397 is going to go much further than we believe possible. 00:04:32.480 --> 00:04:36.157 Third, this technology must empower people individually. 00:04:36.240 --> 00:04:39.437 A great future will come from people realizing their potential and building 00:04:39.520 --> 00:04:43.757 value for themselves, their communities, their countries, and the broader world. 00:04:43.840 --> 00:04:46.837 We might be with the builders, one of the builders of this technology, but we 00:04:46.920 --> 00:04:48.597 are not the heroes of this story. 00:04:48.680 --> 00:04:50.237 Our role is to enable people all over 00:04:50.320 --> 00:04:56.077 the world, and to trust in the magic of humanity's skill and faculties. 00:04:56.160 --> 00:05:00.277 We now have models capable of discovering knowledge that we have never had before. 00:05:00.360 --> 00:05:02.837 Consider the progress we've seen in math. 00:05:02.920 --> 00:05:06.557 Three summers ago, our models were okay at grade school math. 00:05:06.640 --> 00:05:10.437 Two summers ago, they were pretty good at high school math competitions. 00:05:10.520 --> 00:05:13.997 Last summer, we reached gold a gold level medal in the most prestigious 00:05:14.080 --> 00:05:16.197 international math competition. 00:05:16.280 --> 00:05:19.317 And just a few weeks ago, this summer, one of our models solved one of the 00:05:19.400 --> 00:05:23.077 Millennium Prize problems, the Navier-Stokes equations. 00:05:23.160 --> 00:05:26.317 These equations are used for aircraft design, weather forecasting, and the 00:05:26.400 --> 00:05:29.197 study of blood flow. We are not solving math problems for 00:05:29.280 --> 00:05:33.117 their own sake, but to empower people to discover new knowledge, and to use it to 00:05:33.200 --> 00:05:37.237 improve health care, raise standards of livings of living, and expand everyone's 00:05:37.320 --> 00:05:41.757 potential around the world. There remains so much to be discovered. 00:05:41.840 --> 00:05:45.957 But it's very important that companies not substitute for the democratic process. 00:05:46.040 --> 00:05:49.597 If AI is to be democratic, the most important decisions cannot be made by 00:05:49.680 --> 00:05:54.317 labs in San Francisco alone. They must be shaped through democratic processes 00:05:54.400 --> 00:05:57.797 and by governments accountable to the people that they serve. 00:05:57.880 --> 00:06:01.717 At the international level, this will require cooperation. 00:06:01.800 --> 00:06:04.837 In our history, there have been times where countries who compete and don't 00:06:04.920 --> 00:06:08.477 always like each other very much still come together for shared interests and 00:06:08.560 --> 00:06:11.957 the collective good in the face of a powerful new technology. 00:06:12.040 --> 00:06:15.517 We believe this must be one of those times. 00:06:15.600 --> 00:06:18.797 We need a mechanism for complementary national and international frontier AI 00:06:18.880 --> 00:06:23.757 standards. Standards for measuring capabilities, assessing risks, 00:06:23.840 --> 00:06:27.077 determining whether safeguards are sufficient, and preserving meaningful 00:06:27.160 --> 00:06:30.837 human oversight as systems become more autonomous. 00:06:30.920 --> 00:06:35.357 We need common standards so countries can compare evidence, verify compliance, 00:06:35.440 --> 00:06:39.117 and have a shared language and understanding about what is happening. 00:06:39.200 --> 00:06:41.677 We need accurate and speedy incident reporting, 00:06:41.760 --> 00:06:45.477 classification and reporting protocols so the world can learn from failures 00:06:45.560 --> 00:06:48.117 before they become catastrophes. 00:06:48.200 --> 00:06:51.237 And we need secure channels among governments, critical infrastructure 00:06:51.320 --> 00:06:54.877 operators, and technical experts to share emerging vulnerabilities and new 00:06:54.960 --> 00:06:58.277 threats. Obviously, these standards should not 00:06:58.360 --> 00:07:02.117 lock in incumbents or favor one business model over another. 00:07:02.200 --> 00:07:05.637 They must support open and closed model developers, new entrants and established 00:07:05.720 --> 00:07:09.637 labs, national authority, and international cooperation. 00:07:09.720 --> 00:07:12.517 Each government should decide how to incorporate standards into its own legal 00:07:12.600 --> 00:07:16.717 system, but we will all be better off if we can agree on what good evidence, good 00:07:16.800 --> 00:07:21.157 safeguards, and good oversight look like on the global stage. 00:07:21.240 --> 00:07:25.037 We do not have all the answers, and we're very open to any better ideas. 00:07:25.120 --> 00:07:27.877 We're ready to work with you all and try our hardest to figure out the path 00:07:27.960 --> 00:07:31.877 forward. We are at a crossroads. 00:07:31.960 --> 00:07:36.197 One path leads to a world where power is concentrated, decisions are opaque, and 00:07:36.280 --> 00:07:38.997 people feel that the most important technology of their lifetime is 00:07:39.080 --> 00:07:43.397 something being done to them. The other path leads to a world where AI 00:07:43.480 --> 00:07:48.277 works for people because people, through democratic institutions, help decide how 00:07:48.360 --> 00:07:52.717 it should develop and guide it. Where scientific progress improves real lives, 00:07:52.800 --> 00:07:55.837 where more people can participate in the future, and where this technology 00:07:55.920 --> 00:07:59.437 continues to give individuals and communities greater power to live life 00:07:59.520 --> 00:08:02.597 as they choose. This is a future of realized potential, 00:08:02.680 --> 00:08:05.757 excitement, and wonder, and it is the future we believe is worth building. 00:08:05.840 --> 00:08:08.640 Thank you very much. 00:08:09.080 --> 00:08:11.957 >> I thank Mr. Altman for his briefing. 00:08:12.040 --> 00:08:15.960 I now give the floor to Mr. Dario Amodei. 00:08:16.840 --> 00:08:20.757 >> Good afternoon. I'd like to thank France for convening this meeting and the 00:08:20.840 --> 00:08:23.597 members of the council for inviting me to speak. 00:08:23.680 --> 00:08:28.077 AI is advancing very quickly. Four years ago, it could barely write a line of 00:08:28.160 --> 00:08:31.957 code or solve a high school math problem. Today, it writes most of the 00:08:32.040 --> 00:08:36.197 code at Anthropic and solves world-famous unsolved math problems. The 00:08:36.280 --> 00:08:38.237 trajectory only needs to continue for a 00:08:38.320 --> 00:08:40.877 tiny period longer, one or two years, 00:08:40.960 --> 00:08:43.277 maybe less, to reach what I've called a 00:08:43.360 --> 00:08:45.917 country of geniuses in the data center. 00:08:46.000 --> 00:08:50.637 This rapid pace of progress will produce incredible benefits for the world. Just 00:08:50.720 --> 00:08:54.197 today, Anthropic announced the Claude-led discovery of a new molecular 00:08:54.280 --> 00:08:58.717 machine. It's a preliminary finding, but it may constitute a new gene editing 00:08:58.800 --> 00:09:03.077 mechanism that could have applications in in gene therapy. Claude led the 00:09:03.160 --> 00:09:05.117 literature review, theorizing, and 00:09:05.200 --> 00:09:10.237 experimental design, and humans ran and verified the experiments in the lab. 00:09:10.320 --> 00:09:14.557 We believe this type of application will accelerate rapidly in the years ahead 00:09:14.640 --> 00:09:18.477 and that we can make profound progress in the near future on diseases that have 00:09:18.560 --> 00:09:21.197 plagued humanity for millennia. 00:09:21.280 --> 00:09:23.437 But these remarkable capabilities also 00:09:23.520 --> 00:09:25.997 come with risks that must be managed. 00:09:26.080 --> 00:09:30.837 I see two main categories of risk. The first is misuse of AI models, for 00:09:30.920 --> 00:09:35.437 example, misuse by bioterrorists to create biological weapons. The second 00:09:35.520 --> 00:09:39.517 risk is loss of control. Model capabilities accelerating beyond 00:09:39.600 --> 00:09:41.837 developers' ability to control them. 00:09:41.920 --> 00:09:46.437 These questions are global in scope and concern all of humanity. They are grave 00:09:46.520 --> 00:09:48.037 enough that I believe they are important 00:09:48.120 --> 00:09:50.357 questions for this council, not just for 00:09:50.440 --> 00:09:53.517 economic or ICT ministries. If managed 00:09:53.600 --> 00:09:58.277 poorly, I even believe that AI could be a risk to humanity as a whole. 00:09:58.360 --> 00:10:02.037 At Anthropic, we believe we have the best track record in the industry for 00:10:02.120 --> 00:10:06.117 preventing these risks, both operationally and in our research. We 00:10:06.200 --> 00:10:09.997 test every model for dangerous capabilities in areas like cybersecurity 00:10:10.080 --> 00:10:14.917 and biology. We put strong safeguards on our most capable models, and we publish 00:10:15.000 --> 00:10:19.357 our work in these areas. We have pioneered research in areas like 00:10:19.440 --> 00:10:23.037 mechanistic interpretability and alignment science that improve our 00:10:23.120 --> 00:10:26.077 ability to steer and control our models. 00:10:26.160 --> 00:10:30.717 We also have unparalleled transparency practices, releasing risk reports that 00:10:30.800 --> 00:10:32.797 run to hundreds of pages. But we 00:10:32.880 --> 00:10:35.397 understand that as AI models become more 00:10:35.480 --> 00:10:37.997 powerful, even more stringent standards 00:10:38.080 --> 00:10:40.357 of safety will be required. And we are 00:10:40.440 --> 00:10:43.677 always aiming to improve our safety and 00:10:43.760 --> 00:10:48.357 alignment practices, our abilities our ability to prevent misuse, and we are 00:10:48.440 --> 00:10:50.597 always striving to do better. We will 00:10:50.680 --> 00:10:53.477 slow down as much as necessary in order 00:10:53.560 --> 00:10:58.797 to make sure that every successive AI technology that we release is actually 00:10:58.880 --> 00:11:02.397 safe. But regardless of what we do, managing 00:11:02.480 --> 00:11:06.197 these risks is ultimately bigger than anyone company. And it has an 00:11:06.280 --> 00:11:09.077 industry-wide and global scale. 00:11:09.160 --> 00:11:11.197 On September 12th, I called for a 00:11:11.280 --> 00:11:13.077 collaborative effort to pace the 00:11:13.160 --> 00:11:16.197 frontier, not to stop AI progress, but 00:11:16.280 --> 00:11:18.357 to work together to make sure the 00:11:18.440 --> 00:11:20.597 industry and the world can proceed at a 00:11:20.680 --> 00:11:25.597 rate where we can be sure that safety keeps up with capability for everyone in 00:11:25.680 --> 00:11:29.837 the industry. This call had three parts. First, we 00:11:29.920 --> 00:11:32.357 committed to embed external evaluators 00:11:32.440 --> 00:11:34.357 inside Anthropic with employee-like 00:11:34.440 --> 00:11:37.437 access, similar to a food inspector. And 00:11:37.520 --> 00:11:42.357 And we recommended that other companies across the world do the same. Some have 00:11:42.440 --> 00:11:46.197 already agreed uh to adopt this measure. 00:11:46.280 --> 00:11:48.797 Second, we called for cooperation across 00:11:48.880 --> 00:11:53.357 the industry to set standards and modulate the pace of progress. And 00:11:53.440 --> 00:11:58.117 finally, we called for global cooperation across the world between 00:11:58.200 --> 00:12:02.957 governments to set international standards. It is this last area that 00:12:03.040 --> 00:12:04.837 most squarely falls within this 00:12:04.920 --> 00:12:07.637 council's remit. I want to offer three 00:12:07.720 --> 00:12:10.997 ideas in this regard. First, we should 00:12:11.080 --> 00:12:13.197 begin with narrow agreements that every 00:12:13.280 --> 00:12:15.957 member can support, such as a ban on 00:12:16.040 --> 00:12:19.077 using AI to make biological weapons or 00:12:19.160 --> 00:12:21.997 permitting your AI technologies to be 00:12:22.080 --> 00:12:25.837 used to make biological weapons. Second, 00:12:25.920 --> 00:12:27.597 we should build evaluation and 00:12:27.680 --> 00:12:30.677 verification systems that keep pace with 00:12:30.760 --> 00:12:33.037 AI development so that states can have 00:12:33.120 --> 00:12:34.917 visibility into frontier model 00:12:35.000 --> 00:12:37.877 capability and can verify each other's 00:12:37.960 --> 00:12:40.717 commitments. Third, we should establish 00:12:40.800 --> 00:12:43.277 common global standards for testing AI 00:12:43.360 --> 00:12:46.397 models for loss of control risks and 00:12:46.480 --> 00:12:49.597 misuse risks, and a notification system 00:12:49.680 --> 00:12:54.117 for AI incidents that are significant to global security. 00:12:54.200 --> 00:12:58.477 I believe that this is the most important global security issue facing 00:12:58.560 --> 00:13:03.357 the world today. For all the differences of the people in this room, we must put 00:13:03.440 --> 00:13:05.437 aside those differences in order to 00:13:05.520 --> 00:13:08.437 confront this global opportunity and 00:13:08.520 --> 00:13:13.437 global threat that is being presented to us at the same time. No leader, no 00:13:13.520 --> 00:13:16.437 company, and no nation can manage this alone. 00:13:16.520 --> 00:13:18.357 We commit to working with governments in 00:13:18.440 --> 00:13:21.037 this room on this urgent work. 00:13:21.120 --> 00:13:24.200 Thank you for your attention.