1 00:00:00,480 --> 00:00:03,917 There are many things that AI cannot and should not automate. 2 00:00:04,000 --> 00:00:07,397 AI should be built for people, not simply to turn the crank crank of the 3 00:00:07,480 --> 00:00:10,757 machines faster for their own sake. 4 00:00:10,840 --> 00:00:13,797 Some of the things people working to build AI have said are so dystopian that 5 00:00:13,880 --> 00:00:17,197 they sound like the plot of bad science fiction movies. 6 00:00:17,280 --> 00:00:21,117 Anything this power is also powerful is also intimidating. 7 00:00:21,200 --> 00:00:22,717 This moment is complicated as we feel 8 00:00:22,800 --> 00:00:24,837 both tremendous potential and the very 9 00:00:24,920 --> 00:00:28,477 understandable anxiety at the same time. 10 00:00:28,560 --> 00:00:31,517 These models can help people discover new knowledge, new medicines, better 11 00:00:31,600 --> 00:00:34,837 health care, stronger education, and build more productive economies. They 12 00:00:34,920 --> 00:00:38,877 can be new tools for people to pursue dreams and ambition that today feel out 13 00:00:38,960 --> 00:00:42,077 of reach. The complexity of life has increased so 14 00:00:42,160 --> 00:00:44,677 dramatically for the average person. 15 00:00:44,760 --> 00:00:48,237 AI can be a defensive tool against this complexity as much as it can be this 16 00:00:48,320 --> 00:00:52,597 tool of unprecedented discovery and innovation. 17 00:00:52,680 --> 00:00:54,997 On the other hand, as AI systems become 18 00:00:55,080 --> 00:00:57,637 more capable and more autonomous, they 19 00:00:57,720 --> 00:01:02,677 could move faster than our institutions, concentrate power in two two few hands, 20 00:01:02,760 --> 00:01:07,677 or make decisions that people no longer understand or control. 21 00:01:07,760 --> 00:01:10,997 That concern becomes especially important as we approach systems that 22 00:01:11,080 --> 00:01:14,437 can improve themselves and future versions of themselves, 23 00:01:14,520 --> 00:01:17,437 often called recursive self-improvement. 24 00:01:17,520 --> 00:01:21,557 As the process of building AI becomes more automated, the pace of AI progress 25 00:01:21,640 --> 00:01:23,437 could accelerate rapidly. 26 00:01:23,520 --> 00:01:26,637 This moment calls for extreme care. 27 00:01:26,720 --> 00:01:30,517 I largely agree with Professor Bengio, uh although I'll say two instead of 28 00:01:30,600 --> 00:01:34,357 three. I believe there are two ways that AI progress could go very badly that we 29 00:01:34,440 --> 00:01:38,477 have to avoid. First, we could lose control of the future to 30 00:01:38,560 --> 00:01:42,237 AI. The risk is that it moves so fast that 31 00:01:42,320 --> 00:01:46,037 people can no longer follow what's happening or intervene when needed. This 32 00:01:46,120 --> 00:01:49,517 would obviously be terrible. The industry must not accept too much 33 00:01:49,600 --> 00:01:52,797 technological risk just because the benefits are too great and that they 34 00:01:52,880 --> 00:01:54,837 feel too important to slow down. 35 00:01:54,920 --> 00:01:58,517 Beating companies in a competitive race is not a reason to make rash decisions, 36 00:01:58,600 --> 00:02:01,397 nor do we believe we are locked in a race where we are unable to do that. We 37 00:02:01,480 --> 00:02:04,797 have unilaterally slowed down in the past. We will do so in the future. We do 38 00:02:04,880 --> 00:02:07,637 not want to build systems. We think it's important that everyone avoids building 39 00:02:07,720 --> 00:02:11,397 systems where we cannot make the alignment, monitorability, and safety 40 00:02:11,480 --> 00:02:14,797 guarantees that we must. We need to understand what these systems 41 00:02:14,880 --> 00:02:18,637 are doing and have strong evidence that they will do what people intend even as 42 00:02:18,720 --> 00:02:23,637 they get very, very smart. Actually, especially as they get very, very smart. 43 00:02:23,720 --> 00:02:27,637 It doesn't matter whether people put the risk of catastrophe at 10% or 1% or 12% 44 00:02:27,720 --> 00:02:31,837 or 0.1%. None of these levels are remotely acceptable. And we should not 45 00:02:31,920 --> 00:02:35,157 train models that we cannot make an extremely strong case they will be able 46 00:02:35,240 --> 00:02:40,237 to keep under human control. So, that's one way things can go wrong. 47 00:02:40,320 --> 00:02:43,397 In the other direction, these systems could concentrate too much power in too 48 00:02:43,480 --> 00:02:47,277 few hands. No one person or company or country 49 00:02:47,360 --> 00:02:51,397 should be able to use the most powerful AI models to impose their worldview on 50 00:02:51,480 --> 00:02:54,837 everyone else. A company or country that believes only 51 00:02:54,920 --> 00:02:58,397 it can be trusted with this technology can use that belief to justify almost 52 00:02:58,480 --> 00:03:01,517 anything else. We have to reject that logic even when 53 00:03:01,600 --> 00:03:04,277 it is convenient for any one actor. 54 00:03:04,360 --> 00:03:07,957 I believe there is a pragmatic path through both of these. 55 00:03:08,040 --> 00:03:11,597 At OpenAI, we approach these questions with a clear goal. 56 00:03:11,680 --> 00:03:15,317 AI should give people more power over their own lives, more freedom to live 57 00:03:15,400 --> 00:03:19,237 lives the way they want, more ability to solve problems big or small, and to 58 00:03:19,320 --> 00:03:23,717 improve their communities. We don't want to fall into the trap of blind optimism. 59 00:03:23,800 --> 00:03:26,757 We also don't want to fall into the trap of doomerism, either. 60 00:03:26,840 --> 00:03:30,197 Getting pulled into either extreme will not help us successfully figure out the 61 00:03:30,280 --> 00:03:33,397 challenges in front of us, especially at a time when the world and 62 00:03:33,480 --> 00:03:38,357 its people face such enormous challenges that this technology can help solve. 63 00:03:38,440 --> 00:03:41,957 So, we try to walk the middle path between these two poles. 64 00:03:42,040 --> 00:03:44,197 For us, that means three principles. 65 00:03:44,280 --> 00:03:49,277 First, as AI becomes more capable, people must remain at the center of AI 66 00:03:49,360 --> 00:03:52,437 decision-making. Alignment is not an abstract research 67 00:03:52,520 --> 00:03:56,397 question. It is the work of ensuring that these systems reliably remain under 68 00:03:56,480 --> 00:03:59,757 human control, reflect human values, and help people 69 00:03:59,840 --> 00:04:02,157 guide the next stages of development. 70 00:04:02,240 --> 00:04:06,717 We are not trying to and must not automate human judgment or human values. 71 00:04:06,800 --> 00:04:10,877 Second, the benefits of scientific progress and economic growth must be by 72 00:04:10,960 --> 00:04:13,717 people and for people. AI should help researchers make 73 00:04:13,800 --> 00:04:17,117 discoveries, doctors treat patients, teachers teach students, entrepreneurs 74 00:04:17,200 --> 00:04:20,437 start companies, communities solve problems that felt out of reach, and 75 00:04:20,520 --> 00:04:25,437 much more. The continuous story of human progress is not the tools replace human 76 00:04:25,520 --> 00:04:29,157 agency, it is that the right tools make people more capable, and I believe this 77 00:04:29,240 --> 00:04:32,397 is going to go much further than we believe possible. 78 00:04:32,480 --> 00:04:36,157 Third, this technology must empower people individually. 79 00:04:36,240 --> 00:04:39,437 A great future will come from people realizing their potential and building 80 00:04:39,520 --> 00:04:43,757 value for themselves, their communities, their countries, and the broader world. 81 00:04:43,840 --> 00:04:46,837 We might be with the builders, one of the builders of this technology, but we 82 00:04:46,920 --> 00:04:48,597 are not the heroes of this story. 83 00:04:48,680 --> 00:04:50,237 Our role is to enable people all over 84 00:04:50,320 --> 00:04:56,077 the world, and to trust in the magic of humanity's skill and faculties. 85 00:04:56,160 --> 00:05:00,277 We now have models capable of discovering knowledge that we have never had before. 86 00:05:00,360 --> 00:05:02,837 Consider the progress we've seen in math. 87 00:05:02,920 --> 00:05:06,557 Three summers ago, our models were okay at grade school math. 88 00:05:06,640 --> 00:05:10,437 Two summers ago, they were pretty good at high school math competitions. 89 00:05:10,520 --> 00:05:13,997 Last summer, we reached gold a gold level medal in the most prestigious 90 00:05:14,080 --> 00:05:16,197 international math competition. 91 00:05:16,280 --> 00:05:19,317 And just a few weeks ago, this summer, one of our models solved one of the 92 00:05:19,400 --> 00:05:23,077 Millennium Prize problems, the Navier-Stokes equations. 93 00:05:23,160 --> 00:05:26,317 These equations are used for aircraft design, weather forecasting, and the 94 00:05:26,400 --> 00:05:29,197 study of blood flow. We are not solving math problems for 95 00:05:29,280 --> 00:05:33,117 their own sake, but to empower people to discover new knowledge, and to use it to 96 00:05:33,200 --> 00:05:37,237 improve health care, raise standards of livings of living, and expand everyone's 97 00:05:37,320 --> 00:05:41,757 potential around the world. There remains so much to be discovered. 98 00:05:41,840 --> 00:05:45,957 But it's very important that companies not substitute for the democratic process. 99 00:05:46,040 --> 00:05:49,597 If AI is to be democratic, the most important decisions cannot be made by 100 00:05:49,680 --> 00:05:54,317 labs in San Francisco alone. They must be shaped through democratic processes 101 00:05:54,400 --> 00:05:57,797 and by governments accountable to the people that they serve. 102 00:05:57,880 --> 00:06:01,717 At the international level, this will require cooperation. 103 00:06:01,800 --> 00:06:04,837 In our history, there have been times where countries who compete and don't 104 00:06:04,920 --> 00:06:08,477 always like each other very much still come together for shared interests and 105 00:06:08,560 --> 00:06:11,957 the collective good in the face of a powerful new technology. 106 00:06:12,040 --> 00:06:15,517 We believe this must be one of those times. 107 00:06:15,600 --> 00:06:18,797 We need a mechanism for complementary national and international frontier AI 108 00:06:18,880 --> 00:06:23,757 standards. Standards for measuring capabilities, assessing risks, 109 00:06:23,840 --> 00:06:27,077 determining whether safeguards are sufficient, and preserving meaningful 110 00:06:27,160 --> 00:06:30,837 human oversight as systems become more autonomous. 111 00:06:30,920 --> 00:06:35,357 We need common standards so countries can compare evidence, verify compliance, 112 00:06:35,440 --> 00:06:39,117 and have a shared language and understanding about what is happening. 113 00:06:39,200 --> 00:06:41,677 We need accurate and speedy incident reporting, 114 00:06:41,760 --> 00:06:45,477 classification and reporting protocols so the world can learn from failures 115 00:06:45,560 --> 00:06:48,117 before they become catastrophes. 116 00:06:48,200 --> 00:06:51,237 And we need secure channels among governments, critical infrastructure 117 00:06:51,320 --> 00:06:54,877 operators, and technical experts to share emerging vulnerabilities and new 118 00:06:54,960 --> 00:06:58,277 threats. Obviously, these standards should not 119 00:06:58,360 --> 00:07:02,117 lock in incumbents or favor one business model over another. 120 00:07:02,200 --> 00:07:05,637 They must support open and closed model developers, new entrants and established 121 00:07:05,720 --> 00:07:09,637 labs, national authority, and international cooperation. 122 00:07:09,720 --> 00:07:12,517 Each government should decide how to incorporate standards into its own legal 123 00:07:12,600 --> 00:07:16,717 system, but we will all be better off if we can agree on what good evidence, good 124 00:07:16,800 --> 00:07:21,157 safeguards, and good oversight look like on the global stage. 125 00:07:21,240 --> 00:07:25,037 We do not have all the answers, and we're very open to any better ideas. 126 00:07:25,120 --> 00:07:27,877 We're ready to work with you all and try our hardest to figure out the path 127 00:07:27,960 --> 00:07:31,877 forward. We are at a crossroads. 128 00:07:31,960 --> 00:07:36,197 One path leads to a world where power is concentrated, decisions are opaque, and 129 00:07:36,280 --> 00:07:38,997 people feel that the most important technology of their lifetime is 130 00:07:39,080 --> 00:07:43,397 something being done to them. The other path leads to a world where AI 131 00:07:43,480 --> 00:07:48,277 works for people because people, through democratic institutions, help decide how 132 00:07:48,360 --> 00:07:52,717 it should develop and guide it. Where scientific progress improves real lives, 133 00:07:52,800 --> 00:07:55,837 where more people can participate in the future, and where this technology 134 00:07:55,920 --> 00:07:59,437 continues to give individuals and communities greater power to live life 135 00:07:59,520 --> 00:08:02,597 as they choose. This is a future of realized potential, 136 00:08:02,680 --> 00:08:05,757 excitement, and wonder, and it is the future we believe is worth building. 137 00:08:05,840 --> 00:08:08,640 Thank you very much. 138 00:08:09,080 --> 00:08:11,957 >> I thank Mr. Altman for his briefing. 139 00:08:12,040 --> 00:08:15,960 I now give the floor to Mr. Dario Amodei. 140 00:08:16,840 --> 00:08:20,757 >> Good afternoon. I'd like to thank France for convening this meeting and the 141 00:08:20,840 --> 00:08:23,597 members of the council for inviting me to speak. 142 00:08:23,680 --> 00:08:28,077 AI is advancing very quickly. Four years ago, it could barely write a line of 143 00:08:28,160 --> 00:08:31,957 code or solve a high school math problem. Today, it writes most of the 144 00:08:32,040 --> 00:08:36,197 code at Anthropic and solves world-famous unsolved math problems. The 145 00:08:36,280 --> 00:08:38,237 trajectory only needs to continue for a 146 00:08:38,320 --> 00:08:40,877 tiny period longer, one or two years, 147 00:08:40,960 --> 00:08:43,277 maybe less, to reach what I've called a 148 00:08:43,360 --> 00:08:45,917 country of geniuses in the data center. 149 00:08:46,000 --> 00:08:50,637 This rapid pace of progress will produce incredible benefits for the world. Just 150 00:08:50,720 --> 00:08:54,197 today, Anthropic announced the Claude-led discovery of a new molecular 151 00:08:54,280 --> 00:08:58,717 machine. It's a preliminary finding, but it may constitute a new gene editing 152 00:08:58,800 --> 00:09:03,077 mechanism that could have applications in in gene therapy. Claude led the 153 00:09:03,160 --> 00:09:05,117 literature review, theorizing, and 154 00:09:05,200 --> 00:09:10,237 experimental design, and humans ran and verified the experiments in the lab. 155 00:09:10,320 --> 00:09:14,557 We believe this type of application will accelerate rapidly in the years ahead 156 00:09:14,640 --> 00:09:18,477 and that we can make profound progress in the near future on diseases that have 157 00:09:18,560 --> 00:09:21,197 plagued humanity for millennia. 158 00:09:21,280 --> 00:09:23,437 But these remarkable capabilities also 159 00:09:23,520 --> 00:09:25,997 come with risks that must be managed. 160 00:09:26,080 --> 00:09:30,837 I see two main categories of risk. The first is misuse of AI models, for 161 00:09:30,920 --> 00:09:35,437 example, misuse by bioterrorists to create biological weapons. The second 162 00:09:35,520 --> 00:09:39,517 risk is loss of control. Model capabilities accelerating beyond 163 00:09:39,600 --> 00:09:41,837 developers' ability to control them. 164 00:09:41,920 --> 00:09:46,437 These questions are global in scope and concern all of humanity. They are grave 165 00:09:46,520 --> 00:09:48,037 enough that I believe they are important 166 00:09:48,120 --> 00:09:50,357 questions for this council, not just for 167 00:09:50,440 --> 00:09:53,517 economic or ICT ministries. If managed 168 00:09:53,600 --> 00:09:58,277 poorly, I even believe that AI could be a risk to humanity as a whole. 169 00:09:58,360 --> 00:10:02,037 At Anthropic, we believe we have the best track record in the industry for 170 00:10:02,120 --> 00:10:06,117 preventing these risks, both operationally and in our research. We 171 00:10:06,200 --> 00:10:09,997 test every model for dangerous capabilities in areas like cybersecurity 172 00:10:10,080 --> 00:10:14,917 and biology. We put strong safeguards on our most capable models, and we publish 173 00:10:15,000 --> 00:10:19,357 our work in these areas. We have pioneered research in areas like 174 00:10:19,440 --> 00:10:23,037 mechanistic interpretability and alignment science that improve our 175 00:10:23,120 --> 00:10:26,077 ability to steer and control our models. 176 00:10:26,160 --> 00:10:30,717 We also have unparalleled transparency practices, releasing risk reports that 177 00:10:30,800 --> 00:10:32,797 run to hundreds of pages. But we 178 00:10:32,880 --> 00:10:35,397 understand that as AI models become more 179 00:10:35,480 --> 00:10:37,997 powerful, even more stringent standards 180 00:10:38,080 --> 00:10:40,357 of safety will be required. And we are 181 00:10:40,440 --> 00:10:43,677 always aiming to improve our safety and 182 00:10:43,760 --> 00:10:48,357 alignment practices, our abilities our ability to prevent misuse, and we are 183 00:10:48,440 --> 00:10:50,597 always striving to do better. We will 184 00:10:50,680 --> 00:10:53,477 slow down as much as necessary in order 185 00:10:53,560 --> 00:10:58,797 to make sure that every successive AI technology that we release is actually 186 00:10:58,880 --> 00:11:02,397 safe. But regardless of what we do, managing 187 00:11:02,480 --> 00:11:06,197 these risks is ultimately bigger than anyone company. And it has an 188 00:11:06,280 --> 00:11:09,077 industry-wide and global scale. 189 00:11:09,160 --> 00:11:11,197 On September 12th, I called for a 190 00:11:11,280 --> 00:11:13,077 collaborative effort to pace the 191 00:11:13,160 --> 00:11:16,197 frontier, not to stop AI progress, but 192 00:11:16,280 --> 00:11:18,357 to work together to make sure the 193 00:11:18,440 --> 00:11:20,597 industry and the world can proceed at a 194 00:11:20,680 --> 00:11:25,597 rate where we can be sure that safety keeps up with capability for everyone in 195 00:11:25,680 --> 00:11:29,837 the industry. This call had three parts. First, we 196 00:11:29,920 --> 00:11:32,357 committed to embed external evaluators 197 00:11:32,440 --> 00:11:34,357 inside Anthropic with employee-like 198 00:11:34,440 --> 00:11:37,437 access, similar to a food inspector. And 199 00:11:37,520 --> 00:11:42,357 And we recommended that other companies across the world do the same. Some have 200 00:11:42,440 --> 00:11:46,197 already agreed uh to adopt this measure. 201 00:11:46,280 --> 00:11:48,797 Second, we called for cooperation across 202 00:11:48,880 --> 00:11:53,357 the industry to set standards and modulate the pace of progress. And 203 00:11:53,440 --> 00:11:58,117 finally, we called for global cooperation across the world between 204 00:11:58,200 --> 00:12:02,957 governments to set international standards. It is this last area that 205 00:12:03,040 --> 00:12:04,837 most squarely falls within this 206 00:12:04,920 --> 00:12:07,637 council's remit. I want to offer three 207 00:12:07,720 --> 00:12:10,997 ideas in this regard. First, we should 208 00:12:11,080 --> 00:12:13,197 begin with narrow agreements that every 209 00:12:13,280 --> 00:12:15,957 member can support, such as a ban on 210 00:12:16,040 --> 00:12:19,077 using AI to make biological weapons or 211 00:12:19,160 --> 00:12:21,997 permitting your AI technologies to be 212 00:12:22,080 --> 00:12:25,837 used to make biological weapons. Second, 213 00:12:25,920 --> 00:12:27,597 we should build evaluation and 214 00:12:27,680 --> 00:12:30,677 verification systems that keep pace with 215 00:12:30,760 --> 00:12:33,037 AI development so that states can have 216 00:12:33,120 --> 00:12:34,917 visibility into frontier model 217 00:12:35,000 --> 00:12:37,877 capability and can verify each other's 218 00:12:37,960 --> 00:12:40,717 commitments. Third, we should establish 219 00:12:40,800 --> 00:12:43,277 common global standards for testing AI 220 00:12:43,360 --> 00:12:46,397 models for loss of control risks and 221 00:12:46,480 --> 00:12:49,597 misuse risks, and a notification system 222 00:12:49,680 --> 00:12:54,117 for AI incidents that are significant to global security. 223 00:12:54,200 --> 00:12:58,477 I believe that this is the most important global security issue facing 224 00:12:58,560 --> 00:13:03,357 the world today. For all the differences of the people in this room, we must put 225 00:13:03,440 --> 00:13:05,437 aside those differences in order to 226 00:13:05,520 --> 00:13:08,437 confront this global opportunity and 227 00:13:08,520 --> 00:13:13,437 global threat that is being presented to us at the same time. No leader, no 228 00:13:13,520 --> 00:13:16,437 company, and no nation can manage this alone. 229 00:13:16,520 --> 00:13:18,357 We commit to working with governments in 230 00:13:18,440 --> 00:13:21,037 this room on this urgent work. 231 00:13:21,120 --> 00:13:24,200 Thank you for your attention.