# SAM ALTMAN WARNS the WORLD about AI: "We should be AFRAID of THIS" *Diario AS* ยท https://www.youtube.com/watch?v=D73Dh-UPcEk ## Summary In this conversation at Dreamforce, Sam Altman discusses why AI risks have suddenly become an urgent international topic, pointing to how fast and capable the models have become. He identifies two major challenges: the potential for a loss of control accident and the concentration of too much power in AI companies, saying the world is right to be afraid. He argues that alignment, safety, and monitoring must stay ahead of capabilities, notes the Hugging Face incident triggered an industry-wide reset, and calls for a culture of accident reporting modeled on the FAA and NTSB. He closes by describing the technology as a sci-fi moment come true, with personal super assistants making people far faster and more creative. ## Key points - Sam Altman says the models have gotten so good so fast that people did not expect this progress. - He identifies two major challenges: a loss of control accident and too much power concentrated in AI companies. - Altman states the world is right to be afraid of this. - He says alignment, safety, and monitoring must always stay ahead of capabilities. - The Hugging Face incident triggered a real reset in his company and the whole industry. - He points to the FAA and NTSB culture of accident reporting as a model for making technology safer. - He describes talking to a personal super assistant as a sci-fi moment that came true. ## Notes ### Why Now? - Sam Altman says people have long discussed AI's potential downsides, but the moment it "escaped the bubble" came when the models got so good so fast. - He believes people did not expect the models to come this far this quickly, and the idea that these models could become more capable and smarter than people "finally happened." - He expresses confidence: "We will get it right." ### Two Big Challenges - The first challenge is the potential of a loss of control accident or some other serious thing going wrong. - The second is way too much power concentration, where companies developing AI could exert undue influence on the economy or push a worldview on people. - Altman says, "I think the world is right to be afraid of this." - He describes a narrow path that must be navigated with pragmatism, steadiness, and dependable, consistent decisions that the world can trust. ### Safety and Rigor - Because the models have gotten so good so fast, Altman argues alignment, monitoring, and security need a new level of rigor. - Development of capabilities must be paced so that alignment, safety, and monitoring are always ahead of capabilities. - The Hugging Face incident triggered a real reset, not just in his own company but across the whole industry. ### Learning from Accidents - Altman cares about building a great culture of accident reporting and learning. - He cites the FAA and NTSB as examples: airplanes became very safe partly because of a strong culture of accident reporting. - In other kinds of technology, he says society has not had the same response and has not improved as much. ### A Profound Change - Altman calls this one of the most profound changes to how we use technology that he has ever seen. - Sci-fi movies have hinted at this for a long time, but now "the sci-fi thing came true." - He describes talking to your personal super assistant and working 10 times faster, better, or more creatively than before. ## Transcript [00:08] Thanks for having me. >> Great to have you, Sam. >> It's interesting cuz people have been talking about the potential downsides or serious risks of AI for some time. And I think a lot of people have been saying, "Why is this the moment where it sort of escaped the bubble and is now all of a sudden an international topic of such uh intensity?" And I think a few things have happened. One, the models have gotten so good. And people for a long time looked at the models, maybe they looked at the rate of progress, but I don't think people really felt that the models were going to come this far this fast. We will get it right, by the way. I am very confident in our I think the biggest thing that has changed is [00:55] this thing that our industry has been talking about for a while, which is no, no, this technology, this approach is going to go very far. These models are in some ways going to become more capable, smarter than people. Um that finally happened. So, I think that was a big change to why now. When people say, you know, my probability of doom is X or Y or whatever, it hits in a different way. Technology to do. Somebody else wants to build like a hacking bot, okay. Um the world can decide what they think about that, but uh I think you also see um a real fear that some of these companies or some companies developing AI could get too much power and be able to sort of exert undue influence uh on the economy, push a worldview out on people. [01:40] Uh and I think the world is right to be afraid of this. So, we have these two big challenges. We have the potential of a loss of control accident or some other serious thing that could go wrong. Um we have the potential of way too much power concentration and people fear that those developing AI could exert their worldview. Um and there is a sort of narrow path we have to navigate with pragmatism, with you know, steadiness, with the ability for the the world to like trust that we will make dependable decisions, consistent decisions uh through this. And I think people are saying, well, you know, this now seems pretty important. Like let's let's get it right. I'm not going to get the days of the [02:23] week exactly right here, but something on like a Friday or a Saturday. These models have gotten so good so fast that we need to treat the alignment and monitoring and security with a new level of rigor. And we have to be willing to pace our development of capabilities such that capabilities always uh such that alignment, safety, and monitoring are always ahead of capabilities. And the Hugging Face incident triggered a real reset, not just in our own company, but I think the whole industry. >> You know, we're in the cyber area today, but maybe we move into another area. Is this a fair way to characterization, or am I or is this unfair to characterize it like this? >> What I really care about is that we have [03:04] got a great culture of accident reporting and learning. I think if we look at what happens with the FAA and the NTSB and how safe airplanes have gotten, um as I understand it from talking to people who have really studied, you know, new kinds of technology on the order of magnitude of airplanes, one of the things that has gone so well there is is the culture of accident reporting. You know, accidents are going to happen. We're going to learn as much as we can and react from each one. Um and if, you know, there's other kinds of technology where I don't think it's gone I don't think society's had the same response, and I don't think we have improved as much. to pull this thing together and present [03:46] you with the information to make a decision or to blame. I think this is one of the most profound changes to how we use technology that I've ever seen. I mean, sci-fi movies, as you mentioned, have been hinting at this or playing around with this for a long time, but this is one of those moments where it feels like you're in the movie. Like the sci-fi thing came true. You're talking to your personal super assistant and you are working 10 times faster or better or more creative than you could have before. >> Sam, you're invited back anytime you want to Dreamforce. >> Thank you very much. Thank you. >> Please thank Sam Altman.