Sam Altman on Donald Trump, Xi Jinping, and the Risk of Extinction (Part Two)

Intro and the Dots product

00:00from Vanity Fair. This is Fair Game. I'm Mark Aduchi. I sat down with Alman for two interviews in San Francisco. The first on September 11th and then again today, October 2nd. In those weeks, the story about AI seemed to turn upside down about half a dozen times. I should note before we begin that Condas, Vanity Fair's parent company, has a content licensing agreement with OpenAI. Both interviews with Altman are live now. This is [music] part two.

00:31Sam, it's good to see you again. You, too. >> Uh, we sat in this room three weeks ago and so much has happened since. This past week, you introduced the world to your new product, Dots. You described it as like an AI helper that always has your back, inspired by the cool versions of what we watched in movies growing up. Sam, what's a Dot? >> Before I get to literally what it is, this this is the product I have been waiting for. >> Um, I like this. I think more than any product we've ever launched it. The fact that it gets to know me as I work. The fact that it's just kind of like always working and taking on the stuff that I didn't want to do. I I noticed that and I'm sure many other people have this too

01:12that I like let the tasks I most don't want to deal with just pile up and pile up and it becomes this like gravity well of dread and it's just sitting there. And the fact that my dot is going through my email, going through my like to-do list of just stuff that is not pleasant for me to do and where I don't add much value and it's just like knocking that out with undefeable spirit and energy uh is amazing and I and I really feel like I have this thing working for me all >> it somehow feels assistant is the mental model that I had in mind when we started working on the And I think that is probably how people still are talking about it, but it feels much more like a sort of extension of me. >> Is it just an avatar of you? Like is

02:00that a good way of thinking about it that it it knows [clears throat] what you want to say to those emails and is getting getting back to them or >> No, I I like I I think that we've I have figured out this like good workflow where uh I called my dot yesterday while I had some time and I was just like here's a set of ideas. Went off and did a bunch of work. By the time I got back to my computer, there was a bunch of things where my doc had sent me a message in Slack being like, uh, I, you know, did all this work. These it's bad ideas for these reasons. I also found this other stuff on your computer. I proactively talked to three people on the team and I was like, all right, let's do these things and let's keep

02:34going. And it was just a completely new way of trying to get a new idea towards reality. I'm curious like for that to be effective it must have access to everything right because I remember we're almost the same age when I got to college I immediately got on Facebook and was like >> I think we were a little naive we just everyone was like free-for-all you know >> it was a freeall >> and no one even thought about data how does living through Facebook impact how you think about privacy with dots particularly we have to be extremely extremely um rigorous about privacy for this to work. But I I was actually thinking about this the other day. I I am kind of sad about where technology has evolved to in many ways

Privacy and technology's attention cost

03:27because I feel like it is constantly taking my attention. You know, my phone I finally turned off almost all notifications on my phone. It's way better. But it's like a crazy way to live your life with your phone just buzzing at you non-stop. My hope for the world is that we're going to automate the bad parts of life and then give people more effort, ability, time to do the good parts and express their [snorts] creative spirit or, you know, push on something they care about or help their community. And in my own use, I found out that one of the things I value the most, no surprise to anyone, is these small moments of human connection. And if it's like my dot is messaging yours and like that feels really just bad like

Delaying Astra 6.1 over safety

04:10I I want to send you the text. So last time we spoke here you were really eager to talk about Astra the latest version of chat BT but last week you delayed the latest release of Astra 6.1 after the model failed to meet safety standards. How did you come to that decision and what was Astra doing that means it's not ready for the public? these these models are in the like serious capable seriously capable era now. And so the kind of decisions where I think a year ago we could have very reasonably said, you know, just put this out in the world. It's maybe a little unsafe, but nothing that bad is going to happen and people, you know, we want people to benefit from it. Is it's different now. And and I think that

04:57as the capabilities continue to increase, that the bar we will hold for ourselves on safety is going to go higher and higher. Now, of course, you could say, and I think some of our colleagues in the industry would say, that this stuff is just way too dangerous to release. Only a couple of companies should have it or one company or one person should have it. Uh, and that is also not right. So, there are kind of two major ways that this could go wrong. And I think the Astra 6.1 is an interesting thing to study here. You can have a big safety accident eventually. one that leads to like a real loss of control where AI, you know, escapes and it's out doing wrecking havoc on the internet and society can't

05:36really get control back. You know, AI is so much more capable than us and so smart that it's doing things that we don't understand and we don't know how to stop it. Like this is the thing from the sci-fi movies. On the other hand, you could have AI have way too much power concentration, lead to way too much power concentration in a small number of companies or people or maybe like one country and if you don't release at all, you can fall off that way. And again, I think there's a lot of people just like we just want the control here. We have to navigate this like pragmatic middle path down down the center of this and 6.1 Astra the current model we had ready to go I think would have tipped too much

06:19towards the accident risk. So what we're doing instead is pushing towards a model that we will still be able to distribute very broadly and we're committed to do that even though it's going to take a little longer but much we can make much more confident claims about the alignment. So then this is a good example of self-regulation of saying we're not going to release this product because we've determined for XYZ reasons not yet it's not yet safe. Um, which brings us to this week. President Trump and AI executives, including Open AI's President Greg Brockman, signed an accord on super intelligence. It essentially is an agreement for the AI companies to self-regulate. As I read it, I mean, it sounds like Trump wants

The Trump accord on superintelligence

07:00you guys to police yourselves. You told me 3 weeks ago, we need our governments to step up. And then went even further the next week when you went to the UN. You said if AI is to be democratic, the most important decisions cannot be made by labs in San Francisco alone. Does this new accord fall short in your view? I think it's a good start. I think it's like much better than nothing. Um, I remain of the belief that we are going to need more over time. Um, but for like the models of kind of useless this thing, it's a piece of paper that has a typo on it. [laughter] Um I don't think that's quite fair. I mean I mean yes in some sense it has no real power. On the other hand I think there is always like social force behind

07:46these things and um it's a gesture people will point to it and I think it'll have some power inside in between these companies. What do you think is the next step that you would like to see? I think with all of these questions about next steps, we are in the realm of powerful enough models that it is difficult to predict the specific things that will go wrong in what order. Um, and I think politics and policy is going to be very reactive to the specific things that go wrong. So >> if the next big thing that goes wrong is like someone makes a novel pathogen, I think that will have a very different response than if the next thing that goes wrong is more cyber security or people feeling like, you know,

08:35businesses can't succeed because the model companies stop giving access. So I I think policy makers still have not really felt the exponential that we're on. And I would guess the world in general will be caught off guard by the next thing that goes really wrong. But what's your stance on it? like what's your can you clarify you or your company's position on government regulation because you have said you've preached almost like regulate us >> sometimes the actions of the company whether it's lobbying in the EU or California or >> Greg Brockman's political donations don't always seem to be aligned with that idea. So can you clarify your stance on regulation? Yeah, I mean this is obviously not a binary thing. It's not

Regulation stance and open source risk

09:25like I I I think it's like really easy for people to say either you are for all regulation or you're against all regulation. And I don't agree with you in that framing. Um I think it's very reasonable for us to say like given what we see on the technology with the technology and given that we are trying to balance between these two really important things >> um we're going to be in the gray area in between here. there's going to like we really want people to have the technology. We really want the US to win. We really want this to be something that all people get to use and control and have a voice in the future. Not just like okay fine AI is giving you a cure for cancer, but really get to like use

10:03to shape where the future's going to go. And we are willing to accept some things going wrong to make sure that this is like a widely held and used and democratic technology. What do you mean by some things going wrong? Um, I think for example, I support open source models. Uh, there is a coming title wave of cyber security problems that are going to come from these open source models. Now, we're trying to help the world be ready for that by saying we've got a little bit of a lead. We want to help you defend. Um, but there are people in the industry who say we should ban open source because otherwise a bunch of systems are going to be hacked. And I don't think that price of liberty is worth it. And we're going to

10:49as a society, I think have to just accept some fairly severe cyber incidents from open models in exchange for the liberty that comes with that. >> What's the what's the limit of that that we accept? I mean that is the political process like how we determine what >> you know safety is always like negotiated between stakeholders like airplanes are generally thought of as very extremely safe things still sometimes crash so like the limits on how much we accept for the benefits like that is not something that I think highly regulated >> that's that's an industry with extreme regulation >> totally in the past week or so Jensen Hang and Bill Gates have come out as kind of very different uh with different approaches to this. Hang

11:40said recently in the New York Times, "The right answer is don't ship products until they're in control. It's really quite simple." Pacing being self-regulated. Whereas Bill Gates said this is the most dangerous thing that humans have ever gone near. And in other areas, do you just release drugs without the FDA? He brings up aviation. So if they're kind of opposites in those views and you're saying it's a gray area in the middle, can you define that a little bit more clearly? Like what are the principles for safety and regulation that you stick with? >> Here are some things that I think have gone well in other industries we should learn from. Um I think one principle for learning how to make a new technology

12:20safe is very clear [snorts] and transparent and kind of like extract maximum learning out of accident reporting. So, I think the NTSB, we talked about airplanes and cars. These have gotten remarkably safe and I think there there's been an amazing culture of accident reporting or incident reporting, I don't know what they call it, one of those two. Um, that has helped do that. So, I think that's a good thing. Um, another is yeah, I think like pre-release standards and a liability framework. Um, we earlier did uh testing before we released a model but not until after we finished training. We're now saying that actually even during training things can now go wrong. This is like a new stage where

Greg Brockman's donations

12:59the models are so smart that they can get out of a sandbox, hack into a system, things like that. So, also in the news this week, it was reported that OpenAI President Greg Brockman backed out of a $25 million commitment to leading the future super PAC who that is generally speaking opposes strong AI AI regulation. uh he and his wife had previously already donated$2 million and they had donated $25 million to President Trump's re-election campaign. Did you personally ask Greg to withdraw his pledge and how much daylight is there between your views on regulation and Greg's? >> Can I finish my answer the other question because it actually touches on this one but then we'll answer it. Um what I was going to say in terms of like

13:43where we are on the answer for regulation it is uh you don't want to overregulate to a degree that people don't get the benefits and that new entrance and you know random kids with ideas cannot build all the stuff we're going to do in the future. And part of why I think the internet was so magical and why the US did so well is that we accepted some things going a little bit wrong to give people the freedom and liberty and you know that was that was good but unlike other kinds of technology not all like you know we've seen some extraordinarily powerful technologies in our history the potential downside here may not be the kind of thing normally goes wrong that we can correct we have a legal system for um but that something

14:29goes super wrong like in the extreme extreme case back to what we talked about of you know we we lose control of the future to AI which is horrific. So my stance on regulation is pro- innovation, let some things go normally wrong and trust that we have a normal process for doing this. But where this technology is extraordinarily powerful and fundamentally new, have guardrails to ensure that all of society is kept reasonably safe and we don't have we don't kind of fall off that path in either direction in what could be a very difficult or unreoverable way. So, so do you think Greg is on the same page with that? On to that. I was going to say I think Greg is on the same page as that. I don't want to speak for him. You'd

15:14have to ask him, but I think he would give you a very similar take. I did not ask him to donate or not, you know, stop donating. He's like can make his own person >> because when you look at it holistically, it's like you're saying something, the company's saying something, but then there's lobbying efforts or executive donations that seem to speak to a different thing. So, >> I don't think that's a fair cut. Like I I again I don't want to like you should ask Greg. I don't have to speak for him on this but I don't I don't think Greg would say he's like against all AI regulation or has any of his contributions have ever expressed that view. Okay. Let's talk about the Chinese state visit because when we were in this

Xi Jinping state visit

15:49room you told me that you were looking forward to Xiinping meeting with President Trump at the White House. What you said was these two leaders could make tremendous progress and then you attended the state dinner. It didn't feel like there was tremendous progress from that visit, but you know, a lot happens behind closed doors. What was the biggest takeaway from that night that the public has not really heard about yet? >> I don't really know either. Um, I was I mean, it was like a nice dinner, but uh the substant I was not in the room for the substantive conversations. >> Okay. At that state dinner, Trump offered guests a tour of the ballroom. Yeah. >> Did you take him up on the offer? >> Very briefly.

16:25>> What was it like? >> It's a construction site. It's big. Anything more? It's it's still like a very raw construction site. >> Okay, because here's how that dinner looks, right? I mean, you have the leaders of this industry who are some of the richest people in the world with the two most powerful men in the world arguably, and Trump gets up there and doesn't say one thing about AI, which is what the whole world is hoping he will lead on or that someone will lead on. It was an opportunity for leadership that he seems to have just skirted to talk about this ballroom. It's the elephant in the ballroom. [snorts] >> I I heard they did talk about it earlier. I mean, you know, like I think as I have never hosted another president

17:13first day dinner obviously, but when I've done the business version of this, we will often do difficult work all day and then by the time dinner comes, people are like very tired or anything like ahead of time or after the CEOs. Um, but I know that the two presidents spent a lot of time. >> Okay. I worry about him representing an industry that he seems to not quite understand. You have thought about the subject of AI for longer than almost anybody. Is the extinction risk from AI overhyped? >> Well, the fact that there's any at all, the fact that it is like a nonzero zero, >> uh, I think is something that we all should take incredibly seriously. And part of why we formed OpenAI is the mission to make sure that doesn't

Extinction risk and NYT report

18:02happen. >> But you know that there's some people are saying that that it's very canny marketing or that it contributes to a bubble or that it's some collaborative effort to block newcomers to the AI market. Is that just cynical? Is that just um analysis on analysis? >> I assume some people are using it as a way to justify power seeking power concentration. Um, but that doesn't mean there's not a real risk here. The thing I don't like about the conversation is when people say like this is so risky, there's this, you know, whatever percent chance we better hurdle towards it as fast as we can. Like I think the constructive thing here is yes, there are risks. We are all here to mitigate those risks

18:47to let people get the benefits. But before we do that to ensure that there's not going to be a really bad thing and we need to talk with the world and kind of collaborate on how to make sure that that happens and people understand how that's happening. Um, we also need to work to avoid using this as a maybe not maybe marketing is too negative, but like a way to justify power concentration and society has got to like decide on the particulars of the trade-offs here. There was a report this week in the New York Times claiming that your company ignored warnings from employees to top executives months before the Hugging Pace incident. Do you refute that or is that >> I'm not familiar with that specific

19:30report? I mean, look, you can probably you can complain to me like, "Oh, you're delaying a model." Or you can complain to me like, "Oh, you're like, you know, ignoring warning signs or many other things and like the answer is >> we're going to make I'm going to make many individual decisions you clearly won't like and you'll say, "Oh, you're like marketing and you're scaring people or oh, you're delaying a release or oh, you're like trying to, you know, you're ignoring warning signs and you need something to write about every day." I get it. I really do. My goal is to navigate this path and I won't get every decision right. Um, but making sure that we let the whole world have agency and authority and

20:12the ability to self-determine their future and make their own lives and their communities lives better while avoiding a and you know not seeing the power concentrate in my company or any other company while avoiding safety things. And so I'm sure you can always find someone who's going to say like, "Ah, Sam shouldn't have made this decision or that decision." You can come try making them all. They're not so easy. Um, but if you look at the trajectory of what we have done over the last many years we've been offering this. Um, it's imperfect, but I think remarkably good relative to what I feared it could have been. I have two more questions. One is that when we were here last, I asked you if there were any politicians

Washington and legacy

20:50whose positions on AI you approved. If the Democrats do take the House, will that help or hinder OpenAI's goals towards regulation? >> The main thing I have learned about Washington in the last few years that I've, you know, kind of had a more active role is that I cannot predict how that place works at all. I remember when I first testified in front of the Senate in like was that [clears throat] 23 23. I was so convinced and the people seemed so genuine. We're going to get this done in 2 weeks. It's going to be non-controversial. This is just like basic common sense. It's going to be done. We won't need to see you again. I was like great. That's how it works. This is these is, you know, and then

21:34nothing has happened. And I was so convinced then it was going to happen. >> Sam, when the history of AI is written, who do you want to be? What role do you want to have played? Let me put it this way. Elon Musk is like the Mad King Nero character. I think Jensen Hong will be the guy who was selling picks and shovels during the gold rush. If you had to summarize who you would like to be when the history of AI is written, Sam, who do you specifically want to be? >> First of all, I never think this way. And I think it's probably like a super toxic way to live your life. Um like I I try to just stay really focused on my work and you know there's so much still to to do and I don't want to get like too much

22:25main character energy here. I think that's super bad. um that's probably healthy and yet I think that people would benefit from understanding your own personal mission. >> Yeah. I like the thing that has driven me through all of this even especially the hard days is uh I have such a belief that although there are challenges although there's reason for unease although there are things we have to get right as we navigate this path that putting this technology in the hands of everyone and giving people a legitimate new superpower relative to I think what any of us could imagine mentioned a few years ago will do tremendously more good than bad. And a big part of my own motivation to start and keep working in OpenAI is that I

23:21think there are many people many of the traditional parts of the community in the Bay Area that were thinking about AI that really believed only a small priesthood should get access to AI and that maybe that that group would like, you know, they'd be benevolent dictators. they would decide how the rest of us got to use it. I think there are other people pursuing AI that just want like more power for themselves or their company. Like maybe they're newer to this, but they're just like this is, you know, this is how we're going to win. And and and what I would really like if the history books ever write anything about me at all be great if they don't is just like the work that OpenAI did really made sure

24:02that AI was like a fundamental thing that every person got hugely empowered with. It was not something happened to them. It was something that people were like given was like a new fire or electricity or whatever. And it led to this incredibly great new society that people built for each other. Not that like a small group of people worshiping the machine god decided to like bequeath on all of us. Prometheus closer to that than other things you could come up with for sure. Okay, Sam, thank you very much. Thank you.