Dario Amodei, Sam Altman on AI: Amid pace of progress by models, 'the world is right to be afraid'

Why AI fear went mainstream

00:08Thanks for having me. >> Great to have you, Sam. >> It's interesting because people have been talking about the potential downsides uh or serious risks of AI for some time. And I think a lot of people have been saying why is this the moment where it sort of escaped the bubble and is now all of a sudden an international topic of such uh intensity and I think a few things have happened. One the models have gotten so good and people for a long time looked at the models maybe they looked at the rate of progress but I don't think people really felt that the models were going to come this far this fast. We will get it right. By the way, I am very confident in our I think the biggest thing that has changed is this

Two big challenges: control and power

00:55thing that our industry has been talking about for a while, which is no no this technology, this approach is going to go very far. These models are in some ways going to become more capable, smarter than people. Um that finally happened. So I think that was a big change to why now when people say you know my probability of doom is x or y or whatever it hits in a different way technology to do somebody else wants to build like a hacking bot okay um the world can decide what they think about that but I think you also see um a real fear that some of these companies or some companies developing AI could get too much power and be able to sort of exert undue influence uh on the economy push worldview out on people.

01:40Uh, and I think the world is right to be afraid of this. So, we have these two big challenges. We have the potential of a loss of control accident or some other serious thing that could go wrong. Um, we have the potential of way too much power concentration and people fear that those developing AI could exert their worldview. Um and there is a sort of narrow path we have to navigate with pragmatism with you know steadiness with the ability for the the world to like trust that we will make dependable decisions consistent decisions uh through this and I think people are saying well you know this now seems pretty important like let's let's get it right I'm not going to get the days of the week exactly right here but something on

Safety ahead of capabilities

02:25like a Friday or a Saturday these models have gotten so good so fast that we need to treat the alignment and monitoring and security with a new level of rigor. And we have to be willing to pace our development of capabilities such that capabilities always such that alignment, safety and monitoring are always ahead of capabilities and the hugging face incident triggered a real reset not just in our own company but I think the whole industry. >> You know we're in the cyber area today but maybe we move into another area. Is this a fair way to characterization or am I or is this unfair to characterize it like this? >> What I really care about is that we have got a great culture of accident reporting and learning. I think if we

Accident reporting culture

03:09look at what happens with the FAA and the NTSB and how safe airplanes have gotten. Um, as I understand it from talking to people who have really studied, you know, new kinds of technology on the order of magnitude of airplanes, one of the things that has gone so well there is is the culture of accident reporting. You know, accidents are going to happen. We're going to learn as much as we can and react from each one. Um, and if you know there's other kinds of technology where I don't think it's gone, I don't think society's had the same response and I don't think we have improved as much to pull this thing together and present you with the information to make a decision or to bl. I think this is one of the most profound

The sci-fi moment arrived

03:52changes to how we use technology that I've ever seen. I mean, sci-fi movies, as you mentioned, have been hinting at this or playing around with this for a long time, but this is one of those moments where it feels like you're in the movie. Like, the sci-fi thing came true. You're talking to your personal super assistant and you are working 10 times faster or better or more creatively than you could have before. >> Sam, you're invited back anytime you want to Dream Force. >> Hold on. I'm going to hold you back. Thank you. >> Show me what the future was and it's been an incredible experience. show it to you for the first time. The customer at the end of it said for the first time, for the first time they saw that

04:33trop value. Now I want to just tell you we are here to do many things with you. Now as you know we are here to excite you coming to San Francisco and thank you for coming to this incredible dream force. >> We are going to have an amazing day. We're going to have an amazing conference. And let me just tell you this, we have never been more excited about a Dream Force in our history. This is going to be absolutely incredible. [applause] And here we have the CEO of Enthropic right here, Daario. Hello, Dario. How are you? >> Hey, >> great to have you. Fantastic. [applause] Great job, Daario. you're doing an amazing job leading the AI industry, guiding it all right from our Slack headquarters building. We watch you

Dario Amodei's three-step plan

05:28every day. Is to say, hey, first of all, let's look let's look at our own record, right? We may not have had this big high-profile incident, but I'm sure we're not perfect. Let's look at everything and we can always be better. Let's make our practices better. Um, let's recommmit transparency. Let's invest more in safety. Then let's organize the rest of the industry and say, what can we do to set standards for everyone? what can we do to make everyone's standards better? And then finally, there should be an international component to it. And and that's essentially the the three-step plan that I laid out that I laid out in the essay. The the first of which we committed to and the the others of which

Scaling laws and fast growth

06:06we're going to have a dialogue with the rest of the industry about. And I think I think that's the way to lead the industry forward to set an example to say that everyone can always be better. you know it's it's been the pace of the progress not just of the technology but of the economic side of it. So we you know I formulated the scaling laws along with some of my co-founders which tell you that AI models get smarter and smarter the more compute you throw into them. And so we had a prediction of just how capable these models would be. But I think what we didn't appreciate is that it would lead to these companies growing so fast to all these incredible, you know, products that we're working together on and you've been talking

06:49about today and and how quickly they would become central to everything that's going on in the world, right? >> Say, you know, yes, we're have to keep our eye on the pace of AI. You know, it's been amazing what's happened over 10 years. No one could have written the story. When I look at the technology versus how much it has diffused, even if we freeze the technology in place, which we're not doing, pacing the frontier is not not doing that at all. The technology will keep getting better. But just suppose we freezed it in place. Even if that were the case, we're making use of maybe only five or 10% of what the possible value of the technology is. >> Congratulations. Well done. >> We're going to do incredible things

07:32together. You're doing a great job. Thanks for everything you're doing for the whole industry. We're so grateful to you. >> Thank you for having me. >> Really thrilled. Thank you so much. >> All right. >> And thank you for coming to Dreamforce. >> Please thank the anthropic team. [applause] >> Jensen. >> WOOHOO. [cheering] [applause] >> Great to have you, Jensen. Thank you for being here. >> We all have to learn how to walk and talk at the same time. How are you guys doing this >> on our board who I think was at Nvidia for a while as well. >> That's right. Love David Kirk. >> And now give us your vision of the future. You know, where are we going? Because of course you've got these critical components. You've got well

Jensen Huang: safety as engineering

08:18safety is paramount in a lot of ways. It's job one. However, safety is an engineering problem. We're developing software after all. We're developing computing systems after all. It's complicated computing system but it's ultimately a computing system and so so the first thing is to make sure that we create the test environments for testing these complex systems and those test environments have to be built in a good way built in a safe way and then the second thing is we have to test these products if we're if we're not confident about the safety of the products like all companies like you and I any any com all the companies here if you build a product or a service and you're confident in its functionality, capability, or safety,

09:03then don't release it. And so, that's a very obvious thing to do. You pace yourself until you are confident you're releasing something that the market would appreciate. The market forces are already there. We don't need any new laws. We don't need new regulations. Um, we just need uh companies to decide that when it's, you know, run as fast as they can. I think innovation speed and safe products are not it's a false choice. You could definitely have both at the same time. So run as fast as you can, but if you feel at any given point in time the company's out of control or or the product's not going to be safe, you know, take a pause and make sure you get it righties and transform them. The technology can partner with them to do

Regulation debate

09:47this in incredible new ways. That is what I'm really Jensen. We could not be more grateful to you for coming to Dreamforce being >> being a part of this. >> Thank you. >> We are so Please thank Jensen Wall. >> Thank you. Thank [cheering] you. Thank you. Thank you. [applause] Thank you, Mark. >> Thank you, Jensen. >> Dario and pacing and all the rest of it, but I think my colleagues will handle that one. So since then you've done a great deal to bang the lid on the coffin of um sus the suspocalypse >> the ones which are still in the PC phase number one and number two. >> Yeah I thought it was super interesting obviously we have architected the uh uh keynote before this week as you know and so we definitely had Daario and Jensen.

10:33I think that we heard both sides of the pendulum. You know, in my view, technology is never good or bad. It's what you're doing with the technology that matters. And I think that that remains the key thing. And um you know, I think that these companies obviously have to be responsible for what they're building. That is going to be the the key to it being them being ethical companies. Committed to that. We've now given more than $130 million to this. I know it's unusual for a, you know, company to have that exposure. I think that, you know, you're going to find different opinions right now. We're in a very fluid moment in AI. And I think that as we kind of start to get our head around what are

11:18the capability of the models, who can do what and also obviously there's political orientations for everybody's where everybody is. So, um I think that we already have regulations around the models, right? We have laws and I think that companies have to be um you know uh adherent to those laws and regulations. Will those laws and regulations continue to adapt and evolve and move forward? Of course. So that is I think just at the moment we are currently in the industry. We're in a highly evolutionary point and I think that these things will change you know as they're needed. >> It's really up to them. I I think we're doing all right with the investment and we couldn't be more thrilled to have you

12:01at Dream Force. Thank you everybody. Thank [applause] you everybody.