WEBVTT 00:00:03.840 --> 00:00:07.757 Welcome everybody and welcome to those of you joining us on live stream um to 00:00:07.840 --> 00:00:09.677 this conversation that I have to say I 00:00:09.760 --> 00:00:11.997 have been looking forward to for months. 00:00:12.080 --> 00:00:14.797 Uh I had was lucky enough to ch to 00:00:14.880 --> 00:00:17.037 moderate a conversation between Darede 00:00:17.120 --> 00:00:22.157 and Deis Hassabos last year in Paris. Um which I'm afraid got most attention for 00:00:22.240 --> 00:00:26.477 the fact that you two were squashed on a very small love seat while I sat on an 00:00:26.560 --> 00:00:30.477 enormous sofa which was probably my screw up. But I said at that point that 00:00:30.560 --> 00:00:33.757 this was for me like, you know, chairing a conversation between the Beatles and 00:00:33.840 --> 00:00:37.917 the Rolling Stones. And you have not had a conversation on stage since. So this 00:00:38.000 --> 00:00:42.077 is, you know, the sequel, the the the, you know, the bands get together again. 00:00:42.160 --> 00:00:44.557 I'm delighted. You need no introduction. 00:00:44.640 --> 00:00:49.037 Uh the title of our conversation is the day after AGI, which I think is perhaps 00:00:49.120 --> 00:00:51.757 slightly getting ahead of ourselves because we should probably talk about 00:00:51.840 --> 00:00:55.677 how quickly and easily we will get there. And I want to do a bit of a sort 00:00:55.760 --> 00:00:58.717 of update on that and then talk about the consequences. So firstly on the 00:00:58.800 --> 00:01:03.437 timeline Dario you last year in Paris said we'll have a model that can do 00:01:03.520 --> 00:01:05.277 everything a human could do at the level 00:01:05.360 --> 00:01:07.357 of a Nobel laureate across many fields 00:01:07.440 --> 00:01:12.397 by 2627. We're in 26. Uh do you still stand by that timeline? 00:01:12.480 --> 00:01:15.597 >> So you know it's always hard to know exactly when something will happen but 00:01:15.680 --> 00:01:17.597 but I don't I don't think that's going 00:01:17.680 --> 00:01:20.237 to turn out to be that far off. So um 00:01:20.320 --> 00:01:25.357 you know the the the mechanism whereby imagined it would happen is that we 00:01:25.440 --> 00:01:29.437 would make models that were good at coding and good at AI research and we 00:01:29.520 --> 00:01:33.837 would use that to produce the next generation of model and speed it up to 00:01:33.920 --> 00:01:36.317 create a loop that would that would uh 00:01:36.400 --> 00:01:38.477 increase the speed of model development. 00:01:38.560 --> 00:01:40.957 We are now in terms of you know the 00:01:41.040 --> 00:01:45.517 models that write code I have engineers within anthropic who say I don't write 00:01:45.600 --> 00:01:47.757 any code anymore. I just I just let the 00:01:47.840 --> 00:01:50.317 model write the code. I edit it. I do 00:01:50.400 --> 00:01:52.957 the things around it. I think I don't 00:01:53.040 --> 00:01:55.917 know. We might be six to 12 months away 00:01:56.000 --> 00:01:58.397 from when the model is doing most maybe 00:01:58.480 --> 00:02:01.917 all of what sues do end to end. And then 00:02:02.000 --> 00:02:06.797 it's a question of how fast does that loop close. Not every part of that loop 00:02:06.880 --> 00:02:10.397 is something that can be sped up by AI, right? There's like chips, there's 00:02:10.480 --> 00:02:14.637 manufacturer of chips, there's training time for the model. So it's, you know, I 00:02:14.720 --> 00:02:16.077 I think there's a lot of uncertainty. 00:02:16.160 --> 00:02:21.677 It's easy to see how this could take a few years. I don't I I it's very hard 00:02:21.760 --> 00:02:25.997 for me to see how it could take longer than that. Um but if if I had to guess, 00:02:26.080 --> 00:02:29.997 I would guess that this goes faster than people imagine. And that that key 00:02:30.080 --> 00:02:32.877 element of code and increasingly 00:02:32.960 --> 00:02:36.317 research going faster than we imagine. 00:02:36.400 --> 00:02:40.237 That's going to be the key driver. It's it's really hard to predict again how 00:02:40.320 --> 00:02:43.997 much that exponential is going to speed us up, but but something fast is going 00:02:44.080 --> 00:02:48.317 to happen. So you demis were a little more cautious last year. You said a 50% 00:02:48.400 --> 00:02:52.397 chance of a system that can exhibit all the cognitive capabilities humans can by 00:02:52.480 --> 00:02:55.117 the end of the decade. Um clearly in 00:02:55.200 --> 00:02:59.757 coding as Dario says it's been remarkable. What is your sense of do you 00:02:59.840 --> 00:03:02.637 stand by your prediction and what's changed in the past year? 00:03:02.720 --> 00:03:06.317 >> Yeah, look I I I I think I'm still on the same kind of timeline. And I think 00:03:06.400 --> 00:03:08.237 there has been remarkable progress. But 00:03:08.320 --> 00:03:11.917 I think some areas of uh uh um kind of 00:03:12.000 --> 00:03:16.477 engineering work, coding or so you could say mathematics are a little bit easier 00:03:16.560 --> 00:03:20.237 to see how they would be automated partly because they're verifiable what 00:03:20.320 --> 00:03:24.317 the output is. Um some areas of natural science are much harder to do than that. 00:03:24.400 --> 00:03:27.677 You won't necessarily know if the chemical compound you've built or this 00:03:27.760 --> 00:03:31.117 prediction about physics is correct. It may be you may have to test it 00:03:31.200 --> 00:03:35.357 experimentally and that will all take longer. So uh I also think there are 00:03:35.440 --> 00:03:37.837 some missing capabilities at the moment 00:03:37.920 --> 00:03:42.477 uh in terms of like not just solving existing conjectures uh or existing 00:03:42.560 --> 00:03:46.157 problems but actually coming up with the question in the first place or coming up 00:03:46.240 --> 00:03:49.597 with the theory or the hypothesis. I think that's much much harder and I 00:03:49.680 --> 00:03:52.957 think that's the highest level of scientific creativity and it's not 00:03:53.040 --> 00:03:55.837 clear. I think we will have those systems. I don't think it's impossible 00:03:55.920 --> 00:03:59.837 but I think there may be one or two missing ingredients. Um, it remains to 00:03:59.920 --> 00:04:02.957 be seen how, you know, first of all, can this self-improvement loop that we're 00:04:03.040 --> 00:04:06.397 all working on actually close without a human in the loop. I think there are 00:04:06.480 --> 00:04:09.357 also risks to that to that kind of system, by the way, which we should 00:04:09.440 --> 00:04:13.677 discuss and I'm sure we will. But the the but but that could speed things up 00:04:13.760 --> 00:04:15.277 if that kind of system does work. 00:04:15.360 --> 00:04:17.997 >> We'll get to the risks in a minute. But one other change I think of the past 00:04:18.080 --> 00:04:21.917 year has been a kind of change in the pecking order of the race, if you will. 00:04:22.000 --> 00:04:25.597 This time a year ago, we just had the deepseek moment and everyone was 00:04:25.680 --> 00:04:29.677 incredibly excited about what happened there and there was still a sense, you 00:04:29.760 --> 00:04:35.037 know, that Google Deep Mind was kind of lagging open AI. I would say that now uh 00:04:35.120 --> 00:04:38.157 it's looking quite different. I mean, they've declared code red, right? Um 00:04:38.240 --> 00:04:40.397 it's been quite a quite a year. So, talk 00:04:40.480 --> 00:04:45.437 me through what specifically you've been surprised by and how well you've done 00:04:45.520 --> 00:04:48.477 this year and whether you think and then I'm going to ask you about the lineup. 00:04:48.560 --> 00:04:52.957 Well, look, I I think we were I was always very confident we uh would get 00:04:53.040 --> 00:04:56.557 back to sort of the top of the the leaderboards and and the soda type of 00:04:56.640 --> 00:05:00.717 models across the board because I think we've always had like the deepest and 00:05:00.800 --> 00:05:04.397 broadest research bench and it was about kind of marshalling that all together 00:05:04.480 --> 00:05:08.477 and um getting the intensity and focus and the kind of startup mentality back 00:05:08.560 --> 00:05:10.557 to the whole organization and it's been 00:05:10.640 --> 00:05:15.677 a a lot of work and um but I think we're and we're still a lot of work to do um 00:05:15.760 --> 00:05:19.837 but I think you can start seeing the the the the the you know the the kind of um 00:05:19.920 --> 00:05:24.397 the progress that's been made in both the models with Gemini 3 but also uh on 00:05:24.480 --> 00:05:28.877 the product side with Gemini app getting increasing uh market share. So I feel 00:05:28.960 --> 00:05:33.837 like uh we're making great progress um but there's a ton more work to do um and 00:05:33.920 --> 00:05:37.117 you know we're bringing to bear Google DeepMind's kind of like the engine room 00:05:37.200 --> 00:05:40.957 of Google where we're getting used to um shipping our models more and more more 00:05:41.040 --> 00:05:44.717 quickly into the product surfaces. One question for you Daria on on this aspect 00:05:44.800 --> 00:05:48.317 of it because you've just saw you're in the process of you know a new round at 00:05:48.400 --> 00:05:50.397 an extraordinary valuation too. Um but 00:05:50.480 --> 00:05:55.437 you are unlike them as a let's call it an independent model maker and there is 00:05:55.520 --> 00:05:58.637 I think an increasing concern that the independent model makers will not be 00:05:58.720 --> 00:06:02.797 able to continue for long enough until you get to where the revenues come in. 00:06:02.880 --> 00:06:06.797 Um it's made very openly about open AI but talk me through how you think about 00:06:06.880 --> 00:06:09.677 that and then we'll get to the AGI itself. Yeah, I mean that you know I 00:06:09.760 --> 00:06:11.437 think I think I think how we think about 00:06:11.520 --> 00:06:14.477 that is you know as we've built better 00:06:14.560 --> 00:06:19.197 and better models there's been a kind of exponential relationship not only 00:06:19.280 --> 00:06:22.957 between how much compute you put into the model and how cognitively capable it 00:06:23.040 --> 00:06:27.037 is but between how cognitively capable it is and how much revenue it's able to 00:06:27.120 --> 00:06:29.437 generate. So our revenues grown 10x in 00:06:29.520 --> 00:06:31.277 the last three years from 0 to 100 00:06:31.360 --> 00:06:34.077 million in 2023 100 million to a billion 00:06:34.160 --> 00:06:39.277 in 2024 and 1 billion to 10 billion in 2025. And so th those revenue numbers, 00:06:39.360 --> 00:06:41.997 you know, I don't know if that curve will literally continue. It would be 00:06:42.080 --> 00:06:47.117 crazy if it did. Um, but those numbers are starting to get not too far from, 00:06:47.200 --> 00:06:50.317 you know, the sca the scale of the largest companies in the world. So 00:06:50.400 --> 00:06:52.717 there's there's there's always uncertainty. You know, we're trying to 00:06:52.800 --> 00:06:57.037 bootstrap this from nothing. It's it's a crazy thing, but but I have confidence 00:06:57.120 --> 00:07:01.917 that if we're able to produce the best models in the things that we focus on, 00:07:02.000 --> 00:07:05.677 um, uh, then I think then I think things will go well. And you know, I I will I 00:07:05.760 --> 00:07:09.037 will generally say, you know, I think I think it's been a good year for both 00:07:09.120 --> 00:07:13.837 both Google and Anthropic. And I think the thing we actually have in common is 00:07:13.920 --> 00:07:17.277 that they're you know, they're both kind of kind of kind of companies that are, 00:07:17.360 --> 00:07:20.477 you know, or the research part of the company that are kind of led by 00:07:20.560 --> 00:07:25.677 researchers who focus on the models who focus on solving important problems in 00:07:25.760 --> 00:07:27.117 the world, right? Who have these kind of 00:07:27.200 --> 00:07:30.237 hard scientific problems as a as a north 00:07:30.320 --> 00:07:34.717 star. and and and I think those are the kind of companies that are going to 00:07:34.800 --> 00:07:38.717 succeed going forward and you know I think I think we share that between us 00:07:38.800 --> 00:07:41.597 >> very much. Uh I'm I'm going to resist the temptation to ask you what will 00:07:41.680 --> 00:07:45.117 happen to the companies that are not led by researchers 00:07:45.200 --> 00:07:46.477 uh because I know you won't answer it. 00:07:46.560 --> 00:07:51.757 But let's then go on to uh the predictions area now and this we are 00:07:51.840 --> 00:07:54.717 supposed to be talking about the day after AI but let's talk about closing 00:07:54.800 --> 00:07:59.037 the loop. This the odds that you will get models that will close the loop and 00:07:59.120 --> 00:08:02.557 be able to you know power themselves if you will because that's the really the 00:08:02.640 --> 00:08:06.957 crux for the the winner takes all threshold approach. Do you still believe 00:08:07.040 --> 00:08:11.277 that we are likely to see that or is this going to be much more of a normal 00:08:11.360 --> 00:08:15.597 technology where followers and catchup can can compete? 00:08:15.680 --> 00:08:18.237 >> Well, look, I definitely don't think it's going to be a normal technology. 00:08:18.320 --> 00:08:22.077 So, I mean, there are aspects already that as Dario mentioned that it's 00:08:22.160 --> 00:08:26.557 already helping with our coding and and some aspects of research. The full 00:08:26.640 --> 00:08:30.157 closing of the loop though, I think is an unknown. I mean, I think it's 00:08:30.240 --> 00:08:33.117 possible to do. you may need AGI itself 00:08:33.200 --> 00:08:37.197 to be able to do that in some domains again where there these domains you know 00:08:37.280 --> 00:08:40.717 where there's there's more messiness around them it's not so easy to verify 00:08:40.800 --> 00:08:45.917 your answer very quickly um there's kind of MP hard domains so as soon as you 00:08:46.000 --> 00:08:49.917 start getting more and you know I also include by the way for AGI physical AI 00:08:50.000 --> 00:08:53.037 robotics working all of these kind of things and then you've got you know 00:08:53.120 --> 00:08:57.597 hardware in the loop uh that may uh limit how fast the self-improvement 00:08:57.680 --> 00:09:01.837 systems can work but I think in coding and mathematics and these kind areas. I 00:09:01.920 --> 00:09:04.477 can definitely see that working. And then the question is more theoretical 00:09:04.560 --> 00:09:06.557 one is what is the limit of engineering 00:09:06.640 --> 00:09:09.677 and maths uh to solve uh the natural 00:09:09.760 --> 00:09:14.317 sciences. Daria, you um last year, I think it was last year that you 00:09:14.400 --> 00:09:16.317 published Machines of Love and Grace um 00:09:16.400 --> 00:09:19.357 which was a very I would say upbeat 00:09:19.440 --> 00:09:23.837 essay about the potential that that you were going to see unfold and you were 00:09:23.920 --> 00:09:26.157 talking about you know a a what was it a 00:09:26.240 --> 00:09:29.037 genius of data at country 00:09:29.120 --> 00:09:33.677 data center I'm told that you are working on an update to this a new essay 00:09:33.760 --> 00:09:37.917 so you know wait for it guys it's not out yet but it is coming out but perhaps 00:09:38.000 --> 00:09:43.437 you can give us a sort of a sneak preview of what a year later your big 00:09:43.520 --> 00:09:46.557 take is going to be. >> Yes. So, you know, my take my take has 00:09:46.640 --> 00:09:50.477 not changed. It has always been my view that, you know, AI is going to be 00:09:50.560 --> 00:09:53.517 incredibly powerful. I think Demis and I, you know, kind of agree on that. It's 00:09:53.600 --> 00:09:57.997 just a question of exactly when. Um, uh, and because it's incredibly powerful, it 00:09:58.080 --> 00:10:00.957 will do all these wonderful things like the ones I talked about in Machines of 00:10:01.040 --> 00:10:05.437 Loving Grace. It, you know, will help us cure cancer. It may help us to eradicate 00:10:05.520 --> 00:10:09.997 tropical diseases. It will help us understand understand the universe. but 00:10:10.080 --> 00:10:14.237 that there are these, you know, immense and grave risks that, you know, not that 00:10:14.320 --> 00:10:18.397 we can't address them. I'm not a doomer, but but that, you know, we we we we we 00:10:18.480 --> 00:10:21.677 need to think about them and we need to address them. And I wrote Machines of 00:10:21.760 --> 00:10:25.837 Loving Grace first. I' I'd love to give some uh a sophisticated reason why I 00:10:25.920 --> 00:10:29.597 wrote that first, but it was just that the the positive essay was easier and 00:10:29.680 --> 00:10:34.157 more fun to write than than the negative essay. Um, so, you know, I finally spent 00:10:34.240 --> 00:10:37.997 some time on vacation and I was able to write an essay about the risks. Even 00:10:38.080 --> 00:10:40.797 when I'm writing about the risks, um, I 00:10:40.880 --> 00:10:43.677 I I try, you know, I I I'm like an 00:10:43.760 --> 00:10:47.997 optimistic person, right? So, even as I'm writing about these risks, I I I 00:10:48.080 --> 00:10:51.517 wrote about it in a way that was like, how do we overcome these risks? How do 00:10:51.600 --> 00:10:56.077 we have a battle plan to fight them? And and and the way I the way I framed it 00:10:56.160 --> 00:11:00.157 was, you know, there's this scene from Carl Sean's Contact, the movie version 00:11:00.240 --> 00:11:04.077 of it, where, you know, they they kind of discover alien life and this 00:11:04.160 --> 00:11:07.997 international panel that's like interviewing um uh you know, people to, 00:11:08.080 --> 00:11:12.237 you know, to be humanity's representative to meet the alien. Um uh 00:11:12.320 --> 00:11:15.917 and uh one one of the questions they ask one of the candidates is, you know, if 00:11:16.000 --> 00:11:19.197 you could ask the aliens any one question, what it would what what what 00:11:19.280 --> 00:11:23.517 would it be? And one of one of the characters says,"I would ask,"How did 00:11:23.600 --> 00:11:28.557 you do it? How did you manage to get through this technological adolescence 00:11:28.640 --> 00:11:32.717 without destroying yourselves? How did you make it through?" And and and ever 00:11:32.800 --> 00:11:35.437 since I saw it, it was like 20 years ago, I think I saw that movie. It's kind 00:11:35.520 --> 00:11:39.357 of stuck with me. And that that's the frame that I use, which is which is 00:11:39.440 --> 00:11:41.997 that, you know, we we're we're we are 00:11:42.080 --> 00:11:44.557 knocking on the door of these incredible 00:11:44.640 --> 00:11:46.717 capabilities, right? the the ability to 00:11:46.800 --> 00:11:51.677 build basically machines out of sand, right? I think I think it was inevitable 00:11:51.760 --> 00:11:53.517 that the instant we started working with 00:11:53.600 --> 00:11:57.437 fire. Um uh but but how we handle it is 00:11:57.520 --> 00:11:59.597 is not inevitable. And so I think the 00:11:59.680 --> 00:12:04.637 next few years we're going to be dealing with, you know, how do we keep these 00:12:04.720 --> 00:12:06.317 systems under control that are highly 00:12:06.400 --> 00:12:09.197 autonomous and smarter than any human? 00:12:09.280 --> 00:12:12.477 How do we make sure that individuals 00:12:12.560 --> 00:12:16.317 don't misuse them? Right? I have worries about things like bioteterrorism. How do 00:12:16.400 --> 00:12:20.237 we make sure that nation states don't misuse them? That's why I've been so 00:12:20.320 --> 00:12:24.637 concerned about, you know, the CCP, other authoritarian authoritarian 00:12:24.720 --> 00:12:27.917 governments. What are the economic impacts? Right? I've talked about labor 00:12:28.000 --> 00:12:31.277 displacement a lot. And and you know, what what haven't we thought of which 00:12:31.360 --> 00:12:35.037 which in many cases, you know, maybe may be the the hardest thing to deal with at 00:12:35.120 --> 00:12:37.597 all. Um, so, you know, I I'm I'm 00:12:37.680 --> 00:12:42.477 thinking through how to address those risks. you know, for for each of these, 00:12:42.560 --> 00:12:46.557 it's a mixture of things that we individually need to do as as leaders of 00:12:46.640 --> 00:12:50.797 the of of of the companies and that we can do working together. And then there 00:12:50.880 --> 00:12:54.717 there's going to need to be some role for wider societal institutions like the 00:12:54.800 --> 00:12:58.877 like the government in in in addressing all of these. But, you know, I I I just 00:12:58.960 --> 00:13:02.397 feel this urgency that, you know, every day, you know, there's there's all kinds 00:13:02.480 --> 00:13:03.997 of crazy stuff going on in the outside 00:13:04.080 --> 00:13:07.117 world, outside AI, right? Um but but you 00:13:07.200 --> 00:13:09.677 know my my my view is this is happening 00:13:09.760 --> 00:13:14.877 so fast and is such a crisis we should be devoting almost all of our effort to 00:13:14.960 --> 00:13:16.637 thinking about how to get through this. 00:13:16.720 --> 00:13:20.237 >> So I can't decide whether I'm more surprised that you a take a vacation b 00:13:20.320 --> 00:13:24.157 when you take a vacation you think about the risks of AI and c that your essay is 00:13:24.240 --> 00:13:28.557 framed in terms of are we going to get through the technological adolescence of 00:13:28.640 --> 00:13:31.837 this technology without destroying ourselves. So, I'm my head is slightly 00:13:31.920 --> 00:13:35.117 spinning, but you then and I can't wait to read it, but you you you mentioned 00:13:35.200 --> 00:13:39.037 several areas that can guide the rest of our conversation. Let's start with jobs 00:13:39.120 --> 00:13:41.837 um because you actually have been very outspoken about that and I think you 00:13:41.920 --> 00:13:44.877 said that half of entry- level white collar jobs could be gone within the 00:13:44.960 --> 00:13:46.477 next one to five years. But I'm going to 00:13:46.560 --> 00:13:49.997 turn to you Demis because so far we 00:13:50.080 --> 00:13:54.877 haven't actually seen any discernable impact on the labor market. Um, yes, 00:13:54.960 --> 00:13:58.477 unemployment has ticked up in the US, but all of the kind of economic studies 00:13:58.560 --> 00:14:02.637 I've looked at and that we've written about suggest that this is overhiring 00:14:02.720 --> 00:14:04.397 post pandemic that it's really not 00:14:04.480 --> 00:14:07.277 AIdriven. If anything, people are hiring 00:14:07.360 --> 00:14:10.477 to build out AI capability. 00:14:10.560 --> 00:14:15.517 Do you think that this will be as you know economists have always argued that 00:14:15.600 --> 00:14:18.797 it's not a lump of labor fallacy that actually there will be new jobs created 00:14:18.880 --> 00:14:23.197 because so far the evidence seems to suggest that. Yeah, I mean I I think in 00:14:23.280 --> 00:14:26.397 um the near term that is what will happen. The kind of normal evolution 00:14:26.480 --> 00:14:28.317 when a breakthrough technology arrives. 00:14:28.400 --> 00:14:32.637 So some jobs will get disrupted but I think new even more valuable perhaps 00:14:32.720 --> 00:14:34.717 more meaningful jobs will get created. 00:14:34.800 --> 00:14:39.037 Um I think we're going to see this year the beginnings of maybe impacting the 00:14:39.120 --> 00:14:43.037 junior level entry level child of jobs internships this type of thing. And I 00:14:43.120 --> 00:14:47.117 think there is some evidence I can feel that ourselves maybe like a slowdown in 00:14:47.200 --> 00:14:50.637 hiring in that. But I think that can be more than compensated by the fact there 00:14:50.720 --> 00:14:54.877 are these amazing creative tools out there pretty much available for everyone 00:14:54.960 --> 00:14:57.437 uh almost for free that if you know I 00:14:57.520 --> 00:15:00.317 was to talk to a class of undergrads 00:15:00.400 --> 00:15:04.557 right now I would be telling them to get really unbelievably proficient with 00:15:04.640 --> 00:15:08.237 these tools. I think to the extent that even those of us building it, we're so 00:15:08.320 --> 00:15:11.997 busy building it, it's hard to have also time to really explore the almost the 00:15:12.080 --> 00:15:16.717 capability overhang even today's models and products have let alone tomorrow's 00:15:16.800 --> 00:15:20.797 and I think that uh can be maybe better than a traditional internship would have 00:15:20.880 --> 00:15:23.197 been in terms of you sort of leaprogging 00:15:23.280 --> 00:15:28.237 uh yourself to be useful uh in a useful in a profession. So I think there's 00:15:28.320 --> 00:15:32.397 that's what I see happening probably in the next five years. Um maybe we again 00:15:32.480 --> 00:15:35.997 slightly differ on time scales on that but I think what happens after AGI 00:15:36.080 --> 00:15:39.917 arrives that's a different question cuz I think really we would be in uncharted 00:15:40.000 --> 00:15:42.157 territory at that point. >> Do you think it's going to take longer 00:15:42.240 --> 00:15:45.037 than you thought last year when you said half of all white 00:15:45.120 --> 00:15:48.717 >> colors? I have about the same view. I I actually agree with you and with Demis 00:15:48.800 --> 00:15:52.557 that at the time I made the comment there was no impact on the labor market. 00:15:52.640 --> 00:15:56.877 I wasn't saying there was an impact on the labor market at that moment. Um, you 00:15:56.960 --> 00:16:00.637 know, now I think maybe we're starting to see just just the little beginnings 00:16:00.720 --> 00:16:02.317 of it, you know, in software in coding. 00:16:02.400 --> 00:16:04.797 I even see it within within anthropic 00:16:04.880 --> 00:16:07.997 where, you know, I you know, I can look 00:16:08.080 --> 00:16:12.237 forward I can kind of look forward to a time where on the more junior end and 00:16:12.320 --> 00:16:16.157 then on the more on the more on the more on the more intermediate end, we 00:16:16.240 --> 00:16:17.837 actually need less and not more people. 00:16:17.920 --> 00:16:21.277 And you know, we're thinking about how to deal with that within anthropic in a 00:16:21.360 --> 00:16:24.157 in a in a you know, sense in a sensible 00:16:24.240 --> 00:16:28.157 way. Um I, you know, one to five years 00:16:28.240 --> 00:16:32.397 as of six months ago, I would stick with that. You know, if you kind of, you 00:16:32.480 --> 00:16:36.077 know, connect this to what I said before, which is, you know, we we might 00:16:36.160 --> 00:16:40.557 have AI that's better than humans at at everything in, you know, maybe one to 00:16:40.640 --> 00:16:45.357 two years, maybe a little longer than that. The those don't seem to line up. 00:16:45.440 --> 00:16:47.517 The reason is that there's this there's 00:16:47.600 --> 00:16:52.397 this lag and there's this replacement thing, right? I I know the labor market 00:16:52.480 --> 00:16:56.797 is adaptable, right? Just like you know 80% of people used to do farming you 00:16:56.880 --> 00:16:59.757 know farming got automated and then they became factory workers and then 00:16:59.840 --> 00:17:04.957 knowledge workers. So you know there is some level of adaptability here as well 00:17:05.040 --> 00:17:08.077 right we should be economically sophisticated about how the labor market 00:17:08.160 --> 00:17:12.717 works but my worry is as this exponential keeps compounding and I 00:17:12.800 --> 00:17:17.197 don't think it's going to take that long again somewhere between between a year 00:17:17.280 --> 00:17:20.717 and five years it will overwhelm our ability to adapt. I think I may be 00:17:20.800 --> 00:17:25.597 saying the same thing Demis is just factored out of that that difference we 00:17:25.680 --> 00:17:28.797 have about timelines which I think ultimately comes down to how how fast 00:17:28.880 --> 00:17:32.077 you close the loop on CO. >> How much confidence do you have that 00:17:32.160 --> 00:17:35.197 governments get the scale of this and 00:17:35.280 --> 00:17:39.837 have are beginning to think about what policy responses they need to have? 00:17:39.920 --> 00:17:44.157 >> I don't think that that that it's anywhere near enough work going on about 00:17:44.240 --> 00:17:47.197 this. I'm I'm constantly surprised even when I meet economists at places like 00:17:47.280 --> 00:17:51.037 this that they're not more of uh professional economist professors 00:17:51.120 --> 00:17:53.997 thinking about what happens um and not 00:17:54.080 --> 00:17:58.557 just sort of on the way to AGI but um uh even if we get all the technical things 00:17:58.640 --> 00:18:01.917 right that Dario was talking about and the job displacement is one question 00:18:02.000 --> 00:18:04.397 we're worried about the economics of that but maybe there are ways to 00:18:04.480 --> 00:18:08.557 distribute this new productivity this new wealth more fairly I don't know if 00:18:08.640 --> 00:18:11.517 we have the right institutions to do that but that's what should happen at 00:18:11.600 --> 00:18:15.037 that point there should be you know we maybe in a post scarcity world. But then 00:18:15.120 --> 00:18:17.837 there are even the things that keep me up right now. There are even bigger 00:18:17.920 --> 00:18:19.757 questions than that at that point to do 00:18:19.840 --> 00:18:23.597 with meaning and um purpose and a lot of 00:18:23.680 --> 00:18:26.877 the things that we get from our jobs not just economically. That's one question. 00:18:26.960 --> 00:18:31.277 But I think that may be easier to solve strangely than uh what happens to the 00:18:31.360 --> 00:18:33.757 human condition and humanity as a whole. 00:18:33.840 --> 00:18:36.877 And I think I'm also optimistic we'll come up with new answers there. We do a 00:18:36.960 --> 00:18:39.437 lot of things today um from extreme 00:18:39.520 --> 00:18:41.517 sports to art that aren't necessarily 00:18:41.600 --> 00:18:44.717 directly to do with economic gain. So I 00:18:44.800 --> 00:18:48.557 think we will find uh meaning and maybe there'll be even more sort of 00:18:48.640 --> 00:18:52.717 sophisticated versions of those activities. Um plus I think we'll be 00:18:52.800 --> 00:18:57.117 exploring the stars. So there'll be all of that to to factor in as well for in 00:18:57.200 --> 00:19:01.037 terms of purpose. But I think it's really worth thinking now even on my 00:19:01.120 --> 00:19:05.517 timelines of like five to 10 years away that isn't a lot of time uh before this 00:19:05.600 --> 00:19:07.837 comes. How big do you think is the risk 00:19:07.920 --> 00:19:11.197 of a popular backlash against AI that 00:19:11.280 --> 00:19:16.317 will somehow kind of cause governments to do what from your perspective might 00:19:16.400 --> 00:19:20.237 be stupid things? Because I'm just thinking back to the era of, you know, 00:19:20.320 --> 00:19:24.717 globalization in the 1990s when when there was indeed some displacement of 00:19:24.800 --> 00:19:26.957 jobs. Governments didn't do enough. The 00:19:27.040 --> 00:19:31.757 public backlash was such that we've ended up sort of where we are now. uh do 00:19:31.840 --> 00:19:35.997 you think that there is a risk that there will be a growing antipathy 00:19:36.080 --> 00:19:40.637 towards what you are doing and your companies in the kind of body politic? 00:19:40.720 --> 00:19:44.477 >> Um I think there's definitely a risk. I think um I think that's kind of 00:19:44.560 --> 00:19:47.757 reasonable. There's fear and there's worries about these things like jobs and 00:19:47.840 --> 00:19:51.917 livelihoods. Um I think there's a couple of things that I mean it's going to be 00:19:52.000 --> 00:19:55.757 very complicated the next few years I think geopolitically but also the 00:19:55.840 --> 00:19:59.197 various factors here like we want to and we're trying to do this with AlphaFold 00:19:59.280 --> 00:20:03.837 and our science work and isomeorphic our spinout company solve all disease cure 00:20:03.920 --> 00:20:07.517 diseases come up with new energy sources I think as a society it's clear we'd 00:20:07.600 --> 00:20:11.677 want that I think maybe the balance of what the industry is doing is not enough 00:20:11.760 --> 00:20:14.877 balance towards those types of activities I think we should have a lot 00:20:14.960 --> 00:20:19.117 more examples I know Dary agrees with me of like alpha fold like things that help 00:20:19.200 --> 00:20:22.957 sort of unequivocal good in the world and I think actually it's incumbent on 00:20:23.040 --> 00:20:26.477 the industry and and all of us leading players to show that more demonstrate 00:20:26.560 --> 00:20:30.557 that not just talk about it but demonstrate that um and but then it's 00:20:30.640 --> 00:20:34.477 going to come with these other intendent disruptions and um but I don't I think 00:20:34.560 --> 00:20:37.277 the other issue is the geopolitical competition there's obviously 00:20:37.360 --> 00:20:41.837 competition between the companies but also US and China primarily so unless 00:20:41.920 --> 00:20:46.157 there's an international cooperation or or understanding around this um uh which 00:20:46.240 --> 00:20:49.757 I think would be good actually in terms things like minimum safety standards for 00:20:49.840 --> 00:20:52.477 deployment. I think Dario would agree on that as well. I think it's vitally 00:20:52.560 --> 00:20:55.357 needed. This technology is going to be crossborder. It's going to affect 00:20:55.440 --> 00:20:59.117 everyone. It's going to affect all of humanity. Um actually contact is one of 00:20:59.200 --> 00:21:01.357 my favorite films as well. So funny 00:21:01.440 --> 00:21:06.157 enough, I didn't realize it was yours too, Dario. But I I think um um you know 00:21:06.240 --> 00:21:10.237 those kind of things need to be worked through. Um and and if we can maybe it 00:21:10.320 --> 00:21:13.917 would be good to have a bit of slow a slightly slower pace than we're 00:21:14.000 --> 00:21:18.237 currently predicting even my timelines so that we can get this right society 00:21:18.320 --> 00:21:22.077 but it that would require some coordination that is I prefer your 00:21:22.160 --> 00:21:25.517 timelines. Yes, I think I battle concede. 00:21:25.600 --> 00:21:29.117 >> But but but Dario, let's turn to this now because the one thing since we last 00:21:29.200 --> 00:21:31.437 spoke uh in Paris, the geopolitical 00:21:31.520 --> 00:21:36.237 environment has, if anything, I don't know, complicated, mad, crazy, whatever, 00:21:36.320 --> 00:21:38.237 whatever phrase you want to use. 00:21:38.320 --> 00:21:42.557 Secondly, the US has a very different approach now towards China. It's a much 00:21:42.640 --> 00:21:44.877 more it's a kind of no holds bar, go as 00:21:44.960 --> 00:21:50.397 fast as we can, but then sell chips to China. Um, and that is so you've got a 00:21:50.480 --> 00:21:55.357 different attitude towards the United States. you've got a a very um strange 00:21:55.440 --> 00:21:59.517 relationship between the United States and and Europe right now geopolitically 00:21:59.600 --> 00:22:02.717 against that. I mean I hear you talk about it would be nice to have a CERN 00:22:02.800 --> 00:22:06.157 like organization I mean it's a million years from where we are from the real 00:22:06.240 --> 00:22:11.037 world. So in the real world have the geopolitical risks increased and what if 00:22:11.120 --> 00:22:14.317 anything do you think should be done about that and and the administration 00:22:14.400 --> 00:22:17.357 seems to be doing the opposite of what you were suggesting? Yeah, I mean, look, 00:22:17.440 --> 00:22:19.997 you know, we're we're we're just trying to do the best we can to, you know, 00:22:20.080 --> 00:22:22.797 we're just we're just one company and we're we're trying to operate in, you 00:22:22.880 --> 00:22:26.877 know, the the environment that exists, no matter how no matter how crazy it is. 00:22:26.960 --> 00:22:30.877 But, you know, I think I think at least my policy recommendations haven't 00:22:30.960 --> 00:22:33.517 changed that, you know, not selling 00:22:33.600 --> 00:22:36.317 chips is one of the, you know, one of 00:22:36.400 --> 00:22:38.637 the one of the biggest things we can do 00:22:38.720 --> 00:22:41.357 um to, you know, make sure that we have 00:22:41.440 --> 00:22:44.077 the time to handle this. Um, you know, 00:22:44.160 --> 00:22:45.757 you know, I said I said before, you 00:22:45.840 --> 00:22:51.117 know, I I I prefer Demis' timeline. I wish we had 5 to 10 years, you know, so 00:22:51.200 --> 00:22:54.397 it's it's possible he's just right and I'm just wrong, but but assume I'm right 00:22:54.480 --> 00:22:56.397 and it can be done in one to two years. 00:22:56.480 --> 00:22:58.957 Why can't we slow down to to Demis' timeline? 00:22:59.040 --> 00:23:02.797 >> Well, you could just slow down. Well, no. The but but but the reason the 00:23:02.880 --> 00:23:05.197 reason we the reason we can't do that is 00:23:05.280 --> 00:23:07.597 is you know because we have 00:23:07.680 --> 00:23:10.557 >> geopolitical adversaries building the 00:23:10.640 --> 00:23:14.717 same technology at a similar pace, it's very hard to have an enforcable 00:23:14.800 --> 00:23:16.797 agreement where they slow down and we 00:23:16.880 --> 00:23:19.517 slow down and and so if we can just 00:23:19.600 --> 00:23:22.317 >> if we can just not sell the chips, then 00:23:22.400 --> 00:23:26.557 this isn't a question of competition between the US and China. This is a 00:23:26.640 --> 00:23:29.757 question of competition between me and Demis which I'm very confident that we 00:23:29.840 --> 00:23:32.797 can work out. >> And what do you make of the logic of the 00:23:32.880 --> 00:23:34.477 administration which as I understand it 00:23:34.560 --> 00:23:35.997 is we need to sell them chips because we 00:23:36.080 --> 00:23:40.077 need to bind them into US supply chains. 00:23:40.160 --> 00:23:44.157 So you know it's it's I I think it's I 00:23:44.240 --> 00:23:46.637 think it's a question not just of time 00:23:46.720 --> 00:23:48.477 scale but of the significance of the 00:23:48.560 --> 00:23:51.997 technology right if this was telecom or 00:23:52.080 --> 00:23:56.717 something then all this stuff about proliferating the US stack and you know 00:23:56.800 --> 00:23:58.797 wanting to build our you know chips 00:23:58.880 --> 00:24:00.477 around the world to make sure that you 00:24:00.560 --> 00:24:04.237 know you know this c you know the you 00:24:04.320 --> 00:24:07.757 know these random countries in different parts of the world you know build data 00:24:07.840 --> 00:24:12.317 centers that have Nvidia chips instead of Huawei chips. You know, I think of 00:24:12.400 --> 00:24:14.077 this more as like, you know, it's a 00:24:14.160 --> 00:24:16.957 decision. Are we going to, you know, 00:24:17.040 --> 00:24:20.397 sell nuclear weapons to North Korea and, 00:24:20.480 --> 00:24:24.477 you know, because that produces some profit for Boeing. Um, you know, where 00:24:24.560 --> 00:24:27.997 where we can say, okay, yeah, these cases were made by Boeing, like the US 00:24:28.080 --> 00:24:31.917 is winning. Like, this is great. Like, I I I just, you know, that that analogy 00:24:32.000 --> 00:24:34.877 should just make clear how I see this 00:24:34.960 --> 00:24:39.997 trade-off that I just don't think it makes sense. Um and and we've done a lot 00:24:40.080 --> 00:24:43.757 of more aggressive stuff toward, you know, toward towards towards China, 00:24:43.840 --> 00:24:46.557 China and other players that that I think is much less effective than this 00:24:46.640 --> 00:24:50.317 this one this one measure. >> One more area for me and then I hope 00:24:50.400 --> 00:24:51.997 we'll have time for a question or two. 00:24:52.080 --> 00:24:54.557 The other area of potential risk that 00:24:54.640 --> 00:24:59.437 doomers worry about is a kind of all powerful malign AI. Um, and I think 00:24:59.520 --> 00:25:02.877 you've both been somewhat skeptical of the doomer approach, but in the last 00:25:02.960 --> 00:25:05.677 year we have seen, you know, these 00:25:05.760 --> 00:25:07.357 models showing themselves to be capable 00:25:07.440 --> 00:25:11.117 of deception, duplicity. Uh, do you 00:25:11.200 --> 00:25:14.797 think that do you think differently about that risk now than you did a year 00:25:14.880 --> 00:25:19.197 ago? And is there something about the way the models are evolving that we 00:25:19.280 --> 00:25:21.117 should put a little bit more concern on that? 00:25:21.200 --> 00:25:24.237 >> Yeah, I mean, you know, since since the beginning of Enthropic, we've kind of 00:25:24.320 --> 00:25:27.997 thought about this risk. I mean, you know, our our our research at the 00:25:28.080 --> 00:25:31.597 beginning of it was very theoretical, right? You know, we pioneered this idea 00:25:31.680 --> 00:25:35.277 of mechanistic interpretability, which is looking inside the model and and 00:25:35.360 --> 00:25:39.037 trying to understand looking inside its brain, trying to understand why it does 00:25:39.120 --> 00:25:43.357 what it does as it, you know, as as human neuroscientists, which we actually 00:25:43.440 --> 00:25:48.397 both have background in, um, try try to understand try to understand the brain. 00:25:48.480 --> 00:25:52.397 And I think as time has gone on, we've we've increasingly documented the you 00:25:52.480 --> 00:25:55.917 know bad behaviors of the models when they emerge and are now working on 00:25:56.000 --> 00:26:00.397 trying to address them with mechanistic interpretability. So I you know I think 00:26:00.480 --> 00:26:03.997 uh you know I I've always been concerned about these these risks. I've talked to 00:26:04.080 --> 00:26:08.557 Demis many times. I think he has also been um concerned about these risks. I 00:26:08.640 --> 00:26:12.717 think I have definitely been and I I I would guess Demis as well although I'll 00:26:12.800 --> 00:26:17.277 let him speak for himself skeptical of of dumerism which is you know we're 00:26:17.360 --> 00:26:21.597 doomed. there's nothing we can do or this is the most likely outcome. I think 00:26:21.680 --> 00:26:24.237 this is a risk. This is a risk that if 00:26:24.320 --> 00:26:26.477 we work all work together, we can 00:26:26.560 --> 00:26:29.357 address we can learn through science to 00:26:29.440 --> 00:26:31.837 properly, you know, control and and 00:26:31.920 --> 00:26:33.597 direct these creations that we're 00:26:33.680 --> 00:26:36.637 building. But if we build them poorly, 00:26:36.720 --> 00:26:40.157 if we go, you know, if if if we're all 00:26:40.240 --> 00:26:43.837 racing and we go so fast that there's no guardrails, then I think there is risk 00:26:43.920 --> 00:26:46.397 of something going wrong. So I'm going to give you a chance to answer that in 00:26:46.480 --> 00:26:49.997 the context of of a slightly broader question which is over the past year 00:26:50.080 --> 00:26:53.117 have you grown more confident of the 00:26:53.200 --> 00:26:57.117 upside potential of the technology science all of the areas that you have 00:26:57.200 --> 00:27:00.877 talked about a lot or are you more worried about the risks that we've been 00:27:00.960 --> 00:27:04.317 discussing I've been working on this for 20 plus 00:27:04.400 --> 00:27:08.397 years so we we already knew look the reason I've spent my whole career on AI 00:27:08.480 --> 00:27:13.197 is is the upsides of solving basically the ultimate tool for science and 00:27:13.280 --> 00:27:14.877 understanding the universal around us. 00:27:14.960 --> 00:27:18.637 I've I've sort of been obsessed with that since a kid and and and building AI 00:27:18.720 --> 00:27:21.517 is the you know should be the ultimate tool for that if we do it in the right 00:27:21.600 --> 00:27:25.037 way. The risks also we've been thinking about since the start at least the start 00:27:25.120 --> 00:27:29.757 of deep mind 15 years ago and um we kind of sort of foresaw that if you got the 00:27:29.840 --> 00:27:33.917 upsides it's a dual purpose technology so it could be repurposed by say bad 00:27:34.000 --> 00:27:37.277 actors for harmful ends. So we've needed to think about that all the way through 00:27:37.360 --> 00:27:39.037 but I'm a big believer in human 00:27:39.120 --> 00:27:41.677 ingenuity. Um but the question is having 00:27:41.760 --> 00:27:44.957 the time and the focus and all the best 00:27:45.040 --> 00:27:48.077 minds collaborating on it to solve these 00:27:48.160 --> 00:27:52.077 problems. I'm sure if we had that we would solve the technical risk problem. 00:27:52.160 --> 00:27:55.517 It may be we don't have that and then that will introduce risk because we'll 00:27:55.600 --> 00:27:59.037 be sort of it'll be fragmented. There'll be different projects and people be 00:27:59.120 --> 00:28:02.397 racing each other then it's much harder to make sure you know these systems that 00:28:02.480 --> 00:28:04.877 we produce will be technically safe. But 00:28:04.960 --> 00:28:09.437 I I feel like that's a very tractable uh problem if we have the time if we have 00:28:09.520 --> 00:28:13.517 the time and space. I want to make sure there's one question gentlemen. Keep it 00:28:13.600 --> 00:28:16.557 very short because we've got literally two minutes. 00:28:16.640 --> 00:28:19.597 >> Thanks for Hello. >> Yeah. No speak. 00:28:19.680 --> 00:28:20.957 >> Thanks very much. I'm Philip, co-founder 00:28:21.040 --> 00:28:25.037 of StarCloud building data centers in space. Um I wanted to ask a slightly 00:28:25.120 --> 00:28:29.037 philosophical question. The sort of strongest argument for doomerism to me 00:28:29.120 --> 00:28:31.757 is the firmy paradox, the idea that we don't see intelligent life in our 00:28:31.840 --> 00:28:34.237 galaxy. I was wondering if you guys have any thoughts. Yeah, I've thought a lot 00:28:34.320 --> 00:28:38.397 about that. That can't be the reason because we we we should see all the AIS 00:28:38.480 --> 00:28:43.117 that have So, just so everyone know the idea is well, it's sort of unclear why 00:28:43.200 --> 00:28:46.477 that would happen, right? So, if if the reason there's a Firmeny paradox, there 00:28:46.560 --> 00:28:50.237 are no aliens cuz they get taken out by their own technology. We should be 00:28:50.320 --> 00:28:54.157 seeing paper clips coming towards us from some part of the galaxy. And 00:28:54.240 --> 00:28:57.597 apparently, we don't. We don't see any structures. Dyson sphere is nothing 00:28:57.680 --> 00:29:02.317 whether they're AI or natur or sort of biological. So to me um there has to be 00:29:02.400 --> 00:29:04.637 a different answer to FMY powders. I have my own theories about that, but 00:29:04.720 --> 00:29:06.397 it's out of scope for the next minute. 00:29:06.480 --> 00:29:09.357 But um you know I I just feel like uh 00:29:09.440 --> 00:29:13.837 that that I my prediction my feeling is that we're past the great filter. It was 00:29:13.920 --> 00:29:17.757 probably multisellular life if I would have to guess. It was incredibly hard 00:29:17.840 --> 00:29:22.397 for for biology to evolve that. Um so we're on you know there isn't a comfort 00:29:22.480 --> 00:29:25.917 of like what's going to happen next. I think it's for us to write as humanity 00:29:26.000 --> 00:29:28.157 what's going to happen next. >> This this could be a great discussion 00:29:28.240 --> 00:29:31.917 but is out of scope for the next 36 sessions. But what isn't 15 seconds each 00:29:32.000 --> 00:29:36.477 what when we meet again I hope next year uh the three of us which I would love uh 00:29:36.560 --> 00:29:38.557 what will have changed by then 00:29:38.640 --> 00:29:41.597 >> I well I think the biggest thing to 00:29:41.680 --> 00:29:44.877 watch is this issue of AI systems 00:29:44.960 --> 00:29:47.757 building AI systems how that goes whe 00:29:47.840 --> 00:29:49.277 that whether that goes one way or 00:29:49.360 --> 00:29:52.797 another that that will determine you 00:29:52.880 --> 00:29:55.117 know whether it's a few more years until 00:29:55.200 --> 00:29:58.717 we get there or or if we have you know 00:29:58.800 --> 00:30:03.437 you know if if we have wonders and and a great emergency in front of us that we 00:30:03.520 --> 00:30:05.677 have to face. >> AI systems, building AI system. 00:30:05.760 --> 00:30:07.197 >> I agree on that. So, we're we're keeping 00:30:07.280 --> 00:30:11.357 close touch about that. Um but also I think um outside of that, I think there 00:30:11.440 --> 00:30:15.117 are other interesting uh uh uh ideas being researched like world models, 00:30:15.200 --> 00:30:18.317 continual learning. These are the things I think that will need to be cracked. If 00:30:18.400 --> 00:30:22.157 self-improvement doesn't sort of deliver the goods on its own, then we'll need 00:30:22.240 --> 00:30:25.757 these other things to work. And then I think things like robotics may have its 00:30:25.840 --> 00:30:29.037 sort of breakout moment. But maybe on the basis of what you've just said, we 00:30:29.120 --> 00:30:31.757 should all be hoping that it does take you a little bit longer and indeed 00:30:31.840 --> 00:30:33.917 everybody else to give us >> I would prefer that. I think that would 00:30:34.000 --> 00:30:35.117 be better for the world. 00:30:35.200 --> 00:30:41.240 >> Well, you guys could do something about that. Thank you both very much.