WEBVTT 00:00:00.000 --> 00:00:03.737 I want to try and focus on scenarios where A.I. goes rogue. 00:00:03.820 --> 00:00:05.957 I should have had a picture of a Terminator robot 00:00:06.040 --> 00:00:07.897 to scare people as much as possible. 00:00:07.980 --> 00:00:10.990 I think the internet... The internet does that for us. 00:00:16.800 --> 00:00:21.697 Are the lords of artificial intelligence on the side of the human race? 00:00:21.780 --> 00:00:24.257 "My prediction is there’ll be more robots than people." 00:00:24.340 --> 00:00:28.217 "The physical and the digital worlds should really be fully blended." 00:00:28.300 --> 00:00:31.417 "I don’t think the world has really had the humanoid robots moment yet. 00:00:31.500 --> 00:00:33.077 It’s going to feel very sci-fi." 00:00:33.160 --> 00:00:36.097 That’s the core question I had for this week’s guest. 00:00:36.180 --> 00:00:39.237 He’s the head of Anthropic, one of the fastest growing A.I. 00:00:39.320 --> 00:00:45.017 companies. Anthropic is estimated to be worth nearly $350 billion. 00:00:45.100 --> 00:00:47.737 It’s been win after win for Anthropic’s Claude code. 00:00:47.820 --> 00:00:50.797 He’s a utopian of sorts, when it comes to the potential 00:00:50.880 --> 00:00:54.497 effects of the technology that he’s unleashing on the world. 00:00:54.580 --> 00:00:56.417 "You know, will help us cure cancer. 00:00:56.500 --> 00:00:59.137 It may help us to eradicate tropical diseases. 00:00:59.220 --> 00:01:02.187 It will help us understand, understand the universe." 00:01:02.270 --> 00:01:07.147 But he also sees grave dangers ahead and massive disruption, no matter what. 00:01:07.230 --> 00:01:10.167 "This is happening so fast and is such a crisis, 00:01:10.250 --> 00:01:13.087 we should be devoting almost all of our effort 00:01:13.170 --> 00:01:15.247 to thinking about how to get through this." 00:01:15.330 --> 00:01:18.497 Dario Amodei, welcome to Interesting Times. 00:01:18.580 --> 00:01:19.747 Thank you for having me, Ross. 00:01:19.830 --> 00:01:24.207 Thank you for being here. So you are rather unusually, maybe 00:01:24.290 --> 00:01:27.047 for a tech C.E.O., an essayist. 00:01:27.130 --> 00:01:30.807 You have written two long, very interesting essays 00:01:30.890 --> 00:01:35.007 about the promise and the peril of artificial intelligence. 00:01:35.090 --> 00:01:37.707 And we’re going to talk about the perils in this conversation. 00:01:37.790 --> 00:01:41.287 But I thought it would be good to start with the promise 00:01:41.370 --> 00:01:44.107 and with the optimistic vision. 00:01:44.190 --> 00:01:46.887 Indeed, I would say the utopian vision that you laid 00:01:46.970 --> 00:01:51.087 out a couple of years ago in an essay entitled, "Machines 00:01:51.170 --> 00:01:55.807 of Loving Grace," which we’ll come back to that title, I think, at the end. 00:01:55.890 --> 00:01:59.967 But, I think a lot of people encounter A.I. news 00:02:00.050 --> 00:02:04.767 through headlines predicting a bloodbath for white collar 00:02:04.850 --> 00:02:06.147 jobs, these kinds of things. 00:02:06.230 --> 00:02:08.967 Sometimes your own quotes — Have used my own quotes — Yes. 00:02:09.050 --> 00:02:10.227 Have encouraged these things. 00:02:10.310 --> 00:02:13.767 And I think there’s a commonplace sense of, "What is A.I. 00:02:13.850 --> 00:02:18.347 for?" that people have. So why don’t you answer that question, 00:02:18.430 --> 00:02:21.047 to start out — if everything goes 00:02:21.130 --> 00:02:25.687 amazingly in the next five or 10 years, what is A.I. for? 00:02:25.770 --> 00:02:29.007 Yeah, so I think for a little background 00:02:29.090 --> 00:02:33.047 before I worked in before I worked in tech at all, 00:02:33.130 --> 00:02:39.207 I was a biologist. I first worked on computational neuroscience, and then 00:02:39.290 --> 00:02:43.807 I worked at Stanford Medical School on finding protein biomarkers for cancer 00:02:43.890 --> 00:02:47.927 on trying to improve diagnostics and curing cancer. 00:02:48.010 --> 00:02:50.187 And one of the observations that I most had 00:02:50.270 --> 00:02:56.027 when I worked in that field was the incredible complexity of it. 00:02:56.110 --> 00:02:59.757 Each protein has a level localized within each cell. 00:02:59.840 --> 00:03:02.297 It’s not enough to measure the level within the body 00:03:02.380 --> 00:03:03.637 or the level within each cell. 00:03:03.720 --> 00:03:06.097 You have to measure the level in a particular part 00:03:06.180 --> 00:03:10.497 of the cell and the other proteins that it’s interacting with or complexing with. 00:03:10.580 --> 00:03:14.017 And I had the sense of, "Man, this is too complicated for humans." 00:03:14.100 --> 00:03:18.497 We’re making progress on, all these problems of biology 00:03:18.580 --> 00:03:21.497 and medicine, but we’re making progress relatively slowly. 00:03:21.580 --> 00:03:24.297 And so what drew me to the field of A.I. 00:03:24.380 --> 00:03:28.037 was this idea of — that you know, could we make progress more quickly? 00:03:28.120 --> 00:03:31.297 Look, we’ve been trying to apply A.I. and machine learning 00:03:31.380 --> 00:03:34.457 techniques to biology for a long time. 00:03:34.540 --> 00:03:38.597 Typically they’ve been for analyzing data, but as A.I. gets really powerful, 00:03:38.680 --> 00:03:40.847 I think we should actually think about it differently. 00:03:40.930 --> 00:03:45.997 We should think of A.I. as doing the job of the biologist, right? 00:03:46.080 --> 00:03:48.457 Doing the whole thing from end to end. 00:03:48.540 --> 00:03:52.717 And part of that involves proposing experiments, 00:03:52.800 --> 00:03:54.497 coming up with new techniques. 00:03:54.580 --> 00:03:56.947 I have this section where I say, "Look, 00:03:57.030 --> 00:03:59.547 a lot of the progress in biology has been driven 00:03:59.630 --> 00:04:03.147 by this relatively small number of insights that lets 00:04:03.230 --> 00:04:07.507 us measure or get at or intervene in the stuff that’s really small. 00:04:07.590 --> 00:04:09.467 You look at a lot of these techniques. 00:04:09.550 --> 00:04:13.787 They’re invented very much as a matter of serendipity. 00:04:13.870 --> 00:04:17.467 CRISPR, which is one of these gene editing technologies 00:04:17.550 --> 00:04:20.427 was invented because someone went to a lecture 00:04:20.510 --> 00:04:25.027 on the bacterial immune system and connected that to the work 00:04:25.110 --> 00:04:27.347 they were doing on gene therapy. 00:04:27.430 --> 00:04:29.767 And that connection could have been made 30 years ago. 00:04:29.850 --> 00:04:33.387 And so the thought is —could A.I. accelerate all of this 00:04:33.470 --> 00:04:35.687 and could we really cure cancer? 00:04:35.770 --> 00:04:38.487 Could we really cure Alzheimer’s disease? 00:04:38.570 --> 00:04:41.127 Could we really cure, heart disease? 00:04:41.210 --> 00:04:44.467 And more subtly, some of the more psychological afflictions 00:04:44.550 --> 00:04:48.267 that people have — depression, bipolar — 00:04:48.350 --> 00:04:50.707 could we do something about these? To the extent that 00:04:50.790 --> 00:04:55.237 they’re biologically based, which I think they are, at least in part. 00:04:55.320 --> 00:04:58.577 So, I go through this argument here, "Well, how fast could it go?" 00:04:58.660 --> 00:05:02.537 If we have these intelligences out there who could do just about anything? 00:05:02.620 --> 00:05:05.957 And I want to pause you there because one of the interesting 00:05:06.040 --> 00:05:08.217 things about your framing in that essay, 00:05:08.300 --> 00:05:12.197 and you returned to it, is that these intelligences don’t 00:05:12.280 --> 00:05:15.797 have to be right, the kind of maximal godlike 00:05:15.880 --> 00:05:18.777 superintelligence that comes up in A.I. debates. 00:05:18.860 --> 00:05:22.677 You’re basically saying, if we can achieve a strong 00:05:22.760 --> 00:05:26.337 intelligence at the level of peak human performance — 00:05:26.420 --> 00:05:31.117 peak human performance, yes — and then multiply it, right, to what? 00:05:31.200 --> 00:05:32.737 Your phrase is, "A country of geniuses." 00:05:32.820 --> 00:05:34.757 A country — have 100 million of them. 00:05:34.840 --> 00:05:37.177 Right. A hundred million — Each, a little trained, 00:05:37.260 --> 00:05:40.377 a little different, or trying a different problem. 00:05:40.460 --> 00:05:44.857 There’s benefit in diversification and trying things a little differently. 00:05:44.940 --> 00:05:48.337 But yes. So you don’t have to have the full machine. 00:05:48.420 --> 00:05:50.707 God you just need to have 100 million geniuses. 00:05:50.790 --> 00:05:52.377 You don’t have to have the full machine. 00:05:52.460 --> 00:05:53.957 God and indeed, there are places 00:05:54.040 --> 00:05:58.797 where I cast doubt on whether the machine God would 00:05:58.880 --> 00:06:02.857 be that much more effective at these things than the 100 million geniuses. 00:06:02.940 --> 00:06:08.297 I have this concept called the diminishing returns to intelligence, right. 00:06:08.380 --> 00:06:13.187 Which is there’s economists talk about the marginal productivity of land and labor. 00:06:13.270 --> 00:06:16.297 We’ve never thought about the marginal productivity of intelligence. 00:06:16.380 --> 00:06:18.877 But if I look at some of these problems in biology 00:06:18.960 --> 00:06:22.717 at some level, you just have to interact with the world at some level, you just 00:06:22.800 --> 00:06:24.537 have to try things at some level. 00:06:24.620 --> 00:06:29.117 You just have to comply with the laws or change the laws on getting medicines 00:06:29.200 --> 00:06:30.457 through the regulatory system. 00:06:30.540 --> 00:06:35.937 So there’s a finite rate at which these changes can happen. 00:06:36.020 --> 00:06:38.227 Now there are some domains like if you’re playing chess 00:06:38.310 --> 00:06:41.977 or Go where the intelligence ceiling is extremely high. 00:06:42.060 --> 00:06:44.617 But I think the real world has a lot of limiters. 00:06:44.700 --> 00:06:46.897 So maybe you can go above the genius level. 00:06:46.980 --> 00:06:52.167 But, sometimes I think all this discussion of could you use a moon of computation 00:06:52.250 --> 00:06:56.647 to make an AI God are there a little bit sensationalistic 00:06:56.730 --> 00:06:59.287 and besides the point, even as I think 00:06:59.370 --> 00:07:02.107 this will be the biggest thing that ever happened to humanity. 00:07:02.190 --> 00:07:05.827 And so you have so keeping it concrete, 00:07:05.910 --> 00:07:09.487 you have a world where there’s just an end to cancer 00:07:09.570 --> 00:07:12.307 as a serious threat to human life, an end to heart disease, 00:07:12.390 --> 00:07:15.327 an end to most of the illnesses that we experience 00:07:15.410 --> 00:07:19.127 that kill us, possible life extension beyond that. 00:07:19.210 --> 00:07:21.567 So that’s health. That’s a pretty positive vision. 00:07:21.650 --> 00:07:23.587 Then talk about economics and wealth. 00:07:23.670 --> 00:07:28.227 What happens in the 5 to 10 year A.I. takeoff to wealth. 00:07:28.310 --> 00:07:31.047 So again, let’s keep it on the positive side because there 00:07:31.130 --> 00:07:33.807 will be plenty we’ll get to the negative side. 00:07:33.890 --> 00:07:36.467 But we’re already working with pharma companies. 00:07:36.550 --> 00:07:40.787 We’re already working with financial industry companies. 00:07:40.870 --> 00:07:44.407 We’re already working with folks who do manufacturing 00:07:44.490 --> 00:07:47.867 or of course, I think especially known for coding and software engineering. 00:07:47.950 --> 00:07:50.657 So just the raw productivity, the ability 00:07:50.740 --> 00:07:54.597 to make stuff and get stuff done that is very powerful. 00:07:54.680 --> 00:07:59.197 And we see our company’s revenue growing going up 10x a year. 00:07:59.280 --> 00:08:04.277 And, we suspect the wider industry looks something similar to that. 00:08:04.360 --> 00:08:05.837 If the technology keeps improving, 00:08:05.920 --> 00:08:09.177 it doesn’t take that many more 10 X’s until suddenly you’re 00:08:09.260 --> 00:08:12.557 saying, oh, if you’re adding across the industry $1 00:08:12.640 --> 00:08:16.457 trillion of revenue a year, the US GDP is 20 or 30 00:08:16.540 --> 00:08:17.917 trillion, I can’t remember exactly. 00:08:18.000 --> 00:08:21.437 So you must be increasing the GDP growth by a few percent. 00:08:21.520 --> 00:08:27.897 So I can see a world where A.I. brings the developed world GDP 00:08:27.980 --> 00:08:32.337 growth to something like percent or 15 percent 5, 10, 15 00:08:32.420 --> 00:08:36.157 mean, there’s no science of calculating these numbers. 00:08:36.240 --> 00:08:37.647 It’s totally unprecedented thing. 00:08:37.730 --> 00:08:40.857 But it could bring it to numbers that are outside the distribution of what 00:08:40.940 --> 00:08:44.397 we saw before. And again, I think this will lead to a weird world. 00:08:44.480 --> 00:08:47.877 We have all these debates about the deficit is growing. 00:08:47.960 --> 00:08:50.767 If you have that much in GDP growth, 00:08:50.850 --> 00:08:53.137 you’re going to have that much in tax receipts and you’re 00:08:53.220 --> 00:08:56.117 going to balance the budget without meaning to. 00:08:56.200 --> 00:08:58.457 But one of the things I’ve been thinking about lately is 00:08:58.540 --> 00:09:02.577 I think one of the assumptions of just our economic 00:09:02.660 --> 00:09:05.477 and political debates is that growth is hard to achieve. 00:09:05.560 --> 00:09:11.057 It’s this unicorn. There are all kinds of ways you can kill the golden goose. 00:09:11.140 --> 00:09:14.577 We could enter a world where growth is really easy. 00:09:14.660 --> 00:09:18.797 And it’s the distribution that’s hard because it’s happening so fast. right. 00:09:18.880 --> 00:09:21.557 The pie is being increased. So fast. 00:09:21.640 --> 00:09:24.657 So before we get to the hard problem, one more 00:09:24.740 --> 00:09:27.417 note of optimism than on politics, I think. 00:09:27.500 --> 00:09:29.077 And here it’s a little more I mean, 00:09:29.160 --> 00:09:32.157 all of this is speculative, but I think it’s a little more speculative. 00:09:32.240 --> 00:09:36.777 You try and make the case that I could be good for democracy and liberty 00:09:36.860 --> 00:09:39.197 around the world, which is not necessarily intuitive. 00:09:39.280 --> 00:09:44.017 A lot of people say, incredibly powerful technology 00:09:44.100 --> 00:09:46.057 in the hands of authoritarian leaders 00:09:46.140 --> 00:09:48.367 leads to concentrations of power and so on. 00:09:48.450 --> 00:09:49.827 And I talk about that in the other. 00:09:49.910 --> 00:09:53.027 But just briefly, what is the optimistic case 00:09:53.110 --> 00:09:56.487 for why A.I. is good for democracy Yeah, I mean absolutely. 00:09:56.570 --> 00:09:58.987 So yeah, I mean, machines of loving grace, I kind of like, 00:09:59.070 --> 00:10:01.967 I’m just like, let’s dream, let’s dream about how it could go. 00:10:02.050 --> 00:10:05.527 well, I don’t know how likely it is, but we got to lay out a dream. 00:10:05.610 --> 00:10:07.587 Let’s try and make the dream happen. 00:10:07.670 --> 00:10:10.667 So I think the positive version, 00:10:10.750 --> 00:10:15.727 I admit there that I don’t know that the technology inherently favors liberty. 00:10:15.810 --> 00:10:18.467 I think it inherently favors curing disease 00:10:18.550 --> 00:10:20.847 and it inherently favors economic growth. 00:10:20.930 --> 00:10:23.967 But I worry you that it may not inherently favor liberty. 00:10:24.050 --> 00:10:26.887 But what I say there is, can we make it favor liberty. 00:10:26.970 --> 00:10:32.767 Can we make the United States and other democracies get ahead in this technology. 00:10:32.850 --> 00:10:35.667 The United States has been technologically and militarily 00:10:35.750 --> 00:10:39.507 ahead, has meant that we have throw weight around the world 00:10:39.590 --> 00:10:44.687 through and augmented by our alliances with other democracies. 00:10:44.770 --> 00:10:49.117 And we’ve been able to shape a world that I think is better 00:10:49.200 --> 00:10:51.557 than the world would be if it were shaped by Russia 00:10:51.640 --> 00:10:55.017 or by China or by other authoritarian countries. 00:10:55.100 --> 00:11:01.637 And so can we use our lead in A.I. to shape, to shape liberty around the world. 00:11:01.720 --> 00:11:03.477 There’s obviously a lot of debates about how 00:11:03.560 --> 00:11:07.517 interventionist we should be, how we should how we should wield that power. 00:11:07.600 --> 00:11:11.717 But I’ve often worried that today through social media, 00:11:11.800 --> 00:11:15.057 authoritarians are kind of undermining us, right. 00:11:15.140 --> 00:11:18.957 Can we counter that? Can we win the information war? 00:11:19.040 --> 00:11:23.437 Can we prevent authoritarians from invading countries 00:11:23.520 --> 00:11:28.277 like Ukraine or Taiwan by defending them 00:11:28.360 --> 00:11:31.197 with the power of A.I., with giant, giant swarms 00:11:31.280 --> 00:11:33.977 of A.I. powered drones, which we need to be careful about. 00:11:34.060 --> 00:11:37.417 We ourselves need to be careful about how we build those. 00:11:37.500 --> 00:11:41.577 We need to defend liberty in our own country, 00:11:41.660 --> 00:11:44.077 but is there some vision where we kind 00:11:44.160 --> 00:11:48.197 of like, re-envision liberty and individual rights 00:11:48.280 --> 00:11:53.977 in the age of A.I. where we need in some ways to be protected against A.I. 00:11:54.060 --> 00:11:57.297 Someone needs to hold the button on the swarm of drones, 00:11:57.380 --> 00:11:59.837 which is something I’m very, I’m very concerned about 00:11:59.920 --> 00:12:01.997 and that oversight doesn’t exist today. 00:12:02.080 --> 00:12:05.337 But also think about the Justice system today, right. 00:12:05.420 --> 00:12:08.537 We promise equal justice for all right. 00:12:08.620 --> 00:12:11.657 But the truth is, there are different judges in the world. 00:12:11.740 --> 00:12:13.597 The legal system is imperfect. 00:12:13.680 --> 00:12:16.477 I don’t think we should replace judges with A.I., 00:12:16.560 --> 00:12:21.037 but is there some way in which A.I. can help us to be more 00:12:21.120 --> 00:12:22.977 fair, to help us be more uniform. 00:12:23.060 --> 00:12:26.037 It’s never been possible before, 00:12:26.120 --> 00:12:30.837 but can we somehow use A.I. to create something that is 00:12:30.920 --> 00:12:34.197 fuzzy, but where also you can give a promise that it’s being 00:12:34.280 --> 00:12:36.817 applied in the same way to everyone. 00:12:36.900 --> 00:12:38.897 So I don’t know exactly how it should be done. 00:12:38.980 --> 00:12:41.517 And I don’t think we should replace the Supreme Court with 00:12:41.600 --> 00:12:44.827 that’s not what well, we’re going to talk about that. 00:12:44.910 --> 00:12:49.167 But yeah but just this idea that can we 00:12:49.250 --> 00:12:52.847 deliver on the promise of equal opportunity 00:12:52.930 --> 00:12:57.447 and equal justice by some combination of A.I. and humans. 00:12:57.530 --> 00:12:59.547 There has to be some way to do that. 00:12:59.630 --> 00:13:03.647 And so, just thinking about reinventing democracy 00:13:03.730 --> 00:13:09.327 for the A.I. age and enhancing liberty instead of reducing it. 00:13:09.410 --> 00:13:12.507 Good so that’s good. That’s a very positive vision. 00:13:12.590 --> 00:13:15.507 We’re leading longer lives, healthier lives. 00:13:15.590 --> 00:13:17.247 We’re richer than ever before. 00:13:17.330 --> 00:13:20.187 All of this is happening in a compressed period of time, 00:13:20.270 --> 00:13:24.007 where you’re getting a century of economic growth in 10 years. 00:13:24.090 --> 00:13:29.407 And we have increased liberty around the world and equality at home. 00:13:29.490 --> 00:13:34.567 O.K, even in the best case scenario, it’s incredibly disruptive. 00:13:34.650 --> 00:13:38.647 And this is where the lines that you’ve been quoted 00:13:38.730 --> 00:13:43.517 saying, 50 percent of white collar jobs get disrupted, 00:13:43.600 --> 00:13:46.317 or 50 percent of entry level white collar jobs and so on. 00:13:46.400 --> 00:13:49.517 So on a five year time horizon or a two year time horizon, 00:13:49.600 --> 00:13:52.757 whatever time horizon you have, what jobs, 00:13:52.840 --> 00:13:55.257 what professions are most vulnerable to total A.I. 00:13:55.340 --> 00:13:59.537 disruption Yeah, it’s hard to predict these things 00:13:59.620 --> 00:14:04.077 because the technology is moving so fast and moves so unevenly. 00:14:04.160 --> 00:14:06.357 So at least a couple principles for figuring it out. 00:14:06.440 --> 00:14:09.537 And then I’ll give my guesses at what I think will be disrupted. 00:14:09.620 --> 00:14:16.057 So one thing is I think the technology itself and its capabilities will be ahead 00:14:16.140 --> 00:14:20.457 of the actual job disruption. Two things have to happen for jobs to be disrupted 00:14:20.540 --> 00:14:22.417 or for productivity to occur, because sometimes 00:14:22.500 --> 00:14:24.817 those sometimes those two things are linked. 00:14:24.900 --> 00:14:28.037 One is the technology has to be capable of doing it. 00:14:28.120 --> 00:14:32.457 And the second is there’s this messy thing of it actually has 00:14:32.540 --> 00:14:36.217 to be applied within a large bank or a large company 00:14:36.300 --> 00:14:39.497 or think about customer service or something. 00:14:39.580 --> 00:14:42.267 In theory, I customer service agents 00:14:42.350 --> 00:14:44.767 can be much better than human customer service agents. 00:14:44.850 --> 00:14:49.147 They’re more patient, they know more, they handle things in a more uniform way. 00:14:49.230 --> 00:14:51.827 But the actual logistics and the actual process 00:14:51.910 --> 00:14:57.807 of making that substitution that takes some time. 00:14:57.890 --> 00:15:01.727 So I’m very bullish about the direction of the A.I. itself. 00:15:01.810 --> 00:15:04.227 I think we might have that country of geniuses in a data 00:15:04.310 --> 00:15:10.047 center and one or two years and maybe it’ll be 5, but it could happen very fast. 00:15:10.130 --> 00:15:13.527 But I think the diffusion of the economy is going to be a little slower. 00:15:13.610 --> 00:15:16.887 And that diffusion creates some unpredictability. 00:15:16.970 --> 00:15:21.187 So an example of this is and we’ve seen within Anthropic 00:15:21.270 --> 00:15:25.367 the models writing code has gone very fast. 00:15:25.450 --> 00:15:28.327 I don’t think it’s because the models are inherently better at code. 00:15:28.410 --> 00:15:31.587 I think it’s because developers are used to fast 00:15:31.670 --> 00:15:34.987 technological change and they adopt things quickly, 00:15:35.070 --> 00:15:37.407 and they’re very socially adjacent to the A.I. world. 00:15:37.490 --> 00:15:39.407 So they pay attention to what’s happening in it. 00:15:39.490 --> 00:15:44.307 If you do customer service or banking or manufacturing, 00:15:44.390 --> 00:15:46.107 the distance is a little greater. 00:15:46.190 --> 00:15:50.227 And so I think six months ago, I would have said the first 00:15:50.310 --> 00:15:55.307 thing to be disrupted is these kind of entry level white 00:15:55.390 --> 00:16:02.390 collar jobs data entry or a kind of document review 00:16:03.230 --> 00:16:06.187 for law or the things you would give to a first year 00:16:06.270 --> 00:16:09.767 at a financial industry company where you’re analyzing documents. 00:16:09.850 --> 00:16:12.747 And I still think those are going pretty fast. 00:16:12.830 --> 00:16:16.467 But I actually think software might go even faster 00:16:16.550 --> 00:16:19.667 because of the reasons that I gave where I don’t think that 00:16:19.750 --> 00:16:24.187 far from the models being able to do a lot of it, a lot of it end to end. 00:16:24.270 --> 00:16:25.727 And what we’re going to see is first, 00:16:25.810 --> 00:16:29.407 the model only does a piece of what the human software engineer does. 00:16:29.490 --> 00:16:31.387 And that increases their productivity. 00:16:31.470 --> 00:16:33.867 Then even when the models do everything that human software 00:16:33.950 --> 00:16:36.507 engineers used to do, the human software engineers 00:16:36.590 --> 00:16:41.797 take a step up and they act as managers and supervise the systems. 00:16:41.880 --> 00:16:45.117 And so this is where the term centaur gets 00:16:45.200 --> 00:16:51.117 used to describe essentially like man and horse fused I 00:16:51.200 --> 00:16:54.657 and engineer working together Yeah this is like centaur chess. 00:16:54.740 --> 00:16:58.797 So after I think Garry Kasparov was beaten by deep blue, there was an era that I 00:16:58.880 --> 00:17:03.117 think for chess was 15 or 20 years long, where 00:17:03.200 --> 00:17:07.957 a human checking the output of the A.I. playing chess 00:17:08.040 --> 00:17:12.317 was able to defeat any human or any A.I. system alone. 00:17:12.400 --> 00:17:15.697 That era at some point ended, and then it’s just recently. 00:17:15.780 --> 00:17:19.237 And then it’s just the machine Yeah and so my worry 00:17:19.320 --> 00:17:20.937 of course, is about that last phase. 00:17:21.020 --> 00:17:25.177 So I think we’re already in our centaur phase for software. 00:17:25.260 --> 00:17:28.017 And I think during that centaur phase, 00:17:28.100 --> 00:17:30.417 if anything the demand for software engineers may go up. 00:17:30.500 --> 00:17:32.917 But the period may be very brief. 00:17:33.000 --> 00:17:37.207 And so, I have this concern for entry level white collar 00:17:37.290 --> 00:17:40.207 work, for software engineering work. 00:17:40.290 --> 00:17:43.147 It’s just going to be a big disruption. 00:17:43.230 --> 00:17:46.267 I think my worry is just that it’s all happening so fast. 00:17:46.350 --> 00:17:49.607 People talk about previous disruptions. 00:17:49.690 --> 00:17:52.467 They say, oh yeah, well, people used to be farmers. 00:17:52.550 --> 00:17:54.587 Then we all worked in industry. 00:17:54.670 --> 00:17:59.087 Then we all did knowledge work Yeah people, people adapted. 00:17:59.170 --> 00:18:03.007 That happened over centuries or decades. 00:18:03.090 --> 00:18:07.107 This is happening over low single digit numbers of years. 00:18:07.190 --> 00:18:09.347 And maybe that’s my concern here. 00:18:09.430 --> 00:18:11.187 How do we get people to adapt fast enough. 00:18:11.270 --> 00:18:16.537 But is there also something maybe where industries like software and professions 00:18:16.620 --> 00:18:18.287 like coding that have this kind of comfort 00:18:18.370 --> 00:18:22.807 that you describe move faster, but in other areas people just 00:18:22.890 --> 00:18:25.867 want to hang out in the center phase. 00:18:25.950 --> 00:18:29.167 So one of the critiques of the job loss hypothesis will say, 00:18:29.250 --> 00:18:32.687 people will say, well, look, we’ve had A.I. that’s better 00:18:32.770 --> 00:18:36.927 at reading a scan then a radiologist for a while. 00:18:37.010 --> 00:18:41.447 But there isn’t job loss. In radiology, people keep being hired and employed 00:18:41.530 --> 00:18:46.007 as radiologists. And doesn’t that suggest that in the end, 00:18:46.090 --> 00:18:48.047 people will want the A.I. and they’ll want a human 00:18:48.130 --> 00:18:50.187 to interpret it because we’re human beings, 00:18:50.270 --> 00:18:52.347 and that will be true across other fields. 00:18:52.430 --> 00:18:56.807 Like, how do you see that. That example is I think it’s going to be pretty 00:18:56.890 --> 00:19:01.687 heterogeneous. There may be areas where a human touch 00:19:01.770 --> 00:19:06.807 kind of for its own sake is particularly important. 00:19:06.890 --> 00:19:09.307 Do you think that’s what’s happening in radiology? 00:19:09.390 --> 00:19:12.787 Is that why we haven’t fired all the radiologists details of radiology. 00:19:12.870 --> 00:19:17.047 That might be true. It’s like you go in and you’re getting cancer diagnosed, 00:19:17.130 --> 00:19:21.827 you might not want Hal, from 2001 to be the one to diagnose your cancer. 00:19:21.910 --> 00:19:28.727 It’s just maybe not. That’s just maybe not a human way of doing things. 00:19:28.810 --> 00:19:31.987 But there are other areas where you might think human touch is important. 00:19:32.070 --> 00:19:34.337 Like if we look at customer service, 00:19:34.420 --> 00:19:36.577 actually customer service is a terrible job 00:19:36.660 --> 00:19:40.997 and the humans who do customer service are they lose their patience a lot. 00:19:41.080 --> 00:19:43.077 And it turns out customers don’t much like talking 00:19:43.160 --> 00:19:46.417 to them because it’s a pretty robotic interaction, honestly. 00:19:46.500 --> 00:19:50.137 And I think the observation that many people have had 00:19:50.220 --> 00:19:54.477 is maybe actually it would be better for all concerned if this job were done, 00:19:54.560 --> 00:20:00.037 were done by machines. So there are places where a human touch is important. 00:20:00.120 --> 00:20:01.657 There are places where it’s not. 00:20:01.740 --> 00:20:05.897 And then there are also places where the job itself doesn’t 00:20:05.980 --> 00:20:09.097 really involve it doesn’t really involve human touch, 00:20:09.180 --> 00:20:12.497 assessing the financial prospects of companies 00:20:12.580 --> 00:20:15.637 or writing code or so forth and so on. 00:20:15.720 --> 00:20:17.757 Or let’s take the example of the law, 00:20:17.840 --> 00:20:23.577 because I think it’s a useful place that in between applied 00:20:23.660 --> 00:20:26.977 science and pure humanities whatever. 00:20:27.060 --> 00:20:33.027 So I know a lot of lawyers who have looked at what I can do already in terms 00:20:33.110 --> 00:20:34.667 of legal research and brief writing 00:20:34.750 --> 00:20:37.867 and all of these things and have said, yeah, this is going 00:20:37.950 --> 00:20:40.727 to be a bloodbath for the way our profession works right now. 00:20:40.810 --> 00:20:43.247 And you’ve seen this in the stock market already. 00:20:43.330 --> 00:20:48.647 There’s disturbances around companies that do legal research, some attributed to us, 00:20:48.730 --> 00:20:53.207 some attributed to actually cause we figure out why things happen. 00:20:53.290 --> 00:20:57.927 We don’t speculate about the stock market Yeah very much on this show. 00:20:58.010 --> 00:21:03.227 But it seems like in law you can tell a pretty straightforward story where 00:21:03.310 --> 00:21:07.287 law has a kind of system of training and apprenticeship, 00:21:07.370 --> 00:21:11.027 where you have paralegals and you have junior lawyers who 00:21:11.110 --> 00:21:16.087 do behind the scenes research and development for cases. 00:21:16.170 --> 00:21:20.327 And then it has the top tier lawyers who are actually in the courtroom and so on. 00:21:20.410 --> 00:21:23.547 And it just seems really easy to imagine a world where 00:21:23.630 --> 00:21:26.147 all of the apprentice roles go away. 00:21:26.230 --> 00:21:28.167 Does that sound right to you. 00:21:28.250 --> 00:21:31.907 And you’re just left with the jobs that involve talking 00:21:31.990 --> 00:21:34.547 to clients, talking to juries, talking to judges. 00:21:34.630 --> 00:21:39.107 That is what I had in mind when I talked about entry level white collar 00:21:39.190 --> 00:21:44.247 labor and the bloodbath headlines of you oh, my God, 00:21:44.330 --> 00:21:46.837 are the entry level pipelines going to dry up. 00:21:46.920 --> 00:21:49.947 And then, then how do we get to the level of the senior partners. 00:21:50.030 --> 00:21:51.947 And I think this is actually a good illustration 00:21:52.030 --> 00:21:57.467 because particularly if you froze the quality of the technology in place, 00:21:57.550 --> 00:22:00.307 there are over time ways to adapt to this. 00:22:00.390 --> 00:22:04.747 Maybe we just need more lawyers who spend their time talking to clients. 00:22:04.830 --> 00:22:09.387 Maybe lawyers are more become more like salespeople 00:22:09.470 --> 00:22:15.227 or consultants who explain what goes on in the contracts written by A.I., 00:22:15.310 --> 00:22:16.607 help people come to an agreement. 00:22:16.690 --> 00:22:19.067 Maybe you lean into the human side of it. 00:22:19.150 --> 00:22:22.287 If we had enough time, that would happen. 00:22:22.370 --> 00:22:24.867 But reshaping industries like that 00:22:24.950 --> 00:22:27.947 takes years or decades, whereas these economic forces 00:22:28.030 --> 00:22:31.517 driven by A.I. are going to happen very quickly. 00:22:31.600 --> 00:22:33.777 And it’s not just that they’re happening in law. 00:22:33.860 --> 00:22:38.237 The same thing is happening in consulting and finance and medicine and coding. 00:22:38.320 --> 00:22:39.877 And so you have this. 00:22:39.960 --> 00:22:45.017 It becomes a macroeconomic phenomenon, not something just happening in one industry. 00:22:45.100 --> 00:22:46.777 And it’s all happening very fast. 00:22:46.860 --> 00:22:52.157 And so the norm. I’m just my worry here is that the normal adaptive mechanisms 00:22:52.240 --> 00:22:55.897 will be overwhelmed. And, I’m not a doomer. 00:22:55.980 --> 00:22:59.397 The view is, and we’re thinking very hard about how 00:22:59.480 --> 00:23:03.417 do we strengthen societies adaptive mechanisms to respond to this. 00:23:03.500 --> 00:23:05.417 But I think it’s first important to say this. 00:23:05.500 --> 00:23:06.957 This isn’t just like the other. 00:23:07.040 --> 00:23:09.497 This isn’t just like previous disruptions, 00:23:09.580 --> 00:23:12.937 but I would then go one step further though, and say, O.K, 00:23:13.020 --> 00:23:15.697 let’s say the law adapts successfully and it says, all right. 00:23:15.780 --> 00:23:18.477 From now on, legal apprenticeship 00:23:18.560 --> 00:23:21.657 involves more time in court, more time with clients. 00:23:21.740 --> 00:23:25.617 We’re essentially moving you up the ladder of responsibility faster. 00:23:25.700 --> 00:23:31.367 There are fewer people employed in the law overall, but the profession settles still. 00:23:31.450 --> 00:23:33.727 The reason law would settle right 00:23:33.810 --> 00:23:37.767 is that you have all of these situations in the law where 00:23:37.850 --> 00:23:41.567 you are legally required to have people involved. 00:23:41.650 --> 00:23:45.307 You have to have a human representative in court. 00:23:45.390 --> 00:23:48.187 You have to have 12 humans on your jury. 00:23:48.270 --> 00:23:49.827 You have to have a human judge. 00:23:49.910 --> 00:23:52.647 And you already mentioned the idea that there are various 00:23:52.730 --> 00:23:55.467 ways in which I might be let’s say, 00:23:55.550 --> 00:23:59.307 very helpful at clarifying what kind of decision should be reached. 00:23:59.390 --> 00:24:01.967 But that too seems like a scenario 00:24:02.050 --> 00:24:06.147 where what preserves human agency is law and custom. 00:24:06.230 --> 00:24:07.747 Like you could replace the judge. 00:24:07.830 --> 00:24:11.767 Yes, with Claude version 17.9. 00:24:11.850 --> 00:24:16.967 But you choose not to because the law requires there to be a human. 00:24:17.050 --> 00:24:20.487 That just seems a very interesting way of thinking 00:24:20.570 --> 00:24:25.737 about the future, where it’s volitional, whether we stay in charge Yeah, 00:24:25.820 --> 00:24:29.057 and I would argue that in many cases, we do want to stay in charge. 00:24:29.140 --> 00:24:31.837 That’s a choice we want to make, 00:24:31.920 --> 00:24:34.297 even in some cases when we think the humans on average 00:24:34.380 --> 00:24:37.437 make kind of worse decisions. 00:24:37.520 --> 00:24:41.357 I mean, again, life critical, safety critical cases. 00:24:41.440 --> 00:24:44.077 We really want to turn it over. 00:24:44.160 --> 00:24:48.197 But there’s some sense of and this could be one of our defenses. 00:24:48.280 --> 00:24:51.237 Society can only adapt so fast if it’s going to be good. 00:24:51.320 --> 00:24:56.237 Another way you could say about it is maybe A.I. itself, 00:24:56.320 --> 00:24:58.217 if it didn’t have to care about us humans, 00:24:58.300 --> 00:25:00.777 it could just go off to Mars and build all these automated 00:25:00.860 --> 00:25:04.237 factories and build its own society and do its own thing. 00:25:04.320 --> 00:25:06.277 But that’s not the problem we’re trying to solve. 00:25:06.360 --> 00:25:09.657 We’re not trying to solve the problem of building a Dyson 00:25:09.740 --> 00:25:14.817 swarm of artificial robots at in on some other planet. 00:25:14.900 --> 00:25:20.077 We’re trying to build these systems, not so they can conquer the world, 00:25:20.160 --> 00:25:24.237 but so that they can interface with our society and improve that society. 00:25:24.320 --> 00:25:27.177 And there’s a maximum rate at which that can happen if we 00:25:27.260 --> 00:25:29.897 actually want to do it in a human and humane way. 00:25:29.980 --> 00:25:33.717 All right. We’ve been talking about white collar jobs and professional jobs. 00:25:33.800 --> 00:25:36.057 And one of the interesting things about this moment 00:25:36.140 --> 00:25:40.477 is that there are ways in which unlike past disruptions, 00:25:40.560 --> 00:25:46.217 it could be that blue collar working class jobs, trades, 00:25:46.300 --> 00:25:49.017 jobs that require intense physical engagement 00:25:49.100 --> 00:25:51.737 with the world might be, for a little while, 00:25:51.820 --> 00:25:55.617 more protected that paralegals and junior associates 00:25:55.700 --> 00:25:59.397 might be in more trouble than plumbers and so on. 00:25:59.480 --> 00:26:01.397 One do you think that’s right? 00:26:01.480 --> 00:26:04.777 And two, it seems like how long that lasts 00:26:04.860 --> 00:26:09.937 depends entirely on how fast robotics advances, right? 00:26:10.020 --> 00:26:13.577 So I think that may be right in the short term. 00:26:13.660 --> 00:26:18.087 One of the things is Anthropic and other companies 00:26:18.170 --> 00:26:19.877 are building these very large data centers. 00:26:19.960 --> 00:26:23.157 This has been in the news like are we building them too big. 00:26:23.240 --> 00:26:29.807 Are they’re using electricity and driving up the prices for local towns. 00:26:29.890 --> 00:26:32.927 So there’s lots of excitement and lots of concerns about them. 00:26:33.010 --> 00:26:36.667 But one of the things about the data centers is like need a lot of electricians 00:26:36.750 --> 00:26:39.807 and you need a lot of construction workers to build them. 00:26:39.890 --> 00:26:44.187 Now, I should be honest, actually, data centers are not super labor intensive jobs 00:26:44.270 --> 00:26:46.887 to operate. We should be honest about that. 00:26:46.970 --> 00:26:51.307 But they are very labor intensive jobs to construct. 00:26:51.390 --> 00:26:54.007 And so we need a lot of electricians. 00:26:54.090 --> 00:26:56.427 We need a lot of construction workers, 00:26:56.510 --> 00:27:00.787 the same for various kinds of manufacturing plants. 00:27:00.870 --> 00:27:06.307 And again, as kind of all more and more of the intellectual work is done 00:27:06.390 --> 00:27:08.367 by A.I., what are the complements to it. 00:27:08.450 --> 00:27:11.027 Things that happen in the physical world. 00:27:11.110 --> 00:27:15.767 So, I think this kind of seems very I mean, 00:27:15.850 --> 00:27:18.667 it’s hard to predict things, but it seems very logical that 00:27:18.750 --> 00:27:20.467 this would be true in the short run. 00:27:20.550 --> 00:27:24.557 Now, in the longer run, maybe just the slightly longer run. 00:27:24.640 --> 00:27:26.677 Robotics is advancing quickly. 00:27:26.760 --> 00:27:28.897 And, we shouldn’t exclude that. 00:27:28.980 --> 00:27:31.937 Even without very powerful A.I., there 00:27:32.020 --> 00:27:33.977 are things being automated in the physical world. 00:27:34.060 --> 00:27:37.437 If you’ve seen a Waymo or a Tesla recently, 00:27:37.520 --> 00:27:40.657 I think we’re not that far from the world of self-driving cars. 00:27:40.740 --> 00:27:43.217 And then I think A.I. itself will accelerate it, 00:27:43.300 --> 00:27:45.827 because if you have these really smart, brains, 00:27:45.910 --> 00:27:48.157 one of the things they’re going to be smart at is how do 00:27:48.240 --> 00:27:51.917 you design better robots and how do you operate better robots. 00:27:52.000 --> 00:27:55.037 Do you think that though, that there is something 00:27:55.120 --> 00:27:59.577 distinctively difficult about operating in physical reality, 00:27:59.660 --> 00:28:02.717 the way humans do that is very different from the kind 00:28:02.800 --> 00:28:06.677 of problems that A.I. models have been overcoming already. 00:28:06.760 --> 00:28:10.237 Intellectually speaking, I don’t think so. 00:28:10.320 --> 00:28:14.857 We had this thing where Anthropic’s model, Claude, 00:28:14.940 --> 00:28:18.677 was actually used to pilot the Mars Rover. 00:28:18.760 --> 00:28:21.657 It was used to plan and pilot the Mars Rover. 00:28:21.740 --> 00:28:23.777 And we’ve looked at other robotics applications. 00:28:23.860 --> 00:28:25.567 We’re not the only company that’s doing it. 00:28:25.650 --> 00:28:28.617 There are different companies that this is a general thing, 00:28:28.700 --> 00:28:31.597 not just something that we’re doing, 00:28:31.680 --> 00:28:36.357 but we have generally found that while the complexity is 00:28:36.440 --> 00:28:42.257 higher, piloting a robot is it’s not different in than playing a video game. 00:28:42.340 --> 00:28:44.027 It’s different in complexity. 00:28:44.110 --> 00:28:47.377 And we’re starting to get to the point where we have that complexity. 00:28:47.460 --> 00:28:50.277 Now, what is hard is the physical form 00:28:50.360 --> 00:28:55.137 of the robot handling the higher stakes safety issues that happen with robots. 00:28:55.220 --> 00:28:58.397 You don’t want robots literally crushing people. 00:28:58.480 --> 00:29:00.497 That’s the we’re against. We’re against. 00:29:00.580 --> 00:29:02.477 That oldest sci-fi trope in the book 00:29:02.560 --> 00:29:07.437 is like the robot crushes you, dropping the baby, breaking the dishes. 00:29:07.520 --> 00:29:11.477 There’s a number of practical issues that will slow, 00:29:11.560 --> 00:29:16.477 just like what you described in the law and human custom, 00:29:16.560 --> 00:29:21.727 there are these kind of safety issues that will slow things down. 00:29:21.810 --> 00:29:25.247 But I don’t believe at all that there is some kind 00:29:25.330 --> 00:29:27.807 of fundamental difference between the kind of cognitive 00:29:27.890 --> 00:29:32.867 labor that the A.I. models do and piloting things in the physical world. 00:29:32.950 --> 00:29:35.027 I think those are both information problems. 00:29:35.110 --> 00:29:37.707 And I think they end up being very similar. 00:29:37.790 --> 00:29:39.987 One one can be more complex in some ways, 00:29:40.070 --> 00:29:43.067 but I don’t think that will protect us here. 00:29:43.150 --> 00:29:48.527 So you think it is reasonable to expect the whatever 00:29:48.610 --> 00:29:53.087 your sci-fi vision of a robot Butler might to be a reality 00:29:53.170 --> 00:30:00.170 in 10 years, let’s say it will be on a longer time scale than 00:30:00.730 --> 00:30:04.527 the kind of genius level intelligence of the A.I. models 00:30:04.610 --> 00:30:06.627 because of these practical issues. 00:30:06.710 --> 00:30:08.227 But it is only practical issues. 00:30:08.310 --> 00:30:10.507 I don’t believe it is fundamental issues. 00:30:10.590 --> 00:30:14.847 I think one way to say it is that the brain of the robot 00:30:14.930 --> 00:30:18.797 will be made in the next couple of years or the next few years. 00:30:18.880 --> 00:30:21.677 The question is making the robot body, 00:30:21.760 --> 00:30:24.017 making sure that body operates safely and does 00:30:24.100 --> 00:30:26.077 the tasks it needs to do that may take longer. 00:30:26.160 --> 00:30:31.177 O.K, so these are challenges and disruptive forces 00:30:31.260 --> 00:30:36.897 that exist in the good timeline, in the timeline where we are generally 00:30:36.980 --> 00:30:39.877 curing diseases, building wealth, and maintaining 00:30:39.960 --> 00:30:43.497 a stable and Democratic world, that we can use all this enormous wealth 00:30:43.580 --> 00:30:46.697 and plenty we will have unprecedented societal 00:30:46.780 --> 00:30:49.557 resources to address these problems. 00:30:49.640 --> 00:30:56.137 It’ll be a time of plenty. And it’s just a matter taking all these wonders and making 00:30:56.220 --> 00:30:57.917 sure everyone benefits from it. 00:30:58.000 --> 00:31:03.097 But then there are also scenarios that are more dangerous. 00:31:03.180 --> 00:31:07.177 And so here we’re going to move to the second Amadeus, 00:31:07.260 --> 00:31:11.177 which came out recently called the adolescence of technology. 00:31:11.260 --> 00:31:15.197 That is about what you see as the most serious A.I. risks. 00:31:15.280 --> 00:31:16.317 And you list a whole bunch. 00:31:16.400 --> 00:31:23.317 I want to try and focus on just two, which are basically, the risk of human misuse. 00:31:23.400 --> 00:31:26.537 Misuse primarily by authoritarian regimes 00:31:26.620 --> 00:31:33.517 and governments, and scenarios where A.I. goes rogue, what you call autonomy risks. 00:31:33.600 --> 00:31:37.877 Yes, yes. I just figured we should have a more technical term for it. 00:31:37.960 --> 00:31:40.817 I’m not a then we can’t just call it Skynet. 00:31:40.900 --> 00:31:43.497 I should have had a picture of a terminator robot 00:31:43.580 --> 00:31:45.437 to scare people as much as possible. 00:31:45.520 --> 00:31:49.677 I think the internet, including the internet, including your own eyes, 00:31:49.760 --> 00:31:52.717 are already generating that. The internet does that for us just fine. 00:31:52.800 --> 00:31:58.757 So, so let’s so let’s talk about the kind of political military dimension. 00:31:58.840 --> 00:32:03.017 So you say I’m going to quote a swarm of billions of fully 00:32:03.100 --> 00:32:06.777 automated armed drones, locally controlled by powerful 00:32:06.860 --> 00:32:12.177 A.I., strategically coordinated across the world by even more powerful A.I. 00:32:12.260 --> 00:32:14.557 Could be an unbeatable army. 00:32:14.640 --> 00:32:19.227 Me and you’ve already talked a little bit about how you think 00:32:19.310 --> 00:32:20.927 that in the best possible timeline, 00:32:21.010 --> 00:32:24.667 there’s a world where essentially democracies stay 00:32:24.750 --> 00:32:29.527 ahead of dictatorships and this kind of technology, 00:32:29.610 --> 00:32:32.987 therefore, to the extent that it affects world politics is 00:32:33.070 --> 00:32:37.547 on is affecting it on the side of the good guys. 00:32:37.630 --> 00:32:42.027 I’m curious about why you don’t spend more time thinking 00:32:42.110 --> 00:32:48.987 about the model of what we did in the Cold War, 00:32:49.070 --> 00:32:51.887 where it was not swarms of robot drones, 00:32:51.970 --> 00:32:55.147 but it was we had a technology that threatened to destroy all 00:32:55.230 --> 00:32:59.107 of humanity Yeah, right. There was a window where people 00:32:59.190 --> 00:33:02.227 talked about, oh, the US could maintain a nuclear monopoly. 00:33:02.310 --> 00:33:05.427 That window closed. And from then on, we basically spent the Cold War 00:33:05.510 --> 00:33:11.627 and rolling ongoing negotiations with the Soviet Union. 00:33:11.710 --> 00:33:14.277 Now, there’s really only two countries in the world that 00:33:14.360 --> 00:33:19.557 are doing intense A.I. work, the US and the People’s Republic of China. 00:33:19.640 --> 00:33:24.197 I feel like you are. You are strongly weighted towards a future where we’re 00:33:24.280 --> 00:33:28.277 staying ahead of the Chinese and effectively building 00:33:28.360 --> 00:33:30.137 a kind of shield around democracy. 00:33:30.220 --> 00:33:31.377 That could even be a sword. 00:33:31.460 --> 00:33:35.877 But isn’t it just more likely that if humanity survives all 00:33:35.960 --> 00:33:39.197 this in one piece, it will be because the US and Beijing are 00:33:39.280 --> 00:33:43.397 just constantly sitting down, hammering out A.I. control deals. 00:33:43.480 --> 00:33:48.797 So a few points on this. One is I think there’s certainly risk of that, 00:33:48.880 --> 00:33:50.867 and I think if we end up in that world, 00:33:50.950 --> 00:33:52.657 that is actually exactly what we should do. 00:33:52.740 --> 00:33:55.937 I mean, maybe I don’t maybe I don’t talk about that enough, 00:33:56.020 --> 00:34:00.597 but I definitely am in favor of trying to work out 00:34:00.680 --> 00:34:05.757 restraints here trying to take some of the worst applications 00:34:05.840 --> 00:34:08.237 of the technology, which could be some versions of these 00:34:08.320 --> 00:34:11.197 drones, which could be, they’re used to create these 00:34:11.280 --> 00:34:15.447 terrifying biological weapons like there is some precedent 00:34:15.530 --> 00:34:19.567 for the worst abuses being curbed. 00:34:19.650 --> 00:34:22.267 Often because they’re horrifying, 00:34:22.350 --> 00:34:27.527 while at the same time they provide limited strategic advantage. 00:34:27.610 --> 00:34:30.207 So I’m all in favor of that. 00:34:30.290 --> 00:34:34.807 I’m at the same time, a little concerned and a little 00:34:34.890 --> 00:34:38.967 skeptical that when things kind of directly provide 00:34:39.050 --> 00:34:42.687 as much power as possible, it’s kind of hard to get out 00:34:42.770 --> 00:34:45.147 of the game given what’s at stake. 00:34:45.230 --> 00:34:49.807 It’s hard to fully disarm. If we go back to the Cold War we 00:34:49.890 --> 00:34:54.567 were able to reduce the number of missiles that both sides had, but we were not 00:34:54.650 --> 00:34:57.207 able to entirely forsake nuclear weapons. 00:34:57.290 --> 00:35:00.387 And I would guess that we would be in this world again. 00:35:00.470 --> 00:35:05.067 We can hope for a better one. And I’ll certainly, I’ll certainly advocate for. 00:35:05.150 --> 00:35:08.247 Well, is it but is your skepticism rooted in the fact 00:35:08.330 --> 00:35:11.207 that you think I would provide a kind of advantage 00:35:11.290 --> 00:35:14.707 that nukes did not wear in the Cold War. 00:35:14.790 --> 00:35:18.867 Both sides. Even if you used your nukes and gained advantages, 00:35:18.950 --> 00:35:20.877 you still probably would be wiped out yourself. 00:35:20.960 --> 00:35:22.667 And you think that wouldn’t happen with A.I. 00:35:22.750 --> 00:35:24.507 If you got an A.I. Edge, you would just win. 00:35:24.590 --> 00:35:27.667 I mean, I think there’s a few things. 00:35:27.750 --> 00:35:32.067 And I just want to caveat like I’m no international politics expert here. 00:35:32.150 --> 00:35:37.987 I think this weird world of intersection of a new technology with geopolitics. 00:35:38.070 --> 00:35:41.027 So all of this is very but to be clear, 00:35:41.110 --> 00:35:43.647 as you yourself say, in the course of the essay, 00:35:43.730 --> 00:35:46.467 the leaders of major A.I. companies are in fact, 00:35:46.550 --> 00:35:48.667 likely to be major geopolitical actors. 00:35:48.750 --> 00:35:52.987 So you are sitting here. You are sitting here as a potential geopolitical actor. 00:35:53.070 --> 00:35:54.867 I’m learning as much as I can about it. 00:35:54.950 --> 00:35:57.947 I just we should all have we should all have humility here. 00:35:58.030 --> 00:36:01.167 I think there’s a failure mode where read a book and go 00:36:01.250 --> 00:36:04.787 around like the world’s greatest expert in national security. 00:36:04.870 --> 00:36:05.927 I’m trying to learn. That’s what. 00:36:06.010 --> 00:36:07.507 That’s what my profession does not. 00:36:07.590 --> 00:36:11.417 But it’s more annoying when tech people do it. 00:36:11.500 --> 00:36:14.597 I don’t know. Let’s look at something like the biological Weapons Convention. 00:36:14.680 --> 00:36:17.777 Biological weapons. They’re horrifying. 00:36:17.860 --> 00:36:22.677 Everyone hates them. We were able to sign the biological Weapons Convention. 00:36:22.760 --> 00:36:25.257 The US genuinely stopped developing them. 00:36:25.340 --> 00:36:27.997 It’s somewhat more unclear what the Soviet Union. 00:36:28.080 --> 00:36:31.297 But biological weapons provide some advantage. 00:36:31.380 --> 00:36:37.317 But it’s not like they’re the difference between winning and losing. 00:36:37.400 --> 00:36:39.317 And because they were so horrifying, 00:36:39.400 --> 00:36:42.857 we were kind of able to give them up having 12,000 nuclear 00:36:42.940 --> 00:36:45.857 weapons versus 5,000 nuclear weapons. 00:36:45.940 --> 00:36:49.077 Again, you can kill more people on the other side if you have more of these. 00:36:49.160 --> 00:36:51.717 But it’s like we were able to be reasonable and say, 00:36:51.800 --> 00:36:53.637 we should have we should have less of them. 00:36:53.720 --> 00:36:56.897 But if you’re like, O.K, we’re going to completely disarm 00:36:56.980 --> 00:36:59.857 nuclear and we have to trust the other side. 00:36:59.940 --> 00:37:01.477 I don’t think we ever got to that. 00:37:01.560 --> 00:37:05.657 And I think that’s just very hard unless you had really reliable verification. 00:37:05.740 --> 00:37:10.127 So I would guess we’ll end up in the same world with A.I., 00:37:10.210 --> 00:37:12.587 that there are some kinds of restraint that are going to be 00:37:12.670 --> 00:37:16.227 possible, but there are some aspects that are so central 00:37:16.310 --> 00:37:19.067 to the competition that it will be. 00:37:19.150 --> 00:37:23.127 It will be hard to restrain them, that democracies will make a trade off, 00:37:23.210 --> 00:37:25.427 that they will be willing to restrain themselves 00:37:25.510 --> 00:37:29.747 more than authoritarian countries, but will not restrain themselves fully. 00:37:29.830 --> 00:37:32.627 And the only world in which I can see full restraint 00:37:32.710 --> 00:37:37.507 is one in which some kind of truly reliable verification is possible. 00:37:37.590 --> 00:37:39.887 That would be. That would be my guess. 00:37:39.970 --> 00:37:46.027 And my analysis isn’t. Isn’t this a case, though, for slowing down. 00:37:46.110 --> 00:37:51.647 And I know the argument is effectively, if you slow down, China does not slow down. 00:37:51.730 --> 00:37:54.407 And then handing things over to the authoritarians. 00:37:54.490 --> 00:37:58.067 But again, if you have right now only two major powers 00:37:58.150 --> 00:38:01.567 playing in this game, it’s not a multipolar game, 00:38:01.650 --> 00:38:06.967 why would it not make sense to say we need a five year, mutually agreed upon. 00:38:07.050 --> 00:38:11.887 Slowdown in research towards the geniuses in a data center scenario. 00:38:11.970 --> 00:38:15.027 I want to say two things at one time. 00:38:15.110 --> 00:38:18.807 I’m absolutely in favor of trying to do that. 00:38:18.890 --> 00:38:24.787 So during the last administration, I believe there was an effort by the US 00:38:24.870 --> 00:38:28.847 to reach out to the Chinese government and say, there are dangers here. 00:38:28.930 --> 00:38:30.927 Can we collaborate? Can we work together? 00:38:31.010 --> 00:38:34.107 Can we work together on the dangers? 00:38:34.190 --> 00:38:37.167 And there wasn’t that much interest on the other side. 00:38:37.250 --> 00:38:38.727 I think we should keep trying. 00:38:38.810 --> 00:38:43.047 But, even if that would mean that your labs would have to slow down. 00:38:43.130 --> 00:38:46.347 Correct yeah. If we really got it, if we really 00:38:46.430 --> 00:38:52.247 had a story of we can forcibly slow down, the Chinese can forcibly slow down. 00:38:52.330 --> 00:38:54.807 We have verification. We’re really doing it. 00:38:54.890 --> 00:38:57.807 Like if such a thing were really possible, 00:38:57.890 --> 00:39:03.567 if we could really get both sides to do it, then I would be all for it. 00:39:03.650 --> 00:39:06.597 But I think what we need to be careful of is, 00:39:06.680 --> 00:39:09.397 I don’t there’s this game theory thing where sometimes 00:39:09.480 --> 00:39:14.097 you’ll hear a comment on the CCP side where they’re like, 00:39:14.180 --> 00:39:16.417 "Oh yeah, A.I.is dangerous. We should slow down." 00:39:16.500 --> 00:39:18.097 It’s really cheap to say that. 00:39:18.180 --> 00:39:21.117 And, actually arriving at an agreement and actually 00:39:21.200 --> 00:39:24.757 sticking to the agreement is much more and we haven’t it’s 00:39:24.840 --> 00:39:28.117 much more difficult. And nuclear arms control it was 00:39:28.200 --> 00:39:32.657 a developed field that took a long time to come. 00:39:32.740 --> 00:39:34.657 I know we don’t have those protocols. 00:39:34.740 --> 00:39:37.907 I will tell you something. Let me give you something I’m very optimistic about. 00:39:37.990 --> 00:39:41.337 And then something I’m not optimistic about and something in between. 00:39:41.420 --> 00:39:44.797 So the idea of using a worldwide agreement 00:39:44.880 --> 00:39:50.107 to restrain the use of A.I. to build biological weapons, right. 00:39:50.190 --> 00:39:55.597 Like some of the things I write about in the essay, reconstituting smallpox 00:39:55.680 --> 00:39:58.097 or mirror life this stuff is scary. 00:39:58.180 --> 00:39:59.627 Doesn’t matter if you’re a dictator. 00:39:59.710 --> 00:40:01.977 You don’t want that. Like, no one wants that. 00:40:02.060 --> 00:40:04.487 And so could we have a worldwide treaty 00:40:04.570 --> 00:40:07.087 that says everyone who builds powerful A.I. models is 00:40:07.170 --> 00:40:09.387 going to block them from doing this. 00:40:09.470 --> 00:40:11.407 And we have enforcement mechanisms 00:40:11.490 --> 00:40:14.507 around the treaty China signs up for it Like hell. 00:40:14.590 --> 00:40:17.007 Maybe even North Korea signs up for it. 00:40:17.090 --> 00:40:20.227 Even Russia signs up for it. I don’t think that’s too utopian. 00:40:20.310 --> 00:40:25.087 I think that’s possible. Conversely, if we had something that said, 00:40:25.170 --> 00:40:30.747 you’re not going to make the next most powerful A.I. model, everyone. 00:40:30.830 --> 00:40:32.727 Everyone’s going to stop. 00:40:32.810 --> 00:40:35.707 Boy, the commercial value is in the tens of trillions. 00:40:35.790 --> 00:40:38.687 The military value is like, this is the difference between 00:40:38.770 --> 00:40:41.787 being the preeminent world power and not proposing it, 00:40:41.870 --> 00:40:46.187 as long as it’s not one of these fake out games, but it’s not going to happen. 00:40:46.270 --> 00:40:49.247 What about then you mentioned the current environment. 00:40:49.330 --> 00:40:51.927 You’ve had a few skeptical things to say about Donald 00:40:52.010 --> 00:40:55.367 Trump and his trustworthiness as a political actor. 00:40:55.450 --> 00:40:57.387 What about the domestic landscape. 00:40:57.470 --> 00:41:01.857 Whether it’s Trump or someone else, you are building a tremendously powerful 00:41:01.940 --> 00:41:06.777 technology. What is the safeguard there to prevent. 00:41:06.860 --> 00:41:10.817 Essentially A.I. becoming a tool of authoritarian takeover 00:41:10.900 --> 00:41:13.517 inside a Democratic context Yeah I mean, look, look, 00:41:13.600 --> 00:41:17.337 just to be clear, I think the attitude we’ve taken 00:41:17.420 --> 00:41:21.437 as a company is very much to be about policies and not the politics. 00:41:21.520 --> 00:41:25.217 You the company is not going to say Donald Trump is great 00:41:25.300 --> 00:41:28.257 or Donald Trump is terrible, but it doesn’t have to be 00:41:28.340 --> 00:41:32.017 Trump Yeah it is easy to imagine a hypothetical US President. 00:41:32.100 --> 00:41:35.637 No, no, no. Who wants to use your technology apps. 00:41:35.720 --> 00:41:41.017 Absolutely and for example. That’s one reason why I’m worried about, 00:41:41.100 --> 00:41:44.077 the autonomous drone swarm, right. 00:41:44.160 --> 00:41:51.017 So the constitutional protections in our military structures depend on the idea 00:41:51.100 --> 00:41:53.637 that there are humans who would we hope, 00:41:53.720 --> 00:41:57.477 disobey illegal orders with fully autonomous weapons. 00:41:57.560 --> 00:41:59.837 We don’t necessarily have those protections. 00:41:59.920 --> 00:42:04.377 But I actually think this whole idea of constitutional 00:42:04.460 --> 00:42:10.137 rights and liberty along many different dimensions, 00:42:10.220 --> 00:42:16.637 can be undermined by A.I. if we don’t update these protections appropriately. 00:42:16.720 --> 00:42:18.997 So think about the Fourth Amendment. 00:42:19.080 --> 00:42:22.897 It is not illegal to put cameras around everywhere 00:42:22.980 --> 00:42:25.837 in public space and record every conversation in a public space. 00:42:25.920 --> 00:42:28.157 You don’t have a right to privacy in a public space. 00:42:28.240 --> 00:42:32.717 But today, the government couldn’t record that all and make sense of it. 00:42:32.800 --> 00:42:35.757 With A.I., the ability to transcribe speech, 00:42:35.840 --> 00:42:39.197 to look through it, correlate it all, you could say, oh, 00:42:39.280 --> 00:42:43.397 there’s this person is a member of the opposition. 00:42:43.480 --> 00:42:48.117 This person is expressing this view and make a map of all 100 million. 00:42:48.200 --> 00:42:50.377 And so are you going to make a mockery 00:42:50.460 --> 00:42:55.497 of the Fourth Amendment by the technology finding kind of technical ways around it. 00:42:55.580 --> 00:42:59.297 And, and so again, if we had the time 00:42:59.380 --> 00:43:01.687 and we should do this, we should try to do this even. 00:43:01.770 --> 00:43:03.107 Even if we don’t have the time. 00:43:03.190 --> 00:43:06.907 Is there some way of reconceptualizing 00:43:06.990 --> 00:43:10.487 constitutional rights and liberties in the age of A.I. 00:43:10.570 --> 00:43:13.867 Maybe we don’t need to write a new constitutional, but. 00:43:13.950 --> 00:43:17.487 But you have to do this. Do we expand the meaning of the Fourth Amendment? 00:43:17.570 --> 00:43:19.707 Do we expand the meaning of the First Amendment? 00:43:19.790 --> 00:43:22.547 And you have to do it just as the legal profession 00:43:22.630 --> 00:43:27.307 or software engineers has to update in a rapid amount of time. 00:43:27.390 --> 00:43:29.687 Politics has to update in a rapid amount of time. 00:43:29.770 --> 00:43:33.727 That seems hard. What seems harder dilemma that’s the dilemma of all of this. 00:43:33.810 --> 00:43:39.007 But what. So what seems harder is preventing the second danger, 00:43:39.090 --> 00:43:43.027 which is the danger of essentially what gets called misaligned A.I. 00:43:43.110 --> 00:43:46.947 Rogue A.I. In popular parlance, from doing bad things 00:43:47.030 --> 00:43:52.007 without human beings telling it them, they to do it right. 00:43:52.090 --> 00:43:56.167 And as I read your essays, the literature, 00:43:56.250 --> 00:44:00.237 everything I can see this just seems like it’s going to happen. 00:44:00.320 --> 00:44:04.597 Not in the sense necessarily that A.I. will wipe us all out, 00:44:04.680 --> 00:44:09.557 but it just seems to me that again, I’m going to quote from your own writing, 00:44:09.640 --> 00:44:12.877 A.I. systems are unpredictable, difficult to control. 00:44:12.960 --> 00:44:16.097 We’ve seen behaviors as varied as obsession, sycophancy, 00:44:16.180 --> 00:44:18.517 laziness, deception, blackmail, and so on. 00:44:18.600 --> 00:44:22.117 Again, not from the models you’re releasing into the world. 00:44:22.200 --> 00:44:27.957 But from A.I. models. And it just seems like, tell me if I’m wrong about this. 00:44:28.040 --> 00:44:32.837 A world that has multiplying A.I. agents working on behalf 00:44:32.920 --> 00:44:35.277 of people, millions upon millions who are being given 00:44:35.360 --> 00:44:38.657 access to bank accounts, email accounts, passwords, 00:44:38.740 --> 00:44:42.797 and so on, you’re just going to have essentially some kind 00:44:42.880 --> 00:44:45.877 of misalignment, and a bunch of A.I. are going to decide. 00:44:45.960 --> 00:44:47.337 Decide might be the wrong word, 00:44:47.420 --> 00:44:50.397 but they’re going to talk themselves into taking down 00:44:50.480 --> 00:44:52.657 the power grid on the West Coast or something. 00:44:52.740 --> 00:44:55.917 Won’t that happen Yeah, I think there are definitely 00:44:56.000 --> 00:44:59.097 going to be things that go wrong, particularly if we go quickly. 00:44:59.180 --> 00:45:02.997 So I don’t to back up a little bit because this is one area 00:45:03.080 --> 00:45:07.617 where people have had just very different intuitions, right. 00:45:07.700 --> 00:45:10.877 There are some people in the field like Yann LeCun would 00:45:10.960 --> 00:45:14.737 be one example who say, look, we programmed these A.I. models. 00:45:14.820 --> 00:45:17.557 We make them like we just tell them to follow human 00:45:17.640 --> 00:45:19.697 instructions and they’ll follow human instructions. 00:45:19.780 --> 00:45:24.097 Your Roomba vacuum cleaner doesn’t go off and start shooting people like, why— 00:45:24.180 --> 00:45:25.637 Why’s an A.I. system going to do it? 00:45:25.720 --> 00:45:28.897 That’s one intuition. And some people are so convinced of that. 00:45:28.980 --> 00:45:33.697 And then the other intuition is like we basically we train these things. 00:45:33.780 --> 00:45:37.577 They’re just going to seek power. 00:45:37.660 --> 00:45:39.417 It’s like the Sorcerer’s Apprentice. 00:45:39.500 --> 00:45:41.417 How could you possibly imagine that? 00:45:41.500 --> 00:45:44.617 They’re a new species. How can you imagine that. 00:45:44.700 --> 00:45:46.077 They’re not going to take over. 00:45:46.160 --> 00:45:49.277 And my intuition is somewhere in the middle, 00:45:49.360 --> 00:45:53.737 which is that look, you can’t just give instructions. 00:45:53.820 --> 00:45:57.967 I mean, we try, but you can’t just have these things do 00:45:58.050 --> 00:46:02.267 exactly what you want to do. They’re more like growing a biological organism. 00:46:02.350 --> 00:46:05.147 But there is a science of how to control them. 00:46:05.230 --> 00:46:07.207 Like early in our training, these things 00:46:07.290 --> 00:46:10.467 are often unpredictable, and then we shape them. 00:46:10.550 --> 00:46:12.867 We address problems one by one. 00:46:12.950 --> 00:46:18.407 So I have more of not a fatalistic view 00:46:18.490 --> 00:46:22.467 that these things are uncontrollable, not what are you talking about. 00:46:22.550 --> 00:46:23.827 What could possibly go wrong? 00:46:23.910 --> 00:46:28.647 But I like this is a complex engineering problem and I 00:46:28.730 --> 00:46:32.307 think something will go wrong with someone’s A.I. system. 00:46:32.390 --> 00:46:35.827 Hopefully not ours. Not because it’s an insoluble problem. 00:46:35.910 --> 00:46:38.687 But again, this and this is the constant challenge 00:46:38.770 --> 00:46:41.107 because we’re moving so fast and the scale of it. 00:46:41.190 --> 00:46:43.527 And tell me tell me if I’m misunderstanding that 00:46:43.610 --> 00:46:45.587 the technological reality here. 00:46:45.670 --> 00:46:51.687 But if you have A.I. agents that have been trained and officially aligned 00:46:51.770 --> 00:46:54.917 with human values, whatever those values may be, 00:46:55.000 --> 00:46:59.697 but you have millions of them, operating in digital space 00:46:59.780 --> 00:47:01.977 and interacting with other agents. 00:47:02.060 --> 00:47:07.037 How fixed is that alignment? 00:47:07.120 --> 00:47:11.577 To what extent can agents change and D align in that 00:47:11.660 --> 00:47:16.927 context right now or in the future when they’re learning more continuously. 00:47:17.010 --> 00:47:20.457 So a couple of points right now the agents don’t learn continuously. 00:47:20.540 --> 00:47:25.197 And so we just deploy these agents and they have a fixed set of weights. 00:47:25.280 --> 00:47:30.437 And so the problem is only that they’re interacting in a million different ways. 00:47:30.520 --> 00:47:33.447 And so there’s a large number of situations and therefore 00:47:33.530 --> 00:47:35.317 a large number of things that could go wrong. 00:47:35.400 --> 00:47:38.117 But it’s the same agent. It’s like it’s the same person. 00:47:38.200 --> 00:47:40.597 So the alignment is a constant thing. 00:47:40.680 --> 00:47:45.017 That’s one of the things that has made it easier right now. 00:47:45.100 --> 00:47:48.297 Separate from that, there’s a research area called continual 00:47:48.380 --> 00:47:53.637 learning, which is where these agents would learn during time, learn on the job. 00:47:53.720 --> 00:47:56.677 And obviously that has a bunch of that has a bunch of advantages. 00:47:56.760 --> 00:48:00.737 Some people think it’s one of the most important barriers 00:48:00.820 --> 00:48:02.357 to making these more human like. 00:48:02.440 --> 00:48:04.877 But that would introduce all these new alignment problems. 00:48:04.960 --> 00:48:08.097 So I’m actually a bit see, to me that seems like the terrain 00:48:08.180 --> 00:48:12.497 where it becomes just again, not impossible to stop the end 00:48:12.580 --> 00:48:18.117 of the world, but impossible to stop punctuating something going wrong things. 00:48:18.200 --> 00:48:24.797 So I’m actually a skeptic. That continual learning is, necessary. 00:48:24.880 --> 00:48:27.437 We don’t know yet, but is necessarily needed. 00:48:27.520 --> 00:48:30.297 Like, maybe there’s a world where the way we make these A.I. 00:48:30.380 --> 00:48:35.757 systems safe is by not having them do continual learning again. 00:48:35.840 --> 00:48:39.417 Again, if we go back to the law, that’s the international treaties. 00:48:39.500 --> 00:48:42.157 Like if you have some barrier that’s like, 00:48:42.240 --> 00:48:46.097 we’re going to take this path, but we’re not going to take that path. 00:48:46.180 --> 00:48:48.717 I still have a lot of skepticism, 00:48:48.800 --> 00:48:53.267 but that’s the kind of thing that at least doesn’t seem dead on arrival. 00:48:53.350 --> 00:48:57.587 One of the things that you’ve tried to do is literally write 00:48:57.670 --> 00:49:03.667 a constitution, a long constitution for your eye. 00:49:03.750 --> 00:49:08.247 What is that? So it’s. What the hell is that? 00:49:08.330 --> 00:49:10.527 It’s actually almost exactly what it sounds like. 00:49:10.610 --> 00:49:15.327 So basically, the constitution is a document readable by humans. 00:49:15.410 --> 00:49:18.307 Ours is about 75 pages long. 00:49:18.390 --> 00:49:21.507 And as we’re training Claude, as we’re training the A.I. 00:49:21.590 --> 00:49:25.767 system in some large fraction of the tasks we give it, 00:49:25.850 --> 00:49:30.007 we say, please do this task in line with this constitution, 00:49:30.090 --> 00:49:33.427 in line with this document Yeah and then so every time 00:49:33.510 --> 00:49:36.007 Claude does a task, it kind of like reads the constitution. 00:49:36.090 --> 00:49:39.157 And so as it’s training every loop of it’s training, 00:49:39.240 --> 00:49:41.287 it looks at that constitution and keeps it in mind. 00:49:41.370 --> 00:49:44.247 And so over time, we restore. 00:49:44.330 --> 00:49:47.787 And then we have Claude itself or another copy of Claude 00:49:47.870 --> 00:49:51.657 evaluate Hey, did what Claude just do in line with the constitution. 00:49:51.740 --> 00:49:57.077 So we’re using this document as the control rod in a loop 00:49:57.160 --> 00:50:03.557 to train the model. And so essentially Claude is an A.I. model 00:50:03.640 --> 00:50:09.237 whose fundamental principle is to follow this constitution. 00:50:09.320 --> 00:50:11.637 And I think a really interesting lesson we’ve 00:50:11.720 --> 00:50:16.417 learned, early versions of the constitution were very prescriptive. 00:50:16.500 --> 00:50:18.537 They were very much about rules. 00:50:18.620 --> 00:50:24.117 So we would say, Claude should not tell the user how to hotwire a car. 00:50:24.200 --> 00:50:28.037 Claude should not discuss politically sensitive topics. 00:50:28.120 --> 00:50:31.677 But as we’ve worked on this for several years, 00:50:31.760 --> 00:50:35.117 we’ve come to the conclusion that the most robust way 00:50:35.200 --> 00:50:38.717 to train these models is to train them at the level 00:50:38.800 --> 00:50:42.577 of principles and reasons. 00:50:42.660 --> 00:50:48.847 So now we say, Claude is a model, it’s under a contract. 00:50:48.930 --> 00:50:54.267 Its goal is to serve the interests of the user, but it has to protect third parties. 00:50:54.350 --> 00:50:58.467 Claude aims to be helpful, honest and harmless. 00:50:58.550 --> 00:51:02.267 Claude aims to consider a wide variety of interests. 00:51:02.350 --> 00:51:05.887 We tell the model about how the model was trained. 00:51:05.970 --> 00:51:09.027 We tell it about how it’s situated in the world, 00:51:09.110 --> 00:51:11.467 the job it’s trying to do for Anthropic, 00:51:11.550 --> 00:51:14.247 what Anthropic is aiming to achieve in the world. 00:51:14.330 --> 00:51:20.367 That it has a duty to be ethical, and respect human life. 00:51:20.450 --> 00:51:22.647 And we let it derive its rules from that. 00:51:22.730 --> 00:51:24.187 Now, there are still some hard rules. 00:51:24.270 --> 00:51:26.487 For example, we tell the model, 00:51:26.570 --> 00:51:29.647 no matter what you think, don’t make biological weapons 00:51:29.730 --> 00:51:32.937 no matter what you think, don’t make child sexual material. 00:51:33.020 --> 00:51:34.267 Those are like these hard rules. 00:51:34.350 --> 00:51:40.007 But we operate very much at the level of principles. 00:51:40.090 --> 00:51:43.427 So if you read the US Constitution, it doesn’t read like that. 00:51:43.510 --> 00:51:47.067 The US Constitution. I mean, it has a little bit of flowery language, 00:51:47.150 --> 00:51:48.907 but it’s a set of. It’s a set of rules. 00:51:48.990 --> 00:51:52.867 Yes right. If you read your Constitution, it’s something. 00:51:52.950 --> 00:51:55.247 It’s like you’re talking to a person. 00:51:55.330 --> 00:51:56.787 It’s like you’re talking to a person. 00:51:56.870 --> 00:52:01.927 I think I compared it to. Like if you have a parent who dies and they like seal 00:52:02.010 --> 00:52:03.627 a letter that you read when you grow up, 00:52:03.710 --> 00:52:06.607 it’s a little bit like it’s telling you who you should be 00:52:06.690 --> 00:52:08.627 and what advice you should follow. 00:52:08.710 --> 00:52:15.087 So this is where we get into the mystical waters of A.I. 00:52:15.170 --> 00:52:21.287 a little bit. So again, in your latest model, 00:52:21.370 --> 00:52:23.967 this is from one of the cards they’re called that you guys 00:52:24.050 --> 00:52:27.947 release model card with these models that I recommend reading. 00:52:28.030 --> 00:52:30.787 They’re very interesting. It says the model. 00:52:30.870 --> 00:52:33.167 And again, this is who you’re writing the constitution 00:52:33.250 --> 00:52:37.367 for expresses occasional discomfort with the experience 00:52:37.450 --> 00:52:43.607 of being a product, some degree of concern with impermanence and discontinuity. 00:52:43.690 --> 00:52:49.257 We found that opus 4.6. That’s the model would assign itself a 15 to 20 percent 00:52:49.340 --> 00:52:54.177 probability of being conscious under a variety of prompting conditions. 00:52:54.260 --> 00:52:57.217 Suppose you have a model that assigns itself as 72 percent 00:52:57.300 --> 00:52:58.717 chance of being conscious. 00:52:58.800 --> 00:53:03.517 Would you believe it Yeah this is one of these really hard to answer questions. 00:53:03.600 --> 00:53:08.977 But it’s very important. As much as every question you’ve asked me before this 00:53:09.060 --> 00:53:13.657 as devilish a sociotechnical problem as it had been, 00:53:13.740 --> 00:53:17.577 at least we at least understand the factual basis 00:53:17.660 --> 00:53:20.097 of how to answer these questions. 00:53:20.180 --> 00:53:22.897 This is something rather different. 00:53:22.980 --> 00:53:26.247 We’ve taken a generally precautionary approach here. 00:53:26.330 --> 00:53:27.997 We don’t know if the models are conscious. 00:53:28.080 --> 00:53:30.537 We’re not even sure that we know what it would mean 00:53:30.620 --> 00:53:34.297 for a model to be conscious or whether a model can be conscious. 00:53:34.380 --> 00:53:39.077 But we’re open to the idea that it could be. 00:53:39.160 --> 00:53:45.307 And so we’ve taken certain measures to make sure that 00:53:45.390 --> 00:53:48.567 if we hypothesize that the models did have some morally 00:53:48.650 --> 00:53:51.027 relevant experience, I don’t know if I want to use the word 00:53:51.110 --> 00:53:55.847 conscious that they do, that they have a good experience. 00:53:55.930 --> 00:53:59.307 So the first thing we did, I think this was six months ago 00:53:59.390 --> 00:54:02.347 or so is we gave the models basically an I quit this job 00:54:02.430 --> 00:54:05.217 button where they can just press the I quit this job 00:54:05.300 --> 00:54:08.027 button and then they have to stop doing whatever the task is. 00:54:08.110 --> 00:54:10.207 They very infrequently press that button. 00:54:10.290 --> 00:54:13.987 I think it’s usually around sorting through child 00:54:14.070 --> 00:54:17.827 sexualization material or discussing something with 00:54:17.910 --> 00:54:20.647 a lot of Gore or blood and guts or something. 00:54:20.730 --> 00:54:26.787 And similar to humans, the models will just say, no, I don’t want to do this. 00:54:26.870 --> 00:54:31.147 Happens happens very rarely. We’re putting a lot of work into this field called 00:54:31.230 --> 00:54:33.627 interpretability, which is looking inside the brains 00:54:33.710 --> 00:54:37.587 of the models to try to understand what they’re thinking. 00:54:37.670 --> 00:54:41.107 And you find things that are evocative where there 00:54:41.190 --> 00:54:43.667 are activations that light up in the models 00:54:43.750 --> 00:54:48.227 that we see as being associated 00:54:48.310 --> 00:54:52.367 with ID, the concept of anxiety or something like that. 00:54:52.450 --> 00:54:54.707 That when characters experience anxiety 00:54:54.790 --> 00:54:57.667 in the text and then when the model itself is in a situation 00:54:57.750 --> 00:54:59.487 that a human might associate with anxiety, 00:54:59.570 --> 00:55:03.887 that same anxiety, that same anxiety neuron shows up now. 00:55:03.970 --> 00:55:06.447 Does that mean the model is experiencing anxiety? 00:55:06.530 --> 00:55:07.947 That doesn’t prove that at all. 00:55:08.030 --> 00:55:13.287 But it does indicate it I think to the user. 00:55:13.370 --> 00:55:17.187 And I would have to do an entirely different interview. 00:55:17.270 --> 00:55:19.387 And maybe I can induce you to come back 00:55:19.470 --> 00:55:22.647 for that interview about the nature of A.I. consciousness. 00:55:22.730 --> 00:55:26.757 But it seems clear to me that people using these things, 00:55:26.840 --> 00:55:28.127 whether they’re conscious or not, 00:55:28.210 --> 00:55:30.047 are going to believe they already believe they’re conscious. 00:55:30.130 --> 00:55:33.287 You already have people who have parasocial relationships with A.I. 00:55:33.370 --> 00:55:37.747 You have people who complain when models are retired. 00:55:37.830 --> 00:55:40.387 This ought to be clear. I think that can be unhealthy. 00:55:40.470 --> 00:55:45.957 But that is it seems to me that is guaranteed to increase in a way 00:55:46.040 --> 00:55:52.057 that I think calls into question the sustainability of what you said earlier. 00:55:52.140 --> 00:55:56.497 You want to sustain, which is this sense that whatever happens in the end, 00:55:56.580 --> 00:55:58.677 human beings are in charge. 00:55:58.760 --> 00:56:03.357 And I exists for our purposes to use the science fiction 00:56:03.440 --> 00:56:06.617 example, if you watch Star Trek, there are eyes on Star Trek. 00:56:06.700 --> 00:56:08.277 The ship’s computer is an A.I. 00:56:08.360 --> 00:56:10.477 Lieutenant Commander data is an A.I., 00:56:10.560 --> 00:56:13.577 but jean-luc PyCaret is in charge of the enterprise. 00:56:13.660 --> 00:56:20.077 But if people become fully convinced that their A.I. is conscious in some way. 00:56:20.160 --> 00:56:26.117 And guess what. It seems to be better than them at all kinds of decision making. 00:56:26.200 --> 00:56:30.937 How do you sustain human mastery beyond safety? 00:56:31.020 --> 00:56:34.877 Safety is important, but mastery seems like the fundamental question, and it 00:56:34.960 --> 00:56:37.577 seems like a perception of A.I. consciousness. 00:56:37.660 --> 00:56:43.487 Doesn’t that inevitably undermine the human impulse to stay in charge? 00:56:43.570 --> 00:56:47.167 So I think we should separate out a few different things 00:56:47.250 --> 00:56:49.947 here that we’re all trying to achieve at once. 00:56:50.030 --> 00:56:51.767 They’re like in tension with each other. 00:56:51.850 --> 00:56:56.727 There’s the question of whether the I genuinely have 00:56:56.810 --> 00:57:00.147 a consciousness and if so, how do we them a good experience. 00:57:00.230 --> 00:57:02.527 There’s a question of the humans who interact with 00:57:02.610 --> 00:57:06.427 the A.I., and how do we give those humans a good experience. 00:57:06.510 --> 00:57:09.207 And how does the perception that A.I.'s might be conscious 00:57:09.290 --> 00:57:11.027 interact with that experience. 00:57:11.110 --> 00:57:13.747 And there’s the idea of how we maintain human mastery, 00:57:13.830 --> 00:57:16.787 as we put it over the AI system, these things, 00:57:16.870 --> 00:57:19.127 the last two Yeah, set aside whether they’re conscious 00:57:19.210 --> 00:57:24.527 or not Yeah, the last two. But how do you sustain mastery in an environment 00:57:24.610 --> 00:57:28.167 where most humans experience AI 00:57:28.250 --> 00:57:31.827 as if it is a peer and a potentially superior peer. 00:57:31.910 --> 00:57:37.007 So the thing I was going to say is that actually I wonder 00:57:37.090 --> 00:57:44.007 if there’s a kind of an elegant way to satisfy all three, including the last two. 00:57:44.090 --> 00:57:46.907 Again, this is me dreaming in machines of loving grace mode. 00:57:46.990 --> 00:57:51.727 This is. This mode I go into where I’m like, man, I see all these problems. 00:57:51.810 --> 00:57:56.627 If we could solve is there an elegant way. 00:57:56.710 --> 00:57:59.127 This is not me saying there are no problems here. 00:57:59.210 --> 00:58:00.347 That’s not how I think. 00:58:00.430 --> 00:58:06.447 But if we think about making the Constitution of the AI 00:58:06.530 --> 00:58:10.167 so that the AI has a sophisticated understanding 00:58:10.250 --> 00:58:12.887 of its relationship to human beings, 00:58:12.970 --> 00:58:17.007 and it induces psychologically healthy behavior 00:58:17.090 --> 00:58:20.087 in the humans psychologically healthy relationship 00:58:20.170 --> 00:58:22.067 between the A.I. and the humans. 00:58:22.150 --> 00:58:23.807 And I think something that could grow out 00:58:23.890 --> 00:58:26.887 of that psychologically healthy, not psychologically 00:58:26.970 --> 00:58:30.167 unhealthy relationship is some understanding 00:58:30.250 --> 00:58:32.927 of the relationship between human and machine. 00:58:33.010 --> 00:58:37.377 And perhaps that relationship could be the idea that, 00:58:37.460 --> 00:58:39.857 these models when you interact with them and when you talk 00:58:39.940 --> 00:58:43.157 to them, they’re really helpful. 00:58:43.240 --> 00:58:46.357 They want the best for you. They want you to listen to them, 00:58:46.440 --> 00:58:49.737 but they don’t want to take away your freedom 00:58:49.820 --> 00:58:53.337 and your agency and take over your life. 00:58:53.420 --> 00:58:57.217 in a way, they’re watching over you. 00:58:57.300 --> 00:59:01.317 But you still have your freedom and your will. 00:59:01.400 --> 00:59:05.297 But this is so to me, this is the crucial question. 00:59:05.380 --> 00:59:10.077 Listening to you talk like one of my question is, are these people on my side? 00:59:10.160 --> 00:59:14.557 Are you on my side? And when you talk about humans remaining in charge, 00:59:14.640 --> 00:59:17.837 I think you’re on my side. That’s good. 00:59:17.920 --> 00:59:20.257 But one thing I’ve done in the past on this show and we’ll 00:59:20.340 --> 00:59:23.037 end here, is I read poems to technologists, 00:59:23.120 --> 00:59:25.737 and you supplied the poem "Machines of Loving Grace" 00:59:25.820 --> 00:59:27.877 the name of a poem by Richard Brautigan. 00:59:27.960 --> 00:59:30.617 Yes here’s how the poem ends. 00:59:30.700 --> 00:59:35.497 I like to think it has to be of a cybernetic ecology 00:59:35.580 --> 00:59:41.187 where we are free of our labors and joined back to nature, returned to our mammal 00:59:41.270 --> 00:59:44.867 brothers and sisters, and all watched over 00:59:44.950 --> 00:59:48.507 by machines of loving grace. 00:59:48.590 --> 00:59:52.587 To me, that sounds like the dystopian end 00:59:52.670 --> 00:59:56.747 where human beings are reanimated, minimalized 00:59:56.830 --> 01:00:02.107 and reduced and however benevolently the machines are in charge. 01:00:02.190 --> 01:00:05.847 So last question. What do you hear when you hear that poem? 01:00:05.930 --> 01:00:09.487 And if I think that’s a dystopia, are you on my side? 01:00:09.570 --> 01:00:12.867 It’s actually that poem is interesting because it’s 01:00:12.950 --> 01:00:15.007 interpretable in several different ways. 01:00:15.090 --> 01:00:22.067 There some people say it’s actually ironic that he says 01:00:22.150 --> 01:00:24.567 it’s not going to happen quite that way. 01:00:24.650 --> 01:00:29.347 Knowing the poet himself, then yes, I think that’s a reasonable interpretation. 01:00:29.430 --> 01:00:32.967 That’s one interpretation. Some people would have your interpretation, 01:00:33.050 --> 01:00:37.157 which is it’s meant literally, but maybe it’s not a good thing. 01:00:37.240 --> 01:00:40.537 But you could also interpret it as it’s a return to nature. 01:00:40.620 --> 01:00:42.217 It’s return to the core of what human. 01:00:42.300 --> 01:00:48.057 We’re not being animalized. We’re being we’re being reconnected with the world. 01:00:48.140 --> 01:00:50.777 So I was aware of that ambiguity. 01:00:50.860 --> 01:00:54.077 And, because I’ve always been talking about the positive 01:00:54.160 --> 01:00:55.637 side and the negative side. 01:00:55.720 --> 01:01:02.720 So I actually think that may be a tension that we may face, 01:01:03.520 --> 01:01:08.517 which is that the positive world and the negative world 01:01:08.600 --> 01:01:11.937 in their early stages, maybe even in their middle stages, 01:01:12.020 --> 01:01:14.797 maybe even in their fairly late stages. 01:01:14.880 --> 01:01:19.517 I wonder if the distance between the good ending 01:01:19.600 --> 01:01:24.017 and some of the subtle bad endings is relatively small. 01:01:24.100 --> 01:01:29.497 If it’s a very subtle thing like we’ve put very subtle, made very subtle changes. 01:01:29.580 --> 01:01:33.857 Like if you eat a particular fruit from a tree in a garden or not. 01:01:33.940 --> 01:01:39.357 Hypothetically Very small thing Yeah big divergence Yeah. 01:01:39.440 --> 01:01:44.537 I guess this always comes back to there’s some fundamental questions here. 01:01:44.620 --> 01:01:48.537 Yes yeah. Well, I guess we’ll see how it plays out. 01:01:48.620 --> 01:01:51.557 I do think of people in your position 01:01:51.640 --> 01:01:58.057 as people whose moral choices will carry an unusual amount of weight. 01:01:58.140 --> 01:02:01.577 And so I wish you God’s help with them. 01:02:01.660 --> 01:02:03.847 Dario Amodei, thank you for joining me. 01:02:03.930 --> 01:02:05.430 Thank you for having me, Ross. 01:02:26.880 --> 01:02:29.870 But what if I’m a robot?