1 00:00:00,400 --> 00:00:02,717 Is America going to win the AI race? 2 00:00:02,800 --> 00:00:05,117 >> There are three obvious threats right now. 3 00:00:05,200 --> 00:00:09,437 >> Is China winning the global race to develop artificial intelligence 4 00:00:09,520 --> 00:00:13,517 technology? >> So, how far ahead of China do you think we are? 5 00:00:13,600 --> 00:00:15,037 >> Overall, I would say 6 00:00:15,120 --> 00:00:19,437 >> China put in 172 gawatt of solar last 7 00:00:19,520 --> 00:00:21,597 year, I think is the number. It's 8 00:00:21,680 --> 00:00:24,557 remarkable. We needed in our calculation 9 00:00:24,640 --> 00:00:29,677 by 2030 92 gawatt to be built. A big 10 00:00:29,760 --> 00:00:31,597 nuclear power plant is somewhere between 11 00:00:31,680 --> 00:00:35,197 1 and 1 1/2 gawatt. The country needs 12 00:00:35,280 --> 00:00:39,037 more energy. And if we don't get more energy, we're not going to be able to 13 00:00:39,120 --> 00:00:41,517 fully exploit the lead we have in AI and 14 00:00:41,600 --> 00:00:48,157 AGI. It's very clear. It's probably the case that 15 00:00:48,240 --> 00:00:52,840 >> now that's a moonshot, ladies and gentlemen. 16 00:00:53,200 --> 00:00:57,117 >> Hey everybody, welcome to part two of Eric Schmidt week on moonshots. In this 17 00:00:57,200 --> 00:00:59,517 episode, my moonshot mate Dave Blondon 18 00:00:59,600 --> 00:01:01,837 is interviewing Eric Schmidt about US 19 00:01:01,920 --> 00:01:04,637 versus China and how to avoid crisis 20 00:01:04,720 --> 00:01:06,797 during this period of hyperexponential 21 00:01:06,880 --> 00:01:10,797 growth in AI. A heads up, uh, this this 22 00:01:10,880 --> 00:01:14,477 audio recording from Eric is a little bit choppy. He was on Wi-Fi from his 23 00:01:14,560 --> 00:01:18,877 hotel room, but guarantee you the content is valuable, so please listen 24 00:01:18,960 --> 00:01:22,557 in. And also, this was recorded about a month ago. It took us a while to get the 25 00:01:22,640 --> 00:01:24,797 footage out to all of you. All right, 26 00:01:24,880 --> 00:01:30,680 let's jump in this episode with Dave Blondon and Eric Schmidt. 27 00:01:32,960 --> 00:01:37,117 So, why don't I start you with uh is America going to win the AI race? 28 00:01:37,200 --> 00:01:41,197 Because I know that's a topic that you've spoken on quite a bit. It's also 29 00:01:41,280 --> 00:01:45,837 addressed a little bit in your new book, Genesis, AI, Hope, uh and the human 30 00:01:45,920 --> 00:01:50,077 spirit, uh which hopefully everybody will read, uh with Henry Kissinger, 31 00:01:50,160 --> 00:01:53,437 actually as a co-author. So, uh are we going to win the AI race? And you know 32 00:01:53,520 --> 00:01:55,997 what are the scenarios where we win and lose? 33 00:01:56,080 --> 00:01:59,277 >> It looks like we will. And um and let me 34 00:01:59,360 --> 00:02:01,357 define like 35 00:02:01,440 --> 00:02:03,917 so I think the San Francisco consensus 36 00:02:04,000 --> 00:02:06,717 which is I call it is what people in San 37 00:02:06,800 --> 00:02:11,197 Francisco believe which charity was you 38 00:02:11,280 --> 00:02:14,797 is that you're going to see a build from 39 00:02:14,880 --> 00:02:18,797 current agentic concion to various forms 40 00:02:18,880 --> 00:02:21,117 of recursive self-improvement to 41 00:02:21,200 --> 00:02:24,477 eventual AGI and super intelligence. 42 00:02:24,560 --> 00:02:26,637 Um, in order to do that, of course, I 43 00:02:26,720 --> 00:02:29,277 need to use enormous amount of hardware. 44 00:02:29,360 --> 00:02:32,797 Uh, Google TPUs, uh, the biggest ships and silver, and everybody in the 45 00:02:32,880 --> 00:02:35,917 audience knows that. It sure looks like 46 00:02:36,000 --> 00:02:38,797 the hardware restrictions that the Trump 47 00:02:38,880 --> 00:02:43,357 and Biden administrations have put on China are going to prevent them from 48 00:02:43,440 --> 00:02:47,597 competing at that space. I've been recently in Shanghai for a few days. I 49 00:02:47,680 --> 00:02:51,597 have good relationships with the Chinese and my conclusion is they're they're 50 00:02:51,680 --> 00:02:53,517 they're fighting a different game. 51 00:02:53,600 --> 00:02:55,917 They're ready to adopt AI in every 52 00:02:56,000 --> 00:02:58,877 product, every service, everything, but 53 00:02:58,960 --> 00:03:01,277 in a more classical way. But the merit 54 00:03:01,360 --> 00:03:04,237 is going to seek for AGI. I was quite 55 00:03:04,320 --> 00:03:05,997 worried that we would end up in a super 56 00:03:06,080 --> 00:03:08,317 intelligence race where you would end up 57 00:03:08,400 --> 00:03:11,677 with such enormous gains that one side 58 00:03:11,760 --> 00:03:15,720 would have to actually pre-mp. 59 00:03:18,480 --> 00:03:23,197 It looks like that fear from me was not well grounded. In fact, I think we're 60 00:03:23,280 --> 00:03:25,037 going to be okay for a few years. 61 00:03:25,120 --> 00:03:28,477 >> Really? What What about robotics? We seem to be pretty far behind in that 62 00:03:28,560 --> 00:03:31,917 front. >> Yeah. If if if it's okay for me to be 63 00:03:32,000 --> 00:03:35,437 completely blunt, um the Chinese are 64 00:03:35,520 --> 00:03:37,277 doing the same thing in robotics that 65 00:03:37,360 --> 00:03:39,757 they have done in electronic vehicles. 66 00:03:39,840 --> 00:03:42,557 Um, in case you're confused, while the 67 00:03:42,640 --> 00:03:45,357 current government in the US gets rid of 68 00:03:45,440 --> 00:03:48,477 solar and wind subsidies and promos, 69 00:03:48,560 --> 00:03:52,957 China put in 172 gawatt of solar last 70 00:03:53,040 --> 00:03:55,117 year, I think is the number. It's 71 00:03:55,200 --> 00:03:56,717 remarkable. 72 00:03:56,800 --> 00:04:00,797 So, the the Chinese race around uh solar 73 00:04:00,880 --> 00:04:03,037 and EVs, electronic vehicles of one kind 74 00:04:03,120 --> 00:04:05,517 or another, looks like they've won. 75 00:04:05,600 --> 00:04:09,997 They're using all of those technologies, in particular, new ways of building 76 00:04:10,080 --> 00:04:13,837 stepper motors and other very inexpensive, very powerful, you know, 77 00:04:13,920 --> 00:04:16,317 physical things. Good example is Unitry 78 00:04:16,400 --> 00:04:20,157 just launched the R1 uh for $6,000 79 00:04:20,240 --> 00:04:22,397 available in December. I'm I've ordered 80 00:04:22,480 --> 00:04:24,957 one. We'll see how good it is. But the 81 00:04:25,040 --> 00:04:30,237 arrival of humanoid robots um is likely to be dominated by China. I'm not 82 00:04:30,320 --> 00:04:34,957 suggesting there won't be areas where the US will play. uh the US will still 83 00:04:35,040 --> 00:04:39,597 have spaces of very high-end, very sophisticated stuff. Our software is so 84 00:04:39,680 --> 00:04:43,357 much better than the Chinese software, but at the hardware level, I think you 85 00:04:43,440 --> 00:04:47,037 should assume that the world will be a wash in inexpensive 86 00:04:47,120 --> 00:04:49,357 Chinese robots in the same sense that 87 00:04:49,440 --> 00:04:51,357 there'll be a wash in inexpensive 88 00:04:51,440 --> 00:04:56,637 Chinese electric vehicles. Who knew? And so this is going to be my last question, 89 00:04:56,720 --> 00:05:00,637 but let me bump it to the front since you just beautifully segueed into it. In 90 00:05:00,720 --> 00:05:05,917 the race for AGI, but then also in parallel robotics, uh, China's going to 91 00:05:06,000 --> 00:05:09,757 be miles ahead in electricity production. And in the very short term, 92 00:05:09,840 --> 00:05:13,037 we're all going to be chip constrained, TPU constrained. But if you look 3 four 93 00:05:13,120 --> 00:05:17,117 years out, they, you know, the the fabs are running at full throttle, the chips 94 00:05:17,200 --> 00:05:21,677 are coming out by the millions, then you're suddenly electricity constrained. 95 00:05:21,760 --> 00:05:24,957 uh is there a vulnerability for America there? 96 00:05:25,040 --> 00:05:27,597 >> It's a huge issue. So again, let's use 97 00:05:27,680 --> 00:05:32,797 China versus the US as a metaphor. What are China's strengths? Uh and I'm not 98 00:05:32,880 --> 00:05:36,717 praising China. I'm just trying to report it. They have solved their 99 00:05:36,800 --> 00:05:41,517 electric power problem. They also have full control over social media. So they 100 00:05:41,600 --> 00:05:42,717 don't have the kind of problems that 101 00:05:42,800 --> 00:05:45,677 people here complain about. They have 102 00:05:45,760 --> 00:05:48,717 enormously talented software people and 103 00:05:48,800 --> 00:05:51,597 they don't have enough hardware. 104 00:05:51,680 --> 00:05:53,437 I think that's roughly where they are. 105 00:05:53,520 --> 00:05:56,237 They're also incredible capitalists. Uh 106 00:05:56,320 --> 00:05:59,277 they they call it um Chinese Chinese 107 00:05:59,360 --> 00:06:02,637 socialism with uh socialism with Chinese 108 00:06:02,720 --> 00:06:05,917 characteristics. But trust me, it's pure 109 00:06:06,000 --> 00:06:08,237 raw capitalism. Let's call it what it 110 00:06:08,320 --> 00:06:11,597 is. In the US, we have the many benefits 111 00:06:11,680 --> 00:06:16,157 everybody understands. We do not have enough electricity and our hard and our 112 00:06:16,240 --> 00:06:17,837 at least in our consumer stuff, the 113 00:06:17,920 --> 00:06:20,317 Chinese are likely to beat us. Our 114 00:06:20,400 --> 00:06:23,197 hardware architectures are fantastic and 115 00:06:23,280 --> 00:06:27,517 I'm including um the Amazon chip, obviously the embedded chips, the TPU 116 00:06:27,600 --> 00:06:30,157 v5, which I'm happy to say I was part of 117 00:06:30,240 --> 00:06:33,357 TPU version one. Um all of that stuff is 118 00:06:33,440 --> 00:06:37,197 incredible. So, if you think about it, uh, and I 119 00:06:37,280 --> 00:06:39,437 testified in Congress a month or two ago 120 00:06:39,520 --> 00:06:42,877 on this, we looked at the amount of 121 00:06:42,960 --> 00:06:44,557 electricity required in the United 122 00:06:44,640 --> 00:06:49,837 States to power the expected demand of 123 00:06:49,920 --> 00:06:52,077 data centers and we needed in our 124 00:06:52,160 --> 00:06:55,437 calculation by 2030 125 00:06:55,520 --> 00:06:58,797 92 gawatt to be built. And for 126 00:06:58,880 --> 00:07:01,837 reference, a big nuclear power plant is 127 00:07:01,920 --> 00:07:06,477 somewhere between 1 and 1 and a half gigaww. To give you a sense at how many 128 00:07:06,560 --> 00:07:07,917 nuclear power plants are getting started 129 00:07:08,000 --> 00:07:11,757 in America, effectively zero. So we had 130 00:07:11,840 --> 00:07:14,637 hoped and I've hoped in my lobbying and 131 00:07:14,720 --> 00:07:19,277 testim testifying that the government would fast track 132 00:07:19,360 --> 00:07:21,117 availability of all kinds of 133 00:07:21,200 --> 00:07:24,077 electricity. Um and indeed they have 134 00:07:24,160 --> 00:07:26,877 promoted oil and gas, but they've also 135 00:07:26,960 --> 00:07:30,077 hobbled sto um solar and wind to a 136 00:07:30,160 --> 00:07:32,557 terrible degree, which is an error. The 137 00:07:32,640 --> 00:07:36,077 country needs more energy. And if we 138 00:07:36,160 --> 00:07:39,757 don't get more energy, we're not going to be able to fully exploit the lead we 139 00:07:39,840 --> 00:07:43,037 have in AI and AGI. It's very clear. And 140 00:07:43,120 --> 00:07:46,957 by the way, the obvious next question is what do we do? Well, there are 141 00:07:47,040 --> 00:07:51,037 scenarios. For example, the president went to Saudi and the UAE and did huge 142 00:07:51,120 --> 00:07:55,037 deals for multiple gigawatts. And so we might find ourselves in a situation 143 00:07:55,120 --> 00:07:59,117 where our training for our most important thing, the thing which are the 144 00:07:59,200 --> 00:08:01,357 the essence of America, which is 145 00:08:01,440 --> 00:08:06,157 American intelligence, is actually being developed in kingdoms and that may be 146 00:08:06,240 --> 00:08:09,517 the only fall back we have. >> That is really weird and that begs a 147 00:08:09,600 --> 00:08:13,517 very difficult question. Do you mind if I ask you a a tough one? 148 00:08:13,600 --> 00:08:16,477 >> Go ahead. Uh I've been, you know, uh 149 00:08:16,560 --> 00:08:18,717 kind of uh inspired by you for most of 150 00:08:18,800 --> 00:08:23,837 my adult life and uh I I really I've 151 00:08:23,920 --> 00:08:27,757 been watching your uh podcast where you're talking about look when you 152 00:08:27,840 --> 00:08:30,877 remember when you guest lectured Eric Bolson's class over at Stanford actually 153 00:08:30,960 --> 00:08:33,277 you said when you were running Google you felt like you made decisions three 154 00:08:33,360 --> 00:08:36,557 times faster than any company on the planet but then when you got into the 155 00:08:36,640 --> 00:08:40,157 federal government you felt like the decision-m was onethird as fast as even 156 00:08:40,240 --> 00:08:43,837 a slow company. So from your point of view, it's like, you know, a tiny 157 00:08:43,920 --> 00:08:47,437 fraction of the pace that we need to move. But you've been saying recently 158 00:08:47,520 --> 00:08:51,517 that, you know, one of the things that's inevitable is some kind of an AI 159 00:08:51,600 --> 00:08:54,397 disaster and we're hoping that it's like 160 00:08:54,480 --> 00:08:57,837 a 100 people that die and not a thousand 161 00:08:57,920 --> 00:09:02,797 or 10,000 or a million or or even 100 million. Uh but, you know, suppose that 162 00:09:02,880 --> 00:09:04,077 it does play out that there's a 163 00:09:04,160 --> 00:09:07,437 catastrophe. That's a wake-up call. What 164 00:09:07,520 --> 00:09:10,157 do you want to do with that wakeup call? 165 00:09:10,240 --> 00:09:14,077 What what's the next move for Eric Schmidt after the wakeup call? 166 00:09:14,160 --> 00:09:18,557 >> Well, so so the background here is that Dr. Kissinger, Henry, and I spent an 167 00:09:18,640 --> 00:09:21,997 awful lot of time talking about the period in the 1950s where he was a key 168 00:09:22,080 --> 00:09:24,717 component of all of these things. 169 00:09:24,800 --> 00:09:29,677 And the so so what he did was he used 170 00:09:29,760 --> 00:09:31,437 the fact that we had used the nuclear 171 00:09:31,520 --> 00:09:35,757 bomb to negotiate over about a 15-year 172 00:09:35,840 --> 00:09:38,557 period a set of treaties that restricted 173 00:09:38,640 --> 00:09:40,637 nuclear proliferation. 174 00:09:40,720 --> 00:09:44,477 Those treaties when they were um 175 00:09:44,560 --> 00:09:47,357 uh when they were negotiated have 176 00:09:47,440 --> 00:09:52,157 allowed us to be alive today. So these were centrally important without but 177 00:09:52,240 --> 00:09:55,517 without without controlling the spread of enriched uranium and the other 178 00:09:55,600 --> 00:09:57,837 secrets we would all be toast literally 179 00:09:57,920 --> 00:10:00,397 because of crazy people and so forth. Is 180 00:10:00,480 --> 00:10:02,717 there an analogous set of things that we 181 00:10:02,800 --> 00:10:07,757 can do? The problem here is or I guess the good news is we're not in a war. We 182 00:10:07,840 --> 00:10:11,117 haven't had a nuclear bomb. We don't have that thing to discuss. So we can 183 00:10:11,200 --> 00:10:15,597 talk about it but governments tend to act reactively. So let me talk. There 184 00:10:15,680 --> 00:10:19,117 are three obvious threats right now which I think are fairly well 185 00:10:19,200 --> 00:10:21,757 understood. The first is misinformation 186 00:10:21,840 --> 00:10:23,437 and the software that we're all 187 00:10:23,520 --> 00:10:27,037 collectively giving people. Um allows 188 00:10:27,120 --> 00:10:31,677 for all sorts of misinformation, fake videos, fake news, what have you. We all 189 00:10:31,760 --> 00:10:33,997 understand this. It's all open source. 190 00:10:34,080 --> 00:10:37,757 That's done. That's a threat to democracies and maybe to dictatorships, 191 00:10:37,840 --> 00:10:42,157 but certainly to democracies. The second one is cyber. And I think one way to 192 00:10:42,240 --> 00:10:45,917 understand cyber is that if you can write code, you can also write cyber 193 00:10:46,000 --> 00:10:49,037 cyber attacks. It's the same logic. And 194 00:10:49,120 --> 00:10:51,437 you have these incredible gains in 195 00:10:51,520 --> 00:10:56,237 software. It's frightening how good these so remember my career is as a 196 00:10:56,320 --> 00:11:00,077 programmer. These things program better than I ever did. It's like shocking, 197 00:11:00,160 --> 00:11:02,957 right? And then the third one is bio. 198 00:11:03,040 --> 00:11:05,197 And I think most people believe that one 199 00:11:05,280 --> 00:11:08,237 of those three will create some kind of 200 00:11:08,320 --> 00:11:11,117 mini crisis that will then cause the 201 00:11:11,200 --> 00:11:15,357 governments to say, "Hang on, let's have a conversation about how to really deal 202 00:11:15,440 --> 00:11:20,317 with the downsides." The upsides are incredible, right? And I want America to 203 00:11:20,400 --> 00:11:23,437 win, and I want us to run as fast as we can. And we're indeed doing that with 204 00:11:23,520 --> 00:11:27,917 the Trump administration, which is great, right? But we have to be aware 205 00:11:28,000 --> 00:11:31,597 that these pos these things are possible. Um the one I'm particularly 206 00:11:31,680 --> 00:11:34,557 worried about is biological and he goes something like this. You take some 207 00:11:34,640 --> 00:11:39,117 existing pathogen and using biological techniques which I won't discuss. You 208 00:11:39,200 --> 00:11:43,117 can modify it enough that it cannot be detected but it's still quite dangerous. 209 00:11:43,200 --> 00:11:45,917 That's an example of a threat. There are many others. 210 00:11:46,000 --> 00:11:48,717 >> Yeah, that's also the easiest I think 211 00:11:48,800 --> 00:11:51,917 which is scary um of of the immediate 212 00:11:52,000 --> 00:11:56,797 you know CBRN threat threats. So cyber, biological, radiological, nuclear. uh 213 00:11:56,880 --> 00:11:59,677 biological is the one you can kind of do in a basement with three people, but 214 00:11:59,760 --> 00:12:01,197 it's also the one that's hardest to 215 00:12:01,280 --> 00:12:04,477 contain. So anyway, uh I hope the theory 216 00:12:04,560 --> 00:12:06,957 is right. So, so then uh how do you deal 217 00:12:07,040 --> 00:12:09,037 with proliferation? You know, the Biden 218 00:12:09,120 --> 00:12:12,637 approach was okay, let's contain AGI to 219 00:12:12,720 --> 00:12:16,397 these five companies, you know, Google being one of them, and then let's say 220 00:12:16,480 --> 00:12:21,117 any model with over 1 26 training flops has to register with the federal 221 00:12:21,200 --> 00:12:25,757 government. And then we'll keep it all contained. Uh, so that all got scrapped 222 00:12:25,840 --> 00:12:29,197 immediately, you know, after the election and got replaced by the new 223 00:12:29,280 --> 00:12:33,037 David Saxs document, which the David Sachs document is much more about how do 224 00:12:33,120 --> 00:12:35,837 we move as quickly as possible and and 225 00:12:35,920 --> 00:12:38,637 win the race. Um, but it doesn't really 226 00:12:38,720 --> 00:12:43,037 address proliferation. And obviously in America, we want startups uh and 227 00:12:43,120 --> 00:12:47,917 researchers to have incredible access to technology and compute. On the other 228 00:12:48,000 --> 00:12:51,997 hand, you know, the three people in a basement making a biological weapon is 229 00:12:52,080 --> 00:12:55,677 is a real scary thing. So, how do you balance those? 230 00:12:55,760 --> 00:12:59,437 >> It turns out if you read the David Saxs President Trump announcement, they're 231 00:12:59,520 --> 00:13:01,917 very clear that they want to continue to 232 00:13:02,000 --> 00:13:04,797 study the security aspects 233 00:13:04,880 --> 00:13:07,037 of AI, especially in a geopolitical way. 234 00:13:07,120 --> 00:13:09,997 And that's code for China versus the US. 235 00:13:10,080 --> 00:13:14,317 And so, they continue in their proposal to fund things involving nation state 236 00:13:14,400 --> 00:13:16,557 attacks and so forth and so on. and I 237 00:13:16,640 --> 00:13:19,517 fully support that kind of stuff. Um, 238 00:13:19,600 --> 00:13:23,757 there's probably a difference on the misinformation social media stuff 239 00:13:23,840 --> 00:13:26,237 between the two, but for example, the 240 00:13:26,320 --> 00:13:30,957 Biden rule was simply that you had to report if you're doing a training run 10 241 00:13:31,040 --> 00:13:34,957 to the 26 or greater. I was part of the group that made that number up. Uh, we 242 00:13:35,040 --> 00:13:38,157 made it up because we had no better number. I'm not suggesting it's the 243 00:13:38,240 --> 00:13:41,197 right number. And the the conclusion you 244 00:13:41,280 --> 00:13:43,757 come to is 245 00:13:43,840 --> 00:13:48,477 it's probably the case that that we know 246 00:13:48,560 --> 00:13:52,237 our government, I don't know this, but I'm guessing knows where the training 247 00:13:52,320 --> 00:13:55,197 runs are going on in in China because of 248 00:13:55,280 --> 00:13:57,757 espionage. And it's probably the case 249 00:13:57,840 --> 00:14:02,957 that the Chinese have espionage on the US knowing where our training runs are. 250 00:14:03,040 --> 00:14:07,517 So I'm not sure the nature of the training run runs is a secret. And 251 00:14:07,600 --> 00:14:09,597 frankly, everybody knows where the data 252 00:14:09,680 --> 00:14:12,557 centers are cuz they're immense. So, uh, 253 00:14:12,640 --> 00:14:15,197 we will see. Another attack on the 1026 254 00:14:15,280 --> 00:14:20,477 is that the training is getting more um more costefficient. If you look at the 255 00:14:20,560 --> 00:14:25,597 moving, they move from something called FP16 to FP8. That means eight precision 256 00:14:25,680 --> 00:14:29,117 floating point. People are now moving to four fourbit floating point, which is 257 00:14:29,200 --> 00:14:31,837 bizarre. It turns out these training 258 00:14:31,920 --> 00:14:35,117 algorithms seem to be quite tolerant for 259 00:14:35,200 --> 00:14:39,997 floating point impre imprecision which is a shock to me. So again we're getting 260 00:14:40,080 --> 00:14:44,157 more efficient in training. This creates more of a proliferation problem. If I 261 00:14:44,240 --> 00:14:48,157 can if I can say in general the proliferation issue I'm not worried 262 00:14:48,240 --> 00:14:50,797 about big companies in big countries 263 00:14:50,880 --> 00:14:52,877 because I can count them. You know 264 00:14:52,960 --> 00:14:57,917 there'll be 10 huge data centers, 10 huge training runs around the world. I'm 265 00:14:58,000 --> 00:15:02,557 much more worried about the open- source groups which can operate in the shadows. 266 00:15:02,640 --> 00:15:03,997 They don't have to solve every problem. 267 00:15:04,080 --> 00:15:05,917 They just have to do one thing well. 268 00:15:06,000 --> 00:15:09,277 They can patch together the open source which is generally available and it's 269 00:15:09,360 --> 00:15:11,437 good enough. If you look at the quality 270 00:15:11,520 --> 00:15:15,357 of DeepS are R are one and now R2 271 00:15:15,440 --> 00:15:18,077 coming. Sure looks like it's in 80 or 272 00:15:18,160 --> 00:15:22,637 90% of these top closed models. The closed models people, which obviously 273 00:15:22,720 --> 00:15:26,877 includes Google, get very defensive over this because they say, "Look, those are 274 00:15:26,960 --> 00:15:30,717 kind of synthetic. They have diffused the models. They've trained our best 275 00:15:30,800 --> 00:15:34,557 information. They're not." All of that is true. They're correct, but they are 276 00:15:34,640 --> 00:15:37,037 nevertheless useful for specific things, 277 00:15:37,120 --> 00:15:41,677 and they could be used in a proliferation issue to do various forms 278 00:15:41,760 --> 00:15:43,917 of cyber and biological attacks. 279 00:15:44,000 --> 00:15:47,357 >> Well, yeah. The current rule of thumb is that distillation and transfer learning 280 00:15:47,440 --> 00:15:51,837 is about 1% of the cost of the original training but gets you to the same 281 00:15:51,920 --> 00:15:55,437 destination. So if you take you know FP4 which you mentioned you know we get an 282 00:15:55,520 --> 00:15:58,717 8x performance boost over FP32. So you 283 00:15:58,800 --> 00:16:03,277 get an 8x on the ch on the you know the the weights and then you've got another 284 00:16:03,360 --> 00:16:07,437 100x on the transfer learning. And it's really hard to draw an analogy to 285 00:16:07,520 --> 00:16:12,157 nuclear or chemical weapons in the past cuz you couldn't kind of shrink them a 286 00:16:12,240 --> 00:16:14,397 thousandx under the covers and get the 287 00:16:14,480 --> 00:16:19,437 same result. But AI is really weird that way. It's it's very compressible, very 288 00:16:19,520 --> 00:16:21,277 fluid. 289 00:16:21,360 --> 00:16:24,637 Uh well, again, we we we will see. Um it 290 00:16:24,720 --> 00:16:27,037 does not look like we have a very good 291 00:16:27,120 --> 00:16:29,517 solution for distillation. 292 00:16:29,600 --> 00:16:32,957 It looks like an opponent of a company 293 00:16:33,040 --> 00:16:35,837 can mask their queries to look like 294 00:16:35,920 --> 00:16:37,917 normal sets of users and then distill 295 00:16:38,000 --> 00:16:40,877 the models. One of my friends has 296 00:16:40,960 --> 00:16:43,677 thought about this a lot and he thinks 297 00:16:43,760 --> 00:16:45,517 that the eventual state in the United 298 00:16:45,600 --> 00:16:48,477 States is that the biggest models will 299 00:16:48,560 --> 00:16:51,597 never be released and that the companies 300 00:16:51,680 --> 00:16:54,557 will distill their own models down for 301 00:16:54,640 --> 00:16:57,197 that reason. That's his opinion. We'll 302 00:16:57,280 --> 00:17:00,157 see if that's true. And this produces a 303 00:17:00,240 --> 00:17:03,037 bizarre outcome where the biggest models 304 00:17:03,120 --> 00:17:05,517 in the United States are closed source 305 00:17:05,600 --> 00:17:07,997 and the biggest models in China are 306 00:17:08,080 --> 00:17:12,637 open- source. And the geopolitical issue there of course is that open source is 307 00:17:12,720 --> 00:17:17,677 free and the closed source models are not free. And so the vast majority of 308 00:17:17,760 --> 00:17:21,037 governments and countries who don't have the kind of money that the west does and 309 00:17:21,120 --> 00:17:24,877 so forth will end up standardizing on Chinese models not because they're 310 00:17:24,960 --> 00:17:28,397 better, but because they're free. And so we'll see if that's true or not. But I 311 00:17:28,480 --> 00:17:31,197 do worry about that. >> Well, this whole topic of distillation 312 00:17:31,280 --> 00:17:35,037 and proliferation is a good segue into a much happier topic which I really want 313 00:17:35,120 --> 00:17:39,757 to use want to use our remaining time on. Uh I cannot tell you how inspiring 314 00:17:39,840 --> 00:17:43,677 you are to the founders around Cambridge, MIT, Harvard, Northeastern 315 00:17:43,760 --> 00:17:47,677 where where all of our talent comes from. Um, in my perfect world, you would 316 00:17:47,760 --> 00:17:51,757 podcast or say something or publish something every single day and then 317 00:17:51,840 --> 00:17:56,477 people could turn off CNBC and just tune into what you say because well, I mean, 318 00:17:56,560 --> 00:18:00,237 aside from it being brilliant, you have access to knowledge that isn't in anyone 319 00:18:00,320 --> 00:18:01,677 else's brain as far as I can tell. 320 00:18:01,760 --> 00:18:05,997 You've got this really, really unique combination of perspectives and and it's 321 00:18:06,080 --> 00:18:08,157 incredibly valuable. So, you know, using 322 00:18:08,240 --> 00:18:11,997 distillation and uh as a starting point, 323 00:18:12,080 --> 00:18:15,837 what founder advice would you give? you know, we're seeing numbers like they're 324 00:18:15,920 --> 00:18:20,157 completely unprecedented at ages of founders that are also unprecedented. I 325 00:18:20,240 --> 00:18:23,757 mean, you saw Sergey and Larry when they were very very young, but now they're 326 00:18:23,840 --> 00:18:27,677 even younger. We're talking 18, 19 years years old now. So, uh, what advice would 327 00:18:27,760 --> 00:18:30,077 you give them? 328 00:18:30,160 --> 00:18:32,077 >> So, I think the the most important thing 329 00:18:32,160 --> 00:18:35,357 to say is that the barrier to entry to 330 00:18:35,440 --> 00:18:38,237 starting a company is effectively zero 331 00:18:38,320 --> 00:18:41,357 now. So, let's think about it. What do 332 00:18:41,440 --> 00:18:45,597 you need to have a company? You can register it online. You need some money 333 00:18:45,680 --> 00:18:49,277 to get started. You have to pay yourself. Um, you don't really need any 334 00:18:49,360 --> 00:18:53,197 programmers. You need a couple people to w to pay 335 00:18:53,280 --> 00:18:55,517 Google or Claude or what have you to uh 336 00:18:55,600 --> 00:19:00,717 to write the code for you. Um, so you're pretty good there. You can use third 337 00:19:00,800 --> 00:19:04,637 party logistics companies if you're building a hardware device and you can 338 00:19:04,720 --> 00:19:09,357 use essentially contract manufacturers to build whatever hardware device. So it 339 00:19:09,440 --> 00:19:13,117 looks to me like for hardware and software the barrier to entry is almost 340 00:19:13,200 --> 00:19:16,237 zero. That sounds great until you 341 00:19:16,320 --> 00:19:17,757 realize that as a result you're now 342 00:19:17,840 --> 00:19:20,477 competing against everyone all the time. 343 00:19:20,560 --> 00:19:23,597 So as an example in my career which has 344 00:19:23,680 --> 00:19:26,557 spanned more than I guess 50 55 years 345 00:19:26,640 --> 00:19:29,277 now doing this stuff the key thing 346 00:19:29,360 --> 00:19:32,237 that's true is the compression of time. 347 00:19:32,320 --> 00:19:34,237 The other thing I would say to founders 348 00:19:34,320 --> 00:19:37,037 is it's really important that everything 349 00:19:37,120 --> 00:19:41,357 you do be learned and not uh specified. 350 00:19:41,440 --> 00:19:43,197 I'm doing a couple startups on my own. 351 00:19:43,280 --> 00:19:47,357 We'll see how well they do. But with them, I say I know nothing. Learn 352 00:19:47,440 --> 00:19:51,517 everything. So you can learn how to support your customers. You can learn 353 00:19:51,600 --> 00:19:55,197 what the customer wants. You can learn how to and learning meaning in the AI 354 00:19:55,280 --> 00:19:56,797 sense of learning. Learning it as part 355 00:19:56,880 --> 00:20:00,157 of uh either uh supervised or 356 00:20:00,240 --> 00:20:02,957 unsupervised training. U and if you take 357 00:20:03,040 --> 00:20:04,797 a learning approach then you build a 358 00:20:04,880 --> 00:20:07,997 system that if it works it will explode 359 00:20:08,080 --> 00:20:10,637 >> because once the learning accelerates 360 00:20:10,720 --> 00:20:13,917 you get into a quasi monopoly position. 361 00:20:14,000 --> 00:20:18,717 So the philosophy of winning goes something like this. Run as fast you can 362 00:20:18,800 --> 00:20:23,117 get there as quickly as you can. Build it around learning and if it takes off 363 00:20:23,200 --> 00:20:25,677 you'll be a hero because once it learns 364 00:20:25,760 --> 00:20:28,557 it learns how to become stronger. Um, 365 00:20:28,640 --> 00:20:33,437 and eventually two or three years, you'll start to have various forms of 366 00:20:33,520 --> 00:20:36,637 reinforcement learning which are self-replicating. 367 00:20:36,720 --> 00:20:40,397 And so then you're likely to get accelerations further. That's the most 368 00:20:40,480 --> 00:20:42,317 likely path for the next trillion dollar 369 00:20:42,400 --> 00:20:44,637 company past the equivalents of 370 00:20:44,720 --> 00:20:47,837 anthropic, open AI, you know, etc. And 371 00:20:47,920 --> 00:20:52,157 so when you're looking at an investment yourself, uh, by the way, the the 372 00:20:52,240 --> 00:20:56,077 learning loops concept that Eric just mentioned on the Moonshots podcast that 373 00:20:56,160 --> 00:20:59,837 we did with Eric, which you can find online, he actually described in detail 374 00:20:59,920 --> 00:21:03,197 the three or four different types of learning loops that he looks for. So 375 00:21:03,280 --> 00:21:06,477 it's definitely worth, you know, it's a lot more than we have time for right 376 00:21:06,560 --> 00:21:08,637 now, but it's really, really brilliant. 377 00:21:08,720 --> 00:21:12,797 So, uh, aside from the actual business plan and the learning loops, what do you 378 00:21:12,880 --> 00:21:16,877 look for in founder dynamics, founder team team members? 379 00:21:16,960 --> 00:21:20,477 Um, it's always the case that the that the 380 00:21:20,560 --> 00:21:22,557 founders are really really smart. 381 00:21:22,640 --> 00:21:24,957 They're very very quick and they're very 382 00:21:25,040 --> 00:21:28,397 interesting to talk to. It's also true 383 00:21:28,480 --> 00:21:32,717 that you need them to be able to hire a network of people like them. 384 00:21:32,800 --> 00:21:36,157 >> And so a simple way is if you talk to them and they seem really interesting 385 00:21:36,240 --> 00:21:40,317 and really dynamic and you find that they're in a network of such people, you 386 00:21:40,400 --> 00:21:44,877 likely have winners. And then what you do is you say to them, show me, don't 387 00:21:44,960 --> 00:21:47,917 tell me the product, which of course is what they want to talk about. Show me 388 00:21:48,000 --> 00:21:52,877 how you're going to build a system that is that goes from zero to infinity. 389 00:21:52,960 --> 00:21:56,237 There's a lot of discussion about 0ero to one. And there it's complete 390 00:21:56,320 --> 00:21:58,237 compression. Get that thing done. But 391 00:21:58,320 --> 00:22:00,237 once you have one, how are you going to 392 00:22:00,320 --> 00:22:03,517 scale? Our industry 393 00:22:03,600 --> 00:22:07,917 makes enormous wealth for the founders when you build a platform that is 394 00:22:08,000 --> 00:22:12,877 scaling. That's the lesson. What is the lesson to offer? Build a platform that 395 00:22:12,960 --> 00:22:14,797 scales if you can't. And a platform is 396 00:22:14,880 --> 00:22:16,877 defined as something which others depend 397 00:22:16,960 --> 00:22:21,277 on that you provide. Right? And the 398 00:22:21,360 --> 00:22:24,877 stronger the platform, the more it is, the more interconnected it is, the 399 00:22:24,960 --> 00:22:28,477 stronger the network lock in. That's just generally true. It was true for 400 00:22:28,560 --> 00:22:32,717 Microsoft. It was true for what I did 20 years ago. It's true today. And I think 401 00:22:32,800 --> 00:22:34,077 if you look, I'll give you an example. 402 00:22:34,160 --> 00:22:37,917 Everyone's looking at where are the economics for the large LLM companies. 403 00:22:38,000 --> 00:22:42,397 They don't have a strong enough network lockin yet, but you could easily imagine 404 00:22:42,480 --> 00:22:45,677 that they would develop it and I'm sure they don't tell me what they're doing, 405 00:22:45,760 --> 00:22:48,877 but I'm sure that they have that in the back of their mind. That's their that's 406 00:22:48,960 --> 00:22:51,757 part of the reasons why their valuations are so high. 407 00:22:51,840 --> 00:22:56,077 >> So, we have one minute remaining and I really want to use it to milk you for a 408 00:22:56,160 --> 00:22:58,237 quote that we can loop on our wall in 409 00:22:58,320 --> 00:23:01,757 the office. Um, and I it would the 410 00:23:01,840 --> 00:23:03,677 perfect quote to me would be about the 411 00:23:03,760 --> 00:23:06,157 importance of this moment in time. And 412 00:23:06,240 --> 00:23:11,357 so many of the founders uh they weren't around when Microsoft could, you know, 413 00:23:11,440 --> 00:23:13,997 Bill Gates could wake up in the morning and decide, hey, I'm going to destroy 414 00:23:14,080 --> 00:23:18,317 Word Perfect today and I'm going to destroy uh Lotus 123 today cuz they just 415 00:23:18,400 --> 00:23:20,797 had that that that power at that time. 416 00:23:20,880 --> 00:23:25,437 And noll was certainly in that crosshairs, too. uh then there was this 417 00:23:25,520 --> 00:23:27,597 magical moment of the internet explosion 418 00:23:27,680 --> 00:23:29,357 where companies like Google and many 419 00:23:29,440 --> 00:23:32,957 many others could thrive but then after 420 00:23:33,040 --> 00:23:36,077 that you know things got static again 421 00:23:36,160 --> 00:23:41,117 and now we're in the most explosive uh time period for opportunity for 422 00:23:41,200 --> 00:23:45,197 entrepreneurs that I've ever experienced in my life but very few people remember 423 00:23:45,280 --> 00:23:47,837 all the way back to the internet 424 00:23:47,920 --> 00:23:52,557 explosion era so I'd love to get your your quote or your thoughts around what 425 00:23:52,640 --> 00:23:56,237 is the importance of this moment in time. 426 00:23:56,320 --> 00:23:58,237 >> So I firmly believe that that the 427 00:23:58,320 --> 00:24:02,317 arrival of nonhuman intelligence, AI 428 00:24:02,400 --> 00:24:05,597 intelligence is at the level of 429 00:24:05,680 --> 00:24:08,397 electricity or the invention of fire, 430 00:24:08,480 --> 00:24:11,197 transportation, etc. in human history. 431 00:24:11,280 --> 00:24:14,317 We are fortunate to be living in a time 432 00:24:14,400 --> 00:24:19,197 of great historical consequence. The next 10 years are probably the 10 years 433 00:24:19,280 --> 00:24:22,717 that will have a greater determination over the next hundred years than 434 00:24:22,800 --> 00:24:24,157 anything before because of the 435 00:24:24,240 --> 00:24:26,877 inventions of these new tools. And the 436 00:24:26,960 --> 00:24:29,837 tools are very very powerful power. And 437 00:24:29,920 --> 00:24:34,317 remember, they're powerful because they they can equal and in some cases surpass 438 00:24:34,400 --> 00:24:38,957 human intelligence. And human intelligence is everything for society. 439 00:24:39,040 --> 00:24:43,517 And so the countries and companies that embrace this non-human intelligence 440 00:24:43,600 --> 00:24:48,157 correctly and aggressively will be the big winners. And the companies and 441 00:24:48,240 --> 00:24:52,557 countries that are slow or let other people do it, they will lose because 442 00:24:52,640 --> 00:24:56,077 that the source of excellence, the source of leadership, the source of 443 00:24:56,160 --> 00:25:00,157 growth, the source of everything, the source of innovation, everything 444 00:25:00,240 --> 00:25:03,837 economic growth comes from the application of intelligence to discover 445 00:25:03,920 --> 00:25:08,877 new things, to solve new problems. We are on a huge course to accelerate that 446 00:25:08,960 --> 00:25:13,960 here in America and I'm very proud to be part of it.