WEBVTT 00:00:00.400 --> 00:00:02.717 Is America going to win the AI race? 00:00:02.800 --> 00:00:05.117 >> There are three obvious threats right now. 00:00:05.200 --> 00:00:09.437 >> Is China winning the global race to develop artificial intelligence 00:00:09.520 --> 00:00:13.517 technology? >> So, how far ahead of China do you think we are? 00:00:13.600 --> 00:00:15.037 >> Overall, I would say 00:00:15.120 --> 00:00:19.437 >> China put in 172 gawatt of solar last 00:00:19.520 --> 00:00:21.597 year, I think is the number. It's 00:00:21.680 --> 00:00:24.557 remarkable. We needed in our calculation 00:00:24.640 --> 00:00:29.677 by 2030 92 gawatt to be built. A big 00:00:29.760 --> 00:00:31.597 nuclear power plant is somewhere between 00:00:31.680 --> 00:00:35.197 1 and 1 1/2 gawatt. The country needs 00:00:35.280 --> 00:00:39.037 more energy. And if we don't get more energy, we're not going to be able to 00:00:39.120 --> 00:00:41.517 fully exploit the lead we have in AI and 00:00:41.600 --> 00:00:48.157 AGI. It's very clear. It's probably the case that 00:00:48.240 --> 00:00:52.840 >> now that's a moonshot, ladies and gentlemen. 00:00:53.200 --> 00:00:57.117 >> Hey everybody, welcome to part two of Eric Schmidt week on moonshots. In this 00:00:57.200 --> 00:00:59.517 episode, my moonshot mate Dave Blondon 00:00:59.600 --> 00:01:01.837 is interviewing Eric Schmidt about US 00:01:01.920 --> 00:01:04.637 versus China and how to avoid crisis 00:01:04.720 --> 00:01:06.797 during this period of hyperexponential 00:01:06.880 --> 00:01:10.797 growth in AI. A heads up, uh, this this 00:01:10.880 --> 00:01:14.477 audio recording from Eric is a little bit choppy. He was on Wi-Fi from his 00:01:14.560 --> 00:01:18.877 hotel room, but guarantee you the content is valuable, so please listen 00:01:18.960 --> 00:01:22.557 in. And also, this was recorded about a month ago. It took us a while to get the 00:01:22.640 --> 00:01:24.797 footage out to all of you. All right, 00:01:24.880 --> 00:01:30.680 let's jump in this episode with Dave Blondon and Eric Schmidt. 00:01:32.960 --> 00:01:37.117 So, why don't I start you with uh is America going to win the AI race? 00:01:37.200 --> 00:01:41.197 Because I know that's a topic that you've spoken on quite a bit. It's also 00:01:41.280 --> 00:01:45.837 addressed a little bit in your new book, Genesis, AI, Hope, uh and the human 00:01:45.920 --> 00:01:50.077 spirit, uh which hopefully everybody will read, uh with Henry Kissinger, 00:01:50.160 --> 00:01:53.437 actually as a co-author. So, uh are we going to win the AI race? And you know 00:01:53.520 --> 00:01:55.997 what are the scenarios where we win and lose? 00:01:56.080 --> 00:01:59.277 >> It looks like we will. And um and let me 00:01:59.360 --> 00:02:01.357 define like 00:02:01.440 --> 00:02:03.917 so I think the San Francisco consensus 00:02:04.000 --> 00:02:06.717 which is I call it is what people in San 00:02:06.800 --> 00:02:11.197 Francisco believe which charity was you 00:02:11.280 --> 00:02:14.797 is that you're going to see a build from 00:02:14.880 --> 00:02:18.797 current agentic concion to various forms 00:02:18.880 --> 00:02:21.117 of recursive self-improvement to 00:02:21.200 --> 00:02:24.477 eventual AGI and super intelligence. 00:02:24.560 --> 00:02:26.637 Um, in order to do that, of course, I 00:02:26.720 --> 00:02:29.277 need to use enormous amount of hardware. 00:02:29.360 --> 00:02:32.797 Uh, Google TPUs, uh, the biggest ships and silver, and everybody in the 00:02:32.880 --> 00:02:35.917 audience knows that. It sure looks like 00:02:36.000 --> 00:02:38.797 the hardware restrictions that the Trump 00:02:38.880 --> 00:02:43.357 and Biden administrations have put on China are going to prevent them from 00:02:43.440 --> 00:02:47.597 competing at that space. I've been recently in Shanghai for a few days. I 00:02:47.680 --> 00:02:51.597 have good relationships with the Chinese and my conclusion is they're they're 00:02:51.680 --> 00:02:53.517 they're fighting a different game. 00:02:53.600 --> 00:02:55.917 They're ready to adopt AI in every 00:02:56.000 --> 00:02:58.877 product, every service, everything, but 00:02:58.960 --> 00:03:01.277 in a more classical way. But the merit 00:03:01.360 --> 00:03:04.237 is going to seek for AGI. I was quite 00:03:04.320 --> 00:03:05.997 worried that we would end up in a super 00:03:06.080 --> 00:03:08.317 intelligence race where you would end up 00:03:08.400 --> 00:03:11.677 with such enormous gains that one side 00:03:11.760 --> 00:03:15.720 would have to actually pre-mp. 00:03:18.480 --> 00:03:23.197 It looks like that fear from me was not well grounded. In fact, I think we're 00:03:23.280 --> 00:03:25.037 going to be okay for a few years. 00:03:25.120 --> 00:03:28.477 >> Really? What What about robotics? We seem to be pretty far behind in that 00:03:28.560 --> 00:03:31.917 front. >> Yeah. If if if it's okay for me to be 00:03:32.000 --> 00:03:35.437 completely blunt, um the Chinese are 00:03:35.520 --> 00:03:37.277 doing the same thing in robotics that 00:03:37.360 --> 00:03:39.757 they have done in electronic vehicles. 00:03:39.840 --> 00:03:42.557 Um, in case you're confused, while the 00:03:42.640 --> 00:03:45.357 current government in the US gets rid of 00:03:45.440 --> 00:03:48.477 solar and wind subsidies and promos, 00:03:48.560 --> 00:03:52.957 China put in 172 gawatt of solar last 00:03:53.040 --> 00:03:55.117 year, I think is the number. It's 00:03:55.200 --> 00:03:56.717 remarkable. 00:03:56.800 --> 00:04:00.797 So, the the Chinese race around uh solar 00:04:00.880 --> 00:04:03.037 and EVs, electronic vehicles of one kind 00:04:03.120 --> 00:04:05.517 or another, looks like they've won. 00:04:05.600 --> 00:04:09.997 They're using all of those technologies, in particular, new ways of building 00:04:10.080 --> 00:04:13.837 stepper motors and other very inexpensive, very powerful, you know, 00:04:13.920 --> 00:04:16.317 physical things. Good example is Unitry 00:04:16.400 --> 00:04:20.157 just launched the R1 uh for $6,000 00:04:20.240 --> 00:04:22.397 available in December. I'm I've ordered 00:04:22.480 --> 00:04:24.957 one. We'll see how good it is. But the 00:04:25.040 --> 00:04:30.237 arrival of humanoid robots um is likely to be dominated by China. I'm not 00:04:30.320 --> 00:04:34.957 suggesting there won't be areas where the US will play. uh the US will still 00:04:35.040 --> 00:04:39.597 have spaces of very high-end, very sophisticated stuff. Our software is so 00:04:39.680 --> 00:04:43.357 much better than the Chinese software, but at the hardware level, I think you 00:04:43.440 --> 00:04:47.037 should assume that the world will be a wash in inexpensive 00:04:47.120 --> 00:04:49.357 Chinese robots in the same sense that 00:04:49.440 --> 00:04:51.357 there'll be a wash in inexpensive 00:04:51.440 --> 00:04:56.637 Chinese electric vehicles. Who knew? And so this is going to be my last question, 00:04:56.720 --> 00:05:00.637 but let me bump it to the front since you just beautifully segueed into it. In 00:05:00.720 --> 00:05:05.917 the race for AGI, but then also in parallel robotics, uh, China's going to 00:05:06.000 --> 00:05:09.757 be miles ahead in electricity production. And in the very short term, 00:05:09.840 --> 00:05:13.037 we're all going to be chip constrained, TPU constrained. But if you look 3 four 00:05:13.120 --> 00:05:17.117 years out, they, you know, the the fabs are running at full throttle, the chips 00:05:17.200 --> 00:05:21.677 are coming out by the millions, then you're suddenly electricity constrained. 00:05:21.760 --> 00:05:24.957 uh is there a vulnerability for America there? 00:05:25.040 --> 00:05:27.597 >> It's a huge issue. So again, let's use 00:05:27.680 --> 00:05:32.797 China versus the US as a metaphor. What are China's strengths? Uh and I'm not 00:05:32.880 --> 00:05:36.717 praising China. I'm just trying to report it. They have solved their 00:05:36.800 --> 00:05:41.517 electric power problem. They also have full control over social media. So they 00:05:41.600 --> 00:05:42.717 don't have the kind of problems that 00:05:42.800 --> 00:05:45.677 people here complain about. They have 00:05:45.760 --> 00:05:48.717 enormously talented software people and 00:05:48.800 --> 00:05:51.597 they don't have enough hardware. 00:05:51.680 --> 00:05:53.437 I think that's roughly where they are. 00:05:53.520 --> 00:05:56.237 They're also incredible capitalists. Uh 00:05:56.320 --> 00:05:59.277 they they call it um Chinese Chinese 00:05:59.360 --> 00:06:02.637 socialism with uh socialism with Chinese 00:06:02.720 --> 00:06:05.917 characteristics. But trust me, it's pure 00:06:06.000 --> 00:06:08.237 raw capitalism. Let's call it what it 00:06:08.320 --> 00:06:11.597 is. In the US, we have the many benefits 00:06:11.680 --> 00:06:16.157 everybody understands. We do not have enough electricity and our hard and our 00:06:16.240 --> 00:06:17.837 at least in our consumer stuff, the 00:06:17.920 --> 00:06:20.317 Chinese are likely to beat us. Our 00:06:20.400 --> 00:06:23.197 hardware architectures are fantastic and 00:06:23.280 --> 00:06:27.517 I'm including um the Amazon chip, obviously the embedded chips, the TPU 00:06:27.600 --> 00:06:30.157 v5, which I'm happy to say I was part of 00:06:30.240 --> 00:06:33.357 TPU version one. Um all of that stuff is 00:06:33.440 --> 00:06:37.197 incredible. So, if you think about it, uh, and I 00:06:37.280 --> 00:06:39.437 testified in Congress a month or two ago 00:06:39.520 --> 00:06:42.877 on this, we looked at the amount of 00:06:42.960 --> 00:06:44.557 electricity required in the United 00:06:44.640 --> 00:06:49.837 States to power the expected demand of 00:06:49.920 --> 00:06:52.077 data centers and we needed in our 00:06:52.160 --> 00:06:55.437 calculation by 2030 00:06:55.520 --> 00:06:58.797 92 gawatt to be built. And for 00:06:58.880 --> 00:07:01.837 reference, a big nuclear power plant is 00:07:01.920 --> 00:07:06.477 somewhere between 1 and 1 and a half gigaww. To give you a sense at how many 00:07:06.560 --> 00:07:07.917 nuclear power plants are getting started 00:07:08.000 --> 00:07:11.757 in America, effectively zero. So we had 00:07:11.840 --> 00:07:14.637 hoped and I've hoped in my lobbying and 00:07:14.720 --> 00:07:19.277 testim testifying that the government would fast track 00:07:19.360 --> 00:07:21.117 availability of all kinds of 00:07:21.200 --> 00:07:24.077 electricity. Um and indeed they have 00:07:24.160 --> 00:07:26.877 promoted oil and gas, but they've also 00:07:26.960 --> 00:07:30.077 hobbled sto um solar and wind to a 00:07:30.160 --> 00:07:32.557 terrible degree, which is an error. The 00:07:32.640 --> 00:07:36.077 country needs more energy. And if we 00:07:36.160 --> 00:07:39.757 don't get more energy, we're not going to be able to fully exploit the lead we 00:07:39.840 --> 00:07:43.037 have in AI and AGI. It's very clear. And 00:07:43.120 --> 00:07:46.957 by the way, the obvious next question is what do we do? Well, there are 00:07:47.040 --> 00:07:51.037 scenarios. For example, the president went to Saudi and the UAE and did huge 00:07:51.120 --> 00:07:55.037 deals for multiple gigawatts. And so we might find ourselves in a situation 00:07:55.120 --> 00:07:59.117 where our training for our most important thing, the thing which are the 00:07:59.200 --> 00:08:01.357 the essence of America, which is 00:08:01.440 --> 00:08:06.157 American intelligence, is actually being developed in kingdoms and that may be 00:08:06.240 --> 00:08:09.517 the only fall back we have. >> That is really weird and that begs a 00:08:09.600 --> 00:08:13.517 very difficult question. Do you mind if I ask you a a tough one? 00:08:13.600 --> 00:08:16.477 >> Go ahead. Uh I've been, you know, uh 00:08:16.560 --> 00:08:18.717 kind of uh inspired by you for most of 00:08:18.800 --> 00:08:23.837 my adult life and uh I I really I've 00:08:23.920 --> 00:08:27.757 been watching your uh podcast where you're talking about look when you 00:08:27.840 --> 00:08:30.877 remember when you guest lectured Eric Bolson's class over at Stanford actually 00:08:30.960 --> 00:08:33.277 you said when you were running Google you felt like you made decisions three 00:08:33.360 --> 00:08:36.557 times faster than any company on the planet but then when you got into the 00:08:36.640 --> 00:08:40.157 federal government you felt like the decision-m was onethird as fast as even 00:08:40.240 --> 00:08:43.837 a slow company. So from your point of view, it's like, you know, a tiny 00:08:43.920 --> 00:08:47.437 fraction of the pace that we need to move. But you've been saying recently 00:08:47.520 --> 00:08:51.517 that, you know, one of the things that's inevitable is some kind of an AI 00:08:51.600 --> 00:08:54.397 disaster and we're hoping that it's like 00:08:54.480 --> 00:08:57.837 a 100 people that die and not a thousand 00:08:57.920 --> 00:09:02.797 or 10,000 or a million or or even 100 million. Uh but, you know, suppose that 00:09:02.880 --> 00:09:04.077 it does play out that there's a 00:09:04.160 --> 00:09:07.437 catastrophe. That's a wake-up call. What 00:09:07.520 --> 00:09:10.157 do you want to do with that wakeup call? 00:09:10.240 --> 00:09:14.077 What what's the next move for Eric Schmidt after the wakeup call? 00:09:14.160 --> 00:09:18.557 >> Well, so so the background here is that Dr. Kissinger, Henry, and I spent an 00:09:18.640 --> 00:09:21.997 awful lot of time talking about the period in the 1950s where he was a key 00:09:22.080 --> 00:09:24.717 component of all of these things. 00:09:24.800 --> 00:09:29.677 And the so so what he did was he used 00:09:29.760 --> 00:09:31.437 the fact that we had used the nuclear 00:09:31.520 --> 00:09:35.757 bomb to negotiate over about a 15-year 00:09:35.840 --> 00:09:38.557 period a set of treaties that restricted 00:09:38.640 --> 00:09:40.637 nuclear proliferation. 00:09:40.720 --> 00:09:44.477 Those treaties when they were um 00:09:44.560 --> 00:09:47.357 uh when they were negotiated have 00:09:47.440 --> 00:09:52.157 allowed us to be alive today. So these were centrally important without but 00:09:52.240 --> 00:09:55.517 without without controlling the spread of enriched uranium and the other 00:09:55.600 --> 00:09:57.837 secrets we would all be toast literally 00:09:57.920 --> 00:10:00.397 because of crazy people and so forth. Is 00:10:00.480 --> 00:10:02.717 there an analogous set of things that we 00:10:02.800 --> 00:10:07.757 can do? The problem here is or I guess the good news is we're not in a war. We 00:10:07.840 --> 00:10:11.117 haven't had a nuclear bomb. We don't have that thing to discuss. So we can 00:10:11.200 --> 00:10:15.597 talk about it but governments tend to act reactively. So let me talk. There 00:10:15.680 --> 00:10:19.117 are three obvious threats right now which I think are fairly well 00:10:19.200 --> 00:10:21.757 understood. The first is misinformation 00:10:21.840 --> 00:10:23.437 and the software that we're all 00:10:23.520 --> 00:10:27.037 collectively giving people. Um allows 00:10:27.120 --> 00:10:31.677 for all sorts of misinformation, fake videos, fake news, what have you. We all 00:10:31.760 --> 00:10:33.997 understand this. It's all open source. 00:10:34.080 --> 00:10:37.757 That's done. That's a threat to democracies and maybe to dictatorships, 00:10:37.840 --> 00:10:42.157 but certainly to democracies. The second one is cyber. And I think one way to 00:10:42.240 --> 00:10:45.917 understand cyber is that if you can write code, you can also write cyber 00:10:46.000 --> 00:10:49.037 cyber attacks. It's the same logic. And 00:10:49.120 --> 00:10:51.437 you have these incredible gains in 00:10:51.520 --> 00:10:56.237 software. It's frightening how good these so remember my career is as a 00:10:56.320 --> 00:11:00.077 programmer. These things program better than I ever did. It's like shocking, 00:11:00.160 --> 00:11:02.957 right? And then the third one is bio. 00:11:03.040 --> 00:11:05.197 And I think most people believe that one 00:11:05.280 --> 00:11:08.237 of those three will create some kind of 00:11:08.320 --> 00:11:11.117 mini crisis that will then cause the 00:11:11.200 --> 00:11:15.357 governments to say, "Hang on, let's have a conversation about how to really deal 00:11:15.440 --> 00:11:20.317 with the downsides." The upsides are incredible, right? And I want America to 00:11:20.400 --> 00:11:23.437 win, and I want us to run as fast as we can. And we're indeed doing that with 00:11:23.520 --> 00:11:27.917 the Trump administration, which is great, right? But we have to be aware 00:11:28.000 --> 00:11:31.597 that these pos these things are possible. Um the one I'm particularly 00:11:31.680 --> 00:11:34.557 worried about is biological and he goes something like this. You take some 00:11:34.640 --> 00:11:39.117 existing pathogen and using biological techniques which I won't discuss. You 00:11:39.200 --> 00:11:43.117 can modify it enough that it cannot be detected but it's still quite dangerous. 00:11:43.200 --> 00:11:45.917 That's an example of a threat. There are many others. 00:11:46.000 --> 00:11:48.717 >> Yeah, that's also the easiest I think 00:11:48.800 --> 00:11:51.917 which is scary um of of the immediate 00:11:52.000 --> 00:11:56.797 you know CBRN threat threats. So cyber, biological, radiological, nuclear. uh 00:11:56.880 --> 00:11:59.677 biological is the one you can kind of do in a basement with three people, but 00:11:59.760 --> 00:12:01.197 it's also the one that's hardest to 00:12:01.280 --> 00:12:04.477 contain. So anyway, uh I hope the theory 00:12:04.560 --> 00:12:06.957 is right. So, so then uh how do you deal 00:12:07.040 --> 00:12:09.037 with proliferation? You know, the Biden 00:12:09.120 --> 00:12:12.637 approach was okay, let's contain AGI to 00:12:12.720 --> 00:12:16.397 these five companies, you know, Google being one of them, and then let's say 00:12:16.480 --> 00:12:21.117 any model with over 1 26 training flops has to register with the federal 00:12:21.200 --> 00:12:25.757 government. And then we'll keep it all contained. Uh, so that all got scrapped 00:12:25.840 --> 00:12:29.197 immediately, you know, after the election and got replaced by the new 00:12:29.280 --> 00:12:33.037 David Saxs document, which the David Sachs document is much more about how do 00:12:33.120 --> 00:12:35.837 we move as quickly as possible and and 00:12:35.920 --> 00:12:38.637 win the race. Um, but it doesn't really 00:12:38.720 --> 00:12:43.037 address proliferation. And obviously in America, we want startups uh and 00:12:43.120 --> 00:12:47.917 researchers to have incredible access to technology and compute. On the other 00:12:48.000 --> 00:12:51.997 hand, you know, the three people in a basement making a biological weapon is 00:12:52.080 --> 00:12:55.677 is a real scary thing. So, how do you balance those? 00:12:55.760 --> 00:12:59.437 >> It turns out if you read the David Saxs President Trump announcement, they're 00:12:59.520 --> 00:13:01.917 very clear that they want to continue to 00:13:02.000 --> 00:13:04.797 study the security aspects 00:13:04.880 --> 00:13:07.037 of AI, especially in a geopolitical way. 00:13:07.120 --> 00:13:09.997 And that's code for China versus the US. 00:13:10.080 --> 00:13:14.317 And so, they continue in their proposal to fund things involving nation state 00:13:14.400 --> 00:13:16.557 attacks and so forth and so on. and I 00:13:16.640 --> 00:13:19.517 fully support that kind of stuff. Um, 00:13:19.600 --> 00:13:23.757 there's probably a difference on the misinformation social media stuff 00:13:23.840 --> 00:13:26.237 between the two, but for example, the 00:13:26.320 --> 00:13:30.957 Biden rule was simply that you had to report if you're doing a training run 10 00:13:31.040 --> 00:13:34.957 to the 26 or greater. I was part of the group that made that number up. Uh, we 00:13:35.040 --> 00:13:38.157 made it up because we had no better number. I'm not suggesting it's the 00:13:38.240 --> 00:13:41.197 right number. And the the conclusion you 00:13:41.280 --> 00:13:43.757 come to is 00:13:43.840 --> 00:13:48.477 it's probably the case that that we know 00:13:48.560 --> 00:13:52.237 our government, I don't know this, but I'm guessing knows where the training 00:13:52.320 --> 00:13:55.197 runs are going on in in China because of 00:13:55.280 --> 00:13:57.757 espionage. And it's probably the case 00:13:57.840 --> 00:14:02.957 that the Chinese have espionage on the US knowing where our training runs are. 00:14:03.040 --> 00:14:07.517 So I'm not sure the nature of the training run runs is a secret. And 00:14:07.600 --> 00:14:09.597 frankly, everybody knows where the data 00:14:09.680 --> 00:14:12.557 centers are cuz they're immense. So, uh, 00:14:12.640 --> 00:14:15.197 we will see. Another attack on the 1026 00:14:15.280 --> 00:14:20.477 is that the training is getting more um more costefficient. If you look at the 00:14:20.560 --> 00:14:25.597 moving, they move from something called FP16 to FP8. That means eight precision 00:14:25.680 --> 00:14:29.117 floating point. People are now moving to four fourbit floating point, which is 00:14:29.200 --> 00:14:31.837 bizarre. It turns out these training 00:14:31.920 --> 00:14:35.117 algorithms seem to be quite tolerant for 00:14:35.200 --> 00:14:39.997 floating point impre imprecision which is a shock to me. So again we're getting 00:14:40.080 --> 00:14:44.157 more efficient in training. This creates more of a proliferation problem. If I 00:14:44.240 --> 00:14:48.157 can if I can say in general the proliferation issue I'm not worried 00:14:48.240 --> 00:14:50.797 about big companies in big countries 00:14:50.880 --> 00:14:52.877 because I can count them. You know 00:14:52.960 --> 00:14:57.917 there'll be 10 huge data centers, 10 huge training runs around the world. I'm 00:14:58.000 --> 00:15:02.557 much more worried about the open- source groups which can operate in the shadows. 00:15:02.640 --> 00:15:03.997 They don't have to solve every problem. 00:15:04.080 --> 00:15:05.917 They just have to do one thing well. 00:15:06.000 --> 00:15:09.277 They can patch together the open source which is generally available and it's 00:15:09.360 --> 00:15:11.437 good enough. If you look at the quality 00:15:11.520 --> 00:15:15.357 of DeepS are R are one and now R2 00:15:15.440 --> 00:15:18.077 coming. Sure looks like it's in 80 or 00:15:18.160 --> 00:15:22.637 90% of these top closed models. The closed models people, which obviously 00:15:22.720 --> 00:15:26.877 includes Google, get very defensive over this because they say, "Look, those are 00:15:26.960 --> 00:15:30.717 kind of synthetic. They have diffused the models. They've trained our best 00:15:30.800 --> 00:15:34.557 information. They're not." All of that is true. They're correct, but they are 00:15:34.640 --> 00:15:37.037 nevertheless useful for specific things, 00:15:37.120 --> 00:15:41.677 and they could be used in a proliferation issue to do various forms 00:15:41.760 --> 00:15:43.917 of cyber and biological attacks. 00:15:44.000 --> 00:15:47.357 >> Well, yeah. The current rule of thumb is that distillation and transfer learning 00:15:47.440 --> 00:15:51.837 is about 1% of the cost of the original training but gets you to the same 00:15:51.920 --> 00:15:55.437 destination. So if you take you know FP4 which you mentioned you know we get an 00:15:55.520 --> 00:15:58.717 8x performance boost over FP32. So you 00:15:58.800 --> 00:16:03.277 get an 8x on the ch on the you know the the weights and then you've got another 00:16:03.360 --> 00:16:07.437 100x on the transfer learning. And it's really hard to draw an analogy to 00:16:07.520 --> 00:16:12.157 nuclear or chemical weapons in the past cuz you couldn't kind of shrink them a 00:16:12.240 --> 00:16:14.397 thousandx under the covers and get the 00:16:14.480 --> 00:16:19.437 same result. But AI is really weird that way. It's it's very compressible, very 00:16:19.520 --> 00:16:21.277 fluid. 00:16:21.360 --> 00:16:24.637 Uh well, again, we we we will see. Um it 00:16:24.720 --> 00:16:27.037 does not look like we have a very good 00:16:27.120 --> 00:16:29.517 solution for distillation. 00:16:29.600 --> 00:16:32.957 It looks like an opponent of a company 00:16:33.040 --> 00:16:35.837 can mask their queries to look like 00:16:35.920 --> 00:16:37.917 normal sets of users and then distill 00:16:38.000 --> 00:16:40.877 the models. One of my friends has 00:16:40.960 --> 00:16:43.677 thought about this a lot and he thinks 00:16:43.760 --> 00:16:45.517 that the eventual state in the United 00:16:45.600 --> 00:16:48.477 States is that the biggest models will 00:16:48.560 --> 00:16:51.597 never be released and that the companies 00:16:51.680 --> 00:16:54.557 will distill their own models down for 00:16:54.640 --> 00:16:57.197 that reason. That's his opinion. We'll 00:16:57.280 --> 00:17:00.157 see if that's true. And this produces a 00:17:00.240 --> 00:17:03.037 bizarre outcome where the biggest models 00:17:03.120 --> 00:17:05.517 in the United States are closed source 00:17:05.600 --> 00:17:07.997 and the biggest models in China are 00:17:08.080 --> 00:17:12.637 open- source. And the geopolitical issue there of course is that open source is 00:17:12.720 --> 00:17:17.677 free and the closed source models are not free. And so the vast majority of 00:17:17.760 --> 00:17:21.037 governments and countries who don't have the kind of money that the west does and 00:17:21.120 --> 00:17:24.877 so forth will end up standardizing on Chinese models not because they're 00:17:24.960 --> 00:17:28.397 better, but because they're free. And so we'll see if that's true or not. But I 00:17:28.480 --> 00:17:31.197 do worry about that. >> Well, this whole topic of distillation 00:17:31.280 --> 00:17:35.037 and proliferation is a good segue into a much happier topic which I really want 00:17:35.120 --> 00:17:39.757 to use want to use our remaining time on. Uh I cannot tell you how inspiring 00:17:39.840 --> 00:17:43.677 you are to the founders around Cambridge, MIT, Harvard, Northeastern 00:17:43.760 --> 00:17:47.677 where where all of our talent comes from. Um, in my perfect world, you would 00:17:47.760 --> 00:17:51.757 podcast or say something or publish something every single day and then 00:17:51.840 --> 00:17:56.477 people could turn off CNBC and just tune into what you say because well, I mean, 00:17:56.560 --> 00:18:00.237 aside from it being brilliant, you have access to knowledge that isn't in anyone 00:18:00.320 --> 00:18:01.677 else's brain as far as I can tell. 00:18:01.760 --> 00:18:05.997 You've got this really, really unique combination of perspectives and and it's 00:18:06.080 --> 00:18:08.157 incredibly valuable. So, you know, using 00:18:08.240 --> 00:18:11.997 distillation and uh as a starting point, 00:18:12.080 --> 00:18:15.837 what founder advice would you give? you know, we're seeing numbers like they're 00:18:15.920 --> 00:18:20.157 completely unprecedented at ages of founders that are also unprecedented. I 00:18:20.240 --> 00:18:23.757 mean, you saw Sergey and Larry when they were very very young, but now they're 00:18:23.840 --> 00:18:27.677 even younger. We're talking 18, 19 years years old now. So, uh, what advice would 00:18:27.760 --> 00:18:30.077 you give them? 00:18:30.160 --> 00:18:32.077 >> So, I think the the most important thing 00:18:32.160 --> 00:18:35.357 to say is that the barrier to entry to 00:18:35.440 --> 00:18:38.237 starting a company is effectively zero 00:18:38.320 --> 00:18:41.357 now. So, let's think about it. What do 00:18:41.440 --> 00:18:45.597 you need to have a company? You can register it online. You need some money 00:18:45.680 --> 00:18:49.277 to get started. You have to pay yourself. Um, you don't really need any 00:18:49.360 --> 00:18:53.197 programmers. You need a couple people to w to pay 00:18:53.280 --> 00:18:55.517 Google or Claude or what have you to uh 00:18:55.600 --> 00:19:00.717 to write the code for you. Um, so you're pretty good there. You can use third 00:19:00.800 --> 00:19:04.637 party logistics companies if you're building a hardware device and you can 00:19:04.720 --> 00:19:09.357 use essentially contract manufacturers to build whatever hardware device. So it 00:19:09.440 --> 00:19:13.117 looks to me like for hardware and software the barrier to entry is almost 00:19:13.200 --> 00:19:16.237 zero. That sounds great until you 00:19:16.320 --> 00:19:17.757 realize that as a result you're now 00:19:17.840 --> 00:19:20.477 competing against everyone all the time. 00:19:20.560 --> 00:19:23.597 So as an example in my career which has 00:19:23.680 --> 00:19:26.557 spanned more than I guess 50 55 years 00:19:26.640 --> 00:19:29.277 now doing this stuff the key thing 00:19:29.360 --> 00:19:32.237 that's true is the compression of time. 00:19:32.320 --> 00:19:34.237 The other thing I would say to founders 00:19:34.320 --> 00:19:37.037 is it's really important that everything 00:19:37.120 --> 00:19:41.357 you do be learned and not uh specified. 00:19:41.440 --> 00:19:43.197 I'm doing a couple startups on my own. 00:19:43.280 --> 00:19:47.357 We'll see how well they do. But with them, I say I know nothing. Learn 00:19:47.440 --> 00:19:51.517 everything. So you can learn how to support your customers. You can learn 00:19:51.600 --> 00:19:55.197 what the customer wants. You can learn how to and learning meaning in the AI 00:19:55.280 --> 00:19:56.797 sense of learning. Learning it as part 00:19:56.880 --> 00:20:00.157 of uh either uh supervised or 00:20:00.240 --> 00:20:02.957 unsupervised training. U and if you take 00:20:03.040 --> 00:20:04.797 a learning approach then you build a 00:20:04.880 --> 00:20:07.997 system that if it works it will explode 00:20:08.080 --> 00:20:10.637 >> because once the learning accelerates 00:20:10.720 --> 00:20:13.917 you get into a quasi monopoly position. 00:20:14.000 --> 00:20:18.717 So the philosophy of winning goes something like this. Run as fast you can 00:20:18.800 --> 00:20:23.117 get there as quickly as you can. Build it around learning and if it takes off 00:20:23.200 --> 00:20:25.677 you'll be a hero because once it learns 00:20:25.760 --> 00:20:28.557 it learns how to become stronger. Um, 00:20:28.640 --> 00:20:33.437 and eventually two or three years, you'll start to have various forms of 00:20:33.520 --> 00:20:36.637 reinforcement learning which are self-replicating. 00:20:36.720 --> 00:20:40.397 And so then you're likely to get accelerations further. That's the most 00:20:40.480 --> 00:20:42.317 likely path for the next trillion dollar 00:20:42.400 --> 00:20:44.637 company past the equivalents of 00:20:44.720 --> 00:20:47.837 anthropic, open AI, you know, etc. And 00:20:47.920 --> 00:20:52.157 so when you're looking at an investment yourself, uh, by the way, the the 00:20:52.240 --> 00:20:56.077 learning loops concept that Eric just mentioned on the Moonshots podcast that 00:20:56.160 --> 00:20:59.837 we did with Eric, which you can find online, he actually described in detail 00:20:59.920 --> 00:21:03.197 the three or four different types of learning loops that he looks for. So 00:21:03.280 --> 00:21:06.477 it's definitely worth, you know, it's a lot more than we have time for right 00:21:06.560 --> 00:21:08.637 now, but it's really, really brilliant. 00:21:08.720 --> 00:21:12.797 So, uh, aside from the actual business plan and the learning loops, what do you 00:21:12.880 --> 00:21:16.877 look for in founder dynamics, founder team team members? 00:21:16.960 --> 00:21:20.477 Um, it's always the case that the that the 00:21:20.560 --> 00:21:22.557 founders are really really smart. 00:21:22.640 --> 00:21:24.957 They're very very quick and they're very 00:21:25.040 --> 00:21:28.397 interesting to talk to. It's also true 00:21:28.480 --> 00:21:32.717 that you need them to be able to hire a network of people like them. 00:21:32.800 --> 00:21:36.157 >> And so a simple way is if you talk to them and they seem really interesting 00:21:36.240 --> 00:21:40.317 and really dynamic and you find that they're in a network of such people, you 00:21:40.400 --> 00:21:44.877 likely have winners. And then what you do is you say to them, show me, don't 00:21:44.960 --> 00:21:47.917 tell me the product, which of course is what they want to talk about. Show me 00:21:48.000 --> 00:21:52.877 how you're going to build a system that is that goes from zero to infinity. 00:21:52.960 --> 00:21:56.237 There's a lot of discussion about 0ero to one. And there it's complete 00:21:56.320 --> 00:21:58.237 compression. Get that thing done. But 00:21:58.320 --> 00:22:00.237 once you have one, how are you going to 00:22:00.320 --> 00:22:03.517 scale? Our industry 00:22:03.600 --> 00:22:07.917 makes enormous wealth for the founders when you build a platform that is 00:22:08.000 --> 00:22:12.877 scaling. That's the lesson. What is the lesson to offer? Build a platform that 00:22:12.960 --> 00:22:14.797 scales if you can't. And a platform is 00:22:14.880 --> 00:22:16.877 defined as something which others depend 00:22:16.960 --> 00:22:21.277 on that you provide. Right? And the 00:22:21.360 --> 00:22:24.877 stronger the platform, the more it is, the more interconnected it is, the 00:22:24.960 --> 00:22:28.477 stronger the network lock in. That's just generally true. It was true for 00:22:28.560 --> 00:22:32.717 Microsoft. It was true for what I did 20 years ago. It's true today. And I think 00:22:32.800 --> 00:22:34.077 if you look, I'll give you an example. 00:22:34.160 --> 00:22:37.917 Everyone's looking at where are the economics for the large LLM companies. 00:22:38.000 --> 00:22:42.397 They don't have a strong enough network lockin yet, but you could easily imagine 00:22:42.480 --> 00:22:45.677 that they would develop it and I'm sure they don't tell me what they're doing, 00:22:45.760 --> 00:22:48.877 but I'm sure that they have that in the back of their mind. That's their that's 00:22:48.960 --> 00:22:51.757 part of the reasons why their valuations are so high. 00:22:51.840 --> 00:22:56.077 >> So, we have one minute remaining and I really want to use it to milk you for a 00:22:56.160 --> 00:22:58.237 quote that we can loop on our wall in 00:22:58.320 --> 00:23:01.757 the office. Um, and I it would the 00:23:01.840 --> 00:23:03.677 perfect quote to me would be about the 00:23:03.760 --> 00:23:06.157 importance of this moment in time. And 00:23:06.240 --> 00:23:11.357 so many of the founders uh they weren't around when Microsoft could, you know, 00:23:11.440 --> 00:23:13.997 Bill Gates could wake up in the morning and decide, hey, I'm going to destroy 00:23:14.080 --> 00:23:18.317 Word Perfect today and I'm going to destroy uh Lotus 123 today cuz they just 00:23:18.400 --> 00:23:20.797 had that that that power at that time. 00:23:20.880 --> 00:23:25.437 And noll was certainly in that crosshairs, too. uh then there was this 00:23:25.520 --> 00:23:27.597 magical moment of the internet explosion 00:23:27.680 --> 00:23:29.357 where companies like Google and many 00:23:29.440 --> 00:23:32.957 many others could thrive but then after 00:23:33.040 --> 00:23:36.077 that you know things got static again 00:23:36.160 --> 00:23:41.117 and now we're in the most explosive uh time period for opportunity for 00:23:41.200 --> 00:23:45.197 entrepreneurs that I've ever experienced in my life but very few people remember 00:23:45.280 --> 00:23:47.837 all the way back to the internet 00:23:47.920 --> 00:23:52.557 explosion era so I'd love to get your your quote or your thoughts around what 00:23:52.640 --> 00:23:56.237 is the importance of this moment in time. 00:23:56.320 --> 00:23:58.237 >> So I firmly believe that that the 00:23:58.320 --> 00:24:02.317 arrival of nonhuman intelligence, AI 00:24:02.400 --> 00:24:05.597 intelligence is at the level of 00:24:05.680 --> 00:24:08.397 electricity or the invention of fire, 00:24:08.480 --> 00:24:11.197 transportation, etc. in human history. 00:24:11.280 --> 00:24:14.317 We are fortunate to be living in a time 00:24:14.400 --> 00:24:19.197 of great historical consequence. The next 10 years are probably the 10 years 00:24:19.280 --> 00:24:22.717 that will have a greater determination over the next hundred years than 00:24:22.800 --> 00:24:24.157 anything before because of the 00:24:24.240 --> 00:24:26.877 inventions of these new tools. And the 00:24:26.960 --> 00:24:29.837 tools are very very powerful power. And 00:24:29.920 --> 00:24:34.317 remember, they're powerful because they they can equal and in some cases surpass 00:24:34.400 --> 00:24:38.957 human intelligence. And human intelligence is everything for society. 00:24:39.040 --> 00:24:43.517 And so the countries and companies that embrace this non-human intelligence 00:24:43.600 --> 00:24:48.157 correctly and aggressively will be the big winners. And the companies and 00:24:48.240 --> 00:24:52.557 countries that are slow or let other people do it, they will lose because 00:24:52.640 --> 00:24:56.077 that the source of excellence, the source of leadership, the source of 00:24:56.160 --> 00:25:00.157 growth, the source of everything, the source of innovation, everything 00:25:00.240 --> 00:25:03.837 economic growth comes from the application of intelligence to discover 00:25:03.920 --> 00:25:08.877 new things, to solve new problems. We are on a huge course to accelerate that 00:25:08.960 --> 00:25:13.960 here in America and I'm very proud to be part of it.