1 00:00:00,080 --> 00:00:03,517 I think we don't yet know the shape and form of the ultimate products. Just one 2 00:00:03,600 --> 00:00:07,117 just obvious historical analogy is you know the personal computer from sort of 3 00:00:07,200 --> 00:00:09,597 invention in 1975 through to you know 4 00:00:09,680 --> 00:00:14,797 basically 1992 was a text prompt system 17 years in you know the whole industry 5 00:00:14,880 --> 00:00:17,597 took a left turn into gooies and never looked back and then by the way you know 6 00:00:17,680 --> 00:00:20,477 5 years after that the industry took a left turn into web browsers and never 7 00:00:20,560 --> 00:00:22,957 looked back right and you know look I'm I'm sure there will be chat bots 20 8 00:00:23,040 --> 00:00:26,477 years from now but I'm pretty confident that both the current chatbot companies 9 00:00:26,560 --> 00:00:29,357 and many new companies are going to figure out many kinds of user 10 00:00:29,440 --> 00:00:33,680 experiences that are radically different that we don't we don't even know 11 00:00:34,160 --> 00:00:36,557 Please join me in welcoming Mark Andre 12 00:00:36,640 --> 00:00:38,477 and Ben Horowitz with general partner 13 00:00:38,560 --> 00:00:43,597 Eric Torberg. Follow me into our solo. Get in the flow 14 00:00:43,680 --> 00:00:45,437 and you can pitch it like a photo. Music 15 00:00:45,520 --> 00:00:47,517 makes me maintain and make melodies for 16 00:00:47,600 --> 00:00:52,797 MC's motivation breaks. I'm never >> Thank you for the rock Kim. Who did 17 00:00:52,880 --> 00:00:57,117 that? >> Ben picks the Ben picked the music. 18 00:00:57,200 --> 00:00:59,277 Mark, there's been a lot of talk lately 19 00:00:59,360 --> 00:01:02,317 about the limitations of LLMs that they 20 00:01:02,400 --> 00:01:04,957 they can't do true invention of say new 21 00:01:05,040 --> 00:01:09,437 science that they can't do true creative genius that it's just combining or 22 00:01:09,520 --> 00:01:13,117 packaging. You have thoughts here. What say you? 23 00:01:13,200 --> 00:01:15,917 >> Yeah. So, so for me, yes, you get all these questions and yeah, they usually 24 00:01:16,000 --> 00:01:20,557 come in either, you know, sort of are are language models intelligent in the 25 00:01:20,640 --> 00:01:23,117 sense of like can they actually um you know, can they actually process 26 00:01:23,200 --> 00:01:26,877 information and have sort of conceptual breakthroughs? the way that people can. 27 00:01:26,960 --> 00:01:30,077 And then there's are language models or or video models creative, you know, can 28 00:01:30,160 --> 00:01:33,037 they can they create new art? Um actually have genuine creative 29 00:01:33,120 --> 00:01:35,517 breakthroughs. And of course my my my answer to both of those is well can 30 00:01:35,600 --> 00:01:39,677 people do those things? Um and um I think there's two two two questions 31 00:01:39,760 --> 00:01:43,597 there which is like okay even if some people are quote unquote intelligent as 32 00:01:43,680 --> 00:01:46,637 in having original uh conceptual breakthroughs and not just let's just 33 00:01:46,720 --> 00:01:51,437 say regurgitating the training set um uh or following scripts um uh how you know 34 00:01:51,520 --> 00:01:55,117 what percentage of people can actually do that and I say I've only met a few 35 00:01:55,200 --> 00:01:59,197 some of them are here in the room um but uh you know not that many most people 36 00:01:59,280 --> 00:02:01,677 never do and then creativity I mean how many people are actually genuinely 37 00:02:01,760 --> 00:02:04,957 creative right and so you you kind of point to a Beethoven or you know Van 38 00:02:05,040 --> 00:02:07,357 Gogh or something like that you're like Okay, that's creativity and yeah, that's 39 00:02:07,440 --> 00:02:10,637 creativity and then how many Beethovenas and Van Go are there? Obviously, not 40 00:02:10,720 --> 00:02:14,557 very many. So, so one is just like okay, like you know, if it's if it if if if 41 00:02:14,640 --> 00:02:18,637 these things clear the bar of, you know, 99.99% of humanity, you know, then 42 00:02:18,720 --> 00:02:22,317 that's pretty interesting just in and of itself. But then I you dig into it 43 00:02:22,400 --> 00:02:26,237 further and you're like, okay, like how many actual real conceptual 44 00:02:26,320 --> 00:02:29,437 breakthroughs have there ever been actually ever in human history as 45 00:02:29,520 --> 00:02:34,317 compared to sort of remixing remixing ideas. Um and you know like if you look 46 00:02:34,400 --> 00:02:37,197 at the history of technology it's almost always the case that the big 47 00:02:37,280 --> 00:02:41,357 breakthroughs are the result of you know usually at least 40 years of sort of 48 00:02:41,440 --> 00:02:44,797 work ahead of time you know four decades right in fact language models themselves 49 00:02:44,880 --> 00:02:48,557 are the culmination of eight decades right of previous work and so there's 50 00:02:48,640 --> 00:02:51,357 remixing and then in the arts it's the exact same thing you know novels and 51 00:02:51,440 --> 00:02:54,557 music and everything like you know there are clearly creative leaps but you know 52 00:02:54,640 --> 00:02:57,597 there's just tremendous amounts of influence that came in from from people 53 00:02:57,680 --> 00:03:00,877 who came before and even if you think about like somebody with the creativity 54 00:03:00,960 --> 00:03:03,997 of a Beethoven like there There's a lot of Beethoven in Mozart and Heiden and in 55 00:03:04,080 --> 00:03:07,677 the composers that came before and so there's just tremendous amounts of of of 56 00:03:07,760 --> 00:03:11,357 remixing and combination. And so it's it's it's a little bit of an angel's 57 00:03:11,440 --> 00:03:14,317 dancing on the head of a pin question which is like if you can get if you can 58 00:03:14,400 --> 00:03:19,197 get you know within you know I don't know one you know 001% of of kind of 59 00:03:19,280 --> 00:03:22,477 worldbeating uh you know generational creativity intelligence like you're 60 00:03:22,560 --> 00:03:26,397 you're you're probably all the way there. So I so emotionally I want to 61 00:03:26,480 --> 00:03:30,077 like hold out hope that there is you know still something special about human 62 00:03:30,160 --> 00:03:33,197 creativity and I I certainly believe that and I and I and I very much want to 63 00:03:33,280 --> 00:03:37,037 believe that but um I don't know when I use these things I'm like wow they seem 64 00:03:37,120 --> 00:03:40,077 to be awfully smart and awfully creative. So I'm I'm I'm I'm pretty 65 00:03:40,160 --> 00:03:42,637 convinced that they're going to clear the bar. 66 00:03:42,720 --> 00:03:45,597 >> Yeah. the I think that seems to be a common 67 00:03:45,680 --> 00:03:48,957 theme in your analysis when when people talk about the limitations of LMS, you 68 00:03:49,040 --> 00:03:52,317 know, can they do transfer learning or or just learning in general? You seem to 69 00:03:52,400 --> 00:03:55,517 ask, can people do this? >> Yes. Can people do these things? Well, 70 00:03:55,600 --> 00:03:58,237 it's like lateral thinking, right? So, yeah, so it's like reasoning in or out 71 00:03:58,320 --> 00:04:00,477 of distribution, right? And so it's like, okay, I know a lot of people who 72 00:04:00,560 --> 00:04:02,797 are very good at reasoning inside distribution. How many people do I 73 00:04:02,880 --> 00:04:05,517 actually know who are good at reasoning outside of distribution and doing 74 00:04:05,600 --> 00:04:08,557 transfer learning? And and the answer is like I know a handful. Like I I know a 75 00:04:08,640 --> 00:04:12,077 few I know a few people um where whenever you ask them a question, you 76 00:04:12,160 --> 00:04:15,517 get an extremely original answer. And usually that answer involves bringing in 77 00:04:15,600 --> 00:04:19,197 some idea from some adjacent space and basically being able to bridge domains. 78 00:04:19,280 --> 00:04:21,837 Um and so you know you'll ask them a question about I don't know you know 79 00:04:21,920 --> 00:04:24,637 finance and they'll they'll bring you an answer from psychology or you ask them a 80 00:04:24,720 --> 00:04:27,277 question about psychology and they'll bring you an answer from biology right 81 00:04:27,360 --> 00:04:31,437 or whatever it is. And so I I know you know I know I don't know sitting here 82 00:04:31,520 --> 00:04:35,197 today probably three I probably know three people who can do that reliably 83 00:04:35,280 --> 00:04:39,277 out of the you know I I've got you know I've got 10,000 in my address book. Um 84 00:04:39,360 --> 00:04:42,797 and so three out of 10,000 >> Yeah. is not is not that high a 85 00:04:42,880 --> 00:04:46,557 percentage. By by the way, I find this very encouraging. Like I I yeah, 86 00:04:46,640 --> 00:04:50,797 immediately the mood in the room has gone completely to hell. Um I find this 87 00:04:50,880 --> 00:04:53,437 very encouraging. I find this very encouraging because look at what 88 00:04:53,520 --> 00:04:55,437 humanity has been able to build, right? 89 00:04:55,520 --> 00:04:57,437 Despite all of our limitations, right? 90 00:04:57,520 --> 00:04:59,997 And and look at all the creativity that we've been able to exhibit and all the 91 00:05:00,080 --> 00:05:02,877 amazing art and all the amazing movies and all the amazing novels and all the 92 00:05:02,960 --> 00:05:05,917 amazing technical inventions and scientific breakthroughs. And so we, you 93 00:05:06,000 --> 00:05:08,157 know, we've been able to do, you know, everything we've been able to do with 94 00:05:08,240 --> 00:05:12,317 the limitations that we have. Um, and so, you know, I think that, you know, 95 00:05:12,400 --> 00:05:15,597 like, you know, do you need to get to the thing where you are 100% positive 96 00:05:15,680 --> 00:05:19,037 that's actually doing, you know, original thinking? I don't think so. Uh, 97 00:05:19,120 --> 00:05:21,437 I I think it'd be great if you did, and I think ultimately we'll probably 98 00:05:21,520 --> 00:05:22,957 conclude that that's what's happening. 99 00:05:23,040 --> 00:05:26,717 Um, but like I it's not necessary for like just tremendous amounts of 100 00:05:26,800 --> 00:05:30,237 improvement. >> Ben, we were just celebrating some some 101 00:05:30,320 --> 00:05:34,717 hip-hop legends at your paid in full uh event last week and so you think a lot 102 00:05:34,800 --> 00:05:38,157 about creative genius. How do you think about this question? 103 00:05:38,240 --> 00:05:41,437 Yeah, I mean I think that uh I agree 104 00:05:41,520 --> 00:05:43,997 with Mark that it's 105 00:05:44,080 --> 00:05:46,397 whatever it is, it's very useful. um 106 00:05:46,480 --> 00:05:48,797 even if it isn't all the way that level 107 00:05:48,880 --> 00:05:51,757 uh I think that 108 00:05:51,840 --> 00:05:54,637 you know there's something about the 109 00:05:54,720 --> 00:05:57,277 actual like real time human experience 110 00:05:57,360 --> 00:06:00,957 that humans are very into um at least in 111 00:06:01,040 --> 00:06:03,997 art where 112 00:06:04,080 --> 00:06:06,077 you know with the current state of the 113 00:06:06,160 --> 00:06:08,717 technology kind of the the pre-training 114 00:06:08,800 --> 00:06:11,757 doesn't have quite the the right data to 115 00:06:11,840 --> 00:06:15,837 to to get to um what you really want to 116 00:06:15,920 --> 00:06:20,397 Uh but it, you know, it's pretty good. 117 00:06:20,480 --> 00:06:23,997 >> It is pretty good. >> How many So how many true conceptual So 118 00:06:24,080 --> 00:06:27,197 Ben Ben Ben Ben's Ben, one of Ben's nonprofit activities is something called 119 00:06:27,280 --> 00:06:29,997 the Pain and Fall Foundation, which is honoring and actually providing 120 00:06:30,080 --> 00:06:33,197 essentially a pension a pension for uh you know, for sort of, you know, the 121 00:06:33,280 --> 00:06:37,917 great innovators in in in rap and hip-hop. Um and and so he has he knows 122 00:06:38,000 --> 00:06:40,797 and has many of we were just at the event and he you know has many of the 123 00:06:40,880 --> 00:06:43,357 kind of leading lights of that field for the last 50 years you know perform and 124 00:06:43,440 --> 00:06:47,117 and it's really fun to meet them and talk to them. Um but like how many 125 00:06:47,200 --> 00:06:49,917 people in that entire field over the course of the last 50 years would you 126 00:06:50,000 --> 00:06:53,357 classify as like a true conceptual innovator? 127 00:06:53,440 --> 00:06:56,397 >> Yeah. Well, you know it's interesting. 128 00:06:56,480 --> 00:07:00,957 Well, it depends how broadly you define it but 129 00:07:01,040 --> 00:07:05,917 >> you know there were several of them there last you know on Saturday. Rock, I 130 00:07:06,000 --> 00:07:08,957 think. Yeah, Rockm, you'd certainly put in that category. Dr. Dre, you'd 131 00:07:09,040 --> 00:07:12,237 certainly put in that category. George Clinton, you'd certainly put in that 132 00:07:12,320 --> 00:07:14,877 category. Um, 133 00:07:14,960 --> 00:07:17,917 >> you know, in a narrower sense, like Cool 134 00:07:18,000 --> 00:07:22,077 G Rap certainly had a new idea. Um, 135 00:07:22,160 --> 00:07:26,317 but you know, it depends like a 136 00:07:26,400 --> 00:07:28,317 fundamental kind of musical 137 00:07:28,400 --> 00:07:33,837 breakthrough, you probably just say like Rock Kim and George Clinton. Um, are 138 00:07:33,920 --> 00:07:38,877 they excited? >> So, so two out of >> Well, I mean those of the guys who were 139 00:07:38,960 --> 00:07:41,997 there. >> Oh, yeah. Yeah. >> Yeah. But yeah, it's a tiny percentage. 140 00:07:42,080 --> 00:07:45,997 Tiny tiny tiny tiny tiny. >> We had Jared at the fireside last night 141 00:07:46,080 --> 00:07:49,757 with Jared Leto. He was talking about how many people in Hollywood are are 142 00:07:49,840 --> 00:07:54,077 really scared or against this um what's happening here is is what do you see in 143 00:07:54,160 --> 00:07:57,757 you know when you talk to the Dr. JS the nas the Kanye's are they excited? Are 144 00:07:57,840 --> 00:07:59,277 they using it? Are they 145 00:07:59,360 --> 00:08:02,797 >> Yeah. No, I the so everybody uh who I 146 00:08:02,880 --> 00:08:07,117 speak to there are definitely people who are scared in music but like there are a 147 00:08:07,200 --> 00:08:09,677 lot of people who are very very interested in it and particularly the 148 00:08:09,760 --> 00:08:13,357 hip-hop guys are interested because um 149 00:08:13,440 --> 00:08:15,917 it's almost like a replay of what they 150 00:08:16,000 --> 00:08:18,557 did right that they just took other 151 00:08:18,640 --> 00:08:20,797 music and they kind of built new music 152 00:08:20,880 --> 00:08:23,997 out of it and I think that you know AI 153 00:08:24,080 --> 00:08:26,957 is uh a fantastic creative tool for 154 00:08:27,040 --> 00:08:29,037 them. like way opens up the pallet and 155 00:08:29,120 --> 00:08:32,477 then for you know a lot of what hip-hop 156 00:08:32,560 --> 00:08:35,437 is is it's um kind of telling a very 157 00:08:35,520 --> 00:08:38,797 specific story of a specific time and 158 00:08:38,880 --> 00:08:42,477 place which um having intimate knowledge 159 00:08:42,560 --> 00:08:44,877 and being trained just on that thing is 160 00:08:44,960 --> 00:08:46,637 is actually an advantage as opposed 161 00:08:46,720 --> 00:08:49,517 being like a generally smart uh music 162 00:08:49,600 --> 00:08:54,397 model. Um, people also use the same logic of, hey, whatever is more 163 00:08:54,480 --> 00:08:58,157 intelligent will rule whatever is less intelligent. And and Mark, you recently 164 00:08:58,240 --> 00:09:02,077 uh not not said by anybody who owns a cat. 165 00:09:02,160 --> 00:09:06,077 >> Yeah, exactly. Mark, you recently tweeted, "A supreme 166 00:09:06,160 --> 00:09:08,477 shape rotator can only rotate shapes, 167 00:09:08,560 --> 00:09:10,557 but a supreme word cell can rotate shape 168 00:09:10,640 --> 00:09:13,197 rotators." And and also, 169 00:09:13,280 --> 00:09:19,117 >> someone's clapping here. And also, high IQ experts work for midiq generalists. 170 00:09:19,200 --> 00:09:22,637 What means? >> Yeah. What what means? Uh yeah. So it's 171 00:09:22,720 --> 00:09:24,797 the PhDs all work for MBAs, right? So 172 00:09:24,880 --> 00:09:30,397 it's like, you know, okay. So yeah, like so yeah. Well, I just, you know, just 173 00:09:30,480 --> 00:09:32,797 take it up a level. It's just like when you look at the world today, do you 174 00:09:32,880 --> 00:09:37,917 think we're being ruled by the smart ones, right? Like is that is that your big 175 00:09:38,000 --> 00:09:41,277 conclusion from like current events, current affairs, 176 00:09:41,360 --> 00:09:43,197 right? Like okay, we put the geniuses in charge. Like 177 00:09:43,280 --> 00:09:45,197 >> you mean Kamla and Trump aren't the best? 178 00:09:45,280 --> 00:09:48,317 >> Well, now let's not even be specific towards the US. let's just look all over 179 00:09:48,400 --> 00:09:51,837 the world. >> Um, you know, yeah. And so like it's 180 00:09:51,920 --> 00:09:56,317 just like there's this thing. So I think two two 181 00:09:56,400 --> 00:09:59,437 things are true. One is we we probably all kind of underwrite the importance of 182 00:09:59,520 --> 00:10:03,117 intelligence. Um, and actually there's a whole kind of backstory here of like 183 00:10:03,200 --> 00:10:06,397 intelligence actually turns out to be this like incredibly inflammatory, you 184 00:10:06,480 --> 00:10:10,157 know, kind of topic for lots of reasons over the last hundred years. um uh which 185 00:10:10,240 --> 00:10:13,757 which we could we could talk about in great detail but like it it and you know 186 00:10:13,840 --> 00:10:16,557 it's and the even the just the very idea that like some people are smarter than 187 00:10:16,640 --> 00:10:19,117 other people you just like really freaks people out and people people don't like 188 00:10:19,200 --> 00:10:22,797 to talk about it we really struggle with that as a society and so like and then 189 00:10:22,880 --> 00:10:26,077 it is true that intelligence is like in humans intelligence is correlated to 190 00:10:26,160 --> 00:10:30,237 almost every kind of positive life outcome right and so u intelligence 191 00:10:30,320 --> 00:10:32,957 generally in the social sciences what they'll tell you is what they call fluid 192 00:10:33,040 --> 00:10:35,517 intelligence or or the G factor or IQ is 193 00:10:35,600 --> 00:10:40,237 sort of it's sort of 0.4 four correlated to basically everything. Um, and so it's 194 00:10:40,320 --> 00:10:44,317 0.4 correlation to like educational outcomes and like you know professional 195 00:10:44,400 --> 00:10:48,717 outcomes and and you know income and by the way also like life satisfaction and 196 00:10:48,800 --> 00:10:51,837 by the way non-violence you know being able to solve problems without physical 197 00:10:51,920 --> 00:10:54,637 violence and so forth. And so like on the one hand like we probably all 198 00:10:54,720 --> 00:10:58,237 underrate intelligence. Um on the other hand the people who are in the fields 199 00:10:58,320 --> 00:11:01,917 that involve intelligence probably overrate intelligence. Um, and you might 200 00:11:02,000 --> 00:11:05,117 even you might even coin a term like maybe like intelligence supremacist or 201 00:11:05,200 --> 00:11:08,477 something like that where it's just like oh like intelligence is very important 202 00:11:08,560 --> 00:11:12,077 and so therefore maybe it's like the most important thing or the only thing 203 00:11:12,160 --> 00:11:14,797 but but then you look at reality and you're like okay that's clearly not the 204 00:11:14,880 --> 00:11:17,197 case. >> Yeah. It's still zero only 0.4 right. 205 00:11:17,280 --> 00:11:19,277 Yeah. >> Well so to start with it's only 0.4 and 206 00:11:19,360 --> 00:11:23,117 you know in the social sciences 0.4 is a giant correlation factor right like most 207 00:11:23,200 --> 00:11:26,317 most things that where you can correlate whether it's you know genes or observed 208 00:11:26,400 --> 00:11:28,877 behavior whatever to anything in the social sciences the correlations are 209 00:11:28,960 --> 00:11:32,797 much smaller than that. So 0.4 is is tiny, but it's still only 0.4. So, even 210 00:11:32,880 --> 00:11:35,997 if you're like a fullon if you even if you're like a full-on genetic 211 00:11:36,080 --> 00:11:39,197 determinist and you're just like, you know, genetic IQ just like drives all 212 00:11:39,280 --> 00:11:43,997 these outcomes, like it still doesn't explain, you know, 6 uh of the 213 00:11:44,080 --> 00:11:46,957 correlation and so that leaves it but but that's just on the individual level. 214 00:11:47,040 --> 00:11:49,517 Then you just look at the collective level. Well, you just look at the 215 00:11:49,600 --> 00:11:52,797 collective level and it's like a famous famous observation is you take a b you 216 00:11:52,880 --> 00:11:55,677 take a bunch of you take any group of people, you put them in a mob and the 217 00:11:55,760 --> 00:11:59,437 mob is dumber, right, than the average and and and you put a bunch of smart 218 00:11:59,520 --> 00:12:02,797 people in a mob and they definitely turn dumber like and you see that all the 219 00:12:02,880 --> 00:12:07,597 time, right? Um uh and so you put people in groups and they they they behave very 220 00:12:07,680 --> 00:12:11,357 differently and then you you create and then you create these you know questions 221 00:12:11,440 --> 00:12:13,917 around like who's in charge whether who's in charge at a at a company or 222 00:12:14,000 --> 00:12:17,277 who's in charge of a of a country and like it it's whatever the filtration 223 00:12:17,360 --> 00:12:21,277 process it's clearly not it's not it's it's not it's certainly not only on IQ 224 00:12:21,360 --> 00:12:24,957 and it may not even be primarily on IQ and so so therefore it's just like this 225 00:12:25,040 --> 00:12:27,757 assumption that you kind of hear in some of the AI circles uh which is like 226 00:12:27,840 --> 00:12:31,197 inevitably the smart you know kind of thing is going to govern the dumb thing 227 00:12:31,280 --> 00:12:34,397 like I I just think that's like very easily uh it's just sort of very easily 228 00:12:34,480 --> 00:12:37,677 and obviously falsified like intelligence isn't sufficient and then 229 00:12:37,760 --> 00:12:39,277 you just you just you just convey it. 230 00:12:39,360 --> 00:12:41,677 You know, we're all in this room lucky enough to know a lot of smart people and 231 00:12:41,760 --> 00:12:44,637 you you just kind of observe smart people and like some smart people, you 232 00:12:44,720 --> 00:12:46,877 know, really figure out how to have their stuff together and become very 233 00:12:46,960 --> 00:12:51,117 successful and a lot of smart people never do. Um and so there there's there 234 00:12:51,200 --> 00:12:54,877 there must be there obviously are and there and there in fact must be many 235 00:12:54,960 --> 00:12:57,997 other factors that have to do with success um and have to do with like 236 00:12:58,080 --> 00:13:01,677 who's in charge than just raw intelligence. It it begs the the 237 00:13:01,760 --> 00:13:05,277 follow-up question of what are what are some examples of what that might be you 238 00:13:05,360 --> 00:13:07,837 know skills sort of outside of intelligence and more particularly 239 00:13:07,920 --> 00:13:11,037 specifically why couldn't AI systems you know learn them? 240 00:13:11,120 --> 00:13:14,237 >> Yeah. So Ben like what what what other than intelligence what what in your 241 00:13:14,320 --> 00:13:16,717 experience determines for example success in leadership or in 242 00:13:16,800 --> 00:13:21,677 entrepreneurship or in in solving complex problems or organizing people? 243 00:13:21,760 --> 00:13:25,357 >> Yeah. There there are many things um you know 244 00:13:25,440 --> 00:13:30,557 like a lot of it is uh being able to have a confrontation in the correct way 245 00:13:30,640 --> 00:13:34,557 and like there's some intelligence in that but a lot of it is just under 246 00:13:34,640 --> 00:13:37,517 really understanding who you're talking to 247 00:13:37,600 --> 00:13:40,797 you know being able to interpret 248 00:13:40,880 --> 00:13:45,037 everything about how they're thinking about it and just kind of generally 249 00:13:45,120 --> 00:13:47,277 seeing decisions through the eyes of the 250 00:13:47,360 --> 00:13:49,117 people working in the company not 251 00:13:49,200 --> 00:13:54,397 through your eyes is the skill that you you know you develop by 252 00:13:54,480 --> 00:13:56,957 talking to people all the time, understanding what they're saying, so 253 00:13:57,040 --> 00:14:01,517 forth, these kinds of things. And it's just um 254 00:14:01,600 --> 00:14:05,517 you know, it's certainly not an IQ thing. And not that like I I could 255 00:14:05,600 --> 00:14:08,477 imagine an AI training on any individual 256 00:14:08,560 --> 00:14:11,197 and like figuring it all out and knowing 257 00:14:11,280 --> 00:14:14,237 what to say and so forth. Um but then 258 00:14:14,320 --> 00:14:17,437 you also need that integrated with uh 259 00:14:17,520 --> 00:14:21,277 you know like whatever the business ought to be doing. So you're not you're 260 00:14:21,360 --> 00:14:26,477 not trying to do what's popular. You're trying to get people to do what's 261 00:14:26,560 --> 00:14:30,797 correct even if they don't like it. And you know that's a lot of management. So 262 00:14:30,880 --> 00:14:34,877 uh it's not a problem anybody's working on 263 00:14:34,960 --> 00:14:36,317 currently but maybe they will. 264 00:14:36,400 --> 00:14:37,997 >> It's some right some combination of like 265 00:14:38,080 --> 00:14:40,957 courage some combination of motivation 266 00:14:41,040 --> 00:14:45,517 some combination of um of emotional uh understanding theory of mind. 267 00:14:45,600 --> 00:14:47,517 >> Yeah. what you know what do people want 268 00:14:47,600 --> 00:14:50,957 like you know married to you know what 269 00:14:51,040 --> 00:14:56,077 needs to be done and then like how talented are they like which ones can 270 00:14:56,160 --> 00:14:59,357 you afford like if they jump out the window it's fine you know which one's 271 00:14:59,440 --> 00:15:01,357 not fine you know this kind of thing 272 00:15:01,440 --> 00:15:06,477 it's a there's a lot of like weird subtleties to it uh and it's very 273 00:15:06,560 --> 00:15:08,157 situational I think the hardest thing 274 00:15:08,240 --> 00:15:10,637 about it uh and why management books are 275 00:15:10,720 --> 00:15:13,997 so bad is because it's situational um 276 00:15:14,080 --> 00:15:15,677 you know like your company your product, 277 00:15:15,760 --> 00:15:18,557 your people, your org chart is very very 278 00:15:18,640 --> 00:15:23,117 different than you know, yeah, here are the five steps to building a strategy. 279 00:15:23,200 --> 00:15:27,117 It's like, well, that's the most useless thing I ever read because it has 280 00:15:27,200 --> 00:15:30,237 nothing to do with you. >> So, one of the interesting things on 281 00:15:30,320 --> 00:15:33,277 this like on this is right the the concept of theory of mind is really 282 00:15:33,360 --> 00:15:36,157 important, right? So, theory of mind is can you in your head model what's 283 00:15:36,240 --> 00:15:38,557 happening in the other person's head, right? And and and you would think that 284 00:15:38,640 --> 00:15:41,037 maybe that you know maybe obviously people who are smarter should be better 285 00:15:41,120 --> 00:15:44,477 at that. It turns out that that may not be true and I'll the reason to believe 286 00:15:44,560 --> 00:15:48,397 that that's not true which which is as follows. So the so the the US military 287 00:15:48,480 --> 00:15:50,957 is was the early adopter and has continued to be sort of the leading 288 00:15:51,040 --> 00:15:55,597 adopter in in US society of of of actually IQ testing and they uh they 289 00:15:55,680 --> 00:15:58,237 they they basically launder it through something called the ASVAB which is 290 00:15:58,320 --> 00:16:01,597 their they call vocational aptitude battery test but it's basically an I 291 00:16:01,680 --> 00:16:05,917 it's essentially an IQ test. Um and uh so they they they they still use 292 00:16:06,000 --> 00:16:09,117 basically explicit IQ tests and they they slot people uh into different 293 00:16:09,200 --> 00:16:11,197 specialties and roles uh you know in 294 00:16:11,280 --> 00:16:16,157 part with according IQ um including into leadership roles. Um and and so they 295 00:16:16,240 --> 00:16:18,637 they know what everybody's IQ is and they they kind of organize organize 296 00:16:18,720 --> 00:16:21,677 around that. And one of the things that they found over the years is if the 297 00:16:21,760 --> 00:16:24,797 leader is more than one standard deviation of IQ away from the followers, 298 00:16:24,880 --> 00:16:28,957 it's a real problem. Um and and and that's true in both directions, right? 299 00:16:29,040 --> 00:16:33,997 Um if if the leader is not smart enough to be able to right manage the you know 300 00:16:34,080 --> 00:16:37,917 is to be able to mo you know for somebody who is less smart to model the 301 00:16:38,000 --> 00:16:41,117 mental behavior of somebody who's more smart is inher of course inherently very 302 00:16:41,200 --> 00:16:44,477 challenging and maybe impossible. But it turns out the reverse is also true which 303 00:16:44,560 --> 00:16:47,677 is if the leader is two standard deviations above the norm of the 304 00:16:47,760 --> 00:16:51,677 organization that he's running. He also loses theory of mind. Right? it it's 305 00:16:51,760 --> 00:16:56,317 it's actually very hard for very smart people to um to to model uh the internal 306 00:16:56,400 --> 00:16:59,997 thought processes of even moderately smart people. Um and so there there's 307 00:17:00,080 --> 00:17:03,597 actually there's actually a real there's actually a real need to have a level of 308 00:17:03,680 --> 00:17:07,597 connection there that's not just right and therefore by inference if you had a 309 00:17:07,680 --> 00:17:10,717 person or a or a machine that had you know a thousand IQ or something like it 310 00:17:10,800 --> 00:17:14,397 it may just be it would be so alien its understanding of reality would be so 311 00:17:14,480 --> 00:17:17,997 alien to the people or the things that it was managing that it wouldn't it 312 00:17:18,080 --> 00:17:20,717 wouldn't even be able to connect in any sort any sort of realistic way. So again 313 00:17:20,800 --> 00:17:23,677 this is a very good argument that like it yeah this is the world is going to be 314 00:17:23,760 --> 00:17:27,677 far from organized by IQ for yeah for centuries to come. 315 00:17:27,760 --> 00:17:29,517 >> Yeah and Zuckerberg had a great line 316 00:17:29,600 --> 00:17:32,397 which is intelligence is not life and 317 00:17:32,480 --> 00:17:36,717 life life has a lot of dimensionality to it that is independent of intelligence. 318 00:17:36,800 --> 00:17:40,477 I think that you know if you spend all your time 319 00:17:40,560 --> 00:17:45,357 working on intelligence you lose track of that. We sometimes say about uh some 320 00:17:45,440 --> 00:17:47,197 specific people that they're too smart 321 00:17:47,280 --> 00:17:52,397 to properly model or or you know um too they s sort of assume too much 322 00:17:52,480 --> 00:17:55,917 rationality on other people or they just o overthink things or overrationalize 323 00:17:56,000 --> 00:17:59,517 them. Um yeah just to your point that it's it's on everything. 324 00:17:59,600 --> 00:18:03,997 >> Yeah. Yeah. Yeah. People often people seldom do what's in their best interest 325 00:18:04,080 --> 00:18:06,557 I should say. >> You know I also suspect this kind of 326 00:18:06,640 --> 00:18:09,357 gets more into the biology side of things. I you know there's more and more 327 00:18:09,440 --> 00:18:14,477 scientific evidence that basically also that like human human cognition human 328 00:18:14,560 --> 00:18:17,117 cognition or human I don't whatever you want to call it self-awareness 329 00:18:17,200 --> 00:18:19,357 information processing decision-m sort 330 00:18:19,440 --> 00:18:24,477 of experience um uh is is not purely a brain like the basically the d the sort 331 00:18:24,560 --> 00:18:27,997 of mind famous mindbody dualism is just not correct like and again this is an 332 00:18:28,080 --> 00:18:31,917 argument against sort of IQ supremacism or intelligent supremacism is it's not 333 00:18:32,000 --> 00:18:34,957 you know we we human beings didn't experience existence just through the 334 00:18:35,040 --> 00:18:38,077 rational thought u and and and specifically not through just the 335 00:18:38,160 --> 00:18:40,877 rational thought of the brain, but rather it's a whole body experience, 336 00:18:40,960 --> 00:18:43,277 right? And there's there's there's aspects of our nervous system and 337 00:18:43,360 --> 00:18:46,557 there's aspects of everything from our gut biome to, you know, to to you know, 338 00:18:46,640 --> 00:18:50,797 to to smells, you know, to alactory senses and and um you know, and hormones 339 00:18:50,880 --> 00:18:55,837 and like all all kinds of like biochemical kind of aspects to life. Um 340 00:18:55,920 --> 00:18:58,237 I sus if you just kind of track the research, I suspect we're going to find 341 00:18:58,320 --> 00:19:01,837 is human cognition is a full body experience. Uh much much more than much 342 00:19:01,920 --> 00:19:06,397 more than people thought. Um and and so therefore to actually and this you know 343 00:19:06,480 --> 00:19:08,717 this is like a and this is you know one of the kind of big fundamental 344 00:19:08,800 --> 00:19:12,157 challenges in the AI AI field right now which is you know the form of AI that we 345 00:19:12,240 --> 00:19:15,917 have working is is the is the fully mind body dual version of it which is it's 346 00:19:16,000 --> 00:19:20,077 just a disembodi you know like a disembodied brain you know the robotics 347 00:19:20,160 --> 00:19:23,517 revolution for sure is coming when that happens when we put AI in in physical 348 00:19:23,600 --> 00:19:26,317 objects that move around the world you know you're going to be able to get 349 00:19:26,400 --> 00:19:29,997 closer to having that kind of you know inte integrated intellectual physical 350 00:19:30,080 --> 00:19:31,197 you know experience you're going to have 351 00:19:31,280 --> 00:19:37,720 sensors in the robots are going to be a lot more data. 352 00:19:45,040 --> 00:19:48,397 But it is just to me at least reading the research like that all those ideas 353 00:19:48,480 --> 00:19:51,757 feel very nent and we have a lot of work to do to try to figure that out. 354 00:19:51,840 --> 00:19:56,957 >> Do you have a sense for how they are at theory of mind today? Um or do you have 355 00:19:57,040 --> 00:20:00,157 a where the limitations are? you like to talk to them a lot. Are there any 356 00:20:00,240 --> 00:20:02,877 particular things that are particularly surprising to you as you do? 357 00:20:02,960 --> 00:20:05,757 >> Yeah, I would say generally they're really good. Um, yeah. And so like one 358 00:20:05,840 --> 00:20:08,717 of the one one of the more I find one of the more fascinating ways, you know, to 359 00:20:08,800 --> 00:20:12,637 to work with language models is actually have them create personas. Um, and uh 360 00:20:12,720 --> 00:20:16,397 and then you know, basically have well actually so I like I like I like 361 00:20:16,480 --> 00:20:19,117 basically I like Socratic dialogues. I like when things are argued out and like 362 00:20:19,200 --> 00:20:23,117 a Socratic dialogue. And so you you know tell a tell any advanced LLM today to 363 00:20:23,200 --> 00:20:25,757 create a Socratic dialogue and it'll either make up the personas, you can 364 00:20:25,840 --> 00:20:29,037 tell what it is, it does a good job. it has this very very annoying property 365 00:20:29,120 --> 00:20:31,197 which is it wants everybody to be happy. 366 00:20:31,280 --> 00:20:35,437 Um and so it wants all of its personas to agree. Um and so by default it will 367 00:20:35,520 --> 00:20:39,117 have a uh it will have a briefly interesting discussion and then it will 368 00:20:39,200 --> 00:20:42,477 sort of figure out you know basically it like you're watching I don't know PBS 369 00:20:42,560 --> 00:20:44,397 special or something. It'll it'll kind of figure out how to bring everybody 370 00:20:44,480 --> 00:20:46,797 into agreement. Everybody's happy at the end of the discussion. And of course I 371 00:20:46,880 --> 00:20:50,877 hate that. Like it drives me nuts. I don't want that. So instead I I 372 00:20:50,960 --> 00:20:52,637 tell it I'm like make the conversation 373 00:20:52,720 --> 00:20:57,997 more tense, right? And like fraught with like anger and like you know people, you 374 00:20:58,080 --> 00:21:01,357 know, going to get like increasingly upset throughout the conversation. And 375 00:21:01,440 --> 00:21:04,237 then it starts to get really interesting. Um and then I and then I 376 00:21:04,320 --> 00:21:07,997 tell it, you know, bring it, you know, use introduce a lot more cursing. Um you 377 00:21:08,080 --> 00:21:10,797 know, really have them go at it like all the gloves come off, they're going for 378 00:21:10,880 --> 00:21:13,357 pull full, you know, you know, reputational destruction of each other. 379 00:21:13,440 --> 00:21:14,637 >> You do a lot of these skits. 380 00:21:14,720 --> 00:21:16,797 >> Yeah, these skits. And then I get carried away and then I'm like it turns 381 00:21:16,880 --> 00:21:19,197 out they're all like secret ninjas and then they all start fighting and you've 382 00:21:19,280 --> 00:21:22,717 got Einstein you know you know you know hitting you know Neils Bore with 383 00:21:22,800 --> 00:21:26,637 nunchucks and it and by the way it's happy to do that too. Um so you do have 384 00:21:26,720 --> 00:21:29,597 to you have to you have to control yourself but it it is very good a theory 385 00:21:29,680 --> 00:21:32,557 of mind and then I'll give you another example. There's a there's a startup 386 00:21:32,640 --> 00:21:36,637 actually in the UK uh in uh in in the world of politics and and what what they 387 00:21:36,720 --> 00:21:40,797 found is that um they found that language models now are good enough so 388 00:21:40,880 --> 00:21:44,477 specifically for for politics which is sort of a sub subcategory where this 389 00:21:44,560 --> 00:21:48,477 this idea matters. Um so you know in politics people do focus group you do 390 00:21:48,560 --> 00:21:51,757 focus groups of voters all the time and and by the way many businesses also do 391 00:21:51,840 --> 00:21:54,477 that. Um, you know, so you get a bunch of people together from different 392 00:21:54,560 --> 00:21:56,797 backgrounds in a room and you kind of guide them through a discussion and try 393 00:21:56,880 --> 00:21:59,517 to get their their points of view on things. And and focus groups are often 394 00:21:59,600 --> 00:22:02,637 surprising. Like politicians who if you talk to politicians who do focus groups, 395 00:22:02,720 --> 00:22:05,117 they're often surprising. They're often surprised by the things that they 396 00:22:05,200 --> 00:22:07,597 thought voters cared about is actually not the things that voters care about. 397 00:22:07,680 --> 00:22:10,797 And so you can actually learn a lot by doing this. But focus groups are very 398 00:22:10,880 --> 00:22:13,197 expensive to run. And then there's a long lag time because they have to be 399 00:22:13,280 --> 00:22:15,997 actually physically organized and you have to recruit people and vet people 400 00:22:16,080 --> 00:22:19,597 and and so forth. Um, and so it turns out that the the the the 401 00:22:19,680 --> 00:22:23,037 state-of-the-art models now are are good enough at this so they can actually they 402 00:22:23,120 --> 00:22:27,277 can they can correctly um accurately reproduce a focus group of real people 403 00:22:27,360 --> 00:22:30,957 um inside the model. Um so so so they're good enough to clear that bar. In other 404 00:22:31,040 --> 00:22:33,837 words, you you can basically have a focus group actually happening in the 405 00:22:33,920 --> 00:22:36,877 model where you create personas in the model and then it actually accurately 406 00:22:36,960 --> 00:22:40,157 represents, you know, a college student from, you know, Kentucky is contrasted 407 00:22:40,240 --> 00:22:42,797 to a housewife from Tennessee is contrasted to a, you know, whatever 408 00:22:42,880 --> 00:22:44,797 whatever you you just like specify this. 409 00:22:44,880 --> 00:22:47,197 And so, you know, they're good enough to clear they're good enough to clear that 410 00:22:47,280 --> 00:22:50,157 bar and, you know, we'll we'll see how far they get. 411 00:22:50,240 --> 00:22:52,797 >> I want to segue to the bubble 412 00:22:52,880 --> 00:22:56,157 conversation. Uh, Amin and G2, Jensen 413 00:22:56,240 --> 00:22:58,557 and Matt spoke about the enormous scale 414 00:22:58,640 --> 00:23:00,397 of physical infrastructure being built 415 00:23:00,480 --> 00:23:04,237 out. AI capex is 1% of GDP. How should 416 00:23:04,320 --> 00:23:07,437 we understand and think about this bubble question? 417 00:23:07,520 --> 00:23:10,797 >> Well, I think the fact that it's a question means we're not in a bubble. 418 00:23:10,880 --> 00:23:14,557 That's the first thing to understand. I mean, a bubble is a psychological 419 00:23:14,640 --> 00:23:19,437 phenomenon um as much as anything. And in order to get to a bubble, everybody 420 00:23:19,520 --> 00:23:23,757 has to believe it's not a bubble. That that's sort of the the core mechanic of 421 00:23:23,840 --> 00:23:28,157 it. And that, you know, we call that capitulation. Everybody just gives up. 422 00:23:28,240 --> 00:23:31,997 Like, okay, I'm not going to short these stocks anymore. I'm tired of losing all 423 00:23:32,080 --> 00:23:37,277 my money. I'm going to go long. Uh and we saw that actually in, 424 00:23:37,360 --> 00:23:38,877 you know, and a little bit of question 425 00:23:38,960 --> 00:23:41,757 like really what was the tech bubble? Um 426 00:23:41,840 --> 00:23:45,597 but in the kind of.com era right as the 427 00:23:45,680 --> 00:23:49,437 prices went through the roof Warren Buffett started inviting investing in 428 00:23:49,520 --> 00:23:52,717 tech. So like and he swore he would never invest in tech because you didn't 429 00:23:52,800 --> 00:23:57,117 understand it. And so if he capitulated nobody was saying it was a bubble when 430 00:23:57,200 --> 00:23:59,357 it became like a quote unquote bubble. 431 00:23:59,440 --> 00:24:02,957 Now if you look at that phenomenon 432 00:24:03,040 --> 00:24:07,197 um the internet clearly was not a bubble. Uh you know it was a real thing. 433 00:24:07,280 --> 00:24:09,837 It was in the short term there was a 434 00:24:09,920 --> 00:24:12,797 kind of price dislocation that happened 435 00:24:12,880 --> 00:24:17,037 because uh the the market um you know 436 00:24:17,120 --> 00:24:21,197 there were just not enough people on the network to make those products go at the 437 00:24:21,280 --> 00:24:26,317 time uh and then the prices kind of outran the market. 438 00:24:26,400 --> 00:24:29,837 you know, in AI, it's much harder to see that because there's so much demand in 439 00:24:29,920 --> 00:24:34,237 the short term, right? Like we don't have a demand problem right now. And 440 00:24:34,320 --> 00:24:35,917 like the idea that we're going to have a 441 00:24:36,000 --> 00:24:37,997 demand problem five years from now to me 442 00:24:38,080 --> 00:24:41,357 seems quite absurd. Uh, you know, could 443 00:24:41,440 --> 00:24:46,717 there be like weird bottlenecks that that appear, you know, like we just at 444 00:24:46,800 --> 00:24:50,397 some point we just don't have enough cooling or something like that? Yeah, 445 00:24:50,480 --> 00:24:53,677 you know, maybe. But like like right now 446 00:24:53,760 --> 00:24:57,437 if you look at demand and supply and 447 00:24:57,520 --> 00:25:02,557 what's going on and multiples uh against growth, 448 00:25:02,640 --> 00:25:06,637 it doesn't look like a bubble at all to me. Um but 449 00:25:06,720 --> 00:25:08,477 I don't know. Do you think it's a bubble, Mark? 450 00:25:08,560 --> 00:25:10,237 >> Yeah. Look, I I would just say this. 451 00:25:10,320 --> 00:25:13,517 Yeah. Like nobody know So nobody knows in the sense of like the experts like if 452 00:25:13,600 --> 00:25:15,597 you're talking to anybody at like a hedge fun or a bank or whatever, like 453 00:25:15,680 --> 00:25:20,237 they definitely don't know. Um uh generally the CEOs don't know. So it the 454 00:25:20,320 --> 00:25:24,397 >> by the way a lot of VCs don't know that they just get upset like VCs get like 455 00:25:24,480 --> 00:25:26,637 emotionally upset when you guys have 456 00:25:26,720 --> 00:25:29,357 higher valuations like it it makes them 457 00:25:29,440 --> 00:25:32,797 like like angry. Uh and you know and I I 458 00:25:32,880 --> 00:25:34,877 get it all the time and I'm like what 459 00:25:34,960 --> 00:25:37,277 are you mad about? Like the is 460 00:25:37,360 --> 00:25:40,797 working man be happy. Come on. But so so 461 00:25:40,880 --> 00:25:45,837 like there's a lot of emotion around like people wanting it to be be a bubble. 462 00:25:45,920 --> 00:25:48,397 >> Yeah. No, nothing's worse than passing on a deal and then having the company 463 00:25:48,480 --> 00:25:50,717 become a great success. Like it's just I was just just put 464 00:25:50,800 --> 00:25:52,477 >> that that valuation is outrageous. 465 00:25:52,560 --> 00:25:55,117 >> You can be furious about that for 30 years in our business. It's it's 466 00:25:55,200 --> 00:25:58,637 amazing. Um and you can find Yeah. You come up with all kinds of reasons to 467 00:25:58,720 --> 00:26:01,917 cope and and explain why it wasn't your mistake. But it's you know it's the 468 00:26:02,000 --> 00:26:05,997 world it's the world that's wrong, not me. Right. So there there's a lot of 469 00:26:06,080 --> 00:26:09,037 that. Yeah. Uh yeah. So I I just I would just I would just say like I would 470 00:26:09,120 --> 00:26:12,157 always say bring the conversation back to ground truth fundamentals. And the 471 00:26:12,240 --> 00:26:15,517 the two big ground truth fundamentals are number one, does the technology 472 00:26:15,600 --> 00:26:18,637 actually work? Um, you know, can it deliver on its promise? And then number 473 00:26:18,720 --> 00:26:21,997 two, is are customers paying for it? Um, and if those if those two things are 474 00:26:22,080 --> 00:26:25,597 true, then it's very hard to it's very hard to uh like as long as those two 475 00:26:25,680 --> 00:26:28,957 things stay grounded. Um, you know, gen generally generally things are going to 476 00:26:29,040 --> 00:26:31,037 I think are going to be on track. 477 00:26:31,120 --> 00:26:34,477 >> Yeah. When Gavin was up here with DG, he said 478 00:26:34,560 --> 00:26:38,157 chatbt was a Pearl Harbor moment for Google, the moment when the giant wakes 479 00:26:38,240 --> 00:26:41,917 up. when when we look at history and and platform shifts, what determine whether 480 00:26:42,000 --> 00:26:45,117 the incumbent actually wins the next wave versus versus new entrance or how 481 00:26:45,200 --> 00:26:48,637 should we think about that in in AMP? 482 00:26:48,720 --> 00:26:51,597 >> Well, you know, reacting to it is 483 00:26:51,680 --> 00:26:55,357 important. Um, but that doesn't mean 484 00:26:55,440 --> 00:26:58,477 like I it's a Pearl Harbor moment. I 485 00:26:58,560 --> 00:26:59,997 think 486 00:27:00,080 --> 00:27:02,797 Google got their head out of their ass. 487 00:27:02,880 --> 00:27:06,600 It was the sound of it. 488 00:27:06,960 --> 00:27:11,837 Uh so you know they're not going to get completely run over but 489 00:27:11,920 --> 00:27:15,597 nonetheless like I don't think open AI is going away. So like they they 490 00:27:15,680 --> 00:27:20,797 definitely let that happened. Um yeah some of it to speed and then just look 491 00:27:20,880 --> 00:27:22,557 it's execution over a long period of 492 00:27:22,640 --> 00:27:25,197 time and uh you know some of these very 493 00:27:25,280 --> 00:27:28,077 large companies to varying degrees have 494 00:27:28,160 --> 00:27:31,997 lost their ability to execute. And so if 495 00:27:32,080 --> 00:27:35,837 you're talking about a brand new platform and you're talking about, you 496 00:27:35,920 --> 00:27:38,957 know, kind of building for a long time, it's like, you know, Microsoft got 497 00:27:39,040 --> 00:27:41,197 caught with their pants down on Google. 498 00:27:41,280 --> 00:27:44,317 Um, Microsoft's still like very strong, 499 00:27:44,400 --> 00:27:47,277 but they missed that whole opportunity. 500 00:27:47,360 --> 00:27:52,477 They also missed the opportunity. You know, Apple was nothing and Microsoft 501 00:27:52,560 --> 00:27:55,437 fully believed that they were going to own mobile computing. They completely 502 00:27:55,520 --> 00:27:58,797 missed that one. But they were still so big from their Windows monopoly they 503 00:27:58,880 --> 00:28:01,917 could build into other things. So you 504 00:28:02,000 --> 00:28:03,997 know I think generally the new companies 505 00:28:04,080 --> 00:28:07,277 have won the new markets. Uh 506 00:28:07,360 --> 00:28:11,197 and that doesn't mean the big company the biggest companies the biggest 507 00:28:11,280 --> 00:28:15,277 monopolies from the prior generation just last a long time is the way I would 508 00:28:15,360 --> 00:28:18,237 look at it. >> Yeah. I I also think we don't quite know 509 00:28:18,320 --> 00:28:22,157 like it's all happened so fast we we actually don't I think we don't yet know 510 00:28:22,240 --> 00:28:24,557 the shape and form of the ultimate products. 511 00:28:24,640 --> 00:28:27,517 >> Yeah. >> Um Right. And so and so like because 512 00:28:27,600 --> 00:28:29,917 it's it's tempting and this is kind of what what always happens. It's kind of 513 00:28:30,000 --> 00:28:32,397 it's kind of tempting to look I'm not saying what that's what these guys did 514 00:28:32,480 --> 00:28:35,037 on stage but it's kind of tempting to look you sometimes you hear the kind of 515 00:28:35,120 --> 00:28:37,437 reductive version of this which is basically it's like oh there's either 516 00:28:37,520 --> 00:28:40,717 going to be a chatbot or a search engine right the competition is between a 517 00:28:40,800 --> 00:28:44,237 chatbot and a search engine and the problem Google has is the classic 518 00:28:44,320 --> 00:28:47,037 problem of dis you know disruption. are you going to disrupt the 10 blue links 519 00:28:47,120 --> 00:28:50,797 model and swap in you know at you know sort of uh AI answers and you know 520 00:28:50,880 --> 00:28:53,757 potentially disrupt the advertising model and then the problem OpenAI has is 521 00:28:53,840 --> 00:28:56,477 they have the the full you know the full chat product but you know they don't 522 00:28:56,560 --> 00:28:59,437 have the advertising yet and they don't have the distribution Google scale 523 00:28:59,520 --> 00:29:02,877 distribution and so you know you kind of say okay that's a fairly that's a fairly 524 00:29:02,960 --> 00:29:05,917 cl like that'd be straight out of a like you know the innovator's dilemma you 525 00:29:06,000 --> 00:29:10,237 know business textbook like this is just a very clear you know one one versus one 526 00:29:10,320 --> 00:29:14,397 you know kind of dynamic but that assumes that you know the mistake that 527 00:29:14,480 --> 00:29:17,437 you could make in thinking way is that assumes that the forms of the product in 528 00:29:17,520 --> 00:29:21,117 5 10 15 20 years that that are going to be the main things that people use are 529 00:29:21,200 --> 00:29:25,277 going to be either a search engine or a chatbot, right? Um and and you know the 530 00:29:25,360 --> 00:29:28,797 just there's you know there's just obvious historical analogies. One just 531 00:29:28,880 --> 00:29:32,477 obvious historical analogy is, you know, the the personal computer from sort of 532 00:29:32,560 --> 00:29:37,117 invention in 1975 through to, you know, basically 1992, 533 00:29:37,200 --> 00:29:40,797 you know, was was a was a text prompt system, right? Um, you know, and at the 534 00:29:40,880 --> 00:29:43,757 time, by the way, an interactive text prompt was a big advance over the 535 00:29:43,840 --> 00:29:46,957 previous generation of like punch card systems, time sharing systems. Uh, and 536 00:29:47,040 --> 00:29:51,357 then, you know, it was, you know, 1992, so was what, seven, 17 years in, you 537 00:29:51,440 --> 00:29:54,317 know, the whole industry took a left turn into GUIEs and never looked back, 538 00:29:54,400 --> 00:29:56,957 you know. And then by the way, you know, 5 years after that, the industry took a 539 00:29:57,040 --> 00:30:00,557 left turn in web browsers and never looked back, right? And so the very 540 00:30:00,640 --> 00:30:04,077 shape and form and nature of the user experience and how it and how it fits 541 00:30:04,160 --> 00:30:08,077 into our lives, uh, you know, is is is I think still unformed. And so like and 542 00:30:08,160 --> 00:30:10,637 you know, look, I'm I'm sure there will be chat bots 20 years from now, but I 543 00:30:10,720 --> 00:30:14,797 I'm I'm pretty confident that, um, you know, both the current chatbot companies 544 00:30:14,880 --> 00:30:17,677 and many new companies are going to figure out many kinds of user 545 00:30:17,760 --> 00:30:20,957 experiences that are radically different that we don't we don't even know yet. 546 00:30:21,040 --> 00:30:23,277 Yeah. And by and by the way, that's one of the things of course that keeps the 547 00:30:23,360 --> 00:30:26,077 tech industry fun, which is it, you know, especially on the especially on 548 00:30:26,160 --> 00:30:29,197 the software side, you know, is it's not it's not it's not obvious what the shape 549 00:30:29,280 --> 00:30:31,197 and form of the products are. And there's just I think there's just 550 00:30:31,280 --> 00:30:32,557 tremendous headroom for invention. 551 00:30:32,640 --> 00:30:35,597 >> As as you're coaching entrepreneurs and the entrepreneurs in this room, what 552 00:30:35,680 --> 00:30:40,237 what else feels different about this era or or or other advice that you find 553 00:30:40,320 --> 00:30:43,997 yourself whether it's around uh sort of the talent wars that are going on or 554 00:30:44,080 --> 00:30:47,437 other aspects that feel unique to this era? What what other advice do you want 555 00:30:47,520 --> 00:30:50,717 to be leaving our entrepreneurs with? 556 00:30:50,800 --> 00:30:54,157 that's unique to this era. Well, like I I I actually think you said the right 557 00:30:54,240 --> 00:30:57,117 thing, which is this is a unique era. 558 00:30:57,200 --> 00:30:59,277 And so 559 00:30:59,360 --> 00:31:01,757 trying to 560 00:31:01,840 --> 00:31:05,117 learn the organizational design lessons 561 00:31:05,200 --> 00:31:08,717 of the past or trying to learn um kind 562 00:31:08,800 --> 00:31:13,997 of too much from the last generation is can be deceptive 563 00:31:14,080 --> 00:31:15,437 because things really are different. 564 00:31:15,520 --> 00:31:17,357 like the way these you know the way your 565 00:31:17,440 --> 00:31:19,517 companies are getting built is is quite 566 00:31:19,600 --> 00:31:23,837 different in in many aspects. Uh and you 567 00:31:23,920 --> 00:31:26,397 know the types of 568 00:31:26,480 --> 00:31:29,677 you know what the just like our observ 569 00:31:29,760 --> 00:31:33,837 observation on like PhD AI researchers 570 00:31:33,920 --> 00:31:36,317 is just very different than like a 571 00:31:36,400 --> 00:31:39,357 traditional um engineer full stack 572 00:31:39,440 --> 00:31:42,397 engineer or something like that. So you 573 00:31:42,480 --> 00:31:46,397 know I I think you do have to think through a lot of things from first 574 00:31:46,480 --> 00:31:50,797 principles uh because it is different and like you know observing from the 575 00:31:50,880 --> 00:31:52,877 outside it's really different. 576 00:31:52,960 --> 00:31:54,717 >> Yeah. >> Yeah. And I would just offer like I I do 577 00:31:54,800 --> 00:31:56,797 think things are going to change. So I already talked about I think the shape 578 00:31:56,880 --> 00:31:58,477 and form of products is going to change. 579 00:31:58,560 --> 00:32:00,637 Um uh and so like I think there's still 580 00:32:00,720 --> 00:32:02,397 a lot of creativity there. I also think 581 00:32:02,480 --> 00:32:06,157 and I I I let's say I think that um like 582 00:32:06,240 --> 00:32:09,757 in a in a world of supply and demand the thing that creates gluts is shortages. 583 00:32:09,840 --> 00:32:13,197 Um right so like when something becomes too scarce there becomes a massive 584 00:32:13,280 --> 00:32:16,797 economic incentive to figure out how to unlock new supply and so the the the 585 00:32:16,880 --> 00:32:19,917 current generation of AI companies are really struggling with uh particular 586 00:32:20,000 --> 00:32:23,357 shortages of of the really talented AI researchers and engineers and then 587 00:32:23,440 --> 00:32:27,277 they're really you know challenged with shortage of of infrastructure capacity 588 00:32:27,360 --> 00:32:31,597 chips and and data centers and power. Um I I don't want to call timing on this. 589 00:32:31,680 --> 00:32:35,677 There will come a time when both of those things become gluts. Um and so you 590 00:32:35,760 --> 00:32:39,037 I don't know I don't know that we can plan for that. Um although I I would 591 00:32:39,120 --> 00:32:43,437 just say the following. Number one, um the the the researcher engineer side of 592 00:32:43,520 --> 00:32:47,517 things, it is striking. It is striking to the degree to which um there are 593 00:32:47,600 --> 00:32:51,197 excellent, you know, outstanding models coming out of China now. Um you know, 594 00:32:51,280 --> 00:32:54,397 and and in a m from multiple companies and you know, specifically, you know, uh 595 00:32:54,480 --> 00:32:58,397 Deepseek and and Quinn and Kimmy. Um it is striking how the teams that are 596 00:32:58,480 --> 00:33:02,157 making those are not, you know, the name brand, you know, for the most part, 597 00:33:02,240 --> 00:33:05,597 these are not like the name brand people with their names on all the papers. Um 598 00:33:05,680 --> 00:33:09,277 and and so like China is successfully decoding how to like basically take 599 00:33:09,360 --> 00:33:10,877 young people and train them up in the field. 600 00:33:10,960 --> 00:33:13,197 >> Well, and XAI to a large extent too. 601 00:33:13,280 --> 00:33:17,357 >> Yeah. Yeah. And so I I think that I think there's going to be and look it 602 00:33:17,440 --> 00:33:20,317 makes sense up until it it makes sense that for a while it's going to be this 603 00:33:20,400 --> 00:33:23,037 super esoteric skill set and people are going to pay through the nose for it. 604 00:33:23,120 --> 00:33:26,157 But like you know there's no question the information is right being 605 00:33:26,240 --> 00:33:28,957 transferred into the environment. People are learning how to do this. Um you know 606 00:33:29,040 --> 00:33:32,957 college kids are figuring it out. Um, and so, um, you know, there's there's 607 00:33:33,040 --> 00:33:35,597 and I don't know that there's ever going to be a talent glut per se, but like I 608 00:33:35,680 --> 00:33:37,517 think for sure there's gonna there's gonna be a lot more people in the future 609 00:33:37,600 --> 00:33:40,717 who of course know know how to build these things. Um, and then and then by 610 00:33:40,800 --> 00:33:43,437 the way also of course you know AI building AI, right? So that the the the 611 00:33:43,520 --> 00:33:46,797 the tools themselves are going to be better better at at contributing to 612 00:33:46,880 --> 00:33:50,237 that. And so and and I think that I think this is good because I think that 613 00:33:50,320 --> 00:33:52,957 you know the current level of of shortage of of engineers and researchers 614 00:33:53,040 --> 00:33:55,837 is is is too constraining. And then and then on the chip side I don't I don't 615 00:33:55,920 --> 00:33:58,237 want to I'm not a chip guy and I don't want to call call it specifically but 616 00:33:58,320 --> 00:34:00,957 like it it's never been the case. It's never been the case in the ship industry 617 00:34:01,040 --> 00:34:04,237 that there's ever, you know, every every shortage in the ship industry has always 618 00:34:04,320 --> 00:34:08,397 resulted in a glut uh because the the profit the profit pool of a shortage, 619 00:34:08,480 --> 00:34:11,517 the margins get too big, the incentive uh for other people to come in and 620 00:34:11,600 --> 00:34:14,157 figure out how to commoditize the function get too big. And so, you know, 621 00:34:14,240 --> 00:34:16,877 Nvidia has like, you know, the best position probably anybody's ever had in 622 00:34:16,960 --> 00:34:19,917 chips. But notwithstanding that, I I find it hard to believe that there's 623 00:34:20,000 --> 00:34:22,237 going to be this level of pressure on infrastructure in 5 years. 624 00:34:22,320 --> 00:34:24,477 >> Yeah. And even if the bottleneck within 625 00:34:24,560 --> 00:34:29,277 the infrastructure moves, so if if it becomes power, if it becomes cooling or 626 00:34:29,360 --> 00:34:32,957 or or anything else, then you'll have a chip glut for sure. Yeah. 627 00:34:33,040 --> 00:34:36,157 >> So So I I think over the I I would just say this, it's likely the challenges 628 00:34:36,240 --> 00:34:38,477 that we that we all have in five years from now are going to be different 629 00:34:38,560 --> 00:34:42,237 challenges. >> Yeah. Yeah. Yeah. Like don't 630 00:34:42,320 --> 00:34:46,477 don't definitely this industry of all industries don't look at us as static 631 00:34:46,560 --> 00:34:51,437 like you know the positions uh could change very very fast. 632 00:34:51,520 --> 00:34:55,757 Let's actually close on a more of this macro note. Mark, you mentioned China. 633 00:34:55,840 --> 00:34:59,597 Last month, we were in DC and what one of the big questions the senator has is 634 00:34:59,680 --> 00:35:03,037 how should we make sense of sort of the state of the AI race visa v China. Do 635 00:35:03,120 --> 00:35:06,797 you you want to share just the the high level um summary of what what you shared 636 00:35:06,880 --> 00:35:09,597 with them? >> Yeah. So my sense of things and I and I 637 00:35:09,680 --> 00:35:12,637 think the current I think the current if you just observe currently specifically 638 00:35:12,720 --> 00:35:14,637 like deep sea quan and and these models 639 00:35:14,720 --> 00:35:19,997 coming out of China I my my sense basically is like I would say the US 640 00:35:20,080 --> 00:35:22,957 specifically in the west generally but you know more and more specifically the 641 00:35:23,040 --> 00:35:26,557 US is like the conceptual innovations are you know have been you know coming 642 00:35:26,640 --> 00:35:29,197 out coming out of coming out of the US coming out of the west you know kind of 643 00:35:29,280 --> 00:35:33,837 the the big kind of conceptual breakthroughs um uh China is extremely 644 00:35:33,920 --> 00:35:36,637 good at picking up ideas and implementing them and scaling them and 645 00:35:36,720 --> 00:35:39,437 commoditizing them and and you know that's that they do that obviously 646 00:35:39,520 --> 00:35:43,117 throughout the manufacturing world. Um and and they're doing it now very I 647 00:35:43,200 --> 00:35:47,357 think successfully uh sort of in AI. Um and so I would say that they're running 648 00:35:47,440 --> 00:35:50,717 they're running the catch-up game like really well. Um you know and then 649 00:35:50,800 --> 00:35:53,517 there's there's sort of always this question of like how much of that is 650 00:35:53,600 --> 00:35:57,197 like being done let's just say like authentically um uh you know through 651 00:35:57,280 --> 00:36:00,237 hard work um and smart people and then how much is being done with maybe a 652 00:36:00,320 --> 00:36:03,117 little bit of help um maybe a little USB 653 00:36:03,200 --> 00:36:07,997 stick uh in the middle of the night uh you know kind of help um Okay. 654 00:36:08,080 --> 00:36:10,157 >> So, uh, you know, there's always a little bit of a question, but like 655 00:36:10,240 --> 00:36:13,677 either either way, uh, you know, they're they're doing a great job. Uh, obviously 656 00:36:13,760 --> 00:36:17,197 they they aspire to, you know, more than that. Um, and there are many very smart 657 00:36:17,280 --> 00:36:19,997 and creative people in China. And so, you know, it will be interesting now to 658 00:36:20,080 --> 00:36:22,797 see, you know, the level to which the conceptual breakthroughs start to come 659 00:36:22,880 --> 00:36:26,877 from there and whether they whether they pull ahead. Um, and so, but like I would 660 00:36:26,960 --> 00:36:29,437 say like what we tell people in Washington is like look this this is a 661 00:36:29,520 --> 00:36:31,277 foot this is now this is a full-on race. 662 00:36:31,360 --> 00:36:33,517 It's a foot race. It's a game of inches. 663 00:36:33,600 --> 00:36:36,317 Like we're not going to have a 5year lead. we're going to have like maybe a 664 00:36:36,400 --> 00:36:39,757 six-month lead. Like we have to run fast. We have to win. Like we we have to 665 00:36:39,840 --> 00:36:42,717 we have to do this. We we can't and then we can't put constraints on our 666 00:36:42,800 --> 00:36:45,597 companies that the the the Chinese government isn't putting on their own 667 00:36:45,680 --> 00:36:49,677 companies. And so um you know we'll just lose and you know do do you really want 668 00:36:49,760 --> 00:36:51,677 do you really want to wake up in the morning and live in a world you know 669 00:36:51,760 --> 00:36:53,837 really controlled and run by Chinese AI? 670 00:36:53,920 --> 00:36:55,277 Most of us would say no we don't want to 671 00:36:55,360 --> 00:36:59,117 live in that world. Um and so um so so 672 00:36:59,200 --> 00:37:02,317 that so there's that and I would say I feel moderately good about that just 673 00:37:02,400 --> 00:37:05,837 because I think that I think we're we're really good at software. um you know the 674 00:37:05,920 --> 00:37:09,197 minute this goes into you know embodied AI in the form of robotics I think 675 00:37:09,280 --> 00:37:12,317 things get a lot scarier and you know this is the thing I'm now spending time 676 00:37:12,400 --> 00:37:16,637 in DC trying to really educate people on which is you know the ch because the US 677 00:37:16,720 --> 00:37:19,597 and the west have chosen to de-industrialize to the extent that we 678 00:37:19,680 --> 00:37:24,157 have over the last 40 years um you know China China specifically now has this 679 00:37:24,240 --> 00:37:28,717 giant industrial ecosystem for building you know sort of mechanical electrical 680 00:37:28,800 --> 00:37:33,357 and um uh and semiconductor and now software you know devices of all kinds 681 00:37:33,440 --> 00:37:39,037 including phones and drones phones and cars um and robots. Um and so uh you 682 00:37:39,120 --> 00:37:41,757 know there's going to be a phase two to the AI revolution. It's going to be 683 00:37:41,840 --> 00:37:45,917 robotics. It's going to happen you know pretty quickly here I think. Um and when 684 00:37:46,000 --> 00:37:49,437 it does like even if the US stays ahead in software like the robots's got to get 685 00:37:49,520 --> 00:37:53,197 built and that's not an easy thing. And it's not just like a company that does 686 00:37:53,280 --> 00:37:55,997 that. It's got to be an entire ecosystem. Um and it's you know it's 687 00:37:56,080 --> 00:37:59,037 going to be you know it's going to you know like I mean you know the car 688 00:37:59,120 --> 00:38:01,837 industry was not three car companies. It was thousands and thousands of of of 689 00:38:01,920 --> 00:38:04,557 component suppliers building all the parts. And it's been the same thing for 690 00:38:04,640 --> 00:38:07,437 airplanes and the same thing for computers and everything else. Um it's 691 00:38:07,520 --> 00:38:08,797 going to be the same thing for robotics. 692 00:38:08,880 --> 00:38:12,317 Um and you know by default sitting here today that's all going to happen in 693 00:38:12,400 --> 00:38:15,597 China. And so even if they never quite catch us in software they might just 694 00:38:15,680 --> 00:38:19,917 like lap us in hardware and and that'll be that. Um you know the good news is I 695 00:38:20,000 --> 00:38:23,277 I think there's a growing awareness in there's a growing awareness I would say 696 00:38:23,360 --> 00:38:26,797 across the political political spectrum in the US that like de-industrialization 697 00:38:26,880 --> 00:38:29,597 went too far. Uh and there's a growing desire to kind of figure out how to 698 00:38:29,680 --> 00:38:33,757 reverse that. Um, and um, you know, I say I'm guardedly optimistic that we'll 699 00:38:33,840 --> 00:38:37,357 be making progress on that, but I think there's a lot of work to be done 700 00:38:37,440 --> 00:38:39,837 >> on that call to arms. Uh, let's wrap. 701 00:38:39,920 --> 00:38:43,997 Uh, thank you Mark and Ben. To to wrap up, I'd like to welcome back. 702 00:38:44,080 --> 00:38:51,080 >> Thank you, everybody.