WEBVTT 00:00:00.080 --> 00:00:02.637 We're dealing with magic here that we I think probably all would have thought 00:00:02.720 --> 00:00:05.677 was impossible 5 years ago or certainly 10 years ago. This is the most amazing 00:00:05.760 --> 00:00:07.997 technology ever and it's moving really fast and yet we're still like really 00:00:08.080 --> 00:00:11.197 disappointed. Like it's not moving fast enough and like it's like maybe right on 00:00:11.280 --> 00:00:13.917 the verge of stalling out. We should both be like hyper excited but also on 00:00:14.000 --> 00:00:17.117 the verge of like slitting our wrists cuz like you know the gravy train is 00:00:17.200 --> 00:00:20.557 coming to an end, >> right? >> It is faster but it's not at computer 00:00:20.640 --> 00:00:23.597 speed, right? What we expect computer speed to be. It's sort of like watching 00:00:23.680 --> 00:00:24.637 a person work. 00:00:24.720 --> 00:00:28.477 >> It's like watching John Carmarmac 00:00:28.560 --> 00:00:30.637 >> the world. Okay. the world's the world's 00:00:30.720 --> 00:00:32.557 best programmer on a stimulus. 00:00:32.640 --> 00:00:36.680 >> On a stimulant. Yeah, that's right. 00:00:37.200 --> 00:00:41.277 >> So, let's start with um let's assume that I'm a sort of a novice programmer. 00:00:41.360 --> 00:00:44.957 So, maybe I'm a student um uh or maybe I'm just somebody, you know, I took a 00:00:45.040 --> 00:00:47.437 few coding classes and I've hacked around a little bit or like I don't 00:00:47.520 --> 00:00:51.277 know, I do Excel macros or something like that, but I'm like not less. I'm 00:00:51.360 --> 00:00:53.597 not like a master craftsman at coding. 00:00:53.680 --> 00:00:57.917 Um and you know people somebody tells me about replet and and specifically AI um 00:00:58.000 --> 00:01:01.757 AI and Replet like what's my what's my experience uh when when I launch in with 00:01:01.840 --> 00:01:03.677 with what replet is today with AI. 00:01:03.760 --> 00:01:07.597 >> Yeah I I would um I I think the experience of someone with no coding 00:01:07.680 --> 00:01:10.797 experience or some coding experience is largely the same when you go into 00:01:10.880 --> 00:01:13.997 replet. Okay. >> The first thing we try to do is get all 00:01:14.080 --> 00:01:17.757 the nonsense away from like setting up development environment and all of that 00:01:17.840 --> 00:01:21.437 stuff and just have you focus on your idea. So what do you want to build? Do 00:01:21.520 --> 00:01:24.637 you want to build a product? Do you want to solve a problem? Do you want to do a 00:01:24.720 --> 00:01:28.317 data vis visualization? So the prompt box is really open for you. You can put 00:01:28.400 --> 00:01:32.397 in anything there. So let's say you want to, you know, build a startup. You have 00:01:32.480 --> 00:01:34.557 an idea for a startup. I would I would 00:01:34.640 --> 00:01:37.517 start with like a paragraph long kind of 00:01:37.600 --> 00:01:42.477 description of what I want to build. Uh the agents will read that. It will 00:01:42.560 --> 00:01:45.517 >> you just type just type >> standard English. Standard English. You 00:01:45.600 --> 00:01:49.677 just type it in. I want to build a I want to sell I want to sell crepes. I 00:01:49.760 --> 00:01:53.357 want to sell crepes online. So you just like type in I want to talk. 00:01:53.440 --> 00:01:56.637 >> You can it literally could be that four words or five words. Okay. 00:01:56.720 --> 00:02:00.797 >> Or it could be if you're if you have a programming language you prefer or stack 00:02:00.880 --> 00:02:04.317 you prefer, you could do that. But we actually prefer not for you not to do 00:02:04.400 --> 00:02:07.917 that because we're going to pick the best thing for we're going to classify 00:02:08.000 --> 00:02:10.797 the best stack for that request. Right? 00:02:10.880 --> 00:02:14.397 >> It's a if it's a data app, we'll pick Python and stream whatever. If it's like 00:02:14.480 --> 00:02:18.237 a web app, we'll pick JavaScript and Postgress and things like that. So you 00:02:18.320 --> 00:02:19.997 just type that >> or you can decide you can decide you can 00:02:20.080 --> 00:02:23.037 say and I want to do it I know Python or I'm learning Python in school and I want 00:02:23.120 --> 00:02:25.437 to do it in Python. >> That's right. The cool thing about 00:02:25.520 --> 00:02:29.117 Replet is you know we we've been around for almost 10 years now and we built all 00:02:29.200 --> 00:02:32.637 this infrastructure. Replet runs any programming language. So if you're 00:02:32.720 --> 00:02:35.277 comfortable with Python you can go in and do that for sure. 00:02:35.360 --> 00:02:37.437 >> Okay. >> And then just again I know this is 00:02:37.520 --> 00:02:39.997 obvious people have used it but like I'm dealing in English. 00:02:40.080 --> 00:02:45.197 >> Yes. >> So okay go ahead. >> Yes. You're fully in English. I mean, 00:02:45.280 --> 00:02:46.877 you know, just a, you know, a little bit 00:02:46.960 --> 00:02:49.757 of of sort of background here, like when 00:02:49.840 --> 00:02:54.477 um when I I came here and pitched to you like 10 years ago or like whatever 7 00:02:54.560 --> 00:02:57.917 years ago, >> right? >> Uh what we were saying is we were 00:02:58.000 --> 00:03:01.757 exactly describing this future is that >> uh everyone would want to build 00:03:01.840 --> 00:03:04.637 software, right? >> And the thing that's kind of getting in 00:03:04.720 --> 00:03:09.677 in people's ways is all the uh what Fred Brooks called the accidental complexity 00:03:09.760 --> 00:03:12.957 of programming, right? They're like essential complexity which is like how 00:03:13.040 --> 00:03:16.557 do I bring my startup to market and how 00:03:16.640 --> 00:03:20.477 do I build a business and all of that accidental complexity is what package 00:03:20.560 --> 00:03:22.477 manager do I use all of that stuff we've 00:03:22.560 --> 00:03:24.877 been abstracting away that for so many 00:03:24.960 --> 00:03:30.077 years so you can just um and the last thing we had to abstract away is code 00:03:30.160 --> 00:03:32.637 >> right >> I had this realization last year which 00:03:32.720 --> 00:03:36.397 is I think we you know built an amazing platform but the business is not 00:03:36.480 --> 00:03:39.117 performing and the reason the business is not performing is that code is the 00:03:39.200 --> 00:03:41.517 bottleneck like yes all the other stuff 00:03:41.600 --> 00:03:46.157 is important to solve but syntax is still an issue like you know syntax is 00:03:46.240 --> 00:03:50.557 just an unnatural thing for people so ultimately English is the programming 00:03:50.640 --> 00:03:54.957 language >> right >> I I just does it work with other other 00:03:55.040 --> 00:03:56.717 world languages other than English at this point 00:03:56.800 --> 00:04:00.637 >> yes you can write in Japanese and we have a lot of users especially Japanese 00:04:00.720 --> 00:04:02.957 that tends to be very >> so does it support these days like for a 00:04:03.040 --> 00:04:06.077 does a support every language or is it still do you still have to do like 00:04:06.160 --> 00:04:09.677 custom work to craft a new new language >> no most most you know uh mainstream 00:04:09.760 --> 00:04:13.357 dream language that has like 100 million plus people that speak it. AI is pretty 00:04:13.440 --> 00:04:15.197 good at it. >> Okay. Yeah. >> Yeah. Wow. 00:04:15.280 --> 00:04:19.917 >> So, uh I I I did a bit of a bit of historical research recently for for 00:04:20.000 --> 00:04:22.877 some reason. I just want to just understand the moment we're in and 00:04:22.960 --> 00:04:24.637 because it's such a special moment. It's 00:04:24.720 --> 00:04:26.797 I think it's important to contextualize 00:04:26.880 --> 00:04:30.557 it and I I I read this quote from 00:04:30.640 --> 00:04:32.957 Gracehopper. So, Gracehopper invented 00:04:33.040 --> 00:04:36.077 the compiler as you know. uh at the time 00:04:36.160 --> 00:04:40.877 people were uh you know programming in machine code and that's what programmers 00:04:40.960 --> 00:04:45.437 do that's what the specialists do >> yes >> and she said you know specialists will 00:04:45.520 --> 00:04:48.557 always be a specialist they have to learn the underlying machinery of 00:04:48.640 --> 00:04:52.397 computers but I want to get to a world where people are programming English 00:04:52.480 --> 00:04:56.317 that's what she said that's before karpathy right that's like you know 75 00:04:56.400 --> 00:05:00.397 years ago >> uh and and and that's why I invented the 00:05:00.480 --> 00:05:04.477 compiler and in her mind like C programming is English >> right 00:05:04.560 --> 00:05:08.797 >> uh But that, you know, that really didn't uh that was just the start of it. 00:05:08.880 --> 00:05:13.277 You had C and then you go higher level Python and JavaScript. And I think it 00:05:13.360 --> 00:05:15.997 we're at a moment where it's the next step, >> right? 00:05:16.080 --> 00:05:19.597 >> Instead of typing syntax, you're actually typing thoughts, you know, 00:05:19.680 --> 00:05:21.037 which is what we ultimately want. 00:05:21.120 --> 00:05:22.957 >> And the machine writes the code >> and the machine writes the code, 00:05:23.040 --> 00:05:25.197 >> right? Right. >> Um yeah, I remember it. you're you're 00:05:25.280 --> 00:05:28.237 probably not old enough uh to remember but I I remember when when I was a kid 00:05:28.320 --> 00:05:31.517 it was um you know there there were were higher level languages you know by the 00:05:31.600 --> 00:05:33.757 70s like like basic and so forth and forran 00:05:33.840 --> 00:05:37.517 >> and C and C but um uh there were still you know you still would run into people 00:05:37.600 --> 00:05:40.797 who were doing assembly programming assembly language which by the way you 00:05:40.880 --> 00:05:43.837 still do you know like game companies or whatever still do assembly to to to get 00:05:43.920 --> 00:05:46.877 >> and they were hating on the kids that were doing basic. Oh well so so the 00:05:46.960 --> 00:05:48.957 assembly people were hating the kids doing basic but there were also older 00:05:49.040 --> 00:05:51.597 coders who hated on the assembly programmers for doing assembly and not 00:05:51.680 --> 00:05:55.357 and not and not no no no not doing direct machine code right not doing 00:05:55.440 --> 00:05:58.637 direct zero in one machine code because because as because assembly assembly so 00:05:58.720 --> 00:06:01.517 people don't know assembly language is sort of this very low-level programming 00:06:01.600 --> 00:06:05.357 language that sort of compiles to actual actual machine code and if and if it's 00:06:05.440 --> 00:06:08.237 it's it's incomprehensible gibberish to most program even most programmers 00:06:08.320 --> 00:06:10.477 >> you're writing an octal or something >> you're writing like very very close to 00:06:10.560 --> 00:06:14.317 the hardware but even still it's still a language that compiles to zeros and ones 00:06:14.400 --> 00:06:18.077 >> um whereas the actual real programmers actually wrote in zeros and ones. And so 00:06:18.160 --> 00:06:20.797 there there's always there's always this tendency, you know, for the for the for 00:06:20.880 --> 00:06:22.637 the pros to be, you know, look on the nose. 00:06:22.720 --> 00:06:24.317 >> Yeah. >> And say, you know, the new people are 00:06:24.400 --> 00:06:26.637 being are being, you know, basically sloppy. They don't understand what's 00:06:26.720 --> 00:06:28.957 happening. You know, they don't really understand the machine. And then, of 00:06:29.040 --> 00:06:31.597 course, you know, with with the higher level with the higher level abstractions 00:06:31.680 --> 00:06:35.517 do is they make they democratize. The absolute irony is I was part of the 00:06:35.600 --> 00:06:40.397 JavaScript revolution. I was at Facebook uh before starting repled and we built 00:06:40.480 --> 00:06:44.557 the modern JavaScript stack. We built ReactJS and all the tooling around it 00:06:44.640 --> 00:06:48.397 >> and we got a lot of hate from from the programmers that you should type you 00:06:48.480 --> 00:06:52.957 know vanilla JavaScript directly and >> um I was like okay whatever and then 00:06:53.040 --> 00:06:56.797 that you and now that's mainstream and then those guys that built their careers 00:06:56.880 --> 00:07:01.597 on the last wave we invented are hating on this new wave and so just you know 00:07:01.680 --> 00:07:04.957 people never change. Okay, got it. Okay, so you you're typing English I want to 00:07:05.040 --> 00:07:07.437 sell crepes online. I want to do this. I want to have a t-shirt. Whatever the 00:07:07.520 --> 00:07:09.037 business is. Okay. What what happens then? 00:07:09.120 --> 00:07:12.637 >> Yeah. and then uh uh replet agent will 00:07:12.720 --> 00:07:17.357 show you what it understood. So it's trying to build um a common 00:07:17.440 --> 00:07:20.877 understanding between you and it and I think there's a lot of things we can do 00:07:20.960 --> 00:07:24.557 better there in terms of UI but for now it'll show you a list of tasks. 00:07:24.640 --> 00:07:28.237 >> It'll tell you I'm going to go set up a database because you need to store your 00:07:28.320 --> 00:07:32.317 data somewhere. Uh we need to set up Shopify or Stripe because we need to 00:07:32.400 --> 00:07:36.637 accept payments. Uh and then it shows you this list and gives you two options 00:07:36.720 --> 00:07:40.077 initially. Do you want to start with a design so that we can iterate back and 00:07:40.160 --> 00:07:43.757 forth to get lock that design down or do you want to build a full thing? 00:07:43.840 --> 00:07:47.917 >> Hey, if you want to build a full thing, we'll go for 20, 30, 40 minutes. 00:07:48.000 --> 00:07:52.797 >> Uh, and the a and be like the agent will tell you go here, install the app. 00:07:52.880 --> 00:07:54.637 >> Uh, I'm going to go set up the database, 00:07:54.720 --> 00:07:58.637 do the migrations, write the SQL, you know, build the site. I'm going to also 00:07:58.720 --> 00:08:00.717 test it. So this is a recent innovation 00:08:00.800 --> 00:08:03.997 we did with um agent 3 is that after it 00:08:04.080 --> 00:08:06.397 writes the software spins up a browser 00:08:06.480 --> 00:08:11.117 goes around and tests in the browser and then any issue it kind of iterates kind 00:08:11.200 --> 00:08:14.397 of goes and fix the code. So it'll spend 20 30 minutes building that I'll send 00:08:14.480 --> 00:08:17.117 you a notification it'll tell you the app is ready. And so you can test it on 00:08:17.200 --> 00:08:21.757 your phone. You can go back to your computer. You'll see maybe you'll find a 00:08:21.840 --> 00:08:24.957 bug or an issue, you'll describe it to the agent, say, "Hey, it's not exactly 00:08:25.040 --> 00:08:29.437 doing what I expected." Uh or if it's perfect and and you're ready to go and 00:08:29.520 --> 00:08:32.317 that's it. You know, 20 minutes. By the way, there's a lot of examples where 00:08:32.400 --> 00:08:36.957 people just get their idea in 20 30 minutes, which is amazing. Um you just 00:08:37.040 --> 00:08:38.077 hit publish. >> Mhm. 00:08:38.160 --> 00:08:41.997 >> You hit you hit publish. Um 00:08:42.080 --> 00:08:44.237 couple clicks, you'll be up in the 00:08:44.320 --> 00:08:49.357 cloud. we'll set up a a virtual machine in the cloud. The database is deployed. 00:08:49.440 --> 00:08:51.677 Everything's done and now you have a production database. 00:08:51.760 --> 00:08:56.237 >> So, think about the steps needed just two or 3 years ago in order to get to 00:08:56.320 --> 00:08:59.277 that step. You have to set up your local development environment. You have to 00:08:59.360 --> 00:09:02.637 sign up for an AWS account. You have to provision the databases, the virtual 00:09:02.720 --> 00:09:06.957 machines, you have to create the entire pip deployment pipeline. All of that is 00:09:07.040 --> 00:09:09.037 done for you. And it just, you know, a 00:09:09.120 --> 00:09:11.277 kid can do it, a lay person can do it. 00:09:11.360 --> 00:09:16.237 If you're a programmer and uh you're curious about what the agent did, the 00:09:16.320 --> 00:09:19.517 cool thing about replet because we have this history of being an IDE, you can 00:09:19.600 --> 00:09:23.677 peel the layers. You can open the file tree and you could look at the files. 00:09:23.760 --> 00:09:26.557 You can open gits, you can push it to GitHub, you can connect it to your 00:09:26.640 --> 00:09:30.077 editor if you want, you can open it in Emacs. So the cool thing about Replet, 00:09:30.160 --> 00:09:34.157 yes, it is a vibe coding platform that abstracts away all the complexities, but 00:09:34.240 --> 00:09:36.077 all the layers are there for you to look at. 00:09:36.160 --> 00:09:40.237 >> Right. So let's go let's go back to um that was great, but let's go back to you 00:09:40.320 --> 00:09:43.597 said it. It it gives you that the a the agent gives you you you say I've got my 00:09:43.680 --> 00:09:46.317 idea. You plug it in and it says it gives you this list of things and then 00:09:46.400 --> 00:09:48.477 you and then when you describe it you said I'm going to do this. I'm going to 00:09:48.560 --> 00:09:52.477 do that. The eye there in that case was the agent as opposed to the user. Yes. 00:09:52.560 --> 00:09:55.517 >> And so the the agent lists the set of things that it's going to do and then 00:09:55.600 --> 00:09:57.197 the agent actually does those things. 00:09:57.280 --> 00:10:00.237 >> Agent does those things. Yeah. That that's a that's a that's a very 00:10:00.320 --> 00:10:03.357 important point. when we did this shift, 00:10:03.440 --> 00:10:07.917 we hadn't realized internally at Replet how much the actual user stopped being 00:10:08.000 --> 00:10:10.797 the human user and it's actually the agent programmer, >> right? 00:10:10.880 --> 00:10:12.957 >> So, one really uh funny thing happened 00:10:13.040 --> 00:10:17.997 is we had servers in Asia uh and we the reason we had servers in Asia because we 00:10:18.080 --> 00:10:19.917 wanted our Indian or you know Japanese 00:10:20.000 --> 00:10:23.357 users to be to have a you know shorter 00:10:23.440 --> 00:10:27.037 uh time to the servers. uh when we 00:10:27.120 --> 00:10:30.637 launched the agent their experience got significantly worse and we're like what 00:10:30.720 --> 00:10:33.757 happened like it's supposed to be faster well turns out it's worse it's because 00:10:33.840 --> 00:10:38.237 the AIS are sitting in uh in United States and so the the programmer is 00:10:38.320 --> 00:10:41.357 actually in United States it's you're sending the request to the programmer 00:10:41.440 --> 00:10:45.277 and the programmer is interfacing with a machine across the world and so yes 00:10:45.360 --> 00:10:49.997 suddenly the agent is the programmer okay so like the the ter ter you know 00:10:50.080 --> 00:10:53.917 new terminology agent is a software program that is basically using the rest 00:10:54.000 --> 00:10:57.597 of the as if it were a as if it were a human 00:10:57.680 --> 00:10:59.277 user, but it's not. It's a it's a bot. 00:10:59.360 --> 00:11:00.957 >> That's right. It has access to tools 00:11:01.040 --> 00:11:03.357 such as write a file, edit a file, 00:11:03.440 --> 00:11:06.877 delete a file, uh uh search the package 00:11:06.960 --> 00:11:11.677 index, install a package, uh provision a database, provision object object 00:11:11.760 --> 00:11:15.437 storage. It is a programmer that has the tools and interface. It has a sort of an 00:11:15.520 --> 00:11:18.317 interface >> that that is very similar to a human 00:11:18.400 --> 00:11:22.317 programmer. And then um you know we'll talk more about how this all works but a 00:11:22.400 --> 00:11:27.037 debate inside the AI industry um is with these was kind of this you know this 00:11:27.120 --> 00:11:29.917 idea now of having agents that do things on your behalf and then go out you know 00:11:30.000 --> 00:11:32.717 go go out and kind of accomplish missions. Um there's this you know kind 00:11:32.800 --> 00:11:36.237 of debate which is okay how like obviously you know it's a big deal even 00:11:36.320 --> 00:11:38.717 to have an AI agent that can do relatively simple things to do complex 00:11:38.800 --> 00:11:41.357 things of course is you know one of the great technical challenges of the last 00:11:41.440 --> 00:11:44.877 80 years you know to to do that and then there's sort of this question of like 00:11:44.960 --> 00:11:48.157 can the agent go out and run and operate on its own for 5 minutes you know for 00:11:48.240 --> 00:11:53.437 for 15 minutes for an hour for 8 hours and and meaning like sort of like how 00:11:53.520 --> 00:11:56.397 long does it maintain coherence like how long does it actually like stay in full 00:11:56.480 --> 00:12:00.077 control of it of its faculties and not kind of spin out because at least the 00:12:00.160 --> 00:12:03.197 early early agents or the the early AIS, if if you set them off to do this, they 00:12:03.280 --> 00:12:05.357 might be able to run for two or three minutes, then they would they would 00:12:05.440 --> 00:12:07.997 start to get confused and go down rabbit holes and, you know, kind of kind of 00:12:08.080 --> 00:12:11.997 spin out. Um, more recently, more recently, um, uh, you know, we've seen 00:12:12.080 --> 00:12:15.197 that that that that agents can run a lot longer and and do more complex tasks. 00:12:15.280 --> 00:12:19.597 Like, where are we on the curve of agents being able to run for how long 00:12:19.680 --> 00:12:22.477 and for what complexity tasks before before they break? 00:12:22.560 --> 00:12:24.557 >> That's that's absolutely the the I think 00:12:24.640 --> 00:12:29.197 the main metric we're looking at. even back in 2023, you know, had the idea for 00:12:29.280 --> 00:12:33.117 software agents, you know, four or five years ago now. The problem every time we 00:12:33.200 --> 00:12:37.117 attempt them, the the problem of coherence, you know, they'll they'll go 00:12:37.200 --> 00:12:41.357 on for a minute or two and then they'll just, you know, they compound in errors 00:12:41.440 --> 00:12:43.837 in a way that they just can't recover. 00:12:43.920 --> 00:12:45.037 >> Um, >> and you can actually see it, right? 00:12:45.120 --> 00:12:47.757 Because they actually they actually, if you watch watch them operate, they get 00:12:47.840 --> 00:12:51.037 increasingly confused and then, you know, maybe even deranged. Yeah, they 00:12:51.120 --> 00:12:54.237 vary the range and they they go into very weird areas and sometimes they 00:12:54.320 --> 00:12:56.157 start speaking Chinese and doing really 00:12:56.240 --> 00:12:59.437 weird things and um but I would say 00:12:59.520 --> 00:13:03.437 sometime around last year we maybe 00:13:03.520 --> 00:13:05.917 crossed the 3 four five minute mark 00:13:06.000 --> 00:13:09.037 >> and it felt to us that okay we're on a 00:13:09.120 --> 00:13:12.397 path where long re you know long horizon 00:13:12.480 --> 00:13:17.197 reasoning is getting solved >> uh and so we made we made a bet and I I 00:13:17.280 --> 00:13:20.157 tell my team >> so long horizon reasoning meaning 00:13:20.240 --> 00:13:23.837 reasoning meaning like dealing in like facts and logic 00:13:23.920 --> 00:13:28.157 >> um in a in a sort of complex way and then long horizon being over a long 00:13:28.240 --> 00:13:31.037 period of time. Yes. >> With many many steps to a reasoning process. 00:13:31.120 --> 00:13:34.157 >> Yeah, that's right. So if you think about the way large language models work 00:13:34.240 --> 00:13:37.757 is that they have a context. This context is basically the memory all the 00:13:37.840 --> 00:13:42.797 text all your prompt and also all the internal talk that the AI is doing as 00:13:42.880 --> 00:13:45.437 it's reasoning. So when the AI is reasoning it's actually talking to 00:13:45.520 --> 00:13:49.357 itself. It's like oh now I need to go set up a database. Well, what what kind 00:13:49.440 --> 00:13:53.277 of tool do I have? Oh, there's a tool here that says Postgress. Okay, let me 00:13:53.360 --> 00:13:57.677 try using that. Okay, I use that. I got feedback. Let me look at the feedback 00:13:57.760 --> 00:13:58.877 and read it. And it'll read the 00:13:58.960 --> 00:14:01.997 feedback. And so the that that prompt 00:14:02.080 --> 00:14:06.877 box or context is where both the user 00:14:06.960 --> 00:14:11.757 input, the environment input, and the internal thoughts of the machine are all 00:14:11.840 --> 00:14:15.837 within. It's sort of like a program memory in in memory space. And so 00:14:15.920 --> 00:14:17.917 reasoning over that was the challenge 00:14:18.000 --> 00:14:20.797 for a long time. That's when AIs just 00:14:20.880 --> 00:14:23.677 like went off track and now they're able 00:14:23.760 --> 00:14:25.197 to kind of think through this entire 00:14:25.280 --> 00:14:27.917 thing and and maintain coherence. And 00:14:28.000 --> 00:14:30.717 there's there's now techniques around uh 00:14:30.800 --> 00:14:33.437 compression of contacts. So there still 00:14:33.520 --> 00:14:37.917 context length is still a problem, right? So I would say LM today, you 00:14:38.000 --> 00:14:40.077 know, they're marketed as a million uh 00:14:40.160 --> 00:14:42.477 token uh length, which is like a million 00:14:42.560 --> 00:14:44.957 words almost. uh in reality it's about 00:14:45.040 --> 00:14:47.437 200,000 and then they start to struggle. 00:14:47.520 --> 00:14:49.997 So we do a lot of uh you know we stop we 00:14:50.080 --> 00:14:54.957 compress the memory. So if a memory if if a portion of the memory is saying 00:14:55.040 --> 00:14:59.597 that I'm getting all the logs from the database you can summarize you know 00:14:59.680 --> 00:15:03.997 paragraphs of logs with one statement or the database setup that's it right and 00:15:04.080 --> 00:15:08.317 so every once in a while we'll compress the context so that we make sure we 00:15:08.400 --> 00:15:11.997 maintain coherence so that there's a lot of innovation happened outside of the 00:15:12.080 --> 00:15:16.397 foundation models as well in order to to enable that long context coherence. So 00:15:16.480 --> 00:15:19.117 what was the what was the key technical breakthrough at the in the foundation 00:15:19.200 --> 00:15:20.957 models that made this possible do you think? 00:15:21.040 --> 00:15:22.877 >> I think it's RL I think it's uh 00:15:22.960 --> 00:15:24.957 reinforcement learning. So the way 00:15:25.040 --> 00:15:28.477 pre-training works is you know uh they 00:15:28.560 --> 00:15:31.997 uh pre-training is a uh the first step 00:15:32.080 --> 00:15:35.917 of training a large language model. It reads a piece of text. It covers the 00:15:36.000 --> 00:15:37.997 last words and tries to guess it. That's 00:15:38.080 --> 00:15:41.037 how it's trained. That doesn't really 00:15:41.120 --> 00:15:44.317 imply long context reasoning. it it you 00:15:44.400 --> 00:15:47.037 know it it turns out to be very very 00:15:47.120 --> 00:15:48.557 effective. It can learn language that 00:15:48.640 --> 00:15:51.597 way. But the reason we weren't able to 00:15:51.680 --> 00:15:54.957 move past that limitation is that that 00:15:55.040 --> 00:15:57.357 modality of training just wasn't good 00:15:57.440 --> 00:15:59.997 enough. And what you want is you want a 00:16:00.080 --> 00:16:03.597 type of problem solving over a uh over 00:16:03.680 --> 00:16:05.837 long context. So what reinforcement 00:16:05.920 --> 00:16:08.717 learning uh uh especially from code 00:16:08.800 --> 00:16:12.557 execution gave us is the ability to for 00:16:12.640 --> 00:16:16.317 the machine to for the LLM to roll out 00:16:16.400 --> 00:16:18.397 what we call trajectories in AI. So 00:16:18.480 --> 00:16:22.717 trajectory is a uh stepbystep reasoning 00:16:22.800 --> 00:16:27.677 chain in order to reach a solution. So uh the way uh as I understand 00:16:27.760 --> 00:16:29.517 reinforcement learning works is they put 00:16:29.600 --> 00:16:31.917 the LM in a programming environment like 00:16:32.000 --> 00:16:35.757 replet and say hey here's a pro here's a 00:16:35.840 --> 00:16:38.077 codebase here's a bug in the codebase 00:16:38.160 --> 00:16:43.197 and we want you to solve it. Um now the human trainer already knows what the 00:16:43.280 --> 00:16:46.237 solution would look like. So we have a pull request that we have on GitHub so 00:16:46.320 --> 00:16:50.157 we know exactly or we have a unit test that we can run and verify the solution. 00:16:50.240 --> 00:16:53.517 So what it does is it rolls out a lot of different trajectories. Those they 00:16:53.600 --> 00:16:55.917 sample the model and maybe one of those 00:16:56.000 --> 00:16:57.837 trajectories will reach and a lot of 00:16:57.920 --> 00:17:00.877 them will just go go off off track but 00:17:00.960 --> 00:17:05.517 one of them will reach the solution by solving the bug and it reinforces on 00:17:05.600 --> 00:17:09.357 that. So that that gets a reward and the model gets trained that okay you know 00:17:09.440 --> 00:17:12.557 this is how you solve these type of problems. So that's how we're able to 00:17:12.640 --> 00:17:13.917 extend these reasoning chains. 00:17:14.000 --> 00:17:15.917 >> Got it. and and how it's a two-part 00:17:16.000 --> 00:17:18.557 question is how how how good how good 00:17:18.640 --> 00:17:23.757 are the models now at long long long reasoning and and I would say and how do 00:17:23.840 --> 00:17:25.197 we know like how how is that 00:17:25.280 --> 00:17:27.757 established? Um 00:17:27.840 --> 00:17:30.877 there is a nonprofit called meter um 00:17:30.960 --> 00:17:34.077 that is uh measuring 00:17:34.160 --> 00:17:37.837 uh useful has a benchmark to measure uh 00:17:37.920 --> 00:17:43.037 how long a model runs while maintaining coherence and doing useful useful things 00:17:43.120 --> 00:17:46.797 whether it's programming or other benchmark tasks that they've done. uh 00:17:46.880 --> 00:17:49.357 and they put up a paper I think uh late 00:17:49.440 --> 00:17:52.877 last year that said every seven months 00:17:52.960 --> 00:17:57.277 >> uh the the minutes that a model can run 00:17:57.360 --> 00:18:01.517 is doubling. >> So you go from 2 minutes to you know 4 00:18:01.600 --> 00:18:05.197 minutes in 7 months I think they vastly underestimated that. 00:18:05.280 --> 00:18:06.717 >> Is that right? Vastly it's doubling. 00:18:06.800 --> 00:18:08.157 It's doubling more often than 7 months. 00:18:08.240 --> 00:18:10.957 >> We so Asian 3 we measure that you know 00:18:11.040 --> 00:18:16.157 very closely uh and we measure that in real tasks from real users. So we're not 00:18:16.240 --> 00:18:19.357 doing benchmarking. We're actually doing AB tests and we're looking at the data 00:18:19.440 --> 00:18:21.677 that how users are successful or not. 00:18:21.760 --> 00:18:24.477 >> For us, the the absolute sign of success 00:18:24.560 --> 00:18:26.157 is you made an app and you published it. 00:18:26.240 --> 00:18:28.717 Because when you publish it, you're paying extra money. You're saying this 00:18:28.800 --> 00:18:32.637 app is economically useful. I'm going to publish it. So that's as clear-cut as 00:18:32.720 --> 00:18:37.357 possible. And so what we're seeing is in agent one, the agent could run for 2 00:18:37.440 --> 00:18:40.317 minutes >> and then and then perhaps struggle. 00:18:40.400 --> 00:18:45.597 Agent two came out in February, it ran for 20 minutes. Agent 3 200 minutes. 00:18:45.680 --> 00:18:48.797 >> Okay, >> 200. Some users are pushing it to like 00:18:48.880 --> 00:18:50.797 12 hours and things like that. I'm less 00:18:50.880 --> 00:18:53.437 confident that it is as good and when it 00:18:53.520 --> 00:18:56.317 goes to these stratospheres, but at like 00:18:56.400 --> 00:18:59.117 2 3 hours timeline, it is really it's 00:18:59.200 --> 00:19:01.997 it's it's it's insanely good. And and 00:19:02.080 --> 00:19:03.677 the main innovation outside of the 00:19:03.760 --> 00:19:07.117 models is a verification loop. Actually, 00:19:07.200 --> 00:19:09.997 uh I remember reading um a research 00:19:10.080 --> 00:19:12.797 paper from Nvidia. So what Nvidia did is 00:19:12.880 --> 00:19:15.597 they're trying to uh write um GPU 00:19:15.680 --> 00:19:18.877 kernels uh using deepseek and that was 00:19:18.960 --> 00:19:23.597 like perhaps 7 months ago when deepseek came out and what they found is that if 00:19:23.680 --> 00:19:27.837 we add a verifier in the loop if we can run the kernel and verify it's working 00:19:27.920 --> 00:19:31.677 we're able to run deepseeek for like 20 minutes and it it was generating 00:19:31.760 --> 00:19:34.077 actually optimized kernels 00:19:34.160 --> 00:19:36.397 >> and so I was like okay the next thing 00:19:36.480 --> 00:19:39.677 for us obviously as an as a sort of a 00:19:39.760 --> 00:19:43.037 agent lab or like applay our company. 00:19:43.120 --> 00:19:46.157 We're not doing the foundation model stuff, but we're doing a lot of research 00:19:46.240 --> 00:19:50.397 on top of that. And so, okay, we know that agents can run for 10 20 minutes 00:19:50.480 --> 00:19:52.637 now or LLMs can stay coherent for 00:19:52.720 --> 00:19:55.997 longer, but for you to push them to 200, 00:19:56.080 --> 00:19:58.157 300 minutes, you need a verifier in the 00:19:58.240 --> 00:19:59.917 loop. So, that's why we spend all our 00:20:00.000 --> 00:20:03.757 time uh creating scaffolds to make it so 00:20:03.840 --> 00:20:08.797 that the agent can spin up a browser and do computer use style testing. So once 00:20:08.880 --> 00:20:10.397 you put that in the middle, what's 00:20:10.480 --> 00:20:12.877 happening is it works for 20 minutes uh 00:20:12.960 --> 00:20:16.157 spin up another agent uh spins up a a 00:20:16.240 --> 00:20:18.077 browser tests the work of the previous 00:20:18.160 --> 00:20:20.157 agent. So it's a multi- aent system 00:20:20.240 --> 00:20:24.557 >> and if it is uh if it founds a bug it 00:20:24.640 --> 00:20:29.037 starts a new trajectory and says okay good work let's summarize what you did 00:20:29.120 --> 00:20:32.717 the last 20 minutes >> now that be that plus what the bug that 00:20:32.800 --> 00:20:34.957 we found that's a prompt for a new trajectory right 00:20:35.040 --> 00:20:37.837 >> so you stack those on each other and you can go endlessly but 00:20:37.920 --> 00:20:41.037 >> so it's like a mar like setting up a marathon or like a relay race 00:20:41.120 --> 00:20:43.837 >> as long as as long as each step is done properly you could do in sort of an 00:20:43.920 --> 00:20:46.077 infinite number of steps >> that's right that's right you can always 00:20:46.160 --> 00:20:49.197 compress the previous step into a paragraph And that becomes a prompt. So 00:20:49.280 --> 00:20:51.277 it's it's an agent prompting the next agent. 00:20:51.360 --> 00:20:54.237 >> Right. Right. Right. That's amazing. So um and then when when an agent like when 00:20:54.320 --> 00:20:57.677 a modern agent like running on modern modern LM that are trained this way when 00:20:57.760 --> 00:21:01.117 it let's say it runs for 200 minutes like when you watch the agent run is it 00:21:01.200 --> 00:21:05.037 like running is it like processing through like logic and tasks at the same 00:21:05.120 --> 00:21:07.197 pace that like a human being is or 00:21:07.280 --> 00:21:08.797 slower or faster? I 00:21:08.880 --> 00:21:12.637 >> it's actually I would say it is faster 00:21:12.720 --> 00:21:14.957 but not that much significantly faster. 00:21:15.040 --> 00:21:18.397 It's not at computer speed, right? What we expect computer speed to be. 00:21:18.480 --> 00:21:20.877 >> It's like watching a per like if you watch if you if it's describing what 00:21:20.960 --> 00:21:23.037 it's doing, it's sort of like watching a person work. 00:21:23.120 --> 00:21:26.877 >> It's like watching John Carmarmacine work. 00:21:26.960 --> 00:21:31.277 >> The world Okay. The world's the world's best programmer. 00:21:31.360 --> 00:21:33.597 >> Yeah. >> The world's best programmer on a stim on a stimulant. 00:21:33.680 --> 00:21:35.597 >> On a stimulant. Yeah, that's right. 00:21:35.680 --> 00:21:37.037 >> Working for you. Working for you. 00:21:37.120 --> 00:21:41.277 >> Yeah. There. So, it's very fast and you can see the uh file diffs running 00:21:41.360 --> 00:21:44.557 through, but every once in a while it'll stop and it'll start thinking. I'll show 00:21:44.640 --> 00:21:47.757 you the reasoning. Yeah. It's like, I did this and I did this. Am I on the 00:21:47.840 --> 00:21:50.317 right track? It kind of really tries to reflect, right? 00:21:50.400 --> 00:21:54.557 >> Uh and then it might review its work and decide the next step or it might kick 00:21:54.640 --> 00:21:59.677 into the testing agent or you know, so so you're seeing it do all of that and 00:21:59.760 --> 00:22:03.117 every once in a while it calls the tool for example, it stops and says, well, we 00:22:03.200 --> 00:22:05.597 ran into an issue. you know, Postgress 00:22:05.680 --> 00:22:11.437 um 15 is not um compatible with this, 00:22:11.520 --> 00:22:14.717 you know, database ORM package that that I have. 00:22:14.800 --> 00:22:17.837 >> Um okay, this is a problem I haven't seen before. I'm going to go search the 00:22:17.920 --> 00:22:21.997 web. So, it has a web search tool. Go do that. And so, it looks like a human 00:22:22.080 --> 00:22:24.477 programmer, right? >> And it's really fascinating to watch. 00:22:24.560 --> 00:22:28.237 It's one of my favorite things to do is just to watch the tool chain and 00:22:28.320 --> 00:22:30.477 reasoning chain and the testing chain. 00:22:30.560 --> 00:22:34.477 And it's yeah it is like watching a hyperproductive programmer 00:22:34.560 --> 00:22:37.437 >> right so you know we're kind of getting into here kind of the holy grail of AI 00:22:37.520 --> 00:22:41.757 which is sort of you know generalized reasoning um you know by the machine um 00:22:41.840 --> 00:22:46.477 so uh you mentioned this a couple times but this idea of a of a verification so 00:22:46.560 --> 00:22:49.997 so just for folks on the listening to podcast who maybe aren't in the details 00:22:50.080 --> 00:22:53.357 let me try to describe this and see see if I have it right so like a just a just 00:22:53.440 --> 00:22:55.997 a large language model the way you would exper you would have experienced with 00:22:56.080 --> 00:22:59.357 like Chad GPT out of the gate two years ago or whatever would have been it's 00:22:59.440 --> 00:23:03.757 like And it's incredible how fluid uh it is at language. Um it's incredible how 00:23:03.840 --> 00:23:06.877 good it is at like writing Shakespearean sonnetss or rap lyrics. It's it's 00:23:06.960 --> 00:23:09.917 amazing how good it is at human conversation. But if you start to ask it 00:23:10.000 --> 00:23:14.157 like problems that involve like rational thinking uh or problem solving all of a 00:23:14.240 --> 00:23:17.677 sudden like you math or the math the whole show and and in the very beginning 00:23:17.760 --> 00:23:20.557 it was you could ask if you ask it very basic math problems, you know, it would 00:23:20.640 --> 00:23:21.757 it would not be able to do them. 00:23:21.840 --> 00:23:24.397 >> That's right. Uh but then even when it got better at those, if you started to 00:23:24.480 --> 00:23:27.117 ask it to like, you know, it it could maybe add two small numbers together, 00:23:27.200 --> 00:23:29.837 but it couldn't add two large numbers together. Or if it could add two large 00:23:29.920 --> 00:23:31.197 numbers, it couldn't multiply them. 00:23:31.280 --> 00:23:32.877 Yeah. >> And it's just like, all right, this is 00:23:32.960 --> 00:23:35.517 And then it had this there was this famous the famous was the straw the 00:23:35.600 --> 00:23:37.997 strawberry test, the famous strawberry test, which is how many Rs are in the 00:23:38.080 --> 00:23:39.037 word strawberry. >> That's right. 00:23:39.120 --> 00:23:41.837 >> And there was this long period where it it kept it would it would just guess 00:23:41.920 --> 00:23:44.797 wrong. It would say there were only two Rs in the word strawberry. And it turns 00:23:44.880 --> 00:23:49.757 out there are three. Um, so, um, so it it was this thing and so people were and 00:23:49.840 --> 00:23:52.637 there was even this term that was being used kind of the the slur that was being 00:23:52.720 --> 00:23:54.237 used at the time was stoastic parrot. 00:23:54.320 --> 00:23:55.837 >> Yeah, >> I was thinking clanker. 00:23:55.920 --> 00:23:59.997 >> Well, well, clanker is the is the new slur. Clank clanker. Clanker is just the 00:24:00.080 --> 00:24:02.477 full-on racial slur against AI as a 00:24:02.560 --> 00:24:07.437 species. Um, but the technical critique was so-called stoastic par stoastic 00:24:07.520 --> 00:24:10.637 means random. Uh so sort of random parrot me meaning basically that this 00:24:10.720 --> 00:24:13.357 thing was sort of the large language models were like a they were like a 00:24:13.440 --> 00:24:15.997 mirage where they were like repeating back to you things that they thought 00:24:16.080 --> 00:24:19.597 that you wanted to hear but they didn't >> in a way it's true in the in the pure 00:24:19.680 --> 00:24:22.157 pre-training LLM world >> right for the for the very basic layer 00:24:22.240 --> 00:24:24.877 but then what happened is as you said over the last year or something there 00:24:24.960 --> 00:24:28.237 there was this layering in of of reinforcement learning and then but the 00:24:28.320 --> 00:24:32.157 key to >> it's not new crucially it's like it's alpha go right so 00:24:32.240 --> 00:24:33.597 >> describe so describe that for a second. 00:24:33.680 --> 00:24:35.197 Yeah. So we we had this breakthrough 00:24:35.280 --> 00:24:39.597 before in uh 2015 was the Alph Go 00:24:39.680 --> 00:24:42.477 breakthrough. I think 2015 2016 where it 00:24:42.560 --> 00:24:47.357 is emerging of sort of uh you know the the the you would know a lot better than 00:24:47.440 --> 00:24:51.997 me the old AI debate between the connectionists uh the the people who who 00:24:52.080 --> 00:24:53.757 thinks neuronet networks are the true 00:24:53.840 --> 00:24:56.957 sort of way of doing AI and the symbolic 00:24:57.040 --> 00:25:01.837 systems I think or like the people that think that you know discrete reasonings 00:25:01.920 --> 00:25:05.517 fates and knowledge bases whatever this is the way to go and so there was there 00:25:05.600 --> 00:25:09.677 was a merging of these two worlds where the way AlphaG go worked is it had a 00:25:09.760 --> 00:25:13.997 neural network but it had a Monte Carlo research algorithm on top of that. So 00:25:14.080 --> 00:25:18.317 the neural network would generate uh would would like uh generate a list of 00:25:18.400 --> 00:25:20.957 potential moves uh and then you had a 00:25:21.040 --> 00:25:23.597 more discrete algorithm sort those moves 00:25:23.680 --> 00:25:26.797 and find the best based on just uh tree 00:25:26.880 --> 00:25:29.357 search based on just trying to verify 00:25:29.440 --> 00:25:30.877 again this sort of a verifier in the 00:25:30.960 --> 00:25:34.557 loop trying to verify which move might 00:25:34.640 --> 00:25:38.077 yield the best based on more classical 00:25:38.160 --> 00:25:40.877 way of doing algorithms. Um, and so that 00:25:40.960 --> 00:25:43.437 that's a resurgence of of that movement 00:25:43.520 --> 00:25:46.637 where we have this amazing generative uh 00:25:46.720 --> 00:25:51.997 neural network that is the the LLM and now let's layer on more discrete ways of 00:25:52.080 --> 00:25:55.197 trying to verify whether it's doing the right thing or not and let's put that in 00:25:55.280 --> 00:25:57.117 a training loop and once you do that the 00:25:57.200 --> 00:26:02.237 LLM will start gaining new capabilities such as uh reasoning over math and code 00:26:02.320 --> 00:26:04.317 and things like that. >> Exactly. Right. Okay. And then that's 00:26:04.400 --> 00:26:07.677 great. And then and then the the key thing there though for for RL to work 00:26:07.760 --> 00:26:12.717 for LMS to reason the key is that it be a a problem statement that there is a 00:26:12.800 --> 00:26:16.477 defined and verifiable answer. That's right. Is that right? And so and and and 00:26:16.560 --> 00:26:19.117 you might think about this as like let's give a bunch of examples like in 00:26:19.200 --> 00:26:22.637 medicine this might be like um you know a diagnosis that like a panel of human 00:26:22.720 --> 00:26:26.077 doctors agrees with um or or or by the way or a diagnosis that actually you 00:26:26.160 --> 00:26:31.037 know solves the condition. Um in law this would be a um you know a a argument 00:26:31.120 --> 00:26:34.957 that in front of a jury actually results in an acquitt or or something like that. 00:26:35.040 --> 00:26:39.037 Um in u math it's an equation that actually solves properly. Uh in physics 00:26:39.120 --> 00:26:41.277 it's a result that actually works in the real world. 00:26:41.360 --> 00:26:44.077 >> I don't know in civil engineering it's a bridge that doesn't collapse. Right. So 00:26:44.160 --> 00:26:45.837 so so there there there's always some 00:26:45.920 --> 00:26:49.117 some test is that the first two do not 00:26:49.200 --> 00:26:51.597 work very well just yet. like the the 00:26:51.680 --> 00:26:55.997 like I would say uh law and healthcare 00:26:56.080 --> 00:27:01.277 they're still a little too squishy a little too soft it's unlike math or code 00:27:01.360 --> 00:27:04.397 like the way that they're training on math they're using this uh sort of like 00:27:04.480 --> 00:27:08.637 a program language uh provable language called lean for proofs right so you can 00:27:08.720 --> 00:27:13.357 run a lean statement you can run a computer code uh perhaps you can run a 00:27:13.440 --> 00:27:17.677 physics simulation or civil engineering uh sort of physics simulation but you 00:27:17.760 --> 00:27:21.437 can't run a diagnosis >> okay So I would say the 00:27:21.520 --> 00:27:23.757 >> but you could verify it with human answers or or not. 00:27:23.840 --> 00:27:26.557 >> Yeah. So that that's a more RL HF in a 00:27:26.640 --> 00:27:29.277 way. So it is not the like sort of 00:27:29.360 --> 00:27:32.157 autonomous RL train like fully scalable 00:27:32.240 --> 00:27:35.357 autonomous which is why coding is moving 00:27:35.440 --> 00:27:37.357 faster than any other domain is because 00:27:37.440 --> 00:27:40.877 we can we we can generate these problems 00:27:40.960 --> 00:27:44.397 and verify them on the fly. But there's two but with coding as anybody who's 00:27:44.480 --> 00:27:46.717 coded knows there's coding there's two tests which is one is does the code 00:27:46.800 --> 00:27:48.717 compile >> right >> and then the other is does it produce 00:27:48.800 --> 00:27:51.437 the right output and just because it compiles doesn't mean it produces the 00:27:51.520 --> 00:27:54.237 right output and I you tell me but verifying that it's the correct output 00:27:54.320 --> 00:27:55.197 is harder 00:27:55.280 --> 00:27:56.957 >> yeah sobbench 00:27:57.040 --> 00:28:02.877 is a collection of uh verified pull 00:28:02.960 --> 00:28:06.157 request end state uh so so it is it is 00:28:06.240 --> 00:28:09.197 not just about compiling we so they they 00:28:09.280 --> 00:28:11.437 group of scientists sobbench is the main 00:28:11.520 --> 00:28:14.877 benchmark used to test whether AI is 00:28:14.960 --> 00:28:16.877 good at software engineering tasks and 00:28:16.960 --> 00:28:19.197 we're almost saturating that. So last 00:28:19.280 --> 00:28:23.277 year we're at like maybe 5% early 24 or 00:28:23.360 --> 00:28:25.997 less and now we're like 82% or something 00:28:26.080 --> 00:28:30.477 like that with cloth on at 4.5 that's state-of-the-art and that's like a 00:28:30.560 --> 00:28:34.557 really nice health climb that's happening right now. uh and basically 00:28:34.640 --> 00:28:38.557 they went and looked on GitHub. They found the the you know most complex 00:28:38.640 --> 00:28:41.197 repositories. They found bug statements 00:28:41.280 --> 00:28:45.917 that are very clear uh and they found ProQuest that actually solve those bug 00:28:46.000 --> 00:28:49.757 statements with unit tests and everything. So there is an existing 00:28:49.840 --> 00:28:54.557 corpus on GitHub of tasks that that the AIS can can solve and you can also 00:28:54.640 --> 00:28:56.557 generate them. Those are not too hard to 00:28:56.640 --> 00:29:00.317 to generate uh you know what's called 00:29:00.400 --> 00:29:04.157 synthetic uh data. Uh uh but but you're 00:29:04.240 --> 00:29:09.437 right it's not infinitely scalable um because you you some human verifiers 00:29:09.520 --> 00:29:12.557 still need to kind of look at the at the task but maybe the foundation models 00:29:12.640 --> 00:29:15.677 have found a way to have the synthetic training go all the way 00:29:15.760 --> 00:29:17.997 >> right and then what's happening I think I think because what's happening is the 00:29:18.080 --> 00:29:20.877 foundation model companies are in some cases they are hire they're actually 00:29:20.960 --> 00:29:23.517 hiring human experts to generate new training data. Yes. 00:29:23.600 --> 00:29:25.997 >> So they're actually hiring mathematicians and physicists and coders 00:29:26.080 --> 00:29:29.917 to basically sit and you know they're they're hiring they're they're hiring 00:29:30.000 --> 00:29:31.917 human programmers putting them on the cocaine. Yes. 00:29:32.000 --> 00:29:37.437 >> Um and having them probably coffee um uh and having them actually write code and 00:29:37.520 --> 00:29:39.997 then and then write code in a way where there's a known result of the code 00:29:40.080 --> 00:29:43.277 running such that the this RO loop can be trained properly. That's right. And 00:29:43.360 --> 00:29:46.077 then the other the other and then the other thing these companies are doing is 00:29:46.160 --> 00:29:49.677 as you said they're building systems where the software itself generates the 00:29:49.760 --> 00:29:53.277 training data, generates the tests, generates the valid the validated 00:29:53.360 --> 00:29:55.677 results and and that's soal synthetic training data. 00:29:55.760 --> 00:30:00.317 >> That's right. And but yeah, but but again those work in the very hard 00:30:00.400 --> 00:30:03.677 domains. It works to some extent in the software domains 00:30:03.760 --> 00:30:06.397 >> and I think there's some transfer learning we can you can see the 00:30:06.480 --> 00:30:10.157 reasoning work when it comes to you know tools like deep research and things like 00:30:10.240 --> 00:30:12.557 that but we're not making as fast as 00:30:12.640 --> 00:30:15.117 progress in the in the more soft domain. 00:30:15.200 --> 00:30:17.437 >> So so say softer domains meaning like 00:30:17.520 --> 00:30:22.157 domains in which it's harder harder or even impossible to actually verify 00:30:22.240 --> 00:30:26.317 correctness of of result in a sort of a deterministic factual grounded 00:30:26.400 --> 00:30:28.077 >> non-controversial way. Like if you have 00:30:28.160 --> 00:30:29.917 a a chronic disease, you could you could 00:30:30.000 --> 00:30:35.197 have you know you have a POTS or uh you 00:30:35.280 --> 00:30:38.157 know whatever EDS syndrome or and 00:30:38.240 --> 00:30:42.397 they're all they're all clusters and it's because it it is the domain of 00:30:42.480 --> 00:30:46.797 abstraction. It is not as concrete as code and math and things like that. So I 00:30:46.880 --> 00:30:48.397 think there's still a long ways to go there. 00:30:48.480 --> 00:30:51.677 >> Right. So sort of the more concrete the problem like it's the concretness of the 00:30:51.760 --> 00:30:54.477 problem that is the key variable not the difficulty of the problem. Would that be 00:30:54.560 --> 00:30:57.837 a way to think about it? >> Yeah. Yeah. I think the the uh 00:30:57.920 --> 00:31:02.237 concreteness in a sense of can you get a true or false ver verifiable 00:31:02.320 --> 00:31:05.277 >> right but like in any domain in any domain of human effort in which there's 00:31:05.360 --> 00:31:07.677 a verifiable answer we should expect extremely rapid progress. 00:31:07.760 --> 00:31:09.997 >> Yes. >> Right. >> Yes. Absolutely. And I I think that's what we're saying. 00:31:10.080 --> 00:31:12.477 >> Right. And that and that for sure includes math. That for sure includes 00:31:12.560 --> 00:31:15.837 physics for sure includes chemistry. For sure includes >> large areas of code. 00:31:15.920 --> 00:31:18.077 >> That's right. >> Right. What what else does that include do you think? 00:31:18.160 --> 00:31:22.237 >> Bio like we're seeing with a protein >> genomic. Yeah. Okay. Right. 00:31:22.320 --> 00:31:23.917 >> Yeah. Yeah. Things like that. I think 00:31:24.000 --> 00:31:28.237 some some areas of robotics, right? Um 00:31:28.320 --> 00:31:30.637 there's a clear outcome, right? 00:31:30.720 --> 00:31:34.717 >> Uh but but it's not that many. I mean, surprisingly, >> well, it depends. 00:31:34.800 --> 00:31:36.157 >> Yeah, depends on your point of view. 00:31:36.240 --> 00:31:39.917 That's some people might say that's a lot. Um so, uh and then um you you 00:31:40.000 --> 00:31:42.557 mentioned that we you mentioned the pace of improvement. So, what would you 00:31:42.640 --> 00:31:45.277 expect from the pace of improvement going forward for this? 00:31:45.360 --> 00:31:47.437 >> I I think we're we're ripping on coding. 00:31:47.520 --> 00:31:50.077 Like I think I think it's just it's 00:31:50.160 --> 00:31:52.637 going like I think it's going to be like 00:31:52.720 --> 00:31:55.277 what we're working on with with agent 00:31:55.360 --> 00:31:59.837 floor right now um is by by next year we think you're going to be sitting instead 00:31:59.920 --> 00:32:02.237 of rep in front of replet and you're 00:32:02.320 --> 00:32:04.477 shooting off multiple agents at a time. 00:32:04.560 --> 00:32:07.437 You're like planning a new feature. Um 00:32:07.520 --> 00:32:10.157 so I I want you know social network on 00:32:10.240 --> 00:32:12.637 top of my storefront and another one is 00:32:12.720 --> 00:32:16.157 like hey um refactor the database. Hey, 00:32:16.240 --> 00:32:17.997 in and you're running parallel agents. 00:32:18.080 --> 00:32:20.797 So, you have five 10 agents kind of working in the background and they're 00:32:20.880 --> 00:32:25.037 merging the code and taking care of all of that, but you also have a really nice 00:32:25.120 --> 00:32:28.317 interface on top of that that you're doing design and you're interacting with 00:32:28.400 --> 00:32:32.637 AI in a more creative way. Uh maybe using visuals and charts and things like 00:32:32.720 --> 00:32:34.957 that. So, there's a multimodal angle of 00:32:35.040 --> 00:32:37.597 that of that interaction. So I think you 00:32:37.680 --> 00:32:40.237 know creating software is going to be 00:32:40.320 --> 00:32:42.557 such an exciting 00:32:42.640 --> 00:32:45.597 uh area and and and I think that the lay 00:32:45.680 --> 00:32:48.957 person will be as good as a what a 00:32:49.040 --> 00:32:50.477 senior software engineer that works at 00:32:50.560 --> 00:32:52.637 Google uh is today. So I think I think 00:32:52.720 --> 00:32:56.477 that's happening very soon. Um but but 00:32:56.560 --> 00:33:00.877 you know I don't see them and be curious about your point of view but like my 00:33:00.960 --> 00:33:03.277 experience between as as a sort of a you 00:33:03.360 --> 00:33:06.077 know on the let's say healthcare side or 00:33:06.160 --> 00:33:10.797 more you know write me an essay side or more creative side haven't seen as much 00:33:10.880 --> 00:33:14.077 of a rapid improvement as what we're seeing in code. So so I think I think 00:33:14.160 --> 00:33:15.677 code is going to go to the moon. Math is 00:33:15.760 --> 00:33:18.557 probably as well some some you know 00:33:18.640 --> 00:33:22.877 scientific domains bio things like that those are are going to move really fast. 00:33:22.960 --> 00:33:26.077 >> Yeah. So there's this there's this there's this weird dynamic see if you 00:33:26.160 --> 00:33:28.957 agree with this and Eric also curious your point of view on this like there's 00:33:29.040 --> 00:33:31.357 this weird dynamic that we have and we have this in the office here a lot and I 00:33:31.440 --> 00:33:33.837 also have this with like the leading of entrepreneurs a lot which is this thing 00:33:33.920 --> 00:33:36.317 of like >> like wow this is the most amazing 00:33:36.400 --> 00:33:38.637 technology ever and it's moving really fast and yet we're still like really 00:33:38.720 --> 00:33:43.437 disappointed um and like it's not moving fast enough and like it's like maybe 00:33:43.520 --> 00:33:46.637 right on the verge of stalling out >> and like you know we should both be like 00:33:46.720 --> 00:33:49.517 hyper excited but also on the verge of like slitting our wrists because like 00:33:49.600 --> 00:33:51.517 you know the gravy train is coming to an end, 00:33:51.600 --> 00:33:53.757 >> right? And and I always wonder it's like 00:33:53.840 --> 00:33:56.557 you know on the one hand it's like okay like you know not all I don't know 00:33:56.640 --> 00:33:59.277 ladders go to the moon like just because something you know looks like it works 00:33:59.360 --> 00:34:01.517 or you know doesn't mean it's going to you know be able to you're going to be 00:34:01.600 --> 00:34:05.357 able to scale it up and have it work you know to the fullest extent. Um uh you 00:34:05.440 --> 00:34:08.397 know so like it's important to like recognize practical limits and not just 00:34:08.480 --> 00:34:11.677 extrapolate everything to infinity. Um on the other hand like you know we're 00:34:11.760 --> 00:34:14.237 dealing with magic here that we I think probably all would have thought was 00:34:14.320 --> 00:34:16.717 impossible 5 years ago or certainly 10 years ago. 00:34:16.800 --> 00:34:19.597 >> Like I I didn't you know look I I you know I got my CS degree in the late ' 00:34:19.680 --> 00:34:22.637 80s early 90s. I I never I didn't think I would live to see any of this, right? 00:34:22.720 --> 00:34:26.077 Like this is just amazing that this is actually happening in in in my lifetime. 00:34:26.160 --> 00:34:28.797 >> Um >> but but there's a huge bet on AGI, 00:34:28.880 --> 00:34:32.317 right? like whether it's the foundation models uh I think you know now the 00:34:32.400 --> 00:34:34.877 entire US economy is sort of a bet on 00:34:34.960 --> 00:34:37.357 AGI and and there are crucial questions 00:34:37.440 --> 00:34:42.317 to ask whether are we on track to AGI or not because there are some ways that I 00:34:42.400 --> 00:34:46.957 can tell you it doesn't seem like we're on track to AGI because we uh because 00:34:47.040 --> 00:34:50.557 there doesn't seem to be transfer learning across these domains that are 00:34:50.640 --> 00:34:55.677 that are you know significance right so if we get a lot better at code 00:34:55.760 --> 00:34:59.757 we're not immediately getting better at like generalized reasoning we need to go 00:34:59.840 --> 00:35:01.757 also you know get training data and 00:35:01.840 --> 00:35:03.757 create RL environment for bio or 00:35:03.840 --> 00:35:06.957 chemistry or physics or math or law or 00:35:07.040 --> 00:35:09.677 so so and this this has been the sort of 00:35:09.760 --> 00:35:11.277 point of discussion now in the AI 00:35:11.360 --> 00:35:14.077 community after the Darkish and Richard 00:35:14.160 --> 00:35:17.197 Sutton uh interview where uh you know 00:35:17.280 --> 00:35:19.197 Richard Sutton kind of poured this cold 00:35:19.280 --> 00:35:22.157 water on the on the bitter lesson. So 00:35:22.240 --> 00:35:24.397 everyone was using this uh essay that he 00:35:24.480 --> 00:35:26.237 wrote called the bitter lesson. The idea 00:35:26.320 --> 00:35:29.597 is that there are um infinitely scalable 00:35:29.680 --> 00:35:34.397 ways of uh doing uh uh AI research and 00:35:34.480 --> 00:35:37.437 and and and anytime you can pour more 00:35:37.520 --> 00:35:41.597 compute and more data and go more performance out you're just you know 00:35:41.680 --> 00:35:43.277 that's the ultimate way of getting to 00:35:43.360 --> 00:35:47.437 AGI and some people you know interpreted 00:35:47.520 --> 00:35:50.157 that interview that perhaps he's 00:35:50.240 --> 00:35:52.717 doubtful that even we're even on a on a 00:35:52.800 --> 00:35:56.317 bitter uh lessened path here and perhaps 00:35:56.400 --> 00:35:58.797 the current training regime is actually 00:35:58.880 --> 00:36:01.277 very much the opposite in which we we 00:36:01.360 --> 00:36:03.837 are so dependent on human data and human 00:36:03.920 --> 00:36:05.917 annotation and and all of that stuff. So 00:36:06.000 --> 00:36:09.037 I think the I I agree with you. I mean 00:36:09.120 --> 00:36:13.197 as a company we're we're excited about where things are headed but but there's 00:36:13.280 --> 00:36:15.277 there's a question of like are we on 00:36:15.360 --> 00:36:20.397 track to AGI or not and be curious what you think. So, so and you know Ilia I 00:36:20.480 --> 00:36:23.037 think you know Ilioskcover makes a makes a specific form of this argument which 00:36:23.120 --> 00:36:25.357 is basically like we're just literally running out of training data. It's a 00:36:25.440 --> 00:36:28.557 fossil fuel argument right like if we slurped all the training fundamentally 00:36:28.640 --> 00:36:30.797 we've slurped all the data off the internet that is where almost all the 00:36:30.880 --> 00:36:33.597 data is at this point. There's a little bit more data that's in like you know 00:36:33.680 --> 00:36:36.157 private dark pool somewhere that we're going to go get but like 00:36:36.240 --> 00:36:38.477 >> we have it all and then right we're we're in this business now trying to 00:36:38.560 --> 00:36:41.037 generate new data but generating new data is hard and expensive you know 00:36:41.120 --> 00:36:43.517 compared to just like slurping things off the internet. So 00:36:43.600 --> 00:36:46.557 >> there are these arguments. Um you know having said that you know you get into 00:36:46.640 --> 00:36:48.797 definitional questions here really quick which are kind of a rabbit hole but 00:36:48.880 --> 00:36:51.117 having said that like you mentioned transfer learning. So transfer learning 00:36:51.200 --> 00:36:54.237 is the ability of the machine to right to be an expert in one domain and then 00:36:54.320 --> 00:36:56.397 and then generalize that into another domain. 00:36:56.480 --> 00:36:58.957 >> My answer to that is like have you met people? 00:36:59.040 --> 00:37:03.037 >> Um and how many people do you know are able to do transfer learning? 00:37:03.120 --> 00:37:05.437 >> Not many. Right. Well because there's 00:37:05.520 --> 00:37:07.037 >> quite the opposite actually. The nerdier 00:37:07.120 --> 00:37:11.517 they are in a certain domain the kind of you know often they have blind spots. We 00:37:11.600 --> 00:37:14.797 joke about how everyone's just in one area or they make some like 00:37:14.880 --> 00:37:18.157 massive mistake and and like don't trust them on this but on this other topic you 00:37:18.240 --> 00:37:20.717 know >> right? Yeah. Well and this is a well-known thing among like for example 00:37:20.800 --> 00:37:22.717 public intellect. So this happens there's actually been whole books 00:37:22.800 --> 00:37:25.197 written about this on so-called public intellectuals. So you get these people 00:37:25.280 --> 00:37:28.077 who show up on TV and they're experts and what happens is they're like an 00:37:28.160 --> 00:37:30.717 expert in economics right and then they show up on TV and they talk about 00:37:30.800 --> 00:37:33.757 politics and they don't know anything about politics right or they don't know 00:37:33.840 --> 00:37:36.397 anything about like medicine or they don't know anything about the law or 00:37:36.480 --> 00:37:38.797 they don't know anything about computers. You know, this is the Paul 00:37:38.880 --> 00:37:41.197 Gregman talking about how the internet's going to be no more significant than the 00:37:41.280 --> 00:37:43.197 fax machine. >> Facts. >> He's a brilliant economist. He has no 00:37:43.280 --> 00:37:44.237 idea what a how a computer works. 00:37:44.320 --> 00:37:45.517 >> Is he a brilliant economist? 00:37:45.600 --> 00:37:49.437 >> Well, at one at one point at one point 00:37:49.520 --> 00:37:54.397 at one point, let's get even if even if he's a brilliant Well, this is the thing 00:37:54.480 --> 00:37:56.877 like what does that mean? Like should a brilliant economist be able to 00:37:56.960 --> 00:38:00.717 extrapolate, you know, the internet is is a good question. But um but the point 00:38:00.800 --> 00:38:03.837 being like even if he is a you know, take any take anybody Oh, by the way, or 00:38:03.920 --> 00:38:07.757 like Ein like Einstein's like actually my favorite example. I think you'd agree 00:38:07.840 --> 00:38:09.357 Einstein was a brilliant physicist. 00:38:09.440 --> 00:38:11.437 >> He was like a he was he was a Stalinist. 00:38:11.520 --> 00:38:14.237 Like he was just he was Yeah. He was a socialist and he was a Stalinist and he 00:38:14.320 --> 00:38:16.317 was like he thought like Stalin was fantastic. 00:38:16.400 --> 00:38:18.797 >> Out still. >> Yeah. Okay. All right. 00:38:18.880 --> 00:38:24.077 >> True socialism. >> All right. All right. Einstein, you know, I'll I'll 00:38:24.160 --> 00:38:26.957 I'll take your word for it. But like once he got into politics, he was just 00:38:27.040 --> 00:38:29.837 like totally loopy or or you know, even right or wrong. It's just he just 00:38:29.920 --> 00:38:32.557 sounded like all of a sudden like an undergraduate lunatic, like somebody in 00:38:32.640 --> 00:38:35.357 a dorm room. Like he there was no transfer learning from physics into 00:38:35.440 --> 00:38:39.197 politics. like he he didn't listen right or wrong he didn't there was no there 00:38:39.280 --> 00:38:42.477 was clearly there was nothing new in his political analysis it was the same wrote 00:38:42.560 --> 00:38:45.757 routine you get out of >> you know yeah so so in a way the 00:38:45.840 --> 00:38:50.077 argument you're making is like we maybe already a human level AI I mean perhaps 00:38:50.160 --> 00:38:54.157 the definition of AGI is is is something totally different is like above human 00:38:54.240 --> 00:38:57.677 level that something that totally generalizes across domains it's it's not 00:38:57.760 --> 00:39:00.317 something that we've seen >> yeah like we've ideal yeah I was saying 00:39:00.400 --> 00:39:02.877 we we've we've and you know look we should we should shoot big but we we've 00:39:02.960 --> 00:39:07.437 idealized a a we've idealized a goal um that may be idealized in a way that 00:39:07.520 --> 00:39:10.637 like number one it's just it it's it's like so far beyond what people can do 00:39:10.720 --> 00:39:13.277 that it's you're no longer it's no longer relevant comparison to people and 00:39:13.360 --> 00:39:16.397 and usually AGI is defined as you know able to do everything better than a 00:39:16.480 --> 00:39:18.877 person can >> and it's like well okay so if doing 00:39:18.960 --> 00:39:21.357 everything better than a person can it's like if a person can't do any transfer 00:39:21.440 --> 00:39:24.717 learning at all >> right doing even a little little bit a 00:39:24.800 --> 00:39:27.357 marginal bit might might actually be better or it might not matter just 00:39:27.440 --> 00:39:30.077 because no no human can do it and so therefore you just you just stack up the 00:39:30.160 --> 00:39:33.357 domains there's also this well-known phenomenon in AI with you know t 00:39:33.440 --> 00:39:36.317 typically this works the other way which there's a phenomenon AI AI engineers 00:39:36.400 --> 00:39:38.957 always complain about and scientists always complain about which is the 00:39:39.040 --> 00:39:42.477 definition of AI is always the next thing that that the machine can't do and 00:39:42.560 --> 00:39:45.677 so like the definition for of AI for a long time was like can it beat humans at 00:39:45.760 --> 00:39:47.517 chess >> and then the minute it could beat humans 00:39:47.600 --> 00:39:50.717 at chess that was no longer AI that was just like oh that's just like boring 00:39:50.800 --> 00:39:53.437 >> that's computer chess it became >> computer chess it's just like boring and 00:39:53.520 --> 00:39:56.477 now it's an app on your iPhone and nobody nobody and nobody cares right and 00:39:56.560 --> 00:40:00.317 it's immediately then >> cheering test was the test and then we passed it and nobody 00:40:00.400 --> 00:40:02.957 >> we blew this is a really big deal >> there was no celebration 00:40:03.040 --> 00:40:06.317 >> there was no parties That's exactly right. There was no for 80 years the 00:40:06.400 --> 00:40:09.117 Turing test I mean they made a movie about it like the whole thing that was 00:40:09.200 --> 00:40:11.917 the thing and like we blew right through it and nobody even registered it. Nobody 00:40:12.000 --> 00:40:14.317 cares. It gets no credit for it. We're just like ah it's still you know 00:40:14.400 --> 00:40:17.997 complete p piece of like >> right and so there's this thing where so 00:40:18.080 --> 00:40:21.757 the AI scientists are are are used to complaining basically that they're that 00:40:21.840 --> 00:40:24.157 they're they're being they're always being judged against the next thing as 00:40:24.240 --> 00:40:26.317 opposed to all the things they've already they've already solved. 00:40:26.400 --> 00:40:29.757 >> Um uh but but that's maybe the other side of it which is they're also putting 00:40:29.840 --> 00:40:33.757 out for themselves um an unreasonable goal. an an unreasonable goal and then 00:40:33.840 --> 00:40:36.957 doing this sort of self flagagillation kind of along the way and and and I I 00:40:37.040 --> 00:40:39.277 kind of wonder yeah I I wonder kind of which way that cuts. 00:40:39.360 --> 00:40:41.037 >> Yeah. Yeah. It's an interesting question 00:40:41.120 --> 00:40:42.557 like I started thinking about this idea 00:40:42.640 --> 00:40:45.117 of like it doesn't matter whether it's 00:40:45.200 --> 00:40:48.237 truly AGI and the way I define AGI is 00:40:48.320 --> 00:40:51.197 that you put in a AI system in any 00:40:51.280 --> 00:40:53.677 environment and efficiently learns right 00:40:53.760 --> 00:40:55.437 >> um you know it doesn't have to have that 00:40:55.520 --> 00:40:58.557 much prior knowledge in order to kind of learn something but also can transfer 00:40:58.640 --> 00:41:00.877 that knowledge across different domains. 00:41:00.960 --> 00:41:03.677 Um but you know we can get to like 00:41:03.760 --> 00:41:06.157 functional AGI and what functional AGI 00:41:06.240 --> 00:41:09.357 is is just yeah collect data on every uh 00:41:09.440 --> 00:41:12.637 useful uh economic activity in uh in the 00:41:12.720 --> 00:41:15.517 world today and train an LLM on top of 00:41:15.600 --> 00:41:19.677 that or train the same foundation model on top of that and and we we'll go we'll 00:41:19.760 --> 00:41:23.757 target every sector economy and and you can automate a big part of labor that 00:41:23.840 --> 00:41:27.757 way. So I think I think yeah I think we're on that track for sure. 00:41:27.840 --> 00:41:31.197 >> Right. um you tweeted after GBG5 came out that you were feeling the 00:41:31.280 --> 00:41:35.437 diminishing returns. Yeah. What were you expecting and but and and what needs to 00:41:35.520 --> 00:41:38.077 be done? Do we need another breakthrough to get back to the pace of growth or 00:41:38.160 --> 00:41:40.637 what are your thoughts there? >> I mean this this whole discussion is is 00:41:40.720 --> 00:41:42.557 sort of about that and and my feeling is 00:41:42.640 --> 00:41:46.877 that uh you know GPT5 uh got good at 00:41:46.960 --> 00:41:51.837 verifiable domains. It didn't feel that much better at anything else. the more 00:41:51.920 --> 00:41:53.437 human angle of it. It felt like it 00:41:53.520 --> 00:41:55.677 regressed and like you had this uh sort 00:41:55.760 --> 00:41:59.837 of uh Reddit pitchfork uh sort of uh 00:41:59.920 --> 00:42:02.317 movement against against Sam and Open AI 00:42:02.400 --> 00:42:04.157 because they felt like they lost a 00:42:04.240 --> 00:42:06.877 friend. Gupta felt a lot more human and 00:42:06.960 --> 00:42:10.957 closer uh whereas GT5 felt a lot more 00:42:11.040 --> 00:42:14.557 robotic, you know, very in its head kind of trying to think through through 00:42:14.640 --> 00:42:17.437 everything. And um and so I I I would 00:42:17.520 --> 00:42:19.117 have just expected like when we went 00:42:19.200 --> 00:42:23.037 from GP2 2 to 3, it was clear it was 00:42:23.120 --> 00:42:25.757 getting a lot more human. It was uh a 00:42:25.840 --> 00:42:30.557 lot closer to our experience. It can you can feel like it's actually all it gets 00:42:30.640 --> 00:42:32.397 me like there's something about it that 00:42:32.480 --> 00:42:34.557 understands the world better. Similarly 00:42:34.640 --> 00:42:39.277 3 to four to five didn't feel like it 00:42:39.360 --> 00:42:42.717 was a better overall 00:42:42.800 --> 00:42:45.597 being as it were. But is that is that is 00:42:45.680 --> 00:42:47.917 that is that a is the question there 00:42:48.000 --> 00:42:50.717 like is it emotionality? Is it partly 00:42:50.800 --> 00:42:53.437 emotionality but but again partly like I 00:42:53.520 --> 00:42:55.437 like to ask models like very 00:42:55.520 --> 00:42:58.557 controversial uh things. Um can it 00:42:58.640 --> 00:43:01.117 reason through 00:43:01.200 --> 00:43:06.717 uh I don't know how deep we want to go here but like um what happened with 00:43:06.800 --> 00:43:09.837 World Trade 7, >> right? >> Sure. 00:43:09.920 --> 00:43:11.197 >> It's an interesting question, right? 00:43:11.280 --> 00:43:15.437 Like I'm not I'm not putting out a theory, but like it's interesting like 00:43:15.520 --> 00:43:17.517 how did it you know and and can it can 00:43:17.600 --> 00:43:19.917 it think through controversial questions 00:43:20.000 --> 00:43:22.557 >> in the same way that it can go think 00:43:22.640 --> 00:43:27.597 through a coding problem and there hasn't been any movement there like the 00:43:27.680 --> 00:43:30.797 all the reasoning and all of that stuff I haven't se and not just that you know 00:43:30.880 --> 00:43:36.237 that's a cute example but like um co right like you know the origins of co 00:43:36.320 --> 00:43:41.197 right >> you know go you know dig up GPT4 or other models 00:43:41.280 --> 00:43:44.957 and go to GPT5, you're not going to find that much difference of okay, let's 00:43:45.040 --> 00:43:48.157 reason together. Let's try to figure out what was the origins of CO because it's 00:43:48.240 --> 00:43:52.237 still an unanswered question, you know, and I don't see them making progress in 00:43:52.320 --> 00:43:53.517 that. I mean, you play a lot with them. 00:43:53.600 --> 00:43:55.277 Do you feel like >> I use it differently? I don't know, 00:43:55.360 --> 00:43:59.597 maybe I have different expectations. Um, I I'm I the way I my main use case 00:43:59.680 --> 00:44:03.437 actually is sort of sort of PhD and everything at my beck and call. Um, and 00:44:03.520 --> 00:44:05.997 so I'm I'm trying to get it to explain things to me more than I'm trying to 00:44:06.080 --> 00:44:09.357 like, you know, have conversations with it. Maybe maybe I'm just unusual with 00:44:09.440 --> 00:44:12.317 that. But >> and that that that gets back >> well. So what I what I what I found 00:44:12.400 --> 00:44:16.397 specifically is uh a combination of like GPT5 Pro plus deep reasoning or like 00:44:16.480 --> 00:44:21.277 Rock 4 heavy like the you know the the highest end models um u like that um you 00:44:21.360 --> 00:44:24.797 know they now basically generate 30 to 40 page you know essentially books on 00:44:24.880 --> 00:44:29.677 demand on any topic. Um and so anytime I get curious about something you just 00:44:29.760 --> 00:44:32.397 take it maybe it's my version of it but it's something like I don't like a good 00:44:32.480 --> 00:44:36.157 here's a good example. um when when when a when an advanced economy puts a tariff 00:44:36.240 --> 00:44:38.957 on on on a raw m you know on a raw material or on a finished good like who 00:44:39.040 --> 00:44:42.477 pays >> you know is it is it the consumer is it the is it the importer is it the 00:44:42.560 --> 00:44:45.677 exporter or is it the producer and and this actually a very complicated it 00:44:45.760 --> 00:44:48.557 turns out very complicated question it's a big big big thing that economists 00:44:48.640 --> 00:44:51.757 study a lot and it's just like okay who you know who pays and what I found like 00:44:51.840 --> 00:44:53.677 for that kind of thing is it's outstanding 00:44:53.760 --> 00:44:57.437 >> well well but but it's outstanding at um sort of going out of the web getting 00:44:57.520 --> 00:45:00.637 information synthesizing it >> correct it it gives me it gives me a 00:45:00.720 --> 00:45:02.957 synthesized 20 30 40 p basically tops 00:45:03.040 --> 00:45:05.437 out 40 p 40 40 pages of PDF. Yeah. 00:45:05.520 --> 00:45:09.037 >> Um uh but I can get I can get up to 40 pages of PDF but it's a completely 00:45:09.120 --> 00:45:12.317 coherent and as far as I can tell for everything I've cross-cheed a completely 00:45:12.400 --> 00:45:15.997 like it like world class like if I hired you know for a question like that if I 00:45:16.080 --> 00:45:20.317 hired like a great you know econ posttock at Stanford who just like went 00:45:20.400 --> 00:45:22.877 out and did that work like it would maybe be that good. 00:45:22.960 --> 00:45:24.477 >> Yeah. Um but then but then of course the 00:45:24.560 --> 00:45:28.077 significance is it's like it's like you know at least for this is true for many 00:45:28.160 --> 00:45:29.917 domains you know kind of PhD and everything and so 00:45:30.000 --> 00:45:33.277 >> but but this is synthesizing knowledge not trying to create new knowledge. 00:45:33.360 --> 00:45:35.037 >> Well but this this gets to the this sort 00:45:35.120 --> 00:45:37.837 of you know of course the you get into the angels dancing on the head of a pin 00:45:37.920 --> 00:45:40.797 thing which is like what what you know what's the difference how many how much 00:45:40.880 --> 00:45:44.157 new knowledge ever actually is there anyway? What do you actually expect from 00:45:44.240 --> 00:45:47.437 people when you ask them questions? Um, and so what what I'm looking for is 00:45:47.520 --> 00:45:50.157 like, yes, explain this to me in like the the the clearest, most 00:45:50.240 --> 00:45:53.677 sophisticated, most complex, most like complete way that it's possible for 00:45:53.760 --> 00:45:57.757 somebody to, you know, for a real expert to be able to to to explain things to me. 00:45:57.840 --> 00:46:00.077 >> Um, and that's what I use it for. And again, as far as I can tell from the 00:46:00.160 --> 00:46:02.877 crossing, like I'm getting, you know, like almost like basically 100 out of 00:46:02.960 --> 00:46:07.517 100, like I don't even think I've had an issue in months where it's like had had 00:46:07.600 --> 00:46:09.917 a problem in it. >> And it's like, yeah, you can say, yeah, 00:46:10.000 --> 00:46:12.237 synthesizing is supposed to create new information, but like it's it's 00:46:12.320 --> 00:46:15.037 generating a 40 p. He's basically generating a 40-page book. 00:46:15.120 --> 00:46:18.157 >> That's amazing. >> That's like incredibly like fluid. It's, 00:46:18.240 --> 00:46:22.077 you know, it's it's it's it's you know, the the logical coherence of the entire 00:46:22.160 --> 00:46:26.477 like it's it's a great writing. Like if if you if you evaluated an a a human 00:46:26.560 --> 00:46:29.677 author on it, you would say, "Wow, that's a great author." You know, do are 00:46:29.760 --> 00:46:32.877 people who write books, you know, creating new knowledge? Well, yeah. 00:46:32.960 --> 00:46:35.997 Well, sort of not because a lot of what they're doing is building on everything 00:46:36.080 --> 00:46:39.677 that came before them is synthesizing a mind, but also like a book is a creative 00:46:39.760 --> 00:46:43.517 accomplishment, right? And so, >> yeah, one of the thing I'm I'm I'm I'm 00:46:43.600 --> 00:46:45.917 interested in I'm hoping AI could help 00:46:46.000 --> 00:46:49.357 us solve is just like how confusing the 00:46:49.440 --> 00:46:53.037 information ecosystem right now. You know, everything feels like propaganda. 00:46:53.120 --> 00:46:55.997 Like it doesn't feel like you're getting real information from anywhere. So, I I 00:46:56.080 --> 00:46:59.837 really want an AI that could help me reason from first principles about 00:46:59.920 --> 00:47:04.237 what's happening in the world for me to actually get real information. and and 00:47:04.320 --> 00:47:08.877 maybe that's an unreasonable sort of ask of of the AI researchers, but but I 00:47:08.960 --> 00:47:12.077 don't think we're we have made any progress there. So maybe I'm over focus. 00:47:12.160 --> 00:47:15.277 Yeah, maybe I'm over in being in my my line or maybe I'm over focused on ar 00:47:15.360 --> 00:47:18.877 arguing at people as opposed to um trying to get as trying to get underline 00:47:18.960 --> 00:47:21.917 truth. But well here here's the thing I I do a lot with this is I just say like 00:47:22.000 --> 00:47:25.837 take take a provocative point of view um and then steel man the position take 00:47:25.920 --> 00:47:29.277 your co thing steel so I often I often pair these steel man the position that 00:47:29.360 --> 00:47:32.877 it was a lab leak um and steel man the position that it was natural origins 00:47:32.960 --> 00:47:34.317 >> um and and again like is this creativity 00:47:34.400 --> 00:47:37.357 or not? I don't know. But like what comes back is like 30 pages each of like 00:47:37.440 --> 00:47:40.237 wow like that is like the most compelling case in the world I can 00:47:40.320 --> 00:47:42.877 imagine with like every you know everything marshaled against it like the 00:47:42.960 --> 00:47:44.557 argument structured in like the most possible 00:47:44.640 --> 00:47:47.037 >> part of the reason that started happening is because it stopped being 00:47:47.120 --> 00:47:51.277 taboo to talk about a human origin when it was taboo 00:47:51.360 --> 00:47:53.917 >> the the AIS would like talk will you 00:47:54.000 --> 00:47:55.757 know talk down to like oh you're a 00:47:55.840 --> 00:47:58.957 conspiracy theorist and so yes uh so 00:47:59.040 --> 00:48:02.877 there's a there's a you know period of time and so it takes something truly 00:48:02.960 --> 00:48:05.757 controversial and they actually they they can't reason about it because of 00:48:05.840 --> 00:48:09.677 all RLHF and answers all the limitations and as as you know I won't pick no 00:48:09.760 --> 00:48:12.237 specific ones here but like there there are certain certain big models that will 00:48:12.320 --> 00:48:14.877 still lecture you >> that you're a bad person for asking that 00:48:14.960 --> 00:48:17.917 question but but you know like I just there some of them are just like really 00:48:18.000 --> 00:48:19.437 really open now to you know being able 00:48:19.520 --> 00:48:20.797 to do these things 00:48:20.880 --> 00:48:25.517 >> um and then um uh yeah so um okay uh 00:48:25.600 --> 00:48:29.517 yeah so okay so yeah so there's this yeah so so basically like ultimately 00:48:29.600 --> 00:48:32.717 what you're looking for like the ultimate thing would be if there's 00:48:32.800 --> 00:48:34.957 something that's like I don't I think anybody's really defined this really 00:48:35.040 --> 00:48:37.597 well because it's not because again it's like the conventional all the 00:48:37.680 --> 00:48:40.077 conventional definitions of AGI are like basically comparing to people. 00:48:40.160 --> 00:48:41.757 >> Yeah. >> And there there and there it's always 00:48:41.840 --> 00:48:44.317 like you know it's the conventional 00:48:44.400 --> 00:48:49.277 explanations of of of um of uh of AGI always for me struck me a lot like the 00:48:49.360 --> 00:48:51.677 debate around like whether a self-driving car works or not which is 00:48:51.760 --> 00:48:55.277 is does a self-driving car work because it's a perfect driver uh or does it work 00:48:55.360 --> 00:48:58.877 because it is a is better than the human driver and better than the human driver 00:48:58.960 --> 00:49:01.757 I think is actually quite you know just like with the the chess thing and the go 00:49:01.840 --> 00:49:04.477 thing. I actually think like that that that's like a real thing. And then and 00:49:04.560 --> 00:49:07.277 then and then and then there's the like is it a perfect driver which is you know 00:49:07.360 --> 00:49:09.597 what they're obviously the the self-driving car companies are working 00:49:09.680 --> 00:49:12.557 for but then I think you're looking for something beyond the perfect driver. 00:49:12.640 --> 00:49:16.077 You're looking for the car who like knows where to go. So I I I I'm of two 00:49:16.160 --> 00:49:20.157 minds, right? So one mind is the sort of practical entrepreneur, right? 00:49:20.240 --> 00:49:22.797 >> Uh and I just I have so many toys to 00:49:22.880 --> 00:49:27.277 play with to build like stop AI progress today and replet will continue to get 00:49:27.360 --> 00:49:30.477 better for the next 5 years. like there's so much we can do just on the 00:49:30.560 --> 00:49:33.197 app uh app layer and the infrastructure layer. 00:49:33.280 --> 00:49:37.757 >> So you know I but but I think that will you know the the foundation models will 00:49:37.840 --> 00:49:40.877 continue to get better as well and so it's it's a very exciting time in our 00:49:40.960 --> 00:49:45.277 industry. Um the other mind is more academic because as a kid I've always 00:49:45.360 --> 00:49:49.197 been interested in the nature of consciousness, nature of intelligence. I 00:49:49.280 --> 00:49:51.277 was always interested in AI and reading 00:49:51.360 --> 00:49:56.397 the literature there and I would point to the RL uh literature. So Richard 00:49:56.480 --> 00:50:00.397 Soden, there's another guy I think co-founder of deep mind Shane Lag wrote 00:50:00.480 --> 00:50:02.797 wrote a paper trying to define what AGI 00:50:02.880 --> 00:50:06.237 is. Um and in there I think that the 00:50:06.320 --> 00:50:08.717 definition of AI I think is the is the 00:50:08.800 --> 00:50:12.317 original perhaps correct one which is uh 00:50:12.400 --> 00:50:16.237 efficient continual learning. >> Okay. Like if you if you truly want to 00:50:16.320 --> 00:50:20.557 build an artificial general intelligence that you can drop in any domain, you can 00:50:20.640 --> 00:50:23.117 drop in a car without that much prior 00:50:23.200 --> 00:50:27.437 knowledge about cars and within um you 00:50:27.520 --> 00:50:31.117 know how long does it take a human to to learn how to drive you within months be 00:50:31.200 --> 00:50:35.437 able to drive a car really well, you know, generalized skill sort of 00:50:35.520 --> 00:50:39.437 generalized skill acquisition, generalized understanding acquisition, 00:50:39.520 --> 00:50:43.277 generalized reasoning acquisition. And I think that's the thing that will like 00:50:43.360 --> 00:50:47.437 truly change the world. That's the thing that would give us a better 00:50:47.520 --> 00:50:49.597 understanding of of the human mind of 00:50:49.680 --> 00:50:51.917 human consciousness and that's the thing 00:50:52.000 --> 00:50:54.637 that will like propel us to the next uh 00:50:54.720 --> 00:50:57.437 level of human civilization. 00:50:57.520 --> 00:50:59.757 on a civilization level that's a really 00:50:59.840 --> 00:51:04.760 deep question but separ 00:51:05.600 --> 00:51:07.517 but but there's an academic aspect of it that I'm really 00:51:07.600 --> 00:51:10.477 >> so what and what odds what if we're on if we're on Kelsey today what what odds 00:51:10.560 --> 00:51:11.677 do we place on that 00:51:11.760 --> 00:51:15.197 >> I I I'm kind of bearish on on on true 00:51:15.280 --> 00:51:17.757 AGI breakthrough because 00:51:17.840 --> 00:51:20.797 >> what we built is so useful and 00:51:20.880 --> 00:51:23.997 economically valuable uh so in a way 00:51:24.080 --> 00:51:28.157 >> good enough good enough is the enemy Yeah. Yeah. Do do you remember that 00:51:28.240 --> 00:51:29.997 essay? Um, >> worse is better. 00:51:30.080 --> 00:51:32.397 >> Worse is better. >> Worse is better. Worse is better. And and 00:51:32.480 --> 00:51:35.117 >> so there's like a local there's like a trap. There's like a local local maximum 00:51:35.200 --> 00:51:38.557 trap. We're in a local maximum >> local maximum trap where it's because 00:51:38.640 --> 00:51:41.757 it's because it's good enough for so much economically productive work. 00:51:41.840 --> 00:51:44.477 >> Yes. >> It relieves the pressure um in the 00:51:44.560 --> 00:51:46.077 system to create the generalized answer. 00:51:46.160 --> 00:51:49.197 >> Yes. And and then you have the weirdos like Rich Sutton and others that are 00:51:49.280 --> 00:51:51.917 still trying to go that down that path and maybe they'll succeed, 00:51:52.000 --> 00:51:55.997 >> right? Uh but there's enormous optimization energy behind the current 00:51:56.080 --> 00:51:58.637 thing that we're hell climbing on this like local maximum. 00:51:58.720 --> 00:52:01.037 >> Right. Right. Right. And and the irony of it is everybody's worried about like 00:52:01.120 --> 00:52:03.517 the you know the gazillions of dollars going into building out all this stuff 00:52:03.600 --> 00:52:06.797 and and so the the the most ironic thing in the world would be if the gazillions 00:52:06.880 --> 00:52:08.477 of dollars are going into the local maximum. 00:52:08.560 --> 00:52:10.557 >> That's right. >> As as opposed to a counterfactual world 00:52:10.640 --> 00:52:12.397 in which they're going into solving the general problem. 00:52:12.480 --> 00:52:15.197 >> But but it's also potentially irrational. Like maybe the general 00:52:15.280 --> 00:52:20.077 problem is actually you know not within our lifetimes. Who knows? Right. Um, h 00:52:20.160 --> 00:52:22.877 >> how much further do you think like do you think we squeeze most of the juice 00:52:22.960 --> 00:52:26.077 out of out of LLMs in general then? Or are there any other research directions 00:52:26.160 --> 00:52:29.597 that you're particularly um excited about? 00:52:29.680 --> 00:52:31.837 >> Well, that's the thing. I think the 00:52:31.920 --> 00:52:33.997 problem is there aren't that that many. 00:52:34.080 --> 00:52:38.717 I I think the the the breakthroughs in RL are incredibly exciting, but we also 00:52:38.800 --> 00:52:42.637 knew about them now for like over 10 years where you marry generative uh 00:52:42.720 --> 00:52:47.357 systems with uh with tree search and things like that. Um but but there's a 00:52:47.440 --> 00:52:51.997 lot more to go there and I think again the the the the original minds behind 00:52:52.080 --> 00:52:55.517 reinforcement learning are trying to go down that path and try to kind of 00:52:55.600 --> 00:53:00.557 bootstrap intelligence from scratch. Uh Carmarmac is is going down that path as 00:53:00.640 --> 00:53:04.237 as far as I understand Carmarmac you guys may be invested but the the you 00:53:04.320 --> 00:53:07.277 know they're they're not trying to go down the LLM path. So there are people 00:53:07.360 --> 00:53:11.997 that are trying to do that but I'm not seeing a lot of progress or outcome 00:53:12.080 --> 00:53:13.837 there but I watch it kind of from far. 00:53:13.920 --> 00:53:16.237 Although, you know, for all we know, it's already there's already a bot on X 00:53:16.320 --> 00:53:19.837 somewhere. >> You know, you know, you never know. It 00:53:19.920 --> 00:53:21.917 might not be a big announcement. It might just be a you know, one day 00:53:22.000 --> 00:53:24.477 there's just like a bot on X that starts winning all the arguments. 00:53:24.560 --> 00:53:28.957 >> Yeah, it could be >> or a code a user read and all of a 00:53:29.040 --> 00:53:31.677 sudden it's >> generating incredible software. Um, 00:53:31.760 --> 00:53:34.477 okay. Let's uh let's spend our remaining minutes. Let's let's let's talk about 00:53:34.560 --> 00:53:38.717 you. So, uh so uh so how so yeah, take us start from the beginning with your uh 00:53:38.800 --> 00:53:41.197 with your life and how how did you get how did you get from being born to being 00:53:41.280 --> 00:53:45.597 in Silicon Valley? >> Okay. um >> in two minutes. Yeah, 00:53:45.680 --> 00:53:47.197 >> I'm just I'm joking. But 00:53:47.280 --> 00:53:50.557 >> yeah, I I got introduced to computers uh 00:53:50.640 --> 00:53:53.437 very very early on. And so for whatever 00:53:53.520 --> 00:53:58.557 reason, so I was born in Aman, Jordan >> and for whatever reason, my my dad who 00:53:58.640 --> 00:54:00.397 was just a government engineer at the 00:54:00.480 --> 00:54:03.597 time uh decided that computers were 00:54:03.680 --> 00:54:07.757 important and he didn't have a lot of money took out of that, bought a 00:54:07.840 --> 00:54:10.477 computer. It was the first computer in their in our neighborhood. first 00:54:10.560 --> 00:54:14.317 computer of anyone I know. And I just 00:54:14.400 --> 00:54:18.077 one of my earliest memories I was 6 years old just watching my dad unpack 00:54:18.160 --> 00:54:20.877 this machine and sort of open up this 00:54:20.960 --> 00:54:24.317 huge manual and kind of finger type CD 00:54:24.400 --> 00:54:29.197 LS MKDIR and like I would, you know, be behind his shoulder and just like 00:54:29.280 --> 00:54:33.197 watching him, you know, type these commands and seeing the sort of machine 00:54:33.280 --> 00:54:36.157 kind of respond and do exactly what he's 00:54:36.240 --> 00:54:41.680 asked it to do. Um, >> pop in Tylenol as your 00:54:41.840 --> 00:54:47.757 Exactly. Autism activated. >> Of course, you have to. 00:54:47.840 --> 00:54:51.517 >> You have to. >> Exactly. What kind of um what kind of 00:54:51.600 --> 00:54:55.917 computer was it? >> Uh it was uh an IBM as far as I 00:54:56.000 --> 00:55:00.557 remember. It was IBM PC. >> What year was this? Uh 1993. 00:55:00.640 --> 00:55:03.917 >> 1993. Okay. So, it's DOS. So, did it have Windows at that point or 00:55:04.000 --> 00:55:05.197 >> No, it didn't have Windows. 00:55:05.280 --> 00:55:07.917 >> Right before Windows. right before Windows, but I think Windows had been 00:55:08.000 --> 00:55:11.277 out, but you would add >> you it was an add-on. You wouldn't boot 00:55:11.360 --> 00:55:13.597 it up. So, we I think we bought the disc 00:55:13.680 --> 00:55:16.397 for uh for uh for Windows and you had to 00:55:16.480 --> 00:55:20.797 kind of uh bootloaded, you know, from the disk and and it will open Windows 00:55:20.880 --> 00:55:24.077 and you can click around. It wasn't that interesting cuz there wasn't a lot on 00:55:24.160 --> 00:55:25.917 it. So, a lot of time I just spend in 00:55:26.000 --> 00:55:28.477 DOSs and writing batch files and opening 00:55:28.560 --> 00:55:31.597 games and messing around with that. Um 00:55:31.680 --> 00:55:36.637 but it wasn't until Visual Basic that I started. So like after Windows 95 that I 00:55:36.720 --> 00:55:38.317 started making real software, right? 00:55:38.400 --> 00:55:41.997 >> Uh and the first idea I had was um I I 00:55:42.080 --> 00:55:44.477 used to be a huge gamer. So I used I 00:55:44.560 --> 00:55:49.437 used to go to these uh Lang gaming cafes and play Counter Strike and I would go 00:55:49.520 --> 00:55:52.797 there and you know the whole thing is full of computers but they don't use any 00:55:52.880 --> 00:55:56.477 software to run their business. M it was just like people running around just 00:55:56.560 --> 00:56:00.317 like writing down your machine number, how much time you spend on it and how 00:56:00.400 --> 00:56:03.277 much did you pay and kind of tapping your shoulders like hey you need to pay 00:56:03.360 --> 00:56:06.797 a little more for that. And I asked him like why don't you like just build a 00:56:06.880 --> 00:56:09.997 piece of software that allows me to log in and have a time or whatever. And I 00:56:10.080 --> 00:56:13.117 was like yeah we don't know how to do that. And I was like okay I think I know 00:56:13.200 --> 00:56:17.357 how to do that. So I spent I was like 12 or something like that. I spent like 2 00:56:17.440 --> 00:56:21.517 years building that uh and then went out and tried to sell it and was able to 00:56:21.600 --> 00:56:23.597 sell it uh and was making so much money. 00:56:23.680 --> 00:56:26.637 I remember McDonald's opened uh in 00:56:26.720 --> 00:56:30.717 Jordan uh around the time when I was 13 14. I took my entire class to 00:56:30.800 --> 00:56:32.237 McDonald's. It was very expensive, but I 00:56:32.320 --> 00:56:35.677 was balling it all this money and I was 00:56:35.760 --> 00:56:39.677 showing off um and uh and so that was 00:56:39.760 --> 00:56:43.517 the first uh business that I uh created. 00:56:43.600 --> 00:56:46.397 And then when it came to and at the time 00:56:46.480 --> 00:56:48.877 I started kind of learning about AI, you 00:56:48.960 --> 00:56:53.757 know, reading sci-fi and all of that stuff. And when it came time to go to 00:56:53.840 --> 00:56:57.917 college, uh I didn't want to go to computer science because I felt like 00:56:58.000 --> 00:57:02.877 coding is on its way to get automated. I remember using these um wizards. Do you 00:57:02.960 --> 00:57:05.997 remember wizards? >> Yes. >> Wizards basically. It's like extremely 00:57:06.080 --> 00:57:09.197 crude early bots or that generate code. 00:57:09.280 --> 00:57:10.797 Yeah. >> Yeah. And I remember you could like, you 00:57:10.880 --> 00:57:13.837 know, type in a few things like here's my project, here's what it does, 00:57:13.920 --> 00:57:17.197 whatever, and then click click click and just like scaffold a lot of code. I was 00:57:17.280 --> 00:57:18.637 like, oh, I think that's the future. 00:57:18.720 --> 00:57:22.637 Like coding is such a >> it's almost >> yeah it's solved you know why why should 00:57:22.720 --> 00:57:27.197 I go into coding I was okay if AI can do the code what should I do well someone 00:57:27.280 --> 00:57:30.317 needs to build and maintain the computers and so I went to the computer 00:57:30.400 --> 00:57:32.477 engineering and and and did that for a 00:57:32.560 --> 00:57:35.757 while uh but then rediscovered my love 00:57:35.840 --> 00:57:38.397 for for programming uh reading program 00:57:38.480 --> 00:57:41.197 essays on lisp and things like that and 00:57:41.280 --> 00:57:45.597 uh started messing around with scheme and programming languages like that um 00:57:45.680 --> 00:57:49.277 but then I found it incredibly difficult to just like learn different programming 00:57:49.360 --> 00:57:53.277 languages. I didn't have a laptop at the time. And so every time I go to like 00:57:53.360 --> 00:57:56.077 wanting to learn Python or Java, I would 00:57:56.160 --> 00:57:58.717 go to the computer lab, download 00:57:58.800 --> 00:58:01.517 gigabytes of software, try to set it up, 00:58:01.600 --> 00:58:03.677 type a little bit of code, try to run 00:58:03.760 --> 00:58:06.797 it, you know, run into missing DL issue 00:58:06.880 --> 00:58:10.077 or and I was like, man, this is so 00:58:10.160 --> 00:58:13.037 primitive. Like at the time it was 2008 00:58:13.120 --> 00:58:16.157 something like that you know we had uh 00:58:16.240 --> 00:58:18.317 Google Docs, we had Gmail, you could 00:58:18.400 --> 00:58:21.357 like open the browser uh and partly 00:58:21.440 --> 00:58:23.917 thanks to you and be able to kind of uh 00:58:24.000 --> 00:58:28.397 use software on the internet and I thought the web is the ultimate software 00:58:28.480 --> 00:58:31.917 platform like everything should go on the web. Okay, who's building an online 00:58:32.000 --> 00:58:35.837 development environment, right? And and no one, right? And it felt like I w I 00:58:35.920 --> 00:58:40.637 found like $100 bill on the, you know, on the floor of Grand Association. Like 00:58:40.720 --> 00:58:44.237 surely someone should be building this, but no, no one was building this. And so 00:58:44.320 --> 00:58:46.557 I was like, okay, I'll I'll try to build it. 00:58:46.640 --> 00:58:51.437 >> And I got something done in like a couple hours. Uh, which was a text box. 00:58:51.520 --> 00:58:54.797 You type in some JavaScript. We And there's a there's a button that says 00:58:54.880 --> 00:58:57.597 eval. You click eval and evaluates. It 00:58:57.680 --> 00:58:59.757 shows you in a in an alert box, right? 00:58:59.840 --> 00:59:03.917 Right. So oneplus 1 2 I was like oh I have a programming environment. I showed 00:59:04.000 --> 00:59:07.197 it to my friends people started using it. I added a few additional things like 00:59:07.280 --> 00:59:11.037 saving the program. I was like okay all right this is there's there's a real 00:59:11.120 --> 00:59:15.277 idea here. People love it. And then again it took me two two or three years 00:59:15.360 --> 00:59:18.877 to actually be able to build anything because you know the browser can only 00:59:18.960 --> 00:59:21.117 run JavaScript. And it took a 00:59:21.200 --> 00:59:24.077 breakthrough at the time. Uh, Moisella 00:59:24.160 --> 00:59:26.397 had a research project called mcripton 00:59:26.480 --> 00:59:30.077 that allowed you to uh compile different 00:59:30.160 --> 00:59:32.237 uh programming languages like C, C++ 00:59:32.320 --> 00:59:35.597 into JavaScript. And for the browser to 00:59:35.680 --> 00:59:39.437 be able to run something like Python, I needed to compile C, Python to 00:59:39.520 --> 00:59:44.237 JavaScript. So I was the first to do it in the world. uh so built uh contributed 00:59:44.320 --> 00:59:45.677 to that project and built a lot of the 00:59:45.760 --> 00:59:49.357 scaffolding around it and we uh my 00:59:49.440 --> 00:59:53.917 friends and I compiled Python into JavaScript and I was like okay we did it 00:59:54.000 --> 00:59:58.077 for Python let's do it for Ruby let's do it for Lo and that's how the emergence 00:59:58.160 --> 01:00:01.837 of the idea for replet came is that when you need a ripple you should get it you 01:00:01.920 --> 01:00:05.437 should replet it and so ripple is is the most primitive programming environment 01:00:05.520 --> 01:00:08.877 possible so I added all these programming languages and again all this 01:00:08.960 --> 01:00:11.597 time my friends were using it and excited about 01:00:11.680 --> 01:00:15.997 And I was on GitHub at the time and just my standard thing is like when I make a 01:00:16.080 --> 01:00:19.677 piece of software is open source it. And so I was open sourcing all the things I 01:00:19.760 --> 01:00:23.517 was you know years building just like this underlying infrastructure to be 01:00:23.600 --> 01:00:25.757 able to just run code in the browser 01:00:25.840 --> 01:00:28.157 >> and then it went viral uh went viral in 01:00:28.240 --> 01:00:31.357 hacker news and it coincided with the 01:00:31.440 --> 01:00:34.397 MOO era. So massively online courses 01:00:34.480 --> 01:00:38.797 Udacity was coming online Corsera and and most famously Code Academy. Right. 01:00:38.880 --> 01:00:42.717 So, Code Academy uh was the first kind of website that allowed you to code in 01:00:42.800 --> 01:00:46.477 the browser interactively and learn how to code. And they built a lot of it on 01:00:46.560 --> 01:00:49.597 my software that I was open sourcing all the way from Jordan. And so, I remember 01:00:49.680 --> 01:00:52.237 seeing them on Hacker News and they were going super viral. I was like, "Hey, 01:00:52.320 --> 01:00:54.317 that's, you know, I I recognize this. 01:00:54.400 --> 01:00:56.797 What are you using?" And so, I left the Hacker News comments. I was like, "Oh, 01:00:56.880 --> 01:00:59.117 you're using my my open source package." 01:00:59.200 --> 01:01:02.077 And so, they reached out to me. They uh they're like, "Hey, would like to hire 01:01:02.160 --> 01:01:04.877 you." I was like, "I'm not interested. I I want to start a startup. I want to 01:01:04.960 --> 01:01:09.197 start this thing called Replet." and and they're like, "Well, no, you know, you 01:01:09.280 --> 01:01:12.717 should come work with us. We can we can do the same stuff." And I kept saying, 01:01:12.800 --> 01:01:16.157 "No." I was like, "Okay, I'll contract with you." They were paying me $12 an 01:01:16.240 --> 01:01:17.517 hour. I was really excited about it. 01:01:17.600 --> 01:01:21.357 Back from Oman. >> Um, but they came out to their to their 01:01:21.440 --> 01:01:24.397 credit. They came out of Jordan to recruit me and spend a few a few days 01:01:24.480 --> 01:01:28.157 there. And then I, you know, I kept saying no. And in the end, they gave me 01:01:28.240 --> 01:01:33.037 an offer I can't refuse. Um, and they got me an O1 visa. Came to the United States. 01:01:33.120 --> 01:01:37.037 >> That's when you moved. So when when was the first cuz you were born what year to 01:01:37.120 --> 01:01:39.357 >> 1987 >> 87. What was the first year that you 01:01:39.440 --> 01:01:41.917 could remember where you had the idea that you might not live your life in 01:01:42.000 --> 01:01:44.317 Jordan and that you might you might actually move to the US? 01:01:44.400 --> 01:01:46.557 >> Uh when I watched Pirates of Silicon Valley. 01:01:46.640 --> 01:01:47.837 >> Is that right? Okay. Got it. All right. 01:01:47.920 --> 01:01:52.157 >> Uh maybe 98 or 99. I don't know when it came out. 01:01:52.240 --> 01:01:53.917 >> Okay. That might be a good place to >> Yeah. 01:01:54.000 --> 01:01:56.397 >> Is it worth telling the hacker story because there's a version of the world 01:01:56.480 --> 01:01:59.677 where you didn't actually like if that changed maybe you wouldn't have gone to 01:01:59.760 --> 01:02:03.197 America. >> Right. Right. Yeah. So uh in in school I 01:02:03.280 --> 01:02:06.717 was programming the whole time you so I I just want to start businesses. I just 01:02:06.800 --> 01:02:09.997 like I'm exploding with ideas all the time. And like the reason Replet exists 01:02:10.080 --> 01:02:13.357 is because I have ideas all the time. I just want to go type on the computer and 01:02:13.440 --> 01:02:17.117 like build them. Um so I wasn't going to school. It was like incredibly boring 01:02:17.200 --> 01:02:21.117 for me. Uh and part of the reason why Replet has a mobile app today is because 01:02:21.200 --> 01:02:25.037 I always wanted to program under the desk just to do things. 01:02:25.120 --> 01:02:27.917 >> Um and so the at school they kept 01:02:28.000 --> 01:02:30.557 failing me uh for attendance. you know, 01:02:30.640 --> 01:02:35.037 so I would get A's, but I just didn't show up and so they they would fail me. 01:02:35.120 --> 01:02:37.517 And so I felt it was incredibly unfair. 01:02:37.600 --> 01:02:39.517 And all my friends were graduating now. 01:02:39.600 --> 01:02:43.997 This year was like 2011. I've been like for 6 years in college. It should be 01:02:44.080 --> 01:02:45.917 like a three or four year. And I was 01:02:46.000 --> 01:02:47.517 like incredibly depressed. I really 01:02:47.600 --> 01:02:51.037 wanted to be in Silicon Valley. And so I 01:02:51.120 --> 01:02:54.877 was like, "Oh, what if I changed my grades?" 01:02:54.960 --> 01:02:55.917 >> There we go. 01:02:56.000 --> 01:02:59.517 >> The university database. And um and and 01:02:59.600 --> 01:03:05.277 so I went into my parents uh uh basement 01:03:05.360 --> 01:03:10.797 uh and uh implemented uh the polyphasic sleep. Are you familiar with that? 01:03:10.880 --> 01:03:14.477 >> I I I I am >> uh Leonardo da Vinci's uh polyphysic 01:03:14.560 --> 01:03:18.077 sleep. I didn't hear from Ronaldo da Vinci. I heard it from Seinfeld cuz uh 01:03:18.160 --> 01:03:20.957 there's an episode where John Kamemer goes on on 01:03:21.040 --> 01:03:25.917 >> poly sleep what 20 minutes every four hours. 20 minutes every 24 hours. And 01:03:26.000 --> 01:03:27.997 yes, and this this somehow is going to work well. And it 01:03:28.080 --> 01:03:30.477 >> Yeah. And and hacking, if you've ever done anything, 01:03:30.560 --> 01:03:33.277 >> as the meme goes, this this has never worked for anybody else, but it might 01:03:33.360 --> 01:03:36.397 but it might work for me. >> Yes. 01:03:36.480 --> 01:03:40.557 >> And a lot of what hacking is is that you're you're coming up with ideas for 01:03:40.640 --> 01:03:43.597 like finding certain security holes and like writing a script and then running 01:03:43.680 --> 01:03:46.877 that script and that script will take like a 20 30 minutes to run and so 01:03:46.960 --> 01:03:50.317 you'll take that, you know, 20 30 minutes to sleep and go on. So I spent 01:03:50.400 --> 01:03:52.077 two weeks just going mad like trying to 01:03:52.160 --> 01:03:54.877 hack into the university database and uh 01:03:54.960 --> 01:03:58.477 finally I found um a way I found a SQL 01:03:58.560 --> 01:04:02.557 injection somewhere on the site uh and I found a way to like be able to edit the 01:04:02.640 --> 01:04:06.797 the records but I didn't want to risk it. So I went to my neighbor who's going 01:04:06.880 --> 01:04:10.637 to the same school. Uh I think till this day no one caught him. But I went to him 01:04:10.720 --> 01:04:15.357 and I said um hey uh I have this way to change grades like would you want to be 01:04:15.440 --> 01:04:18.397 my guinea pig? And I was honest about it. I was like I'm not going to do it. 01:04:18.480 --> 01:04:22.077 are you open to doing? He's like, "Yeah, yeah, yeah." They call 01:04:22.160 --> 01:04:24.637 his human trials. 01:04:24.720 --> 01:04:27.677 This is how medicine works. 01:04:27.760 --> 01:04:32.397 So, so we we went and and and uh we went and changed his grades and he he went 01:04:32.480 --> 01:04:36.957 and pulled his transcript and the you know, the update wasn't wasn't there and 01:04:37.040 --> 01:04:41.517 went back to the basement. Well, turned out that I had access to the uh slave 01:04:41.600 --> 01:04:43.997 database. I didn't have access to ambassador database. 01:04:44.080 --> 01:04:48.397 >> So, find a way through the network privilege escalation. It was an Oracle 01:04:48.480 --> 01:04:52.317 database that had a vulnerability and then found the real database and then I 01:04:52.400 --> 01:04:54.877 just, you know, did it for myself. Uh, 01:04:54.960 --> 01:04:58.557 changed the grades and went and pulled 01:04:58.640 --> 01:05:00.797 my scrcripts and sure enough it actually 01:05:00.880 --> 01:05:04.077 changed. Went and bought the the the 01:05:04.160 --> 01:05:07.197 gown, went to the graduation parties, 01:05:07.280 --> 01:05:10.637 uh, did all that. We're graduating. Um, 01:05:10.720 --> 01:05:13.357 and then one day I'm at home. It's like 01:05:13.440 --> 01:05:15.757 maybe 6:00 or 7:00 p.m. I get a you know 01:05:15.840 --> 01:05:19.677 the the telephone at home rings 01:05:19.760 --> 01:05:23.357 ominous ominous ring >> Santa 01:05:23.440 --> 01:05:27.357 um hello and he's like hey this is the university registration system and I 01:05:27.440 --> 01:05:31.277 knew the guy that run it. Uh he's like look you know we we we're having this 01:05:31.360 --> 01:05:36.157 problem. The system's been down all day and it keeps coming back to your record. 01:05:36.240 --> 01:05:38.157 there's an anomaly in your record where 01:05:38.240 --> 01:05:40.237 you're both pass you have a passing 01:05:40.320 --> 01:05:43.277 grade but you're also banned from that 01:05:43.360 --> 01:05:47.837 uh final exam of subject I was like oh well turns out the database is not 01:05:47.920 --> 01:05:50.157 normalized so typically that when they 01:05:50.240 --> 01:05:54.877 ban you from an exam the the grades resets to 35 out of 100 but apparently 01:05:54.960 --> 01:05:57.997 there's a boolean flag and by the way all the column names in the database are 01:05:58.080 --> 01:06:02.797 single single letters that was the hardest thing is security by obscurity >> right 01:06:02.880 --> 01:06:04.317 >> and turns out there's a flag that I 01:06:04.400 --> 01:06:06.237 didn't track so when when when when you 01:06:06.320 --> 01:06:09.917 go over attendance um uh when you don't 01:06:10.000 --> 01:06:13.677 attend and they they they want to fail you, they they ban you from the final 01:06:13.760 --> 01:06:18.557 exam. So, I changed the grades and that that that created uh an issue and 01:06:18.640 --> 01:06:22.877 brought down the system. So, they were calling me and I thought at the time I 01:06:22.960 --> 01:06:25.037 was like, you know, I could I could 01:06:25.120 --> 01:06:27.197 potentially lie and I'll it'll be a huge 01:06:27.280 --> 01:06:30.717 issue or I just like I'll just I'll just 01:06:30.800 --> 01:06:34.637 fess up. Yeah. So, I said, hey, listen, look, um yeah, I might know something 01:06:34.720 --> 01:06:38.957 about it. Hey, let me let me come uh tomorrow and kind of talk to you about 01:06:39.040 --> 01:06:43.037 what happened. So, I go in and I open the door and it's the deans of all the 01:06:43.120 --> 01:06:46.317 all the schools. It's a computer science computer. They were all working on it 01:06:46.400 --> 01:06:50.717 for like days because it's like it's like it's a very computerheavy, you 01:06:50.800 --> 01:06:53.197 know, university and it was like a problem 01:06:53.280 --> 01:06:55.517 >> and they're all kind of really intrigued 01:06:55.600 --> 01:06:59.597 about what happened. And so I pull up a whiteboard and started explaining what I 01:06:59.680 --> 01:07:03.517 did and and everyone was engaged. I gave them a lecture basically. your oral exam 01:07:03.600 --> 01:07:07.197 for your PhD. This is great. >> They were they were they were really 01:07:07.280 --> 01:07:11.277 excited and uh and I think I it was endearing to them. I was like, "Oh, wow. 01:07:11.360 --> 01:07:13.597 This is a this is a very interesting problem." 01:07:13.680 --> 01:07:15.997 >> Um and then I was like, "Okay, great. 01:07:16.080 --> 01:07:19.117 Thank you." And I was like, "Hey, wait, 01:07:19.200 --> 01:07:20.957 >> wait. We don't know what to do with you. 01:07:21.040 --> 01:07:24.397 Do we send you to jail? Do we 01:07:24.480 --> 01:07:28.317 >> And uh I was like, "Hey, we have to escalate to the university um uh 01:07:28.400 --> 01:07:30.717 president." and and he he was a great 01:07:30.800 --> 01:07:35.757 man and I think uh he gave me a second chance in life and I went to him and I 01:07:35.840 --> 01:07:39.117 uh you know I I explained the situation I said like I'm really frustrated. I 01:07:39.200 --> 01:07:42.797 need to graduate. I need to get on with my life. I've been here for six years 01:07:42.880 --> 01:07:45.837 and I just can't sit in in in school the 01:07:45.920 --> 01:07:50.157 stuff I already know. I'm a really good programmer. Uh and and he gave me a 01:07:50.240 --> 01:07:52.957 Spider-Man line at the time. was like with great power comes great 01:07:53.040 --> 01:07:55.357 responsibility and you have a great power and you know and it really 01:07:55.440 --> 01:08:00.637 affected me and I think he he was right at the moment and and so he said well 01:08:00.720 --> 01:08:03.597 we're going to let you go but you're going to have to help the system 01:08:03.680 --> 01:08:07.997 administrators secure the system >> uh for the summer I was like happy to do 01:08:08.080 --> 01:08:12.477 it and I show up and all the programmers there hate me hate my gut% 01:08:12.560 --> 01:08:15.917 >> and uh they they would lock me out like I would see them they would be outside I 01:08:16.000 --> 01:08:18.397 would knock on the door and nobody would listen it's like they don't want to let 01:08:18.480 --> 01:08:20.477 me in I try to help them a little bit 01:08:20.560 --> 01:08:25.677 they theyen't and collaborative and so I was like all right whatever. Uh and so 01:08:25.760 --> 01:08:27.117 it came time for me to actually 01:08:27.200 --> 01:08:30.397 graduate. It was the final project and 01:08:30.480 --> 01:08:34.237 one of the computer science dean came to me and he said look I I need to call a 01:08:34.320 --> 01:08:38.237 favor. I was a big part of the reason we kind of let you go and we didn't kind of 01:08:38.320 --> 01:08:43.357 prosecute you. Uh so I want you to work with me on the um on the final project 01:08:43.440 --> 01:08:47.037 and it's going to be around security and hacking. I was like no I'm I'm done with 01:08:47.120 --> 01:08:50.317 that like I just want to I just want to build programming environments 01:08:50.400 --> 01:08:54.317 and things like that. Uh and he's like no you have to do it. I was like okay. 01:08:54.400 --> 01:08:57.117 So I I thought I' do something more productive. So I wrote a security 01:08:57.200 --> 01:09:00.957 scanner uh that was very proud of that that kind of crawls the different side 01:09:01.040 --> 01:09:05.277 that tries to do SQL injection and all sorts of things. Um and actually my 01:09:05.360 --> 01:09:07.677 security scanner found another vulnerability in the system. 01:09:07.760 --> 01:09:10.157 >> Amazing. >> And so I went to the defense and he's 01:09:10.240 --> 01:09:14.157 like you need to run this security scanner live and show that there's a 01:09:14.240 --> 01:09:17.037 vulnerability. And I didn't understand what was going on at the time, but I 01:09:17.120 --> 01:09:21.757 just okay. So I gave the presentation about how the system works and I was 01:09:21.840 --> 01:09:24.557 like, oh, let's run it. And it showed that there's security vulnerability. 01:09:24.640 --> 01:09:28.077 Okay, let's get let's try to get a shell. So the system automatically runs 01:09:28.160 --> 01:09:29.997 all the security stuff and it gets you 01:09:30.080 --> 01:09:33.197 gets you a shell. And then the other 01:09:33.280 --> 01:09:38.157 dean that turned out he was giving the mandate to secure the system. And now I 01:09:38.240 --> 01:09:40.157 started to realize I'm a pawn in some 01:09:40.240 --> 01:09:44.557 kind of rivalry here. and and his his 01:09:44.640 --> 01:09:48.237 face turned red and he's like, "No, it's impossible. You know, we secured the 01:09:48.320 --> 01:09:53.277 system. You're lying." I was like, "You know, you're accusing me of lying." All 01:09:53.360 --> 01:09:57.997 right. What should we know? Should we know your um uh your salary or your 01:09:58.080 --> 01:10:01.037 password? What do you want me to look up? And I was like, "Yeah, I look up my 01:10:01.120 --> 01:10:04.957 password." So, I I look up his his password. Uh and it was like gibberish. 01:10:05.040 --> 01:10:07.357 It was encrypted. And I was like, "Oh, that's not my password. See, you're 01:10:07.440 --> 01:10:10.557 lying." I was like, "Well, there's a decrypt function that the programmers 01:10:10.640 --> 01:10:15.117 put in there." So I I do decrypt and it shows his password. It was something 01:10:15.200 --> 01:10:19.997 embarrassing. I forgot I forgot what it was. And so he gets up really angry, 01:10:20.080 --> 01:10:24.477 shakes my hand and leaves to change his password. Uh so that I I was able to 01:10:24.560 --> 01:10:26.557 hack into the university another time. 01:10:26.640 --> 01:10:29.997 Luckily I I was able to graduate, gave them the software, they secured the 01:10:30.080 --> 01:10:33.357 system. But um but yeah, later on I 01:10:33.440 --> 01:10:37.037 would realize that yeah, he wanted to embarrass the other guy, which was why I 01:10:37.120 --> 01:10:39.357 was in the middle. >> Politics. Well, I think the the moral 01:10:39.440 --> 01:10:42.157 the moral of the story is if if you can successfully hack into your school 01:10:42.240 --> 01:10:45.277 system and change your grade, you deserve the grade and you deserve to graduate. 01:10:45.360 --> 01:10:47.677 >> I I think so. >> And and and just for any for any parents 01:10:47.760 --> 01:10:50.477 out there just children out there, you can just you can see you can site you 01:10:50.560 --> 01:10:53.917 can sight site me as the moral you can site you can set out to me as the moral 01:10:54.000 --> 01:10:55.597 moral authority moral authority on this. 01:10:55.680 --> 01:10:58.477 One maybe lesson I think that is very 01:10:58.560 --> 01:11:02.717 relevant for the AI age. Uh I think that 01:11:02.800 --> 01:11:04.957 the traditional sort of more conformous 01:11:05.040 --> 01:11:08.477 path is paying less and less dividends 01:11:08.560 --> 01:11:10.717 and I think uh you know kids coming up 01:11:10.800 --> 01:11:13.117 today should use all the tools available 01:11:13.200 --> 01:11:15.837 to be able to discover and chart their 01:11:15.920 --> 01:11:18.557 own paths cuz I feel like just you know 01:11:18.640 --> 01:11:21.357 listening to the traditional advice and 01:11:21.440 --> 01:11:23.517 doing the same things that people have 01:11:23.600 --> 01:11:29.117 always done is just not as it's not working out as much as as we'd like. 01:11:29.200 --> 01:11:33.800 >> Thanks for coming on the podcast. Thank you, man. Fantastic. 01:11:41.280 --> 01:11:44.280 Wow. 01:11:45.040 --> 01:11:48.600 Wow. Wow.