WEBVTT 00:00:00.080 --> 00:00:03.837 The following is a conversation with Greg Brockman. He's the co-founder and 00:00:03.920 --> 00:00:06.637 CTO of OpenAI, a world-class research 00:00:06.720 --> 00:00:09.197 organization developing ideas in AI with 00:00:09.280 --> 00:00:12.237 a goal of eventually creating a safe and 00:00:12.320 --> 00:00:14.237 friendly artificial general 00:00:14.320 --> 00:00:16.957 intelligence. One that benefits and 00:00:17.040 --> 00:00:20.157 empowers humanity. OpenAI is not only a 00:00:20.240 --> 00:00:22.557 source of publications, algorithms, 00:00:22.640 --> 00:00:25.837 tools, and data sets. Their mission is a 00:00:25.920 --> 00:00:27.437 catalyst for an important public 00:00:27.520 --> 00:00:30.237 discourse about our future with both 00:00:30.320 --> 00:00:33.997 narrow and general intelligence systems. 00:00:34.080 --> 00:00:38.157 This conversation is part of the artificial intelligence podcast at MIT 00:00:38.240 --> 00:00:40.797 and beyond. If you enjoy it, subscribe 00:00:40.880 --> 00:00:43.437 on YouTube, iTunes, or simply connect 00:00:43.520 --> 00:00:45.677 with me on Twitter at Lex Freedman, 00:00:45.760 --> 00:00:49.357 spelled F R I D. And now, here's my 00:00:49.440 --> 00:00:53.627 conversation with Greg Brockman. 00:00:53.710 --> 00:00:55.730 >> [music] 00:01:01.570 --> 00:01:03.590 [music] 00:01:08.780 --> 00:01:11.757 [music] >> So in high school and right after you 00:01:11.840 --> 00:01:15.677 wrote a draft of a chemistry textbook, I saw that that covers everything from 00:01:15.760 --> 00:01:17.597 basic structure of the atom to quantum 00:01:17.680 --> 00:01:20.397 mechanics. So it's clear you have an 00:01:20.480 --> 00:01:23.277 intuition and a passion for both the uh 00:01:23.360 --> 00:01:25.837 the physical world with chemistry and 00:01:25.920 --> 00:01:29.357 now robotics to the digital world with 00:01:29.440 --> 00:01:33.517 uh AI, deep learning, reinforcement learning so on. Do you see the physical 00:01:33.600 --> 00:01:37.757 world and the digital world as different and what do you think is the gap? 00:01:37.840 --> 00:01:40.957 >> A lot of it actually boils down to iteration speed, right? That I think 00:01:41.040 --> 00:01:44.877 that a lot of what really motivates me is is building things, right? is the uh 00:01:44.960 --> 00:01:47.437 you know think about mathematics for example where you think really hard 00:01:47.520 --> 00:01:50.557 about a problem you understand it you write it down in this very obscure form 00:01:50.640 --> 00:01:55.197 that they call proof but then this is in humanity's library right it's there 00:01:55.280 --> 00:01:58.797 forever this is some truth that we've discovered and you know maybe only five 00:01:58.880 --> 00:02:01.277 people in your field will ever read it but somehow you've kind of moved 00:02:01.360 --> 00:02:04.957 humanity forward and so I actually used to really think that I was going to be a 00:02:05.040 --> 00:02:08.877 mathematician and uh then I actually started writing this chemistry textbook 00:02:08.960 --> 00:02:12.237 one of my friends told me you'll never publish it because you don't have a PhD 00:02:12.320 --> 00:02:16.477 so instead I I decided to build a website and tried to promote my ideas 00:02:16.560 --> 00:02:20.877 that way and then I discovered programming and uh I you know that in 00:02:20.960 --> 00:02:23.997 programming you think hard about a problem you understand it you write down 00:02:24.080 --> 00:02:28.237 in a very obscure form that we call a program but then once again it's in 00:02:28.320 --> 00:02:31.597 humanity's library right and anyone could get the benefit from it and the 00:02:31.680 --> 00:02:35.037 scalability is massive and so I think that the thing that really appeals to me 00:02:35.120 --> 00:02:39.037 about the digital world is that you can have this this this insane leverage 00:02:39.120 --> 00:02:43.517 right a single individual with an idea is able to affect the entire planet Um, 00:02:43.600 --> 00:02:46.397 and that's something I think is really hard to do if you're moving around 00:02:46.480 --> 00:02:50.397 physical atoms. >> But you said mathematics. So if you look 00:02:50.480 --> 00:02:53.277 at the the wet thing over here, our 00:02:53.360 --> 00:02:56.637 mind, do you ultimately see it as just 00:02:56.720 --> 00:03:01.677 math as just information processing or is there some other magic as you've seen 00:03:01.760 --> 00:03:04.717 if you've seen through biology and chemistry and so on? Yeah, I think it's 00:03:04.800 --> 00:03:08.317 really interesting to think about humans as just information processing systems 00:03:08.400 --> 00:03:12.237 and uh that it seems like it's actually a pretty good uh way of describing a lot 00:03:12.320 --> 00:03:16.157 of of kind of how the world works or a lot of what we're capable of to think 00:03:16.240 --> 00:03:20.077 that that you know again if you just look at at technological innovations 00:03:20.160 --> 00:03:23.677 over time that in some ways the most transformative innovation that we've had 00:03:23.760 --> 00:03:27.037 has been the computer right in some ways the internet you know that what has the 00:03:27.120 --> 00:03:30.397 internet done right the internet is not about these physical cables it's about 00:03:30.480 --> 00:03:33.597 the fact that I am suddenly able to instantly communicate with any other 00:03:33.680 --> 00:03:37.757 human on the planet. I'm able to retrieve any piece of knowledge that in 00:03:37.840 --> 00:03:42.397 some ways the human race has ever had and that those are these insane 00:03:42.480 --> 00:03:46.877 transformations. Do you see the our society as a whole, the collective as 00:03:46.960 --> 00:03:50.317 another extension of the intelligence of the human being? So if you look at the 00:03:50.400 --> 00:03:53.437 human being as information processing system, you mentioned the internet, the 00:03:53.520 --> 00:03:57.837 networking, do you see us all together as a civilization as a as a kind of 00:03:57.920 --> 00:04:00.957 intelligence system? Yeah, I think this is actually a really interesting 00:04:01.040 --> 00:04:04.237 perspective uh to take and to think about that you sort of have this 00:04:04.320 --> 00:04:08.077 collective intelligence of all of society. The economy itself is this 00:04:08.160 --> 00:04:10.797 superhuman machine that is optimizing 00:04:10.880 --> 00:04:15.197 something, right? And it's in some ways a company has a will of its own, right? 00:04:15.280 --> 00:04:17.837 That you have all these individuals who are all pursuing their own individual 00:04:17.920 --> 00:04:20.797 goals and thinking really hard and thinking about the right things to do, 00:04:20.880 --> 00:04:22.557 but somehow the company does something 00:04:22.640 --> 00:04:25.437 that is this emergent thing uh and that 00:04:25.520 --> 00:04:27.837 is is it's a really useful abstraction. 00:04:27.920 --> 00:04:31.357 And so I think that in some ways, you know, we think of of ourselves as the 00:04:31.440 --> 00:04:34.077 most intelligent things on the planet and the most powerful things on the 00:04:34.160 --> 00:04:37.197 planet, but there are things that are bigger than us that are these systems 00:04:37.280 --> 00:04:40.797 that we all contribute to. Um, and so I think actually, you know, it's a it's 00:04:40.880 --> 00:04:44.637 interesting to think about uh if you've read Isaac Aisimov's foundation, right, 00:04:44.720 --> 00:04:47.677 that uh that there's this concept of psycho history in there, which is 00:04:47.760 --> 00:04:51.117 effectively this that if you have trillions or quadrillions of of beings, 00:04:51.200 --> 00:04:54.877 then maybe you could actually predict what that being that that huge macro 00:04:54.960 --> 00:04:58.797 being will do. uh and uh almost independent of what the individuals 00:04:58.880 --> 00:05:01.997 want. And I actually have a sec a second angle on this that I think is 00:05:02.080 --> 00:05:05.997 interesting which is thinking about uh technological determinism. One thing 00:05:06.080 --> 00:05:09.517 that that I actually think a lot about with with open AI, right, is that we're 00:05:09.600 --> 00:05:13.677 kind of coming on onto this insanely transformational technology of of 00:05:13.760 --> 00:05:16.637 general intelligence, right? That will happen at some point. And there's a 00:05:16.720 --> 00:05:20.637 question of how can you take actions that will actually steer it to go better 00:05:20.720 --> 00:05:24.637 rather than worse. And that I think one question you need to ask is as a 00:05:24.720 --> 00:05:28.717 scientist, as an inventor, as a creator, what impact can you have in general, 00:05:28.800 --> 00:05:31.437 right? You look at things like the telephone invented by two people on the 00:05:31.520 --> 00:05:34.557 same day. Like what does that mean? Like what does that mean about the shape of 00:05:34.640 --> 00:05:37.677 innovation? And I think that what's going on is everyone's building on the 00:05:37.760 --> 00:05:41.277 shoulders of the same giants. And so you can kind of you can't really hope to 00:05:41.360 --> 00:05:42.877 create something no one else ever would. 00:05:42.960 --> 00:05:45.517 You know, if Einstein wasn't born, someone else would have come up with 00:05:45.600 --> 00:05:48.717 relativity. You know, he changed the timeline a bit, right? that maybe it 00:05:48.800 --> 00:05:51.517 would have taken another 20 years, but it wouldn't be that fundamentally 00:05:51.600 --> 00:05:54.717 humanity would never discover these these fundamental truths. 00:05:54.800 --> 00:05:57.037 >> So there's some kind of invisible 00:05:57.120 --> 00:05:59.997 momentum that some people like Einstein 00:06:00.080 --> 00:06:03.437 or OpenAI is plugging into uh that 00:06:03.520 --> 00:06:07.437 anybody else can also plug into and ultimately that wave takes us into a 00:06:07.520 --> 00:06:09.357 certain direction. That's what you mean by 00:06:09.440 --> 00:06:11.757 >> That's right. That's right. And you know this kind of seems to play out in a 00:06:11.840 --> 00:06:15.117 bunch of different ways. uh that there's some exponential that is being ridden 00:06:15.200 --> 00:06:18.557 and that the exponential itself which one it is changes you think about Moors 00:06:18.640 --> 00:06:23.197 law an entire industry set its clock to it for 50 years like how can that be 00:06:23.280 --> 00:06:28.077 right how is that possible and yet somehow it happened and so I think you 00:06:28.160 --> 00:06:31.277 can't hope to ever invent something that no one else will maybe you can change 00:06:31.360 --> 00:06:34.877 the timeline a little bit but if you really want to make a difference I think 00:06:34.960 --> 00:06:38.397 that the thing that you really have to do the only real degree of freedom you 00:06:38.480 --> 00:06:42.557 have is to set the initial conditions under which a technology is born And so 00:06:42.640 --> 00:06:43.837 you think about the internet, right? 00:06:43.920 --> 00:06:46.077 That there were lots of other competitors trying to build similar 00:06:46.160 --> 00:06:51.037 things and the internet won and that the initial conditions were that it was 00:06:51.120 --> 00:06:55.517 created by this group that really valued people being able to be uh you know 00:06:55.600 --> 00:06:58.637 anyone being able to plug in this very academic mindset of of being open and 00:06:58.720 --> 00:07:01.917 connected. Um and I think that the internet for the next 40 years really 00:07:02.000 --> 00:07:05.997 played out that way. um you know maybe today uh things are starting to to shift 00:07:06.080 --> 00:07:08.637 in in a different direction but I think that those initial conditions were 00:07:08.720 --> 00:07:12.637 really important to determine the next 40 years worth of of progress. That's 00:07:12.720 --> 00:07:16.477 really beautifully put. So another example of that I think about you know I 00:07:16.560 --> 00:07:21.117 recently looked at it I looked at Wikipedia the formation of Wikipedia and 00:07:21.200 --> 00:07:25.037 I wonder what the internet would be like if Wikipedia had ads. 00:07:25.120 --> 00:07:29.837 >> You know there's a interesting argument that uh why they chose not to make it 00:07:29.920 --> 00:07:34.317 put advertisement on Wikipedia. I think it's uh I think Wikipedia is one of the 00:07:34.400 --> 00:07:37.437 greatest resources we have on the internet. It's it's extremely surprising 00:07:37.520 --> 00:07:41.357 how well it works and how well it was able to aggregate all this kind of uh 00:07:41.440 --> 00:07:45.357 good information and the essentially the creator of Wikipedia. I don't know 00:07:45.440 --> 00:07:49.117 there's probably some debates there but set the initial conditions and now it 00:07:49.200 --> 00:07:52.717 carried itself forward. That's really interesting. So you're the way you're 00:07:52.800 --> 00:07:55.917 thinking about AGI or artificial intelligence is you're focused on 00:07:56.000 --> 00:07:58.477 setting the initial conditions for for the progress. 00:07:58.560 --> 00:08:01.837 >> That's right. >> That's powerful. Okay. So looking into 00:08:01.920 --> 00:08:05.357 the future, if you create an AGI system, 00:08:05.440 --> 00:08:07.437 like one that can ace the touring test, 00:08:07.520 --> 00:08:12.557 natural language, what do you think would be the interactions you would have 00:08:12.640 --> 00:08:15.437 with it? What do you think are the questions you would ask? Like what would 00:08:15.520 --> 00:08:19.117 be the first question you would ask it, her, him? 00:08:19.200 --> 00:08:22.797 >> That's right. I think that at that point, if you've really built a powerful 00:08:22.880 --> 00:08:26.317 system that is capable of shaping the future of humanity, the first question 00:08:26.400 --> 00:08:29.677 that you really should ask is how do we make sure that this plays out well? Um, 00:08:29.760 --> 00:08:34.877 and so that's actually the first question that I would ask a powerful AGI system is. 00:08:34.960 --> 00:08:38.397 >> So you wouldn't ask your colleague, you wouldn't ask like Ilia, you would ask 00:08:38.480 --> 00:08:40.957 the AGI system. >> Oh, we've already had the conversation 00:08:41.040 --> 00:08:45.197 with Ilia, right? And everyone here. Uh, and so you want as many perspectives and 00:08:45.280 --> 00:08:48.797 uh a piece of wisdom as you can for for answering this question. So I don't 00:08:48.880 --> 00:08:52.877 think you necessarily defer to whatever your powerful system tells you. Um, but 00:08:52.960 --> 00:08:57.597 you use it as one input uh to try to figure out what to do. But and I guess 00:08:57.680 --> 00:09:00.397 fun fundamentally what it really comes down to is if you built something really 00:09:00.480 --> 00:09:04.317 powerful and you think about think about for example the creation of uh of 00:09:04.400 --> 00:09:06.717 shortly after the creation of nuclear weapons right the most important 00:09:06.800 --> 00:09:10.717 question in the world was what's the world order going to be like how do we 00:09:10.800 --> 00:09:15.437 set ourselves up in a place where we're going to be able to survive as a species 00:09:15.520 --> 00:09:18.637 with AGI I think the question is slightly different right that there is a 00:09:18.720 --> 00:09:21.837 question of how do we make sure that we don't get the negative effects but 00:09:21.920 --> 00:09:25.757 there's also the positive side right you imagine that you know like like what 00:09:25.840 --> 00:09:29.677 will AGI be like like what will it be capable of? And I think that that one of 00:09:29.760 --> 00:09:33.437 the core reasons that an AGI can be powerful and transformative is actually 00:09:33.520 --> 00:09:35.677 due to uh technological development. 00:09:35.760 --> 00:09:37.117 Yeah. >> Right. If you have something that's 00:09:37.200 --> 00:09:42.237 capable as capable as a human and that it's much more scalable uh that you 00:09:42.320 --> 00:09:45.037 absolutely want that thing to go read the whole scientific literature and 00:09:45.120 --> 00:09:47.757 think about how to create cures for all the diseases, right? You want it to 00:09:47.840 --> 00:09:51.277 think about how to go and build uh technologies to help us create material 00:09:51.360 --> 00:09:55.677 abundance and to figure out societal problems that we have trouble with like 00:09:55.760 --> 00:09:58.557 how are we supposed to clean up the environment and you know maybe you want 00:09:58.640 --> 00:10:02.077 this uh to go and invent a bunch of little robots that will go out and uh be 00:10:02.160 --> 00:10:04.557 biodegradable and uh turn ocean debris 00:10:04.640 --> 00:10:08.077 into harmless uh molecules and um I 00:10:08.160 --> 00:10:12.877 think that that that positive side is something that I think people miss 00:10:12.960 --> 00:10:16.237 sometimes when thinking about what an AGI will be like. Um, and so I think 00:10:16.320 --> 00:10:19.837 that if you have a system that's capable of all of that, you absolutely want its 00:10:19.920 --> 00:10:24.957 advice about how do I make sure that we're using your uh your capabilities in 00:10:25.040 --> 00:10:27.517 a positive way for humanity. >> So what do you think about that 00:10:27.600 --> 00:10:30.797 psychology that uh looks at all the 00:10:30.880 --> 00:10:35.357 different possible trajectories of an AGI system, many of which perhaps the 00:10:35.440 --> 00:10:39.597 majority of which are positive and nevertheless focuses on the negative 00:10:39.680 --> 00:10:42.557 trajectories. I mean, you get to interact with folks, you get to think 00:10:42.640 --> 00:10:45.197 about this maybe within yourself as 00:10:45.280 --> 00:10:49.837 well. Um, you look at Sam Harris and so on. It seems to be, sorry to put it this 00:10:49.920 --> 00:10:53.037 way, but almost more fun to think about the negative 00:10:53.120 --> 00:10:56.877 >> possibilities, whatever that's deep in our psychology. 00:10:56.960 --> 00:11:00.797 What do you think about that? And how do we deal with it? Because we want AI to 00:11:00.880 --> 00:11:04.157 help us. >> So, I I think there's kind of two 00:11:04.240 --> 00:11:06.957 problems uh entailed in that question. 00:11:07.040 --> 00:11:11.757 Uh the first is more of the question of how can you even picture what a world 00:11:11.840 --> 00:11:13.757 with a new technology will be like? 00:11:13.840 --> 00:11:19.157 >> Now imagine we're in 1950 and I'm trying to describe Uber to someone. 00:11:19.240 --> 00:11:23.437 [laughter] >> Apps and the internet. Yeah. I mean 00:11:23.520 --> 00:11:27.437 you're that's that's going to be extremely complicated, but it's imaginable. 00:11:27.520 --> 00:11:31.917 >> It's it's imaginable, right? But and now imagine being in 1950 and predicting 00:11:32.000 --> 00:11:35.997 Uber, right? And you need to describe the internet. You need to describe GPS. 00:11:36.080 --> 00:11:39.357 you need to describe the fact that everyone's going to have this phone in 00:11:39.440 --> 00:11:43.037 their pocket. And so I think that that just the first 00:11:43.120 --> 00:11:46.957 truth is that it is hard to picture how a transformative technology will play 00:11:47.040 --> 00:11:50.797 out in the world. Um we've seen that before with technologies that are far 00:11:50.880 --> 00:11:52.797 less transformative than AGI will be. 00:11:52.880 --> 00:11:56.957 And so I think that that one piece is that it's just even hard to imagine and 00:11:57.040 --> 00:11:59.037 to really put yourself in a world where 00:11:59.120 --> 00:12:04.157 you can predict what that that positive vision would would be like. 00:12:04.240 --> 00:12:05.757 And you know I think the second thing is 00:12:05.840 --> 00:12:08.957 that it is I think it is always easier 00:12:09.040 --> 00:12:13.197 to support the negative side than the positive side. It's always easier to 00:12:13.280 --> 00:12:17.357 destroy than create and you know less in a in a in a in a 00:12:17.440 --> 00:12:20.877 physical sense and more just in a in an intellectual sense. Right. Because you 00:12:20.960 --> 00:12:23.837 know I think that that with creating something you need to just get a bunch 00:12:23.920 --> 00:12:27.197 of things right and to destroy you just need to get one thing wrong. 00:12:27.280 --> 00:12:29.037 >> Yeah. And so I think that that what that 00:12:29.120 --> 00:12:32.157 means is that I think a lot of people's thinking dead ends as soon as they see 00:12:32.240 --> 00:12:35.997 the negative story. But that being said, 00:12:36.080 --> 00:12:39.517 I actually actually have some hope, right? I think that that the that the 00:12:39.600 --> 00:12:43.997 positive vision is something that I think can be um is something that we can 00:12:44.080 --> 00:12:47.917 we can talk about. And I think that just simply saying this fact of yeah, like 00:12:48.000 --> 00:12:49.197 there's positive, there's negatives. 00:12:49.280 --> 00:12:50.797 Everyone likes to dwell on the negative. 00:12:50.880 --> 00:12:53.517 People actually respond well to that message and say, "Huh, you're right. 00:12:53.600 --> 00:12:57.037 There's a part of this that we're not talking about, not thinking about." And 00:12:57.120 --> 00:13:01.197 that's actually something that's that's that's I think really been a key part of 00:13:01.280 --> 00:13:03.917 how we think about AGI at OpenAI, right? 00:13:04.000 --> 00:13:07.357 You can kind of look at it as like okay like OpenAI talks about the fact that 00:13:07.440 --> 00:13:10.477 there are risks and yet they're trying to build this system. 00:13:10.560 --> 00:13:14.637 >> Like how do you square those those two facts? So, do you share the intuition 00:13:14.720 --> 00:13:17.437 that some people have uh I mean from Sam 00:13:17.520 --> 00:13:20.637 Harris to even Elon Musk himself that 00:13:20.720 --> 00:13:24.237 it's tricky as you develop AGI to keep 00:13:24.320 --> 00:13:27.277 it from slipping into the existential 00:13:27.360 --> 00:13:29.437 threats into the negative? What's your 00:13:29.520 --> 00:13:32.797 intuition about how hard is it to keep 00:13:32.880 --> 00:13:36.957 AI development on the positive track? 00:13:37.040 --> 00:13:39.197 What's your intuition there? >> To to answer that question, you can 00:13:39.280 --> 00:13:41.117 really look at how we structure Open AI. 00:13:41.200 --> 00:13:44.877 So we really have three main arms. So we have capabilities which is actually 00:13:44.960 --> 00:13:48.557 doing the technical work and pushing forward what these systems can do. 00:13:48.640 --> 00:13:53.437 There's safety which is working on technical mechanisms to ensure that the 00:13:53.520 --> 00:13:57.517 systems we build are aligned with human values. And then there's policy which is 00:13:57.600 --> 00:14:00.637 making sure that we have governance mechanisms answering that question of 00:14:00.720 --> 00:14:03.437 well whose values and so I think that 00:14:03.520 --> 00:14:07.757 the technical safety one is the one that people kind of talk about the most right 00:14:07.840 --> 00:14:11.757 you talk about like think about you know all of the dystopic AI movies a lot of 00:14:11.840 --> 00:14:16.237 that is about not having good technical safety in place um and what we've been 00:14:16.320 --> 00:14:19.117 finding is that you know I think that actually a lot of people look at the 00:14:19.200 --> 00:14:22.397 technical safety problem and think it's just intractable 00:14:22.480 --> 00:14:26.397 right this question of what do humans want how supposed to write that down? 00:14:26.480 --> 00:14:28.557 Can I even write down what I want? 00:14:28.640 --> 00:14:32.237 >> No way. And then they stop there. But 00:14:32.320 --> 00:14:36.237 the thing is, we've already built systems that are able to learn things 00:14:36.320 --> 00:14:40.317 that humans can't specify. You know, even the rules for how to recognize if 00:14:40.400 --> 00:14:42.237 there's a cat or a dog in an image. 00:14:42.320 --> 00:14:43.597 >> Turns out it's intractable to write that 00:14:43.680 --> 00:14:45.677 down. And yet, we're able to learn it. 00:14:45.760 --> 00:14:48.957 And that what we're seeing with systems we build at Open AAI, and they're still 00:14:49.040 --> 00:14:53.597 in early proof of concept stage, is that you are able to learn human preferences. 00:14:53.680 --> 00:14:57.117 you're able to learn what humans want from data. Um, and so that's kind of the 00:14:57.200 --> 00:15:00.717 core focus for our technical safety team. And I think that uh that they're 00:15:00.800 --> 00:15:03.597 actually we've had some pretty encouraging updates in terms of what 00:15:03.680 --> 00:15:07.517 we've been able to make work. >> So you have an intuition and a hope that 00:15:07.600 --> 00:15:09.837 from data, you know, looking at the 00:15:09.920 --> 00:15:11.997 value alignment problem. From data, we 00:15:12.080 --> 00:15:14.957 can build systems that align with the 00:15:15.040 --> 00:15:17.997 collective better angels of our nature. 00:15:18.080 --> 00:15:22.557 So align with the the ethics and the morals of human beings. to even say this 00:15:22.640 --> 00:15:26.077 in a different way. I mean, think about how how do we align humans, right? Think 00:15:26.160 --> 00:15:30.237 about like a human baby can grow up to be an evil person or a great person. And 00:15:30.320 --> 00:15:33.677 a lot of that is from learning from data, right? That you have some feedback 00:15:33.760 --> 00:15:37.037 as a child is growing up. They get to see positive examples. And so I think 00:15:37.120 --> 00:15:40.077 that that just like that the the only 00:15:40.160 --> 00:15:44.077 example we have of a general intelligence uh that is able to learn 00:15:44.160 --> 00:15:46.957 from data uh to align with human values 00:15:47.040 --> 00:15:51.357 and to learn values. um I think we shouldn't be surprised that we can do 00:15:51.440 --> 00:15:54.877 the same sorts of of of techniques or whether the same sort of techniques end 00:15:54.960 --> 00:15:58.317 up being how we we we solve value alignment for AGIS. 00:15:58.400 --> 00:16:00.157 >> So let's go even uh higher. I don't know 00:16:00.240 --> 00:16:02.077 if you've read the book Sapiens 00:16:02.160 --> 00:16:05.437 >> but uh there's an idea that you know um 00:16:05.520 --> 00:16:07.277 that as a collective as us human beings 00:16:07.360 --> 00:16:10.957 we kind of develop together an ideas 00:16:11.040 --> 00:16:15.997 that we hold. There's no in that context objective truth. we just kind of all 00:16:16.080 --> 00:16:19.517 agree to certain ideas and hold them as a collective. So do you have a sense 00:16:19.600 --> 00:16:22.797 that there is >> in the world of good and evil, do you 00:16:22.880 --> 00:16:25.917 have a sense that to the first approximation there are some things that 00:16:26.000 --> 00:16:28.957 are good and that we could teach systems 00:16:29.040 --> 00:16:33.837 to behave to be good? >> So I think that that this actually 00:16:33.920 --> 00:16:38.637 blends into our third team, right, which is the policy team. And this is the one 00:16:38.720 --> 00:16:42.477 the aspect I think people really talk about way less than they should, right? 00:16:42.560 --> 00:16:46.077 Because imagine that we build super powerful systems that we've managed to 00:16:46.160 --> 00:16:49.197 figure out all the mechanisms for these things to do whatever the operator 00:16:49.280 --> 00:16:53.517 wants. The most important question becomes who's the operator? What do they 00:16:53.600 --> 00:16:57.517 want? And how is that going to affect everyone else? Right? And and I think 00:16:57.600 --> 00:17:02.557 that this question of what is good? What are those values? I mean I think you 00:17:02.640 --> 00:17:06.397 don't even have to go to those those very grand existential places to start 00:17:06.480 --> 00:17:08.157 to to realize how hard this problem is. 00:17:08.240 --> 00:17:12.637 you just look at different countries and cultures across the world and that 00:17:12.720 --> 00:17:16.237 there's there's a very different conception of how the world works and 00:17:16.320 --> 00:17:19.117 you know what what what kinds of of ways 00:17:19.200 --> 00:17:21.597 that society wants to operate. And so I 00:17:21.680 --> 00:17:26.157 think that that the the really core question is is is actually very 00:17:26.240 --> 00:17:29.757 concrete. Um and I think it's not a question that we have ready answers to, 00:17:29.840 --> 00:17:32.877 right? is how do you have a world where 00:17:32.960 --> 00:17:37.517 all the different countries that we have, United States, China, Russia and 00:17:37.600 --> 00:17:39.117 you know the the the hundreds of other 00:17:39.200 --> 00:17:41.757 countries out there are able to continue 00:17:41.840 --> 00:17:44.797 to not just operate in the way that that 00:17:44.880 --> 00:17:50.077 they that they see fit but in in a that the world that emerges um in these where 00:17:50.160 --> 00:17:54.957 where you have these very powerful systems uh operating alongside humans 00:17:55.040 --> 00:17:58.797 ends up being something that empowers humans more that makes like ex human 00:17:58.880 --> 00:18:03.677 existence be a more meaningful thing and uh that people are happier and wealthier 00:18:03.760 --> 00:18:07.437 and uh able to live more fulfilling lives. It's not an obvious thing for how 00:18:07.520 --> 00:18:12.157 to design that world once you have that very powerful system. So if we take a 00:18:12.240 --> 00:18:14.317 little step back and we're having like a 00:18:14.400 --> 00:18:19.437 fascinating conversation and open in many ways a tech leader in the world and 00:18:19.520 --> 00:18:23.117 yet we're thinking about these big existential questions which is 00:18:23.200 --> 00:18:26.717 fascinating really important. I think you're a leader in that space and that's 00:18:26.800 --> 00:18:30.877 a really important space of just thinking how AI affects society in in a 00:18:30.960 --> 00:18:35.277 big picture view. So Oscar Wild said we're all in the gutter but some of us 00:18:35.360 --> 00:18:37.037 are looking at the stars and I think 00:18:37.120 --> 00:18:40.317 open AI has a charter that looks to the 00:18:40.400 --> 00:18:44.557 stars I would say to create intelligence to create general intelligence make it 00:18:44.640 --> 00:18:46.877 beneficial safe and collaborative. So 00:18:46.960 --> 00:18:49.037 can you tell me 00:18:49.120 --> 00:18:51.837 uh how that came about? How a mission 00:18:51.920 --> 00:18:54.237 like that and the path to creating a 00:18:54.320 --> 00:18:56.397 mission like that at OpenAI was founded? 00:18:56.480 --> 00:19:01.277 >> Yeah. So I think that in some ways it really boils down to taking a look at 00:19:01.360 --> 00:19:04.957 the landscape, right? So if you think about the history of AI that basically 00:19:05.040 --> 00:19:09.597 for the past 60 or 70 years, people have thought about this goal of what could 00:19:09.680 --> 00:19:12.957 happen if you could automate human intellectual labor, 00:19:13.040 --> 00:19:17.197 >> right? Imagine you can build a computer system that could do that. What becomes 00:19:17.280 --> 00:19:21.197 possible? We have a lot of sci-fi that tells stories of various dystopias and 00:19:21.280 --> 00:19:23.357 you know increasingly you have movies like herd that tell you a little bit 00:19:23.440 --> 00:19:25.357 about maybe more of a little bit utopic 00:19:25.440 --> 00:19:29.037 vision. Uh you think about the impact 00:19:29.120 --> 00:19:31.997 that we've seen from being able to uh 00:19:32.080 --> 00:19:36.477 have bicycles for our minds in computers. uh and that I think that the 00:19:36.560 --> 00:19:38.637 the impact of of computers in the 00:19:38.720 --> 00:19:43.517 internet has just far outstripped what anyone really could have predicted. And 00:19:43.600 --> 00:19:47.677 so I think that it's very clear that if you can build an AGI, it will be the 00:19:47.760 --> 00:19:51.757 most transformative technology that humans will ever create. 00:19:51.840 --> 00:19:56.157 And so what it boils down to then is a question of well is there a path? Is 00:19:56.240 --> 00:19:57.917 there hope? Is there a way to build such 00:19:58.000 --> 00:20:00.397 a system? And I think that for 60 or 70 00:20:00.480 --> 00:20:03.437 years that people got excited and uh uh 00:20:03.520 --> 00:20:07.517 that you know ended up not being able to deliver on the hopes that that that 00:20:07.600 --> 00:20:11.437 people had pinned on them. And I think that then you know that after you know 00:20:11.520 --> 00:20:15.597 two two winters of AI development uh that people uh you know I think kind of 00:20:15.680 --> 00:20:19.437 almost stopped daring to dream right that really talking about AGI or 00:20:19.520 --> 00:20:23.837 thinking about AGI became almost this taboo in the community. 00:20:23.920 --> 00:20:27.677 >> But I actually think that people took the wrong lesson from AI history. And if 00:20:27.760 --> 00:20:31.997 you look back starting in 1959 is when the perceptron was released. And this is 00:20:32.080 --> 00:20:35.677 basically you know one of the earliest neural networks. Um it was released to 00:20:35.760 --> 00:20:39.037 what was perceived as this massive overhype. So in the New York Times in 00:20:39.120 --> 00:20:41.357 1959 you have this article uh saying 00:20:41.440 --> 00:20:46.157 that you know the the perceptron will one day recognize people call out their 00:20:46.240 --> 00:20:50.557 names instantly translate speech between languages. And people at the time looked 00:20:50.640 --> 00:20:53.517 at this and said this is like your system can't do any of that. and 00:20:53.600 --> 00:20:56.637 basically spent 10 years trying to discredit the whole perceptron direction 00:20:56.720 --> 00:21:01.037 and and succeeded and all the funding dried up and you know people kind of uh 00:21:01.120 --> 00:21:04.397 went in other directions and you know in the 80s there was this resurgence and 00:21:04.480 --> 00:21:08.237 I'd always heard that the resurgence in the 80s was due to the invention of back 00:21:08.320 --> 00:21:11.517 propagation and these these algorithms that got people excited but actually the 00:21:11.600 --> 00:21:15.357 causality was due to people building larger computers that you can find these 00:21:15.440 --> 00:21:18.957 these articles from the 80s saying that the democratization of computing power 00:21:19.040 --> 00:21:21.917 suddenly meant that you could run these larger neural networks and then people 00:21:22.000 --> 00:21:23.517 started to do all these amazing things. 00:21:23.600 --> 00:21:26.397 The back propagation algorithm was invented and you know that the the 00:21:26.480 --> 00:21:29.277 neural nets people were running were these tiny little like 20 neuron neural 00:21:29.360 --> 00:21:32.077 nets, right? >> Right. Like what are you supposed to learn with 20 neurons? 00:21:32.160 --> 00:21:34.877 >> Yeah. >> And so of course they weren't able to 00:21:34.960 --> 00:21:39.517 get great results. And it really wasn't until 2012 that this approach that's 00:21:39.600 --> 00:21:44.637 almost the most simple natural approach that people had come up with in the 50s, 00:21:44.720 --> 00:21:47.757 right? In some ways even in the 40s before there were computers with the 00:21:47.840 --> 00:21:50.477 pits neuron. 00:21:50.560 --> 00:21:55.117 Suddenly this became the best way of solving problems right and I think there 00:21:55.200 --> 00:22:00.237 are three core properties that deep learning has that I think are very worth 00:22:00.320 --> 00:22:01.917 paying attention to the first is 00:22:02.000 --> 00:22:04.077 generality we have a very small number 00:22:04.160 --> 00:22:07.277 of deep learning tools SGD deep neural 00:22:07.360 --> 00:22:10.637 net maybe some some you know RL and it 00:22:10.720 --> 00:22:12.877 solves this huge variety of problems 00:22:12.960 --> 00:22:14.557 >> speech recognition machine translation 00:22:14.640 --> 00:22:17.757 game playing all these problems small 00:22:17.840 --> 00:22:21.597 set of tools so there's the generality There's a second piece which is the 00:22:21.680 --> 00:22:25.357 competence. You want to solve any of those problems, throw out 40 years worth 00:22:25.440 --> 00:22:29.197 of normal computer vision research, replace it with a deep neural net. It's 00:22:29.280 --> 00:22:33.277 going to work better. And there's a third piece which is the scalability, 00:22:33.360 --> 00:22:37.037 right? That one thing that has been shown time and time again is that you if 00:22:37.120 --> 00:22:40.797 you have a larger neural network, throw more comput, more data at it, it will 00:22:40.880 --> 00:22:43.517 work better. Those three properties 00:22:43.600 --> 00:22:48.477 together feel like essential parts of building a general intelligence. Now it 00:22:48.560 --> 00:22:52.237 doesn't just mean that if we scale up what we have that we will have an AGI 00:22:52.320 --> 00:22:55.837 right there are clearly missing pieces there are missing ideas we need to have 00:22:55.920 --> 00:22:58.637 answers for reasoning but I think that 00:22:58.720 --> 00:23:02.077 the core here is that for the first time 00:23:02.160 --> 00:23:06.237 it feels that we have a paradigm that gives us hope that general intelligence 00:23:06.320 --> 00:23:10.397 can be achievable and so as soon as you believe that everything else becomes 00:23:10.480 --> 00:23:14.397 comes into focus right if you imagine that you may be able to and you know 00:23:14.480 --> 00:23:18.157 that the timeline I think remains uncertain um that but I think that that 00:23:18.240 --> 00:23:21.357 you know certainly within our lifetimes and possibly within a a much shorter 00:23:21.440 --> 00:23:25.517 period of time than than people would expect if you can really build the most 00:23:25.600 --> 00:23:29.037 transformative technology that will ever exist, you stop thinking about yourself 00:23:29.120 --> 00:23:32.717 so much, right? You start thinking about just like how do you have a world where 00:23:32.800 --> 00:23:35.757 this goes well and that you need to think about the practicalities of how do 00:23:35.840 --> 00:23:39.437 you build an organization and get together a bunch of people and resources 00:23:39.520 --> 00:23:41.837 um and to make sure that people feel uh 00:23:41.920 --> 00:23:45.597 motivated and ready to to do it. 00:23:45.680 --> 00:23:49.917 But I think that then you start thinking about well what if we succeed um and how 00:23:50.000 --> 00:23:52.797 do we make sure that when we succeed that the world is actually the place 00:23:52.880 --> 00:23:56.397 that that that we want ourselves to exist then and you know almost in the 00:23:56.480 --> 00:24:00.237 the Rousey and Bale sense of the word and so that's kind of the the the 00:24:00.320 --> 00:24:02.237 broader landscape and OpenAI was really 00:24:02.320 --> 00:24:05.917 formed in 2015 with that high level 00:24:06.000 --> 00:24:09.117 picture of AGI might be possible sooner 00:24:09.200 --> 00:24:13.677 than people think and that uh we need to try to do our best to make sure it's 00:24:13.760 --> 00:24:17.197 going to go well and then We spent the next couple years really trying to 00:24:17.280 --> 00:24:20.317 figure out what does that mean? How do we do it? Um, and you know, I think that 00:24:20.400 --> 00:24:25.357 typically with a company, you start out very small. It's you and a co-founder 00:24:25.440 --> 00:24:28.397 and you build a product, you get some users, you get product market fit, you 00:24:28.480 --> 00:24:32.557 know, then at some point you raise some money, you hire people, you scale, and 00:24:32.640 --> 00:24:35.517 then uh, you know, down the road then the big companies realize you exist and 00:24:35.600 --> 00:24:38.077 try to kill you. Um, and for OpenAI it 00:24:38.160 --> 00:24:42.877 was basically everything in exactly the opposite order. Uh, [laughter] 00:24:42.960 --> 00:24:46.397 >> let me just pause for a second. and you said a lot of things and let me just 00:24:46.480 --> 00:24:49.277 admire the jarring aspect of what open 00:24:49.360 --> 00:24:53.597 AAI stands for which is uh daring to dream. I mean you said it it's pretty 00:24:53.680 --> 00:24:55.597 powerful. It caught me off guard because 00:24:55.680 --> 00:24:59.757 I think that's very true. The the the 00:24:59.840 --> 00:25:03.917 step of just daring to dream about the possibilities of creating intelligence 00:25:04.000 --> 00:25:06.397 in a positive in a safe way. But just 00:25:06.480 --> 00:25:09.357 even creating intelligence is a a much 00:25:09.440 --> 00:25:12.717 needed refreshing 00:25:12.800 --> 00:25:14.877 uh catalyst for the AI community. So 00:25:14.960 --> 00:25:16.397 that's that's the starting point. Okay. 00:25:16.480 --> 00:25:19.837 So then formation of open AI uh what's 00:25:19.920 --> 00:25:21.197 >> just I would just say that you know when 00:25:21.280 --> 00:25:25.437 we were starting open AI uh that kind of the first question that we had is is it 00:25:25.520 --> 00:25:29.197 too late to start a lab with a bunch of the best people 00:25:29.280 --> 00:25:30.477 >> right? Is that even possible? 00:25:30.560 --> 00:25:33.037 >> That was an actual question. That was that was really that was that was the 00:25:33.120 --> 00:25:36.717 core question of uh of you know we had this dinner in July of 20 2015 and 00:25:36.800 --> 00:25:39.677 there's that was that was really what we spent the whole time talking about 00:25:39.760 --> 00:25:42.717 >> and uh you know cuz it's you think about 00:25:42.800 --> 00:25:44.797 kind of where AI was is that it had 00:25:44.880 --> 00:25:49.757 transitioned from being an academic pursuit to an industrial pursuit and so 00:25:49.840 --> 00:25:53.437 a lot of the best people were in these big research labs and that we wanted to 00:25:53.520 --> 00:25:56.877 start our own one that you know no matter how much resources we could 00:25:56.960 --> 00:26:00.877 accumulate would be you know pale in comparison to the big tech companies and 00:26:00.960 --> 00:26:04.957 we knew at and there's a question of are we going to be actually able to get this 00:26:05.040 --> 00:26:08.157 thing off the ground? You need critical mass. You can't just do you and a 00:26:08.240 --> 00:26:11.357 co-founder build a product, right? You really need to have a group of, you 00:26:11.440 --> 00:26:15.837 know, five to 10 people. And uh >> we kind of concluded it wasn't obviously 00:26:15.920 --> 00:26:19.517 impossible. Uh so it seemed worth trying. 00:26:19.600 --> 00:26:22.637 >> Well, you're also dreamers, so who knows, right? That's right. 00:26:22.720 --> 00:26:25.517 >> Okay. So, so speaking of that competing 00:26:25.600 --> 00:26:28.717 with with the with the big players, um 00:26:28.800 --> 00:26:30.557 let's talk about some of the some of the 00:26:30.640 --> 00:26:32.797 tricky things as you think through this 00:26:32.880 --> 00:26:36.237 process of growing of uh seeing how you 00:26:36.320 --> 00:26:38.557 can develop these systems at a scale 00:26:38.640 --> 00:26:41.117 that competes. So, you recent recently 00:26:41.200 --> 00:26:45.677 formed OpenAI LP, a new cap profit 00:26:45.760 --> 00:26:47.597 company that now carries the name 00:26:47.680 --> 00:26:49.597 OpenAI. So, Open AI is now this official 00:26:49.680 --> 00:26:52.877 company. The original nonprofit company 00:26:52.960 --> 00:26:55.517 still exists and carries the OpenAI 00:26:55.600 --> 00:26:58.077 nonprofit name. So can you explain what 00:26:58.160 --> 00:27:00.237 this company is, what the purpose of its 00:27:00.320 --> 00:27:05.117 creation is, and how did you arrive at the decision? 00:27:05.200 --> 00:27:09.277 >> Yep. >> To create it. >> OpenAI, the whole entity and OpenAI LP 00:27:09.360 --> 00:27:14.077 as a vehicle uh is trying to accomplish the mission of ensuring that artificial 00:27:14.160 --> 00:27:15.997 general intelligence benefits everyone. 00:27:16.080 --> 00:27:19.037 And the main way that we're trying to do that is by actually trying to build 00:27:19.120 --> 00:27:22.317 general intelligence ourselves and make sure the benefits are distributed to the 00:27:22.400 --> 00:27:26.557 world. That's the primary way. We're also fine if someone else does this, 00:27:26.640 --> 00:27:30.317 right? Doesn't have to be us. If someone else is going to build an AGI and make 00:27:30.400 --> 00:27:33.597 sure that the benefits don't get locked up in one company or, you know, one one 00:27:33.680 --> 00:27:36.717 one with one set of people, um, like 00:27:36.800 --> 00:27:39.277 we're actually fine with that. And so 00:27:39.360 --> 00:27:42.637 those ideas are baked into our charter 00:27:42.720 --> 00:27:46.477 which is kind of the the the foundational document that are that 00:27:46.560 --> 00:27:48.157 describes kind of our values and and how 00:27:48.240 --> 00:27:53.517 how we operate. Um but it's also really baked into the structure of open ILP. 00:27:53.600 --> 00:27:54.717 And so the way that we've set up open 00:27:54.800 --> 00:27:59.117 ILP is that in the case where we succeed 00:27:59.200 --> 00:28:02.637 right if we actually build what we're trying to build 00:28:02.720 --> 00:28:06.877 then investors are able to get a return um and but that return is something that 00:28:06.960 --> 00:28:10.637 is capped. And so if you think of AGI in terms of the the value that you could 00:28:10.720 --> 00:28:13.837 really create, you're talking about the most transformative technology ever 00:28:13.920 --> 00:28:15.677 created. It's going to create orders of 00:28:15.760 --> 00:28:18.157 magnitude more value than any existing 00:28:18.240 --> 00:28:21.357 company. And that all of that value will 00:28:21.440 --> 00:28:23.757 be owned by the world like legally 00:28:23.840 --> 00:28:28.637 titled to the nonprofit to fulfill that mission. And so that's that's the 00:28:28.720 --> 00:28:32.637 structure. >> So the mission is a powerful one and 00:28:32.720 --> 00:28:34.877 it's um it's one that I think most 00:28:34.960 --> 00:28:37.517 people would agree with. It's how we 00:28:37.600 --> 00:28:42.317 would hope AI progresses. And so, how do you tie yourself to that 00:28:42.400 --> 00:28:44.397 mission? How do you make sure you do not 00:28:44.480 --> 00:28:48.557 deviate from that mission that um you 00:28:48.640 --> 00:28:53.597 know, other incentives that are profit driven would don't interfere with the 00:28:53.680 --> 00:28:55.837 mission. >> So, this was actually a really core 00:28:55.920 --> 00:28:59.437 question for us uh for the past couple years because, you know, I'd say that 00:28:59.520 --> 00:29:02.637 like the way that our history went was that for the first year we were getting 00:29:02.720 --> 00:29:06.477 off the ground, right? we had this highle picture uh but we didn't know 00:29:06.560 --> 00:29:11.117 exactly how we wanted to accomplish it >> and uh really two years ago is when we 00:29:11.200 --> 00:29:14.477 first started realizing in order to build AGI we're just going to need need 00:29:14.560 --> 00:29:18.157 to raise way more money uh than we can as a nonprofit um and you know we're 00:29:18.240 --> 00:29:23.357 talking many billions of dollars >> and so the first question is how are you 00:29:23.440 --> 00:29:26.717 supposed to do that and stay true to this mission and we looked at every 00:29:26.800 --> 00:29:29.517 legal structure out there and concluded none of them were quite right for what 00:29:29.600 --> 00:29:32.317 we wanted to do and I guess it shouldn't be too surprising if you're going to do 00:29:32.400 --> 00:29:35.277 some like crazy unpreced technology that you're going to have to come up with 00:29:35.360 --> 00:29:36.717 some crazy unprecedented structure to do 00:29:36.800 --> 00:29:39.597 it in. And uh a lot of a lot of our 00:29:39.680 --> 00:29:44.237 conversation was uh with people at OpenAI, right? The people who really 00:29:44.320 --> 00:29:47.917 join because they believe so much in this mission and thinking about how do 00:29:48.000 --> 00:29:52.637 we actually raise the resources to do it and also stay true to to what we stand 00:29:52.720 --> 00:29:55.997 for. And the place you got to start is to really align on what is it that we 00:29:56.080 --> 00:29:57.837 stand for, right? What are those values? 00:29:57.920 --> 00:30:01.117 What's really important to us? And so I'd say that we spent about a year 00:30:01.200 --> 00:30:05.517 really uh compiling the opening charter and that that determines and if you even 00:30:05.600 --> 00:30:08.557 look at the first the first line item in there, it says that look, we expect 00:30:08.640 --> 00:30:12.077 we're going to have to marshall huge amounts of resources, but we're going to 00:30:12.160 --> 00:30:14.877 make sure that we minimize conflicts of interest with the mission. 00:30:14.960 --> 00:30:17.037 >> Um and that kind of aligning on all of 00:30:17.120 --> 00:30:19.757 those pieces was the most important step 00:30:19.840 --> 00:30:22.557 towards figuring out how do we structure 00:30:22.640 --> 00:30:24.797 a company that can actually raise the 00:30:24.880 --> 00:30:27.837 resources to do what we need to do. I 00:30:27.920 --> 00:30:30.637 imagine open AAI uh the decision to 00:30:30.720 --> 00:30:33.117 create open ILP was a really difficult 00:30:33.200 --> 00:30:35.117 one and there was a lot of discussions 00:30:35.200 --> 00:30:38.397 as you mentioned for a year and uh there 00:30:38.480 --> 00:30:41.117 was different ideas perhaps detractors 00:30:41.200 --> 00:30:46.477 within open AI uh sort of uh different paths that you could have taken what 00:30:46.560 --> 00:30:49.837 were those concerns what were the different paths considered what was that 00:30:49.920 --> 00:30:53.197 process of making that decision like >> yep um but so if you look actually at 00:30:53.280 --> 00:30:55.837 the openi charter that there's almost 00:30:55.920 --> 00:30:58.237 two paths embeded within it 00:30:58.320 --> 00:31:03.037 >> there is we are primarily trying to build AGI ourselves but we're also okay 00:31:03.120 --> 00:31:06.317 if someone else does it and this is a weird thing for a company 00:31:06.400 --> 00:31:10.477 >> that's really interesting actually >> there there is an element of competition 00:31:10.560 --> 00:31:15.357 that you do want to be the one that does it but at the same time you're okay if 00:31:15.440 --> 00:31:18.317 somebody else does it and we'll talk about that a little bit that tradeoff 00:31:18.400 --> 00:31:20.157 that's that dance that's really interesting 00:31:20.240 --> 00:31:23.997 >> and I think this was the core tension as we were designing OpenAI LP and really 00:31:24.080 --> 00:31:26.717 the OpenAI strategy is how do you make 00:31:26.800 --> 00:31:28.797 that both you have a shot at being a 00:31:28.880 --> 00:31:31.437 primary actor which really requires 00:31:31.520 --> 00:31:36.397 building an organization raising massive resources and really having the will to 00:31:36.480 --> 00:31:39.837 go and execute on some really really hard vision right you need to really 00:31:39.920 --> 00:31:41.997 sign up for a long period to go and take 00:31:42.080 --> 00:31:44.397 on a lot of pain and a lot of risk. Um 00:31:44.480 --> 00:31:49.437 and to do that normally you just import the startup mindset right and that you 00:31:49.520 --> 00:31:52.397 think about okay like how do we out execute everyone you you have this very 00:31:52.480 --> 00:31:56.237 competitive angle but you also have this second angle of saying that well the 00:31:56.320 --> 00:32:00.717 true mission isn't for open AI to build AGI the true mission is for AGI to go 00:32:00.800 --> 00:32:04.397 well for humanity >> and so how do you take all of those 00:32:04.480 --> 00:32:09.117 first actions and make sure you don't close the door on outcomes that would 00:32:09.200 --> 00:32:12.877 actually be positive and and fulfill the mission and so I think it's a very 00:32:12.960 --> 00:32:17.037 delicate balance Right. And I think that going 100% one direction or the other is 00:32:17.120 --> 00:32:20.797 clearly not the correct answer. And so I think that even in terms of just how we 00:32:20.880 --> 00:32:24.637 talk about OpenAI and think about it, there's just like like one thing that's 00:32:24.720 --> 00:32:29.117 always in the back of my mind is to make sure that we're not just saying OpenAI's 00:32:29.200 --> 00:32:32.877 goal is to build AGI, right? That it's actually much broader than that, right? 00:32:32.960 --> 00:32:37.037 That first of all, you know, it's not just AGI, it's safe AGI. That's very 00:32:37.120 --> 00:32:40.637 important. But secondly, our goal isn't to be the ones to build it. Our goal is 00:32:40.720 --> 00:32:41.997 to make sure it goes well for the world. 00:32:42.080 --> 00:32:45.757 And so I think that figuring out how do you balance all of those um and to to 00:32:45.840 --> 00:32:47.517 get people to really come to the table 00:32:47.600 --> 00:32:50.797 and compile the the like a single 00:32:50.880 --> 00:32:54.797 document that that encompasses all of that wasn't trivial. 00:32:54.880 --> 00:32:57.597 >> So part of the challenge here is uh your 00:32:57.680 --> 00:33:00.797 mission is I would say beautiful 00:33:00.880 --> 00:33:04.957 empowering and a beacon of hope for people in the research community and 00:33:05.040 --> 00:33:07.037 just people thinking about AI. So your 00:33:07.120 --> 00:33:10.637 decisions are scrutinized more than I 00:33:10.720 --> 00:33:13.117 think a regular profit-driven company. 00:33:13.200 --> 00:33:16.397 Do you feel the burden of this in the creation of the charter and just in the 00:33:16.480 --> 00:33:17.517 way you operate? 00:33:17.600 --> 00:33:23.587 >> Yes. [laughter] >> Uh so why do you uh lean into the burden 00:33:23.670 --> 00:33:27.597 [laughter] by creating such a charter? Why not keep it quiet? 00:33:27.680 --> 00:33:31.677 >> I mean it just boils down to the to the mission, right? Like like I'm here and 00:33:31.760 --> 00:33:34.477 everyone else is here because we think this is the most important mission, 00:33:34.560 --> 00:33:37.437 right? Dare to dare to dream, right? So, 00:33:37.520 --> 00:33:40.397 uh, do you think you can be good for the 00:33:40.480 --> 00:33:42.317 world or create an AGI system that's 00:33:42.400 --> 00:33:45.597 good when you're a for-profit company? 00:33:45.680 --> 00:33:47.917 From my perspective, I don't understand 00:33:48.000 --> 00:33:49.917 why profit 00:33:50.000 --> 00:33:53.757 uh interferes with u positive impact on 00:33:53.840 --> 00:33:57.197 society. I don't understand uh why uh 00:33:57.280 --> 00:33:59.517 Google that makes most of its money from 00:33:59.600 --> 00:34:02.797 ads can't also do good for the world or 00:34:02.880 --> 00:34:06.637 other companies, Facebook, anything. I don't I don't understand why those have 00:34:06.720 --> 00:34:09.197 to interfere. You know, you can um 00:34:09.280 --> 00:34:12.637 profit isn't the thing in my view that 00:34:12.720 --> 00:34:14.717 affects the impact of a company. what 00:34:14.800 --> 00:34:17.197 affects the impact of the company is the 00:34:17.280 --> 00:34:20.317 charter is the culture is the you know 00:34:20.400 --> 00:34:24.797 the people inside and profit is the thing that just fuels those people. So 00:34:24.880 --> 00:34:27.437 what are your views there? >> Yeah. So I I think that's a it's a 00:34:27.520 --> 00:34:30.557 really good question and there's there's there's some some you know real like 00:34:30.640 --> 00:34:33.677 long long-standing debates in human society that are wrapped up in it. Uh 00:34:33.760 --> 00:34:35.517 the way that I think about it is just 00:34:35.600 --> 00:34:37.117 think about what what are the most 00:34:37.200 --> 00:34:41.197 impactful nonprofits in the world? 00:34:41.280 --> 00:34:45.197 What are the most impactful for-profits in the world? Right. It's much easier to 00:34:45.280 --> 00:34:47.837 list the forprofits. >> That's right. And I think that there's 00:34:47.920 --> 00:34:52.877 there's some real truth here that the system that we set up, the system for 00:34:52.960 --> 00:34:54.557 kind of how you know today's world is 00:34:54.640 --> 00:34:57.037 organized um is one that that really 00:34:57.120 --> 00:35:01.597 allows for huge impact. Um and that that you know kind of part of that is that 00:35:01.680 --> 00:35:05.277 you need to be that that you know for for profofits are are self-sustaining 00:35:05.360 --> 00:35:08.877 and able to to kind of you know build on their own momentum. Um and I think 00:35:08.960 --> 00:35:13.357 that's a really powerful thing. It's something that when it turns out that we 00:35:13.440 --> 00:35:16.397 haven't set the guardrails correctly causes problems, right? Think about 00:35:16.480 --> 00:35:20.237 logging companies that go and deforest, you know, the the rainforest. That's 00:35:20.320 --> 00:35:23.677 really bad. We don't want that. Um, and it's actually really interesting to me 00:35:23.760 --> 00:35:27.517 that kind of this this question of how do you get positive benefits out of a 00:35:27.600 --> 00:35:30.637 for-profit company? It's actually very similar to how do you get positive 00:35:30.720 --> 00:35:33.197 benefits out of an AGI? Um, right, that 00:35:33.280 --> 00:35:35.197 you have this like very powerful system. 00:35:35.280 --> 00:35:39.037 It's more powerful than any human and is kind of autonomous in some ways. you 00:35:39.120 --> 00:35:42.077 know, it's super human in a lot of axes and somehow you have to set the 00:35:42.160 --> 00:35:45.757 guardrails to get good things to happen, but when you do, the benefits are 00:35:45.840 --> 00:35:48.877 massive. And so, I think that that when 00:35:48.960 --> 00:35:52.957 when I think about nonprofit versus for-profit, I think just not enough 00:35:53.040 --> 00:35:56.397 happens in nonprofits. They're very pure, but it's just kind of, you know, 00:35:56.480 --> 00:35:59.837 it's just hard to do things there. Um, in forprofits in some ways, like too 00:35:59.920 --> 00:36:02.877 much happens. Um, but, um, if if kind of 00:36:02.960 --> 00:36:07.437 shaped in the right way, it can actually be very positive. And so with open ILP, 00:36:07.520 --> 00:36:09.357 we're we're picking a road in between. 00:36:09.440 --> 00:36:12.637 Now the thing I think is really important to recognize is that the way 00:36:12.720 --> 00:36:17.357 that we think about open ILP is that in the world where AGI actually happens, 00:36:17.440 --> 00:36:19.357 right? In a world where we are successful, we build the most 00:36:19.440 --> 00:36:22.157 transformative technology ever, the amount of value we're going to create 00:36:22.240 --> 00:36:24.717 will be astronomical. >> Mhm. 00:36:24.800 --> 00:36:26.157 >> And so [clears throat] then in that case 00:36:26.240 --> 00:36:30.157 that the that the cap that we have will 00:36:30.240 --> 00:36:34.157 be a small fraction of the value we create. Um, and the amount of value that 00:36:34.240 --> 00:36:36.957 goes back to investors and employees looks pretty similar to what would 00:36:37.040 --> 00:36:40.957 happen in a in a pretty successful startup. 00:36:41.040 --> 00:36:43.997 And that's really the case that we're optimizing for, right? That we're 00:36:44.080 --> 00:36:48.157 thinking about in this success case, making sure that the value we create 00:36:48.240 --> 00:36:52.077 doesn't get locked up. And I expect that in other, you know, for-profit companies 00:36:52.160 --> 00:36:56.557 that it's possible to do something like that. I think it's not obvious how to do 00:36:56.640 --> 00:37:00.157 it, right? I think that as a for-profit company, you have a lot of fiduciary 00:37:00.240 --> 00:37:03.437 duty to your shareholders and that there are certain decisions that you just 00:37:03.520 --> 00:37:08.237 cannot make. Um, in our structure, we've set it up so that we have a fiduciary 00:37:08.320 --> 00:37:11.517 duty to the charter, right? >> That we always get to make the decision 00:37:11.600 --> 00:37:14.237 that is right for the charter um rather 00:37:14.320 --> 00:37:18.957 than even if it comes at the expense of our own stakeholders. And uh and so I 00:37:19.040 --> 00:37:22.157 think that when I think about what's really important, it's not really about 00:37:22.240 --> 00:37:27.277 nonprofit versus for-profit. It's really a question of if you build AGI and you 00:37:27.360 --> 00:37:32.237 kind of you know humanity is now in this new age who benefits whose lives are 00:37:32.320 --> 00:37:35.517 better. Um and I think that what's really important is to have an answer 00:37:35.600 --> 00:37:39.677 that is everyone. >> Yeah. Which is one of the core aspects 00:37:39.760 --> 00:37:41.677 of the charter. So one concern people 00:37:41.760 --> 00:37:44.077 have not just with open AI but with 00:37:44.160 --> 00:37:47.437 Google, Facebook, Amazon, anybody 00:37:47.520 --> 00:37:50.717 uh really uh that's that's creating 00:37:50.800 --> 00:37:53.917 impact at scale is how do we avoid as 00:37:54.000 --> 00:37:56.157 your charter says avoid enabling the use 00:37:56.240 --> 00:37:59.517 of AI or AGI to unduly concentrate 00:37:59.600 --> 00:38:04.957 power? Why would not a company like open AI keep all the power of an AGI system 00:38:05.040 --> 00:38:08.477 to itself? >> The charter. >> The charter. So, you know, how does the 00:38:08.560 --> 00:38:12.717 charter actionalize itself 00:38:12.800 --> 00:38:17.037 in uh day-to-day? >> So, I think that that first to to zoom 00:38:17.120 --> 00:38:19.997 out, right? That the way that we structure the company is so that the the 00:38:20.080 --> 00:38:24.397 power for sort of, you know, dictating the actions that OpenAI takes ultimately 00:38:24.480 --> 00:38:28.397 rests with the board, right? The board of of of the nonprofit. Um, and the 00:38:28.480 --> 00:38:30.957 board is set up in certain ways with certain certain restrictions that you 00:38:31.040 --> 00:38:32.797 can read about in the OpenAI LP blog 00:38:32.880 --> 00:38:34.797 post. Um but effectively the board is 00:38:34.880 --> 00:38:38.477 the is the governing body for open LP. 00:38:38.560 --> 00:38:40.957 Um and the board has a duty to fulfill 00:38:41.040 --> 00:38:44.477 the mission of the nonprofit. Um and so 00:38:44.560 --> 00:38:48.397 that's kind of how we tie how we thread all these things together. Um now 00:38:48.480 --> 00:38:52.557 there's a question of so day-to-day how do people the individuals who in some 00:38:52.640 --> 00:38:55.517 ways are the most empowered ones right you know the board sort of gets to call 00:38:55.600 --> 00:38:59.597 the shots at the high level but the people who are actually executing are 00:38:59.680 --> 00:39:02.957 the employees right the people here on a day-to-day basis who have the you know 00:39:03.040 --> 00:39:06.237 the the keys to the technical kingdom 00:39:06.320 --> 00:39:10.637 and there I think that the answer looks a lot like well how does any company's 00:39:10.720 --> 00:39:14.237 values get actualized right I think that a lot of that comes down to that you 00:39:14.320 --> 00:39:18.877 need people who are here because they really believe in that mission um and 00:39:18.960 --> 00:39:22.957 they believe in the charter and that they are willing to take actions uh that 00:39:23.040 --> 00:39:26.557 maybe are worse for them but are better for the charter um and that's something 00:39:26.640 --> 00:39:30.557 that's really baked into the culture and honestly I think it's uh you know I 00:39:30.640 --> 00:39:34.317 think that that's one of the things that we really have to work to preserve as 00:39:34.400 --> 00:39:38.157 time goes on um and that's a really important part of how we think about 00:39:38.240 --> 00:39:39.517 hiring people and bringing people into 00:39:39.600 --> 00:39:41.997 open AI. So there's people here there's 00:39:42.080 --> 00:39:46.237 people here who could speak up and say 00:39:46.320 --> 00:39:50.717 like hold on a second this is totally against what we stand for 00:39:50.800 --> 00:39:53.037 >> uh culture wise. >> Yeah. Yeah. For sure. I mean I think 00:39:53.120 --> 00:39:56.157 that uh that we actually have I think a that's like a pretty important part of 00:39:56.240 --> 00:40:01.277 of how how we operate and how we have even again with designing the charter 00:40:01.360 --> 00:40:05.357 and designing open ALP in the first place. uh that there has been a lot of 00:40:05.440 --> 00:40:09.037 conversation with employees here and a lot of times where employees said, "Wait 00:40:09.120 --> 00:40:11.837 a second, this seems like it's going in the wrong direction and let's talk about 00:40:11.920 --> 00:40:15.117 it." And so I think one thing that's that's I think a really and you know, 00:40:15.200 --> 00:40:18.797 here's here's actually one thing that I think is very unique about us as a small 00:40:18.880 --> 00:40:22.557 company is that if you're at a massive tech giant, it's a little bit hard for 00:40:22.640 --> 00:40:26.717 someone who's a line employee to go and talk to the CEO and say, "I think that 00:40:26.800 --> 00:40:30.237 we're doing this wrong." Um, and you know, you look at companies like Google 00:40:30.320 --> 00:40:34.877 that have had some collective action from employees to, you know, make uh 00:40:34.960 --> 00:40:36.557 ethical change around things like Maven. 00:40:36.640 --> 00:40:40.237 Um, and so maybe there are mechanisms at other companies that work. Um, but here, 00:40:40.320 --> 00:40:43.677 super easy for anyone to pull me aside, to pull Sam aside, to pull aside, and 00:40:43.760 --> 00:40:46.317 people do it all the time. >> One of the interesting things in the 00:40:46.400 --> 00:40:51.357 charter is this idea that it'd be great if you could try to describe or untangle 00:40:51.440 --> 00:40:55.357 of switching from competition to collaboration and latestage AGI 00:40:55.440 --> 00:40:56.957 development. It was really interesting. 00:40:57.040 --> 00:41:00.157 this dance between competition and collaboration. How do you think about 00:41:00.240 --> 00:41:02.237 that? >> Yeah, assuming that you can actually do 00:41:02.320 --> 00:41:05.597 the technical side of AGI development, I think there's going to be two key 00:41:05.680 --> 00:41:08.877 problems with figuring out how do you actually deploy it, make it go well. The 00:41:08.960 --> 00:41:13.277 first one of these is the runup to building the first AGI. 00:41:13.360 --> 00:41:16.477 >> Um, you look at how self-driving cars are being developed and it's a 00:41:16.560 --> 00:41:19.117 competitive race >> and the thing that always happens in 00:41:19.200 --> 00:41:23.917 competitive race is that you have huge amounts of pressure to get rid of safety. 00:41:24.000 --> 00:41:27.277 And so that's one thing we're very concerned about, right? Is that people 00:41:27.360 --> 00:41:29.597 multiple teams figuring out we can 00:41:29.680 --> 00:41:32.797 actually get there, but you know, if we 00:41:32.880 --> 00:41:37.757 took the slower path that is more guaranteed to be safe, we will lose. And 00:41:37.840 --> 00:41:39.597 so we're going to take the fast path. 00:41:39.680 --> 00:41:43.917 And so the more that we can both ourselves be in a position where we 00:41:44.000 --> 00:41:47.437 don't generate that competitive race where we say if the race is being run 00:41:47.520 --> 00:41:50.957 and that, you know, someone else is is further ahead than we are, we're not 00:41:51.040 --> 00:41:54.797 going to try to to leaprog. we're gonna actually work with them, right? We will 00:41:54.880 --> 00:41:58.557 help them succeed. As long as what they're trying to do is to fulfill our 00:41:58.640 --> 00:42:02.477 mission, then we're good. We don't have to build AGI ourselves. And I think 00:42:02.560 --> 00:42:05.277 that's a really important commitment from us. But it can't just be 00:42:05.360 --> 00:42:08.797 unilateral, right? I think that it's really important that other players who 00:42:08.880 --> 00:42:12.237 are serious about building AGI make similar commitments, right? And I think 00:42:12.320 --> 00:42:15.677 that uh that you know, again, to the extent that everyone believes that AGI 00:42:15.760 --> 00:42:18.477 should be something to benefit everyone, then it actually really shouldn't matter 00:42:18.560 --> 00:42:21.997 which company builds it. And we should all be concerned about the case where we 00:42:22.080 --> 00:42:24.797 just race. so hard to get there that something goes wrong. 00:42:24.880 --> 00:42:27.757 >> So what role do you think government 00:42:27.840 --> 00:42:30.157 our favorite entity has in setting 00:42:30.240 --> 00:42:33.837 policy and rules about this domain from 00:42:33.920 --> 00:42:39.597 research to the development to early stage to late stage a AI and AGI 00:42:39.680 --> 00:42:42.797 development. >> So I think that that first of all um 00:42:42.880 --> 00:42:46.237 it's really important the government's in there right in some way shape or 00:42:46.320 --> 00:42:48.877 form. you know, at the end of the day, we're talking about building technology 00:42:48.960 --> 00:42:52.317 that will shape how the world operates 00:42:52.400 --> 00:42:54.717 and that there needs to be government as 00:42:54.800 --> 00:42:59.437 part of that answer. And so that's why we've uh we've we've done a number of 00:42:59.520 --> 00:43:02.557 different congressional testimonies. We uh interact with a number of different 00:43:02.640 --> 00:43:05.357 lawmakers and uh that you know right now 00:43:05.440 --> 00:43:08.317 a lot of our message to them is that 00:43:08.400 --> 00:43:11.837 it's not the time for regulation, it is 00:43:11.920 --> 00:43:15.757 the time for measurement, right? that our main policy recommendation is that 00:43:15.840 --> 00:43:19.837 people and you know the government does this all the time with bodies like NIST 00:43:19.920 --> 00:43:22.717 um spend time trying to figure out just 00:43:22.800 --> 00:43:27.037 where the technology is, how fast it's moving um and can really uh become 00:43:27.120 --> 00:43:31.677 literate and up to speed with respect to what to expect. Um so I think that today 00:43:31.760 --> 00:43:35.437 the answer really is about about about measurement. Um and I think that there 00:43:35.520 --> 00:43:38.157 will be a time uh and place where that 00:43:38.240 --> 00:43:43.037 will change and uh I think it's a little bit hard to predict exactly uh what what 00:43:43.120 --> 00:43:44.557 exactly that trajectory should look like. 00:43:44.640 --> 00:43:48.557 >> So there will be a point at which regulation 00:43:48.640 --> 00:43:50.477 federal in the United States the 00:43:50.560 --> 00:43:54.717 government steps in and and helps be the 00:43:54.800 --> 00:43:58.477 I I don't want to say the adult in the room to make sure that there is strict 00:43:58.560 --> 00:44:02.797 rules maybe conservative rules that nobody can cross. Uh well I think 00:44:02.880 --> 00:44:06.157 there's there's kind of maybe two two angles to it. So today with narrow AI 00:44:06.240 --> 00:44:09.837 applications uh that I think there are already existing bodies that are 00:44:09.920 --> 00:44:12.797 responsible and should be responsible for regulation. You think about for 00:44:12.880 --> 00:44:16.157 example with self-driving cars that you want the uh you know the national 00:44:16.240 --> 00:44:21.437 highway uh exactly to to be regulating that. That makes sense, right? That 00:44:21.520 --> 00:44:24.797 basically what we're saying is that we're going to have these technological 00:44:24.880 --> 00:44:28.317 systems that are going to be do performing applications that humans 00:44:28.400 --> 00:44:32.477 already do. Great. we already have ways of thinking about standards and safety 00:44:32.560 --> 00:44:36.317 for those. Um, so I think actually empowering those regulators today is 00:44:36.400 --> 00:44:41.197 also pretty important. And then I think for for AGI, you know, that there's 00:44:41.280 --> 00:44:43.997 going to be a point where we'll have better answers. And I think that maybe a 00:44:44.080 --> 00:44:47.837 similar approach of first measurement and, you know, start thinking about what 00:44:47.920 --> 00:44:51.837 the rules should be. I think it's really important that we don't prematurely 00:44:51.920 --> 00:44:56.477 squash, you know, progress. Like I think it's very easy to kind of smother the a 00:44:56.560 --> 00:44:59.597 budding field and I think that's something to to really avoid. But I 00:44:59.680 --> 00:45:01.197 don't think the right way of doing it is 00:45:01.280 --> 00:45:04.477 to say let's just try to blaze ahead and 00:45:04.560 --> 00:45:06.797 not involve all these other 00:45:06.880 --> 00:45:08.797 stakeholders. 00:45:08.880 --> 00:45:12.477 >> So uh you recently released a paper on 00:45:12.560 --> 00:45:15.837 GPT2 language modeling uh >> Mhm. 00:45:15.920 --> 00:45:19.277 >> Uh but uh did not release the full model 00:45:19.360 --> 00:45:21.277 because you had concerns about the 00:45:21.360 --> 00:45:23.117 possible negative effects of the 00:45:23.200 --> 00:45:26.397 availability of such model. It's uh 00:45:26.480 --> 00:45:30.397 outside of just that decision, it's super interesting because of uh the 00:45:30.480 --> 00:45:33.197 discussion as at a societal level, the 00:45:33.280 --> 00:45:37.677 discourse it creates. So it's fascinating in that aspect. But if you 00:45:37.760 --> 00:45:41.597 think that's the specifics here at first, what are some negative effects 00:45:41.680 --> 00:45:45.837 that you envisioned and of course what are some of the positive effects? 00:45:45.920 --> 00:45:48.077 >> Yeah. So again I think to zoom out like 00:45:48.160 --> 00:45:51.517 the way that we thought about GPT2 is 00:45:51.600 --> 00:45:53.517 that with language modeling we are 00:45:53.600 --> 00:45:56.397 clearly on a trajectory right now where 00:45:56.480 --> 00:45:59.677 we scale up our models and we get 00:45:59.760 --> 00:46:04.557 qualitatively better performance right gpt2 itself was actually just a scale up 00:46:04.640 --> 00:46:08.477 of a model that we've released pre in the previous June right and we just ran 00:46:08.560 --> 00:46:11.997 it at you know much larger scale and we got these results where suddenly 00:46:12.080 --> 00:46:14.637 starting to write coherent pros uh which 00:46:14.720 --> 00:46:17.277 was not something we'd seen previously 00:46:17.360 --> 00:46:21.837 and what are we doing now? Well, we're going to scale up GPD2 by 10x, by 100x, 00:46:21.920 --> 00:46:25.837 by thousandx and we don't know what we're going to get. And so, it's very 00:46:25.920 --> 00:46:30.317 clear that the model that we that we released last June. You know, I think 00:46:30.400 --> 00:46:34.397 it's kind of like it's it's it's it's a good academic toy. It's not something 00:46:34.480 --> 00:46:37.997 that we think is something that can really have negative implications or you 00:46:38.080 --> 00:46:40.797 know to the extent that it can the positive of people being able to play 00:46:40.880 --> 00:46:43.517 with it is is you know far far outweighs 00:46:43.600 --> 00:46:49.357 the the the possible harms. You fast forward to not GPT2 but GPT20, 00:46:49.440 --> 00:46:51.037 >> right? >> And you think about what that's going to 00:46:51.120 --> 00:46:54.477 be like and I think that the capabilities are going to be 00:46:54.560 --> 00:46:56.557 substantive. And so there needs to be a 00:46:56.640 --> 00:47:01.517 point in between the two where you say this is something where we are drawing 00:47:01.600 --> 00:47:05.517 the line um and that we need to start thinking about the safety aspects. And I 00:47:05.600 --> 00:47:08.397 think for GPT2, we could have gone either way. And in fact, when we had 00:47:08.480 --> 00:47:12.557 conversations internally, uh that we had a bunch of pros and cons. Um and it 00:47:12.640 --> 00:47:16.077 wasn't clear which one which one outweighed the other. Um and I think 00:47:16.160 --> 00:47:19.357 that when we announced that, hey, we decide not to release this model. Um 00:47:19.440 --> 00:47:22.077 then there was a bunch of conversation where various people said it's so 00:47:22.160 --> 00:47:24.237 obvious that you should have just released it. There other people said 00:47:24.320 --> 00:47:27.117 it's so obvious you should not have released it. And I think that that 00:47:27.200 --> 00:47:31.037 almost definitionally means that holding it back was the correct decision. Right? 00:47:31.120 --> 00:47:34.797 if it's if there's if it's not obvious whether something is beneficial or not, 00:47:34.880 --> 00:47:36.957 you should probably default to caution. 00:47:37.040 --> 00:47:40.477 And so I think that the that that the overall landscape for how we think about 00:47:40.560 --> 00:47:44.237 it is that this decision could have gone either way. There's great arguments in 00:47:44.320 --> 00:47:48.397 both directions, but for future models down the road um and possibly sooner 00:47:48.480 --> 00:47:51.037 than than than you'd expect because, you know, scaling these things up doesn't 00:47:51.120 --> 00:47:55.277 actually take that long, those ones you're definitely not going to want to 00:47:55.360 --> 00:47:57.997 to release into the wild. And so um I 00:47:58.080 --> 00:48:02.317 think that we we almost view this as a test case and to see can we even design 00:48:02.400 --> 00:48:05.517 you know how how do you have a society where how do you have a system that goes 00:48:05.600 --> 00:48:10.077 from having no concept of responsible disclosure where the mere idea of not 00:48:10.160 --> 00:48:14.237 releasing something for safety reasons is unfamiliar to a world where you say 00:48:14.320 --> 00:48:17.437 okay we have a powerful model let's at least think about it let's go through 00:48:17.520 --> 00:48:20.797 some process and you think about the security community it took them a long 00:48:20.880 --> 00:48:23.837 time to design responsible disclosure right you know you think about this 00:48:23.920 --> 00:48:27.277 question of well I have a security exploit I send it to the company the 00:48:27.360 --> 00:48:29.517 company's like tries to prosecute me or 00:48:29.600 --> 00:48:34.397 just sit just ignores it what do I do right and so you know the alternatives 00:48:34.480 --> 00:48:37.277 of oh I just just always publish your exploits that doesn't seem good either 00:48:37.360 --> 00:48:41.597 right and so it really took a long time and took this this uh it was bigger than 00:48:41.680 --> 00:48:44.717 any individual right it's really about building a whole community that believe 00:48:44.800 --> 00:48:47.597 that okay we'll have this process where you send it to the company you know if 00:48:47.680 --> 00:48:51.357 they don't act in a certain time then you can go public and you're not a bad 00:48:51.440 --> 00:48:53.357 person you've done the right thing um 00:48:53.440 --> 00:48:56.477 and I think that in AI part of the 00:48:56.560 --> 00:48:59.037 response to GBD2 just proves that we 00:48:59.120 --> 00:49:02.557 don't have any concept of this. Um, so 00:49:02.640 --> 00:49:07.037 that's the highle picture. Um, and so I think that I think this was this was a 00:49:07.120 --> 00:49:11.117 really important move to make. Um, and we could have maybe delayed it for GPT3, 00:49:11.200 --> 00:49:13.197 but I'm really glad we did it for GPT2. 00:49:13.280 --> 00:49:16.637 And so now you look at GPT2 itself and you think about the substance of okay, 00:49:16.720 --> 00:49:19.997 what are potential negative applications? So you have this model 00:49:20.080 --> 00:49:22.717 that's been trained on the internet which you know is also going to be a 00:49:22.800 --> 00:49:24.477 bunch of very biased data a bunch of you 00:49:24.560 --> 00:49:26.477 know very uh offensive content in there 00:49:26.560 --> 00:49:31.677 uh and uh you can ask it to generate content for you on basically any topic 00:49:31.760 --> 00:49:34.317 right you just give it a prompt and it'll just start start writing and it 00:49:34.400 --> 00:49:37.517 writes content like you see on the internet you know even down to like 00:49:37.600 --> 00:49:41.997 saying advertisement [laughter] in the middle of of some of its generations and 00:49:42.080 --> 00:49:46.397 uh you think about the possibilities for generating fake news or abusive content 00:49:46.480 --> 00:49:49.117 >> and you know it's interesting seeing what people have done with uh you know 00:49:49.200 --> 00:49:51.757 we released a smaller version of GPT2 00:49:51.840 --> 00:49:56.317 and uh that people have done things like try to generate uh you know to take my 00:49:56.400 --> 00:50:00.477 own Facebook message history and generate more Facebook messages like me 00:50:00.560 --> 00:50:05.357 um and uh people generating fake politician uh uh content or uh you know 00:50:05.440 --> 00:50:09.597 there there's a bunch of of things there where you at least have to think is this 00:50:09.680 --> 00:50:11.997 going to be good for the world. 00:50:12.080 --> 00:50:14.557 There's the flip side, which is I think that there's a lot of awesome 00:50:14.640 --> 00:50:16.717 applications that we really want to see 00:50:16.800 --> 00:50:21.757 like creative uh applications in terms of if you have sci-fi authors that can 00:50:21.840 --> 00:50:25.597 work with this tool and come up with cool ideas like that seems that seems 00:50:25.680 --> 00:50:28.717 awesome if we can write better sci-fi through the use of of these tools. And 00:50:28.800 --> 00:50:31.997 we've actually had a bunch of people write into us asking, hey, can we use it 00:50:32.080 --> 00:50:35.677 for you know a variety of different creative applications? 00:50:35.760 --> 00:50:37.837 >> So the positive are actually pretty easy 00:50:37.920 --> 00:50:41.357 to imagine. uh there, [snorts] 00:50:41.440 --> 00:50:44.157 you know, the the usual NLP applications 00:50:44.240 --> 00:50:47.917 are really interesting, but let let's go 00:50:48.000 --> 00:50:49.357 there. It's kind of interesting to think 00:50:49.440 --> 00:50:53.517 about a world where uh look at Twitter 00:50:53.600 --> 00:50:56.477 where not just fake news but smarter and 00:50:56.560 --> 00:51:00.477 smarter bots being able to uh spread in 00:51:00.560 --> 00:51:03.357 an interesting complex networking way in 00:51:03.440 --> 00:51:06.317 information that just floods out us 00:51:06.400 --> 00:51:09.037 regular human beings with our original 00:51:09.120 --> 00:51:12.557 thoughts. So uh what are your views of 00:51:12.640 --> 00:51:16.637 this world with GPT20 right? 00:51:16.720 --> 00:51:19.837 >> Mhm. What do you how do we think about it? Again, it's like one of those things 00:51:19.920 --> 00:51:23.277 about in the 50s trying to describe the 00:51:23.360 --> 00:51:26.077 uh the internet or the smartphone. What 00:51:26.160 --> 00:51:27.357 do you think about that world, the 00:51:27.440 --> 00:51:30.557 nature of information? Do we uh one 00:51:30.640 --> 00:51:33.197 possibility is that we'll always try to 00:51:33.280 --> 00:51:35.597 design systems that identify robot 00:51:35.680 --> 00:51:38.477 versus human and will do so successfully 00:51:38.560 --> 00:51:42.957 and so we will authenticate that we're still human. And the other world is that 00:51:43.040 --> 00:51:48.077 we just accept the the fact that we're swimming in a sea of fake news and just 00:51:48.160 --> 00:51:51.037 learn to swim there. >> Well, have have you ever seen the uh 00:51:51.120 --> 00:51:54.077 there there's a you know popular uh meme 00:51:54.160 --> 00:51:59.037 of uh of uh robot uh with with a physical physical arm and pen clicking 00:51:59.120 --> 00:52:02.477 the I'm not a robot button. >> Yeah. [laughter] 00:52:02.560 --> 00:52:05.917 >> I think I think that the truth is that uh that really trying to distinguish 00:52:06.000 --> 00:52:09.437 between robot and human is a losing battle. 00:52:09.520 --> 00:52:12.077 >> Ultimately, you think it's a losing battle? I think it's a losing battle 00:52:12.160 --> 00:52:15.197 ultimately, right? I think that that is that in terms of of the content, in 00:52:15.280 --> 00:52:16.557 terms of the actions that you can take. 00:52:16.640 --> 00:52:19.357 I mean, think about how captures have gone, right? The captures used to be a 00:52:19.440 --> 00:52:23.757 very nice simple. You just have this image, all of our OCR is terrible. You 00:52:23.840 --> 00:52:26.957 put a couple of of artifacts in it, you know, humans are going to be able to 00:52:27.040 --> 00:52:31.517 tell what what what it is. An AI system wouldn't be able to today. Like, I could 00:52:31.600 --> 00:52:34.877 barely do captures. >> Yeah. >> And I think that that that this is just 00:52:34.960 --> 00:52:37.357 kind of where we're going. I think captures were were a moment in time 00:52:37.440 --> 00:52:41.357 thing. And as AI systems become more powerful that there being human 00:52:41.440 --> 00:52:46.157 capabilities that can be measured in a very easy automated way uh that that AIs 00:52:46.240 --> 00:52:49.597 will not be capable of. I think that's just like it's just an increasingly hard 00:52:49.680 --> 00:52:52.397 technical battle. But it's not that all 00:52:52.480 --> 00:52:57.037 hope is lost, right? You think about uh how do we already authenticate 00:52:57.120 --> 00:53:00.637 ourselves, right? that you know we have systems we have social security numbers 00:53:00.720 --> 00:53:04.957 if you're in the US or you know you have you have uh you know ways of identifying 00:53:05.040 --> 00:53:08.877 individual people um and having real world identity tied to to digital 00:53:08.960 --> 00:53:13.197 identity seems like a step uh towards you know authenticating the source of 00:53:13.280 --> 00:53:17.437 content rather than the content itself um now there are problems with that how 00:53:17.520 --> 00:53:21.357 can you have privacy and anonymity in a world where the only content you can 00:53:21.440 --> 00:53:25.037 really trust is or the only way you can trust content is by looking at where it 00:53:25.120 --> 00:53:28.797 comes from. Um, and so I think that building out good reputation networks uh 00:53:28.880 --> 00:53:33.117 may be may be one possible solution. But yeah, I think that this this question is 00:53:33.200 --> 00:53:37.437 is not an obvious one. And I think that we, you know, maybe sooner than we think 00:53:37.520 --> 00:53:41.357 we'll be in a world where, you know, today I often will read a tweet and be 00:53:41.440 --> 00:53:44.157 like, hm, do I feel like a real human wrote this or, you know, do I feel like 00:53:44.240 --> 00:53:46.957 this is like genuine? I feel like I can kind of judge the content a little bit. 00:53:47.040 --> 00:53:50.477 Um, and I think in the future it just won't be the case. You look at, for 00:53:50.560 --> 00:53:55.037 example, the FCC comments on net neutrality. uh it came out later that 00:53:55.120 --> 00:53:58.957 millions of those were autogenerated and that the researchers were able to do 00:53:59.040 --> 00:54:03.357 various statistical tech techniques to do that. What do you do in a world where 00:54:03.440 --> 00:54:06.157 those statistical techniques don't exist? It's just impossible to tell the 00:54:06.240 --> 00:54:10.237 difference between humans and AIs. And in fact, the uh the the the most 00:54:10.320 --> 00:54:12.877 persuasive arguments are written by by 00:54:12.960 --> 00:54:17.517 AI. All that stuff, it's not sci-fi anymore. You look at GPT2 making a great 00:54:17.600 --> 00:54:20.557 argument for why recycling is bad for the world. You got to read that and be 00:54:20.640 --> 00:54:23.917 like, "Huh, you're right. We are addressing different symptoms. 00:54:24.000 --> 00:54:27.757 >> Yeah, that's that's quite interesting. I mean, ultimately it boils down to the 00:54:27.840 --> 00:54:32.237 physical world being the last frontier of proving. So you said like basically 00:54:32.320 --> 00:54:34.397 networks of people humans vouching for 00:54:34.480 --> 00:54:37.917 humans in the physical world and somehow 00:54:38.000 --> 00:54:40.477 the authentication uh ends there. I mean 00:54:40.560 --> 00:54:43.997 if I had to ask you I mean you're way 00:54:44.080 --> 00:54:49.197 too eloquent for a human. So if I had to ask you to authenticate like prove how 00:54:49.280 --> 00:54:50.717 do I know you're not a robot and how do 00:54:50.800 --> 00:54:52.397 you know I'm not a robot? 00:54:52.480 --> 00:54:56.957 >> I think that's so far we're we're this 00:54:57.040 --> 00:54:58.877 in this space this conversation we just 00:54:58.960 --> 00:55:01.757 had the physical movements we did is the 00:55:01.840 --> 00:55:04.557 biggest gap between us and AI systems is 00:55:04.640 --> 00:55:09.037 the physical manipulation. So maybe that's the last frontier. Well, here's 00:55:09.120 --> 00:55:14.317 another question is is you know why why is why is solving this problem important 00:55:14.400 --> 00:55:16.957 right? like what aspects are really important to us. And I think that 00:55:17.040 --> 00:55:21.277 probably where we'll end up is we'll hone in on what do we really want out of 00:55:21.360 --> 00:55:25.917 knowing if we're talking to a human. Um and uh and I think that again this comes 00:55:26.000 --> 00:55:29.437 down to identity. And so I think that that the internet of the future I expect 00:55:29.520 --> 00:55:33.117 to be one that will have lots of agents out there uh that will interact with 00:55:33.200 --> 00:55:37.437 with you. But I think that the question of is this you know a flesh real flesh 00:55:37.520 --> 00:55:41.757 and blood uh human or is this an automated system uh may may actually 00:55:41.840 --> 00:55:43.837 just be less important. Let's actually 00:55:43.920 --> 00:55:47.837 go there. It's GPT2 is impressive and 00:55:47.920 --> 00:55:51.197 let's look at GPT20. Why is it so bad 00:55:51.280 --> 00:55:56.557 that all my friends are GPT20? 00:55:56.640 --> 00:55:59.117 Why why is it so why is it so important 00:55:59.200 --> 00:56:02.477 on the internet do you think uh to 00:56:02.560 --> 00:56:04.797 interact with only human beings? Why 00:56:04.880 --> 00:56:06.957 can't we live in a world where ideas can 00:56:07.040 --> 00:56:10.157 come from models trained on human data? 00:56:10.240 --> 00:56:12.957 Yeah, I I think this is I think this is actually a really interesting question. 00:56:13.040 --> 00:56:15.917 This comes back to the how do you even picture a world with some new 00:56:16.000 --> 00:56:20.157 technology. And I think that that one thing that I think is important is is 00:56:20.240 --> 00:56:24.317 you know, let's say honesty. Um and I think that if you have, you know, almost 00:56:24.400 --> 00:56:29.197 in the the touring test style uh sense sense of of of technology, you have AIs 00:56:29.280 --> 00:56:32.877 that are pretending to be humans and deceiving you. Uh I think that is, you 00:56:32.960 --> 00:56:35.597 know, that that feels like a bad thing, right? I think that it's really 00:56:35.680 --> 00:56:38.717 important that we feel like we're in control of our environment, right? that 00:56:38.800 --> 00:56:43.597 we understand who we're interacting with and if it's an AI or a human um that 00:56:43.680 --> 00:56:46.397 that that's not something that we're being deceived about. But I think that 00:56:46.480 --> 00:56:49.917 the flip side of can I have as meaningful of an interaction with an AI 00:56:50.000 --> 00:56:53.357 as I can with a human. Uh well, I actually think here you can turn to 00:56:53.440 --> 00:56:57.757 sci-fi. Um and her I think is a great example of asking this very question, 00:56:57.840 --> 00:57:00.877 right? And one thing I really love about her is it really starts out almost by 00:57:00.960 --> 00:57:06.077 asking how meaningful are human virtual relationships, right? and uh and then 00:57:06.160 --> 00:57:10.397 you have a human who has a relationship with an AI and uh that you really start 00:57:10.480 --> 00:57:13.597 to be drawn into that, right? And that all of your emotional buttons get 00:57:13.680 --> 00:57:16.637 triggered in the same way as if there was a real human that was on the other 00:57:16.720 --> 00:57:20.877 side of that phone. And so I think that that this is one way of thinking about 00:57:20.960 --> 00:57:25.037 it is that I think that we can have meaningful interactions and that if 00:57:25.120 --> 00:57:28.637 there's a funny joke some sense it doesn't really matter if it was written 00:57:28.720 --> 00:57:32.637 by a human or an AI. But what you don't want and where I think we should really 00:57:32.720 --> 00:57:36.797 draw hard lines is deception. Um and I think that as long as we're in a world 00:57:36.880 --> 00:57:40.317 where you know why why do we build AI systems at all? Right? The reason we 00:57:40.400 --> 00:57:43.437 want to build them is to enhance human lives to make humans be able to do more 00:57:43.520 --> 00:57:47.437 things to have human humans feel more fulfilled. And if we can build AI 00:57:47.520 --> 00:57:50.557 systems that do that, uh, you know, sign me up. 00:57:50.640 --> 00:57:54.317 >> So, the process of language modeling, 00:57:54.400 --> 00:57:56.317 how far do you think it takes us? Let's 00:57:56.400 --> 00:58:00.157 look at movie her. Do you think, uh, a 00:58:00.240 --> 00:58:04.157 dialogue, natural language conversation is formulated by the touring test, for 00:58:04.240 --> 00:58:08.477 example? Do you think that process could be achieved through this kind of 00:58:08.560 --> 00:58:10.797 unsupervised language modeling? So, I 00:58:10.880 --> 00:58:15.677 think the Turing test in its in its real form uh isn't just about language, 00:58:15.760 --> 00:58:18.957 right? It's really about reasoning, too, right? That to really pass the Turing 00:58:19.040 --> 00:58:23.357 test, I should be able to teach calculus to whoever's on the other side and have 00:58:23.440 --> 00:58:26.797 it really understand calculus and be able to, you know, go and and solve new 00:58:26.880 --> 00:58:31.677 calculus problems. And so, I think that to really solve the Turing test, we need 00:58:31.760 --> 00:58:34.637 more than what we're seeing with language models. We need some way of 00:58:34.720 --> 00:58:38.877 plugging in reasoning. Now, how different will that be from what we 00:58:38.960 --> 00:58:42.317 already do? That's an open question, right? It might be that we need some 00:58:42.400 --> 00:58:44.477 sequence of totally radical new ideas. 00:58:44.560 --> 00:58:48.237 Or it might be that we just need to kind of shape our existing systems in a 00:58:48.320 --> 00:58:52.557 slightly different way. But I think that in terms of how far language modeling 00:58:52.640 --> 00:58:56.717 will go, it's already gone way further than many people would have expected, 00:58:56.800 --> 00:58:59.197 right? I think that things like uh and I think there's a lot of really 00:58:59.280 --> 00:59:01.357 interesting angles to poke in terms of 00:59:01.440 --> 00:59:04.717 how much does GPT2 understand physical 00:59:04.800 --> 00:59:08.717 world. like you know you you read a little bit about fire underwater uh in 00:59:08.800 --> 00:59:11.757 in GBT2 so it's like okay maybe it doesn't quite understand what these 00:59:11.840 --> 00:59:16.317 things are um but at the same time I think that you also see various things 00:59:16.400 --> 00:59:19.997 like smoke coming from flame and you know a bunch of these things that GBT2 00:59:20.080 --> 00:59:21.677 it has no body it has no physical 00:59:21.760 --> 00:59:23.757 experience it's just statically read 00:59:23.840 --> 00:59:27.677 data um and I I think that I think that 00:59:27.760 --> 00:59:32.077 the answer is like we don't know yet um and these questions though we're 00:59:32.160 --> 00:59:35.197 starting to be able to actually ask them to physical systems uh that to real 00:59:35.280 --> 00:59:37.117 systems that exist and that's very exciting. 00:59:37.200 --> 00:59:38.477 >> Do you think uh what's your intuition? 00:59:38.560 --> 00:59:42.957 Do you think if you just scale language modeling 00:59:43.040 --> 00:59:47.677 like uh significantly scale that reasoning can emerge from the same exact 00:59:47.760 --> 00:59:49.597 mechanisms? I think it's unlikely that 00:59:49.680 --> 00:59:53.517 if we just scale uh GPT2 that we'll have 00:59:53.600 --> 00:59:57.197 reasoning in in the full-fledged way and I think that there's like you know the 00:59:57.280 --> 01:00:01.117 type signature is a little bit wrong right that like there's something we do 01:00:01.200 --> 01:00:05.197 with that we call thinking right where we spend a lot of compute like a 01:00:05.280 --> 01:00:08.317 variable amount of compute to get to better answers right I think a little 01:00:08.400 --> 01:00:10.557 bit harder I get a better answer and 01:00:10.640 --> 01:00:12.797 that that kind of type signature isn't 01:00:12.880 --> 01:00:17.117 quite encoded in a GBT right GBT will 01:00:17.200 --> 01:00:21.357 kind of like it's spent a long time in it like evolutionary history baking in 01:00:21.440 --> 01:00:24.797 all this information getting very very good at this predictive process and then 01:00:24.880 --> 01:00:26.957 at runtime I just kind of do one forward 01:00:27.040 --> 01:00:29.757 pass and uh and I'm able to generate 01:00:29.840 --> 01:00:34.237 stuff and so you know there might be small tweaks to what we do in order to 01:00:34.320 --> 01:00:37.517 get the type signature right for example well you know it's not really one 01:00:37.600 --> 01:00:40.557 forward pass right you know you generate symbol by symbol and so maybe you 01:00:40.640 --> 01:00:43.917 generate like a whole sequence of of of thoughts and you only keep like the last 01:00:44.000 --> 01:00:47.837 bit or something um but I think that at the very least I would expect you have 01:00:47.920 --> 01:00:52.077 make changes like that. >> Yeah. Yeah. Just exactly how we you said 01:00:52.160 --> 01:00:54.797 think is the process of generating 01:00:54.880 --> 01:00:59.357 thought by thought in the same kind of way like you said keep the last bit the 01:00:59.440 --> 01:01:00.877 thing that we converge towards. 01:01:00.960 --> 01:01:03.357 >> Yep. >> And I think there's there's another 01:01:03.440 --> 01:01:06.317 piece which is which is interesting which is this out of distribution 01:01:06.400 --> 01:01:10.477 generalization right that like thinking somehow lets us do that right that we 01:01:10.560 --> 01:01:13.357 haven't experienced a thing and yet somehow we just kind of keep refining 01:01:13.440 --> 01:01:16.237 our mental model of it. Um this is again 01:01:16.320 --> 01:01:18.877 something that feels tied to whatever 01:01:18.960 --> 01:01:23.597 reasoning is and maybe it's a small tweak to what we do. Maybe it's many 01:01:23.680 --> 01:01:25.357 ideas and will take us many decades. 01:01:25.440 --> 01:01:28.477 >> Yeah. So the the assumption there 01:01:28.560 --> 01:01:33.517 generalization out of distribution is that it's possible to create new new 01:01:33.600 --> 01:01:36.317 ideas. Mhm. >> Uh the you know it's possible that 01:01:36.400 --> 01:01:38.317 nobody's ever created any new ideas and 01:01:38.400 --> 01:01:42.557 then with scaling GPT2 to GPT20 01:01:42.640 --> 01:01:45.357 uh you would you would essentially 01:01:45.440 --> 01:01:50.477 generalize to all possible thoughts humans could have [laughter] 01:01:50.560 --> 01:01:51.757 >> just to play devil's advocate. 01:01:51.840 --> 01:01:55.037 >> Right. Right. I mean how many how many uh new new story ideas have we come up 01:01:55.120 --> 01:01:56.557 with since Shakespeare? Right. 01:01:56.640 --> 01:01:59.277 >> Yeah. Exactly. It's [laughter] it's just 01:01:59.360 --> 01:02:03.997 all different forms of love and drama and so on. Okay. Not sure if you read 01:02:04.080 --> 01:02:06.717 bit of lesson a recent blog post by Ray Sutton. 01:02:06.800 --> 01:02:10.157 >> Yep, I have. >> He basically says uh something that 01:02:10.240 --> 01:02:14.557 echoes some of the ideas that you've been talking about which is uh he says 01:02:14.640 --> 01:02:18.877 the biggest lesson that can be read from 70 years of AI research is that general 01:02:18.960 --> 01:02:21.517 methods that leverage computation are 01:02:21.600 --> 01:02:23.997 ultimately going to uh ultimately win 01:02:24.080 --> 01:02:26.557 out. Uh do you agree with this? So 01:02:26.640 --> 01:02:30.077 basically of and open AI in general 01:02:30.160 --> 01:02:31.837 about the ideas you're exploring about 01:02:31.920 --> 01:02:34.557 coming up with methods whether it's GPT2 01:02:34.640 --> 01:02:37.357 modeling or whether it's uh open AI5 01:02:37.440 --> 01:02:40.957 playing Dota where a general method is 01:02:41.040 --> 01:02:46.157 better than a more fine-tuned expert tuned 01:02:46.240 --> 01:02:49.117 method. >> Yeah. So I I think that well one thing 01:02:49.200 --> 01:02:52.557 that I think was really interesting about the reaction to that blog post was 01:02:52.640 --> 01:02:56.557 that a lot of people have read this as saying that compute is all that matters 01:02:56.640 --> 01:02:59.997 >> and that's a very threatening idea right and I don't think it's a true idea 01:03:00.080 --> 01:03:03.437 either right it's very clear that we have algorithmic ideas that have been 01:03:03.520 --> 01:03:07.437 very important for making progress and to really build AGI you want to push as 01:03:07.520 --> 01:03:10.557 far as you can on the computational scale and you want to push as far as you 01:03:10.640 --> 01:03:14.797 can on human human ingenuity and so I think you need both but I think the way 01:03:14.880 --> 01:03:17.997 that you phrase the question is actually very good right that It's really about 01:03:18.080 --> 01:03:20.477 what kind of ideas should we be striving 01:03:20.560 --> 01:03:23.037 for. And absolutely, if you can find a 01:03:23.120 --> 01:03:25.757 scalable idea, you pour more comput into 01:03:25.840 --> 01:03:30.477 it, you pour more data into it, it gets better. Like that's that's the real holy 01:03:30.560 --> 01:03:34.317 grail. And so I think that uh that that the answer to to the question I think is 01:03:34.400 --> 01:03:38.237 is yes. Um that that that's really how we think about it. that part of why 01:03:38.320 --> 01:03:42.557 we're excited about the power of deep learning, the potential for building AGI 01:03:42.640 --> 01:03:44.477 is because we look at the systems that 01:03:44.560 --> 01:03:47.037 exist in the most successful AI systems 01:03:47.120 --> 01:03:51.517 and we realize that you scale those up, they're going to work better. And I 01:03:51.600 --> 01:03:54.797 think that that scalability is something that really gives us hope for being able 01:03:54.880 --> 01:03:57.197 to build transformative systems. So I'll 01:03:57.280 --> 01:04:02.317 tell you this a partially an emotional you know a thing that a response that 01:04:02.400 --> 01:04:06.717 people often have if comput is so important for state-of-the-art performance 01:04:06.800 --> 01:04:10.157 you know individual developers maybe a 13-year-old sitting somewhere in Kansas 01:04:10.240 --> 01:04:13.997 or something like that you know they're sitting they might not even have a GPU 01:04:14.080 --> 01:04:16.717 and or may have a single GPU a 1080 or 01:04:16.800 --> 01:04:21.837 something like that and there's this feeling like well how can I possibly 01:04:21.920 --> 01:04:23.677 compete or contribute to this world of 01:04:23.760 --> 01:04:26.077 AI if uh scale is so important 01:04:26.160 --> 01:04:28.877 important. So for if you can comment on 01:04:28.960 --> 01:04:31.037 that and in general do you think we need 01:04:31.120 --> 01:04:33.357 to also in the future focus on uh 01:04:33.440 --> 01:04:36.877 democratizing compute resources more 01:04:36.960 --> 01:04:39.037 more or as much as we democratize the 01:04:39.120 --> 01:04:40.877 algorithms well so the way that I think 01:04:40.960 --> 01:04:44.397 about it is that there's this space of 01:04:44.480 --> 01:04:48.237 of possible progress right there's a space of ideas and sort of systems that 01:04:48.320 --> 01:04:52.237 that will work that will move us forward and there's a portion of that space and 01:04:52.320 --> 01:04:55.357 to some extent an increasingly significant portion of that space that 01:04:55.440 --> 01:05:00.317 does just require massive compute resources and for that that I think that 01:05:00.400 --> 01:05:04.477 that the answer is kind of clear and that part of why we have the structure 01:05:04.560 --> 01:05:07.757 that that we do is because we think it's really important to be pushing the scale 01:05:07.840 --> 01:05:12.157 and to be you know building these large clusters and systems but there's another 01:05:12.240 --> 01:05:15.357 part portion of the space that isn't about the large scale compute that are 01:05:15.440 --> 01:05:19.677 these ideas that and again I think that for the ideas to really be impactful and 01:05:19.760 --> 01:05:23.677 really shine that they should be ideas that if you scale them up would work way 01:05:23.760 --> 01:05:27.677 better than they do at small scale um but that you can discover them 01:05:27.760 --> 01:05:29.917 without massive computational resources. 01:05:30.000 --> 01:05:32.877 And if you look at the history of of recent developments, you think about 01:05:32.960 --> 01:05:36.957 things like the GAN or the VAE that these are ones that I think you could 01:05:37.040 --> 01:05:40.557 come up with them uh without having and you know in practice people did come up 01:05:40.640 --> 01:05:43.757 with with them without having massive massive computational resources, 01:05:43.840 --> 01:05:45.277 >> right? I just talked to Ian Goodfellow, 01:05:45.360 --> 01:05:48.797 but the thing is the init initial GAN 01:05:48.880 --> 01:05:51.597 produced pretty terrible results, right? 01:05:51.680 --> 01:05:55.037 So only because it was in a very specific it was only because they're 01:05:55.120 --> 01:05:58.797 smart enough to know that this is quite surprising they can generate anything 01:05:58.880 --> 01:06:01.517 that they know. I do you see a world or 01:06:01.600 --> 01:06:03.757 is that too optimistic and dreamer like 01:06:03.840 --> 01:06:06.957 to imagine that the compute resources 01:06:07.040 --> 01:06:12.317 are something that's owned by governments and provided as a utility. 01:06:12.400 --> 01:06:16.157 >> Actually to some extent this this question reminds me of of a blog post 01:06:16.240 --> 01:06:19.997 from one of my former professors at Harvard this guy Matt Matt Welsh who was 01:06:20.080 --> 01:06:23.277 a systems professor. I remember sitting in his tenure talk, right? And uh you 01:06:23.360 --> 01:06:25.197 know that that he had literally just 01:06:25.280 --> 01:06:27.277 gotten tenure. He went to Google for the 01:06:27.360 --> 01:06:30.797 summer uh and uh then decided he wasn't 01:06:30.880 --> 01:06:33.277 going back to academia, right? And that 01:06:33.360 --> 01:06:38.157 kind of in his blog post he makes this point that look as a systems researcher 01:06:38.240 --> 01:06:41.117 that I come up with these cool system ideas, right? and I kind of build a 01:06:41.200 --> 01:06:45.357 little proof of concept and the best thing I can hope for is that the people 01:06:45.440 --> 01:06:49.757 at Google or Yahoo um which was around at the time uh [laughter] 01:06:49.840 --> 01:06:53.197 will implement it and like actually make it work at scale, right? That's like the 01:06:53.280 --> 01:06:55.677 dream for me, right? I build the little thing and they turn into the big thing 01:06:55.760 --> 01:07:00.317 that's actually working. >> Um and uh for him he said I'm done with 01:07:00.400 --> 01:07:03.437 that. I want to be the person who's who's actually doing building and and 01:07:03.520 --> 01:07:07.117 and deploying. And I think that there's a similar dichotomy here, right? I think 01:07:07.200 --> 01:07:11.277 that there are people who really actually find value and I think it is a 01:07:11.360 --> 01:07:14.797 valuable thing to do to be the person who produces those ideas, right? Who 01:07:14.880 --> 01:07:18.717 builds the proof of concept and yeah, you don't get to generate the coolest 01:07:18.800 --> 01:07:21.517 possible GAN images, but you invented the GAN, 01:07:21.600 --> 01:07:25.037 >> right? And so that there's that there's there's a real trade-off there and I 01:07:25.120 --> 01:07:28.157 think that's a very personal choice, but I think there's value in both sides. So 01:07:28.240 --> 01:07:31.837 do you think uh creating AGI something 01:07:31.920 --> 01:07:36.237 or some new models uh would we would see 01:07:36.320 --> 01:07:40.477 echoes of the brilliance even at the prototype level. So you would be able to 01:07:40.560 --> 01:07:45.277 develop those ideas without scale the initial uh seeds. Well, so take take a 01:07:45.360 --> 01:07:48.797 look at uh you know, I always like to look at at examples that that exist, 01:07:48.880 --> 01:07:53.437 right? Look at real precedent. And so take a look at the June uh 2018 model 01:07:53.520 --> 01:07:56.397 that we released that we scaled up to turn into GPT2. 01:07:56.480 --> 01:08:00.637 >> And you can see that at small scale it set some records, right? This was, you 01:08:00.720 --> 01:08:04.477 know, the original GPT. We actually had some some cool generations. They weren't 01:08:04.560 --> 01:08:08.877 nearly as as as amazing and really stunning as the GPT2 ones, but it was 01:08:08.960 --> 01:08:12.557 promising. It was interesting. And so I think it is the case that with a lot of 01:08:12.640 --> 01:08:16.637 these ideas that you see promise at small scale. But there is an asterisk 01:08:16.720 --> 01:08:21.037 here, a very big asterisk, which is sometimes we see 01:08:21.120 --> 01:08:25.117 behaviors that emerge that are qualitatively different from anything we 01:08:25.200 --> 01:08:30.397 saw at small scale and that the original inventor of whatever algorithm looks at 01:08:30.480 --> 01:08:32.717 and says, I didn't think it could do that. 01:08:32.800 --> 01:08:34.877 >> This is what we saw in Dota. Right? So 01:08:34.960 --> 01:08:36.637 PO was was created by John Scholman 01:08:36.720 --> 01:08:39.757 who's a researcher here and uh and with 01:08:39.840 --> 01:08:44.557 with Dota we basically just ran PO at massive massive scale and uh you know 01:08:44.640 --> 01:08:47.197 there's some tweaks in order to in order to make it work but fundamentally it's 01:08:47.280 --> 01:08:51.037 PO at the core and we were able to get 01:08:51.120 --> 01:08:55.517 this long-term planning these these behaviors to really play out uh on a 01:08:55.600 --> 01:08:59.517 time scale that we just thought was not possible. Um and John looked at that and 01:08:59.600 --> 01:09:02.317 was like I didn't think it could do that. That's what happens when you're at 01:09:02.400 --> 01:09:05.677 three orders of magnitude more scale [laughter] than you test it at. 01:09:05.760 --> 01:09:07.277 >> Yeah. But it still has the same flavors 01:09:07.360 --> 01:09:11.357 of um you know u at least echoes of the 01:09:11.440 --> 01:09:14.077 expected billions. Uh although I suspect 01:09:14.160 --> 01:09:16.877 with GPT scaled more and more you might 01:09:16.960 --> 01:09:19.997 get surprising things. Uh so yeah, 01:09:20.080 --> 01:09:21.757 you're right. It's it's interesting. 01:09:21.840 --> 01:09:24.237 It's it's difficult to see how far an 01:09:24.320 --> 01:09:29.037 idea will go when it's scaled. It's an open question. Well, so to that point 01:09:29.120 --> 01:09:32.157 with with Dota and PO, like I mean here's here's a very concrete one, 01:09:32.240 --> 01:09:34.957 right? Is like um it's actually one thing that's very surprising about Dota 01:09:35.040 --> 01:09:38.797 that I think people don't really pay that much attention to is the degree of 01:09:38.880 --> 01:09:42.957 generalization out of distribution that happens, right? That you have this AI 01:09:43.040 --> 01:09:47.597 that's trained against other bots for its entirety the entirety of it existence. 01:09:47.680 --> 01:09:49.197 >> Sorry to take a step back. Can you can 01:09:49.280 --> 01:09:53.757 you talk through in in you know a story 01:09:53.840 --> 01:09:56.477 of Dota a story of uh leading up to Open 01:09:56.560 --> 01:10:01.677 AI5? Y and uh that passed and what was the process of selfplay and so on of 01:10:01.760 --> 01:10:04.957 training on >> yeah so with Dota >> what is Dota 01:10:05.040 --> 01:10:09.197 >> yeah Dota is a complex video game and we started training we started trying to 01:10:09.280 --> 01:10:13.277 solve Dota because we felt like this was a step towards the real world relative 01:10:13.360 --> 01:10:16.877 to other games like chess or go right those very cerebral games where you just 01:10:16.960 --> 01:10:20.237 kind of have this board very discreet moves Dota starts to be much more 01:10:20.320 --> 01:10:23.837 continuous time that you have this huge variety of different actions that you 01:10:23.920 --> 01:10:27.837 have a 45minute game with all these different units And uh it's got a lot of 01:10:27.920 --> 01:10:31.117 messiness to it uh that that really hasn't been captured by by previous 01:10:31.200 --> 01:10:35.437 games. And famously, all of the hard-coded bots for Dota were terrible, 01:10:35.520 --> 01:10:37.837 right? It's just impossible to write anything good for it because it's so 01:10:37.920 --> 01:10:41.837 complex. And so this seemed like a really good place to push what's the 01:10:41.920 --> 01:10:45.597 state-of-the-art in in reinforcement learning. And so we started by focusing 01:10:45.680 --> 01:10:49.117 on the one versus one version of the game and uh and and were able to to 01:10:49.200 --> 01:10:52.637 solve that. We're able to beat the world champions and that the that the the 01:10:52.720 --> 01:10:56.237 learning, you know, the the skill curve was this crazy exponential, right? And 01:10:56.320 --> 01:10:59.837 it was like constantly we were just scaling up that we were fixing bugs and 01:10:59.920 --> 01:11:02.637 you know that you look at the at the skill curve and it was really a very 01:11:02.720 --> 01:11:06.157 very smooth one. This is actually really interesting to see how that like human 01:11:06.240 --> 01:11:09.997 iteration loop yielded very steady exponential progress. 01:11:10.080 --> 01:11:13.917 >> And uh to one one side note first of all it's an exceptionally popular video 01:11:14.000 --> 01:11:18.797 game. The side effect is that there's a lot of incredible human experts at that 01:11:18.880 --> 01:11:22.877 video game. So the benchmark that you're trying to reach is is very high. and the 01:11:22.960 --> 01:11:26.877 other. Can you talk about the approach that was used initially and throughout 01:11:26.960 --> 01:11:29.357 of training these uh agents to play this game? 01:11:29.440 --> 01:11:33.357 >> Yep. And so the approach that we used is selfplay. And so you have two agents, 01:11:33.440 --> 01:11:37.197 they don't know anything. They battle each other. They discover something a 01:11:37.280 --> 01:11:40.317 little bit good and now they both know it and they just get better and better 01:11:40.400 --> 01:11:45.357 and better without bound. And that's a really powerful idea, right? Uh that we 01:11:45.440 --> 01:11:48.717 then went from the one versus one version of the game and scaled up to 01:11:48.800 --> 01:11:51.677 five versus five, right? So you think about kind of like with basketball where 01:11:51.760 --> 01:11:55.037 you have this like team sport and you need to do all this coordination. Um and 01:11:55.120 --> 01:11:58.397 we're able to push the same idea, the 01:11:58.480 --> 01:12:02.317 same selfplay uh to to to really get to 01:12:02.400 --> 01:12:07.517 the professional level at the full five versus five version of the game. And uh 01:12:07.600 --> 01:12:11.917 and and the things I think are really interesting here is that these agents in 01:12:12.000 --> 01:12:14.957 some ways they're almost like an insect-l like intelligence, right? Where 01:12:15.040 --> 01:12:17.437 you know there's they have a lot in common with how an insect is trained, 01:12:17.520 --> 01:12:20.397 right? insect kind of lives in this environment for a very long time or you 01:12:20.480 --> 01:12:23.597 know the the ancestors of this insect have been around for a long time and had 01:12:23.680 --> 01:12:28.157 a lot of experience that gets baked into into into this agent and you know it's 01:12:28.240 --> 01:12:31.437 not really smart in the sense of a human right it's not able to go and learn 01:12:31.520 --> 01:12:34.877 calculus but it's able to navigate its environment extremely well it's able to 01:12:34.960 --> 01:12:39.437 handle unexpected things in environment that's never seen before pretty well um 01:12:39.520 --> 01:12:42.877 and we see the same sort of thing with our Dota bots right that they're able to 01:12:42.960 --> 01:12:46.637 in within this game they're able to play against humans which is something that 01:12:46.720 --> 01:12:50.077 never existed in its evolutionary environment, totally different play 01:12:50.160 --> 01:12:53.917 styles from humans versus the bots, and yet it's able to handle it extremely 01:12:54.000 --> 01:12:56.557 well. And that's something I think was 01:12:56.640 --> 01:12:59.277 very surprising to us was something that 01:12:59.360 --> 01:13:04.557 doesn't really emerge from what we've seen with PO at smaller scale, right? 01:13:04.640 --> 01:13:07.517 And the kind of scale we're running the stuff at was uh you know, like let's say 01:13:07.600 --> 01:13:11.757 like a 100,000 CPU cores running with like hundreds of GPUs. uh it was 01:13:11.840 --> 01:13:14.157 probably about uh you know like you know 01:13:14.240 --> 01:13:16.877 something like hundreds of of years of 01:13:16.960 --> 01:13:19.037 experience going into this bot every 01:13:19.120 --> 01:13:22.957 single real day and so that scale is 01:13:23.040 --> 01:13:26.317 massive and we start to see very different kinds of behaviors out of the 01:13:26.400 --> 01:13:28.237 algorithms that we all know and love. 01:13:28.320 --> 01:13:31.037 >> Dota you mentioned beat the world expert 01:13:31.120 --> 01:13:35.757 one v one and then uh you didn't weren't 01:13:35.840 --> 01:13:38.557 able to win 5v5 this year. Yeah. 01:13:38.640 --> 01:13:43.837 >> At the best players in the world. Uh so what's what's the comeback story? What's 01:13:43.920 --> 01:13:47.277 first of all talk through that exceptionally exciting event and uh 01:13:47.360 --> 01:13:50.557 what's what's the the following months and this year look like? 01:13:50.640 --> 01:13:53.677 >> Yeah. Yeah. So well one thing that that's interesting is that uh you know 01:13:53.760 --> 01:13:58.157 we lose all the time >> because we play here. So the Dota team 01:13:58.240 --> 01:14:02.797 at OpenAI, we we played the bot against better players than our system all the 01:14:02.880 --> 01:14:06.237 time or at least we we we used to, right? Like you know that the the first 01:14:06.320 --> 01:14:10.077 time we lost publicly was we went up on stage at the International and we played 01:14:10.160 --> 01:14:13.197 against some of the best teams in the world. Um and we ended up losing both 01:14:13.280 --> 01:14:17.037 games, but we gave them a run for their money, right? That both games were kind 01:14:17.120 --> 01:14:20.877 of 30 minutes, 25 minutes and they went back and forth, back and forth, back and 01:14:20.960 --> 01:14:25.357 forth. And so I think that really shows that we're at the professional level. 01:14:25.440 --> 01:14:28.637 Um, and that kind of looking at those games, we think that the coin could have 01:14:28.720 --> 01:14:31.357 gone a different direction and we could have could have had some wins. And that 01:14:31.440 --> 01:14:34.477 that was actually very encouraging for us. Um, and you know, it's interesting 01:14:34.560 --> 01:14:38.797 because the international was at a fixed time, right? So we we knew exactly what 01:14:38.880 --> 01:14:41.757 day we were going to be playing and we pushed as far as we could, as fast as we 01:14:41.840 --> 01:14:45.917 could. Two weeks later, we had a bot that had an 80% win rate versus the one 01:14:46.000 --> 01:14:49.437 that played at TI. Um, so the March of Progress, uh, you know, you should think 01:14:49.520 --> 01:14:53.437 of as a snapshot rather than as an end state. Um, and so in fact, we'll we'll 01:14:53.520 --> 01:14:57.437 be announcing our, uh, our finals, uh, pretty soon. I actually think that, uh, 01:14:57.520 --> 01:15:01.917 we'll announce our final match, uh, prior to, uh, this podcast being 01:15:02.000 --> 01:15:05.517 released. Okay. So, there should be, uh, we'll be playing we'll be playing, uh, 01:15:05.600 --> 01:15:10.077 against the the world champions. And you know for us it's really less about like 01:15:10.160 --> 01:15:14.957 that the way that we think about what what's upcoming is the final milestone 01:15:15.040 --> 01:15:19.037 the final competitive milestone for the project right that our goal in all of 01:15:19.120 --> 01:15:22.557 this isn't really about beating humans at Dota 01:15:22.640 --> 01:15:24.157 >> our goal is to push the state-of-the-art 01:15:24.240 --> 01:15:27.197 in reinforcement learning and we've done that right and we've actually learned a 01:15:27.280 --> 01:15:30.957 lot from our system and that we have uh you know I think a lot of exciting next 01:15:31.040 --> 01:15:34.077 steps that we want to take and so you know kind of the final showcase of what 01:15:34.160 --> 01:15:37.597 we built we're going to do this match um but for us it's not really the success 01:15:37.680 --> 01:15:41.757 or failure um to see you know do do we have the coin flip go in our direction 01:15:41.840 --> 01:15:45.837 or against >> where do you see the field of deep 01:15:45.920 --> 01:15:46.977 learning heading in the next 01:15:47.060 --> 01:15:49.597 [clears throat] few years where do you 01:15:49.680 --> 01:15:52.957 see the work and reinforcement learning 01:15:53.040 --> 01:15:56.157 perhaps heading and uh 01:15:56.240 --> 01:15:59.037 more specifically with open AI all the 01:15:59.120 --> 01:16:03.757 exciting projects that you're working on what does 2019 hold for you 01:16:03.840 --> 01:16:06.957 >> massive scale >> scale >> I will put an asterisk on and just say, 01:16:07.040 --> 01:16:10.237 you know, I think that it's about ideas plus scale. You need both. 01:16:10.320 --> 01:16:12.637 >> So that's a really uh good u point. So 01:16:12.720 --> 01:16:16.397 the question in terms of ideas, you have 01:16:16.480 --> 01:16:19.677 a lot of projects that are exploring 01:16:19.760 --> 01:16:22.397 different areas of intelligence and uh 01:16:22.480 --> 01:16:26.797 the question is when you when you think of scale, do you think about growing the 01:16:26.880 --> 01:16:30.957 scale of those individual projects or do you think about adding new projects? And 01:16:31.040 --> 01:16:35.437 uh sorry to and if you're thinking of adding new projects or if you look at 01:16:35.520 --> 01:16:39.517 the past what's the process of coming up with new project new ideas. 01:16:39.600 --> 01:16:43.597 >> Yep. Uh so we really have a life cycle of project here. Uh so we start with a 01:16:43.680 --> 01:16:46.877 few people uh just working on a small scale idea and language is actually a 01:16:46.960 --> 01:16:50.237 very good example of this that was really you know one person here who was 01:16:50.320 --> 01:16:53.997 pushing on language for a long time. I mean then you get signs of life right 01:16:54.080 --> 01:16:58.717 and so this is like let's say uh you know with with the original GPT we had 01:16:58.800 --> 01:17:02.157 something that was interesting and we said okay it's time to scale this right 01:17:02.240 --> 01:17:05.517 it's time to put more people on it put more computational resources behind it 01:17:05.600 --> 01:17:09.357 and uh and then we just kind of keep pushing and keep pushing and the end 01:17:09.440 --> 01:17:12.557 state is something that looks like Dota or robotics where you have a large team 01:17:12.640 --> 01:17:16.557 of you know 10 or 15 people uh that are running things at very large scale uh 01:17:16.640 --> 01:17:20.957 and that you're able to really have material engineering uh and and uh and 01:17:21.040 --> 01:17:25.517 and you sort of machine learning science coming together to make systems that 01:17:25.600 --> 01:17:28.637 work uh and get material results that just would have been impossible 01:17:28.720 --> 01:17:32.637 otherwise. Um so we do that whole life cycle. We've done it a number of times 01:17:32.720 --> 01:17:37.037 uh you know typically end to end. It's probably two uh two years or so uh to do 01:17:37.120 --> 01:17:39.757 it. You know the organization's been around for three years. So maybe we'll 01:17:39.840 --> 01:17:41.277 find that we also have longer life cycle 01:17:41.360 --> 01:17:44.157 projects. Um but you know we we uh we'll 01:17:44.240 --> 01:17:48.557 work up to those. We have uh so so one one team that we were actually just 01:17:48.640 --> 01:17:51.997 starting Illy and I are are kicking off a new team called the reasoning team and 01:17:52.080 --> 01:17:53.837 that this is to really try to tackle how 01:17:53.920 --> 01:17:59.357 do you get neural networks to reason and uh we think that this will be a longterm 01:17:59.440 --> 01:18:01.597 project uh it's one that we're very excited about. 01:18:01.680 --> 01:18:05.677 >> Uh in terms of reasoning super exciting topic 01:18:05.760 --> 01:18:08.557 what do you what kind of benchmarks 01:18:08.640 --> 01:18:11.597 uh what kind of tests of reasoning do 01:18:11.680 --> 01:18:13.677 you envision? What what would if you sat 01:18:13.760 --> 01:18:16.797 back with whatever drink and you would 01:18:16.880 --> 01:18:21.037 be impressed that this system is able to do something what would that look like? 01:18:21.120 --> 01:18:25.917 uh theorem proving >> theorem proving so some kind of logic 01:18:26.000 --> 01:18:29.117 and especially mathematical logic >> I think so right and I think that 01:18:29.200 --> 01:18:31.917 there's there's there's kind of other problems that are dual to theorem 01:18:32.000 --> 01:18:35.517 proving in particular um you know you think about uh programming uh you think 01:18:35.600 --> 01:18:37.757 about even like security analysis of of 01:18:37.840 --> 01:18:43.037 code um that these all kind of capture the same sorts of core reasoning and and 01:18:43.120 --> 01:18:46.637 being able to do some out of distribution generalization 01:18:46.720 --> 01:18:50.797 >> it would be quite exciting if openai reasoning team was able to prove prove 01:18:50.880 --> 01:18:52.957 that P equals NP. That would be very nice. 01:18:53.040 --> 01:18:56.637 >> Uh it would be very very very exciting especially uh if it turns out the P 01:18:56.720 --> 01:18:58.897 equals NP. That'll be interesting too. 01:18:58.980 --> 01:19:03.357 [laughter] >> I just it would be uh ironic and humorous. 01:19:03.440 --> 01:19:04.157 >> Yep. 01:19:04.240 --> 01:19:07.677 >> Uh so what problem stands out to you as 01:19:07.760 --> 01:19:10.717 uh the most uh exciting and challenging 01:19:10.800 --> 01:19:15.197 impactful to the work for us as a community in general and for Open AI 01:19:15.280 --> 01:19:18.717 this year? You mentioned reasoning. I think that's that's a heck of a problem. 01:19:18.800 --> 01:19:20.637 >> Yeah. So I think reasoning is an important one. And I think it's going to 01:19:20.720 --> 01:19:23.837 be hard to get good results in 2019. Um, you know, again, just like we think 01:19:23.920 --> 01:19:28.077 about the life cycle takes time. Um, I think for 2019, language modeling seems 01:19:28.160 --> 01:19:31.277 to be kind of on that ramp, right? It's at the point that we have a technique 01:19:31.360 --> 01:19:35.357 that works. We want to scale 100x, thousandx, see what happens. 01:19:35.440 --> 01:19:38.797 >> Awesome. Do you think we're living in a simulation? 01:19:38.880 --> 01:19:42.477 >> Uh, I think it's I think it's hard to have a a real opinion about it. Uh, you 01:19:42.560 --> 01:19:45.837 know, it's actually interesting. I separate out things that I think can 01:19:45.920 --> 01:19:49.677 have like, you know, yield materially different predictions about the world. 01:19:49.760 --> 01:19:53.597 um from ones that are just kind of you know fun to speculate about and I I kind 01:19:53.680 --> 01:19:55.517 of view simulation as more like is there 01:19:55.600 --> 01:20:01.037 a flying teapot between Mars and Jupiter like maybe but it's a little bit hard to 01:20:01.120 --> 01:20:02.477 know what that would mean for my life. 01:20:02.560 --> 01:20:05.517 >> So there is something actionable so some 01:20:05.600 --> 01:20:08.477 of the best work open has done is in the 01:20:08.560 --> 01:20:13.197 field of reinforcement learning and uh some of the success of reinforcement 01:20:13.280 --> 01:20:15.277 learning come from being able to 01:20:15.360 --> 01:20:20.237 simulate the problem you're trying to solve. So it do you have a hope for 01:20:20.320 --> 01:20:23.037 reinforcement for the future of reinforcement learning and for the 01:20:23.120 --> 01:20:26.317 future of simulation like whether it's we're talking about autonomous vehicles 01:20:26.400 --> 01:20:29.357 or any kind of system? Do you see that 01:20:29.440 --> 01:20:31.597 scaling or we'll be able to simulate 01:20:31.680 --> 01:20:34.317 systems and and hence be able to create 01:20:34.400 --> 01:20:39.117 a simulator that echoes our real world and uh proving once and for all even 01:20:39.200 --> 01:20:42.317 though you're denying it that we're living in a simulation. 01:20:42.400 --> 01:20:44.397 >> I feel like there's two separate questions, right? So, you know, kind 01:20:44.480 --> 01:20:47.437 kind of at the core there of like can we can we use simulation for self-driving 01:20:47.520 --> 01:20:49.917 cars? Um, take a look at our robotic 01:20:50.000 --> 01:20:54.557 system dactyl, right? That was trained in simulation using the Dota system in 01:20:54.640 --> 01:20:59.037 fact and it transfers to a physical robot. And I think everyone looks at our 01:20:59.120 --> 01:21:01.437 Dota system, they're like, okay, it's just a game. How are you ever going to 01:21:01.520 --> 01:21:04.397 escape to the real world? And the answer is, well, we did it with a physical 01:21:04.480 --> 01:21:07.837 robot that no one can program. And so, I think the answer is simulation goes a 01:21:07.920 --> 01:21:11.437 lot further than you think um if you apply the right techniques to it. Um, 01:21:11.520 --> 01:21:14.237 now there's a question of, you know, are the beings in that simulation going to 01:21:14.320 --> 01:21:17.597 going to wake up and and have consciousness? Um, I think that one 01:21:17.680 --> 01:21:21.117 seems a lot a lot harder to to again reason about. I think that, you know, 01:21:21.200 --> 01:21:24.797 you really should think about like where where exactly does human consciousness 01:21:24.880 --> 01:21:28.157 come from and our own self-awareness and, you know, is it just that like once 01:21:28.240 --> 01:21:30.477 you have like a complicated enough neural net that you have to worry about 01:21:30.560 --> 01:21:35.037 the the agents feeling pain? Um, and uh, you know, I think there's like 01:21:35.120 --> 01:21:38.557 interesting speculation to to do there, but uh, but you know, again, I think 01:21:38.640 --> 01:21:41.597 it's it's a little bit hard to know for sure. Well, let me just keep with the 01:21:41.680 --> 01:21:43.917 speculation. Do you think uh to create 01:21:44.000 --> 01:21:46.237 intelligence, general intelligence, you 01:21:46.320 --> 01:21:50.397 need one consciousness and two a body. 01:21:50.480 --> 01:21:53.517 Do you think any of those elements are needed or is intelligence something 01:21:53.600 --> 01:21:55.757 that's that's orthogonal to those? 01:21:55.840 --> 01:21:59.277 >> I'll stick to the the kind of like the the non- grand answer first, right? So 01:21:59.360 --> 01:22:02.637 the non-grand answer is just to look at, you know, what are we already making 01:22:02.720 --> 01:22:05.917 work? You look at GPT2, a lot of people would have said that to even get these 01:22:06.000 --> 01:22:09.277 kinds of results, you need real world experience. You need a body. You need 01:22:09.360 --> 01:22:12.397 grounding. How are you supposed to reason about any of these things? How 01:22:12.480 --> 01:22:14.957 are you supposed to like even kind of know about smoke and fire and those 01:22:15.040 --> 01:22:16.957 things if you've never experienced them? 01:22:17.040 --> 01:22:19.517 >> And GPD2 shows that you can actually go 01:22:19.600 --> 01:22:22.397 way further than that kind of reasoning 01:22:22.480 --> 01:22:27.277 would predict. So I think that that that in terms of do 01:22:27.360 --> 01:22:30.077 we need consciousness, do we need a body? It seems the answer is probably 01:22:30.160 --> 01:22:33.277 not, right? That we could probably just continue to push kind of the systems we 01:22:33.360 --> 01:22:37.757 have, they already feel general. um they're not as competent or as general 01:22:37.840 --> 01:22:40.077 or able to learn as quickly as an AGI 01:22:40.160 --> 01:22:41.677 would but you know they're at least like 01:22:41.760 --> 01:22:45.357 kind of protoagi in some way and they 01:22:45.440 --> 01:22:47.917 don't need any of those things. Now, now 01:22:48.000 --> 01:22:52.397 let's move to the grand answer, which is, you know, if are are neural nets 01:22:52.480 --> 01:22:56.397 nets conscious already, would we ever know? How can we tell, right? And, you 01:22:56.480 --> 01:23:00.397 know, here here's where the speculation starts to become become, you know, at 01:23:00.480 --> 01:23:04.397 least interesting or fun and maybe a little bit disturbing depending on on 01:23:04.480 --> 01:23:08.397 where you take it. But it certainly seems that when we think about animals 01:23:08.480 --> 01:23:11.437 that there's some continuum of of of consciousness. You know, my cat, I 01:23:11.520 --> 01:23:15.117 think, is uh is conscious in some way, right? Uh you know, not as conscious as 01:23:15.200 --> 01:23:18.077 a human. And you could imagine that you could build like a little consciousness 01:23:18.160 --> 01:23:20.637 meter, right? You point at a cat, it gives you a little reading. Point at a 01:23:20.720 --> 01:23:23.597 human, gives you much bigger reading. 01:23:23.680 --> 01:23:27.597 What would happen if you pointed one of those at a Dota neural net? And if 01:23:27.680 --> 01:23:31.997 you're training in this massive simulation, do the neural nets feel pain? 01:23:32.080 --> 01:23:36.637 You know, it becomes pretty hard to know that the answer is no. Um, and it 01:23:36.720 --> 01:23:40.637 becomes pretty hard to to really think about what that would mean if the answer 01:23:40.720 --> 01:23:44.317 were yes. And it's very possible, you know, for 01:23:44.400 --> 01:23:48.237 example, you could imagine that maybe the reason that humans are have 01:23:48.320 --> 01:23:52.157 consciousness is because it's a it's a convenient computational shortcut, 01:23:52.240 --> 01:23:55.597 right? If you think about it, if you have a being that wants to avoid pain, 01:23:55.680 --> 01:23:59.037 which seems pretty important to survive in this environment um and wants to 01:23:59.120 --> 01:24:03.117 like, you know, eat food um then that maybe the best way of doing it is to 01:24:03.200 --> 01:24:04.557 have a being that's conscious, right? 01:24:04.640 --> 01:24:07.837 That you know, in order to succeed in the environment, you need to have those 01:24:07.920 --> 01:24:10.877 properties. And how are you supposed to implement them? and maybe this this 01:24:10.960 --> 01:24:14.717 consciousness's way of doing that. If that's true, then actually maybe we 01:24:14.800 --> 01:24:17.757 should expect that really competent reinforcement learning agents will also 01:24:17.840 --> 01:24:21.677 have consciousness. But you know that's a big if and I think there are a lot of 01:24:21.760 --> 01:24:24.157 other arguments that you can make in other directions. 01:24:24.240 --> 01:24:27.917 >> I think that's a really interesting idea that even GPT2 has some degree of 01:24:28.000 --> 01:24:30.557 consciousness. That's something uh it's 01:24:30.640 --> 01:24:34.957 actually not as crazy to think about as useful to think about as as we think 01:24:35.040 --> 01:24:37.997 about what it means to create intelligence of a dog, intelligence of a 01:24:38.080 --> 01:24:39.837 cat 01:24:39.920 --> 01:24:42.477 and the intelligence of a human. So last 01:24:42.560 --> 01:24:45.277 question, do you think 01:24:45.360 --> 01:24:48.637 we will ever fall in love like in the 01:24:48.720 --> 01:24:52.797 movie [snorts] her with an artificial intelligence system or an artificial 01:24:52.880 --> 01:24:55.837 intelligence system falling in love with a human? 01:24:55.920 --> 01:24:58.637 >> I hope so. if there's any better way to 01:24:58.720 --> 01:25:01.997 end it is on uh love. So, Greg, thanks 01:25:02.080 --> 01:25:06.680 so much for talking today. >> Thank you for having me. 01:25:15.610 --> 01:25:17.630 [music]