WEBVTT 00:00:02.120 --> 00:00:05.437 So, Greg, uh thank you for coming back here. Um 00:00:05.520 --> 00:00:08.597 I don't think we ever charged you for rent. So, uh maybe we'll send you an 00:00:08.680 --> 00:00:12.517 invoice later. But, uh Greg, you've been part of like two really spectacular 00:00:12.600 --> 00:00:14.677 companies, Stripe as employee number 00:00:14.760 --> 00:00:17.397 four, and then the first CTO. I just 00:00:17.480 --> 00:00:19.757 recently heard that they process 1.6 00:00:19.840 --> 00:00:22.917 billion, sorry, 1.6% of the 00:00:23.000 --> 00:00:25.437 global GDP. You must be proud of that. 00:00:25.520 --> 00:00:29.837 That's amazing. You must be even more proud of the fact that OpenAI has almost 00:00:29.920 --> 00:00:34.397 a billion or maybe more than a billion uh in weekly active users at this point. 00:00:34.480 --> 00:00:35.917 I mean, it's all it's all very exciting. 00:00:36.000 --> 00:00:39.837 It shows you what technology can do. And uh you're not just co-founder and 00:00:39.920 --> 00:00:42.237 president, but you're also chief builder. 00:00:42.320 --> 00:00:45.437 At OpenAI, I heard that that was one of your titles. I'm not sure if it was ever 00:00:45.520 --> 00:00:49.317 an official title, but I've been called many things. Let's just say that. 00:00:49.400 --> 00:00:53.277 Well, you have a audience of great builders here, so we'll start from all 00:00:53.360 --> 00:00:57.597 the way at the bottom of the stack. You OpenAI has multiple stacks of the 00:00:57.680 --> 00:01:01.717 business, one of which is compute, and you guys have been very aggressive, very 00:01:01.800 --> 00:01:05.517 aggressive on securing compute. Why is that? 00:01:05.600 --> 00:01:08.277 Well, in many ways, we have a very simple business. 00:01:08.360 --> 00:01:12.277 We buy, rent, build compute, 00:01:12.360 --> 00:01:16.517 and we resell it at a margin. That's it. As long as the margin's 00:01:16.600 --> 00:01:20.837 positive, then you want to scale it because the demand for 00:01:20.920 --> 00:01:26.117 solving problems, the demand for intelligence, that's unlimited. 00:01:26.200 --> 00:01:29.317 And the AIs that we have right now really are able to rise to the challenge 00:01:29.400 --> 00:01:32.517 of effectively any kind of problem that you want to throw at them. Do you have 00:01:32.600 --> 00:01:36.717 enough compute? No. Really? Yeah, definitely not. 00:01:36.800 --> 00:01:39.397 >> just with Matt Garman, and he says the 00:01:39.480 --> 00:01:42.117 GPU compute availability 00:01:42.200 --> 00:01:46.837 in 2026 rounds to zero. Don't you guys have all of it? I mean, we have we we 00:01:46.920 --> 00:01:50.437 would love more. We're we're constantly out there hunting for more, honestly. 00:01:50.520 --> 00:01:54.597 And I'll tell you like when we first launched when we launched ChatGPT, 00:01:54.680 --> 00:01:58.077 I remember being on a call with my team, and they're like, all right, how much 00:01:58.160 --> 00:02:01.797 computer should we buy? And I said, all of it. 00:02:01.880 --> 00:02:03.797 And they're like, no, no, no, we're seriously, like like come on, how much 00:02:03.880 --> 00:02:07.557 should we buy? I'm like, no matter how fast we try to ramp compute, I guarantee 00:02:07.640 --> 00:02:09.957 you we're not going to be able to keep up with demand and that has been true 00:02:10.040 --> 00:02:11.517 ever since. 00:02:11.600 --> 00:02:14.357 Um, that's that's fascinating. Moving up 00:02:14.440 --> 00:02:19.237 from compute since I don't know if much of this audience can help you with 00:02:19.320 --> 00:02:22.517 securing more compute um because most of them are founders of 00:02:22.600 --> 00:02:25.397 startups. Um, about architecture and 00:02:25.480 --> 00:02:27.637 scaling laws, what are the 00:02:27.720 --> 00:02:30.197 what are the where are we in the scaling 00:02:30.280 --> 00:02:32.677 laws? Are they still doubling each year? 00:02:32.760 --> 00:02:36.797 Are you changing architecture? What's what is 00:02:36.880 --> 00:02:39.797 what are you guys pushing on the frontier on the research side? Well, I 00:02:39.880 --> 00:02:42.277 would say first of all, the scaling laws 00:02:42.360 --> 00:02:45.277 are a deep and very beautiful mystery. 00:02:45.360 --> 00:02:47.957 Right? They feel deeply fundamental. 00:02:48.040 --> 00:02:53.437 It's like the scientific truth that just like you think about physics and you 00:02:53.520 --> 00:02:56.117 know, Newton's laws and things like that, they're somehow this truth of the 00:02:56.200 --> 00:03:00.197 universe. And they're empirical. Like we don't 00:03:00.280 --> 00:03:04.357 necessarily have all the theory to explain exactly why it works, but to me 00:03:04.440 --> 00:03:07.557 the most beautiful thing is that neural networks 00:03:07.640 --> 00:03:10.197 were really designed like in the 1940s 00:03:10.280 --> 00:03:12.077 before there were computers. 00:03:12.160 --> 00:03:14.957 And somehow we've been able to take the 00:03:15.040 --> 00:03:17.277 exact ideas that were developed back 00:03:17.360 --> 00:03:22.157 then and apply increasing amounts of computation and as you pour more compute 00:03:22.240 --> 00:03:25.357 into the models, they get correspondingly more capable and it just 00:03:25.440 --> 00:03:30.117 keeps going. There's no wall. And that's I think that's a beautiful thing. 00:03:30.200 --> 00:03:31.917 That's pretty beautiful. Are there more 00:03:32.000 --> 00:03:35.637 research or more algorithms that are in 00:03:35.720 --> 00:03:39.117 the works that because you know, in the past we had neural 00:03:39.200 --> 00:03:42.797 networks to your point in 1940s, but we couldn't we didn't have the computer for 00:03:42.880 --> 00:03:45.717 it. Now that we have the computer for it, you just where are we just pushing 00:03:45.800 --> 00:03:49.357 the same things or are there new architectures and new ideas coming up? 00:03:49.440 --> 00:03:53.717 Yeah, so I would I would think of it as we we absolutely have new ideas that are 00:03:53.800 --> 00:03:57.157 constantly powering what we do. It's very simplified to say, well, let's take 00:03:57.240 --> 00:04:01.477 a neural network from the 1940s and, you know, put put it in a gigawatt data 00:04:01.560 --> 00:04:05.117 center, right? We have made tons of innovations and we constantly are 00:04:05.200 --> 00:04:08.877 improving things. And sometimes these are micro tweaks. Like you just realize 00:04:08.960 --> 00:04:12.637 that the way you've been formatting data was not quite right and that can 00:04:12.720 --> 00:04:16.597 actually be a very big deal. Sometimes it's larger. You think about the shift 00:04:16.680 --> 00:04:19.997 from the LSTM to the transformer and I don't think the transformer is, you 00:04:20.080 --> 00:04:22.837 know, like everyone's moved past the transformer as described in the other 00:04:22.920 --> 00:04:26.117 2018 paper. So, there's there's constant innovation 00:04:26.200 --> 00:04:29.477 happening and I think of places that have been perhaps the most invested in 00:04:29.560 --> 00:04:32.997 long-term research on how to improve the architectures, how to improve the 00:04:33.080 --> 00:04:36.437 fundamental algorithms, and how to get the paradigm shifts. I think OpenAI has 00:04:36.520 --> 00:04:39.197 been leading the pack there and that's something we continue to invest in and I 00:04:39.280 --> 00:04:42.117 see lots of fruit on the horizon. Got it. 00:04:42.200 --> 00:04:44.437 And on the models, um 00:04:44.520 --> 00:04:49.357 Does OpenAI have a formal definition for AGI? Are we close? Are we not close? I 00:04:49.440 --> 00:04:54.237 Pat and Sam and Ilya published this thing that we are at AGI functionally. 00:04:54.320 --> 00:04:57.477 Do you agree with that? Do you not agree with that? Well, we do have a formal 00:04:57.560 --> 00:05:02.077 definition, but to some extent I one thing I have learned is that everyone 00:05:02.160 --> 00:05:06.397 has their own intuitions about what AGI is and maybe you can view it as like 00:05:06.480 --> 00:05:08.317 according to my view of where we are, I 00:05:08.400 --> 00:05:10.677 think we're about 80% of the way there 00:05:10.760 --> 00:05:13.557 in that we have models that are smart. 00:05:13.640 --> 00:05:18.317 They're very capable. They're able to if you give Are they smarter than you? 00:05:18.400 --> 00:05:21.437 I mean, I they're certainly more capable than I am at writing software, right? If 00:05:21.520 --> 00:05:23.517 you give it all the context, then yes, I 00:05:23.600 --> 00:05:28.477 I I think that they are they're just so capable. It's it's really remarkable. 00:05:28.560 --> 00:05:34.960 Like is anyone here feel better at writing software than GPT-5.4? 00:05:35.280 --> 00:05:37.720 Oh. 00:05:38.320 --> 00:05:43.157 All right, writing kernels. Um so, even there we're seeing massive gains from 00:05:43.240 --> 00:05:46.997 Exactly. We're And for some of our internal results, um that 00:05:47.080 --> 00:05:50.877 we're really seeing if you pour the right kinds of of 00:05:50.960 --> 00:05:53.797 you know, if you have the right setup for your problem, then you're able to 00:05:53.880 --> 00:05:58.477 get really massive results out of very low-level, um, even low-level tasks. And 00:05:58.560 --> 00:06:01.917 just to give you one example of how things have been trending, um, 00:06:02.000 --> 00:06:06.077 one of my systems engineers, also very similar, was like, "Hey, like, I haven't 00:06:06.160 --> 00:06:07.597 been able to get value out of the models 00:06:07.680 --> 00:06:10.317 for GPT-5 or 5.1, 00:06:10.400 --> 00:06:14.917 for 5.2 as well." For 5.3, he on a lark 00:06:15.000 --> 00:06:18.397 had prepared this design document for a very complicated systems optimization he 00:06:18.480 --> 00:06:22.717 was about to do. He handed it over to the model, went to 00:06:22.800 --> 00:06:25.757 sleep, waking up intending to like give this to his team to work on for the next 00:06:25.840 --> 00:06:28.437 week. And when he woke up, it was done. 00:06:28.520 --> 00:06:32.677 That the model had actually implemented the initial spec, had seen 00:06:32.760 --> 00:06:35.877 that it was slow, had added instrumentation, had actually run the 00:06:35.960 --> 00:06:39.637 code, used a profiler to figure out where things were slow, and iterated 00:06:39.720 --> 00:06:43.717 multiple times until it got into an optimized result. And like, like, that 00:06:43.800 --> 00:06:46.477 is incredible. That's where we are. 00:06:46.560 --> 00:06:49.877 And so, how what would you advise all startups here to do? Because the models 00:06:49.960 --> 00:06:52.037 keep getting more and more capable. 00:06:52.120 --> 00:06:55.437 They're kind of, uh, I've asked this, uh, when Sam was here 00:06:55.520 --> 00:06:59.037 in the past, and, you know, what if you're building today, do you need to 00:06:59.120 --> 00:07:03.517 rebuild in 2 years when a new model comes out? Because all the functionality 00:07:03.600 --> 00:07:07.237 and all the capabilities all change around you. 00:07:07.320 --> 00:07:12.637 Um, do you need to make sure that you're not in OpenAI's way because 00:07:12.720 --> 00:07:16.357 you're going to roll, you're just going to run over for startups because the 00:07:16.440 --> 00:07:18.637 models are so much more capable. 00:07:18.720 --> 00:07:21.597 Um, how how would you recommend the, uh, 00:07:21.680 --> 00:07:26.397 a set of, um, startup founders to to build in this environment? Well, 00:07:26.480 --> 00:07:28.437 first of all, I would say to lean in. 00:07:28.520 --> 00:07:32.717 The tools right now have become incredibly useful. And if you look even 00:07:32.800 --> 00:07:36.477 over the course of December, I think that we went from these agentic coding 00:07:36.560 --> 00:07:40.837 tools being like, you know, they're like writing 20% of your code to writing 80% 00:07:40.920 --> 00:07:43.917 of your code. Which means they go from being kind of a side show to being the 00:07:44.000 --> 00:07:48.717 main thing that you're doing. And I think we're doing that across all of 00:07:48.800 --> 00:07:52.037 the work that people do with computers, all computer work this year. And you can 00:07:52.120 --> 00:07:53.517 look at the recent progress on Codex. 00:07:53.600 --> 00:07:57.397 It's really changing from a tool for software engineers to a tool for anyone 00:07:57.480 --> 00:08:01.347 who's doing work with a computer. And just over the past week, we've released 00:08:01.430 --> 00:08:03.877 [snorts] a bunch of features that just make it so much more powerful and 00:08:03.960 --> 00:08:08.677 capable. Um and like one thing we just announced today is a new tool called 00:08:08.760 --> 00:08:12.117 Chronicle that plugs into the Codex, where it actually can see everything 00:08:12.200 --> 00:08:16.237 you're doing with your computer and can form memories of what's going on. And so 00:08:16.320 --> 00:08:19.397 you ask it a question, you just it instantly knows what you're talking 00:08:19.480 --> 00:08:21.917 about. You're like, "Huh, what was I doing 5 minutes ago?" It knows, right? 00:08:22.000 --> 00:08:25.597 You're like, "Oh, what was this person talking about?" It knows. It's To me, it 00:08:25.680 --> 00:08:28.757 was this real wake-up call to realize you spend so much of your effort right 00:08:28.840 --> 00:08:32.917 now just explaining your computer what's going on. Like, why are you explaining 00:08:33.000 --> 00:08:35.837 to your computer what's going on? That makes no sense. And so I think what's 00:08:35.920 --> 00:08:39.397 going to happen over upcoming years is the models are going to get much more 00:08:39.480 --> 00:08:43.437 capable, will have better harnesses, will be able to be able to solve harder 00:08:43.520 --> 00:08:46.677 and harder problems, come up with new knowledge, all of these things. But 00:08:46.760 --> 00:08:49.237 there is a one-time shift that's happening now, which is really about 00:08:49.320 --> 00:08:54.037 context. It's really about is your AI able to 00:08:54.120 --> 00:08:57.277 you have all these meetings, you didn't include the AI. 00:08:57.360 --> 00:09:00.037 You know, that's not very nice to the AI. Like, you're asking it to to help 00:09:00.120 --> 00:09:02.877 you with things and it has no information. So I think really leaning 00:09:02.960 --> 00:09:06.597 into how do you make sure the AI even has enough information in theory to 00:09:06.680 --> 00:09:10.477 solve the problem and then trust the models are going to really get there and 00:09:10.560 --> 00:09:14.317 improve. So I think it will be a constant cycle of improvement and 00:09:14.400 --> 00:09:17.557 iteration and leaning into the tools and kind of talking to your friends if 00:09:17.640 --> 00:09:20.877 you're how they're using it, but that there is this investment that's a 00:09:20.960 --> 00:09:23.237 one-time investment that now is the time 00:09:23.320 --> 00:09:25.677 to make. And in terms of like, let's say 00:09:25.760 --> 00:09:28.237 we you you set that all up, how do you 00:09:28.320 --> 00:09:31.717 how is OpenAI using Codex differently 00:09:31.800 --> 00:09:35.677 than you think everybody else outside is using it? 00:09:35.760 --> 00:09:38.997 Well, I think one of the amazing things about being at OpenAI is you do get to 00:09:39.080 --> 00:09:42.717 live in the future, right? You do get to really see the shape of what's emerging, 00:09:42.800 --> 00:09:46.517 and we can co-design, right? We can really change the models, the harness, 00:09:46.600 --> 00:09:50.997 everything together in order to better serve the needs that we see. 00:09:51.080 --> 00:09:53.477 And a lot of the approach we've been taking is so we started with software 00:09:53.560 --> 00:09:58.437 engineering, and we set some clear guidelines for example saying that 00:09:58.520 --> 00:10:02.397 we still want a human to be accountable for all code that gets merged, right? 00:10:02.480 --> 00:10:07.877 So, at the end of the day, is it a good thing to merge this piece of code? 00:10:07.960 --> 00:10:10.917 Is it well-structured? Is it going to make our codebase more maintainable? We 00:10:11.000 --> 00:10:14.637 want to make sure there's a human who is signing off to say yes. And that's I 00:10:14.720 --> 00:10:17.157 think that thoughtfulness of not just saying, "Okay, let's just blindly use 00:10:17.240 --> 00:10:19.597 this." Or, you know, "Oh, we don't want to use this at all." Like I think 00:10:19.680 --> 00:10:23.677 neither extreme is quite right. And then we are also going vertical by vertical 00:10:23.760 --> 00:10:25.437 within OpenAI to adopt these tools 00:10:25.520 --> 00:10:28.837 within finance, within sales, within 00:10:28.920 --> 00:10:33.757 IT. And there we have a small dedicated team who's really deeply understanding 00:10:33.840 --> 00:10:37.117 the domain, working with the people who are the experts in it in order to build 00:10:37.200 --> 00:10:42.237 skills, in order to modify the Codex UI, whatever it is that is is needed in 00:10:42.320 --> 00:10:45.997 order to get it to be good. And then that's something we can then, once we 00:10:46.080 --> 00:10:49.877 have it in good shape, we will externalize, and that we're able to to 00:10:49.960 --> 00:10:53.677 ship that to all of you. And so we are starting to work with certain customers 00:10:53.760 --> 00:10:57.197 as well. So, for people who want to be very AI forward and want to be part of 00:10:57.280 --> 00:11:00.757 defining this revolution, that there's a place for that, and I'd love to talk 00:11:00.840 --> 00:11:03.597 afterwards. But yeah, I think that just this desire 00:11:03.680 --> 00:11:06.757 to say, "Hey, we really want to be AI forward, really live in the future, and 00:11:06.840 --> 00:11:10.437 experience what it will be like for everyone else 1 year, 2 years, 3 years 00:11:10.520 --> 00:11:13.197 down the road." Do you guys structure your company 00:11:13.280 --> 00:11:17.437 differently with the engineering teams differently because of of the living in 00:11:17.520 --> 00:11:20.597 the future? I mean, if you have to go way back, when my father learned 00:11:20.680 --> 00:11:22.637 computer science, he was just himself, 00:11:22.720 --> 00:11:25.157 and then we had these long software 00:11:25.240 --> 00:11:29.757 releases that became waterfall. And then when the web happened and the cloud 00:11:29.840 --> 00:11:32.997 happened, when these two-pizza teams, and we have scrum. 00:11:33.080 --> 00:11:38.037 Now that we have these coding agents, is it is how do you structure around 00:11:38.120 --> 00:11:41.797 everything differently? I think we're still figuring it out, and 00:11:41.880 --> 00:11:45.077 there's certain places where you really see it. For example, the cost of 00:11:45.160 --> 00:11:49.277 building a prototype is cheap now. It's so cheap, and if you want to build a 00:11:49.360 --> 00:11:53.677 dashboard that used to be like, uh, it'd take like someone like a week to do it, 00:11:53.760 --> 00:11:56.917 and you just do it now. And so actually a lot of the bottleneck has shifted to 00:11:57.000 --> 00:12:01.037 things like sharing. Like how do you And so we we actually have some internal 00:12:01.120 --> 00:12:04.037 work on this as well that again we will be externalizing. If How do you make it 00:12:04.120 --> 00:12:07.797 really easy for anyone in your enterprise to build a dashboard, a 00:12:07.880 --> 00:12:11.037 widget, a bot, whatever the thing is, and then share it with others. And then 00:12:11.120 --> 00:12:14.557 that starts to really put pressure on having good governance. Like you want 00:12:14.640 --> 00:12:17.597 your IT organization to be able to see all these different, 00:12:17.680 --> 00:12:19.717 you know, threads of execution that are happening, all the little things that 00:12:19.800 --> 00:12:23.437 are being shared around, have some control over 00:12:23.520 --> 00:12:26.957 data provenance, right? To really make sure that, okay, like a good example of 00:12:27.040 --> 00:12:30.557 this is, um, I think people are now starting to take 00:12:30.640 --> 00:12:33.837 their internal knowledge from storing them to wikis. We we have some some a 00:12:33.920 --> 00:12:37.237 really cool one of these internally, and the thing you immediately think about 00:12:37.320 --> 00:12:41.677 is, well, if someone has a document in the internal knowledge base that was 00:12:41.760 --> 00:12:44.677 accidentally permissioned incorrectly, and they realize, oh no, I didn't want 00:12:44.760 --> 00:12:49.037 this information to be accessible, how do they fix that, right? So normally 00:12:49.120 --> 00:12:51.717 it's they go into the doc, they change the permissions, but now there's these 00:12:51.800 --> 00:12:54.797 derived artifacts. And so you need to make sure you have some way of tracking 00:12:54.880 --> 00:12:58.837 through the system to say, well, this output document came from the source 00:12:58.920 --> 00:13:01.997 one. The source one's no longer accessible to this Let's go and 00:13:02.080 --> 00:13:05.557 invalidate that as well. And so you have to start really building your technical 00:13:05.640 --> 00:13:09.317 architecture with awareness of the way that people are going to use this 00:13:09.400 --> 00:13:13.597 information, and it really changes how teams relate to each other because you 00:13:13.680 --> 00:13:17.717 can just It really changes where the bottlenecks are and what's hard. 00:13:17.800 --> 00:13:21.077 Uh, do you think team sizes going to be a lot smaller? 00:13:21.160 --> 00:13:24.397 Are we going to have still human software engineers in a 00:13:24.480 --> 00:13:28.437 decade? Well, decade is a long time from now and 00:13:28.520 --> 00:13:32.757 the the ceiling on this technology is hard to is really hard to internalize. I 00:13:32.840 --> 00:13:37.597 think that it is clear that what a company is will change in a lot of ways. 00:13:37.680 --> 00:13:42.397 I think that we're going to have this ability for solopreneurs to build very 00:13:42.480 --> 00:13:45.437 incredible businesses. And so anyone who has a vision, I think will be able to 00:13:45.520 --> 00:13:49.477 realize it. I think the jobs that you all have will become way easier in a lot 00:13:49.560 --> 00:13:52.677 of ways, way more fun. Now, might be more competitive, too. Right? Because 00:13:52.760 --> 00:13:55.197 everyone's going to have these amazing tools. And so really figuring out what 00:13:55.280 --> 00:13:58.837 is your niche, what is your unique angle, is probably going to become kind 00:13:58.920 --> 00:14:02.917 of the most important core. But a lot of how we run organizations right now, and 00:14:03.000 --> 00:14:05.677 it's there's almost only one way to organize large groups of people, where 00:14:05.760 --> 00:14:08.677 you have teams, you have management structures, and you have scopes, and you 00:14:08.760 --> 00:14:11.677 have these hierarchies, and all these things. Maybe that can change. Maybe you 00:14:11.760 --> 00:14:15.597 can be much more flat, small teams that can really just do incredible things. 00:14:15.680 --> 00:14:19.517 Like, we're seeing it right now in mathematics, where 00:14:19.600 --> 00:14:24.677 these individuals on the internet are using GPT-4 Pro to solve these unsolved 00:14:24.760 --> 00:14:28.437 math problems. Normally you need a math team, and they're just doing it. Yeah. 00:14:28.520 --> 00:14:32.437 And my son's a math nerd. I just told him that maybe we should be studying 00:14:32.520 --> 00:14:35.717 something else besides math. But I well, but see, this is the 00:14:35.800 --> 00:14:38.957 question, right? Is if you look at something like AlphaGo, you know, move 00:14:39.040 --> 00:14:43.677 37, this move that just like changed humanity's understanding of the game, 00:14:43.760 --> 00:14:47.597 but the thing that was surprising is it made the game more interesting and 00:14:47.680 --> 00:14:50.517 important for humans. And maybe that will be true for for these other 00:14:50.600 --> 00:14:54.557 domains, too. True. What about uh common failure modes when 00:14:54.640 --> 00:14:58.357 you're building when you're you're you're building with uh production 00:14:58.440 --> 00:15:01.757 agentic workflows? What do you What do you see as the common things that 00:15:01.840 --> 00:15:06.197 founders get wrong and they're building incorrectly these days? Well, I think 00:15:06.280 --> 00:15:09.757 that these models, they have such power, and 00:15:09.840 --> 00:15:13.917 really understanding how to operate them well takes thought. And so we've been 00:15:14.000 --> 00:15:18.597 investing a lot in primitives, security primitives, observability, having again 00:15:18.680 --> 00:15:21.917 good governance, things like that. Um but just to give you one anecdote that I 00:15:22.000 --> 00:15:25.717 think is evocative, um, I asked So, I was working with my co-workers, I asked 00:15:25.800 --> 00:15:28.437 them to install some package that someone at OpenAI had written, ran into 00:15:28.520 --> 00:15:31.517 an error, I was like, "Oh, ping that person on Slack and ask them for help." 00:15:31.600 --> 00:15:34.477 So, ping the person on Slack. Two minutes later, it said, "This is 00:15:34.560 --> 00:15:37.037 taking too long. I've escalated to the person's manager." And it actually 00:15:37.120 --> 00:15:42.557 pinged the person's manager. >> [laughter] >> And And you realize it's like, on the 00:15:42.640 --> 00:15:45.517 one hand, it's kind of a reasonable thing for the model to do. It's being 00:15:45.600 --> 00:15:48.517 proactive, it's trying to solve my problem. It's like, you know, not just 00:15:48.600 --> 00:15:51.797 sitting around waiting to be told what to do. But, on the other hand, like, you 00:15:51.880 --> 00:15:54.077 know, >> [laughter] >> maybe it should have taken a little bit 00:15:54.160 --> 00:15:58.037 longer, maybe should have checked with me. And so, I think that really thinking 00:15:58.120 --> 00:16:03.117 about these questions where we're still building up the EQ of the model. And 00:16:03.200 --> 00:16:06.517 that in some places, it's getting very good. For example, clicking approve, 00:16:06.600 --> 00:16:10.517 approve, approve is kind of where we've been. And humans are not very good at 00:16:10.600 --> 00:16:13.357 that, either, right? >> They just They They just default. 00:16:13.440 --> 00:16:16.757 >> default. And so, now we're starting to have AIs that can actually take care of 00:16:16.840 --> 00:16:18.677 flagging, is this a high-risk action? 00:16:18.760 --> 00:16:21.637 Hey, this one should be escalated, this one's okay to auto-approve. And it 00:16:21.720 --> 00:16:24.797 really makes you realize that human attention is going to be this incredibly 00:16:24.880 --> 00:16:28.717 scarce resource, right? The doing of things now is easy. The Is this a good 00:16:28.800 --> 00:16:32.797 thing? Is this what I wanted? Is this aligned with my values, with my desires? 00:16:32.880 --> 00:16:37.197 That is going to become the single most important bottleneck. And so, I think 00:16:37.280 --> 00:16:40.837 building systems that take that into account and really think about the human 00:16:40.920 --> 00:16:43.997 factor, like, that's the most important thing to do now. 00:16:44.080 --> 00:16:46.837 Another human factor, security. 00:16:46.920 --> 00:16:49.597 Um, how would you advise people to think 00:16:49.680 --> 00:16:52.757 about security in this world of AI and 00:16:52.840 --> 00:16:55.237 heard about breaches left and right with 00:16:55.320 --> 00:16:59.917 Vercel recently and then And these models are incredibly powerful 00:17:00.000 --> 00:17:04.197 at finding security holes. So, how how would you recommend people here use the 00:17:04.280 --> 00:17:06.517 models to to find those security issues? 00:17:06.600 --> 00:17:09.957 Well, I think there's a couple levels to the answer. I do think that this is I 00:17:10.040 --> 00:17:12.797 think that the internet has been a place 00:17:12.880 --> 00:17:17.157 where security has been just like a a ratcheting important concern over time. 00:17:17.240 --> 00:17:20.677 You think about where it started, going through the '90s with viruses and worms 00:17:20.760 --> 00:17:24.357 and malware and those things and we've moved past that. Um I think we are also 00:17:24.440 --> 00:17:28.597 moving now to a much more ultimately secure regime, but it does require kind 00:17:28.680 --> 00:17:30.757 of an internet-wide effort to get there. 00:17:30.840 --> 00:17:33.677 And so a lot of this honestly is just again leaning into the technology, 00:17:33.760 --> 00:17:37.637 having these models they can scan your code base, they can actually be used for 00:17:37.720 --> 00:17:40.917 end-to-end red teaming. Like there's a lot that can be done with them and a lot 00:17:41.000 --> 00:17:44.277 of how we're thinking about further models and improvements there is really 00:17:44.360 --> 00:17:49.117 leaning into how do we how do we actually sort of leverage trusted access 00:17:49.200 --> 00:17:52.277 programs, how do we leverage the community of people who really care 00:17:52.360 --> 00:17:55.997 about being defenders and making the internet more secure. I think that's 00:17:56.080 --> 00:18:00.317 something where everyone has a role to play and can participate, but the number 00:18:00.400 --> 00:18:04.637 one thing is just sort of recognizing that these models are very powerful, but 00:18:04.720 --> 00:18:07.877 they're not magic, right? That they are just like a part of the overall 00:18:07.960 --> 00:18:12.237 resilience ecosystem and I think that we as a society and I think every company 00:18:12.320 --> 00:18:16.357 again really contributes to this have something to build in terms of how do we 00:18:16.440 --> 00:18:18.197 how do we incorporate these in a way 00:18:18.280 --> 00:18:20.677 that results in more 00:18:20.760 --> 00:18:23.397 assurance and more 00:18:23.480 --> 00:18:28.197 sort of certainty on the impacts of of whether it's a particular patch that 00:18:28.280 --> 00:18:31.037 you're taking, whether it's thinking about how do you make sure that you're 00:18:31.120 --> 00:18:34.637 um yeah, just sort of rolling in updates quickly as they're being released. Um so 00:18:34.720 --> 00:18:38.437 I think that there's a lot of work to be done, but I have a lot of optimism for 00:18:38.520 --> 00:18:43.517 where this is going. Um let's switch to speed. Seems like 00:18:43.600 --> 00:18:47.037 things are moving faster and faster and faster when in the world of accelerating 00:18:47.120 --> 00:18:49.757 change. We were talking about it when we when you 00:18:49.840 --> 00:18:52.837 uh you were walking up here around how 00:18:52.920 --> 00:18:57.157 how you're trying to keep up with things. How how do you you keep up with 00:18:57.240 --> 00:19:00.677 all the accelerating change? How would you recommend everybody here keep up 00:19:00.760 --> 00:19:02.677 with everything that's changing? 00:19:02.760 --> 00:19:06.557 Well, I think this is the new normal and I think to some extent it's not really 00:19:06.640 --> 00:19:11.517 because of AI. I think it's just been the trend of technology for the past two 00:19:11.600 --> 00:19:14.757 decades. There's more people doing things. It's 00:19:14.840 --> 00:19:17.997 easier to do things than ever. Barrier to entry goes down. 00:19:18.080 --> 00:19:21.797 Means it's also much more easy to build value, right? To have great successes. 00:19:21.880 --> 00:19:25.357 And so, I think that really trying to keep your ear to the ground and 00:19:25.440 --> 00:19:28.237 understand what's changing. And to some extent, it always starts with the same 00:19:28.320 --> 00:19:29.557 thing, which is play with the technology 00:19:29.640 --> 00:19:32.037 yourself. Like, it's very different to 00:19:32.120 --> 00:19:36.917 hear AI described versus to use it. But, the beautiful thing about AI is it's so 00:19:37.000 --> 00:19:40.037 intuitive. Like, that's the whole point is that rather than have the machine be 00:19:40.120 --> 00:19:43.957 something you have to contort yourself to, the machine contorts itself to you, 00:19:44.040 --> 00:19:47.117 right? It's doing work for you. And it should be something where you ask it and 00:19:47.200 --> 00:19:48.757 does something. And so, I think that 00:19:48.840 --> 00:19:52.077 just really trying to just get your 00:19:52.160 --> 00:19:54.437 finger on the pulse of what's changing, 00:19:54.520 --> 00:19:56.437 what's possible, where the models lag. 00:19:56.520 --> 00:19:59.157 That is, I think, the core skill that is 00:19:59.240 --> 00:20:03.877 going to really determine a lot of the success of of companies in the future. 00:20:03.960 --> 00:20:07.717 And then, on the flip side of that, you guys have held up held back models to 00:20:07.800 --> 00:20:11.357 work with security agents. So, it's like the opposite of like going as fast as 00:20:11.440 --> 00:20:15.397 possible. So, um you're doing things responsibly, too. 00:20:15.480 --> 00:20:18.357 So, how do you kind of like think about the balance? Because you're in a 00:20:18.440 --> 00:20:22.237 competitive environment, you want to ship as quickly as possible, and yet 00:20:22.320 --> 00:20:24.557 you're trying to do the right thing, as well. 00:20:24.640 --> 00:20:28.877 Yeah, I think at a values level, like what OpenAI is about, like we really 00:20:28.960 --> 00:20:33.957 want to put the power of AI in people's hands. Like, we believe that people can 00:20:34.040 --> 00:20:37.037 We want to empower people to build the future with the tools that are being 00:20:37.120 --> 00:20:39.917 created. But, we need to do that in a thoughtful way, right? That we really 00:20:40.000 --> 00:20:42.917 think about both sides of Here are the benefits, here are the risks. How do you 00:20:43.000 --> 00:20:45.757 maximize the benefits? How do you mitigate those risks? And I think that 00:20:45.840 --> 00:20:50.077 in cybersecurity and in biosecurity, those are areas where we're very 00:20:50.160 --> 00:20:53.597 thoughtful. We've been building We've been working on these kinds of both 00:20:53.680 --> 00:20:55.757 mitigations and trusted access programs 00:20:55.840 --> 00:20:58.717 for quite a long time. And that what we 00:20:58.800 --> 00:21:00.917 see coming is models that are going to 00:21:01.000 --> 00:21:03.037 be increasingly powerful and capable in 00:21:03.120 --> 00:21:05.757 a continuous way across all dimensions 00:21:05.840 --> 00:21:09.077 of capability. And the 00:21:09.160 --> 00:21:13.277 you know, we announced last week the expansion of our trusted access for 00:21:13.360 --> 00:21:17.157 cyber program. By the way, has anyone here applied? 00:21:17.240 --> 00:21:18.957 No one. Oh, I see one hand, two hands. 00:21:19.040 --> 00:21:21.357 Okay, more of you should apply. It's great. 00:21:21.440 --> 00:21:25.477 We really need help because and it's very important that people who are 00:21:25.560 --> 00:21:29.277 trustworthy and responsible and really want to push these models are 00:21:29.360 --> 00:21:32.237 participating in this because that is how that's going to pay dividends for 00:21:32.320 --> 00:21:34.557 everyone. We're going to have more to announce 00:21:34.640 --> 00:21:39.477 over upcoming weeks on how we're expanding the program, but and also when 00:21:39.560 --> 00:21:42.797 we release models to everyone kind of the mitigations that that we have and 00:21:42.880 --> 00:21:46.637 how we're going to tune those to be both to really balance, right? To really try 00:21:46.720 --> 00:21:50.797 to bring these capabilities as broadly as possible, while also making sure that 00:21:50.880 --> 00:21:54.837 the ones that are, you know, that we're thinking about the risks and and able to 00:21:54.920 --> 00:21:58.597 have some observability over them and to ensure that this is maximally positive 00:21:58.680 --> 00:22:01.397 in terms of deployment. So, I think the short answer is like it's core to our 00:22:01.480 --> 00:22:04.917 mission. We care a lot about the impacts of what we're doing, not just building 00:22:05.000 --> 00:22:08.997 technology in isolation, but it is a whole community, a whole world effort to 00:22:09.080 --> 00:22:11.237 really get to where we need to be. 00:22:11.320 --> 00:22:13.957 On moving up from the models to the 00:22:14.040 --> 00:22:15.877 application layer, which is what a lot 00:22:15.960 --> 00:22:21.237 of people here are building. How do you How does OpenAI decide what 00:22:21.320 --> 00:22:24.357 in the application layer you're you're going to build and what you're going to 00:22:24.440 --> 00:22:28.757 leave out? Well, people have probably seen the word 00:22:28.840 --> 00:22:34.157 focus being applied to OpenAI quite a lot recently, possibly for the first time 00:22:34.240 --> 00:22:37.747 >> in a while. And uh It's been applied to her, too. 00:22:37.830 --> 00:22:41.317 >> [laughter] >> And it's it's hard because the field of 00:22:41.400 --> 00:22:44.597 AI is one of opportunity, right? It's like anything you're going you can 00:22:44.680 --> 00:22:46.757 imagine is going to be great. No 00:22:46.840 --> 00:22:51.997 question it's going to be great. And we as a company, as a single company, no 00:22:52.080 --> 00:22:54.997 matter how much compute we build, no matter how many people we have, are only 00:22:55.080 --> 00:22:58.757 going to be able to do so much. And so, a lot of where we've been how we've been 00:22:58.840 --> 00:23:01.597 thinking about things is what is the 00:23:01.680 --> 00:23:03.917 sort of most focused strategy that 00:23:04.000 --> 00:23:07.717 covers the parts of the space? 00:23:07.800 --> 00:23:09.997 You know, maybe it's an 80/20 or just like the parts of the space that we 00:23:10.080 --> 00:23:13.797 think we can have most impact on. And I think there it's very clear right now 00:23:13.880 --> 00:23:17.877 we're going through this identity transition. And so >> [clears throat] 00:23:17.960 --> 00:23:21.957 >> products that are And it's not just about enterprise versus consumer, right? 00:23:22.040 --> 00:23:25.197 So it's like clear we are being very serious about enterprise. Like we're 00:23:25.280 --> 00:23:28.637 selling to to big companies and and building a whole muscle and sales motion 00:23:28.720 --> 00:23:33.357 there. But consumer, what consumer is is going to change, right? It's kind of a 00:23:33.440 --> 00:23:37.637 very broad term that buckets in multiple things. But that the slice of consumer 00:23:37.720 --> 00:23:41.557 that's about not just productivity, but about goals, about achieving your goal, 00:23:41.640 --> 00:23:44.517 about even knowing what is your goal, being able to elicit that and having an 00:23:44.600 --> 00:23:48.077 AI that can proactively do that. It's all kind of the same thing. Like in the 00:23:48.160 --> 00:23:52.037 end we're trying to build an AGI that you can talk to, that has all this 00:23:52.120 --> 00:23:54.597 context that you can use in your personal life, your work life, that's 00:23:54.680 --> 00:23:58.117 trustworthy, right? That you can go to it for advice, can give you useful 00:23:58.200 --> 00:24:01.997 information, maybe health information, or maybe about finances, or 00:24:02.080 --> 00:24:04.757 you know, about if you're trying to figure out what to do with your career. 00:24:04.840 --> 00:24:09.077 Like all these things, they all kind of ladder into one thing. And it's meant we 00:24:09.160 --> 00:24:12.637 had to make some very painful decisions about what not to do. But I think I 00:24:12.720 --> 00:24:16.077 would just say that that's the aperture that we look at things through and that 00:24:16.160 --> 00:24:20.477 things that accrue to that singular vision of what we want to build, you 00:24:20.560 --> 00:24:22.957 should expect us to pursue. Got it. 00:24:23.040 --> 00:24:27.197 Um Do you think we'll be coding with 00:24:27.280 --> 00:24:31.877 command lines and agents in in a few years or is it going to be 00:24:31.960 --> 00:24:36.357 completely changed? I mean, I think that we're in a very unnatural state right 00:24:36.440 --> 00:24:40.837 now for how we work. Like we all sit behind this box and kind 00:24:40.920 --> 00:24:44.197 of type away and it's very clear our bodies were not designed for this. We 00:24:44.280 --> 00:24:48.237 got our carpal tunnel and our, you know, hunched shoulders and all these things. 00:24:48.320 --> 00:24:50.997 And I don't think we want that. I don't think any of us wanted that. Like I 00:24:51.080 --> 00:24:53.837 think that we want more free time. We want more time, but it's it's not even 00:24:53.920 --> 00:24:56.477 about free time necessarily, right? It's like you want to spend more time with 00:24:56.560 --> 00:24:59.997 your loved ones? Yes. You want to spend more time like talking to people and 00:25:00.080 --> 00:25:03.237 like coming up with like brilliant visions or just like what you're excited 00:25:03.320 --> 00:25:04.997 about or just understanding yourself. 00:25:05.080 --> 00:25:09.597 So, it's kind of like do you do you want to be a CEO of an organization of like 00:25:09.680 --> 00:25:13.437 100,000 agents? Like that actually seems pretty good. And I think that we're all 00:25:13.520 --> 00:25:17.557 going to be able to get so much more done, but the mechanics of it are going 00:25:17.640 --> 00:25:20.277 to feel as different as like going from 00:25:20.360 --> 00:25:22.317 having to write out things with, you 00:25:22.400 --> 00:25:24.957 know, by hand with a quill or something 00:25:25.040 --> 00:25:29.437 to being able to you know, just send a text message and have people go and and, 00:25:29.520 --> 00:25:32.717 you know, working on your behalf on your goals. 00:25:32.800 --> 00:25:34.317 All right. We talked about compute. We 00:25:34.400 --> 00:25:38.397 talked about model and security and 00:25:38.480 --> 00:25:42.557 agents and app layer. Let's talk about frontier. 00:25:42.640 --> 00:25:44.477 When when are the models going to be 00:25:44.560 --> 00:25:47.597 good enough to push the frontiers of 00:25:47.680 --> 00:25:50.757 science, physical AI? Seems like we had 00:25:50.840 --> 00:25:54.797 Jen Fan here. It seems like LLMs have been a great scaling law for digital 00:25:54.880 --> 00:25:57.637 intelligence. It hasn't been as strong 00:25:57.720 --> 00:26:02.317 for robotics, for physical intelligence, for 00:26:02.400 --> 00:26:05.837 aspects of biology and science where the problems 00:26:05.920 --> 00:26:10.157 are probably a lot harder to verify or takes a long time to verify. 00:26:10.240 --> 00:26:14.117 Well, how are you keeping track of science and and physical AI in in the 00:26:14.200 --> 00:26:18.357 world? Well, well, science is one domain that we're really leaning into and we 00:26:18.440 --> 00:26:21.917 see line of sight to really incredible progress. 00:26:22.000 --> 00:26:24.437 And we're starting to have some signs of life and I think it's always important 00:26:24.520 --> 00:26:28.437 to ground in what is happening today when trying to predict what will happen 00:26:28.520 --> 00:26:30.477 6 months a year from now. So, for 00:26:30.560 --> 00:26:34.157 example, we had a physics result where 00:26:34.240 --> 00:26:38.397 our AI came up with this very beautiful formula that physicists who've been 00:26:38.480 --> 00:26:41.117 working on this for quite some time thought was totally impossible. Thought 00:26:41.200 --> 00:26:43.477 it was like maybe an unsolvable problem. 00:26:43.560 --> 00:26:46.837 And like it's pretty significant, right? 00:26:46.920 --> 00:26:51.077 It's like real serious physicists who who who really view this as a step 00:26:51.160 --> 00:26:55.517 towards really being able to get to um to to some sort of answer for quantum 00:26:55.600 --> 00:26:58.957 gravity and all these things. It's not there, but it's a step. That's much 00:26:59.040 --> 00:27:02.317 bigger than where we were just a couple months ago. And so you it makes you 00:27:02.400 --> 00:27:07.157 really wonder a year from now, like how far will we have traveled? Now, things 00:27:07.240 --> 00:27:10.397 like biology that they are different from physics and math, right? That they 00:27:10.480 --> 00:27:14.637 are you got to leave your beautiful simulated world and you know, deal with 00:27:14.720 --> 00:27:18.997 messy reality. But I think we've been learning how to deal with messy reality 00:27:19.080 --> 00:27:22.757 in other domains. Software engineering is a perfect example where we've really 00:27:22.840 --> 00:27:27.037 realized that just building the thing that solves competition 00:27:27.120 --> 00:27:30.677 you know, programming competitions, like that's not enough. Like you need 00:27:30.760 --> 00:27:33.877 something that's seen real-world messy codebases, humans interrupting it 00:27:33.960 --> 00:27:37.437 different ways, like this adversarial banging at it. And I so I think that 00:27:37.520 --> 00:27:40.597 that on science, I expect we're going to see a real renaissance. You know, maybe 00:27:40.680 --> 00:27:42.237 we'll see some big results this year. 00:27:42.320 --> 00:27:46.237 Next year I think it's going to be a totally wild wild time. 00:27:46.320 --> 00:27:50.477 We live in interesting times. Um I I promised that I get you out on time 00:27:50.560 --> 00:27:53.957 because you're a busy man. Um before we let you leave, we got 1 minute on the 00:27:54.040 --> 00:27:58.557 shot clock. What since you have no time, but soon you will have 00:27:58.640 --> 00:28:00.877 lots of time, um 00:28:00.960 --> 00:28:04.477 what do you and Anna do for fun? Uh fun? 00:28:04.560 --> 00:28:09.557 I mean, same as anyone. Like like to watch movies, go on hikes, that those kinds of 00:28:09.640 --> 00:28:12.757 things. Um you know, not as much time for it as as 00:28:12.840 --> 00:28:16.757 maybe we'll we'll hopefully have post-AGI. Um but you got to kind of 00:28:16.840 --> 00:28:18.997 enjoy the the ride along the way. 00:28:19.080 --> 00:28:21.627 Thank you, Greg, for joining us. Thank you, everyone. 00:28:21.710 --> 00:28:23.730 >> [applause]