1 00:00:00,080 --> 00:00:01,837 You know, Greg, this is our fourth time 2 00:00:01,920 --> 00:00:04,717 speaking and we've spoken every time 3 00:00:04,800 --> 00:00:08,637 about OpenAI's product direction and I 4 00:00:08,720 --> 00:00:11,757 think I'm starting to get it. Um, you 5 00:00:11,840 --> 00:00:14,317 know, there was this conversation that a 6 00:00:14,400 --> 00:00:17,517 super app was the wrong term for what 7 00:00:17,600 --> 00:00:20,797 you were doing with um the app that 8 00:00:20,880 --> 00:00:23,277 you're building bringing codeex which is 9 00:00:23,360 --> 00:00:26,477 coding side of OpenAI's product browser 10 00:00:26,560 --> 00:00:31,037 and chat GPT together. And when you use the word super app, people would be 11 00:00:31,120 --> 00:00:33,997 like, "No, a super app is actually something that you can just use every 12 00:00:34,080 --> 00:00:37,917 other app within." Um, and now as we've 13 00:00:38,000 --> 00:00:41,997 seen these products come together, actually super app might be the correct 14 00:00:42,080 --> 00:00:45,757 term. You know, at least for us on the outside, we're starting to see it that 15 00:00:45,840 --> 00:00:48,477 when you need to do anything, um, you it 16 00:00:48,560 --> 00:00:51,117 will start with a prompt in chat GPT and 17 00:00:51,200 --> 00:00:54,317 then OpenAI's technology will use either 18 00:00:54,400 --> 00:00:58,397 your browser or your computer to get that done for you. Is that the right way 19 00:00:58,480 --> 00:01:01,037 to think about it? >> I I think that's a pretty good 20 00:01:01,120 --> 00:01:04,397 perspective, right? And I think to really zoom out, the thing we're 21 00:01:04,480 --> 00:01:09,037 actually trying to build is in AGI, right? That if you think about what 22 00:01:09,120 --> 00:01:13,037 people have been using since chat GBT, it's a language model, right? There's a 23 00:01:13,120 --> 00:01:15,277 big gap between these. 24 00:01:15,360 --> 00:01:20,157 It's amazing. You can talk to chat talks back to you. Great. Wonderful. But when 25 00:01:20,240 --> 00:01:23,277 we launched in 2022, there was no memory, right? It's not hooked up to any 26 00:01:23,360 --> 00:01:25,597 tools. Has no context. And so it really 27 00:01:25,680 --> 00:01:30,877 is that this conversational intelligence is only one part of what people really 28 00:01:30,960 --> 00:01:34,317 need to get work done to be able to achieve their goals. And where we're 29 00:01:34,400 --> 00:01:37,757 going is to have an AI that's really looking out for you, right? That you can 30 00:01:37,840 --> 00:01:41,757 provide the goals, the directions, that it's constantly thinking about what can 31 00:01:41,840 --> 00:01:45,757 I do for Alex today? Uh that it's able to go and solve super hard problems, 32 00:01:45,840 --> 00:01:48,957 very mundane problems. you wake up your inbox is organized but also if there's 33 00:01:49,040 --> 00:01:53,597 like a health plan that you are thinking about that it can help you help you 34 00:01:53,680 --> 00:01:56,797 achieve that figure out medical treatments uh or you know sort of back 35 00:01:56,880 --> 00:02:00,797 and forth provide you with that kind of information at least and I think that 36 00:02:00,880 --> 00:02:04,157 the question of what's the interface you want what is the product that you want 37 00:02:04,240 --> 00:02:07,357 is what we spend a lot of time thinking about and the answer is you want almost 38 00:02:07,440 --> 00:02:10,957 no interface you want no product right you want this to be like what's the 39 00:02:11,040 --> 00:02:15,197 interface between you and me right just being able to talk to a persistent 40 00:02:15,280 --> 00:02:19,757 entity of some form firm that's able to go and accomplish goals for you. And so 41 00:02:19,840 --> 00:02:23,757 building that is hard. It will take time, but we have a lot of the pieces, 42 00:02:23,840 --> 00:02:26,797 right? We're increasingly bringing together the product layer, trying to 43 00:02:26,880 --> 00:02:31,037 make the models better, trying to make the whole system just so there's less 44 00:02:31,120 --> 00:02:35,917 like clicking buttons and toggles and changing modes and all these things. Not 45 00:02:36,000 --> 00:02:38,957 to say that there won't be some of those along the way, but the long-term 46 00:02:39,040 --> 00:02:42,557 trajectory is towards simplification, unification. Yeah, it's very interesting 47 00:02:42,640 --> 00:02:45,117 that you say the interface uh will melt 48 00:02:45,200 --> 00:02:47,117 away. And so to go a little bit deeper 49 00:02:47,200 --> 00:02:52,157 with my question, um many of us who use products like chatbt today will see that 50 00:02:52,240 --> 00:02:55,517 the bot will make a suggestion at the end. You know, you ask it about 51 00:02:55,600 --> 00:02:59,037 nutrition and it says should I make a health plan for you or make a diet plan 52 00:02:59,120 --> 00:03:02,717 for you? You ask it, sorry Ron John, we just talked about travel, but you ask it 53 00:03:02,800 --> 00:03:03,997 about travel and then it will give you 54 00:03:04,080 --> 00:03:07,757 an agenda for instance. And so am I 55 00:03:07,840 --> 00:03:12,477 hearing you right that what's going to happen within chat GPT just to give an 56 00:03:12,560 --> 00:03:16,557 example is you talk to it about your health decisions and it might say you 57 00:03:16,640 --> 00:03:20,397 know you probably need to you know go to this specialist let me make an 58 00:03:20,480 --> 00:03:24,637 appointment for you and then it will go and actually take that action on your 59 00:03:24,720 --> 00:03:27,917 behalf. So it goes from simply a conversation interface to actually 60 00:03:28,000 --> 00:03:31,837 understanding your intent and then going out and accomplishing that for you. that 61 00:03:31,920 --> 00:03:35,757 that I that's exactly right and I think that if you've used codecs and by the 62 00:03:35,840 --> 00:03:39,437 way how many people in the room have used have used codecs 63 00:03:39,520 --> 00:03:43,037 >> people yeah decent number of people um and that our goal is to really bring the 64 00:03:43,120 --> 00:03:45,997 power of codecs to everyone right to 65 00:03:46,080 --> 00:03:50,077 bring agents to everyone you that technology exists right now right you 66 00:03:50,160 --> 00:03:52,797 can hook up like I hook up my codeex to 67 00:03:52,880 --> 00:03:57,837 slack to my Gmail to my calendar and there are many people within openai 68 00:03:57,920 --> 00:04:01,037 nontechnical users you know it's got code in the name but it's not really 69 00:04:01,120 --> 00:04:04,797 about code. It's really about having this general purpose tool using harness 70 00:04:04,880 --> 00:04:07,677 an agent. And that the kinds of things, 71 00:04:07,760 --> 00:04:11,837 for example, someone on our comm's team does is um she was organizing an event 72 00:04:11,920 --> 00:04:16,557 and it would just ask all of the uh event attendees for their dietary 73 00:04:16,640 --> 00:04:21,757 preferences, set up a whole seating chart, kind of did all of that work uh 74 00:04:21,840 --> 00:04:24,797 so that she could focus on the parts that she wanted to and and really 75 00:04:24,880 --> 00:04:27,677 thinking about the vision of what she wanted to achieve. And I think that 76 00:04:27,760 --> 00:04:31,437 we're going to see this across the board. So the it's not sci-fi anymore to 77 00:04:31,520 --> 00:04:36,237 think about an AI that's hooked up to these tools. And I remember with you 78 00:04:36,320 --> 00:04:37,757 know our very first attempt at tool use 79 00:04:37,840 --> 00:04:41,517 in chatbt was 2023. I think in like 80 00:04:41,600 --> 00:04:45,677 March or April or something we released plugins. Do people remember plugins back 81 00:04:45,760 --> 00:04:48,317 in early chat days? >> That didn't work. 82 00:04:48,400 --> 00:04:51,437 >> It didn't work at all because the models 83 00:04:51,520 --> 00:04:54,637 the models weren't ready. Right. The form factor is correct. Obviously you're 84 00:04:54,720 --> 00:04:58,397 going to have an AI that's able to like talk to your Gmail like no question. But 85 00:04:58,480 --> 00:05:01,917 we could only like have three different connectors exposed to the model at a 86 00:05:02,000 --> 00:05:06,157 time or start forgetting. You know, we had like 2K maybe 4K token context. Like 87 00:05:06,240 --> 00:05:09,357 there's just no memory, right? It's kind of like when you had early computers in 88 00:05:09,440 --> 00:05:13,117 the 60s or 70s or something, right? You know, that you had tiny little me memory 89 00:05:13,200 --> 00:05:16,237 banks and today you have your phone that's like better than any 90 00:05:16,320 --> 00:05:19,117 supercomputer from that era. And I think that's where we're going with these 91 00:05:19,200 --> 00:05:21,917 models, right? That the rate of improvement has been so steep. So now 92 00:05:22,000 --> 00:05:25,757 you can have hundreds of different tools accessible that we have the ability to 93 00:05:25,840 --> 00:05:28,957 hook them up to whole file system. So you can almost have the full power of 94 00:05:29,040 --> 00:05:32,477 the internet and like almost any application you want at the model's 95 00:05:32,560 --> 00:05:34,877 fingertips. And it's smart, right? It's 96 00:05:34,960 --> 00:05:38,717 got 52 million token context, depends 97 00:05:38,800 --> 00:05:41,677 how you squint on it. And the capability 98 00:05:41,760 --> 00:05:45,997 level is also getting so so powerful, right? These models are now solving 99 00:05:46,080 --> 00:05:48,797 unsolved math problems and physics problems, right? And really helping 100 00:05:48,880 --> 00:05:52,077 people be able to achieve things they couldn't otherwise. like we are on the 101 00:05:52,160 --> 00:05:57,117 era on the edge of this era of agents really transforming how we all operate 102 00:05:57,200 --> 00:06:01,357 whether it's in software engineering finance legal sales and in our personal 103 00:06:01,440 --> 00:06:05,437 lives too. So just to unpack that example that you were giving there one 104 00:06:05,520 --> 00:06:10,077 of your colleagues is chatting with Chad GPT about an event and then suggest hey 105 00:06:10,160 --> 00:06:14,557 you know uh how should we you know contact event attendees about something 106 00:06:14,640 --> 00:06:17,997 and instead of like saying okay I have to do that and going into an event 107 00:06:18,080 --> 00:06:22,957 program basically what happens is the the interface will take over from there 108 00:06:23,040 --> 00:06:26,877 once it says it's a good idea and you agree and then hook into whatever tools 109 00:06:26,960 --> 00:06:28,557 you're using and then do it for you. 110 00:06:28,640 --> 00:06:33,357 >> Exactly. So it uses its you know Gmail connector searches through your inbox to 111 00:06:33,440 --> 00:06:35,517 find all the people who are attending 112 00:06:35,600 --> 00:06:38,637 and then if you're on the like what 113 00:06:38,720 --> 00:06:42,717 what'sever in dietary restriction sees oh these people I already have their 114 00:06:42,800 --> 00:06:44,237 dietary restrictions these people I do 115 00:06:44,320 --> 00:06:47,517 not um drafts an email depending on 116 00:06:47,600 --> 00:06:49,837 exactly how you have things set up it 117 00:06:49,920 --> 00:06:54,557 might say hey I drafted these emails can I send them if you have a connector that 118 00:06:54,640 --> 00:06:59,357 doesn't even let it send emails said I drafted it you need to send them And uh 119 00:06:59,440 --> 00:07:02,317 in a different world, you could also imagine that it's you've built enough 120 00:07:02,400 --> 00:07:05,037 trust with the system where it says I drafted the emails and I actually sent 121 00:07:05,120 --> 00:07:07,517 them. And I think that this actually 122 00:07:07,600 --> 00:07:12,397 points to a really important aspect of the agentic era, which is trust, right? 123 00:07:12,480 --> 00:07:15,597 That we need to really learn how to build trust with these systems, where 124 00:07:15,680 --> 00:07:18,957 they're good, where they're not. Figure out what you want to delegate to them 125 00:07:19,040 --> 00:07:23,197 and how you want to entrust them with responsibility. And that's something we 126 00:07:23,280 --> 00:07:25,997 view as earned, right? It's not something that that we can we can grant 127 00:07:26,080 --> 00:07:28,237 but by providing lots of tools and 128 00:07:28,320 --> 00:07:31,437 control and oversight and supervision to 129 00:07:31,520 --> 00:07:35,277 the operator to the person who that this AI is operating on behalf of like we 130 00:07:35,360 --> 00:07:38,317 think that that is going to be such an important thing and so that's a key 131 00:07:38,400 --> 00:07:40,237 product feature and differentiator. 132 00:07:40,320 --> 00:07:41,517 >> Yeah. Yeah. And when you go back to some 133 00:07:41,600 --> 00:07:43,437 of the early attempts at this, there was 134 00:07:43,520 --> 00:07:48,557 this like move that OpenAI had to let you call an Uber within Chat GPT. And it 135 00:07:48,640 --> 00:07:51,757 it followed a long line of companies that have tried to get you to take 136 00:07:51,840 --> 00:07:54,237 action within chat, but it never really 137 00:07:54,320 --> 00:07:57,277 took off. And the difference here might 138 00:07:57,360 --> 00:08:00,557 be that the chatbot can take control of 139 00:08:00,640 --> 00:08:05,197 your browser or take control of your computer and then you don't necessarily 140 00:08:05,280 --> 00:08:09,037 have to worry about like is this plug-in going to work? it goes and accomplishes 141 00:08:09,120 --> 00:08:11,197 that for you by taking over your 142 00:08:11,280 --> 00:08:14,317 machine. So I wonder you know if you 143 00:08:14,400 --> 00:08:17,437 expect a fight from the user interfaces 144 00:08:17,520 --> 00:08:19,837 that we have today aka like all the 145 00:08:19,920 --> 00:08:22,397 other apps all the software where to be 146 00:08:22,480 --> 00:08:25,597 truly useful chat GPT will have to not 147 00:08:25,680 --> 00:08:29,597 be blocked to be able to go out and execute these actions on behalf of a 148 00:08:29,680 --> 00:08:34,317 user. Well, look f first of all, I'd say that it's this is not theoretical at 149 00:08:34,400 --> 00:08:37,037 this point, right? That people have been using codec. So, it's a separate 150 00:08:37,120 --> 00:08:39,917 product, separate app. You have to install it separately. Really starting 151 00:08:40,000 --> 00:08:44,797 to focus on software engineering. But the amount of non-software work that has 152 00:08:44,880 --> 00:08:47,677 been happening codeex has been absolutely exploding, right? It's been 153 00:08:47,760 --> 00:08:51,837 this like incredible exponential curve, exactly the thing that you you would 154 00:08:51,920 --> 00:08:55,437 expect. And within OpenAI, we basically have the same level of penetration now 155 00:08:55,520 --> 00:08:58,957 in usage as Slack, right? It's like everyone and OpenI is like an entirely 156 00:08:59,040 --> 00:09:02,317 slack based company. We do not use email for the most part. It's like really like 157 00:09:02,400 --> 00:09:04,957 if you're not on Slack, you're not going to do any work. Um, and it's kind of 158 00:09:05,040 --> 00:09:09,117 feeling that way now with codeex app as well. And that everyone's codex is 159 00:09:09,200 --> 00:09:14,077 hooked up to all of these tools. Um, how the ecos ecosystem evolves. I think it's 160 00:09:14,160 --> 00:09:17,437 going to be a very nuanced thing because I think one thing that is very important 161 00:09:17,520 --> 00:09:22,317 is that we believe that there should be an ecosystem that gets to be vibrant and 162 00:09:22,400 --> 00:09:26,557 thriving and that people can really build and see the benefits. And so we've 163 00:09:26,640 --> 00:09:30,557 actually seen this from uh partner companies where uh you know that we I 164 00:09:30,640 --> 00:09:33,357 remember there's a couple different partners where we said hey we really 165 00:09:33,440 --> 00:09:36,317 want to train our AI to be really good at using your software and we didn't 166 00:09:36,400 --> 00:09:38,957 know what they would say and actually 167 00:09:39,040 --> 00:09:43,437 the response we got is this is the most partner friendly outreach we've ever had 168 00:09:43,520 --> 00:09:47,757 right that the idea that you will make your AI specifically good at using our 169 00:09:47,840 --> 00:09:51,837 tool and they just see the opportunity because their tool will be used just so 170 00:09:51,920 --> 00:09:56,557 much more as a result and that everyone is trying to think about how do they not 171 00:09:56,640 --> 00:10:00,637 just survive as a company into the AI era but thrive? like how do you really 172 00:10:00,720 --> 00:10:04,317 get the advantages of the fact there's going to be so much more activity and if 173 00:10:04,400 --> 00:10:08,317 you don't have AI in there if you shut it out then you're actually going to be 174 00:10:08,400 --> 00:10:12,077 declining not thriving >> right this this kind of makes open AI 175 00:10:12,160 --> 00:10:15,597 puts open AAI so first of all you're going to bring you talk about people 176 00:10:15,680 --> 00:10:19,757 using codeex so one of your colleagues shared and I think you've talked about 177 00:10:19,840 --> 00:10:22,317 this too that you've brought chat GPT 178 00:10:22,400 --> 00:10:27,277 into codeex so you can bring codeex into chat GPT which is basically like if 179 00:10:27,360 --> 00:10:30,797 we're users of chat GPT this experience that we talked about of 180 00:10:30,880 --> 00:10:34,957 chat GPT not only suggesting what you might want to do next but going to do it 181 00:10:35,040 --> 00:10:38,957 for you that's going to happen and so it makes you effectively an operating 182 00:10:39,040 --> 00:10:43,517 system don't you think but not the operating system like an iOS where you 183 00:10:43,600 --> 00:10:48,237 would like go open up the your phone and then tap different apps it's almost as 184 00:10:48,320 --> 00:10:52,397 if all interaction with all apps will happen through this interface is that 185 00:10:52,480 --> 00:10:57,037 the ambition >> I think that you could describe it that 186 00:10:57,120 --> 00:11:00,397 way but I think of it a little differently. Like the way that I think 187 00:11:00,480 --> 00:11:02,397 about this is that what is the ideal 188 00:11:02,480 --> 00:11:05,197 interface to an AGI or we call it kind 189 00:11:05,280 --> 00:11:10,557 of a personal AGI. And I think that it's again the same interface that you and I 190 00:11:10,640 --> 00:11:14,397 are using right now. You just want to talk to an assistant, right? You want to 191 00:11:14,480 --> 00:11:15,997 talk to something that can go and and 192 00:11:16,080 --> 00:11:18,877 and work and and operate on your behalf. 193 00:11:18,960 --> 00:11:21,997 And so that yes, like that agent, that 194 00:11:22,080 --> 00:11:26,717 AGI, that AI will have its own computer, right? It'll have its own access to 195 00:11:26,800 --> 00:11:30,477 things. maybe can you know like ideal co-orker would be they can come over and 196 00:11:30,560 --> 00:11:32,557 type things on your computer too. So 197 00:11:32,640 --> 00:11:35,037 some access some delegated access to 198 00:11:35,120 --> 00:11:37,917 your own own system and you know maybe 199 00:11:38,000 --> 00:11:42,557 you delegate access to your inbox sometimes maybe it has its own inbox 200 00:11:42,640 --> 00:11:46,557 with some sort of uh some sort of you know window into into the things that it 201 00:11:46,640 --> 00:11:50,957 needs you forward emails to it. Th these are not actually if you if you think 202 00:11:51,040 --> 00:11:52,637 about this is not unprecedented, right? 203 00:11:52,720 --> 00:11:55,277 It's like the way that you work with an 204 00:11:55,360 --> 00:11:59,997 assistant who's a person that we've we've actually or any co-orker really, 205 00:12:00,080 --> 00:12:02,957 we've spent a lot of time really thinking about how do you build these 206 00:12:03,040 --> 00:12:06,637 trust boundaries and make sure that you're able to operate together. And so 207 00:12:06,720 --> 00:12:08,637 I think of it as just a different thing. 208 00:12:08,720 --> 00:12:11,837 It's not it you could think of it as an operating system, but an operating 209 00:12:11,920 --> 00:12:15,597 system is almost something from a different time, right? It's a different 210 00:12:15,680 --> 00:12:19,197 layer of the stack. This is really more about how do you interface with 211 00:12:19,280 --> 00:12:22,157 technology broadly. And I think that the 212 00:12:22,240 --> 00:12:24,157 beautiful thing about AI is it's really 213 00:12:24,240 --> 00:12:27,597 about bringing the machine closer to the 214 00:12:27,680 --> 00:12:29,597 human rather than us having to contort 215 00:12:29,680 --> 00:12:34,797 ourselves into like files and folders and like all these details that somehow 216 00:12:34,880 --> 00:12:37,357 are not natural, right? That are more about how the machine operates rather 217 00:12:37,440 --> 00:12:39,837 than how we operate. >> Yeah. Talking about a personal 218 00:12:39,920 --> 00:12:44,077 intelligence, it sort of um I don't did you watch WWDC last week? 219 00:12:44,160 --> 00:12:46,077 >> Uh no, no, I missed it. I was I was 220 00:12:46,160 --> 00:12:50,237 banned, but um I watched it on TV. Um 221 00:12:50,320 --> 00:12:54,877 come on, Apple. Anyway, it does look like you and and and Siri, the new Siri 222 00:12:54,960 --> 00:12:58,557 are going to come kind of into competition, right? Because they're an 223 00:12:58,640 --> 00:13:01,677 app that's going to sit or an intelligence that will sit on top of all 224 00:13:01,760 --> 00:13:04,157 of your apps and let you take action. 225 00:13:04,240 --> 00:13:07,997 And Chacht will be an app on the iPhone. 226 00:13:08,080 --> 00:13:12,077 So then talk a little bit about whether that positioning is going to be 227 00:13:12,160 --> 00:13:15,357 difficult for open AI and how you're thinking about that strategically. 228 00:13:15,440 --> 00:13:18,717 >> Well, I just think again think of it a little differently. Like I think that 229 00:13:18,800 --> 00:13:21,037 we're in the beginning of this new 230 00:13:21,120 --> 00:13:25,757 agentic era and the way that this has always gone in AI is that when you have 231 00:13:25,840 --> 00:13:29,437 a new level of capability, it means you have an opportunity to rethink 232 00:13:29,520 --> 00:13:31,677 everything, right? Rethink how 233 00:13:31,760 --> 00:13:34,237 >> people interface, how like what the 234 00:13:34,320 --> 00:13:39,277 techn is capable of. And I think that this is no no different, right? In my 235 00:13:39,360 --> 00:13:41,117 mind, like the kinds of things that I 236 00:13:41,200 --> 00:13:43,117 see on the horizon, 237 00:13:43,200 --> 00:13:46,797 for example, AI for solving 238 00:13:46,880 --> 00:13:50,557 scientific problems, right? And I think we're starting to see the inklings of 239 00:13:50,640 --> 00:13:54,397 this. Like for example, today we announced 240 00:13:54,480 --> 00:13:59,997 we have in uh peer-reviewed literature people doctors who are using 03. 241 00:14:00,080 --> 00:14:03,517 Remember 03? >> Yep. >> That was like forever ago now, right? 242 00:14:03,600 --> 00:14:04,797 That was like one of our earliest 243 00:14:04,880 --> 00:14:08,317 reasoning models using that to find 244 00:14:08,400 --> 00:14:13,757 diagnosis for people who I had no no answers from doctors for many many 245 00:14:13,840 --> 00:14:17,037 years. You know, there's an example of someone who had spent 20 years with a 246 00:14:17,120 --> 00:14:20,717 mysterious ailment. Finally, it's been diagnosed through the use of this 247 00:14:20,800 --> 00:14:24,237 technology. And if you're like, okay, you've got models that can do that, they 248 00:14:24,320 --> 00:14:27,597 can do that. And then it's really about like, you know, the same like 249 00:14:27,680 --> 00:14:30,797 distribution and like, you know, can you get access to an app? You know, to me 250 00:14:30,880 --> 00:14:33,837 it's it doesn't type check. It's like we have something fundamentally new. And so 251 00:14:33,920 --> 00:14:36,477 that's not to say that there won't be competition. I actually think that there 252 00:14:36,560 --> 00:14:39,517 will be and it's going to be great for everyone, but I just think that the ways 253 00:14:39,600 --> 00:14:41,997 in which you're going to use this technology, the things it will be 254 00:14:42,080 --> 00:14:45,517 capable of and what it'll make you capable of doing are just totally 255 00:14:45,600 --> 00:14:47,437 different from anything we've seen before. 256 00:14:47,520 --> 00:14:51,117 >> You know, I was going to ask you um well, does it mean that you'll have to 257 00:14:51,200 --> 00:14:55,677 um you know, create your own device assuming that like my concept is is you 258 00:14:55,760 --> 00:14:59,037 know that you're going to have to go through Apple to get to the user. Um 259 00:14:59,120 --> 00:15:03,277 assuming that's somewhat valid. But the answer is you already are right. You're 260 00:15:03,360 --> 00:15:05,597 so OpenAI is working on a device right now. 261 00:15:05,680 --> 00:15:09,517 >> It certainly has been publicly reported. 262 00:15:09,600 --> 00:15:11,997 >> I I was in your office in December and 263 00:15:12,080 --> 00:15:16,557 Sam told me that this is happening. It's it's multiple devices. Um so if you 264 00:15:16,640 --> 00:15:21,037 think about the way that again you're going to interface with um with these 265 00:15:21,120 --> 00:15:26,397 AIs, how does that device play in or series of devices? Well, look, I I think 266 00:15:26,480 --> 00:15:27,837 again I I would just step back and say 267 00:15:27,920 --> 00:15:29,597 that I think this is the beginning of 268 00:15:29,680 --> 00:15:33,917 something very new and that I think 269 00:15:34,000 --> 00:15:37,997 about the way that 270 00:15:38,080 --> 00:15:41,357 I think I want to say like I think the biggest shift that has happened in terms 271 00:15:41,440 --> 00:15:44,157 of interface again it's not even about 272 00:15:44,240 --> 00:15:45,917 devices and and and things like that. 273 00:15:46,000 --> 00:15:49,917 It's really about the shift from conversational intelligence like kind of 274 00:15:50,000 --> 00:15:53,917 the chat paradigm where it's like kind of you have an AI that's personalized 275 00:15:54,000 --> 00:15:57,517 enough to you that it's worth reading its output, right? You ask it a 276 00:15:57,600 --> 00:16:01,677 question, you get an answer, it's something that's useful to you to agents 277 00:16:01,760 --> 00:16:03,277 where they're capable enough to actually 278 00:16:03,360 --> 00:16:05,677 do things for you. Like that is a big 279 00:16:05,760 --> 00:16:09,117 shift and that that 280 00:16:09,200 --> 00:16:12,077 implies a difference in how you want to 281 00:16:12,160 --> 00:16:13,757 interact. And so you kind of are just 282 00:16:13,840 --> 00:16:16,397 going to want a single agent that has 283 00:16:16,480 --> 00:16:21,517 access to your context. And this will be true in personal life. This will be true 284 00:16:21,600 --> 00:16:23,357 in a business context, right? You 285 00:16:23,440 --> 00:16:26,797 imagine, for example, having a uh you 286 00:16:26,880 --> 00:16:28,557 know, imagine you have a PhD in every 287 00:16:28,640 --> 00:16:33,837 field co-worker, you know, Nobel prizes, multiple of them, and you hire one of 288 00:16:33,920 --> 00:16:37,277 these, you hire a hundred of them, and you don't invite them to any meetings. 289 00:16:37,360 --> 00:16:40,877 They're not going to be very useful. And so there's something about how do you 290 00:16:40,960 --> 00:16:44,077 get context into the AI and not just 291 00:16:44,160 --> 00:16:48,077 statically but dynamically right as 292 00:16:48,160 --> 00:16:52,077 context evolves as your business processes evolve how do you have a 293 00:16:52,160 --> 00:16:55,997 context layer that is accessible to an AI that lets the AI operate to the 294 00:16:56,080 --> 00:16:59,037 extent of that raw intelligence and so 295 00:16:59,120 --> 00:17:02,317 finding ways to make that AI be 296 00:17:02,400 --> 00:17:06,637 accessible so available in your meetings to make it very ergonomic it's very easy 297 00:17:06,720 --> 00:17:09,277 to get access to Um, I think all of 298 00:17:09,360 --> 00:17:11,117 that's going to require a rethink, but I 299 00:17:11,200 --> 00:17:14,557 think it again it's just the core for me 300 00:17:14,640 --> 00:17:18,877 starts from thinking about the agentic form factor and then working backwards 301 00:17:18,960 --> 00:17:23,197 to how do you just make this have the context it needs and again the trust is 302 00:17:23,280 --> 00:17:26,877 going to be such a core part of making this whole equation work. 303 00:17:26,960 --> 00:17:30,957 >> So kind of like having this this device with you at all times and being like I 304 00:17:31,040 --> 00:17:34,637 need to get that done and it goes and does it for you. And I think that that 305 00:17:34,720 --> 00:17:38,717 will be part of it. But I almost even think if you don't have a device like 306 00:17:38,800 --> 00:17:43,117 that, it's not like you're going to be out of the game, right? Because it's 307 00:17:43,200 --> 00:17:45,917 this a this AI. It's not because there's one thing there's one version of it 308 00:17:46,000 --> 00:17:48,877 where you think of it where it's like the device is the AI and you want your 309 00:17:48,960 --> 00:17:51,917 phone to be the AI. You want, you know, whatever whatever you know custom device 310 00:17:52,000 --> 00:17:54,797 you're you're thinking about to be the AI, but it's not going to be like that. 311 00:17:54,880 --> 00:17:56,077 It's going to be more like an interface. 312 00:17:56,160 --> 00:17:59,517 Like no more than your phone is you, right? It's an interface to you. It's a 313 00:17:59,600 --> 00:18:02,717 way that I can sort of, you know, call you up whenever I need you. uh whenever 314 00:18:02,800 --> 00:18:05,997 I want to ask you a question and there's different ways of accessing right 315 00:18:06,080 --> 00:18:09,837 there's like synchronous phone call I can text you I can email you and I think 316 00:18:09,920 --> 00:18:13,517 that we're going to be much the same with how we interact with our agents uh 317 00:18:13,600 --> 00:18:17,437 there's been some reports that open is working on these like birectional voice 318 00:18:17,520 --> 00:18:21,117 models I think we've talked about that in the past like the goal is to have 319 00:18:21,200 --> 00:18:26,157 like an AI that you can you can speak with and we'll be able to process that 320 00:18:26,240 --> 00:18:29,837 and speak back with you in a much more natural way can you share anything about 321 00:18:29,920 --> 00:18:35,037 that No, [laughter] >> but no, more seriously, um I mean, look, 322 00:18:35,120 --> 00:18:38,637 I think that the that the general shape of the of the technology, like the way 323 00:18:38,720 --> 00:18:41,437 that like we had we had we've had voice 324 00:18:41,520 --> 00:18:45,757 models, um you know, kind of a really cool voice experience for, you know, 325 00:18:45,840 --> 00:18:47,757 year and a half, two years now. Um you 326 00:18:47,840 --> 00:18:50,237 know, we first demoed it back in March, 327 00:18:50,320 --> 00:18:53,437 April of 2024. Um brought it to market, 328 00:18:53,520 --> 00:18:58,157 um you know, maybe late that late that year. And the way that it works and the 329 00:18:58,240 --> 00:19:03,117 way that everyone's models work is that you basically chain together um well the 330 00:19:03,200 --> 00:19:06,477 original way that these things worked was that you would chain together a text 331 00:19:06,560 --> 00:19:09,437 to or a speechtoext model then you do a 332 00:19:09,520 --> 00:19:13,517 texttoext model and then you would do a texttospech model. Horribleness, right? 333 00:19:13,600 --> 00:19:17,117 Like these three things chained together. Um it still has been the case 334 00:19:17,200 --> 00:19:21,117 that even if you have one unified model that's able to kind of take in input and 335 00:19:21,200 --> 00:19:24,557 then you know able to output a response, you still have this problem of turn 336 00:19:24,640 --> 00:19:28,397 taking, right? Imagine that like we have this like you cannot overlap, you cannot 337 00:19:28,480 --> 00:19:32,877 interrupt. It's just like once you you you speak to me in a turn and then you 338 00:19:32,960 --> 00:19:36,077 got to wait for me to finish my whole response. That is not how human 339 00:19:36,160 --> 00:19:40,637 conversation works. And so that we we basically have like a hack where we have 340 00:19:40,720 --> 00:19:44,477 these models that determine oh it seems like the turn has ended and oh it seems 341 00:19:44,560 --> 00:19:47,757 like the turn has started and we're like why are we talking about turns, right? 342 00:19:47,840 --> 00:19:51,517 Like turns again are so unnatural. This is the humans contorting ourselves to 343 00:19:51,600 --> 00:19:55,037 the machine and its limitations. And so the obvious thing that you want to 344 00:19:55,120 --> 00:19:59,357 accomplish is a model in AI that works much more like you and I do, right? 345 00:19:59,440 --> 00:20:04,397 That's able to process input at the same time it's processing output. And all of 346 00:20:04,480 --> 00:20:07,197 that is of course something that many people in this field are trying to to 347 00:20:07,280 --> 00:20:09,037 run towards. Um I think it's going to be 348 00:20:09,120 --> 00:20:10,957 very very exciting as you move to these 349 00:20:11,040 --> 00:20:16,077 natural very human fluid like conversational interfaces. No one's seen 350 00:20:16,160 --> 00:20:19,837 anything like it. Like one thing that that I think about is the the current 351 00:20:19,920 --> 00:20:23,357 interaction with you know chat GBT voice. many ways it's magical, right? So 352 00:20:23,440 --> 00:20:27,277 many people use it on their commute, able to ask all these questions, but it 353 00:20:27,360 --> 00:20:30,317 also is so frustrating, right? Whenever it breaks the magic because it's like 354 00:20:30,400 --> 00:20:34,317 you realize, oh, I want to like add some followup and it keeps talking over you 355 00:20:34,400 --> 00:20:39,037 and it didn't it's just like that is just it doesn't make sense. And so I 356 00:20:39,120 --> 00:20:43,517 think that part of what we need part of like the whole point of this AI is to be 357 00:20:43,600 --> 00:20:47,117 something that you can interact with interact with fluidly and naturally. And 358 00:20:47,200 --> 00:20:48,477 by the way, I think it's not just going 359 00:20:48,560 --> 00:20:52,957 to be about the sort of use case, like 360 00:20:53,040 --> 00:20:55,917 we kind of think about the the personal use case, but it's also really the work 361 00:20:56,000 --> 00:20:59,357 use case. And I think some of the most magical experiences that that I've had 362 00:20:59,440 --> 00:21:03,917 with codecs have been when operating it through voice. Like many people, we have 363 00:21:04,000 --> 00:21:05,837 a voice built in. Some people use third 364 00:21:05,920 --> 00:21:10,957 party apps for it. Um, and that you just get a very different experience when you 365 00:21:11,040 --> 00:21:14,637 start to realize that like typing a quick message to give some feedback, 366 00:21:14,720 --> 00:21:18,157 easy, but like writing out a whole paragraph and everything you want, 367 00:21:18,240 --> 00:21:21,037 horrible. No one wants to do that, right? You just want to be like saying 368 00:21:21,120 --> 00:21:24,397 things and you want the real-time feedback loop and all of that is going 369 00:21:24,480 --> 00:21:26,557 to happen and it's going to be amazing. 370 00:21:26,640 --> 00:21:31,277 So, let's talk about model improvement briefly. Um, so there was a discussion a 371 00:21:31,360 --> 00:21:33,277 couple years ago that large language 372 00:21:33,360 --> 00:21:35,597 models were about to hit a wall. um that 373 00:21:35,680 --> 00:21:39,277 was wrong. And um something you know 374 00:21:39,360 --> 00:21:43,357 that I'm thinking about is I think we're all thinking about it is how much better 375 00:21:43,440 --> 00:21:45,597 can these models get and when will the 376 00:21:45,680 --> 00:21:49,997 improvement stop? Any thoughts? Well, I 377 00:21:50,080 --> 00:21:52,957 think that this is a place where when 378 00:21:53,040 --> 00:21:57,197 you're kind of building these models, you get kind of a sense and an intuition 379 00:21:57,280 --> 00:22:01,917 that I think is harder to get from the outside because we see all the data 380 00:22:02,000 --> 00:22:05,277 points and we see also the work that goes into these improvements. And so 381 00:22:05,360 --> 00:22:07,037 that there's two parts to the answer. 382 00:22:07,120 --> 00:22:08,477 One is I think that the fundamental 383 00:22:08,560 --> 00:22:11,597 science is one of the most mysterious 384 00:22:11,680 --> 00:22:14,477 and important 385 00:22:14,560 --> 00:22:18,637 just scientific discoveries and empirical observations that that I can 386 00:22:18,720 --> 00:22:21,917 that I that I'm aware of that I can imagine right that we are able to 387 00:22:22,000 --> 00:22:24,397 actually build these models and that the 388 00:22:24,480 --> 00:22:27,277 scaling laws continue right that it just 389 00:22:27,360 --> 00:22:29,357 is the case that you can just keep 390 00:22:29,440 --> 00:22:32,797 training these models more data 391 00:22:32,880 --> 00:22:36,717 more compute better architectures and there's a lot of improvements that go 392 00:22:36,800 --> 00:22:39,837 in. But every time we've kind of run into a like, oh, this isn't quite 393 00:22:39,920 --> 00:22:44,797 scaling the way we expect, it's we have a problem, we have a bug, that our math 394 00:22:44,880 --> 00:22:48,157 wasn't quite right, that oh, our implementation isn't isn't isn't quite 395 00:22:48,240 --> 00:22:53,197 matching the math, whatever the thing is. And that is, I think, a very 396 00:22:53,280 --> 00:22:56,717 important thing to to sort of internalize. And actually if you we've 397 00:22:56,800 --> 00:22:59,997 done studies where you go back to the beginning of the field right that neural 398 00:23:00,080 --> 00:23:02,397 nets themselves were designed in like 399 00:23:02,480 --> 00:23:05,437 1940s right before computers right as a 400 00:23:05,520 --> 00:23:09,677 model of maybe this is maybe this how the brain processes information first 401 00:23:09,760 --> 00:23:13,277 hardware implementation was 1959 with the perceptron and if you look at 402 00:23:13,360 --> 00:23:16,077 landmark results in the field that the 403 00:23:16,160 --> 00:23:20,557 landmark results follow this incredibly smooth deterministic path of more 404 00:23:20,640 --> 00:23:24,877 compute being poured into them and so 70 years of people maybe 80 years now of 405 00:23:24,960 --> 00:23:27,597 people saying this stuff is never going to work, never going to scale, going to 406 00:23:27,680 --> 00:23:29,997 hit the wall. Hasn't hit the wall yet. 407 00:23:30,080 --> 00:23:34,317 There's still no wall in sight. And so I think that the fundamentals allow it. 408 00:23:34,400 --> 00:23:36,797 Now the practicality is hard, right? 409 00:23:36,880 --> 00:23:40,477 Actually building these massive supercomputers. It's hard. It's 410 00:23:40,560 --> 00:23:44,477 expensive. It's not easy, right? That we have teams that just like work so hard 411 00:23:44,560 --> 00:23:48,797 to solve these incredibly hard technical problems. We have our own network 412 00:23:48,880 --> 00:23:52,637 protocol that we've had to design. um that we have people who look at every 413 00:23:52,720 --> 00:23:56,157 single layer of the stack that there's weird wiggles in the graph and you the 414 00:23:56,240 --> 00:23:58,717 way to think about these neural nets is that there's like there's no 415 00:23:58,800 --> 00:24:02,317 abstractions, right? It's almost like any little piece that's wrong can have a 416 00:24:02,400 --> 00:24:05,677 ripple effect that only shows up down there. And so you need people to deeply 417 00:24:05,760 --> 00:24:09,277 understand all of it. And yet if you get 418 00:24:09,360 --> 00:24:11,037 the right team together, put the right 419 00:24:11,120 --> 00:24:13,597 mission in front of people and people do 420 00:24:13,680 --> 00:24:18,637 that grind, the outcome, it's worth it, right? And it's achievable and it's 421 00:24:18,720 --> 00:24:22,157 possible. And so I think that for those reasons the progress will continue. 422 00:24:22,240 --> 00:24:25,837 >> So then I'd love to hear your perspective if if models can basically 423 00:24:25,920 --> 00:24:28,157 progress much further from where they 424 00:24:28,240 --> 00:24:31,357 are today. Um let's say let's say open 425 00:24:31,440 --> 00:24:33,997 AI builds the best model and it the 426 00:24:34,080 --> 00:24:38,397 equivalent of like something with like 15 PhDs with excellent emotional 427 00:24:38,480 --> 00:24:40,557 intelligence that doesn't complain and 428 00:24:40,640 --> 00:24:43,517 goes out and does stuff for you. Um, and 429 00:24:43,600 --> 00:24:47,437 then the the next model maker will build 430 00:24:47,520 --> 00:24:50,077 a a less good, but it has 13 PhDs and 431 00:24:50,160 --> 00:24:54,477 it's like pretty good uh, you know, EQ and we'll still go and do things for 432 00:24:54,560 --> 00:24:58,717 you. So, where does the differentiation come in when you get to that level of 433 00:24:58,800 --> 00:25:02,957 intelligence? Because we've seen the model makers kind of move in lock step. 434 00:25:03,040 --> 00:25:06,557 One makes an advance, the next one comes in and makes the advance. So, they all 435 00:25:06,640 --> 00:25:11,037 become that smart. Do they is it possible to differentiate? Well, I think 436 00:25:11,120 --> 00:25:15,757 there are several dimensions to the answer. Um, number one is I do think 437 00:25:15,840 --> 00:25:19,917 there's a bit of an attractor state where just like from a business model 438 00:25:20,000 --> 00:25:23,837 perspective, every provider sells out all their compute. 439 00:25:23,920 --> 00:25:26,797 Okay. Right. I think that is just like the world that we're heading towards 440 00:25:26,880 --> 00:25:29,997 where there just is not going to be enough compute to serve all the demand. 441 00:25:30,080 --> 00:25:32,797 Right? That we're heading to this compute powered economy that everyone's 442 00:25:32,880 --> 00:25:36,477 going to be using these models all the time to be able to accomplish tasks of 443 00:25:36,560 --> 00:25:40,797 interest. And we just see it. It's like right now we're talking about compute 444 00:25:40,880 --> 00:25:43,197 constraints and like like the number of 445 00:25:43,280 --> 00:25:45,837 people using these agents is like order 446 00:25:45,920 --> 00:25:47,997 of 10 million 20 million maybe you know 447 00:25:48,080 --> 00:25:53,117 it's like we're not at planet scale chatbt is like a billion users right but 448 00:25:53,200 --> 00:25:55,597 we haven't brought the agentic power there yet so you're just looking at 449 00:25:55,680 --> 00:25:59,597 these factors and the depth of usage is also tiny compared to where we're going 450 00:25:59,680 --> 00:26:03,277 and so I think that we're just going to be in a world where even if you have 451 00:26:03,360 --> 00:26:07,677 different vendors different capability level open source models all these 452 00:26:07,760 --> 00:26:11,437 things these neoclouds like I think that compute is just going to be the scarce 453 00:26:11,520 --> 00:26:16,317 resource and I think that it's going to go go to use. Um so to some extent I 454 00:26:16,400 --> 00:26:20,077 think that the the like is this a good business to be in and for new entrance 455 00:26:20,160 --> 00:26:22,877 to come into and things like that. My answer is actually yes. I think that 456 00:26:22,960 --> 00:26:27,437 there is like a huge market um that we are just not going to be able to address 457 00:26:27,520 --> 00:26:31,917 and we need much more energy momentum there. Um but a second thing is that it 458 00:26:32,000 --> 00:26:33,677 also misses the fact that intelligence 459 00:26:33,760 --> 00:26:36,477 is not a undimensional thing right that 460 00:26:36,560 --> 00:26:41,837 that If you really zoom in, being good at different domains is something where 461 00:26:41,920 --> 00:26:45,357 even if you you have a lot of raw intelligence, getting good if you've 462 00:26:45,440 --> 00:26:49,597 never practiced like you've never actually done a pitch or something like 463 00:26:49,680 --> 00:26:52,717 you're not going to be good at it your first time, right? And that there's lots 464 00:26:52,800 --> 00:26:55,517 of different you've never operated a spreadsheet, right? You're not going to 465 00:26:55,600 --> 00:27:00,317 be able to succeed at doing some complex modeling. And so I think that there is 466 00:27:00,400 --> 00:27:03,437 something that we have been internalizing which is that we look 467 00:27:03,520 --> 00:27:05,917 across different industries and different domains and we have to 468 00:27:06,000 --> 00:27:11,117 prioritize. We can't possibly be great at every single area at once. There is 469 00:27:11,200 --> 00:27:14,477 definitely a lot of like, hey, you just get the general intelligence up and 470 00:27:14,560 --> 00:27:18,157 it'll experience a lot of these things, but to really become a domain expert, to 471 00:27:18,240 --> 00:27:22,237 really be that PhD, and to really be something that can help push forward the 472 00:27:22,320 --> 00:27:24,957 ambition of a field, like that's hard. 473 00:27:25,040 --> 00:27:28,077 And and by the way, one thing I also want to say is that I think 474 00:27:28,160 --> 00:27:31,357 understanding what happens when you successfully do that. I think that 475 00:27:31,440 --> 00:27:33,677 having a good mental model of that is important, which is you look at 476 00:27:33,760 --> 00:27:36,157 something like rewind to AlphaGo, right? 477 00:27:36,240 --> 00:27:38,317 you remember move 37, this move that 478 00:27:38,400 --> 00:27:43,517 like changed people's understanding of the game and then now more people play 479 00:27:43,600 --> 00:27:47,037 go than ever, right? And that it actually inspired people to do even 480 00:27:47,120 --> 00:27:50,397 more. I think we're just going to see that. And so I think that the depth is 481 00:27:50,480 --> 00:27:54,877 never going to stop, right? How deep can you go on science, right? Like I I think 482 00:27:54,960 --> 00:27:58,237 that people have thought sometimes that, hey, we found out all the physics, it's 483 00:27:58,320 --> 00:28:02,157 all good. We're all done. And I don't think that that's the future we're 484 00:28:02,240 --> 00:28:04,797 signed up for. I think we're signed up for one where we're got to keep find 485 00:28:04,880 --> 00:28:08,157 every time you unlock one mystery, >> every every time you solve one mystery, 486 00:28:08,240 --> 00:28:11,037 it unlocks like 10 more, right? So, I think that there's just going to be so 487 00:28:11,120 --> 00:28:14,237 much more to do and tons of room for differentiation across different 488 00:28:14,320 --> 00:28:15,757 companies. >> So, I think I'm reading you right and 489 00:28:15,840 --> 00:28:20,797 that your belief is um maybe there's a way that everybody can scale up these 490 00:28:20,880 --> 00:28:25,197 models, but ultimately the company with the most compute is going to win. And 491 00:28:25,280 --> 00:28:29,197 you know, we spoke a couple months ago and you had mentioned that like you were 492 00:28:29,280 --> 00:28:32,877 asked internally, how much compute should we buy? And you said, "All of 493 00:28:32,960 --> 00:28:36,237 it." And they said, "No, really, how much should we buy?" And you said, "No, 494 00:28:36,320 --> 00:28:40,717 buy all of it." And and OpenAI is definitely the leader in in buying 495 00:28:40,800 --> 00:28:44,637 compute. I mean, we see the the the money going out. Obviously, a lot of 496 00:28:44,720 --> 00:28:48,557 money coming in through investment and and now you you're you've built a 497 00:28:48,640 --> 00:28:52,237 business with customers, but there's a lot of money going out. Do you ever 498 00:28:52,320 --> 00:28:56,557 wonder, hey, are like, do you ever wonder maybe we're 499 00:28:56,640 --> 00:28:58,957 not going to be able to pay all this money back because it's a brand new 500 00:28:59,040 --> 00:29:03,437 category? Well, the way that I look at it is on the fundamentals, right? You 501 00:29:03,520 --> 00:29:05,677 you need to really look at the fact that 502 00:29:05,760 --> 00:29:10,957 the way that compute goes is that it's multiple years out before compute 503 00:29:11,040 --> 00:29:13,837 actually arrives, right? Depending on exactly what you're doing. For example, 504 00:29:13,920 --> 00:29:18,557 we've been investing in our own chip program now for multiple years. And 505 00:29:18,640 --> 00:29:20,557 super exciting progress like you know, 506 00:29:20,640 --> 00:29:22,557 we'll we'll we'll have more to announce 507 00:29:22,640 --> 00:29:26,157 uh actually pretty soon. Um but the fact 508 00:29:26,240 --> 00:29:29,837 that we're able to do that is something very unique, right? really think about 509 00:29:29,920 --> 00:29:32,717 the full vertical integration of the supply chain and I think that the world 510 00:29:32,800 --> 00:29:36,317 we're heading towards is one where again there's just not going to be enough 511 00:29:36,400 --> 00:29:40,157 compute in the world to satisfy all the demand and we see this very concretely 512 00:29:40,240 --> 00:29:44,157 like you look at the exponential I mean rewind to the exponential of chatbt um 513 00:29:44,240 --> 00:29:46,157 look at the exponentials we're on now uh 514 00:29:46,240 --> 00:29:47,517 you think about the problems that we 515 00:29:47,600 --> 00:29:51,837 that we are able to solve you know 516 00:29:51,920 --> 00:29:56,397 it's it's actually kind of interesting that yeah we just yesterday announced um 517 00:29:56,480 --> 00:29:58,557 it was two days ago announced a new 518 00:29:58,640 --> 00:30:02,077 result in uh uh basically chemistry and 519 00:30:02,160 --> 00:30:06,077 being able to synthesize you know new new improved reaction and all of this is 520 00:30:06,160 --> 00:30:09,357 without much attention you know the thing I just said of if you go deep in a 521 00:30:09,440 --> 00:30:12,557 domain you can really transform it and we're not even scratching the surface 522 00:30:12,640 --> 00:30:16,717 yet and so the way to think about is the economy is so massive right and we see 523 00:30:16,800 --> 00:30:19,837 it very concretely in terms of our own growth in terms of what people are 524 00:30:19,920 --> 00:30:23,757 willing to pay and kind of the size and growth of this whole industry and so I 525 00:30:23,840 --> 00:30:27,997 think that the thing that I think about the most is how do we meet the demand 526 00:30:28,080 --> 00:30:31,517 and how do you actually have something that can help support all of the work 527 00:30:31,600 --> 00:30:32,957 that people want to do in the economy? 528 00:30:33,040 --> 00:30:35,837 And I think that is such a vast thing. I don't think any of us have internalized 529 00:30:35,920 --> 00:30:39,677 it yet. Hi everyone, Alex Canitz here. I want to tell you about a documentary 530 00:30:39,760 --> 00:30:41,917 I've made with Gravity to explore the 531 00:30:42,000 --> 00:30:44,717 future of AI agent security. To find out 532 00:30:44,800 --> 00:30:46,717 if we're truly ready for autonomous 533 00:30:46,800 --> 00:30:49,597 agents, I sat down with MIT professor 534 00:30:49,680 --> 00:30:52,077 Romesh Rosker, former White House CIO 535 00:30:52,160 --> 00:30:54,397 Terresa Payton, Michelin's group chief 536 00:30:54,480 --> 00:30:57,677 data and AI officer Ambbea Roger Gopal, 537 00:30:57,760 --> 00:31:01,837 and Sharon Guy, a former executive at Alibaba. [music] 538 00:31:01,920 --> 00:31:04,237 They each offer unique insights into 539 00:31:04,320 --> 00:31:06,797 this evolving landscape. We conclude 540 00:31:06,880 --> 00:31:09,597 with Rory Blendell, CEO of Gravity, to 541 00:31:09,680 --> 00:31:13,917 discuss the path forward. with gravity leading the way. Join us on this 542 00:31:14,000 --> 00:31:17,357 journey. You [music] can watch the full documentary at the link in the show 543 00:31:17,440 --> 00:31:21,890 notes. [music] 544 00:31:27,120 --> 00:31:31,197 Yeah, but if I may, there is a price war brewing. I mean, at least that's 545 00:31:31,280 --> 00:31:33,917 according to the reports. It's great to have you here to talk about it. The Wall 546 00:31:34,000 --> 00:31:35,917 Street Journal recently had a report 547 00:31:36,000 --> 00:31:38,797 that an upcoming OpenAI model might have 548 00:31:38,880 --> 00:31:42,077 significant price cuts. And so again, 549 00:31:42,160 --> 00:31:45,117 like how can you know if it requires so 550 00:31:45,200 --> 00:31:50,477 much resources to serve this demand and it is growing demand um in an 551 00:31:50,560 --> 00:31:54,157 environment where there might be price cuts, how do you make that math work? 552 00:31:54,240 --> 00:31:58,317 >> Well, again, I look at it from a different angle. So if you look at the 553 00:31:58,400 --> 00:32:02,237 whole history of what we've done, we actually have been increasing the 554 00:32:02,320 --> 00:32:06,637 intelligence cutting price right for a fixed amount of intelligence and people 555 00:32:06,720 --> 00:32:09,837 somehow just like like the Japanese paradox just keeps happening. And so I 556 00:32:09,920 --> 00:32:13,197 think frontier intelligence will always be something that is going to be you 557 00:32:13,280 --> 00:32:15,837 know it's always going to be the priciest thing but I think that a year 558 00:32:15,920 --> 00:32:18,957 from now that level of intelligence is going to feel pretty mundane and like 559 00:32:19,040 --> 00:32:23,437 you know going to be much more available. Um, and I think that that the 560 00:32:23,520 --> 00:32:26,957 the world that we're in is one where people are starting to really think 561 00:32:27,040 --> 00:32:29,917 about value. And it's actually been a 562 00:32:30,000 --> 00:32:34,557 very interesting shift where over the past, you know, first quarter, maybe up 563 00:32:34,640 --> 00:32:38,557 until now, people have just been like this AI agent stuff, it's all new. We 564 00:32:38,640 --> 00:32:41,197 need to bring into our enterprise. Like, we don't want to be left behind. How do 565 00:32:41,280 --> 00:32:45,117 we be part of this this future? And now people are like, okay, like let's make 566 00:32:45,200 --> 00:32:48,077 sure this actually delivering ROI and value. And I actually think that's a 567 00:32:48,160 --> 00:32:51,357 great place to be, right? because people are asking the right questions. And I 568 00:32:51,440 --> 00:32:54,237 hear this I I had some customer meetings today where people were saying exactly 569 00:32:54,320 --> 00:32:58,077 this. They were like, "How can we have even just like good spend controls? How 570 00:32:58,160 --> 00:33:02,637 can we have observability?" And I I think we literally today just released 571 00:33:02,720 --> 00:33:05,117 spend controls. So, you know, it's like >> Okay. 572 00:33:05,200 --> 00:33:09,117 >> Exactly. We're we are really investing hard in enterprise readiness and the 573 00:33:09,200 --> 00:33:10,877 tools that our customers are telling us 574 00:33:10,960 --> 00:33:13,277 that they need. And I think that that 575 00:33:13,360 --> 00:33:18,317 for me is the shift that we've also been going through as a company is really not 576 00:33:18,400 --> 00:33:21,117 just thinking about hey we're just going to release models and you know have a 577 00:33:21,200 --> 00:33:24,077 model really thinking about the endto end of the business. How do we bring 578 00:33:24,160 --> 00:33:26,397 this into solving real problems for real 579 00:33:26,480 --> 00:33:29,677 customers and that is happening so 580 00:33:29,760 --> 00:33:32,397 quickly across every single industry and 581 00:33:32,480 --> 00:33:37,357 the number of different companies that still feel like they're wrapping their 582 00:33:37,440 --> 00:33:40,317 mind around how to best make use of these models we're learning at the same 583 00:33:40,400 --> 00:33:44,877 time. I think it's just so early in this whole game to me the the like the 584 00:33:44,960 --> 00:33:48,797 absolute size of the market growing so quickly, our own revenue ramp growing so 585 00:33:48,880 --> 00:33:52,077 quickly. I think it's still just like none of us are anticipating how steep 586 00:33:52,160 --> 00:33:55,837 that's all going to go. >> Are you going to cut prices? 587 00:33:55,920 --> 00:34:00,317 >> So again, the answer is always yes, right? But it's about like I think that 588 00:34:00,400 --> 00:34:03,677 what's going to keep happening is that we're going to have frontier models. Um 589 00:34:03,760 --> 00:34:06,957 I don't think there's going to be like a massive shift in in the short term. I 590 00:34:07,040 --> 00:34:09,197 don't think that that is the kind of thing that's going to happen. But I 591 00:34:09,280 --> 00:34:14,077 think the thing you should anticipate is that over a year-long time horizon to 592 00:34:14,160 --> 00:34:17,517 get to today's level of intelligence that feels very premier, it's going to 593 00:34:17,600 --> 00:34:21,197 be much cheaper. But there's going to be a new thing that is going to be so much 594 00:34:21,280 --> 00:34:23,757 better and you're going to be like, why would I ever use this other one, right? 595 00:34:23,840 --> 00:34:26,397 It's [clears throat] just it's just how it's always going to be. 596 00:34:26,480 --> 00:34:28,317 >> So Sadia Nadell has had some interesting 597 00:34:28,400 --> 00:34:32,557 tweets and interviews recently. He recently said, "The model is becoming a 598 00:34:32,640 --> 00:34:35,917 commodity and the valuable asset is company," or this might be a paraphrase, 599 00:34:36,000 --> 00:34:39,597 "The valuable asset is a company specific AI system that continually 600 00:34:39,680 --> 00:34:42,877 learns from your data." Um, what do you 601 00:34:42,960 --> 00:34:44,957 think about that? And is it weird to be 602 00:34:45,040 --> 00:34:46,877 competing with Microsoft now? 603 00:34:46,960 --> 00:34:49,917 >> Well, look, I don't think that there's 604 00:34:50,000 --> 00:34:53,917 any layer of the stack here that is going to just kind of be removed from 605 00:34:54,000 --> 00:34:57,997 the value chain. I think that these things multiplied together. And if you 606 00:34:58,080 --> 00:35:00,317 think about the most the base layer of 607 00:35:00,400 --> 00:35:05,597 compute right that is something where it's just like no compute no AI 608 00:35:05,680 --> 00:35:08,397 and to some extent you could say oh comput is commoditized it's just flops 609 00:35:08,480 --> 00:35:12,077 who cares about it but in reality like you look at you look at today's chip 610 00:35:12,160 --> 00:35:15,277 stocks you look at the uh you know people who are selling compute kind of 611 00:35:15,360 --> 00:35:18,717 what what is what the market is valuing people at and they see that there's a 612 00:35:18,800 --> 00:35:22,397 fundamental asset here that is just so critical and I think that is because it 613 00:35:22,480 --> 00:35:27,437 is a revenue center it is something that anyone who's building AI has to rely on 614 00:35:27,520 --> 00:35:30,797 and that there's a bunch of very interesting dynamics in terms of the 615 00:35:30,880 --> 00:35:33,597 efficiencies that you can squeeze out and the margins, all these things. But 616 00:35:33,680 --> 00:35:36,477 fundamentally, even though it's like you can kind of squint it and say it's 617 00:35:36,560 --> 00:35:40,637 commoditized, it's not. It's it's it's not that that the value goes away. It's 618 00:35:40,720 --> 00:35:43,757 not that the margins go away. It's like something that the market will reward 619 00:35:43,840 --> 00:35:47,677 because it has fundamental value and that the importance of it's going to go 620 00:35:47,760 --> 00:35:50,477 up over time. You can see that with some of the prices that people are paying for 621 00:35:50,560 --> 00:35:55,357 H100s, right? Hoppers are, you know, kind of, you know, not not obsolete, 622 00:35:55,440 --> 00:36:00,317 right? They're they're they're a a previous gen chip and uh nor in any 623 00:36:00,400 --> 00:36:03,357 normal situation where you're not totally supply constrained, no one would 624 00:36:03,440 --> 00:36:07,197 be buying them. But instead, the market prices are up relative to to where they 625 00:36:07,280 --> 00:36:10,157 were before. So there's this inversion that's happening and again I think it's 626 00:36:10,240 --> 00:36:13,517 going to keep happening where because 627 00:36:13,600 --> 00:36:17,917 everyone has this avalanche of demand that you're going to see prices and 628 00:36:18,000 --> 00:36:21,837 margins and all of these things AC continuing to increase at various levels 629 00:36:21,920 --> 00:36:24,717 of the stack. And >> I think the same kind of applies for 630 00:36:24,800 --> 00:36:28,317 models where the models themselves are 631 00:36:28,400 --> 00:36:32,717 also again they're not there's there's a lot of competition there and I think 632 00:36:32,800 --> 00:36:35,357 that's very good. I think it's good for the enterprise. I think it's good for 633 00:36:35,440 --> 00:36:37,517 customers, consumers. Um but I think 634 00:36:37,600 --> 00:36:42,397 that there's a lot of areas where for example our models have always been the 635 00:36:42,480 --> 00:36:45,437 sort of smartest ones, right? The ones that are able to solve these incredibly 636 00:36:45,520 --> 00:36:49,197 hard problems. I think we're just starting to reach a phase where you're 637 00:36:49,280 --> 00:36:52,157 going to see the transformative impact from that, right? It's like if we're 638 00:36:52,240 --> 00:36:56,877 really able to speed up science through models, the smarter the model, the 639 00:36:56,960 --> 00:37:00,557 faster it's going to go. And it's very very different from a model that has a 640 00:37:00,640 --> 00:37:03,677 conversational interface that you're able to, you know, is able to book your 641 00:37:03,760 --> 00:37:07,037 travel, right? Or organize your calendar. So that's also a dimension I 642 00:37:07,120 --> 00:37:09,837 think we're going to do a very good job in. But I'm just saying it's it's a 643 00:37:09,920 --> 00:37:14,317 different area. Um and then I think that the question of well how do you actually 644 00:37:14,400 --> 00:37:19,277 connect the intelligence to your own customers right to real value to you 645 00:37:19,360 --> 00:37:23,277 have all these enterprises that have built incredible businesses in different 646 00:37:23,360 --> 00:37:26,797 domains and it's a huge thing and it's not something where if you don't have 647 00:37:26,880 --> 00:37:29,837 domain expertise that you're just going to be able to do right and part of it is 648 00:37:29,920 --> 00:37:33,037 that you need you think about regulated industries you think about any area 649 00:37:33,120 --> 00:37:37,037 where there's like you know think about education where you have a parent you 650 00:37:37,120 --> 00:37:39,997 have a teacher you have student you have these different parties that need to 651 00:37:40,080 --> 00:37:44,877 interact in very thoughtful ways for all of these areas, all of these domains 652 00:37:44,960 --> 00:37:49,757 that there's a lot of value to be built by being in that area and thinking about 653 00:37:49,840 --> 00:37:53,117 how the workflow should work, how these models should be orchestrated. And so I 654 00:37:53,200 --> 00:37:57,437 I really think that there's more than enough to go around and I think that we 655 00:37:57,520 --> 00:38:00,557 have to work together as a whole ecosystem in order to deliver the kind 656 00:38:00,640 --> 00:38:02,797 of value that I think is possible from these systems. 657 00:38:02,880 --> 00:38:06,957 >> Okay, just to go back to the SA point one more time, uh he's called models a 658 00:38:07,040 --> 00:38:10,877 commodity. He's trying to build his own frontier intelligence. He's telling 659 00:38:10,960 --> 00:38:14,077 potentially your customers, hey, you got to come work with us because we're going 660 00:38:14,160 --> 00:38:18,317 to help build these loops that will learn from your data. Um, he's got 661 00:38:18,400 --> 00:38:20,877 access to your IP, I think, till 2032. 662 00:38:20,960 --> 00:38:24,077 So, how does it make you feel to hear this coming from Saudia? 663 00:38:24,160 --> 00:38:25,997 >> Look, I think that the most important 664 00:38:26,080 --> 00:38:31,277 thing that is happening right now is the usage of AI in the economy to really 665 00:38:31,360 --> 00:38:34,957 transform the economy and to uplift everyone. And so, I think that that is 666 00:38:35,040 --> 00:38:36,157 something that I'm really focused on. 667 00:38:36,240 --> 00:38:39,997 And the more that people are trying to make that happen, I think that that's 668 00:38:40,080 --> 00:38:41,597 better for everyone. 669 00:38:41,680 --> 00:38:45,277 >> Um, GPT 5.6 is rumored to be on its way. 670 00:38:45,360 --> 00:38:49,197 Supposed to be this just a Twitter rumor, but I'm going 671 00:38:49,280 --> 00:38:51,517 to read it to you. Um, always the best 672 00:38:51,600 --> 00:38:54,157 rumors. Three times cheaper than Fable 673 00:38:54,240 --> 00:38:57,277 up to 1.5 million token context. Uh, 674 00:38:57,360 --> 00:39:00,557 stronger agentic coding workflows. Um, 675 00:39:00,640 --> 00:39:05,497 how much of that is true? What should we expect for GPT 5.6? 676 00:39:05,580 --> 00:39:08,397 >> [laughter] >> I mean, look, you should always expect 677 00:39:08,480 --> 00:39:11,837 better, faster, smarter, the whole 678 00:39:11,920 --> 00:39:15,977 thing. >> So, everything confirmed. 679 00:39:16,060 --> 00:39:19,197 [laughter] >> Definitely believe everything you read 680 00:39:19,280 --> 00:39:21,597 on Twitter. Yeah, maybe not. 681 00:39:21,680 --> 00:39:22,797 >> That has actually been a source of 682 00:39:22,880 --> 00:39:26,237 problems in my personal life. Um, okay. 683 00:39:26,320 --> 00:39:29,837 So, uh, I want to end on health. Um, you brought it up a couple times. You 684 00:39:29,920 --> 00:39:33,597 actually had a question in the audience about it earlier. Um, you know, 685 00:39:33,680 --> 00:39:35,677 sometimes there's a story and you read 686 00:39:35,760 --> 00:39:38,957 it and you say to yourself, I know this 687 00:39:39,040 --> 00:39:41,197 person is speaking to the media and I 688 00:39:41,280 --> 00:39:43,357 know that what they're saying sounds 689 00:39:43,440 --> 00:39:46,317 like maybe it's true. Um, but there's 690 00:39:46,400 --> 00:39:50,477 something wrong with the story and we're not going to see more of it. Um, and 691 00:39:50,560 --> 00:39:52,477 I've read a couple of those recently. 692 00:39:52,560 --> 00:39:54,637 Um, one is I think is it your friend the 693 00:39:54,720 --> 00:39:59,437 GitLab CEO um, Sid Sberage? He had um he 694 00:39:59,520 --> 00:40:02,877 he got cancer and used uh he got all the 695 00:40:02,960 --> 00:40:05,677 diagnostic testing uh he could have so 696 00:40:05,760 --> 00:40:10,477 just went out and tested like crazy and fed that data into chatt with the 697 00:40:10,560 --> 00:40:12,477 assistance of some people who had built 698 00:40:12,560 --> 00:40:17,517 purpose-built application for it and was able I don't know if cure is the right 699 00:40:17,600 --> 00:40:22,077 word but to beat back the cancer to a degree. There was also this dog Rosie 700 00:40:22,160 --> 00:40:26,797 the dog in Australia. You guys heard of Rosie? like the craziest story where 701 00:40:26,880 --> 00:40:28,637 this guy I'm gonna get some detail wrong 702 00:40:28,720 --> 00:40:33,117 but a guy um biopsied his dog which had 703 00:40:33,200 --> 00:40:36,717 cancer um ran the mutations across of 704 00:40:36,800 --> 00:40:39,277 across Alphafold and then was able to 705 00:40:39,360 --> 00:40:41,917 design an mRNA vaccine that he injected 706 00:40:42,000 --> 00:40:44,877 into the dog um with the assistance of 707 00:40:44,960 --> 00:40:47,197 chatbots to build this thing which ended 708 00:40:47,280 --> 00:40:49,757 up being able to jump over tables again 709 00:40:49,840 --> 00:40:52,157 and the tumor shrunk. Um, when we think 710 00:40:52,240 --> 00:40:55,757 about the future of AI and health, um, 711 00:40:55,840 --> 00:40:57,837 help us sort out the truth with this 712 00:40:57,920 --> 00:41:02,877 question. Are these a couple of outliers that made good headlines, but there was 713 00:41:02,960 --> 00:41:05,677 something about the story we weren't hearing, or is this going to become 714 00:41:05,760 --> 00:41:09,037 standard in the future? >> Absolutely going to become standard. 715 00:41:09,120 --> 00:41:11,277 Absolutely. And it's I I personally have 716 00:41:11,360 --> 00:41:14,557 a number of friends who have done very 717 00:41:14,640 --> 00:41:18,477 similar things of get the data right 718 00:41:18,560 --> 00:41:20,797 your health diagnostics 719 00:41:20,880 --> 00:41:26,237 and use codecs right use these models to get insights from them and I think that 720 00:41:26,320 --> 00:41:27,757 there are many people like I think that 721 00:41:27,840 --> 00:41:31,277 there's about 230 million people each 722 00:41:31,360 --> 00:41:35,437 week who use chat GBT for health queries 723 00:41:35,520 --> 00:41:39,597 right and that's been that's this is like a staggering scale Right. And this 724 00:41:39,680 --> 00:41:44,157 is pe these are people sometimes you upload a scan. Sometimes you have 725 00:41:44,240 --> 00:41:48,477 doctors who are telling you conflicting information. And I think that we've been 726 00:41:48,560 --> 00:41:51,997 in a world where patients are not empowered, right? Patients have to be 727 00:41:52,080 --> 00:41:53,837 the doctor, right? You're the decider. 728 00:41:53,920 --> 00:41:57,597 You are accountable, right? You know, doctor makes a mistake and you're going 729 00:41:57,680 --> 00:42:01,597 to be paying the price for the rest of your life. Like it's just it's a very 730 00:42:01,680 --> 00:42:05,677 different kind of incentive. And this is very personal for me. you know, my wife 731 00:42:05,760 --> 00:42:08,797 has a number of health conditions and I think that we've just been we've not 732 00:42:08,880 --> 00:42:11,437 like I don't even know how we'd be able to manage many of her conditions right 733 00:42:11,520 --> 00:42:15,037 now without the use of of chat. And I think we're just at the beginning of 734 00:42:15,120 --> 00:42:19,677 this journey, right? That I think that the degree to which 735 00:42:19,760 --> 00:42:24,237 even if you have the best medical team, the best access, the best best experts, 736 00:42:24,320 --> 00:42:28,957 there's only so much that can be done, right? That you think about the things 737 00:42:29,040 --> 00:42:31,757 that are just outside of the reach of humanity or even just sometimes it's 738 00:42:31,840 --> 00:42:35,677 like someone didn't even read the chart, right? and kind of missed a detail. All 739 00:42:35,760 --> 00:42:39,517 of that we should be able to improve massively through these tools. And so I 740 00:42:39,600 --> 00:42:42,317 think that the personalized medicine and 741 00:42:42,400 --> 00:42:47,357 sometimes it's going to be about drugs and drug discovery that are of you know 742 00:42:47,440 --> 00:42:51,837 for for mass market but sometimes it'll be even for the kind of NF1 things like 743 00:42:51,920 --> 00:42:54,317 the the disease um diagnoses that I 744 00:42:54,400 --> 00:42:56,317 mentioned earlier today. Sometimes it 745 00:42:56,400 --> 00:42:59,277 will be for just like trying to 746 00:42:59,360 --> 00:43:03,677 understand conditions and trying to come up with with new potential therapeutics. 747 00:43:03,760 --> 00:43:07,517 All of that we're seeing it happening right now in front of our eyes. It's not 748 00:43:07,600 --> 00:43:11,517 theoretical. It's really happening. And so one of the most I think it's like one 749 00:43:11,600 --> 00:43:15,517 of the most astounding possibilities of AI is how much it can improve our 750 00:43:15,600 --> 00:43:17,837 health. And you think about the ripple 751 00:43:17,920 --> 00:43:19,677 effects of the system, right? Where so 752 00:43:19,760 --> 00:43:23,277 much spending on the health care system 753 00:43:23,360 --> 00:43:25,197 happens right now. that's a massive part 754 00:43:25,280 --> 00:43:27,357 of the economy and that if you're 755 00:43:27,440 --> 00:43:32,637 actually able to help people prevent issues, right, to to get ahead of 756 00:43:32,720 --> 00:43:36,237 potential, you know, health health problems, that's 757 00:43:36,320 --> 00:43:39,597 something that actually then alleviates a lot of burden, a lot of strain. And 758 00:43:39,680 --> 00:43:42,557 we're in a world where doctors are burned out, nurses are burned out, like 759 00:43:42,640 --> 00:43:46,237 there's like a real crisis that's happening in front of us. And I think AI 760 00:43:46,320 --> 00:43:48,717 will be able to help with all of that. 761 00:43:48,800 --> 00:43:50,717 Like we have that potential if we deploy 762 00:43:50,800 --> 00:43:54,157 it and use it wisely and well. And so I 763 00:43:54,240 --> 00:43:58,957 think that applying AI to medicine, like that's something that I is really a 764 00:43:59,040 --> 00:44:02,477 personal motivation for [music] me in in thinking about this whole journey of of 765 00:44:02,560 --> 00:44:05,357 what we're building, what we're trying to do with Open AI. And I'm hopeful that 766 00:44:05,440 --> 00:44:08,557 we as a as a world and a community can make the most of that. 767 00:44:08,640 --> 00:44:12,237 >> Let's hope. I think we will. I'm very very confident. 768 00:44:12,320 --> 00:44:13,757 >> Greg, thank you so much. 769 00:44:13,840 --> 00:44:16,317 >> Thank you. [applause] 770 00:44:16,400 --> 00:44:19,247 Great. Thank you. Thank you so much. 771 00:44:19,330 --> 00:44:21,350 [applause] 772 00:44:25,200 --> 00:44:29,077 Oh my god. Thank you everyone. You have a good time today. 773 00:44:29,160 --> 00:44:31,180 [applause] 774 00:44:33,440 --> 00:44:36,927 >> Thank you. Should we do it again next year? 775 00:44:37,010 --> 00:44:39,597 [applause] >> You going to come? Yeah. 776 00:44:39,680 --> 00:44:41,760 >> All