1
00:00:00,160 --> 00:00:02,797
We are here with Greg Brockman. He is

2
00:00:02,880 --> 00:00:05,117
the president and one of the co-founders

3
00:00:05,200 --> 00:00:08,317
of Open AI and we get the opportunity to

4
00:00:08,400 --> 00:00:12,477
chat with him, ask him some follow-up
questions following Sam Alman's keynote,

5
00:00:12,560 --> 00:00:17,437
kicking off the day. So, Greg, thank you
so much for being here and for helping

6
00:00:17,520 --> 00:00:20,237
millions of people figure out
what the heck is coming next.

7
00:00:20,320 --> 00:00:22,637
>> I'm excited to be here.
>> Yeah, >> happy to chats.

8
00:00:22,720 --> 00:00:24,877
>> So, you were the founding engineer and

9
00:00:24,960 --> 00:00:29,677
first CTO at Stripe. you are on a
mission to become the best engineer in

10
00:00:29,760 --> 00:00:32,317
the world. if you were meeting with
the best engineer in the world and maybe

11
00:00:32,400 --> 00:00:35,757
it's you. What is the best engineer
in the world doing right now

12
00:00:35,840 --> 00:00:37,517
>> and then building
with codeex but I mean

13
00:00:37,600 --> 00:00:42,157
I think I think that being an AI is just
such an incredible uh opportunity right

14
00:00:42,240 --> 00:00:46,317
now and I think that we've very much
been seeing that uh people who are

15
00:00:46,400 --> 00:00:48,957
engineers in startups or in other

16
00:00:49,040 --> 00:00:53,677
domains HFT anything like that coming
into AI can have this massive impact and

17
00:00:53,760 --> 00:00:57,837
so I I think that the best engineering
is possible and is really rewarding

18
00:00:57,920 --> 00:01:02,157
within AI. Do you have like specific
features that you're thinking about when

19
00:01:02,240 --> 00:01:06,557
you think of the best engineer that is
super ahead of the curve like day-to-day

20
00:01:06,640 --> 00:01:09,997
what are they doing? >> Well,
I guess the way that I think about

21
00:01:10,080 --> 00:01:12,797
it, I mean, I can't I can't comment
necessarily on every engineer. I could

22
00:01:12,880 --> 00:01:17,437
talk, right? So, the way that I work
is that um I'm constantly context

23
00:01:17,520 --> 00:01:19,997
switching, >> right? So,
I'll like be sort of, you

24
00:01:20,080 --> 00:01:22,957
know, writing some function or something
and I'll hit a little mental block. I'm

25
00:01:23,040 --> 00:01:25,837
like, oh, exactly how should this be
structured? What should what should this

26
00:01:25,920 --> 00:01:30,317
abstraction be? Something like that.
and I'll flip to Slack for like 10 seconds

27
00:01:30,400 --> 00:01:33,677
and then I'll flip back or I'll go
to email and it's just like I find this

28
00:01:33,760 --> 00:01:37,357
constant reset is something that means
that I can background process a problem

29
00:01:37,440 --> 00:01:41,117
um and then by the time I get back to it
I know the answer and so I think it's

30
00:01:41,200 --> 00:01:45,277
just about like really about iteration
speed about problem flux and just trying

31
00:01:45,360 --> 00:01:48,317
to get as much like context
as much speed as possible.

32
00:01:48,400 --> 00:01:49,997
>> God, this reminds
me of like when people

33
00:01:50,080 --> 00:01:53,357
would go to sleep with a math problem in
their head in high school and they would

34
00:01:53,440 --> 00:01:56,157
wake up and it wouldn't solve
but you're doing that while awake.

35
00:01:56,240 --> 00:01:59,197
>> Yes. And I I actually literally did the
fall asleep. I had this research problem

36
00:01:59,280 --> 00:02:01,757
I been working on in high school uh in

37
00:02:01,840 --> 00:02:06,957
math for months and I remember I was >>
for math's last theorem or something

38
00:02:07,040 --> 00:02:11,277
quite wasn't wasn't quite that that
advanced. It was a problem in uh in uh

39
00:02:11,360 --> 00:02:15,597
in duch sequences uh which is pretty
cool stuff. U but I remember falling

40
00:02:15,680 --> 00:02:19,517
asleep and I was half
asleep and I saw the answer.

41
00:02:19,600 --> 00:02:23,437
It was like it was this phenomenal
experience. I remember feeling like I

42
00:02:23,520 --> 00:02:25,837
need to figure out how to replicate this.
There is something about like

43
00:02:25,920 --> 00:02:28,717
their subconscious just like
just chewing on a problem.

44
00:02:28,800 --> 00:02:33,677
>> So almost like early exposure is one
of the most helpful things. So getting in

45
00:02:33,760 --> 00:02:36,637
practice cycles is definitely
one of those starting

46
00:02:36,720 --> 00:02:40,557
>> I So right and I think it's also just
like it's just doing like I think the

47
00:02:40,640 --> 00:02:45,117
most important thing if you're starting
out is not to be afraid right like

48
00:02:45,200 --> 00:02:48,477
there's you write some
code it doesn't work fine

49
00:02:48,560 --> 00:02:51,997
>> there's no consequence to it right
I think it's really about trying to find

50
00:02:52,080 --> 00:02:55,357
projects where you can learn where
you're a little bit uncomfortable right

51
00:02:55,440 --> 00:02:59,117
and where you can just like go and try
and especially with tools like codeex

52
00:02:59,200 --> 00:03:02,797
like it is just so much more
accessible to get into programming

53
00:03:02,880 --> 00:03:06,877
>> for people when I was showing them
chatbt and they had never seen it for

54
00:03:06,960 --> 00:03:11,437
more often than not I would ask it
to like write a poem about their hobby and

55
00:03:11,520 --> 00:03:13,197
that was just a really fast way of

56
00:03:13,280 --> 00:03:18,877
seeing the magic. Is there like a do x
for y that is a great starter spot for

57
00:03:18,960 --> 00:03:22,557
codeex users? >> Well, codeex is
incredible at front-end engineering. Yeah.

58
00:03:22,640 --> 00:03:26,077
>> And so, okay, I have a I have
a particular hobby project that I started

59
00:03:26,160 --> 00:03:27,597
working on like the first time I built

60
00:03:27,680 --> 00:03:29,677
it and I actually built it myself was

61
00:03:29,760 --> 00:03:33,357
back in 2008 or something which is a uh

62
00:03:33,440 --> 00:03:35,917
competitive chatbot game. I call it a

63
00:03:36,000 --> 00:03:37,677
reverse touring test, right? Traditional

64
00:03:37,760 --> 00:03:41,357
touring test, you have um you have two

65
00:03:41,440 --> 00:03:45,917
human participants um and that one of

66
00:03:46,000 --> 00:03:49,357
them is a judge, one of them is a is a
player. Um and the goal is for the judge

67
00:03:49,440 --> 00:03:51,037
to be able to tell are they talking to a

68
00:03:51,120 --> 00:03:54,637
human or an AI. Now, in my competitive

69
00:03:54,720 --> 00:03:56,557
uh reverse turning test, the way that it

70
00:03:56,640 --> 00:03:59,837
works is that you are hooked up to two

71
00:03:59,920 --> 00:04:05,037
terminals, each player. Um, one is your
opponent, a human, one is an AI. Your

72
00:04:05,120 --> 00:04:08,477
goal is to tell which one's which
before your opponent does the same.

73
00:04:08,560 --> 00:04:10,237
>> Okay. >> Now, the thing
that's interesting about

74
00:04:10,320 --> 00:04:15,357
this is that the optimal strategy for
the bot is to act uh kind of botlike uh

75
00:04:15,440 --> 00:04:18,637
or so for the human is to act kind
of botlike, right? Because you don't want

76
00:04:18,720 --> 00:04:23,277
to sort of distinguish whether or not
you are the human. Um, but also to ask

77
00:04:23,360 --> 00:04:26,957
somewhat discerning questions to figure
out is your opponent that or not. And so

78
00:04:27,040 --> 00:04:31,837
back in 2008, I remember I wrote like
this huristic based bot that would like

79
00:04:31,920 --> 00:04:34,957
replay conversations from from its
database that were similar to the ones

80
00:04:35,040 --> 00:04:36,557
that was having currently. And I got

81
00:04:36,640 --> 00:04:40,237
like a 30 or 40% win rate. Um and so it

82
00:04:40,320 --> 00:04:42,397
shows you that that it was just a cool

83
00:04:42,480 --> 00:04:47,597
problem. It was a cool setup. Um and uh
the thing the reason I bring this up is

84
00:04:47,680 --> 00:04:51,197
that I've actually started using this
as just a test of vibe coding. I just be

85
00:04:51,280 --> 00:04:54,557
like, describe that to an AI. And I've

86
00:04:54,640 --> 00:04:59,437
made versions of that with zero lines of
code written by myself that are like 10x

87
00:04:59,520 --> 00:05:01,597
better than the thing
that I wrote back in the day.

88
00:05:01,680 --> 00:05:05,517
>> Really? And when you see that happen,
you're excited because you're adding on

89
00:05:05,600 --> 00:05:09,117
new things. You're bummed because it
took your best idea and just did it.

90
00:05:09,200 --> 00:05:13,117
>> I mean, for me, it feels so empowering
because I remember the struggle of

91
00:05:13,200 --> 00:05:16,397
implementing this back in the day and
there were a bunch of CS fundamentals I

92
00:05:16,480 --> 00:05:19,997
did not know. Like for example,
um serialization was something I'd not

93
00:05:20,080 --> 00:05:22,957
heard of like JSON, you know,
JSON formatting of something or you know,

94
00:05:23,040 --> 00:05:26,477
whatever your favorite serialization
library is. And I remember I just

95
00:05:26,560 --> 00:05:30,317
invented my own like, okay, I'll like
have this like secret delimiter that was

96
00:05:30,400 --> 00:05:33,197
this like, you know, sequence of of
characters that would be unlikely to

97
00:05:33,280 --> 00:05:36,397
not. Exactly. Exactly. I can't tell you
what it is because then you'd be able to

98
00:05:36,480 --> 00:05:40,397
break my relation scheme and I would
just use that to to separate records.

99
00:05:40,480 --> 00:05:44,477
And I, you know, it's just like
a concept where when you're starting out,

100
00:05:44,560 --> 00:05:48,797
you just wouldn't like it's not
it's like a fairly deep thing. Um,

101
00:05:48,880 --> 00:05:53,117
>> and to even realize what the question
is you should ask, whereas today an AI

102
00:05:53,200 --> 00:05:57,437
looking at that code would point out to
you, hey, this is like a much better way

103
00:05:57,520 --> 00:06:00,717
of doing what you're trying to accomplish.
Um, or you can even just

104
00:06:00,800 --> 00:06:02,877
like do so much more than is possible

105
00:06:02,960 --> 00:06:07,757
otherwise. Like for myself, I've always
been like a infrastructure backend kind

106
00:06:07,840 --> 00:06:11,277
of person. my front-end skills are
a little bit atrocious. Um, [laughter] I

107
00:06:11,360 --> 00:06:14,637
used to, the way that I used to work
is that I would basically just do like the

108
00:06:14,720 --> 00:06:17,277
hackiest possible HTML, JavaScript, CSS,

109
00:06:17,360 --> 00:06:21,837
and then have a designer come and like,
you know, clean up my mess. Um, and now

110
00:06:21,920 --> 00:06:26,077
I don't need to. And so for me, I just
feel like it just supplements me in a

111
00:06:26,160 --> 00:06:29,517
way that I've always wanted.
>> I love that. And as someone who's only

112
00:06:29,600 --> 00:06:32,237
capable of making horrific HTML, I'm

113
00:06:32,320 --> 00:06:37,117
with you. Have you interacted with
an engineer recently where they said

114
00:06:37,200 --> 00:06:40,797
something about the way they're working
and you said, "Wow." Like what was that

115
00:06:40,880 --> 00:06:42,637
person doing that might have surprised

116
00:06:42,720 --> 00:06:46,317
you? Well, I actually remember around

117
00:06:46,400 --> 00:06:50,077
December or so, one of the engineers on

118
00:06:50,160 --> 00:06:53,997
my team who's absolutely phenomenal, one
one of the best engineers I would say in

119
00:06:54,080 --> 00:06:56,557
the world, um telling me that he was

120
00:06:56,640 --> 00:06:59,917
getting a lot of value out of the chatbt

121
00:07:00,000 --> 00:07:03,997
desktop integration with his terminal
and that it was a very simple thing. He

122
00:07:04,080 --> 00:07:06,317
was just like, >> if there's
an error in my terminal, I

123
00:07:06,400 --> 00:07:09,037
just ask chatbt like what went wrong here?

124
00:07:09,120 --> 00:07:11,517
>> Yeah. >> And it debugs
it for me. And it's just

125
00:07:11,600 --> 00:07:16,477
like you just realize the like little
pieces of friction, right? It's like

126
00:07:16,560 --> 00:07:19,357
obviously this engineer is capable
of reading the error message,

127
00:07:19,440 --> 00:07:21,677
>> but it's annoying. It's like overhead.

128
00:07:21,760 --> 00:07:25,117
It's slowness. >> And I think
that what I see is people

129
00:07:25,200 --> 00:07:28,877
trying to find where does AI map
to their workflow? Where can it remove

130
00:07:28,960 --> 00:07:32,717
little bits of friction? And sometimes
you're just like that was it. That was

131
00:07:32,800 --> 00:07:35,757
the thing that that that made
the difference. But the thing is

132
00:07:35,840 --> 00:07:39,037
>> all these things compound. And I think
that people who are very discerning and

133
00:07:39,120 --> 00:07:42,157
people who are just really focused on
their own productivity, their own tools,

134
00:07:42,240 --> 00:07:45,357
and it propagates into how people
structure code bases, right? Because you

135
00:07:45,440 --> 00:07:46,957
think about, well, if I want to be using

136
00:07:47,040 --> 00:07:49,357
AI tools as much as possible, then how

137
00:07:49,440 --> 00:07:52,397
do I have code set up so that it's most

138
00:07:52,480 --> 00:07:57,197
amendable to AI? So fast tests, modular
code, good readies for um for you know

139
00:07:57,280 --> 00:08:00,237
the API definitions between
things, all of that stuff.

140
00:08:00,320 --> 00:08:04,557
>> It's almost like I know everyone talks
about AI as an operating system. I lived

141
00:08:04,640 --> 00:08:07,757
in a 20 person AI house in San Francisco
and I think that that term was like

142
00:08:07,840 --> 00:08:11,277
thrown around all the time. And what I'm
seeing now more and more with like these

143
00:08:11,360 --> 00:08:14,557
real time overlays is that it's not just

144
00:08:14,640 --> 00:08:16,557
the foundation of how things are built.

145
00:08:16,640 --> 00:08:20,717
It's also watching over I mean for lack
of a better word and co-working or

146
00:08:20,800 --> 00:08:24,077
pairing with everyone in every
single line of business. It's amazing.

147
00:08:24,160 --> 00:08:27,917
>> Yeah, it it is really phenomenal. And I
think we're still at the beginning to

148
00:08:28,000 --> 00:08:30,557
>> Okay, even at the beginning
that's this slope.

149
00:08:30,640 --> 00:08:33,837
>> Oh, that that was that was an
exponential. That was an exponential. I

150
00:08:33,920 --> 00:08:35,757
think I think we're we're
still just like at the point

151
00:08:35,840 --> 00:08:38,797
>> my acid reflux feels
like it's a little bit super

152
00:08:38,880 --> 00:08:42,397
>> for for early
founders right now. Um and

153
00:08:42,480 --> 00:08:44,477
that could be Silicon Valley early

154
00:08:44,560 --> 00:08:47,437
founders or it could be you know an SMB

155
00:08:47,520 --> 00:08:52,397
solarreneur coach in the middle
of you know their career. What are they

156
00:08:52,480 --> 00:08:56,957
building? What should they be building
today? Well, I think two things. Like

157
00:08:57,040 --> 00:09:00,317
one is honestly there
is so much to explore

158
00:09:00,400 --> 00:09:03,917
>> that I think people should just like
if you have an idea, build it, right? It's

159
00:09:04,000 --> 00:09:06,877
like as simple as that. Like there
is the barriers to entry are the lowest

160
00:09:06,960 --> 00:09:10,397
they've ever been. They're going to get
lower. Don't worry. But we're at a point

161
00:09:10,480 --> 00:09:13,677
now where it's just like if you have
a creative idea, you should see it,

162
00:09:13,760 --> 00:09:18,077
implement it, try it out. And >> I
think that is maybe the highest order

163
00:09:18,160 --> 00:09:21,117
bit. um if you're looking to build
like a durable business I think thinking

164
00:09:21,200 --> 00:09:23,277
about what are things that will benefit

165
00:09:23,360 --> 00:09:28,237
as the models get better not be obsolete
and I that I think that things that

166
00:09:28,320 --> 00:09:31,597
require going into specific domains
building a lot of expertise there that's

167
00:09:31,680 --> 00:09:35,197
the kind of thing that I think can be can
be really uh durable in this in this time

168
00:09:35,280 --> 00:09:39,757
>> okay so vertical AI applying your
expertise using AI to support that mission

169
00:09:39,840 --> 00:09:42,317
>> I think so yes >> yeah
you know there's a lot of talk

170
00:09:42,400 --> 00:09:45,837
around like lean teams and I work with

171
00:09:45,920 --> 00:09:50,237
Fortune 500 enterprises that might have
tens thousands of employees, but we're

172
00:09:50,320 --> 00:09:54,797
seeing that there's a lot of traction
that comes from having a lean team. What

173
00:09:54,880 --> 00:09:57,197
does a lean team look like today? Like

174
00:09:57,280 --> 00:10:00,717
are you envisioning new roles, new uh

175
00:10:00,800 --> 00:10:04,637
processes, you know, do they still have
daily standups? Like do you have a sense

176
00:10:04,720 --> 00:10:07,437
of how a team right now, a lean
team is functioning in the world?

177
00:10:07,520 --> 00:10:10,477
>> You know, it's a great question. Uh
and I think that this is the kind of thing

178
00:10:10,560 --> 00:10:15,357
where I think that almost the way that
people operate has not yet caught up to

179
00:10:15,440 --> 00:10:16,957
the capability of the tools. Totally.

180
00:10:17,040 --> 00:10:21,357
And so, you know, even simple things
like AI note-taking tools really just

181
00:10:21,440 --> 00:10:25,037
open up a new possibility of like,
do you really need everyone to sit in the

182
00:10:25,120 --> 00:10:28,237
meeting, right? It's like, do you
actually have really good tools that can

183
00:10:28,320 --> 00:10:31,357
summarize and surface to you the right
information? And some of these require

184
00:10:31,440 --> 00:10:34,317
integrations that I just haven't seen be
good yet, right? What you really want is

185
00:10:34,400 --> 00:10:37,437
you want something like chat pulse
that just tells you here's everything

186
00:10:37,520 --> 00:10:40,397
happening across your team. Here's
what you need to know. This person was

187
00:10:40,480 --> 00:10:42,877
missing this thing. You want that to
happen in real time, right? That the AI

188
00:10:42,960 --> 00:10:47,117
is really being this mediator. Um,
but maybe you can build that, right? Maybe

189
00:10:47,200 --> 00:10:51,117
you actually can do that. And maybe it
requires changing how your team operates

190
00:10:51,200 --> 00:10:54,237
to make sure that the AI has that
context and that if it's something

191
00:10:54,320 --> 00:10:57,597
that's like very sophisticated, requires
a lot of the back context, you have all

192
00:10:57,680 --> 00:11:01,197
all that there. Um, this is probably
true for development environments too of

193
00:11:01,280 --> 00:11:05,757
like if you want AI to not just sort
of spit out code at you, which is only a

194
00:11:05,840 --> 00:11:08,317
small fraction of what software
engineering is, right? Be able to

195
00:11:08,400 --> 00:11:11,197
operate your infrastructure, things like
that. It would be very nice to be in a

196
00:11:11,280 --> 00:11:15,437
world where the AI has exactly the same
environment and affordances that human

197
00:11:15,520 --> 00:11:19,437
developers do. And so I think that maybe
it starts for me from really thinking

198
00:11:19,520 --> 00:11:24,157
about how to give AI all of that
accessibility and all those affordances

199
00:11:24,240 --> 00:11:26,397
and then you build
your processes around it.

200
00:11:26,480 --> 00:11:30,957
>> I'm obsessed with this because
enterprise change is taking two 3 years.

201
00:11:31,040 --> 00:11:34,797
So having that builder mindset even if
you're in finance is brilliant. Can we

202
00:11:34,880 --> 00:11:37,517
talk a little bit about chach pulse?
We didn't get to hear it a little bit in

203
00:11:37,600 --> 00:11:40,877
the in the keynote, but I would love
to hear your perspective. You're always

204
00:11:40,960 --> 00:11:42,957
thinking about AI in the future. Chachet

205
00:11:43,040 --> 00:11:47,917
Falls is one of the best proactive AI
systems that I've seen and it is putting

206
00:11:48,000 --> 00:11:52,477
proactive AI into the hands of really
any user of Chhatab. Can you talk a

207
00:11:52,560 --> 00:11:55,277
little bit about where you
see proactive AI heading?

208
00:11:55,360 --> 00:11:58,957
>> Yes. So, you know, I think that at the

209
00:11:59,040 --> 00:12:02,637
beginning of the year, I tweeted
something like, you know, 2025 is the

210
00:12:02,720 --> 00:12:05,837
year of agents. Um, I think that's
a a theme that a lot of people have have

211
00:12:05,920 --> 00:12:09,197
said. Um, and I think that pulse is an

212
00:12:09,280 --> 00:12:14,077
example of an agent which is very
different from how people have started

213
00:12:14,160 --> 00:12:18,077
to think about agents, right? I think
that this proactive agent category is

214
00:12:18,160 --> 00:12:21,117
something that just doesn't really exist
yet because the models have not been

215
00:12:21,200 --> 00:12:25,757
good enough. But it's kind of no longer
an excuse because the models are really

216
00:12:25,840 --> 00:12:27,277
smart. Yeah. And I think that what we

217
00:12:27,360 --> 00:12:29,917
have found is that I it's just like

218
00:12:30,000 --> 00:12:33,597
where you want to go is you want this

219
00:12:33,680 --> 00:12:37,197
amplification of human intent through AI

220
00:12:37,280 --> 00:12:41,437
and we view chatbt as we want to shift
it to be something that is a goal

221
00:12:41,520 --> 00:12:45,197
amplifier that helping you achieve your
goals and that part of that is knowing

222
00:12:45,280 --> 00:12:47,997
what your goals are and part of that is
actually then being able to do something

223
00:12:48,080 --> 00:12:50,557
intelligent be able to add value to them

224
00:12:50,640 --> 00:12:55,357
and I think that it just makes sense
that there's only so much that you

225
00:12:55,440 --> 00:12:58,957
either have attention for or really want
to do in terms of managing a fleet of

226
00:12:59,040 --> 00:13:01,917
agents. Like, you know, you want to do
some of that, right? It's just like if

227
00:13:02,000 --> 00:13:04,877
you have a whole team of people working
for you, you want to do a little bit of

228
00:13:04,960 --> 00:13:08,317
of micromanaging, but >> yeah,
>> not that much. You really want people

229
00:13:08,400 --> 00:13:11,757
who are going to go and be trustworthy
and dependable and going to go add value

230
00:13:11,840 --> 00:13:15,597
and do good things and come back with
great reports or come back with like a

231
00:13:15,680 --> 00:13:18,557
suggestion of like, oh, you probably
should, you know, not be eating as much

232
00:13:18,640 --> 00:13:23,517
red meat or, you know, whatever whatever
the thing is. Um, and so, you know, and

233
00:13:23,600 --> 00:13:27,117
something like the the example I just
said really requires the AI to have the

234
00:13:27,200 --> 00:13:30,557
relevant context, right? It needs
information to be able to give you good

235
00:13:30,640 --> 00:13:33,357
actionable suggestions. And so,
I think that that's going to be a really

236
00:13:33,440 --> 00:13:35,997
interesting part of how these
kinds of platforms get built.

237
00:13:36,080 --> 00:13:39,597
>> And what I'm also telling audience and

238
00:13:39,680 --> 00:13:44,157
community and clients is that they also
need like an SOP for goals. That a lot

239
00:13:44,240 --> 00:13:45,997
of these documents are knowledge

240
00:13:46,080 --> 00:13:48,397
gathering or uh relationship gathering.

241
00:13:48,480 --> 00:13:51,437
who's working on what uh time gathering,

242
00:13:51,520 --> 00:13:55,917
when did we announce this versus this,
but there aren't a lot of documents that

243
00:13:56,000 --> 00:13:57,917
completely declare what the goal of

244
00:13:58,000 --> 00:14:03,357
something is. And so, do you think it's
worth it making like a goals doc, like a

245
00:14:03,440 --> 00:14:06,157
goal engineering?
>> I I think it's really interesting,

246
00:14:06,240 --> 00:14:10,317
right? It's like we we're almost moving
from like these like self-contained

247
00:14:10,400 --> 00:14:14,157
prompts. Yeah. couple years ago to now
this like context engineering to you

248
00:14:14,240 --> 00:14:17,197
think about like the gold and the high
level you know really almost containing

249
00:14:17,280 --> 00:14:19,037
all of the knowledge work in one place

250
00:14:19,120 --> 00:14:21,517
and I think once we have AI that are as

251
00:14:21,600 --> 00:14:24,397
capable as a remote human then you

252
00:14:24,480 --> 00:14:27,677
really need to start thinking about well

253
00:14:27,760 --> 00:14:32,957
what would a remote human coworker need
in order to be effective and you know

254
00:14:33,040 --> 00:14:36,877
you see this all the time where it's
like someone wasn't there for the

255
00:14:36,960 --> 00:14:39,037
>> Davy standup
or whatever and now they're

256
00:14:39,120 --> 00:14:42,317
confused they don't know what's going
on they're on the wrong thing, right? Like

257
00:14:42,400 --> 00:14:46,157
all that happens for humans who have
a lot of context. They're, you know, very

258
00:14:46,240 --> 00:14:51,037
plugged in. And so I think that we need
as much sort of thought and attention to

259
00:14:51,120 --> 00:14:54,237
what does the AI need as we
do for for human co-workers.

260
00:14:54,320 --> 00:14:59,037
>> W and so just tapping into that if it's
as good as a remote worker, I get a lot

261
00:14:59,120 --> 00:15:02,397
of questions. Do you imagine that there's
going to be like an AI workforce

262
00:15:02,480 --> 00:15:05,517
or we're going to name all these AI
agents like Derek, John, whatever. Do

263
00:15:05,600 --> 00:15:09,277
you think that we're going to have every
single employee have a suite of agents?

264
00:15:09,360 --> 00:15:12,557
Do you think that there's management
of that or is it just they're working on

265
00:15:12,640 --> 00:15:16,477
their behalf? Like what is that structure?
What are you imagining the AI

266
00:15:16,560 --> 00:15:18,637
first enterprise with agents looking like?

267
00:15:18,720 --> 00:15:20,397
>> Well, to some extent all of the above.

268
00:15:20,480 --> 00:15:23,917
To some extent we don't know. Like I
think that AI presents a new opportunity

269
00:15:24,000 --> 00:15:27,197
to just like work in a totally different
way from what we can imagine. We're

270
00:15:27,280 --> 00:15:30,157
seeing a little bit of it where it's
like okay you take CEX, you ask it to do

271
00:15:30,240 --> 00:15:32,877
things, you sit there staring
at your terminal. Okay, that starts to get

272
00:15:32,960 --> 00:15:36,397
boring. So you can run five of your
codeexes, right? But then it's like well

273
00:15:36,480 --> 00:15:39,117
actually managing five codex's cotices

274
00:15:39,200 --> 00:15:44,477
uh cotices uh is like it's just not very
scalable. It's not very fun. So you're

275
00:15:44,560 --> 00:15:48,557
like okay I want really good interfaces
for actually managing the agents. Um so

276
00:15:48,640 --> 00:15:52,157
that makes sense. And so will they have
persistent personalities and memory?

277
00:15:52,240 --> 00:15:56,477
Well in some ways it makes sense but
in some ways it's almost a waste of the

278
00:15:56,560 --> 00:15:58,317
possibilities of an of an AI, right?

279
00:15:58,400 --> 00:16:04,077
Because for AIs you can control so many
aspects of them that are difficult to

280
00:16:04,160 --> 00:16:06,637
control for humans, right? You can
control what do they remember, right?

281
00:16:06,720 --> 00:16:07,837
What information do they still have

282
00:16:07,920 --> 00:16:10,717
access to? Um you can you can sort of

283
00:16:10,800 --> 00:16:15,597
fork them. You can um make decisions
about how much computational power they

284
00:16:15,680 --> 00:16:18,797
get. And so that there is some amount
of threaded identity that I think makes

285
00:16:18,880 --> 00:16:21,837
sense, but it also feels like it's kind
of limiting to just think of it as okay,

286
00:16:21,920 --> 00:16:24,717
this is like one agent who, you know,
did all this stuff yesterday and is

287
00:16:24,800 --> 00:16:29,597
doing more stuff now. Uh so I think
that I'll be very interested to see how we

288
00:16:29,680 --> 00:16:33,277
build around all these new possibilities.
Um, but I can tell you

289
00:16:33,360 --> 00:16:36,957
just my main observation from 10 years
of working in this field is that the

290
00:16:37,040 --> 00:16:40,237
answer will be more surprising
than any of us can anticipate.

291
00:16:40,320 --> 00:16:42,157
>> And how do you think
we should deal with

292
00:16:42,240 --> 00:16:47,277
mass surprise? Like how are we ought
to maintain sanity through this?

293
00:16:47,360 --> 00:16:50,237
>> Well, honestly, I think that a lot
of this starts with small scale

294
00:16:50,320 --> 00:16:53,837
experimentation. Like you really got
to try it out because hearing the words,

295
00:16:53,920 --> 00:16:57,037
describing it, thinking about it is so
different from just seeing it. And

296
00:16:57,120 --> 00:17:00,557
chatting was a very surprising thing but

297
00:17:00,640 --> 00:17:02,717
now feels very natural right and and I

298
00:17:02,800 --> 00:17:05,037
think that the fact that we got to try

299
00:17:05,120 --> 00:17:08,477
out chatting when it was still 3.5 right

300
00:17:08,560 --> 00:17:12,157
it's like that model compared to

301
00:17:12,240 --> 00:17:16,717
>> you know like the 20 billion parameter
model today right it's like night and

302
00:17:16,800 --> 00:17:20,397
day how much better today's stuff
is versus 3.5 but that was a moment where

303
00:17:20,480 --> 00:17:24,237
the world got to learn and got to adapt
and so for me I I think we just need to

304
00:17:24,320 --> 00:17:26,157
keep doing that >> it's so
interesting just how you were

305
00:17:26,240 --> 00:17:29,517
talking about forking because when
people are looking at the new Sora app

306
00:17:29,600 --> 00:17:31,837
and then they think about it as just
entertainment, I'm like, you're looking

307
00:17:31,920 --> 00:17:33,597
at it completely wrong. Like the fact

308
00:17:33,680 --> 00:17:36,957
that one person can create X and that 50

309
00:17:37,040 --> 00:17:39,917
or 50,000 people can fork off of that

310
00:17:40,000 --> 00:17:42,077
and continue to create like why is that

311
00:17:42,160 --> 00:17:45,997
limited to art or to video like you

312
00:17:46,080 --> 00:17:48,077
should be thinking about code like that,

313
00:17:48,160 --> 00:17:50,797
images like that, processes like that?

314
00:17:50,880 --> 00:17:53,197
Like I really do think that there is a

315
00:17:53,280 --> 00:17:57,837
infinite amount of forking. Yes.
and that kind of finding those seeds are

316
00:17:57,920 --> 00:18:00,077
going to be even more
important inside the enterprise.

317
00:18:00,160 --> 00:18:03,837
>> I think it's a I think it's a great
perspective. I remember when we first

318
00:18:03,920 --> 00:18:07,357
were looking at GPD3 thinking about well
how will this change things? What can

319
00:18:07,440 --> 00:18:11,037
people use it for? um one of my
co-workers named Yakob our chief

320
00:18:11,120 --> 00:18:15,277
scientist uh and his collaborator
Shimone were saying well I think that

321
00:18:15,360 --> 00:18:18,877
there's going to be a new way of writing
right new way of collaborating right you

322
00:18:18,960 --> 00:18:22,557
could imagine that you have like you
know today you kind of have these sort

323
00:18:22,640 --> 00:18:25,197
of single documents that get produced
like Wikipedia in a collaborative

324
00:18:25,280 --> 00:18:29,197
fashion but what if you can easily have
this AI collaborator who's like swapping

325
00:18:29,280 --> 00:18:32,797
different parts of it and we haven't
seen that yet um but I think we're

326
00:18:32,880 --> 00:18:35,437
starting to see it with
Sora which is super cool

327
00:18:35,520 --> 00:18:41,037
>> like canvas was almost that a little
bit It was more human with AI and I was

328
00:18:41,120 --> 00:18:43,037
hoping, you know, for more collaboration,
but then all of a sudden

329
00:18:43,120 --> 00:18:44,957
we're seeing that in a multimodal way.

330
00:18:45,040 --> 00:18:47,837
It's really exciting stuff. >> It is.
And I think part of the reason we

331
00:18:47,920 --> 00:18:50,237
didn't see it as much with text
is honestly because the models just have

332
00:18:50,320 --> 00:18:52,877
not been smart enough, >> right?
It's like if the models were

333
00:18:52,960 --> 00:18:56,397
really coming up with really clever
stuff that's like kind of funny or

334
00:18:56,480 --> 00:18:59,117
that's like really insightful that adds
a lot of value. You could imagine these

335
00:18:59,200 --> 00:19:01,117
remixes starting to come to life there.

336
00:19:01,200 --> 00:19:04,477
So, I think we will see it. I just
think the models aren't aren't there yet.

337
00:19:04,560 --> 00:19:06,877
>> Wow. And when do
you think they will be?

338
00:19:06,960 --> 00:19:10,477
two years, one year, three years,
something like that. But the the

339
00:19:10,560 --> 00:19:13,917
predicting the exact timeline is is
the difficult part. Um, but I think it's

340
00:19:14,000 --> 00:19:15,757
like very clear it's
in the possibility space.

341
00:19:15,840 --> 00:19:20,157
>> Yeah, there's so much that you've
shared. You've shared proactive AI,

342
00:19:20,240 --> 00:19:24,157
vertical AI, the importance of just
getting your hands in there and actually

343
00:19:24,240 --> 00:19:28,237
building, the importance of figuring out
what everyone else in your org is doing,

344
00:19:28,320 --> 00:19:32,397
right? What processes can we actually
change? Lean teams and how they should

345
00:19:32,480 --> 00:19:33,517
function, collaborating with patients.

346
00:19:33,600 --> 00:19:37,117
We've covered so much. Do you have
one piece of advice for every business

347
00:19:37,200 --> 00:19:39,597
leader watching this? And before you

348
00:19:39,680 --> 00:19:42,317
answer, some of them are incredibly

349
00:19:42,400 --> 00:19:44,557
excited about AI. They are using it

350
00:19:44,640 --> 00:19:49,917
every single day. Their kid is building
bedtime stories every single night,

351
00:19:50,000 --> 00:19:53,597
right? And then you've got some that are
like, "Yeah, I use it every so often. I

352
00:19:53,680 --> 00:19:57,037
feel like I'm not even using it to the
full extent." And then you've got some

353
00:19:57,120 --> 00:20:00,877
that are still hesitant, which I still
can't believe that. But you've got those

354
00:20:00,960 --> 00:20:05,277
three categories. How are you trying
to inspire and lift up the business space?

355
00:20:05,360 --> 00:20:08,157
What's one piece of, you know, advice,
one final thought that you would share

356
00:20:08,240 --> 00:20:12,477
with those folks? >> Well,
I think perspective is the most

357
00:20:12,560 --> 00:20:15,677
important thing here.
>> And the way I think about AI is it's

358
00:20:15,760 --> 00:20:18,237
really about making computers useful,

359
00:20:18,320 --> 00:20:19,757
right? And you think about computers

360
00:20:19,840 --> 00:20:22,717
back in like I don't know 1950 or 1960,

361
00:20:22,800 --> 00:20:26,317
right? Compared to 1980s, compared to a

362
00:20:26,400 --> 00:20:31,357
modern laptop and you just think about
the internet, right? Just so you think

363
00:20:31,440 --> 00:20:35,037
about how much more useful these
computers are than what we had. Um, and

364
00:20:35,120 --> 00:20:39,277
how much if your workforce was just
to lose access to all computers right now,

365
00:20:39,360 --> 00:20:41,757
like there's no way you'd be competitive.
There's no way that your

366
00:20:41,840 --> 00:20:44,797
business would continue. >> And I
think we're moving to a world very

367
00:20:44,880 --> 00:20:47,197
rapidly where the same will be true for

368
00:20:47,280 --> 00:20:52,717
AI adoption, right? Because AI
is about imbuing this this sort of human

369
00:20:52,800 --> 00:20:55,837
amplifier, this amplifier of intelligence
into the computer, bringing

370
00:20:55,920 --> 00:21:00,237
it much closer to you to then be able
to carry out your intent and do, you know,

371
00:21:00,320 --> 00:21:03,117
whatever whatever business function
people want. So, I think that having

372
00:21:03,200 --> 00:21:07,597
this perspective of both how important
AI will be to your company and that it's

373
00:21:07,680 --> 00:21:10,877
an exciting thing, right, that there's
an opportunity here that no one else has

374
00:21:10,960 --> 00:21:14,557
tapped, no one else in your market has
tapped. And so I think that those who

375
00:21:14,640 --> 00:21:19,757
really lean in and figure out how it can
add value um and you know that we try to

376
00:21:19,840 --> 00:21:23,197
add a bunch of you you can look
at things like our agent kit and and these

377
00:21:23,280 --> 00:21:26,397
other tools that we bring uh bring to bear
that let you add in the guard rails

378
00:21:26,480 --> 00:21:27,837
control what information is in there.

379
00:21:27,920 --> 00:21:31,837
Think about enterprise controls all
of those things like we're here to help.

380
00:21:31,920 --> 00:21:36,237
>> Unbelievable. And also I just want
to say the guard rail guard rail release.

381
00:21:36,320 --> 00:21:38,637
Love that. Like I'm very excited to show

382
00:21:38,720 --> 00:21:43,357
that off. Let me just ask a followup and
then and then we'll wrap. Um, we talked

383
00:21:43,440 --> 00:21:47,037
earlier about competitive advantage.
You had mentioned speed of iteration and

384
00:21:47,120 --> 00:21:50,797
then you're also saying, listen, we're
going to be at a point in time where if

385
00:21:50,880 --> 00:21:53,917
you haven't adopted this, not only have
you lost your competitive advantage, you

386
00:21:54,000 --> 00:21:58,637
don't exist anymore. So, what does
competitive advantage look like? And

387
00:21:58,720 --> 00:22:02,637
maybe it is speed of iteration.
Maybe it's just constantly changing and

388
00:22:02,720 --> 00:22:07,437
adapting your perspective. Do you have
a perspective on that? Yeah, I do think

389
00:22:07,520 --> 00:22:12,317
that there is something that is very
hard for AI to magically replace, right,

390
00:22:12,400 --> 00:22:16,877
of like real domain knowledge and we
see it is that there is a G in the AGI in

391
00:22:16,960 --> 00:22:20,637
general where the AI just gets better at
all these different domains at once. But

392
00:22:20,720 --> 00:22:24,477
to really figure out how do you make AI
useful in education, there's all these

393
00:22:24,560 --> 00:22:27,037
different stakeholders, right? There's
the parent, there's the teacher, there's

394
00:22:27,120 --> 00:22:30,717
the student, you need to think about if
the student is using one of these tools,

395
00:22:30,800 --> 00:22:34,077
like what information is shared with
with the teacher and the parent, all

396
00:22:34,160 --> 00:22:37,517
those things. And that's something where
having these relationships, having a

397
00:22:37,600 --> 00:22:39,997
track record, really figuring out how to

398
00:22:40,080 --> 00:22:42,237
get good feedback, those are very

399
00:22:42,320 --> 00:22:47,197
difficult to have the technology itself
be the main, at least at the current

400
00:22:47,280 --> 00:22:48,717
moment, the main the main driver for.

401
00:22:48,800 --> 00:22:50,157
And so I think that there is something

402
00:22:50,240 --> 00:22:55,917
that's very sticky about these deep
domain uh expertise. And uh that's

403
00:22:56,000 --> 00:22:57,837
something that I think
you can really lean into.

404
00:22:57,920 --> 00:23:01,757
>> Sounds like the people who have loved
people watching are going to be at a

405
00:23:01,840 --> 00:23:05,197
really strong advantage. people that are
just able to witness a market and are

406
00:23:05,280 --> 00:23:07,837
able to then adapt ability
to grab feedback and jump on.

407
00:23:07,920 --> 00:23:10,237
>> I think that's a great way to put it
and I think there's something, you know,

408
00:23:10,320 --> 00:23:12,237
we're building AI but it's
really about the humans.

409
00:23:12,320 --> 00:23:15,997
>> Yeah, I love that. Thank you so much
for sharing and again if you guys have any

410
00:23:16,080 --> 00:23:19,917
questions, drop them below and I will
try and jump on them. But Greg, thank

411
00:23:20,000 --> 00:23:21,597
you so much for the chat.
Really appreciate it.

412
00:23:21,680 --> 00:23:24,720
>> Thank you. It's been a lot of fun.