WEBVTT

00:00:00.160 --> 00:00:02.797
We are here with Greg Brockman. He is

00:00:02.880 --> 00:00:05.117
the president and one of the co-founders

00:00:05.200 --> 00:00:08.317
of Open AI and we get the opportunity to

00:00:08.400 --> 00:00:12.477
chat with him, ask him some follow-up
questions following Sam Alman's keynote,

00:00:12.560 --> 00:00:17.437
kicking off the day. So, Greg, thank you
so much for being here and for helping

00:00:17.520 --> 00:00:20.237
millions of people figure out
what the heck is coming next.

00:00:20.320 --> 00:00:22.637
>> I'm excited to be here.
>> Yeah, >> happy to chats.

00:00:22.720 --> 00:00:24.877
>> So, you were the founding engineer and

00:00:24.960 --> 00:00:29.677
first CTO at Stripe. you are on a
mission to become the best engineer in

00:00:29.760 --> 00:00:32.317
the world. if you were meeting with
the best engineer in the world and maybe

00:00:32.400 --> 00:00:35.757
it's you. What is the best engineer
in the world doing right now

00:00:35.840 --> 00:00:37.517
>> and then building
with codeex but I mean

00:00:37.600 --> 00:00:42.157
I think I think that being an AI is just
such an incredible uh opportunity right

00:00:42.240 --> 00:00:46.317
now and I think that we've very much
been seeing that uh people who are

00:00:46.400 --> 00:00:48.957
engineers in startups or in other

00:00:49.040 --> 00:00:53.677
domains HFT anything like that coming
into AI can have this massive impact and

00:00:53.760 --> 00:00:57.837
so I I think that the best engineering
is possible and is really rewarding

00:00:57.920 --> 00:01:02.157
within AI. Do you have like specific
features that you're thinking about when

00:01:02.240 --> 00:01:06.557
you think of the best engineer that is
super ahead of the curve like day-to-day

00:01:06.640 --> 00:01:09.997
what are they doing? >> Well,
I guess the way that I think about

00:01:10.080 --> 00:01:12.797
it, I mean, I can't I can't comment
necessarily on every engineer. I could

00:01:12.880 --> 00:01:17.437
talk, right? So, the way that I work
is that um I'm constantly context

00:01:17.520 --> 00:01:19.997
switching, >> right? So,
I'll like be sort of, you

00:01:20.080 --> 00:01:22.957
know, writing some function or something
and I'll hit a little mental block. I'm

00:01:23.040 --> 00:01:25.837
like, oh, exactly how should this be
structured? What should what should this

00:01:25.920 --> 00:01:30.317
abstraction be? Something like that.
and I'll flip to Slack for like 10 seconds

00:01:30.400 --> 00:01:33.677
and then I'll flip back or I'll go
to email and it's just like I find this

00:01:33.760 --> 00:01:37.357
constant reset is something that means
that I can background process a problem

00:01:37.440 --> 00:01:41.117
um and then by the time I get back to it
I know the answer and so I think it's

00:01:41.200 --> 00:01:45.277
just about like really about iteration
speed about problem flux and just trying

00:01:45.360 --> 00:01:48.317
to get as much like context
as much speed as possible.

00:01:48.400 --> 00:01:49.997
>> God, this reminds
me of like when people

00:01:50.080 --> 00:01:53.357
would go to sleep with a math problem in
their head in high school and they would

00:01:53.440 --> 00:01:56.157
wake up and it wouldn't solve
but you're doing that while awake.

00:01:56.240 --> 00:01:59.197
>> Yes. And I I actually literally did the
fall asleep. I had this research problem

00:01:59.280 --> 00:02:01.757
I been working on in high school uh in

00:02:01.840 --> 00:02:06.957
math for months and I remember I was >>
for math's last theorem or something

00:02:07.040 --> 00:02:11.277
quite wasn't wasn't quite that that
advanced. It was a problem in uh in uh

00:02:11.360 --> 00:02:15.597
in duch sequences uh which is pretty
cool stuff. U but I remember falling

00:02:15.680 --> 00:02:19.517
asleep and I was half
asleep and I saw the answer.

00:02:19.600 --> 00:02:23.437
It was like it was this phenomenal
experience. I remember feeling like I

00:02:23.520 --> 00:02:25.837
need to figure out how to replicate this.
There is something about like

00:02:25.920 --> 00:02:28.717
their subconscious just like
just chewing on a problem.

00:02:28.800 --> 00:02:33.677
>> So almost like early exposure is one
of the most helpful things. So getting in

00:02:33.760 --> 00:02:36.637
practice cycles is definitely
one of those starting

00:02:36.720 --> 00:02:40.557
>> I So right and I think it's also just
like it's just doing like I think the

00:02:40.640 --> 00:02:45.117
most important thing if you're starting
out is not to be afraid right like

00:02:45.200 --> 00:02:48.477
there's you write some
code it doesn't work fine

00:02:48.560 --> 00:02:51.997
>> there's no consequence to it right
I think it's really about trying to find

00:02:52.080 --> 00:02:55.357
projects where you can learn where
you're a little bit uncomfortable right

00:02:55.440 --> 00:02:59.117
and where you can just like go and try
and especially with tools like codeex

00:02:59.200 --> 00:03:02.797
like it is just so much more
accessible to get into programming

00:03:02.880 --> 00:03:06.877
>> for people when I was showing them
chatbt and they had never seen it for

00:03:06.960 --> 00:03:11.437
more often than not I would ask it
to like write a poem about their hobby and

00:03:11.520 --> 00:03:13.197
that was just a really fast way of

00:03:13.280 --> 00:03:18.877
seeing the magic. Is there like a do x
for y that is a great starter spot for

00:03:18.960 --> 00:03:22.557
codeex users? >> Well, codeex is
incredible at front-end engineering. Yeah.

00:03:22.640 --> 00:03:26.077
>> And so, okay, I have a I have
a particular hobby project that I started

00:03:26.160 --> 00:03:27.597
working on like the first time I built

00:03:27.680 --> 00:03:29.677
it and I actually built it myself was

00:03:29.760 --> 00:03:33.357
back in 2008 or something which is a uh

00:03:33.440 --> 00:03:35.917
competitive chatbot game. I call it a

00:03:36.000 --> 00:03:37.677
reverse touring test, right? Traditional

00:03:37.760 --> 00:03:41.357
touring test, you have um you have two

00:03:41.440 --> 00:03:45.917
human participants um and that one of

00:03:46.000 --> 00:03:49.357
them is a judge, one of them is a is a
player. Um and the goal is for the judge

00:03:49.440 --> 00:03:51.037
to be able to tell are they talking to a

00:03:51.120 --> 00:03:54.637
human or an AI. Now, in my competitive

00:03:54.720 --> 00:03:56.557
uh reverse turning test, the way that it

00:03:56.640 --> 00:03:59.837
works is that you are hooked up to two

00:03:59.920 --> 00:04:05.037
terminals, each player. Um, one is your
opponent, a human, one is an AI. Your

00:04:05.120 --> 00:04:08.477
goal is to tell which one's which
before your opponent does the same.

00:04:08.560 --> 00:04:10.237
>> Okay. >> Now, the thing
that's interesting about

00:04:10.320 --> 00:04:15.357
this is that the optimal strategy for
the bot is to act uh kind of botlike uh

00:04:15.440 --> 00:04:18.637
or so for the human is to act kind
of botlike, right? Because you don't want

00:04:18.720 --> 00:04:23.277
to sort of distinguish whether or not
you are the human. Um, but also to ask

00:04:23.360 --> 00:04:26.957
somewhat discerning questions to figure
out is your opponent that or not. And so

00:04:27.040 --> 00:04:31.837
back in 2008, I remember I wrote like
this huristic based bot that would like

00:04:31.920 --> 00:04:34.957
replay conversations from from its
database that were similar to the ones

00:04:35.040 --> 00:04:36.557
that was having currently. And I got

00:04:36.640 --> 00:04:40.237
like a 30 or 40% win rate. Um and so it

00:04:40.320 --> 00:04:42.397
shows you that that it was just a cool

00:04:42.480 --> 00:04:47.597
problem. It was a cool setup. Um and uh
the thing the reason I bring this up is

00:04:47.680 --> 00:04:51.197
that I've actually started using this
as just a test of vibe coding. I just be

00:04:51.280 --> 00:04:54.557
like, describe that to an AI. And I've

00:04:54.640 --> 00:04:59.437
made versions of that with zero lines of
code written by myself that are like 10x

00:04:59.520 --> 00:05:01.597
better than the thing
that I wrote back in the day.

00:05:01.680 --> 00:05:05.517
>> Really? And when you see that happen,
you're excited because you're adding on

00:05:05.600 --> 00:05:09.117
new things. You're bummed because it
took your best idea and just did it.

00:05:09.200 --> 00:05:13.117
>> I mean, for me, it feels so empowering
because I remember the struggle of

00:05:13.200 --> 00:05:16.397
implementing this back in the day and
there were a bunch of CS fundamentals I

00:05:16.480 --> 00:05:19.997
did not know. Like for example,
um serialization was something I'd not

00:05:20.080 --> 00:05:22.957
heard of like JSON, you know,
JSON formatting of something or you know,

00:05:23.040 --> 00:05:26.477
whatever your favorite serialization
library is. And I remember I just

00:05:26.560 --> 00:05:30.317
invented my own like, okay, I'll like
have this like secret delimiter that was

00:05:30.400 --> 00:05:33.197
this like, you know, sequence of of
characters that would be unlikely to

00:05:33.280 --> 00:05:36.397
not. Exactly. Exactly. I can't tell you
what it is because then you'd be able to

00:05:36.480 --> 00:05:40.397
break my relation scheme and I would
just use that to to separate records.

00:05:40.480 --> 00:05:44.477
And I, you know, it's just like
a concept where when you're starting out,

00:05:44.560 --> 00:05:48.797
you just wouldn't like it's not
it's like a fairly deep thing. Um,

00:05:48.880 --> 00:05:53.117
>> and to even realize what the question
is you should ask, whereas today an AI

00:05:53.200 --> 00:05:57.437
looking at that code would point out to
you, hey, this is like a much better way

00:05:57.520 --> 00:06:00.717
of doing what you're trying to accomplish.
Um, or you can even just

00:06:00.800 --> 00:06:02.877
like do so much more than is possible

00:06:02.960 --> 00:06:07.757
otherwise. Like for myself, I've always
been like a infrastructure backend kind

00:06:07.840 --> 00:06:11.277
of person. my front-end skills are
a little bit atrocious. Um, [laughter] I

00:06:11.360 --> 00:06:14.637
used to, the way that I used to work
is that I would basically just do like the

00:06:14.720 --> 00:06:17.277
hackiest possible HTML, JavaScript, CSS,

00:06:17.360 --> 00:06:21.837
and then have a designer come and like,
you know, clean up my mess. Um, and now

00:06:21.920 --> 00:06:26.077
I don't need to. And so for me, I just
feel like it just supplements me in a

00:06:26.160 --> 00:06:29.517
way that I've always wanted.
>> I love that. And as someone who's only

00:06:29.600 --> 00:06:32.237
capable of making horrific HTML, I'm

00:06:32.320 --> 00:06:37.117
with you. Have you interacted with
an engineer recently where they said

00:06:37.200 --> 00:06:40.797
something about the way they're working
and you said, "Wow." Like what was that

00:06:40.880 --> 00:06:42.637
person doing that might have surprised

00:06:42.720 --> 00:06:46.317
you? Well, I actually remember around

00:06:46.400 --> 00:06:50.077
December or so, one of the engineers on

00:06:50.160 --> 00:06:53.997
my team who's absolutely phenomenal, one
one of the best engineers I would say in

00:06:54.080 --> 00:06:56.557
the world, um telling me that he was

00:06:56.640 --> 00:06:59.917
getting a lot of value out of the chatbt

00:07:00.000 --> 00:07:03.997
desktop integration with his terminal
and that it was a very simple thing. He

00:07:04.080 --> 00:07:06.317
was just like, >> if there's
an error in my terminal, I

00:07:06.400 --> 00:07:09.037
just ask chatbt like what went wrong here?

00:07:09.120 --> 00:07:11.517
>> Yeah. >> And it debugs
it for me. And it's just

00:07:11.600 --> 00:07:16.477
like you just realize the like little
pieces of friction, right? It's like

00:07:16.560 --> 00:07:19.357
obviously this engineer is capable
of reading the error message,

00:07:19.440 --> 00:07:21.677
>> but it's annoying. It's like overhead.

00:07:21.760 --> 00:07:25.117
It's slowness. >> And I think
that what I see is people

00:07:25.200 --> 00:07:28.877
trying to find where does AI map
to their workflow? Where can it remove

00:07:28.960 --> 00:07:32.717
little bits of friction? And sometimes
you're just like that was it. That was

00:07:32.800 --> 00:07:35.757
the thing that that that made
the difference. But the thing is

00:07:35.840 --> 00:07:39.037
>> all these things compound. And I think
that people who are very discerning and

00:07:39.120 --> 00:07:42.157
people who are just really focused on
their own productivity, their own tools,

00:07:42.240 --> 00:07:45.357
and it propagates into how people
structure code bases, right? Because you

00:07:45.440 --> 00:07:46.957
think about, well, if I want to be using

00:07:47.040 --> 00:07:49.357
AI tools as much as possible, then how

00:07:49.440 --> 00:07:52.397
do I have code set up so that it's most

00:07:52.480 --> 00:07:57.197
amendable to AI? So fast tests, modular
code, good readies for um for you know

00:07:57.280 --> 00:08:00.237
the API definitions between
things, all of that stuff.

00:08:00.320 --> 00:08:04.557
>> It's almost like I know everyone talks
about AI as an operating system. I lived

00:08:04.640 --> 00:08:07.757
in a 20 person AI house in San Francisco
and I think that that term was like

00:08:07.840 --> 00:08:11.277
thrown around all the time. And what I'm
seeing now more and more with like these

00:08:11.360 --> 00:08:14.557
real time overlays is that it's not just

00:08:14.640 --> 00:08:16.557
the foundation of how things are built.

00:08:16.640 --> 00:08:20.717
It's also watching over I mean for lack
of a better word and co-working or

00:08:20.800 --> 00:08:24.077
pairing with everyone in every
single line of business. It's amazing.

00:08:24.160 --> 00:08:27.917
>> Yeah, it it is really phenomenal. And I
think we're still at the beginning to

00:08:28.000 --> 00:08:30.557
>> Okay, even at the beginning
that's this slope.

00:08:30.640 --> 00:08:33.837
>> Oh, that that was that was an
exponential. That was an exponential. I

00:08:33.920 --> 00:08:35.757
think I think we're we're
still just like at the point

00:08:35.840 --> 00:08:38.797
>> my acid reflux feels
like it's a little bit super

00:08:38.880 --> 00:08:42.397
>> for for early
founders right now. Um and

00:08:42.480 --> 00:08:44.477
that could be Silicon Valley early

00:08:44.560 --> 00:08:47.437
founders or it could be you know an SMB

00:08:47.520 --> 00:08:52.397
solarreneur coach in the middle
of you know their career. What are they

00:08:52.480 --> 00:08:56.957
building? What should they be building
today? Well, I think two things. Like

00:08:57.040 --> 00:09:00.317
one is honestly there
is so much to explore

00:09:00.400 --> 00:09:03.917
>> that I think people should just like
if you have an idea, build it, right? It's

00:09:04.000 --> 00:09:06.877
like as simple as that. Like there
is the barriers to entry are the lowest

00:09:06.960 --> 00:09:10.397
they've ever been. They're going to get
lower. Don't worry. But we're at a point

00:09:10.480 --> 00:09:13.677
now where it's just like if you have
a creative idea, you should see it,

00:09:13.760 --> 00:09:18.077
implement it, try it out. And >> I
think that is maybe the highest order

00:09:18.160 --> 00:09:21.117
bit. um if you're looking to build
like a durable business I think thinking

00:09:21.200 --> 00:09:23.277
about what are things that will benefit

00:09:23.360 --> 00:09:28.237
as the models get better not be obsolete
and I that I think that things that

00:09:28.320 --> 00:09:31.597
require going into specific domains
building a lot of expertise there that's

00:09:31.680 --> 00:09:35.197
the kind of thing that I think can be can
be really uh durable in this in this time

00:09:35.280 --> 00:09:39.757
>> okay so vertical AI applying your
expertise using AI to support that mission

00:09:39.840 --> 00:09:42.317
>> I think so yes >> yeah
you know there's a lot of talk

00:09:42.400 --> 00:09:45.837
around like lean teams and I work with

00:09:45.920 --> 00:09:50.237
Fortune 500 enterprises that might have
tens thousands of employees, but we're

00:09:50.320 --> 00:09:54.797
seeing that there's a lot of traction
that comes from having a lean team. What

00:09:54.880 --> 00:09:57.197
does a lean team look like today? Like

00:09:57.280 --> 00:10:00.717
are you envisioning new roles, new uh

00:10:00.800 --> 00:10:04.637
processes, you know, do they still have
daily standups? Like do you have a sense

00:10:04.720 --> 00:10:07.437
of how a team right now, a lean
team is functioning in the world?

00:10:07.520 --> 00:10:10.477
>> You know, it's a great question. Uh
and I think that this is the kind of thing

00:10:10.560 --> 00:10:15.357
where I think that almost the way that
people operate has not yet caught up to

00:10:15.440 --> 00:10:16.957
the capability of the tools. Totally.

00:10:17.040 --> 00:10:21.357
And so, you know, even simple things
like AI note-taking tools really just

00:10:21.440 --> 00:10:25.037
open up a new possibility of like,
do you really need everyone to sit in the

00:10:25.120 --> 00:10:28.237
meeting, right? It's like, do you
actually have really good tools that can

00:10:28.320 --> 00:10:31.357
summarize and surface to you the right
information? And some of these require

00:10:31.440 --> 00:10:34.317
integrations that I just haven't seen be
good yet, right? What you really want is

00:10:34.400 --> 00:10:37.437
you want something like chat pulse
that just tells you here's everything

00:10:37.520 --> 00:10:40.397
happening across your team. Here's
what you need to know. This person was

00:10:40.480 --> 00:10:42.877
missing this thing. You want that to
happen in real time, right? That the AI

00:10:42.960 --> 00:10:47.117
is really being this mediator. Um,
but maybe you can build that, right? Maybe

00:10:47.200 --> 00:10:51.117
you actually can do that. And maybe it
requires changing how your team operates

00:10:51.200 --> 00:10:54.237
to make sure that the AI has that
context and that if it's something

00:10:54.320 --> 00:10:57.597
that's like very sophisticated, requires
a lot of the back context, you have all

00:10:57.680 --> 00:11:01.197
all that there. Um, this is probably
true for development environments too of

00:11:01.280 --> 00:11:05.757
like if you want AI to not just sort
of spit out code at you, which is only a

00:11:05.840 --> 00:11:08.317
small fraction of what software
engineering is, right? Be able to

00:11:08.400 --> 00:11:11.197
operate your infrastructure, things like
that. It would be very nice to be in a

00:11:11.280 --> 00:11:15.437
world where the AI has exactly the same
environment and affordances that human

00:11:15.520 --> 00:11:19.437
developers do. And so I think that maybe
it starts for me from really thinking

00:11:19.520 --> 00:11:24.157
about how to give AI all of that
accessibility and all those affordances

00:11:24.240 --> 00:11:26.397
and then you build
your processes around it.

00:11:26.480 --> 00:11:30.957
>> I'm obsessed with this because
enterprise change is taking two 3 years.

00:11:31.040 --> 00:11:34.797
So having that builder mindset even if
you're in finance is brilliant. Can we

00:11:34.880 --> 00:11:37.517
talk a little bit about chach pulse?
We didn't get to hear it a little bit in

00:11:37.600 --> 00:11:40.877
the in the keynote, but I would love
to hear your perspective. You're always

00:11:40.960 --> 00:11:42.957
thinking about AI in the future. Chachet

00:11:43.040 --> 00:11:47.917
Falls is one of the best proactive AI
systems that I've seen and it is putting

00:11:48.000 --> 00:11:52.477
proactive AI into the hands of really
any user of Chhatab. Can you talk a

00:11:52.560 --> 00:11:55.277
little bit about where you
see proactive AI heading?

00:11:55.360 --> 00:11:58.957
>> Yes. So, you know, I think that at the

00:11:59.040 --> 00:12:02.637
beginning of the year, I tweeted
something like, you know, 2025 is the

00:12:02.720 --> 00:12:05.837
year of agents. Um, I think that's
a a theme that a lot of people have have

00:12:05.920 --> 00:12:09.197
said. Um, and I think that pulse is an

00:12:09.280 --> 00:12:14.077
example of an agent which is very
different from how people have started

00:12:14.160 --> 00:12:18.077
to think about agents, right? I think
that this proactive agent category is

00:12:18.160 --> 00:12:21.117
something that just doesn't really exist
yet because the models have not been

00:12:21.200 --> 00:12:25.757
good enough. But it's kind of no longer
an excuse because the models are really

00:12:25.840 --> 00:12:27.277
smart. Yeah. And I think that what we

00:12:27.360 --> 00:12:29.917
have found is that I it's just like

00:12:30.000 --> 00:12:33.597
where you want to go is you want this

00:12:33.680 --> 00:12:37.197
amplification of human intent through AI

00:12:37.280 --> 00:12:41.437
and we view chatbt as we want to shift
it to be something that is a goal

00:12:41.520 --> 00:12:45.197
amplifier that helping you achieve your
goals and that part of that is knowing

00:12:45.280 --> 00:12:47.997
what your goals are and part of that is
actually then being able to do something

00:12:48.080 --> 00:12:50.557
intelligent be able to add value to them

00:12:50.640 --> 00:12:55.357
and I think that it just makes sense
that there's only so much that you

00:12:55.440 --> 00:12:58.957
either have attention for or really want
to do in terms of managing a fleet of

00:12:59.040 --> 00:13:01.917
agents. Like, you know, you want to do
some of that, right? It's just like if

00:13:02.000 --> 00:13:04.877
you have a whole team of people working
for you, you want to do a little bit of

00:13:04.960 --> 00:13:08.317
of micromanaging, but >> yeah,
>> not that much. You really want people

00:13:08.400 --> 00:13:11.757
who are going to go and be trustworthy
and dependable and going to go add value

00:13:11.840 --> 00:13:15.597
and do good things and come back with
great reports or come back with like a

00:13:15.680 --> 00:13:18.557
suggestion of like, oh, you probably
should, you know, not be eating as much

00:13:18.640 --> 00:13:23.517
red meat or, you know, whatever whatever
the thing is. Um, and so, you know, and

00:13:23.600 --> 00:13:27.117
something like the the example I just
said really requires the AI to have the

00:13:27.200 --> 00:13:30.557
relevant context, right? It needs
information to be able to give you good

00:13:30.640 --> 00:13:33.357
actionable suggestions. And so,
I think that that's going to be a really

00:13:33.440 --> 00:13:35.997
interesting part of how these
kinds of platforms get built.

00:13:36.080 --> 00:13:39.597
>> And what I'm also telling audience and

00:13:39.680 --> 00:13:44.157
community and clients is that they also
need like an SOP for goals. That a lot

00:13:44.240 --> 00:13:45.997
of these documents are knowledge

00:13:46.080 --> 00:13:48.397
gathering or uh relationship gathering.

00:13:48.480 --> 00:13:51.437
who's working on what uh time gathering,

00:13:51.520 --> 00:13:55.917
when did we announce this versus this,
but there aren't a lot of documents that

00:13:56.000 --> 00:13:57.917
completely declare what the goal of

00:13:58.000 --> 00:14:03.357
something is. And so, do you think it's
worth it making like a goals doc, like a

00:14:03.440 --> 00:14:06.157
goal engineering?
>> I I think it's really interesting,

00:14:06.240 --> 00:14:10.317
right? It's like we we're almost moving
from like these like self-contained

00:14:10.400 --> 00:14:14.157
prompts. Yeah. couple years ago to now
this like context engineering to you

00:14:14.240 --> 00:14:17.197
think about like the gold and the high
level you know really almost containing

00:14:17.280 --> 00:14:19.037
all of the knowledge work in one place

00:14:19.120 --> 00:14:21.517
and I think once we have AI that are as

00:14:21.600 --> 00:14:24.397
capable as a remote human then you

00:14:24.480 --> 00:14:27.677
really need to start thinking about well

00:14:27.760 --> 00:14:32.957
what would a remote human coworker need
in order to be effective and you know

00:14:33.040 --> 00:14:36.877
you see this all the time where it's
like someone wasn't there for the

00:14:36.960 --> 00:14:39.037
>> Davy standup
or whatever and now they're

00:14:39.120 --> 00:14:42.317
confused they don't know what's going
on they're on the wrong thing, right? Like

00:14:42.400 --> 00:14:46.157
all that happens for humans who have
a lot of context. They're, you know, very

00:14:46.240 --> 00:14:51.037
plugged in. And so I think that we need
as much sort of thought and attention to

00:14:51.120 --> 00:14:54.237
what does the AI need as we
do for for human co-workers.

00:14:54.320 --> 00:14:59.037
>> W and so just tapping into that if it's
as good as a remote worker, I get a lot

00:14:59.120 --> 00:15:02.397
of questions. Do you imagine that there's
going to be like an AI workforce

00:15:02.480 --> 00:15:05.517
or we're going to name all these AI
agents like Derek, John, whatever. Do

00:15:05.600 --> 00:15:09.277
you think that we're going to have every
single employee have a suite of agents?

00:15:09.360 --> 00:15:12.557
Do you think that there's management
of that or is it just they're working on

00:15:12.640 --> 00:15:16.477
their behalf? Like what is that structure?
What are you imagining the AI

00:15:16.560 --> 00:15:18.637
first enterprise with agents looking like?

00:15:18.720 --> 00:15:20.397
>> Well, to some extent all of the above.

00:15:20.480 --> 00:15:23.917
To some extent we don't know. Like I
think that AI presents a new opportunity

00:15:24.000 --> 00:15:27.197
to just like work in a totally different
way from what we can imagine. We're

00:15:27.280 --> 00:15:30.157
seeing a little bit of it where it's
like okay you take CEX, you ask it to do

00:15:30.240 --> 00:15:32.877
things, you sit there staring
at your terminal. Okay, that starts to get

00:15:32.960 --> 00:15:36.397
boring. So you can run five of your
codeexes, right? But then it's like well

00:15:36.480 --> 00:15:39.117
actually managing five codex's cotices

00:15:39.200 --> 00:15:44.477
uh cotices uh is like it's just not very
scalable. It's not very fun. So you're

00:15:44.560 --> 00:15:48.557
like okay I want really good interfaces
for actually managing the agents. Um so

00:15:48.640 --> 00:15:52.157
that makes sense. And so will they have
persistent personalities and memory?

00:15:52.240 --> 00:15:56.477
Well in some ways it makes sense but
in some ways it's almost a waste of the

00:15:56.560 --> 00:15:58.317
possibilities of an of an AI, right?

00:15:58.400 --> 00:16:04.077
Because for AIs you can control so many
aspects of them that are difficult to

00:16:04.160 --> 00:16:06.637
control for humans, right? You can
control what do they remember, right?

00:16:06.720 --> 00:16:07.837
What information do they still have

00:16:07.920 --> 00:16:10.717
access to? Um you can you can sort of

00:16:10.800 --> 00:16:15.597
fork them. You can um make decisions
about how much computational power they

00:16:15.680 --> 00:16:18.797
get. And so that there is some amount
of threaded identity that I think makes

00:16:18.880 --> 00:16:21.837
sense, but it also feels like it's kind
of limiting to just think of it as okay,

00:16:21.920 --> 00:16:24.717
this is like one agent who, you know,
did all this stuff yesterday and is

00:16:24.800 --> 00:16:29.597
doing more stuff now. Uh so I think
that I'll be very interested to see how we

00:16:29.680 --> 00:16:33.277
build around all these new possibilities.
Um, but I can tell you

00:16:33.360 --> 00:16:36.957
just my main observation from 10 years
of working in this field is that the

00:16:37.040 --> 00:16:40.237
answer will be more surprising
than any of us can anticipate.

00:16:40.320 --> 00:16:42.157
>> And how do you think
we should deal with

00:16:42.240 --> 00:16:47.277
mass surprise? Like how are we ought
to maintain sanity through this?

00:16:47.360 --> 00:16:50.237
>> Well, honestly, I think that a lot
of this starts with small scale

00:16:50.320 --> 00:16:53.837
experimentation. Like you really got
to try it out because hearing the words,

00:16:53.920 --> 00:16:57.037
describing it, thinking about it is so
different from just seeing it. And

00:16:57.120 --> 00:17:00.557
chatting was a very surprising thing but

00:17:00.640 --> 00:17:02.717
now feels very natural right and and I

00:17:02.800 --> 00:17:05.037
think that the fact that we got to try

00:17:05.120 --> 00:17:08.477
out chatting when it was still 3.5 right

00:17:08.560 --> 00:17:12.157
it's like that model compared to

00:17:12.240 --> 00:17:16.717
>> you know like the 20 billion parameter
model today right it's like night and

00:17:16.800 --> 00:17:20.397
day how much better today's stuff
is versus 3.5 but that was a moment where

00:17:20.480 --> 00:17:24.237
the world got to learn and got to adapt
and so for me I I think we just need to

00:17:24.320 --> 00:17:26.157
keep doing that >> it's so
interesting just how you were

00:17:26.240 --> 00:17:29.517
talking about forking because when
people are looking at the new Sora app

00:17:29.600 --> 00:17:31.837
and then they think about it as just
entertainment, I'm like, you're looking

00:17:31.920 --> 00:17:33.597
at it completely wrong. Like the fact

00:17:33.680 --> 00:17:36.957
that one person can create X and that 50

00:17:37.040 --> 00:17:39.917
or 50,000 people can fork off of that

00:17:40.000 --> 00:17:42.077
and continue to create like why is that

00:17:42.160 --> 00:17:45.997
limited to art or to video like you

00:17:46.080 --> 00:17:48.077
should be thinking about code like that,

00:17:48.160 --> 00:17:50.797
images like that, processes like that?

00:17:50.880 --> 00:17:53.197
Like I really do think that there is a

00:17:53.280 --> 00:17:57.837
infinite amount of forking. Yes.
and that kind of finding those seeds are

00:17:57.920 --> 00:18:00.077
going to be even more
important inside the enterprise.

00:18:00.160 --> 00:18:03.837
>> I think it's a I think it's a great
perspective. I remember when we first

00:18:03.920 --> 00:18:07.357
were looking at GPD3 thinking about well
how will this change things? What can

00:18:07.440 --> 00:18:11.037
people use it for? um one of my
co-workers named Yakob our chief

00:18:11.120 --> 00:18:15.277
scientist uh and his collaborator
Shimone were saying well I think that

00:18:15.360 --> 00:18:18.877
there's going to be a new way of writing
right new way of collaborating right you

00:18:18.960 --> 00:18:22.557
could imagine that you have like you
know today you kind of have these sort

00:18:22.640 --> 00:18:25.197
of single documents that get produced
like Wikipedia in a collaborative

00:18:25.280 --> 00:18:29.197
fashion but what if you can easily have
this AI collaborator who's like swapping

00:18:29.280 --> 00:18:32.797
different parts of it and we haven't
seen that yet um but I think we're

00:18:32.880 --> 00:18:35.437
starting to see it with
Sora which is super cool

00:18:35.520 --> 00:18:41.037
>> like canvas was almost that a little
bit It was more human with AI and I was

00:18:41.120 --> 00:18:43.037
hoping, you know, for more collaboration,
but then all of a sudden

00:18:43.120 --> 00:18:44.957
we're seeing that in a multimodal way.

00:18:45.040 --> 00:18:47.837
It's really exciting stuff. >> It is.
And I think part of the reason we

00:18:47.920 --> 00:18:50.237
didn't see it as much with text
is honestly because the models just have

00:18:50.320 --> 00:18:52.877
not been smart enough, >> right?
It's like if the models were

00:18:52.960 --> 00:18:56.397
really coming up with really clever
stuff that's like kind of funny or

00:18:56.480 --> 00:18:59.117
that's like really insightful that adds
a lot of value. You could imagine these

00:18:59.200 --> 00:19:01.117
remixes starting to come to life there.

00:19:01.200 --> 00:19:04.477
So, I think we will see it. I just
think the models aren't aren't there yet.

00:19:04.560 --> 00:19:06.877
>> Wow. And when do
you think they will be?

00:19:06.960 --> 00:19:10.477
two years, one year, three years,
something like that. But the the

00:19:10.560 --> 00:19:13.917
predicting the exact timeline is is
the difficult part. Um, but I think it's

00:19:14.000 --> 00:19:15.757
like very clear it's
in the possibility space.

00:19:15.840 --> 00:19:20.157
>> Yeah, there's so much that you've
shared. You've shared proactive AI,

00:19:20.240 --> 00:19:24.157
vertical AI, the importance of just
getting your hands in there and actually

00:19:24.240 --> 00:19:28.237
building, the importance of figuring out
what everyone else in your org is doing,

00:19:28.320 --> 00:19:32.397
right? What processes can we actually
change? Lean teams and how they should

00:19:32.480 --> 00:19:33.517
function, collaborating with patients.

00:19:33.600 --> 00:19:37.117
We've covered so much. Do you have
one piece of advice for every business

00:19:37.200 --> 00:19:39.597
leader watching this? And before you

00:19:39.680 --> 00:19:42.317
answer, some of them are incredibly

00:19:42.400 --> 00:19:44.557
excited about AI. They are using it

00:19:44.640 --> 00:19:49.917
every single day. Their kid is building
bedtime stories every single night,

00:19:50.000 --> 00:19:53.597
right? And then you've got some that are
like, "Yeah, I use it every so often. I

00:19:53.680 --> 00:19:57.037
feel like I'm not even using it to the
full extent." And then you've got some

00:19:57.120 --> 00:20:00.877
that are still hesitant, which I still
can't believe that. But you've got those

00:20:00.960 --> 00:20:05.277
three categories. How are you trying
to inspire and lift up the business space?

00:20:05.360 --> 00:20:08.157
What's one piece of, you know, advice,
one final thought that you would share

00:20:08.240 --> 00:20:12.477
with those folks? >> Well,
I think perspective is the most

00:20:12.560 --> 00:20:15.677
important thing here.
>> And the way I think about AI is it's

00:20:15.760 --> 00:20:18.237
really about making computers useful,

00:20:18.320 --> 00:20:19.757
right? And you think about computers

00:20:19.840 --> 00:20:22.717
back in like I don't know 1950 or 1960,

00:20:22.800 --> 00:20:26.317
right? Compared to 1980s, compared to a

00:20:26.400 --> 00:20:31.357
modern laptop and you just think about
the internet, right? Just so you think

00:20:31.440 --> 00:20:35.037
about how much more useful these
computers are than what we had. Um, and

00:20:35.120 --> 00:20:39.277
how much if your workforce was just
to lose access to all computers right now,

00:20:39.360 --> 00:20:41.757
like there's no way you'd be competitive.
There's no way that your

00:20:41.840 --> 00:20:44.797
business would continue. >> And I
think we're moving to a world very

00:20:44.880 --> 00:20:47.197
rapidly where the same will be true for

00:20:47.280 --> 00:20:52.717
AI adoption, right? Because AI
is about imbuing this this sort of human

00:20:52.800 --> 00:20:55.837
amplifier, this amplifier of intelligence
into the computer, bringing

00:20:55.920 --> 00:21:00.237
it much closer to you to then be able
to carry out your intent and do, you know,

00:21:00.320 --> 00:21:03.117
whatever whatever business function
people want. So, I think that having

00:21:03.200 --> 00:21:07.597
this perspective of both how important
AI will be to your company and that it's

00:21:07.680 --> 00:21:10.877
an exciting thing, right, that there's
an opportunity here that no one else has

00:21:10.960 --> 00:21:14.557
tapped, no one else in your market has
tapped. And so I think that those who

00:21:14.640 --> 00:21:19.757
really lean in and figure out how it can
add value um and you know that we try to

00:21:19.840 --> 00:21:23.197
add a bunch of you you can look
at things like our agent kit and and these

00:21:23.280 --> 00:21:26.397
other tools that we bring uh bring to bear
that let you add in the guard rails

00:21:26.480 --> 00:21:27.837
control what information is in there.

00:21:27.920 --> 00:21:31.837
Think about enterprise controls all
of those things like we're here to help.

00:21:31.920 --> 00:21:36.237
>> Unbelievable. And also I just want
to say the guard rail guard rail release.

00:21:36.320 --> 00:21:38.637
Love that. Like I'm very excited to show

00:21:38.720 --> 00:21:43.357
that off. Let me just ask a followup and
then and then we'll wrap. Um, we talked

00:21:43.440 --> 00:21:47.037
earlier about competitive advantage.
You had mentioned speed of iteration and

00:21:47.120 --> 00:21:50.797
then you're also saying, listen, we're
going to be at a point in time where if

00:21:50.880 --> 00:21:53.917
you haven't adopted this, not only have
you lost your competitive advantage, you

00:21:54.000 --> 00:21:58.637
don't exist anymore. So, what does
competitive advantage look like? And

00:21:58.720 --> 00:22:02.637
maybe it is speed of iteration.
Maybe it's just constantly changing and

00:22:02.720 --> 00:22:07.437
adapting your perspective. Do you have
a perspective on that? Yeah, I do think

00:22:07.520 --> 00:22:12.317
that there is something that is very
hard for AI to magically replace, right,

00:22:12.400 --> 00:22:16.877
of like real domain knowledge and we
see it is that there is a G in the AGI in

00:22:16.960 --> 00:22:20.637
general where the AI just gets better at
all these different domains at once. But

00:22:20.720 --> 00:22:24.477
to really figure out how do you make AI
useful in education, there's all these

00:22:24.560 --> 00:22:27.037
different stakeholders, right? There's
the parent, there's the teacher, there's

00:22:27.120 --> 00:22:30.717
the student, you need to think about if
the student is using one of these tools,

00:22:30.800 --> 00:22:34.077
like what information is shared with
with the teacher and the parent, all

00:22:34.160 --> 00:22:37.517
those things. And that's something where
having these relationships, having a

00:22:37.600 --> 00:22:39.997
track record, really figuring out how to

00:22:40.080 --> 00:22:42.237
get good feedback, those are very

00:22:42.320 --> 00:22:47.197
difficult to have the technology itself
be the main, at least at the current

00:22:47.280 --> 00:22:48.717
moment, the main the main driver for.

00:22:48.800 --> 00:22:50.157
And so I think that there is something

00:22:50.240 --> 00:22:55.917
that's very sticky about these deep
domain uh expertise. And uh that's

00:22:56.000 --> 00:22:57.837
something that I think
you can really lean into.

00:22:57.920 --> 00:23:01.757
>> Sounds like the people who have loved
people watching are going to be at a

00:23:01.840 --> 00:23:05.197
really strong advantage. people that are
just able to witness a market and are

00:23:05.280 --> 00:23:07.837
able to then adapt ability
to grab feedback and jump on.

00:23:07.920 --> 00:23:10.237
>> I think that's a great way to put it
and I think there's something, you know,

00:23:10.320 --> 00:23:12.237
we're building AI but it's
really about the humans.

00:23:12.320 --> 00:23:15.997
>> Yeah, I love that. Thank you so much
for sharing and again if you guys have any

00:23:16.080 --> 00:23:19.917
questions, drop them below and I will
try and jump on them. But Greg, thank

00:23:20.000 --> 00:23:21.597
you so much for the chat.
Really appreciate it.

00:23:21.680 --> 00:23:24.720
>> Thank you. It's been a lot of fun.