Do you think you have AGI?
I think about this a lot. I talked to I
talked to different people about where
they believe we're at. And the best
answer I've gotten from our team and I
it really resonates for me is that AGI
is a gradual thing, right? It's not
really like a moment in time, like a you
know
>> So you talked about it like a moment in
time at one point.
>> That's right.
>> definitely did.
>> That's right. That's what we used to
think, right? We we really when we set
out, we almost thought of it as nothing
and then we'll put together the pieces
and that's the day of AGI.
>> Greg Brockman's point is important. AGI
may not arrive as one dramatic
breakthrough. Instead, we may already be
watching it emerge gradually. As AI
moves from answering questions to
actually completing complex work on our
behalf.
>> Well, I think we started late. And
specifically the way to think about the
overall field is that for the past
couple years the chat market was the
main growing one. And the reason for
this was simple. GPT-4 was the first
model where the results were
personalized enough that it made sense
for you to ask a question, read the
answer. It's interesting to you. And so
it very naturally lends itself to a
chat. Right? To a chat bot. And that is
something we're able to scale, deliver
value. Now a billion people every single
week use ChatGPT. Like a billion people.
Like that is scale. That is real scale.
No one else is doing that. But this year
something else happened. Is the agentic
moment. This really was December of last
year is when both we and competition had
a moment where agents were really able
to first start doing software
engineering in a significant way. It
went from saying that, you know, you
could speed up your work with maybe 20%
productive productivity increase to 80%
of the work could now be done with an
AI. And so suddenly the way that
software engineers operated it inverted
that you became much more of a manager
of agents. And since then the growth has
been unprecedented, absolutely
unprecedented how these agents have been
adopted.
>> And this shift is already visible in
software development. The important
change isn't simply that AI can write
better code. It's that developers are
beginning to hand entire tasks to agents
and increasingly act as managers rather
than doing every step themselves.
>> I asked Codex, and this is just
off-the-shelf Codex, no special access,
no special models. I just use my
personal Codex to say,
"Scan gregbrockman.com. Tell me what
vulnerabilities are there."
And gregbrockman.com is very simple.
It's just a static site. I was like,
"Come on, what what's it going to find?"
And it came back with like 13 findings.
And each one of them, I was like, "You
know, I kind of knew about this, but
it's so much work to fix it. And like,
oh yeah, I kind of forgot that I got
that open. Or you know, here's some
domain that was still, you know, it was
was legacy. I'm not using it anymore. I
hadn't set the records to prevent people
from spoofing your email address." A
bunch of little things like that. That
one seems like a big deal. Well, yeah,
you're right now that you say it. Now,
here's the thing.
Honestly,
to fix those kinds of things, you have
to set up like DMARC to do it right. And
there's this like whole like process you
have to do. Well, I'm just like, I don't
know, it just takes time. It takes a lot
of expertise. So, instead I just asked
Codex, "Can you fix it?"
>> Right.
>> And so,
it opened up my Cloudflare control panel
and just used computer use. It clicked
around. It like set a bunch of headers,
changed a bunch of settings. I used to
have it so that I had Cloudflare in
front and then AWS on the back end. I
was going over insecure HTTP here. Um
Codex had figured that out. Actually,
I'm not even sure how it how it saw that
cuz I thought everything was hidden
behind Cloudflare. It actually was just
like, "You know what? I'm just going to
migrate you off of AWS." So, it set up
Cloudflare pages, moved over the
content. I had an old jQuery version
that was insecure. It just got rid of
jQuery entirely. And it was like 15
minutes to find all these findings. It
was another hour to fix all of them. And
for me, I just was the manager. I just
gave it the intent. I said, "What I want
is I want to know where the problems are
and I want you to fix them." And it did
it. And that to me is more the future
that we're promised.
>> But perhaps the biggest acceleration
happens when these systems are used to
improve AI itself. Brockman says
engineering problems that once took
weeks can sometimes be investigated in
hours with Codex.
And that creates potentially powerful
feedback loop.
>> Well, I would think of this as an
overall pacing, right? An overall pacing
of of that frontier.
Well, back and another thing to think
about is the fact that
the
our ability to produce these models is
also and to get those capability gains
is also increasing, right? Our raw
capacity is because partly we understand
this technology a lot better than we did
in the past. And that's actually very
beneficial from a safety and alignment
perspective, too, right? That we really
understand a lot of how the technology
works. There's still many many pieces
that we don't understand, but our
ability to engineer it and to really do
good job of of it technically, um I
think has has increased a lot. Um and
that there are certain problems that
really used to hold us back that are
much diminished. For example,
these big training runs, you know, they
run for many months. And so, typically
we'd launch them from a branch of code
and so we'd be adding little fixes to
that that are specific to that run. At
the same time, everyone's continuing
their main line development. And every
so often there's something on the main
line that we're like, "Oh, we really
want that change. We really want that
fix." And then we're like, "All right,
we got to bring these two branches of
development together." And then we merge
them and then we launch the run and we
find that there's a bug because no one
has actually tested all these main line
fixes at the massive scale cuz we only
have one instance where we can do it.
And then we spend the next three to four
weeks with like all of our best people
scouring trying to figure out where did
the bug come from cuz you can't possibly
read all of the code that's been
written. Like no human can possibly do
that. Today,
you're running at that same type of
issue,
Codex just reads all the code, right?
Finds the bug. It says, "Here it is. No
problem."
And so there's something about
the partnership of
this amplification of what the models
can do that knock out whole classes of
problems where we know it would take 3
to 4 weeks and now takes a few hours.
We've seen this concretely multiple
times. And so what I'm getting at is
that there are a lot of limitations for
how we've had to develop in the past
that have made it a lot harder for us to
move forward on problems that are
important. And now we have the
opportunity to move forward on them
in a much more efficient fashion. And
that that to me is like let's use that
extra juice to
increase the alignment properties, to
increase the safety properties. Let's
make sure that we have models that
monitor other models in a trustworthy
fashion. Let's really implement some of
these ideas that we were talking about
back in 2017, right? That we have things
like
iterated amplification, things like
debate where the models can really spend
a bunch of compute in order to make sure
that what they're doing is actually
reasonable. To me that's the opportunity
in front of us.
>> And this brings us back to the real
question.
If AI can write software, use tools,
perform increasingly long tasks, and
even help researchers build the next
generation of AI,
at what point do we actually start
calling it AGI?
>> And I think what we've learned over the
years is that was a naive perspective.
And you can see this in the evidence of
like how
smooth in many ways the creation of
these agents has been, how the
transformation of software engineering
has been, now the transformation of
knowledge work generally, how cyber
capabilities, all these things, they're
gradually increasing. It's like one of
these things where there are moments
where you realize what has happened.
Hugging Face is a moment where you
realize where you are, but it's all
gradual. Like if you look back at kind
of these
different models that have been released
over time and their performance on
different benchmarks,
just you know sort of
that that exponential.
And so I think
that being said, the rest of the answer
is that if you go
if we go forward two years and we look
back, I think that that we will say this
is when AGI was created. So, I think we
are going through that moment.
>> But, it's not it's not something where
you can just say it now.
>> I don't think that it's ever going to be
as simple as you can say this was the
date. There's so many axes, and I think
that maybe one lesson in general to the
whole AGI conversation and why it's so
complicated is because humans, I think,
are capable of much more than
a naive view would give credit to. I
think that the diversity of what we do,
how we set goals, how we accomplish
goals, how tasks work, how jobs work,
all these things extremely deep.
And so, I think that what we're seeing
is that there's different aspects of
either how people accomplish tasks or
limitations that are hard for humans to
overcome, like the thing I mentioned
with read all the code, where these AIs
are becoming able to fill in those gaps,
are able to do more and more, and that
humans it turns out that there are some
things that we also really find are
extremely important for it to be a human
there. Like, where does the judgment
come from ultimately? Where does the
taste come from? Who decides what's good
and what's not good? Like, that is
something that should be uniquely human.
And I'll tell you, just to give you an
example,
one thing we've seen with image
generation, when you just generate an
image with ChatGPT and you show it to
someone, no one cares. It is
uninteresting to anyone besides you.
But, if you take a photo of your family
and you say turn this into the
into a caricature, turn this into, you
know, a cartoon style,
people love it. Goes viral.
And so, there's something about that
element of humanity that matters so
deeply. People want to connect with
other people. And that is actually, in
in my mind, the purpose of this
technology, the thing that I think is is
really emerging is for deepening human
connection so you can spend more time
with your family, can spend more time
with people you want to spend time with,
but less time, you know, typing away at
a machine,
less time, you know, getting carpal
tunnel, the whole thing. And so, I think
that there is something about what AGI
is as it's starting to unfold, it's
almost this like fog that's lifting,
it's almost teaching us more about
ourselves and what it means to be human.
>> OpenAI's vision also appears to be
moving beyond one universal definition
of AI.
For an individual user, the more
practical milestone may be something
much simpler.
An AI that knows your context, has
access to your tools, and can actually
accomplish things for you.
>> What people do see is how chat is
evolving, how Codex is evolving, and one
of your big priorities in this role, as
I understand it, is what you guys call
the merge internally, which is creating
this super app, this combined chat GPT
and Codex. Um and you've got chat GPT
for work now and the and the tabs.
Um but at the same time, like it doesn't
feel like that's the end state, you
know? And I'm a user of it and I see
realness in how it's uh currently
executed. Some things don't sync. You
can tell you're still combining tech
stacks. And I'm curious once you're done
with that work, what is the end product
for people that you imagine?
>> A personal AGI.
>> Personal AGI. And what How does that
manifest as software?
>> Yeah, it's
it's it's a really really exciting
vision in my mind, right? Because where
we started with chat GPT, it was a text
box, you could ask anything.
But if you think about it, it was
limited in a very fundamental way. The
only thing it could do is output some
text for you. The only impact it could
have in the world was, you know, kind of
in
your brain, right? That it wasn't able
to look at your calendar, it wasn't able
to look at your email.
And all of that is now changed.
Chat GPT work is really, by the way, a
Codex harnessed in the cloud, right? So,
we had chat that was this, you know,
question-answer system. Um iterate
turns, you know, people actually then go
beyond just, you know, their questions
to, you know, health, like all these
kinds of queries they're able to help
them.
We had Codex, which was an agentic
system that can go do work.
And so obvious it's like chat has to
become agentic. And so the place we
started at the beginning of the year was
to say, well, let's let's upgrade chat.
Let's, you know, make it also do agent
stuff. And it just turned out to be so
hard to execute because now we have two
agentic stacks and then we compare
between one and the other and there'll
be something that will work in this one.
We're like, why isn't it working in that
one? And it was just so slow. And so the
decision we made instead is say, let's
take the agentic stack we have, the
Codex stack, and bring it to ChatGPT.
And so what you see with chat work is
exactly right. This is an intermediate
state.
In some ways
it's kind of funny if you think about
it. ChatGPT was a text box. ChatGPT work
as a text box. But this text box is way
more awesome. It way more things.
Exactly. And so where we want to be is
you should
it should be simple. User should not
have to think about this. You shouldn't
have to think about tabs or switching
and shouldn't have model pickers. You
shouldn't have thinking levels. You
should have an AGI. You should be able
to ask it. And in fact, you shouldn't
even have to ask. It should proactively
say, hey, I noticed that your favorite
band, cuz it knows you well, is in town.
Tickets went on sale. I got you the
tickets exactly where you want for you
and your family. And you know, I only
had 10 minutes to do it cuz everything
is going to go so fast. So I just
purchased it proactively. Hope that's
okay. And if you've built enough trust
with the agent so you've given it
permission to do that and it kind of
knows that well, these tickets are
within your your price range, within
your budget. These other ones, oh no,
no, I I got to get permission for that.
Right? That's the kind of system that we
want. So trust is the keyword. EQ also
very important that if you've shared
some information, like you've talked
about a health condition, maybe that's
okay information for the AI to talk
about with a parent, but maybe it's not
okay to talk about with a stranger.
Right? All of that judgment and and also
observability and really helping you
build up that trust relationship with
the AI so that you know what
you know, when it's operating on your
behalf, it'll pursue your goals. That's
very, very critical.
But the same thing, same kind of
technology, the underlying AGI stack,
one stack, one way of doing connectors,
so it's able to hook up to this context,
also applies to the enterprise. And
we're really seeing it not just for
software engineering, which by the way
has been totally transformed this year.
And to the point of AGI being a gradual
thing, or a thing where we just
constantly update our expectations, like
we all kind of forget what software was
1 year ago. Totally different. A very
interesting thing within OpenAI is that
we're at basically 100% adoption of
ChatGPT work across every function,
right? Sales, finance, everyone. And
whenever, you know, we roll out a new
model and it goes away, or, you know,
the system breaks for whatever internal
reason, um
everyone's so upset. I cannot do my
work. And you're like, I don't get it. 6
months ago you didn't have this and you
were you were working just fine. Like,
why are you, you know, why are you
complaining? And you realize people
adapt so quickly and learn how to get
the most out of these tools. And once
you're there, you also just sort of
forget that there was any other way.
>> And the implications of that are that
what the coding AI tools have done to
engineering, this could do to all
knowledge work, all kinds of fields. And
that's how you guys used to talk about
AGI is like,
you know, it was basically AI that could
do most economically viable work better
than humans. And what you're describing
now in the product kind of sounds like
that.
>> I think that the amplification we've
seen in software engineering we expect
across every field for sure. And there
is something that I think that we
is also worth really thinking about,
which is the software engineers,
rather than having more vacation time
and having more chill job, they're all
working harder than ever before.
>> Yeah, you guys work a lot of hours here.
>> Yes. I think across the whole industry
people are talking about that. Because
now you feel like if your agents aren't
running,
it's just wasted opportunity. Right?
It's like, and I know that feeling
because we have spent years of we have
these GPU clusters, you're either
running something on them or you've lost
the opportunity. You can't get that time
back. It's not like the GPU hours that
you're not using while you're asleep
overnight, you can bank them up and use
them later. It's use it or lose it. And
that is very much the opportunity that
everyone has in front of them is you
have
almost an AGI, maybe soon truly an AGI
in your pocket. Can do anything you
want.
What is it you want? Waiting on you.
It's waiting on your direction, your
guidance, your management.
>> It would be nice if eventually we can
get some time back from AI.
>> That's right.
>> Like right now, I do feel like it it
will help you work more.
>> That's right. And so I think that I
think that that is that is exactly a
next step, right? It's really moving
from this like you're micromanaging this
agent. You have to provide it with these
tasks and you have to give the context
cuz it just doesn't have it.
Exactly. You want something that has
much more like you think about like a
really good assistant who really is
plugged in and you've given it again,
you have to build up trust with it,
right? You have to really say like,
okay, I trust you with this context, I
trust you with this responsibility, I
trust you with this job, whatever it is.
And once it's there, it should be able
to proactively run and really execute.
You wake up in the morning, you know, it
could say, "Hey, here are five things I
need from you to unblock me. You know, I
just need approval on this. Like here's
a question I have." Those kinds of
things. As you drink your morning
coffee, you say yes, no, yes, no.
Then you go, come back, you know, late
at night, and again you've got five more
things. And so I think that it's going
to be a world where it's going to be
different from I think the kind of AGI
world that many of us would have
pictured 5 10 years ago.
>> Software development may simply be the
first profession showing us what this
transition looks like.
If similar agents move into this area,
finance, marketing, design, and
everyday's business world, the biggest
question may no longer be what AI can
do, but what humans choose to ask it to
do.
So perhaps AGI won't arrive with an
announcement. We may simply look back
and realize that the transition had
already begun.
>> Mhm.