1
00:00:00,080 --> 00:00:03,837
The following is a conversation with
Greg Brockman. He's the co-founder and

2
00:00:03,920 --> 00:00:06,637
CTO of OpenAI, a world-class research

3
00:00:06,720 --> 00:00:09,197
organization developing ideas in AI with

4
00:00:09,280 --> 00:00:12,237
a goal of eventually creating a safe and

5
00:00:12,320 --> 00:00:14,237
friendly artificial general

6
00:00:14,320 --> 00:00:16,957
intelligence. One that benefits and

7
00:00:17,040 --> 00:00:20,157
empowers humanity. OpenAI is not only a

8
00:00:20,240 --> 00:00:22,557
source of publications, algorithms,

9
00:00:22,640 --> 00:00:25,837
tools, and data sets. Their mission is a

10
00:00:25,920 --> 00:00:27,437
catalyst for an important public

11
00:00:27,520 --> 00:00:30,237
discourse about our future with both

12
00:00:30,320 --> 00:00:33,997
narrow and general intelligence systems.

13
00:00:34,080 --> 00:00:38,157
This conversation is part of the
artificial intelligence podcast at MIT

14
00:00:38,240 --> 00:00:40,797
and beyond. If you enjoy it, subscribe

15
00:00:40,880 --> 00:00:43,437
on YouTube, iTunes, or simply connect

16
00:00:43,520 --> 00:00:45,677
with me on Twitter at Lex Freedman,

17
00:00:45,760 --> 00:00:49,357
spelled F R I D. And now, here's my

18
00:00:49,440 --> 00:00:53,627
conversation with Greg Brockman.

19
00:00:53,710 --> 00:00:55,730
>> [music]

20
00:01:01,570 --> 00:01:03,590
[music]

21
00:01:08,780 --> 00:01:11,757
[music] >> So in high
school and right after you

22
00:01:11,840 --> 00:01:15,677
wrote a draft of a chemistry textbook,
I saw that that covers everything from

23
00:01:15,760 --> 00:01:17,597
basic structure of the atom to quantum

24
00:01:17,680 --> 00:01:20,397
mechanics. So it's clear you have an

25
00:01:20,480 --> 00:01:23,277
intuition and a passion for both the uh

26
00:01:23,360 --> 00:01:25,837
the physical world with chemistry and

27
00:01:25,920 --> 00:01:29,357
now robotics to the digital world with

28
00:01:29,440 --> 00:01:33,517
uh AI, deep learning, reinforcement
learning so on. Do you see the physical

29
00:01:33,600 --> 00:01:37,757
world and the digital world as different
and what do you think is the gap?

30
00:01:37,840 --> 00:01:40,957
>> A lot of it actually boils down
to iteration speed, right? That I think

31
00:01:41,040 --> 00:01:44,877
that a lot of what really motivates me
is is building things, right? is the uh

32
00:01:44,960 --> 00:01:47,437
you know think about mathematics
for example where you think really hard

33
00:01:47,520 --> 00:01:50,557
about a problem you understand it you
write it down in this very obscure form

34
00:01:50,640 --> 00:01:55,197
that they call proof but then this is
in humanity's library right it's there

35
00:01:55,280 --> 00:01:58,797
forever this is some truth that we've
discovered and you know maybe only five

36
00:01:58,880 --> 00:02:01,277
people in your field will ever read
it but somehow you've kind of moved

37
00:02:01,360 --> 00:02:04,957
humanity forward and so I actually used
to really think that I was going to be a

38
00:02:05,040 --> 00:02:08,877
mathematician and uh then I actually
started writing this chemistry textbook

39
00:02:08,960 --> 00:02:12,237
one of my friends told me you'll never
publish it because you don't have a PhD

40
00:02:12,320 --> 00:02:16,477
so instead I I decided to build
a website and tried to promote my ideas

41
00:02:16,560 --> 00:02:20,877
that way and then I discovered
programming and uh I you know that in

42
00:02:20,960 --> 00:02:23,997
programming you think hard about
a problem you understand it you write down

43
00:02:24,080 --> 00:02:28,237
in a very obscure form that we call
a program but then once again it's in

44
00:02:28,320 --> 00:02:31,597
humanity's library right and anyone
could get the benefit from it and the

45
00:02:31,680 --> 00:02:35,037
scalability is massive and so I think
that the thing that really appeals to me

46
00:02:35,120 --> 00:02:39,037
about the digital world is that you
can have this this this insane leverage

47
00:02:39,120 --> 00:02:43,517
right a single individual with an idea
is able to affect the entire planet Um,

48
00:02:43,600 --> 00:02:46,397
and that's something I think is really
hard to do if you're moving around

49
00:02:46,480 --> 00:02:50,397
physical atoms. >> But you
said mathematics. So if you look

50
00:02:50,480 --> 00:02:53,277
at the the wet thing over here, our

51
00:02:53,360 --> 00:02:56,637
mind, do you ultimately see it as just

52
00:02:56,720 --> 00:03:01,677
math as just information processing or
is there some other magic as you've seen

53
00:03:01,760 --> 00:03:04,717
if you've seen through biology and
chemistry and so on? Yeah, I think it's

54
00:03:04,800 --> 00:03:08,317
really interesting to think about humans
as just information processing systems

55
00:03:08,400 --> 00:03:12,237
and uh that it seems like it's actually
a pretty good uh way of describing a lot

56
00:03:12,320 --> 00:03:16,157
of of kind of how the world works
or a lot of what we're capable of to think

57
00:03:16,240 --> 00:03:20,077
that that you know again if you just
look at at technological innovations

58
00:03:20,160 --> 00:03:23,677
over time that in some ways the most
transformative innovation that we've had

59
00:03:23,760 --> 00:03:27,037
has been the computer right in some ways
the internet you know that what has the

60
00:03:27,120 --> 00:03:30,397
internet done right the internet is not
about these physical cables it's about

61
00:03:30,480 --> 00:03:33,597
the fact that I am suddenly able
to instantly communicate with any other

62
00:03:33,680 --> 00:03:37,757
human on the planet. I'm able
to retrieve any piece of knowledge that in

63
00:03:37,840 --> 00:03:42,397
some ways the human race has ever
had and that those are these insane

64
00:03:42,480 --> 00:03:46,877
transformations. Do you see the our
society as a whole, the collective as

65
00:03:46,960 --> 00:03:50,317
another extension of the intelligence
of the human being? So if you look at the

66
00:03:50,400 --> 00:03:53,437
human being as information processing
system, you mentioned the internet, the

67
00:03:53,520 --> 00:03:57,837
networking, do you see us all together
as a civilization as a as a kind of

68
00:03:57,920 --> 00:04:00,957
intelligence system? Yeah, I think
this is actually a really interesting

69
00:04:01,040 --> 00:04:04,237
perspective uh to take and to think
about that you sort of have this

70
00:04:04,320 --> 00:04:08,077
collective intelligence of all of society.
The economy itself is this

71
00:04:08,160 --> 00:04:10,797
superhuman machine that is optimizing

72
00:04:10,880 --> 00:04:15,197
something, right? And it's in some ways
a company has a will of its own, right?

73
00:04:15,280 --> 00:04:17,837
That you have all these individuals who
are all pursuing their own individual

74
00:04:17,920 --> 00:04:20,797
goals and thinking really hard
and thinking about the right things to do,

75
00:04:20,880 --> 00:04:22,557
but somehow the company does something

76
00:04:22,640 --> 00:04:25,437
that is this emergent thing uh and that

77
00:04:25,520 --> 00:04:27,837
is is it's a really useful abstraction.

78
00:04:27,920 --> 00:04:31,357
And so I think that in some ways,
you know, we think of of ourselves as the

79
00:04:31,440 --> 00:04:34,077
most intelligent things on the planet
and the most powerful things on the

80
00:04:34,160 --> 00:04:37,197
planet, but there are things that are
bigger than us that are these systems

81
00:04:37,280 --> 00:04:40,797
that we all contribute to. Um, and so
I think actually, you know, it's a it's

82
00:04:40,880 --> 00:04:44,637
interesting to think about uh if you've
read Isaac Aisimov's foundation, right,

83
00:04:44,720 --> 00:04:47,677
that uh that there's this concept
of psycho history in there, which is

84
00:04:47,760 --> 00:04:51,117
effectively this that if you have
trillions or quadrillions of of beings,

85
00:04:51,200 --> 00:04:54,877
then maybe you could actually predict
what that being that that huge macro

86
00:04:54,960 --> 00:04:58,797
being will do. uh and uh almost
independent of what the individuals

87
00:04:58,880 --> 00:05:01,997
want. And I actually have a sec
a second angle on this that I think is

88
00:05:02,080 --> 00:05:05,997
interesting which is thinking about
uh technological determinism. One thing

89
00:05:06,080 --> 00:05:09,517
that that I actually think a lot about
with with open AI, right, is that we're

90
00:05:09,600 --> 00:05:13,677
kind of coming on onto this insanely
transformational technology of of

91
00:05:13,760 --> 00:05:16,637
general intelligence, right? That will
happen at some point. And there's a

92
00:05:16,720 --> 00:05:20,637
question of how can you take actions
that will actually steer it to go better

93
00:05:20,720 --> 00:05:24,637
rather than worse. And that I think
one question you need to ask is as a

94
00:05:24,720 --> 00:05:28,717
scientist, as an inventor, as a creator,
what impact can you have in general,

95
00:05:28,800 --> 00:05:31,437
right? You look at things like the
telephone invented by two people on the

96
00:05:31,520 --> 00:05:34,557
same day. Like what does that mean? Like
what does that mean about the shape of

97
00:05:34,640 --> 00:05:37,677
innovation? And I think that what's
going on is everyone's building on the

98
00:05:37,760 --> 00:05:41,277
shoulders of the same giants. And so
you can kind of you can't really hope to

99
00:05:41,360 --> 00:05:42,877
create something no one else ever would.

100
00:05:42,960 --> 00:05:45,517
You know, if Einstein wasn't born,
someone else would have come up with

101
00:05:45,600 --> 00:05:48,717
relativity. You know, he changed
the timeline a bit, right? that maybe it

102
00:05:48,800 --> 00:05:51,517
would have taken another 20 years,
but it wouldn't be that fundamentally

103
00:05:51,600 --> 00:05:54,717
humanity would never discover
these these fundamental truths.

104
00:05:54,800 --> 00:05:57,037
>> So there's some kind of invisible

105
00:05:57,120 --> 00:05:59,997
momentum that some people like Einstein

106
00:06:00,080 --> 00:06:03,437
or OpenAI is plugging into uh that

107
00:06:03,520 --> 00:06:07,437
anybody else can also plug into
and ultimately that wave takes us into a

108
00:06:07,520 --> 00:06:09,357
certain direction. That's what you mean by

109
00:06:09,440 --> 00:06:11,757
>> That's right. That's right. And you
know this kind of seems to play out in a

110
00:06:11,840 --> 00:06:15,117
bunch of different ways. uh that there's
some exponential that is being ridden

111
00:06:15,200 --> 00:06:18,557
and that the exponential itself which
one it is changes you think about Moors

112
00:06:18,640 --> 00:06:23,197
law an entire industry set its clock
to it for 50 years like how can that be

113
00:06:23,280 --> 00:06:28,077
right how is that possible and yet
somehow it happened and so I think you

114
00:06:28,160 --> 00:06:31,277
can't hope to ever invent something
that no one else will maybe you can change

115
00:06:31,360 --> 00:06:34,877
the timeline a little bit but if you
really want to make a difference I think

116
00:06:34,960 --> 00:06:38,397
that the thing that you really have
to do the only real degree of freedom you

117
00:06:38,480 --> 00:06:42,557
have is to set the initial conditions
under which a technology is born And so

118
00:06:42,640 --> 00:06:43,837
you think about the internet, right?

119
00:06:43,920 --> 00:06:46,077
That there were lots of other
competitors trying to build similar

120
00:06:46,160 --> 00:06:51,037
things and the internet won and that
the initial conditions were that it was

121
00:06:51,120 --> 00:06:55,517
created by this group that really valued
people being able to be uh you know

122
00:06:55,600 --> 00:06:58,637
anyone being able to plug in this very
academic mindset of of being open and

123
00:06:58,720 --> 00:07:01,917
connected. Um and I think that the
internet for the next 40 years really

124
00:07:02,000 --> 00:07:05,997
played out that way. um you know maybe
today uh things are starting to to shift

125
00:07:06,080 --> 00:07:08,637
in in a different direction but I
think that those initial conditions were

126
00:07:08,720 --> 00:07:12,637
really important to determine the next
40 years worth of of progress. That's

127
00:07:12,720 --> 00:07:16,477
really beautifully put. So another
example of that I think about you know I

128
00:07:16,560 --> 00:07:21,117
recently looked at it I looked at
Wikipedia the formation of Wikipedia and

129
00:07:21,200 --> 00:07:25,037
I wonder what the internet would
be like if Wikipedia had ads.

130
00:07:25,120 --> 00:07:29,837
>> You know there's a interesting argument
that uh why they chose not to make it

131
00:07:29,920 --> 00:07:34,317
put advertisement on Wikipedia. I think
it's uh I think Wikipedia is one of the

132
00:07:34,400 --> 00:07:37,437
greatest resources we have on the
internet. It's it's extremely surprising

133
00:07:37,520 --> 00:07:41,357
how well it works and how well it
was able to aggregate all this kind of uh

134
00:07:41,440 --> 00:07:45,357
good information and the essentially
the creator of Wikipedia. I don't know

135
00:07:45,440 --> 00:07:49,117
there's probably some debates there
but set the initial conditions and now it

136
00:07:49,200 --> 00:07:52,717
carried itself forward. That's really
interesting. So you're the way you're

137
00:07:52,800 --> 00:07:55,917
thinking about AGI or artificial
intelligence is you're focused on

138
00:07:56,000 --> 00:07:58,477
setting the initial conditions
for for the progress.

139
00:07:58,560 --> 00:08:01,837
>> That's right. >> That's powerful.
Okay. So looking into

140
00:08:01,920 --> 00:08:05,357
the future, if you create an AGI system,

141
00:08:05,440 --> 00:08:07,437
like one that can ace the touring test,

142
00:08:07,520 --> 00:08:12,557
natural language, what do you think
would be the interactions you would have

143
00:08:12,640 --> 00:08:15,437
with it? What do you think are the
questions you would ask? Like what would

144
00:08:15,520 --> 00:08:19,117
be the first question you
would ask it, her, him?

145
00:08:19,200 --> 00:08:22,797
>> That's right. I think that at that
point, if you've really built a powerful

146
00:08:22,880 --> 00:08:26,317
system that is capable of shaping
the future of humanity, the first question

147
00:08:26,400 --> 00:08:29,677
that you really should ask is how do we
make sure that this plays out well? Um,

148
00:08:29,760 --> 00:08:34,877
and so that's actually the first question
that I would ask a powerful AGI system is.

149
00:08:34,960 --> 00:08:38,397
>> So you wouldn't ask your colleague,
you wouldn't ask like Ilia, you would ask

150
00:08:38,480 --> 00:08:40,957
the AGI system. >> Oh, we've
already had the conversation

151
00:08:41,040 --> 00:08:45,197
with Ilia, right? And everyone here. Uh,
and so you want as many perspectives and

152
00:08:45,280 --> 00:08:48,797
uh a piece of wisdom as you can for
for answering this question. So I don't

153
00:08:48,880 --> 00:08:52,877
think you necessarily defer to whatever
your powerful system tells you. Um, but

154
00:08:52,960 --> 00:08:57,597
you use it as one input uh to try
to figure out what to do. But and I guess

155
00:08:57,680 --> 00:09:00,397
fun fundamentally what it really comes
down to is if you built something really

156
00:09:00,480 --> 00:09:04,317
powerful and you think about think
about for example the creation of uh of

157
00:09:04,400 --> 00:09:06,717
shortly after the creation of nuclear
weapons right the most important

158
00:09:06,800 --> 00:09:10,717
question in the world was what's
the world order going to be like how do we

159
00:09:10,800 --> 00:09:15,437
set ourselves up in a place where we're
going to be able to survive as a species

160
00:09:15,520 --> 00:09:18,637
with AGI I think the question is slightly
different right that there is a

161
00:09:18,720 --> 00:09:21,837
question of how do we make sure that we
don't get the negative effects but

162
00:09:21,920 --> 00:09:25,757
there's also the positive side right
you imagine that you know like like what

163
00:09:25,840 --> 00:09:29,677
will AGI be like like what will it be
capable of? And I think that that one of

164
00:09:29,760 --> 00:09:33,437
the core reasons that an AGI can be
powerful and transformative is actually

165
00:09:33,520 --> 00:09:35,677
due to uh technological development.

166
00:09:35,760 --> 00:09:37,117
Yeah. >> Right.
If you have something that's

167
00:09:37,200 --> 00:09:42,237
capable as capable as a human and that
it's much more scalable uh that you

168
00:09:42,320 --> 00:09:45,037
absolutely want that thing to go read
the whole scientific literature and

169
00:09:45,120 --> 00:09:47,757
think about how to create cures for all
the diseases, right? You want it to

170
00:09:47,840 --> 00:09:51,277
think about how to go and build uh
technologies to help us create material

171
00:09:51,360 --> 00:09:55,677
abundance and to figure out societal
problems that we have trouble with like

172
00:09:55,760 --> 00:09:58,557
how are we supposed to clean up the
environment and you know maybe you want

173
00:09:58,640 --> 00:10:02,077
this uh to go and invent a bunch of little
robots that will go out and uh be

174
00:10:02,160 --> 00:10:04,557
biodegradable and uh turn ocean debris

175
00:10:04,640 --> 00:10:08,077
into harmless uh molecules and um I

176
00:10:08,160 --> 00:10:12,877
think that that that positive side
is something that I think people miss

177
00:10:12,960 --> 00:10:16,237
sometimes when thinking about what
an AGI will be like. Um, and so I think

178
00:10:16,320 --> 00:10:19,837
that if you have a system that's capable
of all of that, you absolutely want its

179
00:10:19,920 --> 00:10:24,957
advice about how do I make sure that we're
using your uh your capabilities in

180
00:10:25,040 --> 00:10:27,517
a positive way for humanity.
>> So what do you think about that

181
00:10:27,600 --> 00:10:30,797
psychology that uh looks at all the

182
00:10:30,880 --> 00:10:35,357
different possible trajectories of an
AGI system, many of which perhaps the

183
00:10:35,440 --> 00:10:39,597
majority of which are positive
and nevertheless focuses on the negative

184
00:10:39,680 --> 00:10:42,557
trajectories. I mean, you get
to interact with folks, you get to think

185
00:10:42,640 --> 00:10:45,197
about this maybe within yourself as

186
00:10:45,280 --> 00:10:49,837
well. Um, you look at Sam Harris and so
on. It seems to be, sorry to put it this

187
00:10:49,920 --> 00:10:53,037
way, but almost more fun
to think about the negative

188
00:10:53,120 --> 00:10:56,877
>> possibilities, whatever
that's deep in our psychology.

189
00:10:56,960 --> 00:11:00,797
What do you think about that? And how
do we deal with it? Because we want AI to

190
00:11:00,880 --> 00:11:04,157
help us. >> So,
I I think there's kind of two

191
00:11:04,240 --> 00:11:06,957
problems uh entailed in that question.

192
00:11:07,040 --> 00:11:11,757
Uh the first is more of the question
of how can you even picture what a world

193
00:11:11,840 --> 00:11:13,757
with a new technology will be like?

194
00:11:13,840 --> 00:11:19,157
>> Now imagine we're in 1950 and I'm
trying to describe Uber to someone.

195
00:11:19,240 --> 00:11:23,437
[laughter] >> Apps and the internet.
Yeah. I mean

196
00:11:23,520 --> 00:11:27,437
you're that's that's going to be extremely
complicated, but it's imaginable.

197
00:11:27,520 --> 00:11:31,917
>> It's it's imaginable, right? But and
now imagine being in 1950 and predicting

198
00:11:32,000 --> 00:11:35,997
Uber, right? And you need to describe
the internet. You need to describe GPS.

199
00:11:36,080 --> 00:11:39,357
you need to describe the fact that
everyone's going to have this phone in

200
00:11:39,440 --> 00:11:43,037
their pocket. And so I think
that that just the first

201
00:11:43,120 --> 00:11:46,957
truth is that it is hard to picture how
a transformative technology will play

202
00:11:47,040 --> 00:11:50,797
out in the world. Um we've seen that
before with technologies that are far

203
00:11:50,880 --> 00:11:52,797
less transformative than AGI will be.

204
00:11:52,880 --> 00:11:56,957
And so I think that that one piece
is that it's just even hard to imagine and

205
00:11:57,040 --> 00:11:59,037
to really put yourself in a world where

206
00:11:59,120 --> 00:12:04,157
you can predict what that that
positive vision would would be like.

207
00:12:04,240 --> 00:12:05,757
And you know I think the second thing is

208
00:12:05,840 --> 00:12:08,957
that it is I think it is always easier

209
00:12:09,040 --> 00:12:13,197
to support the negative side than
the positive side. It's always easier to

210
00:12:13,280 --> 00:12:17,357
destroy than create and you
know less in a in a in a in a

211
00:12:17,440 --> 00:12:20,877
physical sense and more just in a in an
intellectual sense. Right. Because you

212
00:12:20,960 --> 00:12:23,837
know I think that that with creating
something you need to just get a bunch

213
00:12:23,920 --> 00:12:27,197
of things right and to destroy you
just need to get one thing wrong.

214
00:12:27,280 --> 00:12:29,037
>> Yeah. And so I think
that that what that

215
00:12:29,120 --> 00:12:32,157
means is that I think a lot of people's
thinking dead ends as soon as they see

216
00:12:32,240 --> 00:12:35,997
the negative story. But that being said,

217
00:12:36,080 --> 00:12:39,517
I actually actually have some hope,
right? I think that that the that the

218
00:12:39,600 --> 00:12:43,997
positive vision is something that I
think can be um is something that we can

219
00:12:44,080 --> 00:12:47,917
we can talk about. And I think that just
simply saying this fact of yeah, like

220
00:12:48,000 --> 00:12:49,197
there's positive, there's negatives.

221
00:12:49,280 --> 00:12:50,797
Everyone likes to dwell on the negative.

222
00:12:50,880 --> 00:12:53,517
People actually respond well to that
message and say, "Huh, you're right.

223
00:12:53,600 --> 00:12:57,037
There's a part of this that we're not
talking about, not thinking about." And

224
00:12:57,120 --> 00:13:01,197
that's actually something that's that's
that's I think really been a key part of

225
00:13:01,280 --> 00:13:03,917
how we think about AGI at OpenAI, right?

226
00:13:04,000 --> 00:13:07,357
You can kind of look at it as like okay
like OpenAI talks about the fact that

227
00:13:07,440 --> 00:13:10,477
there are risks and yet they're
trying to build this system.

228
00:13:10,560 --> 00:13:14,637
>> Like how do you square those those
two facts? So, do you share the intuition

229
00:13:14,720 --> 00:13:17,437
that some people have uh I mean from Sam

230
00:13:17,520 --> 00:13:20,637
Harris to even Elon Musk himself that

231
00:13:20,720 --> 00:13:24,237
it's tricky as you develop AGI to keep

232
00:13:24,320 --> 00:13:27,277
it from slipping into the existential

233
00:13:27,360 --> 00:13:29,437
threats into the negative? What's your

234
00:13:29,520 --> 00:13:32,797
intuition about how hard is it to keep

235
00:13:32,880 --> 00:13:36,957
AI development on the positive track?

236
00:13:37,040 --> 00:13:39,197
What's your intuition there?
>> To to answer that question, you can

237
00:13:39,280 --> 00:13:41,117
really look at how we structure Open AI.

238
00:13:41,200 --> 00:13:44,877
So we really have three main arms.
So we have capabilities which is actually

239
00:13:44,960 --> 00:13:48,557
doing the technical work and pushing
forward what these systems can do.

240
00:13:48,640 --> 00:13:53,437
There's safety which is working
on technical mechanisms to ensure that the

241
00:13:53,520 --> 00:13:57,517
systems we build are aligned with human
values. And then there's policy which is

242
00:13:57,600 --> 00:14:00,637
making sure that we have governance
mechanisms answering that question of

243
00:14:00,720 --> 00:14:03,437
well whose values and so I think that

244
00:14:03,520 --> 00:14:07,757
the technical safety one is the one that
people kind of talk about the most right

245
00:14:07,840 --> 00:14:11,757
you talk about like think about you know
all of the dystopic AI movies a lot of

246
00:14:11,840 --> 00:14:16,237
that is about not having good technical
safety in place um and what we've been

247
00:14:16,320 --> 00:14:19,117
finding is that you know I think
that actually a lot of people look at the

248
00:14:19,200 --> 00:14:22,397
technical safety problem
and think it's just intractable

249
00:14:22,480 --> 00:14:26,397
right this question of what do humans
want how supposed to write that down?

250
00:14:26,480 --> 00:14:28,557
Can I even write down what I want?

251
00:14:28,640 --> 00:14:32,237
>> No way. And then they stop there. But

252
00:14:32,320 --> 00:14:36,237
the thing is, we've already built
systems that are able to learn things

253
00:14:36,320 --> 00:14:40,317
that humans can't specify. You know,
even the rules for how to recognize if

254
00:14:40,400 --> 00:14:42,237
there's a cat or a dog in an image.

255
00:14:42,320 --> 00:14:43,597
>> Turns out it's
intractable to write that

256
00:14:43,680 --> 00:14:45,677
down. And yet, we're able to learn it.

257
00:14:45,760 --> 00:14:48,957
And that what we're seeing with systems
we build at Open AAI, and they're still

258
00:14:49,040 --> 00:14:53,597
in early proof of concept stage, is that
you are able to learn human preferences.

259
00:14:53,680 --> 00:14:57,117
you're able to learn what humans want
from data. Um, and so that's kind of the

260
00:14:57,200 --> 00:15:00,717
core focus for our technical safety team.
And I think that uh that they're

261
00:15:00,800 --> 00:15:03,597
actually we've had some pretty
encouraging updates in terms of what

262
00:15:03,680 --> 00:15:07,517
we've been able to make work. >> So
you have an intuition and a hope that

263
00:15:07,600 --> 00:15:09,837
from data, you know, looking at the

264
00:15:09,920 --> 00:15:11,997
value alignment problem. From data, we

265
00:15:12,080 --> 00:15:14,957
can build systems that align with the

266
00:15:15,040 --> 00:15:17,997
collective better angels of our nature.

267
00:15:18,080 --> 00:15:22,557
So align with the the ethics and the
morals of human beings. to even say this

268
00:15:22,640 --> 00:15:26,077
in a different way. I mean, think about
how how do we align humans, right? Think

269
00:15:26,160 --> 00:15:30,237
about like a human baby can grow up to be
an evil person or a great person. And

270
00:15:30,320 --> 00:15:33,677
a lot of that is from learning from data,
right? That you have some feedback

271
00:15:33,760 --> 00:15:37,037
as a child is growing up. They get
to see positive examples. And so I think

272
00:15:37,120 --> 00:15:40,077
that that just like that the the only

273
00:15:40,160 --> 00:15:44,077
example we have of a general
intelligence uh that is able to learn

274
00:15:44,160 --> 00:15:46,957
from data uh to align with human values

275
00:15:47,040 --> 00:15:51,357
and to learn values. um I think we
shouldn't be surprised that we can do

276
00:15:51,440 --> 00:15:54,877
the same sorts of of of techniques
or whether the same sort of techniques end

277
00:15:54,960 --> 00:15:58,317
up being how we we we solve
value alignment for AGIS.

278
00:15:58,400 --> 00:16:00,157
>> So let's go even
uh higher. I don't know

279
00:16:00,240 --> 00:16:02,077
if you've read the book Sapiens

280
00:16:02,160 --> 00:16:05,437
>> but uh there's an idea that you know um

281
00:16:05,520 --> 00:16:07,277
that as a collective as us human beings

282
00:16:07,360 --> 00:16:10,957
we kind of develop together an ideas

283
00:16:11,040 --> 00:16:15,997
that we hold. There's no in that context
objective truth. we just kind of all

284
00:16:16,080 --> 00:16:19,517
agree to certain ideas and hold them
as a collective. So do you have a sense

285
00:16:19,600 --> 00:16:22,797
that there is >> in the world
of good and evil, do you

286
00:16:22,880 --> 00:16:25,917
have a sense that to the first
approximation there are some things that

287
00:16:26,000 --> 00:16:28,957
are good and that we could teach systems

288
00:16:29,040 --> 00:16:33,837
to behave to be good?
>> So I think that that this actually

289
00:16:33,920 --> 00:16:38,637
blends into our third team, right, which
is the policy team. And this is the one

290
00:16:38,720 --> 00:16:42,477
the aspect I think people really talk
about way less than they should, right?

291
00:16:42,560 --> 00:16:46,077
Because imagine that we build super
powerful systems that we've managed to

292
00:16:46,160 --> 00:16:49,197
figure out all the mechanisms for these
things to do whatever the operator

293
00:16:49,280 --> 00:16:53,517
wants. The most important question
becomes who's the operator? What do they

294
00:16:53,600 --> 00:16:57,517
want? And how is that going to affect
everyone else? Right? And and I think

295
00:16:57,600 --> 00:17:02,557
that this question of what is good?
What are those values? I mean I think you

296
00:17:02,640 --> 00:17:06,397
don't even have to go to those those
very grand existential places to start

297
00:17:06,480 --> 00:17:08,157
to to realize how hard this problem is.

298
00:17:08,240 --> 00:17:12,637
you just look at different countries
and cultures across the world and that

299
00:17:12,720 --> 00:17:16,237
there's there's a very different
conception of how the world works and

300
00:17:16,320 --> 00:17:19,117
you know what what what kinds of of ways

301
00:17:19,200 --> 00:17:21,597
that society wants to operate. And so I

302
00:17:21,680 --> 00:17:26,157
think that that the the really
core question is is is actually very

303
00:17:26,240 --> 00:17:29,757
concrete. Um and I think it's not
a question that we have ready answers to,

304
00:17:29,840 --> 00:17:32,877
right? is how do you have a world where

305
00:17:32,960 --> 00:17:37,517
all the different countries that we have,
United States, China, Russia and

306
00:17:37,600 --> 00:17:39,117
you know the the the hundreds of other

307
00:17:39,200 --> 00:17:41,757
countries out there are able to continue

308
00:17:41,840 --> 00:17:44,797
to not just operate in the way that that

309
00:17:44,880 --> 00:17:50,077
they that they see fit but in in a that
the world that emerges um in these where

310
00:17:50,160 --> 00:17:54,957
where you have these very powerful
systems uh operating alongside humans

311
00:17:55,040 --> 00:17:58,797
ends up being something that empowers
humans more that makes like ex human

312
00:17:58,880 --> 00:18:03,677
existence be a more meaningful thing and
uh that people are happier and wealthier

313
00:18:03,760 --> 00:18:07,437
and uh able to live more fulfilling lives.
It's not an obvious thing for how

314
00:18:07,520 --> 00:18:12,157
to design that world once you have
that very powerful system. So if we take a

315
00:18:12,240 --> 00:18:14,317
little step back and we're having like a

316
00:18:14,400 --> 00:18:19,437
fascinating conversation and open in many
ways a tech leader in the world and

317
00:18:19,520 --> 00:18:23,117
yet we're thinking about these
big existential questions which is

318
00:18:23,200 --> 00:18:26,717
fascinating really important. I think
you're a leader in that space and that's

319
00:18:26,800 --> 00:18:30,877
a really important space of just
thinking how AI affects society in in a

320
00:18:30,960 --> 00:18:35,277
big picture view. So Oscar Wild said
we're all in the gutter but some of us

321
00:18:35,360 --> 00:18:37,037
are looking at the stars and I think

322
00:18:37,120 --> 00:18:40,317
open AI has a charter that looks to the

323
00:18:40,400 --> 00:18:44,557
stars I would say to create intelligence
to create general intelligence make it

324
00:18:44,640 --> 00:18:46,877
beneficial safe and collaborative. So

325
00:18:46,960 --> 00:18:49,037
can you tell me

326
00:18:49,120 --> 00:18:51,837
uh how that came about? How a mission

327
00:18:51,920 --> 00:18:54,237
like that and the path to creating a

328
00:18:54,320 --> 00:18:56,397
mission like that at OpenAI was founded?

329
00:18:56,480 --> 00:19:01,277
>> Yeah. So I think that in some ways
it really boils down to taking a look at

330
00:19:01,360 --> 00:19:04,957
the landscape, right? So if you think
about the history of AI that basically

331
00:19:05,040 --> 00:19:09,597
for the past 60 or 70 years, people have
thought about this goal of what could

332
00:19:09,680 --> 00:19:12,957
happen if you could automate
human intellectual labor,

333
00:19:13,040 --> 00:19:17,197
>> right? Imagine you can build a computer
system that could do that. What becomes

334
00:19:17,280 --> 00:19:21,197
possible? We have a lot of sci-fi that
tells stories of various dystopias and

335
00:19:21,280 --> 00:19:23,357
you know increasingly you have movies
like herd that tell you a little bit

336
00:19:23,440 --> 00:19:25,357
about maybe more of a little bit utopic

337
00:19:25,440 --> 00:19:29,037
vision. Uh you think about the impact

338
00:19:29,120 --> 00:19:31,997
that we've seen from being able to uh

339
00:19:32,080 --> 00:19:36,477
have bicycles for our minds in computers.
uh and that I think that the

340
00:19:36,560 --> 00:19:38,637
the impact of of computers in the

341
00:19:38,720 --> 00:19:43,517
internet has just far outstripped what
anyone really could have predicted. And

342
00:19:43,600 --> 00:19:47,677
so I think that it's very clear that if
you can build an AGI, it will be the

343
00:19:47,760 --> 00:19:51,757
most transformative technology
that humans will ever create.

344
00:19:51,840 --> 00:19:56,157
And so what it boils down to then
is a question of well is there a path? Is

345
00:19:56,240 --> 00:19:57,917
there hope? Is there a way to build such

346
00:19:58,000 --> 00:20:00,397
a system? And I think that for 60 or 70

347
00:20:00,480 --> 00:20:03,437
years that people got excited and uh uh

348
00:20:03,520 --> 00:20:07,517
that you know ended up not being able
to deliver on the hopes that that that

349
00:20:07,600 --> 00:20:11,437
people had pinned on them. And I think
that then you know that after you know

350
00:20:11,520 --> 00:20:15,597
two two winters of AI development uh
that people uh you know I think kind of

351
00:20:15,680 --> 00:20:19,437
almost stopped daring to dream right
that really talking about AGI or

352
00:20:19,520 --> 00:20:23,837
thinking about AGI became almost
this taboo in the community.

353
00:20:23,920 --> 00:20:27,677
>> But I actually think that people took
the wrong lesson from AI history. And if

354
00:20:27,760 --> 00:20:31,997
you look back starting in 1959 is when
the perceptron was released. And this is

355
00:20:32,080 --> 00:20:35,677
basically you know one of the earliest
neural networks. Um it was released to

356
00:20:35,760 --> 00:20:39,037
what was perceived as this massive
overhype. So in the New York Times in

357
00:20:39,120 --> 00:20:41,357
1959 you have this article uh saying

358
00:20:41,440 --> 00:20:46,157
that you know the the perceptron will
one day recognize people call out their

359
00:20:46,240 --> 00:20:50,557
names instantly translate speech between
languages. And people at the time looked

360
00:20:50,640 --> 00:20:53,517
at this and said this is like
your system can't do any of that. and

361
00:20:53,600 --> 00:20:56,637
basically spent 10 years trying to
discredit the whole perceptron direction

362
00:20:56,720 --> 00:21:01,037
and and succeeded and all the funding
dried up and you know people kind of uh

363
00:21:01,120 --> 00:21:04,397
went in other directions and you know
in the 80s there was this resurgence and

364
00:21:04,480 --> 00:21:08,237
I'd always heard that the resurgence in
the 80s was due to the invention of back

365
00:21:08,320 --> 00:21:11,517
propagation and these these algorithms
that got people excited but actually the

366
00:21:11,600 --> 00:21:15,357
causality was due to people building
larger computers that you can find these

367
00:21:15,440 --> 00:21:18,957
these articles from the 80s saying that
the democratization of computing power

368
00:21:19,040 --> 00:21:21,917
suddenly meant that you could run these
larger neural networks and then people

369
00:21:22,000 --> 00:21:23,517
started to do all these amazing things.

370
00:21:23,600 --> 00:21:26,397
The back propagation algorithm
was invented and you know that the the

371
00:21:26,480 --> 00:21:29,277
neural nets people were running were
these tiny little like 20 neuron neural

372
00:21:29,360 --> 00:21:32,077
nets, right? >> Right. Like what are
you supposed to learn with 20 neurons?

373
00:21:32,160 --> 00:21:34,877
>> Yeah. >> And so of course
they weren't able to

374
00:21:34,960 --> 00:21:39,517
get great results. And it really wasn't
until 2012 that this approach that's

375
00:21:39,600 --> 00:21:44,637
almost the most simple natural approach
that people had come up with in the 50s,

376
00:21:44,720 --> 00:21:47,757
right? In some ways even in the 40s
before there were computers with the

377
00:21:47,840 --> 00:21:50,477
pits neuron.

378
00:21:50,560 --> 00:21:55,117
Suddenly this became the best way of
solving problems right and I think there

379
00:21:55,200 --> 00:22:00,237
are three core properties that deep
learning has that I think are very worth

380
00:22:00,320 --> 00:22:01,917
paying attention to the first is

381
00:22:02,000 --> 00:22:04,077
generality we have a very small number

382
00:22:04,160 --> 00:22:07,277
of deep learning tools SGD deep neural

383
00:22:07,360 --> 00:22:10,637
net maybe some some you know RL and it

384
00:22:10,720 --> 00:22:12,877
solves this huge variety of problems

385
00:22:12,960 --> 00:22:14,557
>> speech recognition machine translation

386
00:22:14,640 --> 00:22:17,757
game playing all these problems small

387
00:22:17,840 --> 00:22:21,597
set of tools so there's the generality
There's a second piece which is the

388
00:22:21,680 --> 00:22:25,357
competence. You want to solve any of those
problems, throw out 40 years worth

389
00:22:25,440 --> 00:22:29,197
of normal computer vision research,
replace it with a deep neural net. It's

390
00:22:29,280 --> 00:22:33,277
going to work better. And there's
a third piece which is the scalability,

391
00:22:33,360 --> 00:22:37,037
right? That one thing that has been
shown time and time again is that you if

392
00:22:37,120 --> 00:22:40,797
you have a larger neural network, throw
more comput, more data at it, it will

393
00:22:40,880 --> 00:22:43,517
work better. Those three properties

394
00:22:43,600 --> 00:22:48,477
together feel like essential parts
of building a general intelligence. Now it

395
00:22:48,560 --> 00:22:52,237
doesn't just mean that if we scale up
what we have that we will have an AGI

396
00:22:52,320 --> 00:22:55,837
right there are clearly missing pieces
there are missing ideas we need to have

397
00:22:55,920 --> 00:22:58,637
answers for reasoning but I think that

398
00:22:58,720 --> 00:23:02,077
the core here is that for the first time

399
00:23:02,160 --> 00:23:06,237
it feels that we have a paradigm that
gives us hope that general intelligence

400
00:23:06,320 --> 00:23:10,397
can be achievable and so as soon as you
believe that everything else becomes

401
00:23:10,480 --> 00:23:14,397
comes into focus right if you imagine
that you may be able to and you know

402
00:23:14,480 --> 00:23:18,157
that the timeline I think remains
uncertain um that but I think that that

403
00:23:18,240 --> 00:23:21,357
you know certainly within our lifetimes
and possibly within a a much shorter

404
00:23:21,440 --> 00:23:25,517
period of time than than people would
expect if you can really build the most

405
00:23:25,600 --> 00:23:29,037
transformative technology that will ever
exist, you stop thinking about yourself

406
00:23:29,120 --> 00:23:32,717
so much, right? You start thinking about
just like how do you have a world where

407
00:23:32,800 --> 00:23:35,757
this goes well and that you need to think
about the practicalities of how do

408
00:23:35,840 --> 00:23:39,437
you build an organization and get
together a bunch of people and resources

409
00:23:39,520 --> 00:23:41,837
um and to make sure that people feel uh

410
00:23:41,920 --> 00:23:45,597
motivated and ready to to do it.

411
00:23:45,680 --> 00:23:49,917
But I think that then you start thinking
about well what if we succeed um and how

412
00:23:50,000 --> 00:23:52,797
do we make sure that when we succeed
that the world is actually the place

413
00:23:52,880 --> 00:23:56,397
that that that we want ourselves
to exist then and you know almost in the

414
00:23:56,480 --> 00:24:00,237
the Rousey and Bale sense of the word
and so that's kind of the the the

415
00:24:00,320 --> 00:24:02,237
broader landscape and OpenAI was really

416
00:24:02,320 --> 00:24:05,917
formed in 2015 with that high level

417
00:24:06,000 --> 00:24:09,117
picture of AGI might be possible sooner

418
00:24:09,200 --> 00:24:13,677
than people think and that uh we need
to try to do our best to make sure it's

419
00:24:13,760 --> 00:24:17,197
going to go well and then We spent
the next couple years really trying to

420
00:24:17,280 --> 00:24:20,317
figure out what does that mean? How do
we do it? Um, and you know, I think that

421
00:24:20,400 --> 00:24:25,357
typically with a company, you start out
very small. It's you and a co-founder

422
00:24:25,440 --> 00:24:28,397
and you build a product, you get some
users, you get product market fit, you

423
00:24:28,480 --> 00:24:32,557
know, then at some point you raise some
money, you hire people, you scale, and

424
00:24:32,640 --> 00:24:35,517
then uh, you know, down the road then
the big companies realize you exist and

425
00:24:35,600 --> 00:24:38,077
try to kill you. Um, and for OpenAI it

426
00:24:38,160 --> 00:24:42,877
was basically everything in exactly
the opposite order. Uh, [laughter]

427
00:24:42,960 --> 00:24:46,397
>> let me just pause for a second. and you
said a lot of things and let me just

428
00:24:46,480 --> 00:24:49,277
admire the jarring aspect of what open

429
00:24:49,360 --> 00:24:53,597
AAI stands for which is uh daring
to dream. I mean you said it it's pretty

430
00:24:53,680 --> 00:24:55,597
powerful. It caught me off guard because

431
00:24:55,680 --> 00:24:59,757
I think that's very true. The the the

432
00:24:59,840 --> 00:25:03,917
step of just daring to dream about
the possibilities of creating intelligence

433
00:25:04,000 --> 00:25:06,397
in a positive in a safe way. But just

434
00:25:06,480 --> 00:25:09,357
even creating intelligence is a a much

435
00:25:09,440 --> 00:25:12,717
needed refreshing

436
00:25:12,800 --> 00:25:14,877
uh catalyst for the AI community. So

437
00:25:14,960 --> 00:25:16,397
that's that's the starting point. Okay.

438
00:25:16,480 --> 00:25:19,837
So then formation of open AI uh what's

439
00:25:19,920 --> 00:25:21,197
>> just I would just
say that you know when

440
00:25:21,280 --> 00:25:25,437
we were starting open AI uh that kind
of the first question that we had is is it

441
00:25:25,520 --> 00:25:29,197
too late to start a lab
with a bunch of the best people

442
00:25:29,280 --> 00:25:30,477
>> right? Is that even possible?

443
00:25:30,560 --> 00:25:33,037
>> That was an actual question. That
was that was really that was that was the

444
00:25:33,120 --> 00:25:36,717
core question of uh of you know we
had this dinner in July of 20 2015 and

445
00:25:36,800 --> 00:25:39,677
there's that was that was really what
we spent the whole time talking about

446
00:25:39,760 --> 00:25:42,717
>> and uh you know cuz
it's you think about

447
00:25:42,800 --> 00:25:44,797
kind of where AI was is that it had

448
00:25:44,880 --> 00:25:49,757
transitioned from being an academic
pursuit to an industrial pursuit and so

449
00:25:49,840 --> 00:25:53,437
a lot of the best people were in these
big research labs and that we wanted to

450
00:25:53,520 --> 00:25:56,877
start our own one that you know no
matter how much resources we could

451
00:25:56,960 --> 00:26:00,877
accumulate would be you know pale in
comparison to the big tech companies and

452
00:26:00,960 --> 00:26:04,957
we knew at and there's a question of are
we going to be actually able to get this

453
00:26:05,040 --> 00:26:08,157
thing off the ground? You need critical
mass. You can't just do you and a

454
00:26:08,240 --> 00:26:11,357
co-founder build a product, right?
You really need to have a group of, you

455
00:26:11,440 --> 00:26:15,837
know, five to 10 people. And uh >> we
kind of concluded it wasn't obviously

456
00:26:15,920 --> 00:26:19,517
impossible. Uh so it seemed worth trying.

457
00:26:19,600 --> 00:26:22,637
>> Well, you're also dreamers,
so who knows, right? That's right.

458
00:26:22,720 --> 00:26:25,517
>> Okay. So, so speaking of that competing

459
00:26:25,600 --> 00:26:28,717
with with the with the big players, um

460
00:26:28,800 --> 00:26:30,557
let's talk about some of the some of the

461
00:26:30,640 --> 00:26:32,797
tricky things as you think through this

462
00:26:32,880 --> 00:26:36,237
process of growing of uh seeing how you

463
00:26:36,320 --> 00:26:38,557
can develop these systems at a scale

464
00:26:38,640 --> 00:26:41,117
that competes. So, you recent recently

465
00:26:41,200 --> 00:26:45,677
formed OpenAI LP, a new cap profit

466
00:26:45,760 --> 00:26:47,597
company that now carries the name

467
00:26:47,680 --> 00:26:49,597
OpenAI. So, Open AI is now this official

468
00:26:49,680 --> 00:26:52,877
company. The original nonprofit company

469
00:26:52,960 --> 00:26:55,517
still exists and carries the OpenAI

470
00:26:55,600 --> 00:26:58,077
nonprofit name. So can you explain what

471
00:26:58,160 --> 00:27:00,237
this company is, what the purpose of its

472
00:27:00,320 --> 00:27:05,117
creation is, and how did
you arrive at the decision?

473
00:27:05,200 --> 00:27:09,277
>> Yep. >> To create it. >> OpenAI,
the whole entity and OpenAI LP

474
00:27:09,360 --> 00:27:14,077
as a vehicle uh is trying to accomplish
the mission of ensuring that artificial

475
00:27:14,160 --> 00:27:15,997
general intelligence benefits everyone.

476
00:27:16,080 --> 00:27:19,037
And the main way that we're trying
to do that is by actually trying to build

477
00:27:19,120 --> 00:27:22,317
general intelligence ourselves and make
sure the benefits are distributed to the

478
00:27:22,400 --> 00:27:26,557
world. That's the primary way.
We're also fine if someone else does this,

479
00:27:26,640 --> 00:27:30,317
right? Doesn't have to be us. If someone
else is going to build an AGI and make

480
00:27:30,400 --> 00:27:33,597
sure that the benefits don't get locked
up in one company or, you know, one one

481
00:27:33,680 --> 00:27:36,717
one with one set of people, um, like

482
00:27:36,800 --> 00:27:39,277
we're actually fine with that. And so

483
00:27:39,360 --> 00:27:42,637
those ideas are baked into our charter

484
00:27:42,720 --> 00:27:46,477
which is kind of the the the
foundational document that are that

485
00:27:46,560 --> 00:27:48,157
describes kind of our values and and how

486
00:27:48,240 --> 00:27:53,517
how we operate. Um but it's also really
baked into the structure of open ILP.

487
00:27:53,600 --> 00:27:54,717
And so the way that we've set up open

488
00:27:54,800 --> 00:27:59,117
ILP is that in the case where we succeed

489
00:27:59,200 --> 00:28:02,637
right if we actually build
what we're trying to build

490
00:28:02,720 --> 00:28:06,877
then investors are able to get a return
um and but that return is something that

491
00:28:06,960 --> 00:28:10,637
is capped. And so if you think of AGI
in terms of the the value that you could

492
00:28:10,720 --> 00:28:13,837
really create, you're talking about
the most transformative technology ever

493
00:28:13,920 --> 00:28:15,677
created. It's going to create orders of

494
00:28:15,760 --> 00:28:18,157
magnitude more value than any existing

495
00:28:18,240 --> 00:28:21,357
company. And that all of that value will

496
00:28:21,440 --> 00:28:23,757
be owned by the world like legally

497
00:28:23,840 --> 00:28:28,637
titled to the nonprofit to fulfill
that mission. And so that's that's the

498
00:28:28,720 --> 00:28:32,637
structure. >> So the mission
is a powerful one and

499
00:28:32,720 --> 00:28:34,877
it's um it's one that I think most

500
00:28:34,960 --> 00:28:37,517
people would agree with. It's how we

501
00:28:37,600 --> 00:28:42,317
would hope AI progresses. And so,
how do you tie yourself to that

502
00:28:42,400 --> 00:28:44,397
mission? How do you make sure you do not

503
00:28:44,480 --> 00:28:48,557
deviate from that mission that um you

504
00:28:48,640 --> 00:28:53,597
know, other incentives that are profit
driven would don't interfere with the

505
00:28:53,680 --> 00:28:55,837
mission. >> So,
this was actually a really core

506
00:28:55,920 --> 00:28:59,437
question for us uh for the past couple
years because, you know, I'd say that

507
00:28:59,520 --> 00:29:02,637
like the way that our history went was
that for the first year we were getting

508
00:29:02,720 --> 00:29:06,477
off the ground, right? we had this
highle picture uh but we didn't know

509
00:29:06,560 --> 00:29:11,117
exactly how we wanted to accomplish it
>> and uh really two years ago is when we

510
00:29:11,200 --> 00:29:14,477
first started realizing in order
to build AGI we're just going to need need

511
00:29:14,560 --> 00:29:18,157
to raise way more money uh than we can
as a nonprofit um and you know we're

512
00:29:18,240 --> 00:29:23,357
talking many billions of dollars >>
and so the first question is how are you

513
00:29:23,440 --> 00:29:26,717
supposed to do that and stay true
to this mission and we looked at every

514
00:29:26,800 --> 00:29:29,517
legal structure out there and concluded
none of them were quite right for what

515
00:29:29,600 --> 00:29:32,317
we wanted to do and I guess it shouldn't
be too surprising if you're going to do

516
00:29:32,400 --> 00:29:35,277
some like crazy unpreced technology
that you're going to have to come up with

517
00:29:35,360 --> 00:29:36,717
some crazy unprecedented structure to do

518
00:29:36,800 --> 00:29:39,597
it in. And uh a lot of a lot of our

519
00:29:39,680 --> 00:29:44,237
conversation was uh with people at OpenAI,
right? The people who really

520
00:29:44,320 --> 00:29:47,917
join because they believe so much
in this mission and thinking about how do

521
00:29:48,000 --> 00:29:52,637
we actually raise the resources to do
it and also stay true to to what we stand

522
00:29:52,720 --> 00:29:55,997
for. And the place you got to start
is to really align on what is it that we

523
00:29:56,080 --> 00:29:57,837
stand for, right? What are those values?

524
00:29:57,920 --> 00:30:01,117
What's really important to us?
And so I'd say that we spent about a year

525
00:30:01,200 --> 00:30:05,517
really uh compiling the opening charter
and that that determines and if you even

526
00:30:05,600 --> 00:30:08,557
look at the first the first line item
in there, it says that look, we expect

527
00:30:08,640 --> 00:30:12,077
we're going to have to marshall huge
amounts of resources, but we're going to

528
00:30:12,160 --> 00:30:14,877
make sure that we minimize conflicts
of interest with the mission.

529
00:30:14,960 --> 00:30:17,037
>> Um and that kind of aligning on all of

530
00:30:17,120 --> 00:30:19,757
those pieces was the most important step

531
00:30:19,840 --> 00:30:22,557
towards figuring out how do we structure

532
00:30:22,640 --> 00:30:24,797
a company that can actually raise the

533
00:30:24,880 --> 00:30:27,837
resources to do what we need to do. I

534
00:30:27,920 --> 00:30:30,637
imagine open AAI uh the decision to

535
00:30:30,720 --> 00:30:33,117
create open ILP was a really difficult

536
00:30:33,200 --> 00:30:35,117
one and there was a lot of discussions

537
00:30:35,200 --> 00:30:38,397
as you mentioned for a year and uh there

538
00:30:38,480 --> 00:30:41,117
was different ideas perhaps detractors

539
00:30:41,200 --> 00:30:46,477
within open AI uh sort of uh different
paths that you could have taken what

540
00:30:46,560 --> 00:30:49,837
were those concerns what were the
different paths considered what was that

541
00:30:49,920 --> 00:30:53,197
process of making that decision like
>> yep um but so if you look actually at

542
00:30:53,280 --> 00:30:55,837
the openi charter that there's almost

543
00:30:55,920 --> 00:30:58,237
two paths embeded within it

544
00:30:58,320 --> 00:31:03,037
>> there is we are primarily trying
to build AGI ourselves but we're also okay

545
00:31:03,120 --> 00:31:06,317
if someone else does it and this
is a weird thing for a company

546
00:31:06,400 --> 00:31:10,477
>> that's really interesting actually >>
there there is an element of competition

547
00:31:10,560 --> 00:31:15,357
that you do want to be the one that does
it but at the same time you're okay if

548
00:31:15,440 --> 00:31:18,317
somebody else does it and we'll talk
about that a little bit that tradeoff

549
00:31:18,400 --> 00:31:20,157
that's that dance
that's really interesting

550
00:31:20,240 --> 00:31:23,997
>> and I think this was the core tension
as we were designing OpenAI LP and really

551
00:31:24,080 --> 00:31:26,717
the OpenAI strategy is how do you make

552
00:31:26,800 --> 00:31:28,797
that both you have a shot at being a

553
00:31:28,880 --> 00:31:31,437
primary actor which really requires

554
00:31:31,520 --> 00:31:36,397
building an organization raising massive
resources and really having the will to

555
00:31:36,480 --> 00:31:39,837
go and execute on some really really
hard vision right you need to really

556
00:31:39,920 --> 00:31:41,997
sign up for a long period to go and take

557
00:31:42,080 --> 00:31:44,397
on a lot of pain and a lot of risk. Um

558
00:31:44,480 --> 00:31:49,437
and to do that normally you just import
the startup mindset right and that you

559
00:31:49,520 --> 00:31:52,397
think about okay like how do we out
execute everyone you you have this very

560
00:31:52,480 --> 00:31:56,237
competitive angle but you also have
this second angle of saying that well the

561
00:31:56,320 --> 00:32:00,717
true mission isn't for open AI to build
AGI the true mission is for AGI to go

562
00:32:00,800 --> 00:32:04,397
well for humanity >> and so
how do you take all of those

563
00:32:04,480 --> 00:32:09,117
first actions and make sure you don't
close the door on outcomes that would

564
00:32:09,200 --> 00:32:12,877
actually be positive and and fulfill
the mission and so I think it's a very

565
00:32:12,960 --> 00:32:17,037
delicate balance Right. And I think that
going 100% one direction or the other is

566
00:32:17,120 --> 00:32:20,797
clearly not the correct answer. And so
I think that even in terms of just how we

567
00:32:20,880 --> 00:32:24,637
talk about OpenAI and think about it,
there's just like like one thing that's

568
00:32:24,720 --> 00:32:29,117
always in the back of my mind is to make
sure that we're not just saying OpenAI's

569
00:32:29,200 --> 00:32:32,877
goal is to build AGI, right? That it's
actually much broader than that, right?

570
00:32:32,960 --> 00:32:37,037
That first of all, you know, it's not
just AGI, it's safe AGI. That's very

571
00:32:37,120 --> 00:32:40,637
important. But secondly, our goal isn't
to be the ones to build it. Our goal is

572
00:32:40,720 --> 00:32:41,997
to make sure it goes well for the world.

573
00:32:42,080 --> 00:32:45,757
And so I think that figuring out how
do you balance all of those um and to to

574
00:32:45,840 --> 00:32:47,517
get people to really come to the table

575
00:32:47,600 --> 00:32:50,797
and compile the the like a single

576
00:32:50,880 --> 00:32:54,797
document that that encompasses
all of that wasn't trivial.

577
00:32:54,880 --> 00:32:57,597
>> So part of the challenge
here is uh your

578
00:32:57,680 --> 00:33:00,797
mission is I would say beautiful

579
00:33:00,880 --> 00:33:04,957
empowering and a beacon of hope
for people in the research community and

580
00:33:05,040 --> 00:33:07,037
just people thinking about AI. So your

581
00:33:07,120 --> 00:33:10,637
decisions are scrutinized more than I

582
00:33:10,720 --> 00:33:13,117
think a regular profit-driven company.

583
00:33:13,200 --> 00:33:16,397
Do you feel the burden of this in the
creation of the charter and just in the

584
00:33:16,480 --> 00:33:17,517
way you operate?

585
00:33:17,600 --> 00:33:23,587
>> Yes. [laughter] >> Uh so why
do you uh lean into the burden

586
00:33:23,670 --> 00:33:27,597
[laughter] by creating such a charter?
Why not keep it quiet?

587
00:33:27,680 --> 00:33:31,677
>> I mean it just boils down to the to
the mission, right? Like like I'm here and

588
00:33:31,760 --> 00:33:34,477
everyone else is here because we think
this is the most important mission,

589
00:33:34,560 --> 00:33:37,437
right? Dare to dare to dream, right? So,

590
00:33:37,520 --> 00:33:40,397
uh, do you think you can be good for the

591
00:33:40,480 --> 00:33:42,317
world or create an AGI system that's

592
00:33:42,400 --> 00:33:45,597
good when you're a for-profit company?

593
00:33:45,680 --> 00:33:47,917
From my perspective, I don't understand

594
00:33:48,000 --> 00:33:49,917
why profit

595
00:33:50,000 --> 00:33:53,757
uh interferes with u positive impact on

596
00:33:53,840 --> 00:33:57,197
society. I don't understand uh why uh

597
00:33:57,280 --> 00:33:59,517
Google that makes most of its money from

598
00:33:59,600 --> 00:34:02,797
ads can't also do good for the world or

599
00:34:02,880 --> 00:34:06,637
other companies, Facebook, anything.
I don't I don't understand why those have

600
00:34:06,720 --> 00:34:09,197
to interfere. You know, you can um

601
00:34:09,280 --> 00:34:12,637
profit isn't the thing in my view that

602
00:34:12,720 --> 00:34:14,717
affects the impact of a company. what

603
00:34:14,800 --> 00:34:17,197
affects the impact of the company is the

604
00:34:17,280 --> 00:34:20,317
charter is the culture is the you know

605
00:34:20,400 --> 00:34:24,797
the people inside and profit is the
thing that just fuels those people. So

606
00:34:24,880 --> 00:34:27,437
what are your views there? >> Yeah.
So I I think that's a it's a

607
00:34:27,520 --> 00:34:30,557
really good question and there's there's
there's some some you know real like

608
00:34:30,640 --> 00:34:33,677
long long-standing debates in human
society that are wrapped up in it. Uh

609
00:34:33,760 --> 00:34:35,517
the way that I think about it is just

610
00:34:35,600 --> 00:34:37,117
think about what what are the most

611
00:34:37,200 --> 00:34:41,197
impactful nonprofits in the world?

612
00:34:41,280 --> 00:34:45,197
What are the most impactful for-profits
in the world? Right. It's much easier to

613
00:34:45,280 --> 00:34:47,837
list the forprofits. >> That's right.
And I think that there's

614
00:34:47,920 --> 00:34:52,877
there's some real truth here that the
system that we set up, the system for

615
00:34:52,960 --> 00:34:54,557
kind of how you know today's world is

616
00:34:54,640 --> 00:34:57,037
organized um is one that that really

617
00:34:57,120 --> 00:35:01,597
allows for huge impact. Um and that that
you know kind of part of that is that

618
00:35:01,680 --> 00:35:05,277
you need to be that that you know
for for profofits are are self-sustaining

619
00:35:05,360 --> 00:35:08,877
and able to to kind of you know build
on their own momentum. Um and I think

620
00:35:08,960 --> 00:35:13,357
that's a really powerful thing. It's
something that when it turns out that we

621
00:35:13,440 --> 00:35:16,397
haven't set the guardrails correctly
causes problems, right? Think about

622
00:35:16,480 --> 00:35:20,237
logging companies that go and deforest,
you know, the the rainforest. That's

623
00:35:20,320 --> 00:35:23,677
really bad. We don't want that. Um,
and it's actually really interesting to me

624
00:35:23,760 --> 00:35:27,517
that kind of this this question of how
do you get positive benefits out of a

625
00:35:27,600 --> 00:35:30,637
for-profit company? It's actually
very similar to how do you get positive

626
00:35:30,720 --> 00:35:33,197
benefits out of an AGI? Um, right, that

627
00:35:33,280 --> 00:35:35,197
you have this like very powerful system.

628
00:35:35,280 --> 00:35:39,037
It's more powerful than any human and
is kind of autonomous in some ways. you

629
00:35:39,120 --> 00:35:42,077
know, it's super human in a lot
of axes and somehow you have to set the

630
00:35:42,160 --> 00:35:45,757
guardrails to get good things to happen,
but when you do, the benefits are

631
00:35:45,840 --> 00:35:48,877
massive. And so, I think that that when

632
00:35:48,960 --> 00:35:52,957
when I think about nonprofit versus
for-profit, I think just not enough

633
00:35:53,040 --> 00:35:56,397
happens in nonprofits. They're very pure,
but it's just kind of, you know,

634
00:35:56,480 --> 00:35:59,837
it's just hard to do things there.
Um, in forprofits in some ways, like too

635
00:35:59,920 --> 00:36:02,877
much happens. Um, but, um, if if kind of

636
00:36:02,960 --> 00:36:07,437
shaped in the right way, it can actually
be very positive. And so with open ILP,

637
00:36:07,520 --> 00:36:09,357
we're we're picking a road in between.

638
00:36:09,440 --> 00:36:12,637
Now the thing I think is really
important to recognize is that the way

639
00:36:12,720 --> 00:36:17,357
that we think about open ILP is that
in the world where AGI actually happens,

640
00:36:17,440 --> 00:36:19,357
right? In a world where we are
successful, we build the most

641
00:36:19,440 --> 00:36:22,157
transformative technology ever,
the amount of value we're going to create

642
00:36:22,240 --> 00:36:24,717
will be astronomical. >> Mhm.

643
00:36:24,800 --> 00:36:26,157
>> And so [clears
throat] then in that case

644
00:36:26,240 --> 00:36:30,157
that the that the cap that we have will

645
00:36:30,240 --> 00:36:34,157
be a small fraction of the value we
create. Um, and the amount of value that

646
00:36:34,240 --> 00:36:36,957
goes back to investors and employees
looks pretty similar to what would

647
00:36:37,040 --> 00:36:40,957
happen in a in a pretty
successful startup.

648
00:36:41,040 --> 00:36:43,997
And that's really the case that we're
optimizing for, right? That we're

649
00:36:44,080 --> 00:36:48,157
thinking about in this success case,
making sure that the value we create

650
00:36:48,240 --> 00:36:52,077
doesn't get locked up. And I expect that
in other, you know, for-profit companies

651
00:36:52,160 --> 00:36:56,557
that it's possible to do something like
that. I think it's not obvious how to do

652
00:36:56,640 --> 00:37:00,157
it, right? I think that as a for-profit
company, you have a lot of fiduciary

653
00:37:00,240 --> 00:37:03,437
duty to your shareholders and that there
are certain decisions that you just

654
00:37:03,520 --> 00:37:08,237
cannot make. Um, in our structure, we've
set it up so that we have a fiduciary

655
00:37:08,320 --> 00:37:11,517
duty to the charter, right?
>> That we always get to make the decision

656
00:37:11,600 --> 00:37:14,237
that is right for the charter um rather

657
00:37:14,320 --> 00:37:18,957
than even if it comes at the expense
of our own stakeholders. And uh and so I

658
00:37:19,040 --> 00:37:22,157
think that when I think about what's
really important, it's not really about

659
00:37:22,240 --> 00:37:27,277
nonprofit versus for-profit. It's really
a question of if you build AGI and you

660
00:37:27,360 --> 00:37:32,237
kind of you know humanity is now in
this new age who benefits whose lives are

661
00:37:32,320 --> 00:37:35,517
better. Um and I think that what's
really important is to have an answer

662
00:37:35,600 --> 00:37:39,677
that is everyone. >> Yeah.
Which is one of the core aspects

663
00:37:39,760 --> 00:37:41,677
of the charter. So one concern people

664
00:37:41,760 --> 00:37:44,077
have not just with open AI but with

665
00:37:44,160 --> 00:37:47,437
Google, Facebook, Amazon, anybody

666
00:37:47,520 --> 00:37:50,717
uh really uh that's that's creating

667
00:37:50,800 --> 00:37:53,917
impact at scale is how do we avoid as

668
00:37:54,000 --> 00:37:56,157
your charter says avoid enabling the use

669
00:37:56,240 --> 00:37:59,517
of AI or AGI to unduly concentrate

670
00:37:59,600 --> 00:38:04,957
power? Why would not a company like open
AI keep all the power of an AGI system

671
00:38:05,040 --> 00:38:08,477
to itself? >> The charter.
>> The charter. So, you know, how does the

672
00:38:08,560 --> 00:38:12,717
charter actionalize itself

673
00:38:12,800 --> 00:38:17,037
in uh day-to-day? >> So,
I think that that first to to zoom

674
00:38:17,120 --> 00:38:19,997
out, right? That the way that we
structure the company is so that the the

675
00:38:20,080 --> 00:38:24,397
power for sort of, you know, dictating
the actions that OpenAI takes ultimately

676
00:38:24,480 --> 00:38:28,397
rests with the board, right? The board
of of of the nonprofit. Um, and the

677
00:38:28,480 --> 00:38:30,957
board is set up in certain ways with
certain certain restrictions that you

678
00:38:31,040 --> 00:38:32,797
can read about in the OpenAI LP blog

679
00:38:32,880 --> 00:38:34,797
post. Um but effectively the board is

680
00:38:34,880 --> 00:38:38,477
the is the governing body for open LP.

681
00:38:38,560 --> 00:38:40,957
Um and the board has a duty to fulfill

682
00:38:41,040 --> 00:38:44,477
the mission of the nonprofit. Um and so

683
00:38:44,560 --> 00:38:48,397
that's kind of how we tie how we
thread all these things together. Um now

684
00:38:48,480 --> 00:38:52,557
there's a question of so day-to-day how
do people the individuals who in some

685
00:38:52,640 --> 00:38:55,517
ways are the most empowered ones right
you know the board sort of gets to call

686
00:38:55,600 --> 00:38:59,597
the shots at the high level but the
people who are actually executing are

687
00:38:59,680 --> 00:39:02,957
the employees right the people here on
a day-to-day basis who have the you know

688
00:39:03,040 --> 00:39:06,237
the the keys to the technical kingdom

689
00:39:06,320 --> 00:39:10,637
and there I think that the answer looks
a lot like well how does any company's

690
00:39:10,720 --> 00:39:14,237
values get actualized right I think
that a lot of that comes down to that you

691
00:39:14,320 --> 00:39:18,877
need people who are here because they
really believe in that mission um and

692
00:39:18,960 --> 00:39:22,957
they believe in the charter and that they
are willing to take actions uh that

693
00:39:23,040 --> 00:39:26,557
maybe are worse for them but are better
for the charter um and that's something

694
00:39:26,640 --> 00:39:30,557
that's really baked into the culture
and honestly I think it's uh you know I

695
00:39:30,640 --> 00:39:34,317
think that that's one of the things
that we really have to work to preserve as

696
00:39:34,400 --> 00:39:38,157
time goes on um and that's a really
important part of how we think about

697
00:39:38,240 --> 00:39:39,517
hiring people and bringing people into

698
00:39:39,600 --> 00:39:41,997
open AI. So there's people here there's

699
00:39:42,080 --> 00:39:46,237
people here who could speak up and say

700
00:39:46,320 --> 00:39:50,717
like hold on a second this is
totally against what we stand for

701
00:39:50,800 --> 00:39:53,037
>> uh culture wise. >> Yeah.
Yeah. For sure. I mean I think

702
00:39:53,120 --> 00:39:56,157
that uh that we actually have I think
a that's like a pretty important part of

703
00:39:56,240 --> 00:40:01,277
of how how we operate and how we have
even again with designing the charter

704
00:40:01,360 --> 00:40:05,357
and designing open ALP in the first place.
uh that there has been a lot of

705
00:40:05,440 --> 00:40:09,037
conversation with employees here and a lot
of times where employees said, "Wait

706
00:40:09,120 --> 00:40:11,837
a second, this seems like it's going in
the wrong direction and let's talk about

707
00:40:11,920 --> 00:40:15,117
it." And so I think one thing that's
that's I think a really and you know,

708
00:40:15,200 --> 00:40:18,797
here's here's actually one thing that I
think is very unique about us as a small

709
00:40:18,880 --> 00:40:22,557
company is that if you're at a massive
tech giant, it's a little bit hard for

710
00:40:22,640 --> 00:40:26,717
someone who's a line employee to go
and talk to the CEO and say, "I think that

711
00:40:26,800 --> 00:40:30,237
we're doing this wrong." Um, and you know,
you look at companies like Google

712
00:40:30,320 --> 00:40:34,877
that have had some collective action
from employees to, you know, make uh

713
00:40:34,960 --> 00:40:36,557
ethical change around things like Maven.

714
00:40:36,640 --> 00:40:40,237
Um, and so maybe there are mechanisms at
other companies that work. Um, but here,

715
00:40:40,320 --> 00:40:43,677
super easy for anyone to pull me aside,
to pull Sam aside, to pull aside, and

716
00:40:43,760 --> 00:40:46,317
people do it all the time.
>> One of the interesting things in the

717
00:40:46,400 --> 00:40:51,357
charter is this idea that it'd be great
if you could try to describe or untangle

718
00:40:51,440 --> 00:40:55,357
of switching from competition
to collaboration and latestage AGI

719
00:40:55,440 --> 00:40:56,957
development. It was really interesting.

720
00:40:57,040 --> 00:41:00,157
this dance between competition
and collaboration. How do you think about

721
00:41:00,240 --> 00:41:02,237
that? >> Yeah, assuming
that you can actually do

722
00:41:02,320 --> 00:41:05,597
the technical side of AGI development,
I think there's going to be two key

723
00:41:05,680 --> 00:41:08,877
problems with figuring out how do you
actually deploy it, make it go well. The

724
00:41:08,960 --> 00:41:13,277
first one of these is the runup
to building the first AGI.

725
00:41:13,360 --> 00:41:16,477
>> Um, you look at how self-driving
cars are being developed and it's a

726
00:41:16,560 --> 00:41:19,117
competitive race >> and the
thing that always happens in

727
00:41:19,200 --> 00:41:23,917
competitive race is that you have huge
amounts of pressure to get rid of safety.

728
00:41:24,000 --> 00:41:27,277
And so that's one thing we're very
concerned about, right? Is that people

729
00:41:27,360 --> 00:41:29,597
multiple teams figuring out we can

730
00:41:29,680 --> 00:41:32,797
actually get there, but you know, if we

731
00:41:32,880 --> 00:41:37,757
took the slower path that is more
guaranteed to be safe, we will lose. And

732
00:41:37,840 --> 00:41:39,597
so we're going to take the fast path.

733
00:41:39,680 --> 00:41:43,917
And so the more that we can both
ourselves be in a position where we

734
00:41:44,000 --> 00:41:47,437
don't generate that competitive race
where we say if the race is being run

735
00:41:47,520 --> 00:41:50,957
and that, you know, someone else
is is further ahead than we are, we're not

736
00:41:51,040 --> 00:41:54,797
going to try to to leaprog. we're gonna
actually work with them, right? We will

737
00:41:54,880 --> 00:41:58,557
help them succeed. As long as what
they're trying to do is to fulfill our

738
00:41:58,640 --> 00:42:02,477
mission, then we're good. We don't
have to build AGI ourselves. And I think

739
00:42:02,560 --> 00:42:05,277
that's a really important commitment
from us. But it can't just be

740
00:42:05,360 --> 00:42:08,797
unilateral, right? I think that it's
really important that other players who

741
00:42:08,880 --> 00:42:12,237
are serious about building AGI make
similar commitments, right? And I think

742
00:42:12,320 --> 00:42:15,677
that uh that you know, again, to the
extent that everyone believes that AGI

743
00:42:15,760 --> 00:42:18,477
should be something to benefit everyone,
then it actually really shouldn't matter

744
00:42:18,560 --> 00:42:21,997
which company builds it. And we should
all be concerned about the case where we

745
00:42:22,080 --> 00:42:24,797
just race. so hard to get
there that something goes wrong.

746
00:42:24,880 --> 00:42:27,757
>> So what role do you think government

747
00:42:27,840 --> 00:42:30,157
our favorite entity has in setting

748
00:42:30,240 --> 00:42:33,837
policy and rules about this domain from

749
00:42:33,920 --> 00:42:39,597
research to the development to early
stage to late stage a AI and AGI

750
00:42:39,680 --> 00:42:42,797
development. >> So I think
that that first of all um

751
00:42:42,880 --> 00:42:46,237
it's really important the government's
in there right in some way shape or

752
00:42:46,320 --> 00:42:48,877
form. you know, at the end of the day,
we're talking about building technology

753
00:42:48,960 --> 00:42:52,317
that will shape how the world operates

754
00:42:52,400 --> 00:42:54,717
and that there needs to be government as

755
00:42:54,800 --> 00:42:59,437
part of that answer. And so that's why
we've uh we've we've done a number of

756
00:42:59,520 --> 00:43:02,557
different congressional testimonies.
We uh interact with a number of different

757
00:43:02,640 --> 00:43:05,357
lawmakers and uh that you know right now

758
00:43:05,440 --> 00:43:08,317
a lot of our message to them is that

759
00:43:08,400 --> 00:43:11,837
it's not the time for regulation, it is

760
00:43:11,920 --> 00:43:15,757
the time for measurement, right? that
our main policy recommendation is that

761
00:43:15,840 --> 00:43:19,837
people and you know the government does
this all the time with bodies like NIST

762
00:43:19,920 --> 00:43:22,717
um spend time trying to figure out just

763
00:43:22,800 --> 00:43:27,037
where the technology is, how fast
it's moving um and can really uh become

764
00:43:27,120 --> 00:43:31,677
literate and up to speed with respect to
what to expect. Um so I think that today

765
00:43:31,760 --> 00:43:35,437
the answer really is about about about
measurement. Um and I think that there

766
00:43:35,520 --> 00:43:38,157
will be a time uh and place where that

767
00:43:38,240 --> 00:43:43,037
will change and uh I think it's a little
bit hard to predict exactly uh what what

768
00:43:43,120 --> 00:43:44,557
exactly that trajectory should look like.

769
00:43:44,640 --> 00:43:48,557
>> So there will be
a point at which regulation

770
00:43:48,640 --> 00:43:50,477
federal in the United States the

771
00:43:50,560 --> 00:43:54,717
government steps in and and helps be the

772
00:43:54,800 --> 00:43:58,477
I I don't want to say the adult in the
room to make sure that there is strict

773
00:43:58,560 --> 00:44:02,797
rules maybe conservative rules
that nobody can cross. Uh well I think

774
00:44:02,880 --> 00:44:06,157
there's there's kind of maybe two two
angles to it. So today with narrow AI

775
00:44:06,240 --> 00:44:09,837
applications uh that I think there
are already existing bodies that are

776
00:44:09,920 --> 00:44:12,797
responsible and should be responsible
for regulation. You think about for

777
00:44:12,880 --> 00:44:16,157
example with self-driving cars that you
want the uh you know the national

778
00:44:16,240 --> 00:44:21,437
highway uh exactly to to be regulating
that. That makes sense, right? That

779
00:44:21,520 --> 00:44:24,797
basically what we're saying is that we're
going to have these technological

780
00:44:24,880 --> 00:44:28,317
systems that are going to be do
performing applications that humans

781
00:44:28,400 --> 00:44:32,477
already do. Great. we already have ways
of thinking about standards and safety

782
00:44:32,560 --> 00:44:36,317
for those. Um, so I think actually
empowering those regulators today is

783
00:44:36,400 --> 00:44:41,197
also pretty important. And then I think
for for AGI, you know, that there's

784
00:44:41,280 --> 00:44:43,997
going to be a point where we'll have
better answers. And I think that maybe a

785
00:44:44,080 --> 00:44:47,837
similar approach of first measurement and,
you know, start thinking about what

786
00:44:47,920 --> 00:44:51,837
the rules should be. I think it's really
important that we don't prematurely

787
00:44:51,920 --> 00:44:56,477
squash, you know, progress. Like I think
it's very easy to kind of smother the a

788
00:44:56,560 --> 00:44:59,597
budding field and I think that's
something to to really avoid. But I

789
00:44:59,680 --> 00:45:01,197
don't think the right way of doing it is

790
00:45:01,280 --> 00:45:04,477
to say let's just try to blaze ahead and

791
00:45:04,560 --> 00:45:06,797
not involve all these other

792
00:45:06,880 --> 00:45:08,797
stakeholders.

793
00:45:08,880 --> 00:45:12,477
>> So uh you recently released a paper on

794
00:45:12,560 --> 00:45:15,837
GPT2 language modeling uh >> Mhm.

795
00:45:15,920 --> 00:45:19,277
>> Uh but uh did not
release the full model

796
00:45:19,360 --> 00:45:21,277
because you had concerns about the

797
00:45:21,360 --> 00:45:23,117
possible negative effects of the

798
00:45:23,200 --> 00:45:26,397
availability of such model. It's uh

799
00:45:26,480 --> 00:45:30,397
outside of just that decision,
it's super interesting because of uh the

800
00:45:30,480 --> 00:45:33,197
discussion as at a societal level, the

801
00:45:33,280 --> 00:45:37,677
discourse it creates. So it's
fascinating in that aspect. But if you

802
00:45:37,760 --> 00:45:41,597
think that's the specifics here at first,
what are some negative effects

803
00:45:41,680 --> 00:45:45,837
that you envisioned and of course
what are some of the positive effects?

804
00:45:45,920 --> 00:45:48,077
>> Yeah. So again I think to zoom out like

805
00:45:48,160 --> 00:45:51,517
the way that we thought about GPT2 is

806
00:45:51,600 --> 00:45:53,517
that with language modeling we are

807
00:45:53,600 --> 00:45:56,397
clearly on a trajectory right now where

808
00:45:56,480 --> 00:45:59,677
we scale up our models and we get

809
00:45:59,760 --> 00:46:04,557
qualitatively better performance right
gpt2 itself was actually just a scale up

810
00:46:04,640 --> 00:46:08,477
of a model that we've released pre
in the previous June right and we just ran

811
00:46:08,560 --> 00:46:11,997
it at you know much larger scale
and we got these results where suddenly

812
00:46:12,080 --> 00:46:14,637
starting to write coherent pros uh which

813
00:46:14,720 --> 00:46:17,277
was not something we'd seen previously

814
00:46:17,360 --> 00:46:21,837
and what are we doing now? Well, we're
going to scale up GPD2 by 10x, by 100x,

815
00:46:21,920 --> 00:46:25,837
by thousandx and we don't know what
we're going to get. And so, it's very

816
00:46:25,920 --> 00:46:30,317
clear that the model that we that we
released last June. You know, I think

817
00:46:30,400 --> 00:46:34,397
it's kind of like it's it's it's it's
a good academic toy. It's not something

818
00:46:34,480 --> 00:46:37,997
that we think is something that can
really have negative implications or you

819
00:46:38,080 --> 00:46:40,797
know to the extent that it can
the positive of people being able to play

820
00:46:40,880 --> 00:46:43,517
with it is is you know far far outweighs

821
00:46:43,600 --> 00:46:49,357
the the the possible harms.
You fast forward to not GPT2 but GPT20,

822
00:46:49,440 --> 00:46:51,037
>> right? >> And you think
about what that's going to

823
00:46:51,120 --> 00:46:54,477
be like and I think that the
capabilities are going to be

824
00:46:54,560 --> 00:46:56,557
substantive. And so there needs to be a

825
00:46:56,640 --> 00:47:01,517
point in between the two where you say
this is something where we are drawing

826
00:47:01,600 --> 00:47:05,517
the line um and that we need to start
thinking about the safety aspects. And I

827
00:47:05,600 --> 00:47:08,397
think for GPT2, we could have gone
either way. And in fact, when we had

828
00:47:08,480 --> 00:47:12,557
conversations internally, uh that we
had a bunch of pros and cons. Um and it

829
00:47:12,640 --> 00:47:16,077
wasn't clear which one which one
outweighed the other. Um and I think

830
00:47:16,160 --> 00:47:19,357
that when we announced that, hey,
we decide not to release this model. Um

831
00:47:19,440 --> 00:47:22,077
then there was a bunch of conversation
where various people said it's so

832
00:47:22,160 --> 00:47:24,237
obvious that you should have just
released it. There other people said

833
00:47:24,320 --> 00:47:27,117
it's so obvious you should not have
released it. And I think that that

834
00:47:27,200 --> 00:47:31,037
almost definitionally means that holding
it back was the correct decision. Right?

835
00:47:31,120 --> 00:47:34,797
if it's if there's if it's not obvious
whether something is beneficial or not,

836
00:47:34,880 --> 00:47:36,957
you should probably default to caution.

837
00:47:37,040 --> 00:47:40,477
And so I think that the that that the
overall landscape for how we think about

838
00:47:40,560 --> 00:47:44,237
it is that this decision could have gone
either way. There's great arguments in

839
00:47:44,320 --> 00:47:48,397
both directions, but for future models
down the road um and possibly sooner

840
00:47:48,480 --> 00:47:51,037
than than than you'd expect because,
you know, scaling these things up doesn't

841
00:47:51,120 --> 00:47:55,277
actually take that long, those ones
you're definitely not going to want to

842
00:47:55,360 --> 00:47:57,997
to release into the wild. And so um I

843
00:47:58,080 --> 00:48:02,317
think that we we almost view this as a
test case and to see can we even design

844
00:48:02,400 --> 00:48:05,517
you know how how do you have a society
where how do you have a system that goes

845
00:48:05,600 --> 00:48:10,077
from having no concept of responsible
disclosure where the mere idea of not

846
00:48:10,160 --> 00:48:14,237
releasing something for safety reasons
is unfamiliar to a world where you say

847
00:48:14,320 --> 00:48:17,437
okay we have a powerful model let's
at least think about it let's go through

848
00:48:17,520 --> 00:48:20,797
some process and you think about
the security community it took them a long

849
00:48:20,880 --> 00:48:23,837
time to design responsible disclosure
right you know you think about this

850
00:48:23,920 --> 00:48:27,277
question of well I have a security
exploit I send it to the company the

851
00:48:27,360 --> 00:48:29,517
company's like tries to prosecute me or

852
00:48:29,600 --> 00:48:34,397
just sit just ignores it what do I do
right and so you know the alternatives

853
00:48:34,480 --> 00:48:37,277
of oh I just just always publish your
exploits that doesn't seem good either

854
00:48:37,360 --> 00:48:41,597
right and so it really took a long time
and took this this uh it was bigger than

855
00:48:41,680 --> 00:48:44,717
any individual right it's really about
building a whole community that believe

856
00:48:44,800 --> 00:48:47,597
that okay we'll have this process where
you send it to the company you know if

857
00:48:47,680 --> 00:48:51,357
they don't act in a certain time then
you can go public and you're not a bad

858
00:48:51,440 --> 00:48:53,357
person you've done the right thing um

859
00:48:53,440 --> 00:48:56,477
and I think that in AI part of the

860
00:48:56,560 --> 00:48:59,037
response to GBD2 just proves that we

861
00:48:59,120 --> 00:49:02,557
don't have any concept of this. Um, so

862
00:49:02,640 --> 00:49:07,037
that's the highle picture. Um, and so
I think that I think this was this was a

863
00:49:07,120 --> 00:49:11,117
really important move to make. Um, and we
could have maybe delayed it for GPT3,

864
00:49:11,200 --> 00:49:13,197
but I'm really glad we did it for GPT2.

865
00:49:13,280 --> 00:49:16,637
And so now you look at GPT2 itself
and you think about the substance of okay,

866
00:49:16,720 --> 00:49:19,997
what are potential negative applications?
So you have this model

867
00:49:20,080 --> 00:49:22,717
that's been trained on the internet
which you know is also going to be a

868
00:49:22,800 --> 00:49:24,477
bunch of very biased data a bunch of you

869
00:49:24,560 --> 00:49:26,477
know very uh offensive content in there

870
00:49:26,560 --> 00:49:31,677
uh and uh you can ask it to generate
content for you on basically any topic

871
00:49:31,760 --> 00:49:34,317
right you just give it a prompt
and it'll just start start writing and it

872
00:49:34,400 --> 00:49:37,517
writes content like you see on the
internet you know even down to like

873
00:49:37,600 --> 00:49:41,997
saying advertisement [laughter] in the
middle of of some of its generations and

874
00:49:42,080 --> 00:49:46,397
uh you think about the possibilities for
generating fake news or abusive content

875
00:49:46,480 --> 00:49:49,117
>> and you know it's interesting seeing
what people have done with uh you know

876
00:49:49,200 --> 00:49:51,757
we released a smaller version of GPT2

877
00:49:51,840 --> 00:49:56,317
and uh that people have done things like
try to generate uh you know to take my

878
00:49:56,400 --> 00:50:00,477
own Facebook message history and generate
more Facebook messages like me

879
00:50:00,560 --> 00:50:05,357
um and uh people generating fake
politician uh uh content or uh you know

880
00:50:05,440 --> 00:50:09,597
there there's a bunch of of things there
where you at least have to think is this

881
00:50:09,680 --> 00:50:11,997
going to be good for the world.

882
00:50:12,080 --> 00:50:14,557
There's the flip side, which is I
think that there's a lot of awesome

883
00:50:14,640 --> 00:50:16,717
applications that we really want to see

884
00:50:16,800 --> 00:50:21,757
like creative uh applications in terms
of if you have sci-fi authors that can

885
00:50:21,840 --> 00:50:25,597
work with this tool and come up with
cool ideas like that seems that seems

886
00:50:25,680 --> 00:50:28,717
awesome if we can write better sci-fi
through the use of of these tools. And

887
00:50:28,800 --> 00:50:31,997
we've actually had a bunch of people
write into us asking, hey, can we use it

888
00:50:32,080 --> 00:50:35,677
for you know a variety
of different creative applications?

889
00:50:35,760 --> 00:50:37,837
>> So the positive are
actually pretty easy

890
00:50:37,920 --> 00:50:41,357
to imagine. uh there, [snorts]

891
00:50:41,440 --> 00:50:44,157
you know, the the usual NLP applications

892
00:50:44,240 --> 00:50:47,917
are really interesting, but let let's go

893
00:50:48,000 --> 00:50:49,357
there. It's kind of interesting to think

894
00:50:49,440 --> 00:50:53,517
about a world where uh look at Twitter

895
00:50:53,600 --> 00:50:56,477
where not just fake news but smarter and

896
00:50:56,560 --> 00:51:00,477
smarter bots being able to uh spread in

897
00:51:00,560 --> 00:51:03,357
an interesting complex networking way in

898
00:51:03,440 --> 00:51:06,317
information that just floods out us

899
00:51:06,400 --> 00:51:09,037
regular human beings with our original

900
00:51:09,120 --> 00:51:12,557
thoughts. So uh what are your views of

901
00:51:12,640 --> 00:51:16,637
this world with GPT20 right?

902
00:51:16,720 --> 00:51:19,837
>> Mhm. What do you how do we think about
it? Again, it's like one of those things

903
00:51:19,920 --> 00:51:23,277
about in the 50s trying to describe the

904
00:51:23,360 --> 00:51:26,077
uh the internet or the smartphone. What

905
00:51:26,160 --> 00:51:27,357
do you think about that world, the

906
00:51:27,440 --> 00:51:30,557
nature of information? Do we uh one

907
00:51:30,640 --> 00:51:33,197
possibility is that we'll always try to

908
00:51:33,280 --> 00:51:35,597
design systems that identify robot

909
00:51:35,680 --> 00:51:38,477
versus human and will do so successfully

910
00:51:38,560 --> 00:51:42,957
and so we will authenticate that we're
still human. And the other world is that

911
00:51:43,040 --> 00:51:48,077
we just accept the the fact that we're
swimming in a sea of fake news and just

912
00:51:48,160 --> 00:51:51,037
learn to swim there. >> Well,
have have you ever seen the uh

913
00:51:51,120 --> 00:51:54,077
there there's a you know popular uh meme

914
00:51:54,160 --> 00:51:59,037
of uh of uh robot uh with with a
physical physical arm and pen clicking

915
00:51:59,120 --> 00:52:02,477
the I'm not a robot button.
>> Yeah. [laughter]

916
00:52:02,560 --> 00:52:05,917
>> I think I think that the truth is
that uh that really trying to distinguish

917
00:52:06,000 --> 00:52:09,437
between robot and human
is a losing battle.

918
00:52:09,520 --> 00:52:12,077
>> Ultimately, you think it's a losing
battle? I think it's a losing battle

919
00:52:12,160 --> 00:52:15,197
ultimately, right? I think that that
is that in terms of of the content, in

920
00:52:15,280 --> 00:52:16,557
terms of the actions that you can take.

921
00:52:16,640 --> 00:52:19,357
I mean, think about how captures have
gone, right? The captures used to be a

922
00:52:19,440 --> 00:52:23,757
very nice simple. You just have this
image, all of our OCR is terrible. You

923
00:52:23,840 --> 00:52:26,957
put a couple of of artifacts in it,
you know, humans are going to be able to

924
00:52:27,040 --> 00:52:31,517
tell what what what it is. An AI system
wouldn't be able to today. Like, I could

925
00:52:31,600 --> 00:52:34,877
barely do captures. >> Yeah.
>> And I think that that that this is just

926
00:52:34,960 --> 00:52:37,357
kind of where we're going. I think
captures were were a moment in time

927
00:52:37,440 --> 00:52:41,357
thing. And as AI systems become
more powerful that there being human

928
00:52:41,440 --> 00:52:46,157
capabilities that can be measured in a
very easy automated way uh that that AIs

929
00:52:46,240 --> 00:52:49,597
will not be capable of. I think that's
just like it's just an increasingly hard

930
00:52:49,680 --> 00:52:52,397
technical battle. But it's not that all

931
00:52:52,480 --> 00:52:57,037
hope is lost, right? You think about
uh how do we already authenticate

932
00:52:57,120 --> 00:53:00,637
ourselves, right? that you know we have
systems we have social security numbers

933
00:53:00,720 --> 00:53:04,957
if you're in the US or you know you have
you have uh you know ways of identifying

934
00:53:05,040 --> 00:53:08,877
individual people um and having real
world identity tied to to digital

935
00:53:08,960 --> 00:53:13,197
identity seems like a step uh towards
you know authenticating the source of

936
00:53:13,280 --> 00:53:17,437
content rather than the content itself
um now there are problems with that how

937
00:53:17,520 --> 00:53:21,357
can you have privacy and anonymity
in a world where the only content you can

938
00:53:21,440 --> 00:53:25,037
really trust is or the only way you can
trust content is by looking at where it

939
00:53:25,120 --> 00:53:28,797
comes from. Um, and so I think that
building out good reputation networks uh

940
00:53:28,880 --> 00:53:33,117
may be may be one possible solution. But
yeah, I think that this this question is

941
00:53:33,200 --> 00:53:37,437
is not an obvious one. And I think that
we, you know, maybe sooner than we think

942
00:53:37,520 --> 00:53:41,357
we'll be in a world where, you know,
today I often will read a tweet and be

943
00:53:41,440 --> 00:53:44,157
like, hm, do I feel like a real human
wrote this or, you know, do I feel like

944
00:53:44,240 --> 00:53:46,957
this is like genuine? I feel like I can
kind of judge the content a little bit.

945
00:53:47,040 --> 00:53:50,477
Um, and I think in the future it just
won't be the case. You look at, for

946
00:53:50,560 --> 00:53:55,037
example, the FCC comments on net
neutrality. uh it came out later that

947
00:53:55,120 --> 00:53:58,957
millions of those were autogenerated
and that the researchers were able to do

948
00:53:59,040 --> 00:54:03,357
various statistical tech techniques to do
that. What do you do in a world where

949
00:54:03,440 --> 00:54:06,157
those statistical techniques don't exist?
It's just impossible to tell the

950
00:54:06,240 --> 00:54:10,237
difference between humans and AIs.
And in fact, the uh the the the most

951
00:54:10,320 --> 00:54:12,877
persuasive arguments are written by by

952
00:54:12,960 --> 00:54:17,517
AI. All that stuff, it's not sci-fi
anymore. You look at GPT2 making a great

953
00:54:17,600 --> 00:54:20,557
argument for why recycling is bad
for the world. You got to read that and be

954
00:54:20,640 --> 00:54:23,917
like, "Huh, you're right.
We are addressing different symptoms.

955
00:54:24,000 --> 00:54:27,757
>> Yeah, that's that's quite interesting.
I mean, ultimately it boils down to the

956
00:54:27,840 --> 00:54:32,237
physical world being the last frontier
of proving. So you said like basically

957
00:54:32,320 --> 00:54:34,397
networks of people humans vouching for

958
00:54:34,480 --> 00:54:37,917
humans in the physical world and somehow

959
00:54:38,000 --> 00:54:40,477
the authentication uh ends there. I mean

960
00:54:40,560 --> 00:54:43,997
if I had to ask you I mean you're way

961
00:54:44,080 --> 00:54:49,197
too eloquent for a human. So if I had
to ask you to authenticate like prove how

962
00:54:49,280 --> 00:54:50,717
do I know you're not a robot and how do

963
00:54:50,800 --> 00:54:52,397
you know I'm not a robot?

964
00:54:52,480 --> 00:54:56,957
>> I think that's so far we're we're this

965
00:54:57,040 --> 00:54:58,877
in this space this conversation we just

966
00:54:58,960 --> 00:55:01,757
had the physical movements we did is the

967
00:55:01,840 --> 00:55:04,557
biggest gap between us and AI systems is

968
00:55:04,640 --> 00:55:09,037
the physical manipulation. So maybe
that's the last frontier. Well, here's

969
00:55:09,120 --> 00:55:14,317
another question is is you know why why
is why is solving this problem important

970
00:55:14,400 --> 00:55:16,957
right? like what aspects are really
important to us. And I think that

971
00:55:17,040 --> 00:55:21,277
probably where we'll end up is we'll
hone in on what do we really want out of

972
00:55:21,360 --> 00:55:25,917
knowing if we're talking to a human. Um
and uh and I think that again this comes

973
00:55:26,000 --> 00:55:29,437
down to identity. And so I think that
that the internet of the future I expect

974
00:55:29,520 --> 00:55:33,117
to be one that will have lots of agents
out there uh that will interact with

975
00:55:33,200 --> 00:55:37,437
with you. But I think that the question
of is this you know a flesh real flesh

976
00:55:37,520 --> 00:55:41,757
and blood uh human or is this an
automated system uh may may actually

977
00:55:41,840 --> 00:55:43,837
just be less important. Let's actually

978
00:55:43,920 --> 00:55:47,837
go there. It's GPT2 is impressive and

979
00:55:47,920 --> 00:55:51,197
let's look at GPT20. Why is it so bad

980
00:55:51,280 --> 00:55:56,557
that all my friends are GPT20?

981
00:55:56,640 --> 00:55:59,117
Why why is it so why is it so important

982
00:55:59,200 --> 00:56:02,477
on the internet do you think uh to

983
00:56:02,560 --> 00:56:04,797
interact with only human beings? Why

984
00:56:04,880 --> 00:56:06,957
can't we live in a world where ideas can

985
00:56:07,040 --> 00:56:10,157
come from models trained on human data?

986
00:56:10,240 --> 00:56:12,957
Yeah, I I think this is I think this is
actually a really interesting question.

987
00:56:13,040 --> 00:56:15,917
This comes back to the how do you
even picture a world with some new

988
00:56:16,000 --> 00:56:20,157
technology. And I think that that one
thing that I think is important is is

989
00:56:20,240 --> 00:56:24,317
you know, let's say honesty. Um and I
think that if you have, you know, almost

990
00:56:24,400 --> 00:56:29,197
in the the touring test style uh sense
sense of of of technology, you have AIs

991
00:56:29,280 --> 00:56:32,877
that are pretending to be humans
and deceiving you. Uh I think that is, you

992
00:56:32,960 --> 00:56:35,597
know, that that feels like a bad thing,
right? I think that it's really

993
00:56:35,680 --> 00:56:38,717
important that we feel like we're
in control of our environment, right? that

994
00:56:38,800 --> 00:56:43,597
we understand who we're interacting
with and if it's an AI or a human um that

995
00:56:43,680 --> 00:56:46,397
that that's not something that we're
being deceived about. But I think that

996
00:56:46,480 --> 00:56:49,917
the flip side of can I have
as meaningful of an interaction with an AI

997
00:56:50,000 --> 00:56:53,357
as I can with a human. Uh well,
I actually think here you can turn to

998
00:56:53,440 --> 00:56:57,757
sci-fi. Um and her I think is a great
example of asking this very question,

999
00:56:57,840 --> 00:57:00,877
right? And one thing I really love about
her is it really starts out almost by

1000
00:57:00,960 --> 00:57:06,077
asking how meaningful are human virtual
relationships, right? and uh and then

1001
00:57:06,160 --> 00:57:10,397
you have a human who has a relationship
with an AI and uh that you really start

1002
00:57:10,480 --> 00:57:13,597
to be drawn into that, right?
And that all of your emotional buttons get

1003
00:57:13,680 --> 00:57:16,637
triggered in the same way as if there
was a real human that was on the other

1004
00:57:16,720 --> 00:57:20,877
side of that phone. And so I think that
that this is one way of thinking about

1005
00:57:20,960 --> 00:57:25,037
it is that I think that we can have
meaningful interactions and that if

1006
00:57:25,120 --> 00:57:28,637
there's a funny joke some sense it
doesn't really matter if it was written

1007
00:57:28,720 --> 00:57:32,637
by a human or an AI. But what you don't
want and where I think we should really

1008
00:57:32,720 --> 00:57:36,797
draw hard lines is deception. Um and I
think that as long as we're in a world

1009
00:57:36,880 --> 00:57:40,317
where you know why why do we build AI
systems at all? Right? The reason we

1010
00:57:40,400 --> 00:57:43,437
want to build them is to enhance human
lives to make humans be able to do more

1011
00:57:43,520 --> 00:57:47,437
things to have human humans feel
more fulfilled. And if we can build AI

1012
00:57:47,520 --> 00:57:50,557
systems that do that,
uh, you know, sign me up.

1013
00:57:50,640 --> 00:57:54,317
>> So, the process of language modeling,

1014
00:57:54,400 --> 00:57:56,317
how far do you think it takes us? Let's

1015
00:57:56,400 --> 00:58:00,157
look at movie her. Do you think, uh, a

1016
00:58:00,240 --> 00:58:04,157
dialogue, natural language conversation
is formulated by the touring test, for

1017
00:58:04,240 --> 00:58:08,477
example? Do you think that process
could be achieved through this kind of

1018
00:58:08,560 --> 00:58:10,797
unsupervised language modeling? So, I

1019
00:58:10,880 --> 00:58:15,677
think the Turing test in its in its
real form uh isn't just about language,

1020
00:58:15,760 --> 00:58:18,957
right? It's really about reasoning, too,
right? That to really pass the Turing

1021
00:58:19,040 --> 00:58:23,357
test, I should be able to teach calculus
to whoever's on the other side and have

1022
00:58:23,440 --> 00:58:26,797
it really understand calculus and be
able to, you know, go and and solve new

1023
00:58:26,880 --> 00:58:31,677
calculus problems. And so, I think that
to really solve the Turing test, we need

1024
00:58:31,760 --> 00:58:34,637
more than what we're seeing with
language models. We need some way of

1025
00:58:34,720 --> 00:58:38,877
plugging in reasoning. Now,
how different will that be from what we

1026
00:58:38,960 --> 00:58:42,317
already do? That's an open question,
right? It might be that we need some

1027
00:58:42,400 --> 00:58:44,477
sequence of totally radical new ideas.

1028
00:58:44,560 --> 00:58:48,237
Or it might be that we just need to kind
of shape our existing systems in a

1029
00:58:48,320 --> 00:58:52,557
slightly different way. But I think
that in terms of how far language modeling

1030
00:58:52,640 --> 00:58:56,717
will go, it's already gone way further
than many people would have expected,

1031
00:58:56,800 --> 00:58:59,197
right? I think that things like uh
and I think there's a lot of really

1032
00:58:59,280 --> 00:59:01,357
interesting angles to poke in terms of

1033
00:59:01,440 --> 00:59:04,717
how much does GPT2 understand physical

1034
00:59:04,800 --> 00:59:08,717
world. like you know you you read
a little bit about fire underwater uh in

1035
00:59:08,800 --> 00:59:11,757
in GBT2 so it's like okay maybe it
doesn't quite understand what these

1036
00:59:11,840 --> 00:59:16,317
things are um but at the same time I
think that you also see various things

1037
00:59:16,400 --> 00:59:19,997
like smoke coming from flame and you
know a bunch of these things that GBT2

1038
00:59:20,080 --> 00:59:21,677
it has no body it has no physical

1039
00:59:21,760 --> 00:59:23,757
experience it's just statically read

1040
00:59:23,840 --> 00:59:27,677
data um and I I think that I think that

1041
00:59:27,760 --> 00:59:32,077
the answer is like we don't know yet
um and these questions though we're

1042
00:59:32,160 --> 00:59:35,197
starting to be able to actually ask
them to physical systems uh that to real

1043
00:59:35,280 --> 00:59:37,117
systems that exist
and that's very exciting.

1044
00:59:37,200 --> 00:59:38,477
>> Do you think uh what's your intuition?

1045
00:59:38,560 --> 00:59:42,957
Do you think if you just
scale language modeling

1046
00:59:43,040 --> 00:59:47,677
like uh significantly scale that reasoning
can emerge from the same exact

1047
00:59:47,760 --> 00:59:49,597
mechanisms? I think it's unlikely that

1048
00:59:49,680 --> 00:59:53,517
if we just scale uh GPT2 that we'll have

1049
00:59:53,600 --> 00:59:57,197
reasoning in in the full-fledged way
and I think that there's like you know the

1050
00:59:57,280 --> 01:00:01,117
type signature is a little bit wrong
right that like there's something we do

1051
01:00:01,200 --> 01:00:05,197
with that we call thinking right
where we spend a lot of compute like a

1052
01:00:05,280 --> 01:00:08,317
variable amount of compute to get
to better answers right I think a little

1053
01:00:08,400 --> 01:00:10,557
bit harder I get a better answer and

1054
01:00:10,640 --> 01:00:12,797
that that kind of type signature isn't

1055
01:00:12,880 --> 01:00:17,117
quite encoded in a GBT right GBT will

1056
01:00:17,200 --> 01:00:21,357
kind of like it's spent a long time
in it like evolutionary history baking in

1057
01:00:21,440 --> 01:00:24,797
all this information getting very very
good at this predictive process and then

1058
01:00:24,880 --> 01:00:26,957
at runtime I just kind of do one forward

1059
01:00:27,040 --> 01:00:29,757
pass and uh and I'm able to generate

1060
01:00:29,840 --> 01:00:34,237
stuff and so you know there might be
small tweaks to what we do in order to

1061
01:00:34,320 --> 01:00:37,517
get the type signature right for example
well you know it's not really one

1062
01:00:37,600 --> 01:00:40,557
forward pass right you know you generate
symbol by symbol and so maybe you

1063
01:00:40,640 --> 01:00:43,917
generate like a whole sequence of of of
thoughts and you only keep like the last

1064
01:00:44,000 --> 01:00:47,837
bit or something um but I think that at
the very least I would expect you have

1065
01:00:47,920 --> 01:00:52,077
make changes like that. >> Yeah.
Yeah. Just exactly how we you said

1066
01:00:52,160 --> 01:00:54,797
think is the process of generating

1067
01:00:54,880 --> 01:00:59,357
thought by thought in the same kind
of way like you said keep the last bit the

1068
01:00:59,440 --> 01:01:00,877
thing that we converge towards.

1069
01:01:00,960 --> 01:01:03,357
>> Yep. >> And I think
there's there's another

1070
01:01:03,440 --> 01:01:06,317
piece which is which is interesting
which is this out of distribution

1071
01:01:06,400 --> 01:01:10,477
generalization right that like thinking
somehow lets us do that right that we

1072
01:01:10,560 --> 01:01:13,357
haven't experienced a thing and yet
somehow we just kind of keep refining

1073
01:01:13,440 --> 01:01:16,237
our mental model of it. Um this is again

1074
01:01:16,320 --> 01:01:18,877
something that feels tied to whatever

1075
01:01:18,960 --> 01:01:23,597
reasoning is and maybe it's a small
tweak to what we do. Maybe it's many

1076
01:01:23,680 --> 01:01:25,357
ideas and will take us many decades.

1077
01:01:25,440 --> 01:01:28,477
>> Yeah. So the the assumption there

1078
01:01:28,560 --> 01:01:33,517
generalization out of distribution
is that it's possible to create new new

1079
01:01:33,600 --> 01:01:36,317
ideas. Mhm. >> Uh the you
know it's possible that

1080
01:01:36,400 --> 01:01:38,317
nobody's ever created any new ideas and

1081
01:01:38,400 --> 01:01:42,557
then with scaling GPT2 to GPT20

1082
01:01:42,640 --> 01:01:45,357
uh you would you would essentially

1083
01:01:45,440 --> 01:01:50,477
generalize to all possible thoughts
humans could have [laughter]

1084
01:01:50,560 --> 01:01:51,757
>> just to play devil's advocate.

1085
01:01:51,840 --> 01:01:55,037
>> Right. Right. I mean how many how many
uh new new story ideas have we come up

1086
01:01:55,120 --> 01:01:56,557
with since Shakespeare? Right.

1087
01:01:56,640 --> 01:01:59,277
>> Yeah. Exactly.
It's [laughter] it's just

1088
01:01:59,360 --> 01:02:03,997
all different forms of love and drama
and so on. Okay. Not sure if you read

1089
01:02:04,080 --> 01:02:06,717
bit of lesson a recent
blog post by Ray Sutton.

1090
01:02:06,800 --> 01:02:10,157
>> Yep, I have.
>> He basically says uh something that

1091
01:02:10,240 --> 01:02:14,557
echoes some of the ideas that you've
been talking about which is uh he says

1092
01:02:14,640 --> 01:02:18,877
the biggest lesson that can be read from
70 years of AI research is that general

1093
01:02:18,960 --> 01:02:21,517
methods that leverage computation are

1094
01:02:21,600 --> 01:02:23,997
ultimately going to uh ultimately win

1095
01:02:24,080 --> 01:02:26,557
out. Uh do you agree with this? So

1096
01:02:26,640 --> 01:02:30,077
basically of and open AI in general

1097
01:02:30,160 --> 01:02:31,837
about the ideas you're exploring about

1098
01:02:31,920 --> 01:02:34,557
coming up with methods whether it's GPT2

1099
01:02:34,640 --> 01:02:37,357
modeling or whether it's uh open AI5

1100
01:02:37,440 --> 01:02:40,957
playing Dota where a general method is

1101
01:02:41,040 --> 01:02:46,157
better than a more fine-tuned expert tuned

1102
01:02:46,240 --> 01:02:49,117
method. >> Yeah.
So I I think that well one thing

1103
01:02:49,200 --> 01:02:52,557
that I think was really interesting
about the reaction to that blog post was

1104
01:02:52,640 --> 01:02:56,557
that a lot of people have read this as
saying that compute is all that matters

1105
01:02:56,640 --> 01:02:59,997
>> and that's a very threatening idea
right and I don't think it's a true idea

1106
01:03:00,080 --> 01:03:03,437
either right it's very clear that we
have algorithmic ideas that have been

1107
01:03:03,520 --> 01:03:07,437
very important for making progress and
to really build AGI you want to push as

1108
01:03:07,520 --> 01:03:10,557
far as you can on the computational
scale and you want to push as far as you

1109
01:03:10,640 --> 01:03:14,797
can on human human ingenuity and so I
think you need both but I think the way

1110
01:03:14,880 --> 01:03:17,997
that you phrase the question is actually
very good right that It's really about

1111
01:03:18,080 --> 01:03:20,477
what kind of ideas should we be striving

1112
01:03:20,560 --> 01:03:23,037
for. And absolutely, if you can find a

1113
01:03:23,120 --> 01:03:25,757
scalable idea, you pour more comput into

1114
01:03:25,840 --> 01:03:30,477
it, you pour more data into it, it gets
better. Like that's that's the real holy

1115
01:03:30,560 --> 01:03:34,317
grail. And so I think that uh that that
the answer to to the question I think is

1116
01:03:34,400 --> 01:03:38,237
is yes. Um that that that's really
how we think about it. that part of why

1117
01:03:38,320 --> 01:03:42,557
we're excited about the power of deep
learning, the potential for building AGI

1118
01:03:42,640 --> 01:03:44,477
is because we look at the systems that

1119
01:03:44,560 --> 01:03:47,037
exist in the most successful AI systems

1120
01:03:47,120 --> 01:03:51,517
and we realize that you scale those up,
they're going to work better. And I

1121
01:03:51,600 --> 01:03:54,797
think that that scalability is something
that really gives us hope for being able

1122
01:03:54,880 --> 01:03:57,197
to build transformative systems. So I'll

1123
01:03:57,280 --> 01:04:02,317
tell you this a partially an emotional
you know a thing that a response that

1124
01:04:02,400 --> 01:04:06,717
people often have if comput is so
important for state-of-the-art performance

1125
01:04:06,800 --> 01:04:10,157
you know individual developers maybe
a 13-year-old sitting somewhere in Kansas

1126
01:04:10,240 --> 01:04:13,997
or something like that you know they're
sitting they might not even have a GPU

1127
01:04:14,080 --> 01:04:16,717
and or may have a single GPU a 1080 or

1128
01:04:16,800 --> 01:04:21,837
something like that and there's this
feeling like well how can I possibly

1129
01:04:21,920 --> 01:04:23,677
compete or contribute to this world of

1130
01:04:23,760 --> 01:04:26,077
AI if uh scale is so important

1131
01:04:26,160 --> 01:04:28,877
important. So for if you can comment on

1132
01:04:28,960 --> 01:04:31,037
that and in general do you think we need

1133
01:04:31,120 --> 01:04:33,357
to also in the future focus on uh

1134
01:04:33,440 --> 01:04:36,877
democratizing compute resources more

1135
01:04:36,960 --> 01:04:39,037
more or as much as we democratize the

1136
01:04:39,120 --> 01:04:40,877
algorithms well so the way that I think

1137
01:04:40,960 --> 01:04:44,397
about it is that there's this space of

1138
01:04:44,480 --> 01:04:48,237
of possible progress right there's
a space of ideas and sort of systems that

1139
01:04:48,320 --> 01:04:52,237
that will work that will move us forward
and there's a portion of that space and

1140
01:04:52,320 --> 01:04:55,357
to some extent an increasingly
significant portion of that space that

1141
01:04:55,440 --> 01:05:00,317
does just require massive compute
resources and for that that I think that

1142
01:05:00,400 --> 01:05:04,477
that the answer is kind of clear
and that part of why we have the structure

1143
01:05:04,560 --> 01:05:07,757
that that we do is because we think it's
really important to be pushing the scale

1144
01:05:07,840 --> 01:05:12,157
and to be you know building these large
clusters and systems but there's another

1145
01:05:12,240 --> 01:05:15,357
part portion of the space that isn't
about the large scale compute that are

1146
01:05:15,440 --> 01:05:19,677
these ideas that and again I think that
for the ideas to really be impactful and

1147
01:05:19,760 --> 01:05:23,677
really shine that they should be ideas
that if you scale them up would work way

1148
01:05:23,760 --> 01:05:27,677
better than they do at small scale
um but that you can discover them

1149
01:05:27,760 --> 01:05:29,917
without massive computational resources.

1150
01:05:30,000 --> 01:05:32,877
And if you look at the history of of
recent developments, you think about

1151
01:05:32,960 --> 01:05:36,957
things like the GAN or the VAE that
these are ones that I think you could

1152
01:05:37,040 --> 01:05:40,557
come up with them uh without having and
you know in practice people did come up

1153
01:05:40,640 --> 01:05:43,757
with with them without having massive
massive computational resources,

1154
01:05:43,840 --> 01:05:45,277
>> right? I just talked to Ian Goodfellow,

1155
01:05:45,360 --> 01:05:48,797
but the thing is the init initial GAN

1156
01:05:48,880 --> 01:05:51,597
produced pretty terrible results, right?

1157
01:05:51,680 --> 01:05:55,037
So only because it was in a very
specific it was only because they're

1158
01:05:55,120 --> 01:05:58,797
smart enough to know that this is quite
surprising they can generate anything

1159
01:05:58,880 --> 01:06:01,517
that they know. I do you see a world or

1160
01:06:01,600 --> 01:06:03,757
is that too optimistic and dreamer like

1161
01:06:03,840 --> 01:06:06,957
to imagine that the compute resources

1162
01:06:07,040 --> 01:06:12,317
are something that's owned
by governments and provided as a utility.

1163
01:06:12,400 --> 01:06:16,157
>> Actually to some extent this this
question reminds me of of a blog post

1164
01:06:16,240 --> 01:06:19,997
from one of my former professors at
Harvard this guy Matt Matt Welsh who was

1165
01:06:20,080 --> 01:06:23,277
a systems professor. I remember sitting
in his tenure talk, right? And uh you

1166
01:06:23,360 --> 01:06:25,197
know that that he had literally just

1167
01:06:25,280 --> 01:06:27,277
gotten tenure. He went to Google for the

1168
01:06:27,360 --> 01:06:30,797
summer uh and uh then decided he wasn't

1169
01:06:30,880 --> 01:06:33,277
going back to academia, right? And that

1170
01:06:33,360 --> 01:06:38,157
kind of in his blog post he makes this
point that look as a systems researcher

1171
01:06:38,240 --> 01:06:41,117
that I come up with these cool system
ideas, right? and I kind of build a

1172
01:06:41,200 --> 01:06:45,357
little proof of concept and the best
thing I can hope for is that the people

1173
01:06:45,440 --> 01:06:49,757
at Google or Yahoo um which
was around at the time uh [laughter]

1174
01:06:49,840 --> 01:06:53,197
will implement it and like actually make
it work at scale, right? That's like the

1175
01:06:53,280 --> 01:06:55,677
dream for me, right? I build the little
thing and they turn into the big thing

1176
01:06:55,760 --> 01:07:00,317
that's actually working.
>> Um and uh for him he said I'm done with

1177
01:07:00,400 --> 01:07:03,437
that. I want to be the person who's
who's actually doing building and and

1178
01:07:03,520 --> 01:07:07,117
and deploying. And I think that there's
a similar dichotomy here, right? I think

1179
01:07:07,200 --> 01:07:11,277
that there are people who really
actually find value and I think it is a

1180
01:07:11,360 --> 01:07:14,797
valuable thing to do to be the person
who produces those ideas, right? Who

1181
01:07:14,880 --> 01:07:18,717
builds the proof of concept and yeah,
you don't get to generate the coolest

1182
01:07:18,800 --> 01:07:21,517
possible GAN images,
but you invented the GAN,

1183
01:07:21,600 --> 01:07:25,037
>> right? And so that there's that there's
there's a real trade-off there and I

1184
01:07:25,120 --> 01:07:28,157
think that's a very personal choice, but I
think there's value in both sides. So

1185
01:07:28,240 --> 01:07:31,837
do you think uh creating AGI something

1186
01:07:31,920 --> 01:07:36,237
or some new models uh would we would see

1187
01:07:36,320 --> 01:07:40,477
echoes of the brilliance even at the
prototype level. So you would be able to

1188
01:07:40,560 --> 01:07:45,277
develop those ideas without scale
the initial uh seeds. Well, so take take a

1189
01:07:45,360 --> 01:07:48,797
look at uh you know, I always like
to look at at examples that that exist,

1190
01:07:48,880 --> 01:07:53,437
right? Look at real precedent. And so
take a look at the June uh 2018 model

1191
01:07:53,520 --> 01:07:56,397
that we released that we
scaled up to turn into GPT2.

1192
01:07:56,480 --> 01:08:00,637
>> And you can see that at small scale
it set some records, right? This was, you

1193
01:08:00,720 --> 01:08:04,477
know, the original GPT. We actually had
some some cool generations. They weren't

1194
01:08:04,560 --> 01:08:08,877
nearly as as as amazing and really
stunning as the GPT2 ones, but it was

1195
01:08:08,960 --> 01:08:12,557
promising. It was interesting. And so I
think it is the case that with a lot of

1196
01:08:12,640 --> 01:08:16,637
these ideas that you see promise
at small scale. But there is an asterisk

1197
01:08:16,720 --> 01:08:21,037
here, a very big asterisk,
which is sometimes we see

1198
01:08:21,120 --> 01:08:25,117
behaviors that emerge that are
qualitatively different from anything we

1199
01:08:25,200 --> 01:08:30,397
saw at small scale and that the original
inventor of whatever algorithm looks at

1200
01:08:30,480 --> 01:08:32,717
and says, I didn't think it could do that.

1201
01:08:32,800 --> 01:08:34,877
>> This is what we saw in Dota. Right? So

1202
01:08:34,960 --> 01:08:36,637
PO was was created by John Scholman

1203
01:08:36,720 --> 01:08:39,757
who's a researcher here and uh and with

1204
01:08:39,840 --> 01:08:44,557
with Dota we basically just ran PO
at massive massive scale and uh you know

1205
01:08:44,640 --> 01:08:47,197
there's some tweaks in order to in order
to make it work but fundamentally it's

1206
01:08:47,280 --> 01:08:51,037
PO at the core and we were able to get

1207
01:08:51,120 --> 01:08:55,517
this long-term planning these these
behaviors to really play out uh on a

1208
01:08:55,600 --> 01:08:59,517
time scale that we just thought was not
possible. Um and John looked at that and

1209
01:08:59,600 --> 01:09:02,317
was like I didn't think it could do that.
That's what happens when you're at

1210
01:09:02,400 --> 01:09:05,677
three orders of magnitude more
scale [laughter] than you test it at.

1211
01:09:05,760 --> 01:09:07,277
>> Yeah. But it still has the same flavors

1212
01:09:07,360 --> 01:09:11,357
of um you know u at least echoes of the

1213
01:09:11,440 --> 01:09:14,077
expected billions. Uh although I suspect

1214
01:09:14,160 --> 01:09:16,877
with GPT scaled more and more you might

1215
01:09:16,960 --> 01:09:19,997
get surprising things. Uh so yeah,

1216
01:09:20,080 --> 01:09:21,757
you're right. It's it's interesting.

1217
01:09:21,840 --> 01:09:24,237
It's it's difficult to see how far an

1218
01:09:24,320 --> 01:09:29,037
idea will go when it's scaled. It's
an open question. Well, so to that point

1219
01:09:29,120 --> 01:09:32,157
with with Dota and PO, like I mean
here's here's a very concrete one,

1220
01:09:32,240 --> 01:09:34,957
right? Is like um it's actually one
thing that's very surprising about Dota

1221
01:09:35,040 --> 01:09:38,797
that I think people don't really pay
that much attention to is the degree of

1222
01:09:38,880 --> 01:09:42,957
generalization out of distribution
that happens, right? That you have this AI

1223
01:09:43,040 --> 01:09:47,597
that's trained against other bots for its
entirety the entirety of it existence.

1224
01:09:47,680 --> 01:09:49,197
>> Sorry to take a step back. Can you can

1225
01:09:49,280 --> 01:09:53,757
you talk through in in you know a story

1226
01:09:53,840 --> 01:09:56,477
of Dota a story of uh leading up to Open

1227
01:09:56,560 --> 01:10:01,677
AI5? Y and uh that passed and what
was the process of selfplay and so on of

1228
01:10:01,760 --> 01:10:04,957
training on >> yeah so
with Dota >> what is Dota

1229
01:10:05,040 --> 01:10:09,197
>> yeah Dota is a complex video game and
we started training we started trying to

1230
01:10:09,280 --> 01:10:13,277
solve Dota because we felt like this was
a step towards the real world relative

1231
01:10:13,360 --> 01:10:16,877
to other games like chess or go right
those very cerebral games where you just

1232
01:10:16,960 --> 01:10:20,237
kind of have this board very discreet
moves Dota starts to be much more

1233
01:10:20,320 --> 01:10:23,837
continuous time that you have this huge
variety of different actions that you

1234
01:10:23,920 --> 01:10:27,837
have a 45minute game with all these
different units And uh it's got a lot of

1235
01:10:27,920 --> 01:10:31,117
messiness to it uh that that really
hasn't been captured by by previous

1236
01:10:31,200 --> 01:10:35,437
games. And famously, all of the
hard-coded bots for Dota were terrible,

1237
01:10:35,520 --> 01:10:37,837
right? It's just impossible to write
anything good for it because it's so

1238
01:10:37,920 --> 01:10:41,837
complex. And so this seemed like
a really good place to push what's the

1239
01:10:41,920 --> 01:10:45,597
state-of-the-art in in reinforcement
learning. And so we started by focusing

1240
01:10:45,680 --> 01:10:49,117
on the one versus one version of the
game and uh and and were able to to

1241
01:10:49,200 --> 01:10:52,637
solve that. We're able to beat the world
champions and that the that the the

1242
01:10:52,720 --> 01:10:56,237
learning, you know, the the skill curve
was this crazy exponential, right? And

1243
01:10:56,320 --> 01:10:59,837
it was like constantly we were just
scaling up that we were fixing bugs and

1244
01:10:59,920 --> 01:11:02,637
you know that you look at the at the
skill curve and it was really a very

1245
01:11:02,720 --> 01:11:06,157
very smooth one. This is actually really
interesting to see how that like human

1246
01:11:06,240 --> 01:11:09,997
iteration loop yielded very
steady exponential progress.

1247
01:11:10,080 --> 01:11:13,917
>> And uh to one one side note first
of all it's an exceptionally popular video

1248
01:11:14,000 --> 01:11:18,797
game. The side effect is that there's
a lot of incredible human experts at that

1249
01:11:18,880 --> 01:11:22,877
video game. So the benchmark that you're
trying to reach is is very high. and the

1250
01:11:22,960 --> 01:11:26,877
other. Can you talk about the approach
that was used initially and throughout

1251
01:11:26,960 --> 01:11:29,357
of training these uh
agents to play this game?

1252
01:11:29,440 --> 01:11:33,357
>> Yep. And so the approach that we used
is selfplay. And so you have two agents,

1253
01:11:33,440 --> 01:11:37,197
they don't know anything. They battle
each other. They discover something a

1254
01:11:37,280 --> 01:11:40,317
little bit good and now they both know
it and they just get better and better

1255
01:11:40,400 --> 01:11:45,357
and better without bound. And that's
a really powerful idea, right? Uh that we

1256
01:11:45,440 --> 01:11:48,717
then went from the one versus one
version of the game and scaled up to

1257
01:11:48,800 --> 01:11:51,677
five versus five, right? So you think
about kind of like with basketball where

1258
01:11:51,760 --> 01:11:55,037
you have this like team sport and you
need to do all this coordination. Um and

1259
01:11:55,120 --> 01:11:58,397
we're able to push the same idea, the

1260
01:11:58,480 --> 01:12:02,317
same selfplay uh to to to really get to

1261
01:12:02,400 --> 01:12:07,517
the professional level at the full five
versus five version of the game. And uh

1262
01:12:07,600 --> 01:12:11,917
and and the things I think are really
interesting here is that these agents in

1263
01:12:12,000 --> 01:12:14,957
some ways they're almost like an insect-l
like intelligence, right? Where

1264
01:12:15,040 --> 01:12:17,437
you know there's they have a lot
in common with how an insect is trained,

1265
01:12:17,520 --> 01:12:20,397
right? insect kind of lives in this
environment for a very long time or you

1266
01:12:20,480 --> 01:12:23,597
know the the ancestors of this insect
have been around for a long time and had

1267
01:12:23,680 --> 01:12:28,157
a lot of experience that gets baked into
into into this agent and you know it's

1268
01:12:28,240 --> 01:12:31,437
not really smart in the sense of a human
right it's not able to go and learn

1269
01:12:31,520 --> 01:12:34,877
calculus but it's able to navigate its
environment extremely well it's able to

1270
01:12:34,960 --> 01:12:39,437
handle unexpected things in environment
that's never seen before pretty well um

1271
01:12:39,520 --> 01:12:42,877
and we see the same sort of thing with
our Dota bots right that they're able to

1272
01:12:42,960 --> 01:12:46,637
in within this game they're able to play
against humans which is something that

1273
01:12:46,720 --> 01:12:50,077
never existed in its evolutionary
environment, totally different play

1274
01:12:50,160 --> 01:12:53,917
styles from humans versus the bots,
and yet it's able to handle it extremely

1275
01:12:54,000 --> 01:12:56,557
well. And that's something I think was

1276
01:12:56,640 --> 01:12:59,277
very surprising to us was something that

1277
01:12:59,360 --> 01:13:04,557
doesn't really emerge from what we've
seen with PO at smaller scale, right?

1278
01:13:04,640 --> 01:13:07,517
And the kind of scale we're running the
stuff at was uh you know, like let's say

1279
01:13:07,600 --> 01:13:11,757
like a 100,000 CPU cores running
with like hundreds of GPUs. uh it was

1280
01:13:11,840 --> 01:13:14,157
probably about uh you know like you know

1281
01:13:14,240 --> 01:13:16,877
something like hundreds of of years of

1282
01:13:16,960 --> 01:13:19,037
experience going into this bot every

1283
01:13:19,120 --> 01:13:22,957
single real day and so that scale is

1284
01:13:23,040 --> 01:13:26,317
massive and we start to see very
different kinds of behaviors out of the

1285
01:13:26,400 --> 01:13:28,237
algorithms that we all know and love.

1286
01:13:28,320 --> 01:13:31,037
>> Dota you mentioned
beat the world expert

1287
01:13:31,120 --> 01:13:35,757
one v one and then uh you didn't weren't

1288
01:13:35,840 --> 01:13:38,557
able to win 5v5 this year. Yeah.

1289
01:13:38,640 --> 01:13:43,837
>> At the best players in the world. Uh so
what's what's the comeback story? What's

1290
01:13:43,920 --> 01:13:47,277
first of all talk through that
exceptionally exciting event and uh

1291
01:13:47,360 --> 01:13:50,557
what's what's the the following
months and this year look like?

1292
01:13:50,640 --> 01:13:53,677
>> Yeah. Yeah. So well one thing that
that's interesting is that uh you know

1293
01:13:53,760 --> 01:13:58,157
we lose all the time >> because
we play here. So the Dota team

1294
01:13:58,240 --> 01:14:02,797
at OpenAI, we we played the bot against
better players than our system all the

1295
01:14:02,880 --> 01:14:06,237
time or at least we we we used to,
right? Like you know that the the first

1296
01:14:06,320 --> 01:14:10,077
time we lost publicly was we went up on
stage at the International and we played

1297
01:14:10,160 --> 01:14:13,197
against some of the best teams in the
world. Um and we ended up losing both

1298
01:14:13,280 --> 01:14:17,037
games, but we gave them a run for their
money, right? That both games were kind

1299
01:14:17,120 --> 01:14:20,877
of 30 minutes, 25 minutes and they went
back and forth, back and forth, back and

1300
01:14:20,960 --> 01:14:25,357
forth. And so I think that really shows
that we're at the professional level.

1301
01:14:25,440 --> 01:14:28,637
Um, and that kind of looking at those
games, we think that the coin could have

1302
01:14:28,720 --> 01:14:31,357
gone a different direction and we could
have could have had some wins. And that

1303
01:14:31,440 --> 01:14:34,477
that was actually very encouraging for us.
Um, and you know, it's interesting

1304
01:14:34,560 --> 01:14:38,797
because the international was at a fixed
time, right? So we we knew exactly what

1305
01:14:38,880 --> 01:14:41,757
day we were going to be playing and we
pushed as far as we could, as fast as we

1306
01:14:41,840 --> 01:14:45,917
could. Two weeks later, we had a bot
that had an 80% win rate versus the one

1307
01:14:46,000 --> 01:14:49,437
that played at TI. Um, so the March of
Progress, uh, you know, you should think

1308
01:14:49,520 --> 01:14:53,437
of as a snapshot rather than as an end
state. Um, and so in fact, we'll we'll

1309
01:14:53,520 --> 01:14:57,437
be announcing our, uh, our finals, uh,
pretty soon. I actually think that, uh,

1310
01:14:57,520 --> 01:15:01,917
we'll announce our final match,
uh, prior to, uh, this podcast being

1311
01:15:02,000 --> 01:15:05,517
released. Okay. So, there should be, uh,
we'll be playing we'll be playing, uh,

1312
01:15:05,600 --> 01:15:10,077
against the the world champions. And you
know for us it's really less about like

1313
01:15:10,160 --> 01:15:14,957
that the way that we think about what
what's upcoming is the final milestone

1314
01:15:15,040 --> 01:15:19,037
the final competitive milestone for the
project right that our goal in all of

1315
01:15:19,120 --> 01:15:22,557
this isn't really about
beating humans at Dota

1316
01:15:22,640 --> 01:15:24,157
>> our goal is to push
the state-of-the-art

1317
01:15:24,240 --> 01:15:27,197
in reinforcement learning and we've done
that right and we've actually learned a

1318
01:15:27,280 --> 01:15:30,957
lot from our system and that we have uh
you know I think a lot of exciting next

1319
01:15:31,040 --> 01:15:34,077
steps that we want to take and so you
know kind of the final showcase of what

1320
01:15:34,160 --> 01:15:37,597
we built we're going to do this match
um but for us it's not really the success

1321
01:15:37,680 --> 01:15:41,757
or failure um to see you know do do we
have the coin flip go in our direction

1322
01:15:41,840 --> 01:15:45,837
or against >> where do
you see the field of deep

1323
01:15:45,920 --> 01:15:46,977
learning heading in the next

1324
01:15:47,060 --> 01:15:49,597
[clears throat] few years where do you

1325
01:15:49,680 --> 01:15:52,957
see the work and reinforcement learning

1326
01:15:53,040 --> 01:15:56,157
perhaps heading and uh

1327
01:15:56,240 --> 01:15:59,037
more specifically with open AI all the

1328
01:15:59,120 --> 01:16:03,757
exciting projects that you're
working on what does 2019 hold for you

1329
01:16:03,840 --> 01:16:06,957
>> massive scale >> scale >> I will
put an asterisk on and just say,

1330
01:16:07,040 --> 01:16:10,237
you know, I think that it's about
ideas plus scale. You need both.

1331
01:16:10,320 --> 01:16:12,637
>> So that's a really uh good u point. So

1332
01:16:12,720 --> 01:16:16,397
the question in terms of ideas, you have

1333
01:16:16,480 --> 01:16:19,677
a lot of projects that are exploring

1334
01:16:19,760 --> 01:16:22,397
different areas of intelligence and uh

1335
01:16:22,480 --> 01:16:26,797
the question is when you when you think
of scale, do you think about growing the

1336
01:16:26,880 --> 01:16:30,957
scale of those individual projects or do
you think about adding new projects? And

1337
01:16:31,040 --> 01:16:35,437
uh sorry to and if you're thinking
of adding new projects or if you look at

1338
01:16:35,520 --> 01:16:39,517
the past what's the process of coming
up with new project new ideas.

1339
01:16:39,600 --> 01:16:43,597
>> Yep. Uh so we really have a life cycle
of project here. Uh so we start with a

1340
01:16:43,680 --> 01:16:46,877
few people uh just working on a small
scale idea and language is actually a

1341
01:16:46,960 --> 01:16:50,237
very good example of this that was really
you know one person here who was

1342
01:16:50,320 --> 01:16:53,997
pushing on language for a long time.
I mean then you get signs of life right

1343
01:16:54,080 --> 01:16:58,717
and so this is like let's say uh you
know with with the original GPT we had

1344
01:16:58,800 --> 01:17:02,157
something that was interesting and we
said okay it's time to scale this right

1345
01:17:02,240 --> 01:17:05,517
it's time to put more people on it put
more computational resources behind it

1346
01:17:05,600 --> 01:17:09,357
and uh and then we just kind of keep
pushing and keep pushing and the end

1347
01:17:09,440 --> 01:17:12,557
state is something that looks like Dota
or robotics where you have a large team

1348
01:17:12,640 --> 01:17:16,557
of you know 10 or 15 people uh that are
running things at very large scale uh

1349
01:17:16,640 --> 01:17:20,957
and that you're able to really have
material engineering uh and and uh and

1350
01:17:21,040 --> 01:17:25,517
and you sort of machine learning science
coming together to make systems that

1351
01:17:25,600 --> 01:17:28,637
work uh and get material results
that just would have been impossible

1352
01:17:28,720 --> 01:17:32,637
otherwise. Um so we do that whole life
cycle. We've done it a number of times

1353
01:17:32,720 --> 01:17:37,037
uh you know typically end to end. It's
probably two uh two years or so uh to do

1354
01:17:37,120 --> 01:17:39,757
it. You know the organization's been
around for three years. So maybe we'll

1355
01:17:39,840 --> 01:17:41,277
find that we also have longer life cycle

1356
01:17:41,360 --> 01:17:44,157
projects. Um but you know we we uh we'll

1357
01:17:44,240 --> 01:17:48,557
work up to those. We have uh so so
one one team that we were actually just

1358
01:17:48,640 --> 01:17:51,997
starting Illy and I are are kicking off
a new team called the reasoning team and

1359
01:17:52,080 --> 01:17:53,837
that this is to really try to tackle how

1360
01:17:53,920 --> 01:17:59,357
do you get neural networks to reason and
uh we think that this will be a longterm

1361
01:17:59,440 --> 01:18:01,597
project uh it's one that
we're very excited about.

1362
01:18:01,680 --> 01:18:05,677
>> Uh in terms of reasoning
super exciting topic

1363
01:18:05,760 --> 01:18:08,557
what do you what kind of benchmarks

1364
01:18:08,640 --> 01:18:11,597
uh what kind of tests of reasoning do

1365
01:18:11,680 --> 01:18:13,677
you envision? What what would if you sat

1366
01:18:13,760 --> 01:18:16,797
back with whatever drink and you would

1367
01:18:16,880 --> 01:18:21,037
be impressed that this system is able
to do something what would that look like?

1368
01:18:21,120 --> 01:18:25,917
uh theorem proving >> theorem
proving so some kind of logic

1369
01:18:26,000 --> 01:18:29,117
and especially mathematical logic
>> I think so right and I think that

1370
01:18:29,200 --> 01:18:31,917
there's there's there's kind of other
problems that are dual to theorem

1371
01:18:32,000 --> 01:18:35,517
proving in particular um you know you
think about uh programming uh you think

1372
01:18:35,600 --> 01:18:37,757
about even like security analysis of of

1373
01:18:37,840 --> 01:18:43,037
code um that these all kind of capture
the same sorts of core reasoning and and

1374
01:18:43,120 --> 01:18:46,637
being able to do some out
of distribution generalization

1375
01:18:46,720 --> 01:18:50,797
>> it would be quite exciting if openai
reasoning team was able to prove prove

1376
01:18:50,880 --> 01:18:52,957
that P equals NP. That would be very nice.

1377
01:18:53,040 --> 01:18:56,637
>> Uh it would be very very very exciting
especially uh if it turns out the P

1378
01:18:56,720 --> 01:18:58,897
equals NP. That'll be interesting too.

1379
01:18:58,980 --> 01:19:03,357
[laughter] >> I just it would
be uh ironic and humorous.

1380
01:19:03,440 --> 01:19:04,157
>> Yep.

1381
01:19:04,240 --> 01:19:07,677
>> Uh so what problem stands out to you as

1382
01:19:07,760 --> 01:19:10,717
uh the most uh exciting and challenging

1383
01:19:10,800 --> 01:19:15,197
impactful to the work for us as a
community in general and for Open AI

1384
01:19:15,280 --> 01:19:18,717
this year? You mentioned reasoning.
I think that's that's a heck of a problem.

1385
01:19:18,800 --> 01:19:20,637
>> Yeah. So I think reasoning is an
important one. And I think it's going to

1386
01:19:20,720 --> 01:19:23,837
be hard to get good results in 2019.
Um, you know, again, just like we think

1387
01:19:23,920 --> 01:19:28,077
about the life cycle takes time. Um,
I think for 2019, language modeling seems

1388
01:19:28,160 --> 01:19:31,277
to be kind of on that ramp, right?
It's at the point that we have a technique

1389
01:19:31,360 --> 01:19:35,357
that works. We want to scale 100x,
thousandx, see what happens.

1390
01:19:35,440 --> 01:19:38,797
>> Awesome. Do you think
we're living in a simulation?

1391
01:19:38,880 --> 01:19:42,477
>> Uh, I think it's I think it's hard
to have a a real opinion about it. Uh, you

1392
01:19:42,560 --> 01:19:45,837
know, it's actually interesting.
I separate out things that I think can

1393
01:19:45,920 --> 01:19:49,677
have like, you know, yield materially
different predictions about the world.

1394
01:19:49,760 --> 01:19:53,597
um from ones that are just kind of you
know fun to speculate about and I I kind

1395
01:19:53,680 --> 01:19:55,517
of view simulation as more like is there

1396
01:19:55,600 --> 01:20:01,037
a flying teapot between Mars and Jupiter
like maybe but it's a little bit hard to

1397
01:20:01,120 --> 01:20:02,477
know what that would mean for my life.

1398
01:20:02,560 --> 01:20:05,517
>> So there is something
actionable so some

1399
01:20:05,600 --> 01:20:08,477
of the best work open has done is in the

1400
01:20:08,560 --> 01:20:13,197
field of reinforcement learning and uh
some of the success of reinforcement

1401
01:20:13,280 --> 01:20:15,277
learning come from being able to

1402
01:20:15,360 --> 01:20:20,237
simulate the problem you're trying
to solve. So it do you have a hope for

1403
01:20:20,320 --> 01:20:23,037
reinforcement for the future
of reinforcement learning and for the

1404
01:20:23,120 --> 01:20:26,317
future of simulation like whether it's
we're talking about autonomous vehicles

1405
01:20:26,400 --> 01:20:29,357
or any kind of system? Do you see that

1406
01:20:29,440 --> 01:20:31,597
scaling or we'll be able to simulate

1407
01:20:31,680 --> 01:20:34,317
systems and and hence be able to create

1408
01:20:34,400 --> 01:20:39,117
a simulator that echoes our real world
and uh proving once and for all even

1409
01:20:39,200 --> 01:20:42,317
though you're denying it that
we're living in a simulation.

1410
01:20:42,400 --> 01:20:44,397
>> I feel like there's two separate
questions, right? So, you know, kind

1411
01:20:44,480 --> 01:20:47,437
kind of at the core there of like can
we can we use simulation for self-driving

1412
01:20:47,520 --> 01:20:49,917
cars? Um, take a look at our robotic

1413
01:20:50,000 --> 01:20:54,557
system dactyl, right? That was trained
in simulation using the Dota system in

1414
01:20:54,640 --> 01:20:59,037
fact and it transfers to a physical robot.
And I think everyone looks at our

1415
01:20:59,120 --> 01:21:01,437
Dota system, they're like, okay, it's
just a game. How are you ever going to

1416
01:21:01,520 --> 01:21:04,397
escape to the real world? And the answer
is, well, we did it with a physical

1417
01:21:04,480 --> 01:21:07,837
robot that no one can program. And so,
I think the answer is simulation goes a

1418
01:21:07,920 --> 01:21:11,437
lot further than you think um if you
apply the right techniques to it. Um,

1419
01:21:11,520 --> 01:21:14,237
now there's a question of, you know,
are the beings in that simulation going to

1420
01:21:14,320 --> 01:21:17,597
going to wake up and and have
consciousness? Um, I think that one

1421
01:21:17,680 --> 01:21:21,117
seems a lot a lot harder to to again
reason about. I think that, you know,

1422
01:21:21,200 --> 01:21:24,797
you really should think about like where
where exactly does human consciousness

1423
01:21:24,880 --> 01:21:28,157
come from and our own self-awareness and,
you know, is it just that like once

1424
01:21:28,240 --> 01:21:30,477
you have like a complicated enough
neural net that you have to worry about

1425
01:21:30,560 --> 01:21:35,037
the the agents feeling pain? Um,
and uh, you know, I think there's like

1426
01:21:35,120 --> 01:21:38,557
interesting speculation to to do there,
but uh, but you know, again, I think

1427
01:21:38,640 --> 01:21:41,597
it's it's a little bit hard to know
for sure. Well, let me just keep with the

1428
01:21:41,680 --> 01:21:43,917
speculation. Do you think uh to create

1429
01:21:44,000 --> 01:21:46,237
intelligence, general intelligence, you

1430
01:21:46,320 --> 01:21:50,397
need one consciousness and two a body.

1431
01:21:50,480 --> 01:21:53,517
Do you think any of those elements
are needed or is intelligence something

1432
01:21:53,600 --> 01:21:55,757
that's that's orthogonal to those?

1433
01:21:55,840 --> 01:21:59,277
>> I'll stick to the the kind of like
the the non- grand answer first, right? So

1434
01:21:59,360 --> 01:22:02,637
the non-grand answer is just to look at,
you know, what are we already making

1435
01:22:02,720 --> 01:22:05,917
work? You look at GPT2, a lot of people
would have said that to even get these

1436
01:22:06,000 --> 01:22:09,277
kinds of results, you need real world
experience. You need a body. You need

1437
01:22:09,360 --> 01:22:12,397
grounding. How are you supposed
to reason about any of these things? How

1438
01:22:12,480 --> 01:22:14,957
are you supposed to like even kind
of know about smoke and fire and those

1439
01:22:15,040 --> 01:22:16,957
things if you've never experienced them?

1440
01:22:17,040 --> 01:22:19,517
>> And GPD2 shows that you can actually go

1441
01:22:19,600 --> 01:22:22,397
way further than that kind of reasoning

1442
01:22:22,480 --> 01:22:27,277
would predict. So I think
that that that in terms of do

1443
01:22:27,360 --> 01:22:30,077
we need consciousness, do we need a body?
It seems the answer is probably

1444
01:22:30,160 --> 01:22:33,277
not, right? That we could probably just
continue to push kind of the systems we

1445
01:22:33,360 --> 01:22:37,757
have, they already feel general.
um they're not as competent or as general

1446
01:22:37,840 --> 01:22:40,077
or able to learn as quickly as an AGI

1447
01:22:40,160 --> 01:22:41,677
would but you know they're at least like

1448
01:22:41,760 --> 01:22:45,357
kind of protoagi in some way and they

1449
01:22:45,440 --> 01:22:47,917
don't need any of those things. Now, now

1450
01:22:48,000 --> 01:22:52,397
let's move to the grand answer,
which is, you know, if are are neural nets

1451
01:22:52,480 --> 01:22:56,397
nets conscious already, would we ever
know? How can we tell, right? And, you

1452
01:22:56,480 --> 01:23:00,397
know, here here's where the speculation
starts to become become, you know, at

1453
01:23:00,480 --> 01:23:04,397
least interesting or fun and maybe
a little bit disturbing depending on on

1454
01:23:04,480 --> 01:23:08,397
where you take it. But it certainly
seems that when we think about animals

1455
01:23:08,480 --> 01:23:11,437
that there's some continuum of of
of consciousness. You know, my cat, I

1456
01:23:11,520 --> 01:23:15,117
think, is uh is conscious in some way,
right? Uh you know, not as conscious as

1457
01:23:15,200 --> 01:23:18,077
a human. And you could imagine that you
could build like a little consciousness

1458
01:23:18,160 --> 01:23:20,637
meter, right? You point at a cat,
it gives you a little reading. Point at a

1459
01:23:20,720 --> 01:23:23,597
human, gives you much bigger reading.

1460
01:23:23,680 --> 01:23:27,597
What would happen if you pointed one
of those at a Dota neural net? And if

1461
01:23:27,680 --> 01:23:31,997
you're training in this massive
simulation, do the neural nets feel pain?

1462
01:23:32,080 --> 01:23:36,637
You know, it becomes pretty hard
to know that the answer is no. Um, and it

1463
01:23:36,720 --> 01:23:40,637
becomes pretty hard to to really think
about what that would mean if the answer

1464
01:23:40,720 --> 01:23:44,317
were yes. And it's very
possible, you know, for

1465
01:23:44,400 --> 01:23:48,237
example, you could imagine that maybe
the reason that humans are have

1466
01:23:48,320 --> 01:23:52,157
consciousness is because it's a it's
a convenient computational shortcut,

1467
01:23:52,240 --> 01:23:55,597
right? If you think about it, if you
have a being that wants to avoid pain,

1468
01:23:55,680 --> 01:23:59,037
which seems pretty important to survive
in this environment um and wants to

1469
01:23:59,120 --> 01:24:03,117
like, you know, eat food um then
that maybe the best way of doing it is to

1470
01:24:03,200 --> 01:24:04,557
have a being that's conscious, right?

1471
01:24:04,640 --> 01:24:07,837
That you know, in order to succeed
in the environment, you need to have those

1472
01:24:07,920 --> 01:24:10,877
properties. And how are you supposed
to implement them? and maybe this this

1473
01:24:10,960 --> 01:24:14,717
consciousness's way of doing that.
If that's true, then actually maybe we

1474
01:24:14,800 --> 01:24:17,757
should expect that really competent
reinforcement learning agents will also

1475
01:24:17,840 --> 01:24:21,677
have consciousness. But you know that's
a big if and I think there are a lot of

1476
01:24:21,760 --> 01:24:24,157
other arguments that you
can make in other directions.

1477
01:24:24,240 --> 01:24:27,917
>> I think that's a really interesting
idea that even GPT2 has some degree of

1478
01:24:28,000 --> 01:24:30,557
consciousness. That's something uh it's

1479
01:24:30,640 --> 01:24:34,957
actually not as crazy to think about
as useful to think about as as we think

1480
01:24:35,040 --> 01:24:37,997
about what it means to create
intelligence of a dog, intelligence of a

1481
01:24:38,080 --> 01:24:39,837
cat

1482
01:24:39,920 --> 01:24:42,477
and the intelligence of a human. So last

1483
01:24:42,560 --> 01:24:45,277
question, do you think

1484
01:24:45,360 --> 01:24:48,637
we will ever fall in love like in the

1485
01:24:48,720 --> 01:24:52,797
movie [snorts] her with an artificial
intelligence system or an artificial

1486
01:24:52,880 --> 01:24:55,837
intelligence system falling
in love with a human?

1487
01:24:55,920 --> 01:24:58,637
>> I hope so. if there's any better way to

1488
01:24:58,720 --> 01:25:01,997
end it is on uh love. So, Greg, thanks

1489
01:25:02,080 --> 01:25:06,680
so much for talking today.
>> Thank you for having me.

1490
01:25:15,610 --> 01:25:17,630
[music]