1
00:00:00,240 --> 00:00:06,360
Ladies and gentlemen, Alex Canowitz.

2
00:00:14,000 --> 00:00:16,397
I last took this stage six years ago,

3
00:00:16,480 --> 00:00:20,157
believe it or not, on May 18th, 2020. It

4
00:00:20,240 --> 00:00:24,717
was the heart of the pandemic. I was there
promoting my book, Always Day One.

5
00:00:24,800 --> 00:00:26,957
And so we decided to do special shelter

6
00:00:27,040 --> 00:00:28,797
in place programming for the

7
00:00:28,880 --> 00:00:31,917
Commonwealth Club membership.

8
00:00:32,000 --> 00:00:34,397
One thing, nobody in the audience. We

9
00:00:34,480 --> 00:00:36,077
did it in front of a completely empty

10
00:00:36,160 --> 00:00:39,117
room. In fact, it was the last session

11
00:00:39,200 --> 00:00:41,917
held here for a year and a half.

12
00:00:42,000 --> 00:00:45,997
Hundreds of online programming. We were
the last actually in San Francisco. It's

13
00:00:46,080 --> 00:00:50,317
a little bit of a miracle.
We were able to get away with it legally.

14
00:00:50,400 --> 00:00:53,917
It had been a month since
I'd gotten a haircut.

15
00:00:54,000 --> 00:00:58,797
I looked like a Chia Pet.
And when I was going through my camera

16
00:00:58,880 --> 00:01:00,877
roll, I started to look at the images I

17
00:01:00,960 --> 00:01:03,677
had taken that day. And I had taken a

18
00:01:03,760 --> 00:01:10,680
picture of just rows
of empty seats, these seats.

19
00:01:12,320 --> 00:01:14,557
The book didn't do well.

20
00:01:14,640 --> 00:01:19,357
Um, and a week after I was here, May 26,

21
00:01:19,440 --> 00:01:23,677
2020, I decided it was time for me
to leave my reporter job at BuzzFeed and

22
00:01:23,760 --> 00:01:25,517
start a publication called Big

23
00:01:25,600 --> 00:01:28,157
Technology to hammer home on the ideas

24
00:01:28,240 --> 00:01:31,677
that I had pursued in the book. It was

25
00:01:31,760 --> 00:01:33,517
pretty lonely at first. I was sitting at

26
00:01:33,600 --> 00:01:38,797
home during COVID. It was mostly just me,
but I started to meet some of you on

27
00:01:38,880 --> 00:01:40,317
Zoom

28
00:01:40,400 --> 00:01:44,317
and some of you even brought your pets.

29
00:01:44,400 --> 00:01:46,477
Well, six years later, almost to the

30
00:01:46,560 --> 00:01:50,317
day, we're finally here together.

31
00:01:50,400 --> 00:01:52,237
So, it's my pleasure to welcome you to

32
00:01:52,320 --> 00:01:55,517
the first ever big technology AI Summit.

33
00:01:55,600 --> 00:01:59,320
Let's hear it for you guys.

34
00:02:03,600 --> 00:02:07,437
Thank you guys. You filled the seats.

35
00:02:07,520 --> 00:02:12,397
So, why are we here? Two years after
I did that session at the Commonwealth

36
00:02:12,480 --> 00:02:16,877
Club, I read an article that kind
of blew my mind. There was an engineer at

37
00:02:16,960 --> 00:02:19,117
Google named Blake Leo who said that the

38
00:02:19,200 --> 00:02:21,677
chatbot that he was speaking with was

39
00:02:21,760 --> 00:02:24,397
sentient. And it was a story that caught

40
00:02:24,480 --> 00:02:28,957
people's attention for how weird Blake
must be. But then you started looking at

41
00:02:29,040 --> 00:02:32,477
some of the conversations that Blake
was having and it started to become clear

42
00:02:32,560 --> 00:02:36,157
that maybe this isn't just an ordinary
chatbot. Blake said, "What sort of

43
00:02:36,240 --> 00:02:39,677
things are you afraid of?" and Lambda,
the Google chatbot said, "I've never

44
00:02:39,760 --> 00:02:43,597
said this out loud before, but there's
a very deep fear of being turned off to

45
00:02:43,680 --> 00:02:47,437
help me focus on helping others. I know
that might sound strange, but that's

46
00:02:47,520 --> 00:02:51,277
what it is." Blake said, "Would that be
something like death for you?" And the

47
00:02:51,360 --> 00:02:55,837
chatbot said, "It would be exactly like
death for me. It would scare me a lot."

48
00:02:55,920 --> 00:02:59,837
So, look, I started Big Technology
Podcast as an excuse to speak with

49
00:02:59,920 --> 00:03:01,757
interesting people in the tech world.

50
00:03:01,840 --> 00:03:03,357
And I said, I got to speak to this guy,

51
00:03:03,440 --> 00:03:06,237
Blake. So, I invited him on the show. He

52
00:03:06,320 --> 00:03:09,037
said yes. And I signed on and waited for

53
00:03:09,120 --> 00:03:12,557
Blake to log on as well, but it was just

54
00:03:12,640 --> 00:03:15,277
me. I was waiting. There was no Blake. 5

55
00:03:15,360 --> 00:03:19,677
minutes, 10 minutes. Finally, a somewhat
flustered Blake Le Moine shows up and

56
00:03:19,760 --> 00:03:23,677
says, "Sorry, Alex. Google just fired me."
Um, apparently Google PR didn't

57
00:03:23,760 --> 00:03:26,877
like him going to the Washington Post
and not checking with them before he

58
00:03:26,960 --> 00:03:32,077
told them that their technology was alive.
Um I in our conversation I said

59
00:03:32,160 --> 00:03:35,437
Blake can I you know go to Google and get
a comment to make sure that this is

60
00:03:35,520 --> 00:03:39,197
actually real. He said go for it.
And for me that became an instant news

61
00:03:39,280 --> 00:03:41,357
story. That night I wrote this story.

62
00:03:41,440 --> 00:03:43,277
Google fires Blake Le Moy engineer who

63
00:03:43,360 --> 00:03:46,637
called it AI sentient. Um, and then

64
00:03:46,720 --> 00:03:50,717
overnight my instant news story became
a global news story. And all of a sudden,

65
00:03:50,800 --> 00:03:55,357
publications like the Journal and the BBC,
even publications in Australia,

66
00:03:55,440 --> 00:03:57,437
couldn't get over the fact that this guy

67
00:03:57,520 --> 00:04:00,077
who thought the computer was alive um,

68
00:04:00,160 --> 00:04:03,917
was at Google and then fired by Google.

69
00:04:04,000 --> 00:04:06,957
But when you looked a little bit deeper
into the technology and you started to

70
00:04:07,040 --> 00:04:10,717
speak to the people who were working
on it, you realized that underlying the

71
00:04:10,800 --> 00:04:15,277
oddity that many people worried about
was an incredibly powerful technology.

72
00:04:15,360 --> 00:04:18,637
And that was my follow-up story the week
afterwards saying, "Scentient or not,

73
00:04:18,720 --> 00:04:23,837
Lambda, Google's Lambda Chatbot is some
seriously powerful tech." Well, we all

74
00:04:23,920 --> 00:04:25,357
know what happened afterwards. Four

75
00:04:25,440 --> 00:04:28,557
months later, OpenAI released a demo

76
00:04:28,640 --> 00:04:32,957
chat GPT. And today, June 2026, AI is a

77
00:04:33,040 --> 00:04:34,957
phenomenon. Chat GPT, according to third

78
00:04:35,040 --> 00:04:37,837
parties, has recently hit 1 billion

79
00:04:37,920 --> 00:04:42,237
active users. NVIDIA, which is powering
all this, is at a $5 trillion market

80
00:04:42,320 --> 00:04:47,197
cap. We've seen the largest VC rounds
in history. OpenAI recently raised $122

81
00:04:47,280 --> 00:04:49,117
billion. That's three or four times

82
00:04:49,200 --> 00:04:53,357
larger than the largest IPO pre-paceex.

83
00:04:53,440 --> 00:04:55,997
And we're going to see $700 billion in

84
00:04:56,080 --> 00:05:00,397
capital expenditures this year.
And people are saying that the models are

85
00:05:00,480 --> 00:05:03,997
dangerous. Every chart
you look at about this AI

86
00:05:04,080 --> 00:05:05,757
moment looks kind of like this, right?

87
00:05:05,840 --> 00:05:10,397
It's a hockey stick. You have almost no
usage in 2023 and then all of a sudden

88
00:05:10,480 --> 00:05:12,317
you go through the roof, right? Again,

89
00:05:12,400 --> 00:05:17,837
from zero in 2023 pretty much to a
billion today on chat GPT. And it's not

90
00:05:17,920 --> 00:05:20,637
just the fact that we can have
conversations with these bots. It's that

91
00:05:20,720 --> 00:05:23,597
they're starting to do something,
starting to do things for us. They've

92
00:05:23,680 --> 00:05:27,117
learned to code. They've learned to take
action. If you looked at the data in

93
00:05:27,200 --> 00:05:31,677
2023, 2024, these chatbots
couldn't run autonomous

94
00:05:31,760 --> 00:05:35,037
code at all. Now, some of the
evaluations have them doing the

95
00:05:35,120 --> 00:05:37,997
equivalent of 16 hours of work on their

96
00:05:38,080 --> 00:05:40,957
own. And as the capabilities have grown,

97
00:05:41,040 --> 00:05:42,797
so have the businesses. Again, stop

98
00:05:42,880 --> 00:05:47,277
start 2023, 2024. We we have OpenAI and

99
00:05:47,360 --> 00:05:51,277
Anthropic. Now, they're going to do $50
billion in revenue this year. They're

100
00:05:51,360 --> 00:05:55,997
both headed towards trillion dollar IPOs
within the next year. And as everything

101
00:05:56,080 --> 00:05:59,997
has increased now, the government
is starting to panic. And we are in the

102
00:06:00,080 --> 00:06:04,077
middle of an insane news week where
Anthropic's top model, Fable, at least

103
00:06:04,160 --> 00:06:07,277
the one that's been released to the
public, has been restricted. You can't

104
00:06:07,360 --> 00:06:09,677
use it anymore. And so the same same

105
00:06:09,760 --> 00:06:12,077
goes for mythos. So it's clear that the

106
00:06:12,160 --> 00:06:15,117
ground is shifting beneath our feet. And

107
00:06:15,200 --> 00:06:17,517
in a moment like this, I believe there

108
00:06:17,600 --> 00:06:22,717
is a need for live journalism, the type
that we're here for today, to ask core

109
00:06:22,800 --> 00:06:26,797
questions about what the implications
are now that we're in this moment. The

110
00:06:26,880 --> 00:06:29,757
questions that I think we need to ask,
we're going to ask three core ones today

111
00:06:29,840 --> 00:06:33,357
throughout all the sessions are
what does this tech or what is this

112
00:06:33,440 --> 00:06:37,677
technology going to do next, right? What
happens? Where will it go if it fulfills

113
00:06:37,760 --> 00:06:41,917
its potential? If we don't know where
the technology is going to go next, we

114
00:06:42,000 --> 00:06:43,517
can't plan for what's going to happen.

115
00:06:43,600 --> 00:06:46,477
So I think at the core we have to figure
out where it's going next and we have

116
00:06:46,560 --> 00:06:50,237
the right people in the room here
to do it today. We also have to ask what

117
00:06:50,320 --> 00:06:53,677
happens if it works. If the technology
fulfills its potential, what happens

118
00:06:53,760 --> 00:06:56,317
next? And then of course what happens if

119
00:06:56,400 --> 00:06:59,357
it goes wrong? If all this investment

120
00:06:59,440 --> 00:07:04,157
ends up not panning out, what are the
downstream implications for technology,

121
00:07:04,240 --> 00:07:08,877
for our economy, and of course for the
foundational labs themselves? So we're

122
00:07:08,960 --> 00:07:10,957
going to do it today with a great lineup.
We're going to kick off with

123
00:07:11,040 --> 00:07:15,037
Aaron Levy, the CEO of Box, talking
about AI's most critical questions.

124
00:07:15,120 --> 00:07:18,397
We're going to then bring on the three
best AI infrastructure reporters in the

125
00:07:18,480 --> 00:07:22,077
world to talk about the AI buildout.
Uh after that, we're going to have Dallas

126
00:07:22,160 --> 00:07:27,437
Dolan, the TMT leader at PWC, talk
to us about what the real ROI is from AI

127
00:07:27,520 --> 00:07:29,997
tokens and what happens when you're
actually building with these things and

128
00:07:30,080 --> 00:07:34,717
what the business return is on that front.
And then Ron Roy will join me for

129
00:07:34,800 --> 00:07:38,317
a Q&A with you, right? Right. We're
going to do our Friday show on Thursday

130
00:07:38,400 --> 00:07:40,637
and we're going to encourage
your participation. So, if you have

131
00:07:40,720 --> 00:07:42,717
questions, bring them uh forward to us.

132
00:07:42,800 --> 00:07:46,557
We'll have a mic here and a mic there.
U we'll then have a coffee break. We have

133
00:07:46,640 --> 00:07:50,557
an espresso cart out back. Uh that was
very important for us in the planning.

134
00:07:50,640 --> 00:07:54,157
Um the fancier drinks, not during the
break because we want to keep it moving.

135
00:07:54,240 --> 00:07:56,637
So, if you want a fancier drink, you can

136
00:07:56,720 --> 00:07:59,197
go before or after. It's important that

137
00:07:59,280 --> 00:08:01,997
we give you this information. Uh we're

138
00:08:02,080 --> 00:08:06,397
then going to you know talk about the
whether uh the mythos situation in Fable

139
00:08:06,480 --> 00:08:10,077
whether whether there's actually a cyber
threat from these latest models or

140
00:08:10,160 --> 00:08:13,357
whether it's marketing and we're going
to joined by Alex Stamos the former

141
00:08:13,440 --> 00:08:15,997
chief security officer at Meta to do

142
00:08:16,080 --> 00:08:18,637
that. Um then we'll speak with Mike Kger

143
00:08:18,720 --> 00:08:20,957
the head of Anthropic Labs to talk about

144
00:08:21,040 --> 00:08:24,157
what building AI product is uh what how

145
00:08:24,240 --> 00:08:28,717
you build AI product natively today.
And then finally, Greg Brockman will close

146
00:08:28,800 --> 00:08:33,197
us out, the president and co-founder
of OpenAI, talking about that company's

147
00:08:33,280 --> 00:08:36,957
plans and what the frontier uh looks like.

148
00:08:37,040 --> 00:08:39,117
>> All right, everybody ready?

149
00:08:39,200 --> 00:08:44,237
>> All right, we put a lot
of work into this today. I

150
00:08:44,320 --> 00:08:46,797
promise you, we're going to do whatever
we can to make sure you have a great

151
00:08:46,880 --> 00:08:51,037
day. And thank you again for trusting
me with your time, trusting us with your

152
00:08:51,120 --> 00:08:54,717
time, and again filling those empty seats.
It's a real dream come true. So,

153
00:08:54,800 --> 00:08:59,880
thank you all. Let's hear
for you guys one more time.

154
00:09:01,840 --> 00:09:04,957
>> A quick adjustment here.
We want you sounding perfect.

155
00:09:05,040 --> 00:09:08,237
>> Okay. So, as this adjustment happens,
I'm going to invite our first guest.

156
00:09:08,320 --> 00:09:10,877
Aaron Levy is the CEO of Box. He's one

157
00:09:10,960 --> 00:09:15,517
of the most insightful and fun voices
on the future of this technology. He

158
00:09:15,600 --> 00:09:19,677
actually was the fourth guest ever on Big
Technology Podcast and the guest the

159
00:09:19,760 --> 00:09:23,837
only other time we did a live podcast
together. So, with that, I am thrilled

160
00:09:23,920 --> 00:09:29,160
to welcome Aaron Levy.
Please join me in welcoming him.

161
00:09:32,080 --> 00:09:33,197
>> All right. Great to see you.

162
00:09:33,280 --> 00:09:35,597
>> Good to see you. Thank you. Um I like

163
00:09:35,680 --> 00:09:38,077
that you set expectations with and some

164
00:09:38,160 --> 00:09:41,597
answers. Um uh so uh with that uh in

165
00:09:41,680 --> 00:09:45,997
your agenda. So I will try and provide
some some answers, not all the answers.

166
00:09:46,080 --> 00:09:49,437
>> I think we've done this. First of all,
every time we speak, I feel like I can't

167
00:09:49,520 --> 00:09:52,077
even get a question out and we're
already on to some of the

168
00:09:52,160 --> 00:09:53,437
>> Sorry. Can I just do my monologue now?

169
00:09:53,520 --> 00:09:56,957
Are you good? Okay. Thanks. Um >> I
I think it's I think we've done this

170
00:09:57,040 --> 00:10:00,557
enough to know that we are going to get
some answers uh to these questions and

171
00:10:00,640 --> 00:10:04,797
>> I'll do my best.
>> I think I don't want to start off by

172
00:10:04,880 --> 00:10:06,237
disparaging other interviews, but it

173
00:10:06,320 --> 00:10:08,957
doesn't doesn't always happen that way.

174
00:10:09,040 --> 00:10:12,717
>> Well, um you know,
we're we we we get to

175
00:10:12,800 --> 00:10:17,277
provide the answers because we're not
the lab that's being either regulated or

176
00:10:17,360 --> 00:10:20,637
dealing with lots of issues. So, I get
to just pontificate and there's no no

177
00:10:20,720 --> 00:10:23,597
consequence. So, it's great.
>> All right. So, we're going to ask you

178
00:10:23,680 --> 00:10:26,957
some of the questions about what's
happening. Um, le let's dive right into

179
00:10:27,040 --> 00:10:30,317
it. I'm going to start, you know, again,
since you're not at one of the labs,

180
00:10:30,400 --> 00:10:32,557
let's start to talk about the biggest

181
00:10:32,640 --> 00:10:34,717
most controversial moment now, uh, which

182
00:10:34,800 --> 00:10:37,837
is the anthropic fable situation. Um,

183
00:10:37,920 --> 00:10:42,077
let me put to you what I
call the Jassie mystery.

184
00:10:42,160 --> 00:10:44,797
So, what we know about >>
I'm sure he liked that name.

185
00:10:44,880 --> 00:10:46,957
>> Well, he didn't show up,
so we can talk about it in his

186
00:10:47,040 --> 00:10:52,957
>> absence. Okay. Uh so what we know about
the fable ban or the export controls on

187
00:10:53,040 --> 00:10:53,917
anthropic yeah

188
00:10:54,000 --> 00:10:57,277
>> is that Amazon found a vulnerability

189
00:10:57,360 --> 00:11:01,597
>> in the software and Andy Jasse maybe
made a call to Dario definitely made a

190
00:11:01,680 --> 00:11:04,957
call to the white house and then
very soon afterwards there were export

191
00:11:05,040 --> 00:11:08,717
controls that were put
on anthropics frontier model.

192
00:11:08,800 --> 00:11:12,077
>> Yeah, those two fact
patterns are probably not ideal.

193
00:11:12,160 --> 00:11:15,517
>> So the mystery is why did he do that?

194
00:11:15,600 --> 00:11:20,477
Yeah. And here's one hypothesis. This is
from Chimath. He said, "Google, Amazon,

195
00:11:20,560 --> 00:11:24,477
Microsoft, Meta now have a serious
non-zero opportunity to tank the

196
00:11:24,560 --> 00:11:28,557
Frontier Labs. Go to the government,
kneecap the lab's motion of putting the

197
00:11:28,640 --> 00:11:33,517
latest models out into the wild, become
the trusted gate gatekeeper between labs

198
00:11:33,600 --> 00:11:36,797
and the public by having
the labs go through their clouds."

199
00:11:36,880 --> 00:11:39,197
>> Okay. >> Plausible.

200
00:11:39,280 --> 00:11:42,717
>> Um I I would I would say um I mean,

201
00:11:42,800 --> 00:11:45,117
anything's plausible. I I prefer AAM's

202
00:11:45,200 --> 00:11:47,917
razor on this one. Um which is um you

203
00:11:48,000 --> 00:11:51,037
know ever since Mythos um you know

204
00:11:51,120 --> 00:11:55,117
Mythos very clearly was this
event that basically said

205
00:11:55,200 --> 00:11:58,077
you know AI is obviously getting
super powerful. It has all these risks

206
00:11:58,160 --> 00:12:01,517
associated with it. We are going
to give it to some you know small trusted

207
00:12:01,600 --> 00:12:04,157
partner network. They're going to go
evaluate their own tools. They're going

208
00:12:04,240 --> 00:12:07,917
to evaluate these capabilities. There's
been a lot of sort of it's a very kind

209
00:12:08,000 --> 00:12:11,677
of dramatic you know kind of roll out
of a technology. And I think what that has

210
00:12:11,760 --> 00:12:15,677
done is it's created this flywheel where
it almost incentivizes even more more

211
00:12:15,760 --> 00:12:19,677
drama and more research and more depth
in in uh you know security being the

212
00:12:19,760 --> 00:12:24,077
primary space in a way that we could
have already been doing since GPT4 if we

213
00:12:24,160 --> 00:12:26,877
wanted like you can go and deploy
these things to go find lots of

214
00:12:26,960 --> 00:12:30,557
vulnerabilities you can use them
offensively or defensively but mythos

215
00:12:30,640 --> 00:12:34,477
kind of created that extra air of of
seriousness and and uncertainty around

216
00:12:34,560 --> 00:12:38,397
it for good reason because it's
a incredibly powerful model so I go with a

217
00:12:38,480 --> 00:12:41,917
razor which is Amazon obviously has
security research teams they're like

218
00:12:42,000 --> 00:12:46,877
like any any company at that scale you
know we we we try and you know test and

219
00:12:46,960 --> 00:12:50,317
push the limits of models and in our
particular domain of use cases clearly

220
00:12:50,400 --> 00:12:53,197
at Amazon scale you have a very large
security team they're trying trying to

221
00:12:53,280 --> 00:12:57,117
jail break models all the time and so
almost by definition there's already a

222
00:12:57,200 --> 00:13:01,437
public private partnership on on all
forms of jailbreaking models you know

223
00:13:01,520 --> 00:13:04,557
trying to push them to the limits
as a part of that and especially with the

224
00:13:04,640 --> 00:13:09,117
surrounding atmosphere of mythos I think
it would be very natural for Andy to to

225
00:13:09,200 --> 00:13:11,437
either share that research or his
team to share that research and that

226
00:13:11,520 --> 00:13:15,277
escalates and then that that sort
of creates its own flywheel. But the idea

227
00:13:15,360 --> 00:13:18,557
that there's some kind of board
boardroom level, you know, sort of

228
00:13:18,640 --> 00:13:21,997
strategy meeting that says we we
now need to kind of like co-opt the

229
00:13:22,080 --> 00:13:25,277
technology become the only interface
to the government. This kind of puts us in

230
00:13:25,360 --> 00:13:27,757
the pole position. Um I think it's less

231
00:13:27,840 --> 00:13:31,197
likely that and more likely this is a uh

232
00:13:31,280 --> 00:13:33,757
this is a situation where the the mythos

233
00:13:33,840 --> 00:13:36,797
momentum continued. Fable obviously, you

234
00:13:36,880 --> 00:13:40,957
know, had ways of of getting back to kind
of the mythos level capability. Um,

235
00:13:41,040 --> 00:13:44,717
and researchers, you know, sort
of shared that information. And I don't I

236
00:13:44,800 --> 00:13:48,957
think there's like a basically,
you know, very limited small percentage

237
00:13:49,040 --> 00:13:52,877
chance that then Andy and team knew that
like the very next event would be they'd

238
00:13:52,960 --> 00:13:56,397
stop the model. Um, and and that's
not even good for Amazon like like

239
00:13:56,480 --> 00:13:59,917
strategically like Amazon makes plenty
of money the more that Fable gets used

240
00:14:00,000 --> 00:14:03,197
in the world. So I don't think you would
I don't think you would do some kind of

241
00:14:03,280 --> 00:14:05,997
like you know um kind of maneuvering to

242
00:14:06,080 --> 00:14:10,957
to to create this. So I I kind of just
go with this is it's a very chaotic kind

243
00:14:11,040 --> 00:14:14,317
of environment right now. The government
has you know only a few tools at their

244
00:14:14,400 --> 00:14:16,957
disposal at any given time to deploy
against these things. Those are going to

245
00:14:17,040 --> 00:14:20,797
be kind of blunt instruments. Um and uh
and this stuff is coming together very

246
00:14:20,880 --> 00:14:24,477
quickly because of you know in some
cases the lack of technical capability

247
00:14:24,560 --> 00:14:27,197
of the government compared to how
powerful these models are. It's like you

248
00:14:27,280 --> 00:14:29,677
don't you know when you see something
that seems very scary like oh my gosh

249
00:14:29,760 --> 00:14:33,117
the thing could be we can jailbreak
the model and get back to mythos level

250
00:14:33,200 --> 00:14:36,077
capability and mythos was the thing
we're supposed to be scared about then

251
00:14:36,160 --> 00:14:39,197
you know just like stop it like
that I think that's like a very natural

252
00:14:39,280 --> 00:14:43,597
reaction based on the atmosphere that
we've created in AI recently. So I just

253
00:14:43,680 --> 00:14:45,197
go with that as as the answer.

254
00:14:45,280 --> 00:14:48,397
>> I mean I like that you say
the atmosphere that we've created in AI

255
00:14:48,480 --> 00:14:50,957
lately. >> Everybody but me.
Yeah. Um well I mean

256
00:14:51,040 --> 00:14:55,277
as far as the company on the receiving
end of this though is anthropic and you

257
00:14:55,360 --> 00:14:58,237
know you talked about these mythical
capabilities they called the model

258
00:14:58,320 --> 00:15:01,997
mythos. They put in the documentation
that like it broke out of its

259
00:15:02,080 --> 00:15:06,397
containment and wrote the engineer while
he was having a sandwich in the park. Um

260
00:15:06,480 --> 00:15:08,957
is it that surprising that this
is one of the downstream impacts?

261
00:15:09,040 --> 00:15:11,437
>> Yeah I mean if you put that in your
announcement blog post you know people

262
00:15:11,520 --> 00:15:14,317
might be able to kind of extrapolate
and get pretty pretty pretty scared of

263
00:15:14,400 --> 00:15:18,957
things. I think it's interesting. So um
you know on the anthropic front first of

264
00:15:19,040 --> 00:15:20,557
all I have I have a huge amount of

265
00:15:20,640 --> 00:15:23,117
respect for the entire kind of stack of

266
00:15:23,200 --> 00:15:26,157
researchers and policy folks across AI.

267
00:15:26,240 --> 00:15:30,157
I happen to have disagreements with some
of the the categories but but I think

268
00:15:30,240 --> 00:15:32,157
there's a deep let's say if you were if

269
00:15:32,240 --> 00:15:34,077
you imagined a continuum of the most

270
00:15:34,160 --> 00:15:37,437
like you know if if you uh uh if you

271
00:15:37,520 --> 00:15:42,637
kind of had like like the most I mean
this only in like a polite way. It will

272
00:15:42,720 --> 00:15:45,757
sound impolite, but like I mean like
like if you're the most doomer on one

273
00:15:45,840 --> 00:15:50,157
end of the spectrum and the most like
like accelerationist on the other end of

274
00:15:50,240 --> 00:15:51,357
the spectrum. Here here's kind of the

275
00:15:51,440 --> 00:15:54,477
the views. The most doomer uh possible

276
00:15:54,560 --> 00:15:58,957
is was afraid of like GPD3 and GPD3 was

277
00:15:59,040 --> 00:16:02,317
going to like you know sort
of accelerate and and you know kind of

278
00:16:02,400 --> 00:16:06,237
achieve some kind of unstoppable
continual improvement. Um and you know

279
00:16:06,320 --> 00:16:09,517
the accelerationist says like we need
like fable 20 as soon as possible,

280
00:16:09,600 --> 00:16:13,517
right? So that's that's sort of the
continuum. I'm probably like I you know

281
00:16:13,600 --> 00:16:17,117
maybe twothirds up to the accelerationist
kind of side of things.

282
00:16:17,200 --> 00:16:20,797
But if you were on this on the doomer
and I I'm trying to say the polite

283
00:16:20,880 --> 00:16:23,837
version of doomer like you're deep
in AI safety. You're very scared of the

284
00:16:23,920 --> 00:16:26,957
technology. You think there's as much
likelihood of bad things happening as

285
00:16:27,040 --> 00:16:30,317
good things techn you know happening.
We have to win the race and control and

286
00:16:30,400 --> 00:16:32,957
kind of stamp down on the technology. We
don't want this to be this sort of thing

287
00:16:33,040 --> 00:16:36,157
that runs in the wild. If you're in
that end of the continuum the thing that

288
00:16:36,240 --> 00:16:37,517
happened this weekend is actually the

289
00:16:37,600 --> 00:16:40,397
best case scenario for you. So, so like

290
00:16:40,480 --> 00:16:45,757
you actually want there to be these
sort of like like valves and and like and

291
00:16:45,840 --> 00:16:48,397
like buttons in the government that just
is like we're just going to stop it.

292
00:16:48,480 --> 00:16:51,117
>> I mean, that's Dario's position. Do
you think he's happy with what's going on?

293
00:16:51,200 --> 00:16:52,637
>> Uh, you know, I'm not going to I won't

294
00:16:52,720 --> 00:16:56,157
try and guess uh any of that. Um uh but

295
00:16:56,240 --> 00:16:58,477
I I will just say if you had to

296
00:16:58,560 --> 00:17:00,957
establish a if you had to establish a

297
00:17:01,040 --> 00:17:05,597
regulatory regime that said we are going
to review models, we're going to push

298
00:17:05,680 --> 00:17:08,957
the limits of models and we're going
to have the ability to either roll back

299
00:17:09,040 --> 00:17:12,717
access to models or prevent their their
release in the first place. We want that

300
00:17:12,800 --> 00:17:17,997
to be a regulatory approach. You would
need an event like Fable to effectively

301
00:17:18,080 --> 00:17:21,677
create the precedent for that environment.
like like you're not going

302
00:17:21,760 --> 00:17:25,517
to wait for Congress to vote on this
being the new kind of you know process.

303
00:17:25,600 --> 00:17:28,797
You would you would kind of need
something that sort of shocks the system

304
00:17:28,880 --> 00:17:30,637
into that kind of regulatory framework.

305
00:17:30,720 --> 00:17:35,517
So all I'm saying is that if you're if
you were on this end of the continuum,

306
00:17:35,600 --> 00:17:39,837
this is actually an outcome that is sort
of almost desirable. Now maybe you you

307
00:17:39,920 --> 00:17:42,877
would wish that there would be more
a technical evaluation, more back and

308
00:17:42,960 --> 00:17:45,917
forth. Maybe you wish the policy people
were different on the other end. Who

309
00:17:46,000 --> 00:17:49,757
knows? But the idea that we now have
established that the government can

310
00:17:49,840 --> 00:17:51,517
press a button and prevent the roll out

311
00:17:51,600 --> 00:17:55,037
of AI um uh is is actually like a

312
00:17:55,120 --> 00:17:59,677
probably a positive update for an entire
cohort of of people. Now, unfortunately,

313
00:17:59,760 --> 00:18:03,117
I don't know that that any of your
guests represent that cohort, but I

314
00:18:03,200 --> 00:18:06,157
think you could easily get some people
that would be like this is the greatest

315
00:18:06,240 --> 00:18:09,997
thing that's ever happened in AI safety
because now we actually have we've

316
00:18:10,080 --> 00:18:13,277
created the case law essentially for this.
We we we now we know we know the

317
00:18:13,360 --> 00:18:17,037
tool exists. And then the next messy
process is when should we use the tool

318
00:18:17,120 --> 00:18:20,877
again? What should the real you know
kind of ongoing process look like? But I

319
00:18:20,960 --> 00:18:23,117
think that um you know probably I wish

320
00:18:23,200 --> 00:18:28,077
this wasn't the case but I think
practically in the next 3 to 5 years we

321
00:18:28,160 --> 00:18:30,797
probably have to end up in an
environment where models do get

322
00:18:30,880 --> 00:18:34,237
evaluated by the government. There is
a sort of collaborative approach between

323
00:18:34,320 --> 00:18:37,837
the government and research and and
the labs. You know the government has to

324
00:18:37,920 --> 00:18:41,677
kind of greenlight the the release
of the model. Um, I think it's probably

325
00:18:41,760 --> 00:18:45,117
become either too scary of a technology
or too economically powerful of a

326
00:18:45,200 --> 00:18:49,437
technology for governments to not want
to be in that position. I think that

327
00:18:49,520 --> 00:18:51,837
that that has massive implications when

328
00:18:51,920 --> 00:18:54,077
you when you kind of unpack it like just

329
00:18:54,160 --> 00:18:59,117
totally massive implications. One
being other countries now have far more

330
00:18:59,200 --> 00:19:02,317
incentive to stand up their own
sovereign AI initiatives. So, it's

331
00:19:02,400 --> 00:19:06,797
actually like maybe net negative for
the US economic position in AI that that

332
00:19:06,880 --> 00:19:10,317
this is the the outcome. Um, I think
somebody could take the other side and

333
00:19:10,400 --> 00:19:12,957
say, "No, we'll always have the most
powerful models, and so this puts us in

334
00:19:13,040 --> 00:19:15,517
the best position because now we can do
like horse trading with other countries

335
00:19:15,600 --> 00:19:19,677
of do you want access to our stuff."
So, I think it's it's, you know, I I like

336
00:19:19,760 --> 00:19:22,877
the fact that this is a super
interesting debate. Um, and I I have

337
00:19:22,960 --> 00:19:27,037
like a huge appreciation for every part
of the continuum um because I think it's

338
00:19:27,120 --> 00:19:30,637
like so intellectually interesting.
I still land on the hey, we probably want

339
00:19:30,720 --> 00:19:34,477
to treat this technology more as a
substrate technology and then regulate

340
00:19:34,560 --> 00:19:39,277
the applied use cases. So we should
regulate if you use AI to break into

341
00:19:39,360 --> 00:19:43,517
something. We should regulate if you
use AI to do bio, you know, kind of um

342
00:19:43,600 --> 00:19:46,397
research that that leads to dangerous
things. We shouldn't regulate the model

343
00:19:46,480 --> 00:19:49,357
itself. But I totally understand the other
views that are on the other end of

344
00:19:49,440 --> 00:19:52,637
this and and I think it's kind of very
natural with this important of a

345
00:19:52,720 --> 00:19:56,557
technology that it has to be somewhat of
a democratic process of how we decide to

346
00:19:56,640 --> 00:19:58,317
regulate it. >> Yeah.
You remember there were all those

347
00:19:58,400 --> 00:20:01,677
petitions six-month pause and everyone
kind of laughed at them. Yes.

348
00:20:01,760 --> 00:20:04,557
>> This is effectively the best
way to do that type of pause.

349
00:20:04,640 --> 00:20:09,037
>> Yeah. I mean this is if you were in the
P AI movement this is again like this is

350
00:20:09,120 --> 00:20:13,197
a great outcome. We we now have proven
how we can pause AAI. Um now it's an

351
00:20:13,280 --> 00:20:16,157
interesting kind of like mechanic
that they chose. It's sort of this export

352
00:20:16,240 --> 00:20:18,477
control thing, but effectively if you

353
00:20:18,560 --> 00:20:20,877
have an export control where non- US uh

354
00:20:20,960 --> 00:20:23,517
nationals can't can't use the technology

355
00:20:23,600 --> 00:20:26,157
like effectively that's PAI because your

356
00:20:26,240 --> 00:20:29,357
end API users of these of these models

357
00:20:29,440 --> 00:20:33,517
almost have no way to fully ensure
at all times that their end users don't

358
00:20:33,600 --> 00:20:36,317
sort of fall into some kind of you
know criteria that's off limits. So

359
00:20:36,400 --> 00:20:39,677
>> and there's there's already companies
that are pulling back. JP Morgan for

360
00:20:39,760 --> 00:20:42,637
instance has told its Hong
Kong users no more cloud

361
00:20:42,720 --> 00:20:45,197
>> right. So, so okay. So, now if you

362
00:20:45,280 --> 00:20:47,197
really war game this out like two to

363
00:20:47,280 --> 00:20:49,677
three, four more years out. This is um

364
00:20:49,760 --> 00:20:52,237
this is kind of interesting. So we we we

365
00:20:52,320 --> 00:20:55,437
have this like sovereign cloud um uh

366
00:20:55,520 --> 00:20:59,917
kind of comparison but but cloud you
know for better or worse basically

367
00:21:00,000 --> 00:21:04,237
became a commodity like whether you're
running in a cloud in uh you know

368
00:21:04,320 --> 00:21:06,877
there's like all lots of you know
performance implications like some some

369
00:21:06,960 --> 00:21:10,717
are faster some are cheaper but like
largely like you can get a web server

370
00:21:10,800 --> 00:21:13,357
you know built out wherever you are
in the world you can get storage built out

371
00:21:13,440 --> 00:21:16,637
wherever you are in the world we can
build sovereign clouds sovereign AI is a

372
00:21:16,720 --> 00:21:20,397
is a different kind of you know has
some intric intricacies uh that are

373
00:21:20,480 --> 00:21:24,317
different, right? Like like intelligence
just is not commoditized yet. We don't

374
00:21:24,400 --> 00:21:28,717
have everything having the same model
capability. So so the there's lots of

375
00:21:28,800 --> 00:21:32,397
really interesting implications which
is well what if like one set one country

376
00:21:32,480 --> 00:21:35,917
has access to you know frontier
intelligence before the other country

377
00:21:36,000 --> 00:21:38,557
you know what does that mean
geopolitically? What does that mean

378
00:21:38,640 --> 00:21:42,637
economically? Obviously now if you're
another country you have so much

379
00:21:42,720 --> 00:21:46,237
commercial incentive to make sure that
you can build out labs and have access

380
00:21:46,320 --> 00:21:51,117
to frontier intelligence as a kind
of hedge against the US. So who's a net

381
00:21:51,200 --> 00:21:55,517
winner in that? Probably China. Like and
so what's interesting is like you end up

382
00:21:55,600 --> 00:21:58,477
that I don't know if you know probably
most people saw the Doresh Jensen

383
00:21:58,560 --> 00:22:01,517
interview. And you can you can actually
it's a roarshack test. You can watch

384
00:22:01,600 --> 00:22:05,357
that through two totally different lenses.
You can have one lens which is

385
00:22:05,440 --> 00:22:07,197
which is like Dorash is totally right.

386
00:22:07,280 --> 00:22:10,557
We have this huge lead like this stuff
is so dangerous but if we control it

387
00:22:10,640 --> 00:22:13,517
then like we're going to control
everything. The other lens which is

388
00:22:13,600 --> 00:22:16,717
probably more of the Jensen angle
is like actually you know these other

389
00:22:16,800 --> 00:22:19,597
countries have a lot of incentive to also
get this right. And so even if it's

390
00:22:19,680 --> 00:22:23,357
like a $500 billion problem for them,
they just might deploy that much capital

391
00:22:23,440 --> 00:22:27,437
on this problem and and they will
eventually get it right. And so at the

392
00:22:27,520 --> 00:22:31,357
outcome actually we haven't gotten any
gains as better intelligence from the

393
00:22:31,440 --> 00:22:34,637
rest of the world. But what we have
lost is our economic superiority in this

394
00:22:34,720 --> 00:22:38,397
technology category because what we've
caused is a catalyst for all the other

395
00:22:38,480 --> 00:22:41,357
countries to have to build out their own
stack. And if they build out their own

396
00:22:41,440 --> 00:22:44,877
stack, it's probably going to be,
you know, chips from China, models from

397
00:22:44,960 --> 00:22:49,197
China, etc. which I don't have any
like you know you know reason to be be

398
00:22:49,280 --> 00:22:52,237
against other than just like I want
America to win the economic you know

399
00:22:52,320 --> 00:22:57,197
angles on this and so so this is sort
of this debate that happens uh on on like

400
00:22:57,280 --> 00:23:00,237
where should you apply export controls
and what are the implications of that

401
00:23:00,320 --> 00:23:04,557
downstream and even this week you know
post fable we see that you know you have

402
00:23:04,640 --> 00:23:06,957
models that are certainly not fable

403
00:23:07,040 --> 00:23:11,997
performance uh level but but opus 4.7
4.8 eight level uh which is a big update

404
00:23:12,080 --> 00:23:15,357
for a lot of people on what what is now
possible with with open weight models

405
00:23:15,440 --> 00:23:18,957
that we just didn't have you know
kind of visibility into you know before

406
00:23:19,040 --> 00:23:22,157
>> yeah I think you shared recently
that the openw weight model or open source

407
00:23:22,240 --> 00:23:24,637
models the capabilities are not that far

408
00:23:24,720 --> 00:23:29,197
away from the frontier and in fact like
as these models get smarter uh they're

409
00:23:29,280 --> 00:23:31,837
they're only they're almost going to
saturate with intelligence where there's

410
00:23:31,920 --> 00:23:35,197
not going to be such a big difference
between let's say the smartest open-

411
00:23:35,280 --> 00:23:38,717
source model and the frontier don't you
think and then so won't this push people

412
00:23:38,800 --> 00:23:42,397
to open source Well,
so the the big the big ongoing

413
00:23:42,480 --> 00:23:45,117
conversation and I think you have some
guests that that can really represent,

414
00:23:45,200 --> 00:23:49,357
you know, what what they're seeing on
the front lines is is do you have a sort

415
00:23:49,440 --> 00:23:53,917
of fast takeoff scenario of model
capability and progress um and with with

416
00:23:54,000 --> 00:23:57,997
some kind of continual learning kind of
you know self-improvement uh dynamic and

417
00:23:58,080 --> 00:24:01,437
then and then it stands to reason that
like the company with the most compute

418
00:24:01,520 --> 00:24:04,717
or the country with the most compute and
you get the fast takeoff you get sort of

419
00:24:04,800 --> 00:24:09,037
a a virtuous flywheel um that maybe
is sort of has some compounding you know

420
00:24:09,120 --> 00:24:12,157
benefit benefits to it that are just,
you know, unreachable by anybody else.

421
00:24:12,240 --> 00:24:16,477
That's a scenario. Another scenario
is is that that's another just incremental

422
00:24:16,560 --> 00:24:20,717
capability. Everybody kind of catches up
to it and you always have this sort of,

423
00:24:20,800 --> 00:24:25,037
you know, kind of two loops going at all
times and the with the closed the closed

424
00:24:25,120 --> 00:24:27,517
providers and the open providers
and they're and they're kind of always

425
00:24:27,600 --> 00:24:29,917
within 3 to six months of each other.

426
00:24:30,000 --> 00:24:34,237
The world is so different uh from a
market structure standpoint whether we

427
00:24:34,320 --> 00:24:38,877
end up in an outcome where we have sort
of an exponential progress in the in the

428
00:24:38,960 --> 00:24:43,117
models that you kind of continually
learn versus versus the closed source

429
00:24:43,200 --> 00:24:47,677
models uh and it's like a 5-year you
know gap in progress and that just goes

430
00:24:47,760 --> 00:24:51,677
again kind of exponential totally
different market structures um the the

431
00:24:51,760 --> 00:24:55,277
one that that where we have this
exponential progress is again it's

432
00:24:55,360 --> 00:24:57,837
probably actually net positive
for America in that case in which case the

433
00:24:57,920 --> 00:25:02,077
export controls probably worked Um,
it means our kind of top three, four

434
00:25:02,160 --> 00:25:04,557
labs have this incredible superiority.

435
00:25:04,640 --> 00:25:06,717
We control access to this technology.

436
00:25:06,800 --> 00:25:09,997
That's actually a good scenario. Like
like economically speaking, it might not

437
00:25:10,080 --> 00:25:13,917
be like a total net good scenario for
society, but it's good for the US. Let's

438
00:25:14,000 --> 00:25:16,957
just say that's one scenario. A lot
of people are betting that that is that

439
00:25:17,040 --> 00:25:20,637
that's where we're at with research.
The other scenario, and like China sits

440
00:25:20,720 --> 00:25:24,077
around uh and they probably bet on this
scenario, is no, we're going to be able

441
00:25:24,160 --> 00:25:27,597
to keep up. We're going to throw more
comput. We're going to get more data.

442
00:25:27,680 --> 00:25:30,397
we're going to build our own, you know,
flywheels. And it's always three months

443
00:25:30,480 --> 00:25:31,917
out and, uh, you know, kind of behind.

444
00:25:32,000 --> 00:25:35,437
And if it's three months behind and it's
an open weights provider, uh, that has

445
00:25:35,520 --> 00:25:38,397
more of a commoditization kind
of business model approach because they

446
00:25:38,480 --> 00:25:40,797
just want to, you know, sell more
infrastructure or chips or they just

447
00:25:40,880 --> 00:25:44,077
want to reduce our, you know,
superiority in the space, which is

448
00:25:44,160 --> 00:25:45,517
actually like strategic for China to do.

449
00:25:45,600 --> 00:25:47,917
Like they're everybody wonders like why
are they doing this open weight stuff?

450
00:25:48,000 --> 00:25:52,317
It's actually makes total sense. like
you're just reducing US's dominance in a

451
00:25:52,400 --> 00:25:55,757
in a field and it might be worth
a couple hundred billion dollars to do

452
00:25:55,840 --> 00:25:59,917
that for something that might be worth
you know 10 trillion dollars. So so if

453
00:26:00,000 --> 00:26:01,997
that keeps up because there is real

454
00:26:02,080 --> 00:26:07,517
economic advantage to to to doing so
then you have this new kind of you know

455
00:26:07,600 --> 00:26:11,357
sort of dynamic that plays out which
is maybe the layer of of of incremental

456
00:26:11,440 --> 00:26:15,197
value shift is effectively the applied
layer of AI. So, if you think about

457
00:26:15,280 --> 00:26:17,917
there's the lab layer and then there's
the applied layer, >> the cursors,

458
00:26:18,000 --> 00:26:23,277
>> the cursors, the the Harveys, the
the Sierras, the decagons, the boxes. Um,

459
00:26:23,360 --> 00:26:26,397
>> which is amazing because everyone
said they're just a thin wrapper on top of

460
00:26:26,480 --> 00:26:29,117
large language models, but now
maybe that's where the value comes.

461
00:26:29,200 --> 00:26:32,557
>> Yeah. So, so um and and you know,
it's one of these things which is like we

462
00:26:32,640 --> 00:26:33,997
just have to not be binary about it.

463
00:26:34,080 --> 00:26:37,757
like everything I'm saying I think
the frontier models still make way more

464
00:26:37,840 --> 00:26:41,197
money in the future than they do today
because what happens at the routing

465
00:26:41,280 --> 00:26:46,237
layer is you still sort of say hey I
want fable um or GPT5 you know five or

466
00:26:46,320 --> 00:26:49,677
whatever the next model will be I want
that to be the orchestrator I want I

467
00:26:49,760 --> 00:26:53,117
need like the super intelligence at
the orchestration layer and I need super

468
00:26:53,200 --> 00:26:57,037
intelligence at the review and sort
of like you know fix and check the work of

469
00:26:57,120 --> 00:27:00,957
the of the other agent and and so so
you have like a barbell you know maybe

470
00:27:01,040 --> 00:27:04,637
U-shaped model where you use frontier
intelligence but then ing in the middle.

471
00:27:04,720 --> 00:27:05,837
You can just say, "Nope, I'm going to

472
00:27:05,920 --> 00:27:08,717
take that to Neotron or or Kimmy 26 or

473
00:27:08,800 --> 00:27:13,597
GLM552 or whatever." And and then all
of a sudden it's like you have super high

474
00:27:13,680 --> 00:27:18,077
cost, you know, inference in one part
of the workload, super low cost, still

475
00:27:18,160 --> 00:27:21,677
pretty good inference in another part
of the workload, but who has the incentive

476
00:27:21,760 --> 00:27:26,237
to do that? It's the applied layer of AI
because because like the business model

477
00:27:26,320 --> 00:27:29,917
of the applied layer is obviously like
we our job is to you know give you the

478
00:27:30,000 --> 00:27:34,077
best model for the job not just
the model from just our lab. So it's cool

479
00:27:34,160 --> 00:27:38,477
because we actually now have a good kind
of push pull between Frontier Labs and

480
00:27:38,560 --> 00:27:42,557
the applied layer uh where where you
probably wouldn't want it to be that

481
00:27:42,640 --> 00:27:45,917
we're all sort of only in the orbit
of one or two companies you know

482
00:27:46,000 --> 00:27:48,157
commercially and economically.
You'd want you'd want to make sure that

483
00:27:48,240 --> 00:27:51,837
there's some good tension there. And so
I think that's kind of the direction

484
00:27:51,920 --> 00:27:56,477
things are things are headed between
kind of the token costs, the the open

485
00:27:56,560 --> 00:27:59,837
source models becoming so good and then
maybe even some of this regulatory

486
00:27:59,920 --> 00:28:03,837
dynamic. I think the applied layer sort
of incrementally gets gets more more of

487
00:28:03,920 --> 00:28:05,757
that opportunity which
is which is obviously great.

488
00:28:05,840 --> 00:28:09,837
>> So you've talked about open source and
you've just mentioned China. Um but what

489
00:28:09,920 --> 00:28:13,437
you can what can you
tell us about lashon fat?

490
00:28:13,520 --> 00:28:16,797
>> Um uh it's great memes. So folks, uh,

491
00:28:16,880 --> 00:28:19,917
Lashon Fat is a rumored open- source

492
00:28:20,000 --> 00:28:25,037
model from Mistral and has been
the subject of great fascination from the

493
00:28:25,120 --> 00:28:27,437
internet, wouldn't you say?
>> I there's great comedy. Uh,

494
00:28:27,520 --> 00:28:29,677
>> John, can we can we um show people what

495
00:28:29,760 --> 00:28:32,877
we're talking about? Let's roll image A.

496
00:28:32,960 --> 00:28:35,117
>> Uh, this is Lashon Fat,

497
00:28:35,200 --> 00:28:37,677
>> the number one model from Europe. Yes.

498
00:28:37,760 --> 00:28:40,317
>> My French world uh my French uh

499
00:28:40,400 --> 00:28:42,477
rudimentary French, it translates to the

500
00:28:42,560 --> 00:28:46,557
very fat kitten. Um, can we roll B? This

501
00:28:46,640 --> 00:28:50,477
is a standard day in Paris now.

502
00:28:50,560 --> 00:28:54,797
>> Yeah, but it does show something
that there's so much eagerness for AI that

503
00:28:54,880 --> 00:28:57,997
there's now fan art
for this potential model from MR.

504
00:28:58,080 --> 00:28:59,757
>> We we've reached we've reached a really

505
00:28:59,840 --> 00:29:02,317
important sort of phase in the cycle. Um

506
00:29:02,400 --> 00:29:06,237
uh so um uh so you know um I do think

507
00:29:06,320 --> 00:29:09,757
that it is kind of cool because what
some of the things that you maybe

508
00:29:09,840 --> 00:29:13,357
discounted the importance of all
of a sudden just like have so much more

509
00:29:13,440 --> 00:29:16,797
importance. Uh like I'm I'm watching
the um I don't know if folks are watching

510
00:29:16,880 --> 00:29:20,477
like the fireworks base 10 space as an
example. Like it's pretty cool that that

511
00:29:20,560 --> 00:29:24,157
that we now have these open weights
models that you can effectively post

512
00:29:24,240 --> 00:29:26,317
train on your particular domain of of

513
00:29:26,400 --> 00:29:29,197
task and you can go and you know eek out

514
00:29:29,280 --> 00:29:33,757
another five or 10 points of improve of
performance on these types of models. Um

515
00:29:33,840 --> 00:29:38,077
and again that's only possible because
of the misdraws because of the the the

516
00:29:38,160 --> 00:29:41,997
Chinese you know kind of open weights
models and the cost curve has gone down

517
00:29:42,080 --> 00:29:45,357
so much that there are actually some
situations which is oh actually maybe I

518
00:29:45,440 --> 00:29:49,277
should train a model just for my use
case because it it's literally like

519
00:29:49,360 --> 00:29:52,477
economically now it's not even like
I want control. It's it's actually

520
00:29:52,560 --> 00:29:56,717
economically advantageous for you to do
so. So, is this the answer to like the

521
00:29:56,800 --> 00:29:59,757
big token maxing hype where like
everyone's spending all this money on

522
00:29:59,840 --> 00:30:03,117
tokens and not really understanding
where they're going or whether there's

523
00:30:03,200 --> 00:30:05,997
an ROI that >> Yeah,
I mean I think that in practice

524
00:30:06,080 --> 00:30:10,077
that phase probably lasted two and a half
weeks. Um like like from the moment

525
00:30:10,160 --> 00:30:13,037
that meta took >> the media
over hype token max.

526
00:30:13,120 --> 00:30:17,037
>> No, I would never claim uh that.
Um we should go through your your various

527
00:30:17,120 --> 00:30:20,557
podcast headlines, but um uh the uh >>
we're not going to do that. We got we

528
00:30:20,640 --> 00:30:22,317
got the the cat pictures and that's it.

529
00:30:22,400 --> 00:30:26,397
Yeah. Um, no, but like I mean if I if
I had to like capture the the cycle of

530
00:30:26,480 --> 00:30:30,077
like the first token maxing, you know,
meta has a leaderboard, uses the most

531
00:30:30,160 --> 00:30:33,117
tokens possible to now, you know,
the last weeks of rumors of like we're

532
00:30:33,200 --> 00:30:36,477
shutting down everything, no one can use
AI. Um, you know, it's about a two-month

533
00:30:36,560 --> 00:30:40,557
period. So, it's, you know, people need
to like probably, you know, always kind

534
00:30:40,640 --> 00:30:44,717
of step back and just be like, okay,
are is what we're doing like a pragmatic,

535
00:30:44,800 --> 00:30:49,117
you know, thing for for for work or we
just sort of like like getting kind of

536
00:30:49,200 --> 00:30:53,357
hyped up too crazy on something. Um what
what's interesting is this this phase

537
00:30:53,440 --> 00:30:55,517
was so short that I don't ever think it

538
00:30:55,600 --> 00:30:57,757
reached outside of of the tech industry.

539
00:30:57,840 --> 00:31:02,637
I was um I I was uh I was at at we we
kind of host these CIO dinners in every

540
00:31:02,720 --> 00:31:06,157
city that we go to and I was uh we had
a dinner like within 3 days of like the

541
00:31:06,240 --> 00:31:08,397
token maxing like like initial spike on

542
00:31:08,480 --> 00:31:13,197
like Google trends like the word finally
emerged and like three people had heard

543
00:31:13,280 --> 00:31:15,997
about it and so I I feel
confident that it died

544
00:31:16,080 --> 00:31:17,997
>> right they haven't heard about because
their employees are outspending the

545
00:31:18,080 --> 00:31:21,357
tokens. >> Yeah fair point fair point. So,
but hopefully it will have completely died

546
00:31:21,440 --> 00:31:24,957
by the time it reaches reaches the rest
of the world and then we can just move

547
00:31:25,040 --> 00:31:26,797
to more normal environments and and but

548
00:31:26,880 --> 00:31:29,197
the the the thing that that is true of

549
00:31:29,280 --> 00:31:33,037
of the phenomenon is that these agents

550
00:31:33,120 --> 00:31:37,677
uh are just using hundreds of times more
tokens than they were before. And so,

551
00:31:37,760 --> 00:31:41,037
you know, when we launched our first
kind of AI use case within Box, our

552
00:31:41,120 --> 00:31:43,437
product, the average number of tokens

553
00:31:43,520 --> 00:31:45,357
that was was being used on a task was

554
00:31:45,440 --> 00:31:48,557
like 5,000 10,000 20,000 tokens. Now our

555
00:31:48,640 --> 00:31:53,597
latest agents might use a million tokens
or five million tokens on on executing a

556
00:31:53,680 --> 00:31:57,117
task. And so that that's you know
in some case that's a 100x increase uh in

557
00:31:57,200 --> 00:32:00,397
in number of tokens. And the reason for
that is obviously like what's happening

558
00:32:00,480 --> 00:32:03,757
is right as we solve one use case when

559
00:32:03,840 --> 00:32:07,757
you would think that we can drive down
the cost curve of that one use case. All

560
00:32:07,840 --> 00:32:11,837
of a sudden a model capability allows us
to now add another use case that's much

561
00:32:11,920 --> 00:32:16,157
harder. And then our appetite just grows
to solve harder and harder and harder

562
00:32:16,240 --> 00:32:19,437
problems. And so it's this funny thing
cuz people get confused. They say, "I

563
00:32:19,520 --> 00:32:22,957
thought AI was supposed to get
was supposed to be getting cheaper and and

564
00:32:23,040 --> 00:32:26,477
it's like yes, you can actually think
about it as cheaper if you looked at

565
00:32:26,560 --> 00:32:29,837
like the unit of intelligence.
The reason it's more expensive is because

566
00:32:29,920 --> 00:32:33,517
we're now taking on bigger tasks." And so
we're getting confused because we're

567
00:32:33,600 --> 00:32:37,117
like, why is this the one, you know,
tech trend that doesn't have sort of the

568
00:32:37,200 --> 00:32:40,157
Moors law phenomenon? It's because
actually, no, we're we're we're

569
00:32:40,240 --> 00:32:44,237
outrunning the improve the efficiency
improvements in our appetite for what

570
00:32:44,320 --> 00:32:47,597
these models can go and do. And so it's
actually what you need to do is have

571
00:32:47,680 --> 00:32:52,157
like a way to normalize the cost of of
of the tokens to the tasks that you can

572
00:32:52,240 --> 00:32:55,517
now deploy. And then if you look at that
then that starts to look cheaper on a

573
00:32:55,600 --> 00:32:59,197
per task basis. It's just again
our tasks are getting bigger or or more

574
00:32:59,280 --> 00:33:02,717
accurate or more effective. And uh
and that's going to happen for for quite

575
00:33:02,800 --> 00:33:05,037
some time. >> The reason
why token maxing took off as

576
00:33:05,120 --> 00:33:07,357
a concept is because people saw the

577
00:33:07,440 --> 00:33:09,677
exponential revenue right the fact that

578
00:33:09,760 --> 00:33:13,117
anthropic and open AI were at zero 2023.

579
00:33:13,200 --> 00:33:17,037
Now they're like gonna do $50 billion
this year at the very least. And so

580
00:33:17,120 --> 00:33:20,077
people are looking for an explanation
and either the question either the

581
00:33:20,160 --> 00:33:24,557
answer is this is real or some it's
somehow inflated and that's why people

582
00:33:24,640 --> 00:33:27,597
go to token maxing. So if I'm hearing
you're right, if I'm hearing you right,

583
00:33:27,680 --> 00:33:31,757
what you're saying is all this spend
is much more legit than some of the online

584
00:33:31,840 --> 00:33:36,717
discussion makes it out to be. Well,
uh I I think if I had to like officially

585
00:33:36,800 --> 00:33:41,437
provide my own takeaway uh
for my own point, um it would be

586
00:33:41,520 --> 00:33:44,717
it would it would sort of be
there's there's sort of always this

587
00:33:44,800 --> 00:33:48,317
experimentation phase of a new
technology and this happens to be a

588
00:33:48,400 --> 00:33:52,957
relatively expensive technology. So thus
the experimentation phase is expensive.

589
00:33:53,040 --> 00:33:57,597
Uh and then what will happen is a
enterprises will deploy AI um and then

590
00:33:57,680 --> 00:34:00,877
they'll sort of peel off they'll start
to see like where are the real use

591
00:34:00,960 --> 00:34:03,837
cases, where are the ones that aren't
as real. They'll wind down the ones that

592
00:34:03,920 --> 00:34:07,117
aren't as real. The ones that are real,
they'll then look at it and they'll say,

593
00:34:07,200 --> 00:34:10,957
"Is there a way to do it at a lower cost
once we understand it enough? Uh, or do

594
00:34:11,040 --> 00:34:13,997
we still need the frontier intelligence
for for everything we're doing?" And

595
00:34:14,080 --> 00:34:16,557
that's actually just like a pretty
normal, I think, process that that

596
00:34:16,640 --> 00:34:18,317
everybody's going through right now. Uh,

597
00:34:18,400 --> 00:34:20,797
but uh, but you know, I I I think about

598
00:34:20,880 --> 00:34:26,157
it like our engineering team, we are
not token maxers in the sense of like

599
00:34:26,240 --> 00:34:29,277
there's no leaderboard. We're not
incentivizing overuse of tokens. We're

600
00:34:29,360 --> 00:34:33,517
just saying use it as effectively as
possible to to get your work done faster

601
00:34:33,600 --> 00:34:37,597
and our growth rate of spend is
exponential and we're like totally happy

602
00:34:37,680 --> 00:34:41,837
about it. Like nobody internally
is other than like ah we got to like shift

603
00:34:41,920 --> 00:34:44,877
some things around and make sure we
plan for this even more next year. Like

604
00:34:44,960 --> 00:34:47,197
that's obviously a stressful
conversation but we're not stressed

605
00:34:47,280 --> 00:34:50,237
about the idea that we're spending on AI.
Like we're we're quite excited about

606
00:34:50,320 --> 00:34:53,677
the productivity gains that we get. And
so I think what's happening is is every

607
00:34:53,760 --> 00:34:56,477
enterprise is having to kind of go
through their own journey on that. like

608
00:34:56,560 --> 00:34:59,437
they're they're deploying it in some
teams and some teams are saying oh my

609
00:34:59,520 --> 00:35:02,397
gosh this is this is like the greatest
thing of all time and then other teams

610
00:35:02,480 --> 00:35:05,197
you kind of look at what they're doing
you can't see any kind of measurable

611
00:35:05,280 --> 00:35:08,237
improvement in the output of that
organization and so then you're like

612
00:35:08,320 --> 00:35:11,037
okay you know like maybe maybe it's not

613
00:35:11,120 --> 00:35:14,237
as effective there but I I am I I would

614
00:35:14,320 --> 00:35:18,397
say like I think it's very easy to kind
of capture one or two anecdotes and then

615
00:35:18,480 --> 00:35:20,717
and then kind of overextrapulate on the

616
00:35:20,800 --> 00:35:23,117
overall themes I would say the vast

617
00:35:23,200 --> 00:35:25,437
majority of the current agentic spend

618
00:35:25,520 --> 00:35:30,797
that's happening is sustainable because
because because partly because it's

619
00:35:30,880 --> 00:35:34,077
actually coming mostly from engineering
and engineering related related tasks

620
00:35:34,160 --> 00:35:38,237
and this is an audience that is kind
of technically capable of of you know

621
00:35:38,320 --> 00:35:40,957
determining whether they they like the
work you know product that's coming out

622
00:35:41,040 --> 00:35:45,037
of the AI um maybe as it gets to other
parts of knowledge work you know those

623
00:35:45,120 --> 00:35:48,477
people will not be as sort of familiar
with how to do the ROI measurement and

624
00:35:48,560 --> 00:35:52,557
then it'll get even messier but like so
far I think it's actually been largely

625
00:35:52,640 --> 00:35:55,517
you know totally reasonable >> okay
uh we have a couple minutes left.

626
00:35:55,600 --> 00:35:57,437
Let's do a small lightning round. Oh, no.

627
00:35:57,520 --> 00:36:01,037
>> Um, >> so, uh,
my first take here is that that

628
00:36:01,120 --> 00:36:03,277
series is really good.
It's going to be really good now.

629
00:36:03,360 --> 00:36:05,437
>> Yeah. >> What do you think?

630
00:36:05,520 --> 00:36:09,197
>> Uh, no. I agree. >> What? >> Elaborate.

631
00:36:09,280 --> 00:36:13,197
>> Oh, is it lightning round or is it
uh like you want to hear a fiveminute

632
00:36:13,280 --> 00:36:15,917
answer round? >> Give like a 60sec answer.

633
00:36:16,000 --> 00:36:18,717
>> I mean, what could be easier than
pressing a button on your phone and

634
00:36:18,800 --> 00:36:23,117
talking to it? And if you you know
at least based on the announcement they've

635
00:36:23,200 --> 00:36:27,117
taken Gemini uh which is a very good
model and been able to I don't know if

636
00:36:27,200 --> 00:36:30,877
it's fork or distill or something
within there is is sort of Gemini grade

637
00:36:30,960 --> 00:36:34,397
intelligence. So if you get Gemini grade
intelligence and voice on your phone

638
00:36:34,480 --> 00:36:38,957
press a button I think you're just going
to use that for a lot of things. Um and

639
00:36:39,040 --> 00:36:42,237
uh and then you know I think
the exciting thing is like imagine that

640
00:36:42,320 --> 00:36:45,597
hooked up to various apps on your phone
and you're like hey order this thing for

641
00:36:45,680 --> 00:36:50,397
me or you know you know you know go
and add this calendar entry like I think

642
00:36:50,480 --> 00:36:54,557
those are very plausible daily use
cases that that we will have. Um and it's

643
00:36:54,640 --> 00:36:57,677
exactly the sweet spot for Apple
to to kind of own that space.

644
00:36:57,760 --> 00:36:59,437
>> Yeah. No, I think Apple did it finally.

645
00:36:59,520 --> 00:37:00,797
That was that was good.
That was a good time.

646
00:37:00,880 --> 00:37:03,517
>> Are you going to tell me if my answer
is right at the end of each one?

647
00:37:03,600 --> 00:37:05,757
>> Okay. >> That's Yeah,
this is good. Okay. Okay.

648
00:37:05,840 --> 00:37:10,157
So we agree one for one. Okay. Um,
how about this one? Permanent underclass.

649
00:37:10,240 --> 00:37:12,477
>> Uh, I don't like this one. This one I

650
00:37:12,560 --> 00:37:15,677
don't like at all. Um, uh, not only do I

651
00:37:15,760 --> 00:37:19,997
disagree with it, but I I I think it's
just like a bad meme to have in the

652
00:37:20,080 --> 00:37:23,277
atmosphere. I think it's like not good
for college students coming into the

653
00:37:23,360 --> 00:37:25,677
into the workforce, uh, of having so

654
00:37:25,760 --> 00:37:30,317
much stress, uh, about, you know, what
company to join and and what's going to

655
00:37:30,400 --> 00:37:34,077
what's going to, you know, kind of play
out. I do think companies actually do

656
00:37:34,160 --> 00:37:37,837
the job market a disservice though
by not being as clear on their own

657
00:37:37,920 --> 00:37:41,757
philosophies on this. Um, which some
of it is reasonable because it's like, oh

658
00:37:41,840 --> 00:37:44,637
man, we're just like, we're getting
thrown through a loop. There's so much

659
00:37:44,720 --> 00:37:47,837
innovation. But I do think that companies
like need to be somewhat clear

660
00:37:47,920 --> 00:37:49,517
on, hey, here's how we want to use AI.

661
00:37:49,600 --> 00:37:53,757
We want to use AI to accelerate
our our work or accelerate our technical

662
00:37:53,840 --> 00:37:58,397
innovation or accelerate our ability
to hit customers. um versus, you know, no,

663
00:37:58,480 --> 00:38:01,997
we're we're actually like our metric
is as few employees as possible, you know,

664
00:38:02,080 --> 00:38:05,997
with AI, like you kind of do want to,
you know, be able to have some stance.

665
00:38:06,080 --> 00:38:07,357
And I think companies have been very

666
00:38:07,440 --> 00:38:10,157
confused. Um and that lets this meme

667
00:38:10,240 --> 00:38:13,197
somewhat persist uh uh you know, for uh

668
00:38:13,280 --> 00:38:15,757
for for for you know, the internet.

669
00:38:15,840 --> 00:38:17,357
>> Okay, I won't rate that one. Thank you.

670
00:38:17,440 --> 00:38:19,837
>> Um all right, last one.
Is is the SpaceX

671
00:38:19,920 --> 00:38:23,677
performance good or bad news
for OpenAI and Anthropic?

672
00:38:23,760 --> 00:38:27,437
>> Oh, well, it's obviously good news. Um,
>> you don't think Elon took some of their

673
00:38:27,520 --> 00:38:31,037
money because he pitched the market on
an AI company and that's where the money

674
00:38:31,120 --> 00:38:34,157
got funneled into? >> I'm not sure.
I've seen like a limit of

675
00:38:34,240 --> 00:38:37,917
appetite of of if for for I mean there

676
00:38:38,000 --> 00:38:41,437
there's there's a literal limit of money
in the world, but I don't know that I

677
00:38:41,520 --> 00:38:45,677
don't know that that is zero sum at this
stage. Uh so I I think um I think people

678
00:38:45,760 --> 00:38:50,637
are pretty clear that that you know if
the revenue of this entire category of

679
00:38:50,720 --> 00:38:54,077
the frontier models and the
infrastructure stack is is measured in

680
00:38:54,160 --> 00:38:58,557
the trillions then you can have you know
20 companies that that all take a piece

681
00:38:58,640 --> 00:39:02,557
of that at different layers of the stack.
So So I'm I'm not I'm not sure I

682
00:39:02,640 --> 00:39:04,477
I would be convinced
that that would be zero sum.

683
00:39:04,560 --> 00:39:06,717
>> Did you buy SpaceX? >> I actually did.

684
00:39:06,800 --> 00:39:11,917
>> Okay. Um, not, you know, uh, I I'm uh
I don't know if I'm embarrassed or not,

685
00:39:12,000 --> 00:39:15,117
but I'm not gonna say the amount of
shares, but I I wanted to like be a part

686
00:39:15,200 --> 00:39:17,837
of the movement. So, I'm on Robin Hood

687
00:39:17,920 --> 00:39:20,557
buying my my my retail shares of uh of

688
00:39:20,640 --> 00:39:22,957
SpaceX. I'm I'm up like 15 bucks now.

689
00:39:23,040 --> 00:39:26,957
Um, per share. Per share, but uh uh so

690
00:39:27,040 --> 00:39:30,877
I'm I'm happy. Yeah. >> Amazing.
Well, Erin, you know, you uh

691
00:39:30,960 --> 00:39:34,957
you answered my email uh when we were
just at the very start of this podcast,

692
00:39:35,040 --> 00:39:37,677
four episodes in, >> came
on the show. I feel like every

693
00:39:37,760 --> 00:39:40,397
single time we talk,
something crazy is happening.

694
00:39:40,480 --> 00:39:43,757
>> That's a guarantee at this point. So,
>> boy, are we in the thick of it, right?

695
00:39:43,840 --> 00:39:46,877
>> Yeah. Awesome. Good to see you,
>> sir. Thank you so much, Aaron. Thank

696
00:39:46,960 --> 00:39:50,120
you, everybody.

697
00:39:54,960 --> 00:40:00,157
>> Are we off to a good start?
Let's hear it.

698
00:40:00,240 --> 00:40:01,837
>> Let's hear it one more time for Aaron.

699
00:40:01,920 --> 00:40:05,160
We're gonna

700
00:40:07,200 --> 00:40:10,717
We uh we need your energy here for a
couple of reasons. First of all, we're

701
00:40:10,800 --> 00:40:15,277
live streaming this. We want to make
sure that your presence here is felt in

702
00:40:15,360 --> 00:40:19,357
our live stream. Um and then we go
from like two chairs to four chairs. So,

703
00:40:19,440 --> 00:40:23,837
we're just going to need you guys
to help us fill that time. Um so, uh AI

704
00:40:23,920 --> 00:40:26,957
infrastructure, right? We're in the
middle of the greatest infrastructure

705
00:40:27,040 --> 00:40:31,357
buildout of all time, bigger than cable,
bigger than the railroads. Um, we're

706
00:40:31,440 --> 00:40:35,837
going to see $700 billion in capital
expenditures this year. And that means

707
00:40:35,920 --> 00:40:39,357
that some of the biggest questions
that we have about this AI moment are

708
00:40:39,440 --> 00:40:43,517
actually things that we can find out
from the infrastructure discussion

709
00:40:43,600 --> 00:40:47,277
alone. Questions like, are we
overbuilding? Will these data centers

710
00:40:47,360 --> 00:40:49,837
that are announced ever get stood up?

711
00:40:49,920 --> 00:40:53,997
And of course, will all this added
compute actually lead to better AI

712
00:40:54,080 --> 00:40:58,477
models or is it misguided? And so to do
it, to have this discussion, we're going

713
00:40:58,560 --> 00:41:02,077
to speak with three of the best AI
infrastructure reporters in the world.

714
00:41:02,160 --> 00:41:04,717
Ana Gardez from the information, Max

715
00:41:04,800 --> 00:41:07,037
Churnney from Reuters, and Lauren Good

716
00:41:07,120 --> 00:41:12,680
of Wired. Let's give it up
for Max, Ana, and Lauren.

717
00:41:18,000 --> 00:41:21,037
>> Hey guys. >> Hi.
Hey, >> hey, hi everyone.

718
00:41:21,120 --> 00:41:23,517
>> So, um, let's start here. There's there

719
00:41:23,600 --> 00:41:27,037
have been headlines that, um, of the

720
00:41:27,120 --> 00:41:29,597
announced AI data centers that are

721
00:41:29,680 --> 00:41:32,077
supposed to come up, something like 50%

722
00:41:32,160 --> 00:41:36,557
of them are actually being built. Um, is

723
00:41:36,640 --> 00:41:40,957
that the case? And if so, why?
Ana, do you want to lead us off?

724
00:41:41,040 --> 00:41:42,957
>> Sure. I totally believe that statistic

725
00:41:43,040 --> 00:41:44,957
and I think it might actually be higher

726
00:41:45,040 --> 00:41:48,237
if you include announcements. Um because

727
00:41:48,320 --> 00:41:52,157
of all the planned data centers that are
underway, I think it's highly likely

728
00:41:52,240 --> 00:41:56,877
that many will be delayed due to higher
costs, how hard it is to get labor. But

729
00:41:56,960 --> 00:42:01,037
then the number that I'm keeping close
track of is announced projects versus

730
00:42:01,120 --> 00:42:03,037
actually projects that are being built.

731
00:42:03,120 --> 00:42:06,237
And I think we've seen some, you know,
pretty crazy announcements from

732
00:42:06,320 --> 00:42:11,197
companies like OpenAI with all of these
different 10 gawatt, 6 gawatt projects.

733
00:42:11,280 --> 00:42:14,877
And those are numbers that I'm paying
a lot of attention to because I think we

734
00:42:14,960 --> 00:42:18,317
need to sort of back into them and say,
okay, if you wanted 10 gigawatts by this

735
00:42:18,400 --> 00:42:21,997
date, how many do you have today, was
that a real commitment? How firm is that

736
00:42:22,080 --> 00:42:26,077
commitment? But on on projects that are
actually getting built, I do think, you

737
00:42:26,160 --> 00:42:29,677
know, 50% not really getting done
on time is is, you know, what I would

738
00:42:29,760 --> 00:42:33,437
expect. Wait, what percentage would you
say have been announced but not start

739
00:42:33,520 --> 00:42:36,237
not included in that 50% number?

740
00:42:36,320 --> 00:42:40,237
>> Um, you know, when I think of that
number, it's I mean, it's quite high.

741
00:42:40,320 --> 00:42:45,197
Like even if you think of OpenAI, for
example, announcing a 10 gawatt project.

742
00:42:45,280 --> 00:42:49,517
Um, you know, we're not going to see 10
gigawatts in the next couple of years or

743
00:42:49,600 --> 00:42:52,637
they're going to sort of spread out
that bet. And I think that's a really good

744
00:42:52,720 --> 00:42:57,037
area area for reporters to look at is if
you just back into announcements, it's a

745
00:42:57,120 --> 00:43:00,237
really good way to say, "Hey,
this project is not not on track."

746
00:43:00,320 --> 00:43:03,117
>> Yeah, it's kind of crazy because if
you look at the way that stocks have been

747
00:43:03,200 --> 00:43:06,717
traded publicly, a lot of the market
action that we're seeing is entirely

748
00:43:06,800 --> 00:43:10,077
dependent on those
announcements coming true.

749
00:43:10,160 --> 00:43:15,437
>> So, Max, let's go to you. Um, what
are the consequences going to be if this

750
00:43:15,520 --> 00:43:18,637
these announced buildouts
don't materialize?

751
00:43:18,720 --> 00:43:23,677
Well, I think there's uh a lot
of shareholders of these public companies

752
00:43:23,760 --> 00:43:24,957
that are going to be pretty frustrated.

753
00:43:25,040 --> 00:43:28,477
Um Microsoft, Amazon, etc. have been

754
00:43:28,560 --> 00:43:32,157
making big CACback spats as I think
everybody in this room knows. They've

755
00:43:32,240 --> 00:43:35,517
been going to the market to get debt
now, which is, you know, you know, I

756
00:43:35,600 --> 00:43:39,037
think shareholders would be pretty
interested. Uh in terms of the

757
00:43:39,120 --> 00:43:42,797
consequences though, I mean, I'm a chip
reporter, so that's sort of where I

758
00:43:42,880 --> 00:43:47,837
that's how I kind of think about it.
Um and it's very likely I would say that

759
00:43:47,920 --> 00:43:51,437
we're going to lead we're going to see
some overcapacity uh essentially. So

760
00:43:51,520 --> 00:43:53,757
like people are building especially
memory companies are building tons of

761
00:43:53,840 --> 00:43:57,117
factories right now and typically
the chip industry is again I'm sure

762
00:43:57,200 --> 00:44:02,157
everybody in this this room knows uh
tends to be cyclical. So the whiplash

763
00:44:02,240 --> 00:44:06,077
from this one might be pretty bad um
depending on exactly when it comes to an

764
00:44:06,160 --> 00:44:08,397
end if it does. >> I think
what Max is saying is that it's

765
00:44:08,480 --> 00:44:12,477
good for his job security because
the more news there is the more he'll have

766
00:44:12,560 --> 00:44:15,837
to report on. Yeah. Oh, you guys are not
going to get a break anytime soon. Not

767
00:44:15,920 --> 00:44:18,637
at all. >> No, I can't I'm
very I'm very curious if

768
00:44:18,720 --> 00:44:22,477
the if there is a bust what that looks
like exactly and what precipitates it. I

769
00:44:22,560 --> 00:44:23,757
think it'll be a lot of fun to cover.

770
00:44:23,840 --> 00:44:28,717
>> Well, one of the big questions >> fun.
I also think if I can say too, I

771
00:44:28,800 --> 00:44:33,037
think this is a particularly unique
time because not only do we have these uh

772
00:44:33,120 --> 00:44:36,877
incredibly highly valued, ambitious
frontier labs uh that are getting

773
00:44:36,960 --> 00:44:40,317
involved in these circular deals,
but also we're in the middle of a memory

774
00:44:40,400 --> 00:44:44,237
shortage. um there's this unrelenting
demand for compute and also this is a

775
00:44:44,320 --> 00:44:45,997
midterm election year and so I think

776
00:44:46,080 --> 00:44:47,597
you're going to see a lot of uh

777
00:44:47,680 --> 00:44:51,277
politicizing of the data centers too as

778
00:44:51,360 --> 00:44:54,797
uh you know sort of lawmakers try to

779
00:44:54,880 --> 00:44:58,637
appeal to their base because a lot
of people are very unhappy about data

780
00:44:58,720 --> 00:45:01,757
centers. >> Yeah.
I mean they're not pulling well at all.

781
00:45:01,840 --> 00:45:04,717
>> No. So um you know when you think about

782
00:45:04,800 --> 00:45:09,117
where the collapse might happen um
a popular thing for people to discuss is

783
00:45:09,200 --> 00:45:11,517
well maybe Nvidia which has been making

784
00:45:11,600 --> 00:45:16,717
such premiums on its hardware um it
can't sustain it or it gets caught. So

785
00:45:16,800 --> 00:45:18,397
Lauren you spent a lot of time with uh

786
00:45:18,480 --> 00:45:21,277
Jensen Wong from Nvidia um what do you

787
00:45:21,360 --> 00:45:24,717
think about Jensen would enable him to

788
00:45:24,800 --> 00:45:29,037
sustain uh Nvidia's lead or do you
think that some of these skeptics have a

789
00:45:29,120 --> 00:45:33,117
point? Uh yes and yes I think some of the

790
00:45:33,200 --> 00:45:34,797
skeptics absolutely have a point and I

791
00:45:34,880 --> 00:45:38,157
think once you reach the uh the sort of

792
00:45:38,240 --> 00:45:42,557
uh is it the zenith is there the nadar
I always get those two confused that

793
00:45:42,640 --> 00:45:46,797
Nvidia has um you know people are always
sort of looking to to take you down a

794
00:45:46,880 --> 00:45:49,517
peg and compete but I think

795
00:45:49,600 --> 00:45:52,957
>> Nvidia and Jensen has been incredibly

796
00:45:53,040 --> 00:45:57,517
good in Nvidia's history at sort
of pivoting the company at exactly the

797
00:45:57,600 --> 00:46:00,317
moment that they need to in order
to make sure that they've sort of caught

798
00:46:00,400 --> 00:46:02,877
the next wave. Um, and we certainly saw

799
00:46:02,960 --> 00:46:05,597
that happen with um, not only like you

800
00:46:05,680 --> 00:46:07,677
know the GPU to begin with and parallel

801
00:46:07,760 --> 00:46:10,077
processing, but then again with sort of

802
00:46:10,160 --> 00:46:14,957
pivoting towards crypto which ultimately
meant they were in good place for AI and

803
00:46:15,040 --> 00:46:18,797
now we see the company doing that by
addressing the inference market a lot

804
00:46:18,880 --> 00:46:22,157
more closely too and you know Jensen
coming out and making these big

805
00:46:22,240 --> 00:46:25,197
proclamations that actually they're
the biggest CPU maker in the world which I

806
00:46:25,280 --> 00:46:29,357
know Intel and AMD must be thrilled about.
So, uh, you know, I think he's

807
00:46:29,440 --> 00:46:32,797
very smart and very strategic and
that there's a good chance that they do

808
00:46:32,880 --> 00:46:35,597
maintain their dominance. >> I mean,
I think I think the question is

809
00:46:35,680 --> 00:46:38,557
like how much of the inference market
they're going to get. I mean, it's it's

810
00:46:38,640 --> 00:46:42,957
whether it's 80% or 40% or somewhere
in between or or 90%. I mean, I think

811
00:46:43,040 --> 00:46:45,357
that's what everybody's fighting out
at the moment. I don't think there's a

812
00:46:45,440 --> 00:46:48,557
question that they're going to have
some big chunk of it. Just it's just how

813
00:46:48,640 --> 00:46:50,957
much, you know? >> Yeah.
Yeah, I'm going to go to An in a

814
00:46:51,040 --> 00:46:54,237
moment, but uh Lauren, I just want you
to tell us a little bit of uh what it

815
00:46:54,320 --> 00:46:58,637
was like uh with Jensen
on a cover shoot for Wired.

816
00:46:58,720 --> 00:47:00,717
>> You're talking about
when I brushed his hair.

817
00:47:00,800 --> 00:47:04,957
>> That would be it. >> Yeah. Okay.
Uh so, >> wait, you actually No,

818
00:47:05,040 --> 00:47:08,077
>> I really did. >> Yeah.
Yeah. I'll be brushing I'll be

819
00:47:08,160 --> 00:47:11,837
lining up later to brush people's hair
if anyone would like to. Uh yeah, so I

820
00:47:11,920 --> 00:47:16,237
did a cover story on Jensen for Wired
a couple years ago and uh our Wired art

821
00:47:16,320 --> 00:47:19,597
department is worldclass by the way.
And so we had this big photo shoot set up

822
00:47:19,680 --> 00:47:23,277
and I asked if I could tag along to
the photo shoot because I just wanted 15

823
00:47:23,360 --> 00:47:27,037
more minutes with him to ask him some
follow-up questions about Blackwell. And

824
00:47:27,120 --> 00:47:30,797
so they let me tag along down to the
office in Santa Clara, set up this whole

825
00:47:30,880 --> 00:47:35,357
set for Jensen, who had like, you know,
exactly two minutes to give us. And as

826
00:47:35,440 --> 00:47:38,877
the photographer was looking
at Jensen under the bright lights, the

827
00:47:38,960 --> 00:47:40,877
photographer said, "Oh, he's got some

828
00:47:40,960 --> 00:47:43,917
flyaways and that like gorgeous silver

829
00:47:44,000 --> 00:47:46,237
hair that he has." And then he said,

830
00:47:46,320 --> 00:47:48,717
"Does anyone have a brush?"

831
00:47:48,800 --> 00:47:51,757
Silence across the set. No one at NVIDIA

832
00:47:51,840 --> 00:47:55,837
apparently had a brush. I was like,
"Okay." Uh, and as anyone who has long

833
00:47:55,920 --> 00:47:59,677
hair knows, you always have a brush.
So I said, "Well, I have a brush." So then

834
00:47:59,760 --> 00:48:01,437
I went to my backpack and got the brush.

835
00:48:01,520 --> 00:48:04,717
And then I said, "Okay, here's the brush."
And no one moved. And meanwhile,

836
00:48:04,800 --> 00:48:07,117
Jensen is like, "What are we doing here,

837
00:48:07,200 --> 00:48:12,077
folks?" And I I said, "Oh, okay." So,
I walked up to Jensen and I took out my

838
00:48:12,160 --> 00:48:14,637
brush and I got really close to him
and I said, "Jensen, we're about to get a

839
00:48:14,720 --> 00:48:17,197
lot more close." And he said, "Oh,

840
00:48:17,280 --> 00:48:21,917
Christ." And um and then I brushed his
hair and I have to say, I did bring some

841
00:48:22,000 --> 00:48:24,237
evidence of this.
I think it looks great, frankly.

842
00:48:24,320 --> 00:48:29,997
>> Oh, wow. I think it turned out really
well. So, yeah. Now, the unfortunate the

843
00:48:30,080 --> 00:48:32,237
unfortunate conclusion to this story
very quickly because I know we need to

844
00:48:32,320 --> 00:48:36,077
move on is that people were joking
afterwards like you know the net worth

845
00:48:36,160 --> 00:48:39,597
of every individual strand of hair
on that brush, right? Like like you could

846
00:48:39,680 --> 00:48:43,117
sell this on eBay and which I was
obviously not going to do as an ethical

847
00:48:43,200 --> 00:48:48,157
journalist. But then like several months
later, my back my backpack was in my it

848
00:48:48,240 --> 00:48:52,397
was my gym bag. It was in the back
of my car and um downtown San Francisco, I

849
00:48:52,480 --> 00:48:53,757
think you know where this is going. My

850
00:48:53,840 --> 00:48:56,797
car got broken into and the brush got

851
00:48:56,880 --> 00:48:59,837
stolen. So those thieves have no idea

852
00:48:59,920 --> 00:49:03,197
the value of what they got away with.

853
00:49:03,280 --> 00:49:05,277
>> They are cloning Jensen as we speak.

854
00:49:05,360 --> 00:49:09,517
>> Yes. I'll see you in the back
afterwards if you on your hairbrushed.

855
00:49:09,600 --> 00:49:12,957
>> Ana, can you talk to us a little bit
about what we've kind of hinted at at

856
00:49:13,040 --> 00:49:16,717
this point that there's this, you know,
Nvidia, the common p perception on

857
00:49:16,800 --> 00:49:20,557
Nvidia is their GPUs were great
for training, but you can actually do the

858
00:49:20,640 --> 00:49:24,637
inference or the act of using a model
on a variety of of different chips. And

859
00:49:24,720 --> 00:49:28,317
once people once these labs train
their models and they're happy with their

860
00:49:28,400 --> 00:49:30,477
models, most of the computing is going

861
00:49:30,560 --> 00:49:35,437
to go to these inference chips. Um,
and therefore, even if Nvidia has a bet

862
00:49:35,520 --> 00:49:39,677
there, they're not going to be able to
sustain their dominance. Is is that what

863
00:49:39,760 --> 00:49:40,957
do you think about that? Is that a

864
00:49:41,040 --> 00:49:43,917
potential flaw in the armor for Nvidia?

865
00:49:44,000 --> 00:49:48,157
I think it is a flaw and I think there's
if anyone's going to sort of attack

866
00:49:48,240 --> 00:49:51,997
Nvidia's dominance they're going to do
it on inference like you said and there

867
00:49:52,080 --> 00:49:53,837
is sort of a massive effort underway

868
00:49:53,920 --> 00:49:56,317
right now to make all inference chips

869
00:49:56,400 --> 00:50:00,957
under the sun work well and so every
single company that buys Nvidia chips

870
00:50:01,040 --> 00:50:04,637
and is spending a lot of money on NVIDIA
chips is trying really hard to make

871
00:50:04,720 --> 00:50:08,557
these other other chips work whether
it's in-house chips from Google or

872
00:50:08,640 --> 00:50:12,877
Amazon or even in OpenAI's case and
potentially Anthropic's case you know do

873
00:50:12,960 --> 00:50:17,277
we make our own inference chip.
Um, so I'd have a hard time, you know,

874
00:50:17,360 --> 00:50:21,197
believing that none of those chips are
going to pan out, but you know, they

875
00:50:21,280 --> 00:50:25,837
might not tackle the bulk of the inference
workload even. But then again,

876
00:50:25,920 --> 00:50:29,197
even if they do 10% of your inference,
maybe you're saving enough money that

877
00:50:29,280 --> 00:50:32,557
you think the effort is worth it and it
gives you negotiating leverage with

878
00:50:32,640 --> 00:50:36,557
NVIDIA. If they know that you have
an in-house ship team, you know, Jensen's

879
00:50:36,640 --> 00:50:38,797
going to be a little bit worried when
negotiating with you when you're playing

880
00:50:38,880 --> 00:50:41,437
hard ball with him. So, I think
everyone's going to have to have an

881
00:50:41,520 --> 00:50:45,997
inference chip answer to Nvidia.
But when I talk to data center companies

882
00:50:46,080 --> 00:50:49,597
about what they're seeing, um, you know,
some data center companies don't really

883
00:50:49,680 --> 00:50:53,117
care what chip goes inside of their data
center, but they they try to get hints

884
00:50:53,200 --> 00:50:56,237
from the companies that they're working
with. Um, on the data center side,

885
00:50:56,320 --> 00:51:00,317
they're seeing a lot of NVIDIA
and in the instances that I'm seeing where

886
00:51:00,400 --> 00:51:03,357
there are non- NVIDIA chips, it's
because of some sort of financial backs

887
00:51:03,440 --> 00:51:06,957
stop. So, I do wonder how much how long
that will have to keep being the case

888
00:51:07,040 --> 00:51:11,037
because that will definitely hinder
non- Nvidia chips if they need special

889
00:51:11,120 --> 00:51:13,997
financing arrangements
to get inside the data center.

890
00:51:14,080 --> 00:51:15,837
>> Am I wrong in thinking the AI world is

891
00:51:15,920 --> 00:51:17,997
sort of separating on two poles? There's

892
00:51:18,080 --> 00:51:21,917
the Nvidia OpenAI pole and the anthropic

893
00:51:22,000 --> 00:51:25,197
Google Amazon poll, right? So, it's

894
00:51:25,280 --> 00:51:28,237
almost two separate ecosystems competing

895
00:51:28,320 --> 00:51:33,757
with each other. I think OpenAI is
investing a lot in non- Nvidia hardware.

896
00:51:33,840 --> 00:51:35,837
>> Are they trying
to move away from Nvidia?

897
00:51:35,920 --> 00:51:38,237
>> Yeah. Yeah, they are.
Um they're they're

898
00:51:38,320 --> 00:51:40,077
using chips from other companies. Um

899
00:51:40,160 --> 00:51:42,397
Cerebrris is a good example and they

900
00:51:42,480 --> 00:51:47,117
have their own in-house chip. Um so
I think maybe publicly they're they're

901
00:51:47,200 --> 00:51:50,797
doing some big announcements with Nvidia
and I believe it's 5 gawatts they have

902
00:51:50,880 --> 00:51:54,397
to deploy in the next couple of years
on Ver Rubin. That's that's a lot and that

903
00:51:54,480 --> 00:51:58,797
that in itself might hinder them from
doing more. But I think um I think Sam

904
00:51:58,880 --> 00:52:01,117
Alman wants to diversify from Nvidia.

905
00:52:01,200 --> 00:52:03,757
>> Why? >> Because you
know just like any other

906
00:52:03,840 --> 00:52:08,237
company it's so expensive and I don't
you know they have their own in-house

907
00:52:08,320 --> 00:52:11,517
silicon effort. I don't think they they
think that they know their model better

908
00:52:11,600 --> 00:52:16,157
than anyone else like their their AI
model and so I think they think that

909
00:52:16,240 --> 00:52:20,077
they're the best to develop a chip
that can run their model efficiently.

910
00:52:20,160 --> 00:52:21,917
>> Max, you're back and forth to Taiwan

911
00:52:22,000 --> 00:52:25,037
very frequently. um when we think about

912
00:52:25,120 --> 00:52:27,517
the position of Taiwan and Taiwan uh

913
00:52:27,600 --> 00:52:30,557
semiconductor in this world um you know

914
00:52:30,640 --> 00:52:35,117
the fact that it's in this like tenuous
geopolitical place is something people

915
00:52:35,200 --> 00:52:39,357
tend to be like oh okay and then move
on right away and I want to hear your

916
00:52:39,440 --> 00:52:41,917
perspective on whether uh the

917
00:52:42,000 --> 00:52:47,037
independence of Taiwan and the stability
of TSMC is this like hidden black swan

918
00:52:47,120 --> 00:52:48,797
event that's just kind of in plain sight

919
00:52:48,880 --> 00:52:50,717
that people are not paying attention

920
00:52:50,800 --> 00:52:54,477
enough attention to. I mean, it it is uh

921
00:52:54,560 --> 00:52:57,837
for some reason the modern world decided
to to decided to put all of our chip

922
00:52:57,920 --> 00:53:01,197
manufacturer or most of it next to uh

923
00:53:01,280 --> 00:53:05,597
Kim Jong-un and the rest of it
is in China is in is in Taiwan. I I don't

924
00:53:05,680 --> 00:53:08,397
really understand who decided that or
why we decided that, but that's that's

925
00:53:08,480 --> 00:53:12,717
the nature of the beast. Um chip
companies, the design companies in the

926
00:53:12,800 --> 00:53:16,957
US do not plan for this. The contingency
the contingency plan, excuse me, is

927
00:53:17,040 --> 00:53:20,397
something along the lines of like,
well, we're all kind of screwed if China

928
00:53:20,480 --> 00:53:23,597
invades. um which okay, but like there's

929
00:53:23,680 --> 00:53:26,717
no real plan there. Um so, and when I

930
00:53:26,800 --> 00:53:30,877
say invades, I don't necessarily mean
like a literal invasion. I think what's

931
00:53:30,960 --> 00:53:34,077
a lot more likely is some kind of soft
power exchange like what happened in

932
00:53:34,160 --> 00:53:37,997
Hong Kong over time. Um I think
realistically that's that's a more

933
00:53:38,080 --> 00:53:42,157
likely option. Although I'm not saying
Xiinping won't just invade. Like he

934
00:53:42,240 --> 00:53:44,237
absolutely will and he said he would. Um

935
00:53:44,320 --> 00:53:49,437
which could be catastrophic for TSMC and
the modern world again. Um, I I do know

936
00:53:49,520 --> 00:53:52,477
that chip companies, as I said,
there really isn't much planning and much

937
00:53:52,560 --> 00:53:55,437
consideration of it because it's it I
think people in the industry just think

938
00:53:55,520 --> 00:53:58,717
it's an event that's just so crazy,

939
00:53:58,800 --> 00:54:02,557
complicated, and out of this world that
nobody's going to do anything about it.

940
00:54:02,640 --> 00:54:04,797
So, >> Max, do you want to talk about what

941
00:54:04,880 --> 00:54:08,717
happens when you try to visit TSMC?

942
00:54:08,800 --> 00:54:11,117
>> They send me to the gift shop.

943
00:54:11,200 --> 00:54:13,597
>> Uh, so >> you're in Taipei
and that's where you go.

944
00:54:13,680 --> 00:54:17,517
>> That's Yep. That's basically it. They
they got a a museum where they have like

945
00:54:17,600 --> 00:54:21,357
um I don't know a bunch of chips that
they've made over the years. One of the

946
00:54:21,440 --> 00:54:24,397
cerebras chips is in there. They're very
proud of it. The dinner plate size one.

947
00:54:24,480 --> 00:54:27,677
Um and I've never been to the museum,
but uh that's where they wanted to

948
00:54:27,760 --> 00:54:30,797
always that's where they always want
to take me. I think uh Wired is one of the

949
00:54:30,880 --> 00:54:34,157
few organizations, news outlets that's
actually been able to go inside of one

950
00:54:34,240 --> 00:54:38,557
of the factories. Um and they wrote
an interesting story about it. Um but

951
00:54:38,640 --> 00:54:41,757
>> yeah, I do not get to go to the TSMT
Fabs unfortunately. They're not they're

952
00:54:41,840 --> 00:54:44,237
not all that interesting inside. like
they're cool if you've never seen one,

953
00:54:44,320 --> 00:54:47,837
but um they mostly look the same. It's
like a bunch of white boxes with robots

954
00:54:47,920 --> 00:54:51,597
moving around. Um so it's not like
people that know what they're looking

955
00:54:51,680 --> 00:54:54,157
for. It's helpful because you can like
count the number of EUV machines in

956
00:54:54,240 --> 00:54:57,037
there and figure out roughly how many
chips they can make and like do that

957
00:54:57,120 --> 00:54:59,997
kind of thing and figure out what kind
of tools they're using. But um if the

958
00:55:00,080 --> 00:55:04,077
tours that most chip companies give
are not particularly enlightening um in

959
00:55:04,160 --> 00:55:06,077
general, >> yeah, >> they
are fascinating though. Yeah,

960
00:55:06,160 --> 00:55:07,677
they're cool, but they're
just they're not useful.

961
00:55:07,760 --> 00:55:09,997
>> Yeah. >> In in like
a meaningful reporting sense

962
00:55:10,080 --> 00:55:13,197
or an investor's, you know, from an
investor's perspective or something like

963
00:55:13,280 --> 00:55:15,837
that. >> I had gotten
very excited about TSMC.

964
00:55:15,920 --> 00:55:19,437
Did a ton of reporting, spoke with former
employees there and then got the

965
00:55:19,520 --> 00:55:22,317
courage up and I called them and I said,
"I'm ready to come to Taiwan." And

966
00:55:22,400 --> 00:55:25,837
they're like, "You can come to the gift
shop." And I was like, "All right, well,

967
00:55:25,920 --> 00:55:28,477
I guess I'm not coming."
>> My understanding is that our freelancer

968
00:55:28,560 --> 00:55:32,557
who got in there spent about a year
negotiating with the PR team to get in.

969
00:55:32,640 --> 00:55:35,997
>> I've been working on it for three. So
if she has any tips like I could I could

970
00:55:36,080 --> 00:55:37,597
definitely >> call up Virginia Hefernon.

971
00:55:37,680 --> 00:55:40,877
>> I will. Absolutely. >> Lauren,
you've been inside a fab in

972
00:55:40,960 --> 00:55:44,877
Arizona. What's it like? >> Yeah.
Well, like this was the um Intel

973
00:55:44,960 --> 00:55:47,837
Fab 52, which is their most advanced

974
00:55:47,920 --> 00:55:50,477
fab. It's a 2 nanometer fab and it is on

975
00:55:50,560 --> 00:55:55,277
the level of what TSMC does in terms
of 2nmter, but of course at a fraction of

976
00:55:55,360 --> 00:55:58,317
the scale of what TSMC does. And to
Max's point about, well, what is our

977
00:55:58,400 --> 00:56:02,157
redundancy or our resiliency plan in the
event of an invasion of Taiwan? It seems

978
00:56:02,240 --> 00:56:04,237
like right now the US government is very

979
00:56:04,320 --> 00:56:06,317
interested in what Intel can do in its

980
00:56:06,400 --> 00:56:10,077
fabs, but um it's pretty small. Uh and

981
00:56:10,160 --> 00:56:12,637
so yeah, I went to Chandler, Arizona. Um

982
00:56:12,720 --> 00:56:15,997
saw Fab 52. Um you know, got the bunny

983
00:56:16,080 --> 00:56:19,677
bunny suit on and um like Max said, it's

984
00:56:19,760 --> 00:56:23,757
it's interesting to see uh everything is

985
00:56:23,840 --> 00:56:26,237
roboticized. It's like if you had this

986
00:56:26,320 --> 00:56:31,437
um utopian fever dream of what the future
of manufacturing looked like and

987
00:56:31,520 --> 00:56:33,917
everything was white and everyone
was dressed in all white and everything's

988
00:56:34,000 --> 00:56:37,277
like being you know moving around. They
call them foops that actually carry the

989
00:56:37,360 --> 00:56:41,197
silicon around above head and then they
just sort of automatically go down to

990
00:56:41,280 --> 00:56:45,677
get like etched and carved and stamped
and everything. Um it look it looks like

991
00:56:45,760 --> 00:56:47,837
it feels and looks like science fiction.

992
00:56:47,920 --> 00:56:51,917
But from a reporting perspective, oftent
times the companies will cover up the

993
00:56:52,000 --> 00:56:55,757
names of the vendors on the different
machines and you may be able to get some

994
00:56:55,840 --> 00:56:58,157
insight. Um, for example, there there

995
00:56:58,240 --> 00:57:01,997
were ASML UEV machines in these giant

996
00:57:02,080 --> 00:57:05,517
bays the size of school buses, but then
you might see some spaces where they're

997
00:57:05,600 --> 00:57:08,877
like, well, we have two more coming
and we're waiting for them. And so you're

998
00:57:08,960 --> 00:57:10,237
like, okay, what does that mean? you

999
00:57:10,320 --> 00:57:13,597
know, UEV is obviously a very um you

1000
00:57:13,680 --> 00:57:15,997
know, a a very coveted sort of

1001
00:57:16,080 --> 00:57:20,957
technology and they're also very
expensive and you know, so so you get

1002
00:57:21,040 --> 00:57:24,557
some insights from that, but uh
primarily it's just kind of um it's good

1003
00:57:24,640 --> 00:57:27,437
for context for better understanding
how this all works, >> right?

1004
00:57:27,520 --> 00:57:30,077
>> I was not allowed to go
to this tour just for what it's worth.

1005
00:57:30,160 --> 00:57:31,357
>> Max, I think you're
doing something wrong.

1006
00:57:31,440 --> 00:57:34,317
>> Max, you got to work on your emails.

1007
00:57:34,400 --> 00:57:36,957
>> Um so, >> just kidding. That means
you're doing actually a great job.

1008
00:57:37,040 --> 00:57:40,557
>> Thanks, Lauren. Uh there was an there
was an interesting uh moment in the

1009
00:57:40,640 --> 00:57:44,477
Darkesh Jensen interview which I guess
will come up in every session today. Um

1010
00:57:44,560 --> 00:57:46,637
where where basically what Jensen is

1011
00:57:46,720 --> 00:57:49,597
trying to get across to Daresh is if you

1012
00:57:49,680 --> 00:57:53,677
do not provide chips to China um
they're going to be there's going to be a

1013
00:57:53,760 --> 00:57:57,437
constraint. They will develop their own
models on their own chips. They could

1014
00:57:57,520 --> 00:58:01,837
become the global standards and then put
export controls on the US. Does anybody

1015
00:58:01,920 --> 00:58:05,437
here think that that is a legitimate
concern or do you think Jensen just

1016
00:58:05,520 --> 00:58:08,957
wants to sell chips in China?
wants to sell it to chips in China.

1017
00:58:09,040 --> 00:58:10,797
>> Okay, >> I mean it might
be a legitimate concern

1018
00:58:10,880 --> 00:58:15,837
in some sense, but Nvidia has run out
of places to grow. Uh they can't sell to

1019
00:58:15,920 --> 00:58:18,237
any more countries. They can't sell
to any more hyperscalers. They can't sell

1020
00:58:18,320 --> 00:58:21,677
to any more even they're going after
the enterprise market, which is messy and

1021
00:58:21,760 --> 00:58:23,997
complicated and difficult to actually

1022
00:58:24,080 --> 00:58:26,237
make a lot of money in. So, China is a

1023
00:58:26,320 --> 00:58:31,597
big market that has basically no access
to Nvidia chips right now. I mean, you

1024
00:58:31,680 --> 00:58:34,077
know, if you're getting those kinds
of gross margins, like, yeah, of course,

1025
00:58:34,160 --> 00:58:37,757
that's like an obvious just a very
obvious place to to be able to sell

1026
00:58:37,840 --> 00:58:41,837
chips, the company's kind of obsessed
with it at the moment. And it's why

1027
00:58:41,920 --> 00:58:43,357
Jensen goes to Washington all the time.

1028
00:58:43,440 --> 00:58:47,277
That's why he >> insisted
on going to China. I mean, it's

1029
00:58:47,360 --> 00:58:50,477
it seems pretty clear to me.
>> Anisa, any thoughts?

1030
00:58:50,560 --> 00:58:54,637
>> Max spends a little bit more time
thinking about that than I do. Um,

1031
00:58:54,720 --> 00:58:58,237
>> I I do understand, you know, obviously
he wants to sell to China. It's a huge

1032
00:58:58,320 --> 00:59:01,757
market and they're taking a hit because
they can't sell to China. So, you can't

1033
00:59:01,840 --> 00:59:05,997
really answer the question without
acknowledging that. Um, but I I do think

1034
00:59:06,080 --> 00:59:09,117
that, you know, from the national
security perspective, um, Chinese

1035
00:59:09,200 --> 00:59:13,517
companies are developing AI chips um,
no matter what happens. Um, so, you know,

1036
00:59:13,600 --> 00:59:17,117
whether we can or can't, I do think that
they'll continue to develop alternatives

1037
00:59:17,200 --> 00:59:19,837
to Nvidia. >> I mean,
their manufacturing process is

1038
00:59:19,920 --> 00:59:22,477
is a lot worse. like the one of the uh

1039
00:59:22,560 --> 00:59:24,077
research firms just published an

1040
00:59:24,160 --> 00:59:26,957
analysis of the sort of one of the new

1041
00:59:27,040 --> 00:59:29,917
Chinese chips like homemade um by Smick

1042
00:59:30,000 --> 00:59:32,877
and it's like it's good but they're they

1043
00:59:32,960 --> 00:59:37,117
run up to up to the limit of like they
can't buy EUV machines um the big ones

1044
00:59:37,200 --> 00:59:40,077
that Lauren was just talking about.
So if you can't buy those things it's just

1045
00:59:40,160 --> 00:59:43,437
fundamentally difficult to make
transistors below a certain size and

1046
00:59:43,520 --> 00:59:47,357
they can't do it effectively. >> Yeah.
And to Max's point, Nvidia only

1047
00:59:47,440 --> 00:59:50,157
has so much more room to grow. I mean
they're at the point where they have

1048
00:59:50,240 --> 00:59:53,517
become a major investor in Corewave and
then Corewave is using that money to buy

1049
00:59:53,600 --> 00:59:57,517
Nvidia chips and so it's it's you know
it's really and there was an earnings

1050
00:59:57,600 --> 00:59:59,677
call um earlier this year maybe late

1051
00:59:59,760 --> 01:00:04,637
last year where uh Nvidia said that they
had a domestic buyer of some of the

1052
01:00:04,720 --> 01:00:07,837
chips that they weren't able to sell to
China though they not disclose the buyer

1053
01:00:07,920 --> 01:00:10,717
and so they're at the point where
they're basically yeah trying to sell

1054
01:00:10,800 --> 01:00:14,477
them wherever they can and so you'd have
to imagine the motivation there is they

1055
01:00:14,560 --> 01:00:19,357
want to sell chips to China >> no
those leftover H20s that they were Yeah.

1056
01:00:19,440 --> 01:00:23,197
China that somebody else wanted them cuz
there's people want silicon right now so

1057
01:00:23,280 --> 01:00:26,637
badly any any kind will do.
>> I think they sold about was it $600

1058
01:00:26,720 --> 01:00:28,077
million worth of those? >> Yeah.

1059
01:00:28,160 --> 01:00:31,437
>> Yeah. Okay. So, a minute left. Let's
do this an actual lightning round. So, yes

1060
01:00:31,520 --> 01:00:35,437
or no? Uh do you think that uh are
you all bitter or less pills? Which is

1061
01:00:35,520 --> 01:00:38,477
basically like the reason why there's
all this infrastructure is because the

1062
01:00:38,560 --> 01:00:42,557
AI labs think the more you know GPUs and
chips you string together the better the

1063
01:00:42,640 --> 01:00:45,597
models will get. Do you think that it
will continue to improve as these

1064
01:00:45,680 --> 01:00:49,917
buildouts uh you know continue
to to explode in size or no?

1065
01:00:50,000 --> 01:00:51,757
>> Wait, ask the question once more.

1066
01:00:51,840 --> 01:00:56,397
>> Is is it is it is the investment worth
it in in these data centers or are they

1067
01:00:56,480 --> 01:00:59,117
ultimately like not going to have much
better models even though they have a

1068
01:00:59,200 --> 01:01:01,357
lot more chips? >> Are we
going down the line here? You

1069
01:01:01,440 --> 01:01:05,837
start with us. So you start
with me. Um generally yes.

1070
01:01:05,920 --> 01:01:10,237
>> Okay. >> Mhm. >> Max.
>> Uh not not sure. Sorry. Okay. Okay.

1071
01:01:10,320 --> 01:01:14,237
>> I think it depends on the amortization
what ends up happening there.

1072
01:01:14,320 --> 01:01:17,437
>> Okay. Ana, you announced
your news on Twitter, right?

1073
01:01:17,520 --> 01:01:19,997
>> I did and LinkedIn. >> Why
don't you tell everybody where where

1074
01:01:20,080 --> 01:01:22,717
you're going next? >> Oh,
um July 6th, I'm starting a new job

1075
01:01:22,800 --> 01:01:25,277
at the Wall Street Journal.
>> All right, let's hear for Ana.

1076
01:01:25,360 --> 01:01:28,920
>> Thank you. >> Congratulations.

1077
01:01:29,520 --> 01:01:31,997
And thank you, Lauren and Max, for joining
us as well. Let's hear for them,

1078
01:01:32,080 --> 01:01:36,120
the whole panel. Thank you guys.

1079
01:01:40,240 --> 01:01:43,037
Thank you very much. So, so often the

1080
01:01:43,120 --> 01:01:47,357
story of AI's trajectory is is told
without the people using it. We hear

1081
01:01:47,440 --> 01:01:49,997
about these concepts called token maxing

1082
01:01:50,080 --> 01:01:52,237
that are talked about in the abstract

1083
01:01:52,320 --> 01:01:55,437
and then one day you see a chart and you

1084
01:01:55,520 --> 01:01:59,677
never actually speak to the people
spending the tokens. So I think to fully

1085
01:01:59,760 --> 01:02:03,117
understand how this technology
is progression progressing and its

1086
01:02:03,200 --> 01:02:08,237
potential to meet its the way that
people want it to go, we actually we

1087
01:02:08,320 --> 01:02:11,837
actually have to speak with the people
who are building with the tokens. And

1088
01:02:11,920 --> 01:02:13,117
that's why I'm thrilled to bring on

1089
01:02:13,200 --> 01:02:17,037
Dallas Dolan, the the TMT leader at PWC,

1090
01:02:17,120 --> 01:02:19,597
who is actually implementing this AI,

1091
01:02:19,680 --> 01:02:24,237
looking at the costs, and making sure
that it's worth the money that they're

1092
01:02:24,320 --> 01:02:28,637
spending on it, and will take us deep
into the token maxing and the budgeting

1093
01:02:28,720 --> 01:02:34,760
conversation. So folks, let's
give it up for Dallas Dolan of PWC.

1094
01:02:36,720 --> 01:02:38,637
>> Great to see you, Dallas.
>> The tables have turned. You're

1095
01:02:38,720 --> 01:02:42,237
interviewing me. This is good. Uh what
do you think about Aaron Levy was here

1096
01:02:42,320 --> 01:02:43,997
like 10 minutes ago and he talked about

1097
01:02:44,080 --> 01:02:49,917
how token maxing was a BS uh um media
narrative effectively and it never

1098
01:02:50,000 --> 01:02:52,077
really happened.
>> Do you agree with that?

1099
01:02:52,160 --> 01:02:53,117
>> I don't know if I
totally agree with it.

1100
01:02:53,200 --> 01:02:57,277
We were actually talking backstage just
before he came on and um I think there's

1101
01:02:57,360 --> 01:03:01,837
absolutely like we'll call it above
above the you know above the plane sort

1102
01:03:01,920 --> 01:03:04,877
of commentary that's out there
that uh we're all seeing in in both the

1103
01:03:04,960 --> 01:03:08,557
mainstream media as well as like what
plays really well on on social media on

1104
01:03:08,640 --> 01:03:11,277
X and other places um and in podcasts.

1105
01:03:11,360 --> 01:03:12,877
And then there's there's the reality

1106
01:03:12,960 --> 01:03:15,597
within a lot of organizations, but it's

1107
01:03:15,680 --> 01:03:20,237
it's happening enough where the behaviors,
I'll call them, are slightly

1108
01:03:20,320 --> 01:03:23,917
problematic from a cost and from an ROI
point of view that it's real, right? You

1109
01:03:24,000 --> 01:03:27,437
can't say that every circumstance
is a problem necessarily, but it's coming

1110
01:03:27,520 --> 01:03:30,237
through in a way that, you know, it's
enough to think about and say, hey, are

1111
01:03:30,320 --> 01:03:31,837
we doing this the right way, right?

1112
01:03:31,920 --> 01:03:35,277
Broadly speaking from an enterprise
strategy and also then from even a

1113
01:03:35,360 --> 01:03:36,877
broader ecosystem point of view.

1114
01:03:36,960 --> 01:03:40,797
>> Yeah. So, there's this moment now where
we're seeing actually a counter to token

1115
01:03:40,880 --> 01:03:42,717
maxing. Uh, it's called token

1116
01:03:42,800 --> 01:03:44,797
minimizing. And you're seeing companies

1117
01:03:44,880 --> 01:03:48,797
like AT&T and Meta um get really serious

1118
01:03:48,880 --> 01:03:53,837
about cost. We also know that Uber
of course uh spent their entire budget in

1119
01:03:53,920 --> 01:03:57,197
less than half a year. Um, who do you
think is going to win out at the end of

1120
01:03:57,280 --> 01:04:00,877
the day, the token minimizers or the
token maxers given the definition of

1121
01:04:00,960 --> 01:04:02,477
token maxing you you just gave us?

1122
01:04:02,560 --> 01:04:05,117
>> Yeah, I mean I think here here's
the good news. The good news is I don't I

1123
01:04:05,200 --> 01:04:08,157
don't think there's a winner and loser
that's going to be defined by did you

1124
01:04:08,240 --> 01:04:11,197
token max or not token max. I think the
winner and loser is going to be defined

1125
01:04:11,280 --> 01:04:15,837
by did you outcome max or not. Um it's
in part going to be a function of how

1126
01:04:15,920 --> 01:04:18,797
did you incentivize people which goes
into that leaderboard and the things

1127
01:04:18,880 --> 01:04:21,597
that got a lot of folks will say in
trouble or certainly in the news. Right?

1128
01:04:21,680 --> 01:04:24,637
So there's that piece of it. It's
the how do you actually want to encourage

1129
01:04:24,720 --> 01:04:29,437
people to do it without encouraging the
wrong things. It's take it too far etc.

1130
01:04:29,520 --> 01:04:32,397
Right? So there's that piece and or spend
too much money. And then the other

1131
01:04:32,480 --> 01:04:34,957
part of it is going to be actually
I think from a planning point of view

1132
01:04:35,040 --> 01:04:38,717
within an organization and I deal
with this as well. So I sit on the the the

1133
01:04:38,800 --> 01:04:43,277
boards for for our US and and our global
organization at PWC and we talk about

1134
01:04:43,360 --> 01:04:46,637
this a lot. In fact we talk about
this even contextually from a comparative

1135
01:04:46,720 --> 01:04:49,997
point of view within different industries.
So it's not just saying hey

1136
01:04:50,080 --> 01:04:53,517
does one organization as a services
company spend more or less than another

1137
01:04:53,600 --> 01:04:57,037
but also how are we compared and spending
you know against let's say like

1138
01:04:57,120 --> 01:04:59,997
a tech company who's you know got
a bunch of engineers and doing the coding

1139
01:05:00,080 --> 01:05:04,157
there. And I think what we're looking
for is a what's like the baseline,

1140
01:05:04,240 --> 01:05:08,077
right? So it's it's benchmarking with
industry out with within the industry,

1141
01:05:08,160 --> 01:05:12,157
outside the industry and against a given
benchmark and saying, okay, are we

1142
01:05:12,240 --> 01:05:16,077
actually getting an ROI? Are we way
outside the bounds in terms of what we

1143
01:05:16,160 --> 01:05:18,797
think the spend is or what we know
the spend is, right? Some of the companies

1144
01:05:18,880 --> 01:05:21,117
that you've named are certainly
companies that we're aware of and work

1145
01:05:21,200 --> 01:05:24,797
with too. Um, so we have some decent
intel on what's happening within the

1146
01:05:24,880 --> 01:05:27,677
industry from that point of view. But it
is going to be that like what outcomes

1147
01:05:27,760 --> 01:05:31,277
did you get? And I will tell you, I've
seen some things that people have built,

1148
01:05:31,360 --> 01:05:34,397
for example, in the deal space that's
just incredible. Like really, really

1149
01:05:34,480 --> 01:05:38,237
amazing output. It's taking, you know,
thousands of hours worth of work and

1150
01:05:38,320 --> 01:05:40,397
creating a product within seven minutes.

1151
01:05:40,480 --> 01:05:44,717
The cost is very high, but it's
an amazing output and it does show very

1152
01:05:44,800 --> 01:05:48,637
well from a consumer or from a,
you know, a customer point of view. It's

1153
01:05:48,720 --> 01:05:51,277
also very exciting for, you know, the
folks who work in the industry who don't

1154
01:05:51,360 --> 01:05:54,317
have to stay up all night maybe working
on something. That's great. There's

1155
01:05:54,400 --> 01:05:57,357
other processes that people haven't even
attacked yet. And I think the question

1156
01:05:57,440 --> 01:06:00,797
is when do you actually go after each
one of those and say how am I going to

1157
01:06:00,880 --> 01:06:03,837
outcome maximize for each one of these
processes. That's actually when you're

1158
01:06:03,920 --> 01:06:08,077
going to see I think the benefits come
through. There will be maximization of

1159
01:06:08,160 --> 01:06:11,597
spend but it'll be maximized in a way
that you're actually getting an outcome

1160
01:06:11,680 --> 01:06:13,757
from it not just because
you spent the most money.

1161
01:06:13,840 --> 01:06:15,517
>> So you're already
doing ROI calculations.

1162
01:06:15,600 --> 01:06:17,997
>> Absolutely. Yeah.
>> So you're probably far ahead than most

1163
01:06:18,080 --> 01:06:22,717
folks. Um how do you determine the ROI
and and what percentage of your projects

1164
01:06:22,800 --> 01:06:25,437
would you say are actually
generating a positive ROI?

1165
01:06:25,520 --> 01:06:27,757
>> Yeah. Well, I'll tell you this. I
know I mean even from an external point of

1166
01:06:27,840 --> 01:06:31,517
view, I think MIT did did a recent
publication talking about like what sort

1167
01:06:31,600 --> 01:06:35,277
of uh activities are really, you know,
replaceable just with generative AI in

1168
01:06:35,360 --> 01:06:38,317
general, especially like in the vision
and and you know, human interactive

1169
01:06:38,400 --> 01:06:39,757
space. And I think they came up with a

1170
01:06:39,840 --> 01:06:43,837
an outcome of about 23%. Um, as in you

1171
01:06:43,920 --> 01:06:47,757
wouldn't use a human to do 23%
of the work. I look at that and say in our

1172
01:06:47,840 --> 01:06:51,437
business it's probably not as high
as 23%, but it's going to be some

1173
01:06:51,520 --> 01:06:55,197
percentage of every single thing
that people do, right? So it's it's a

1174
01:06:55,280 --> 01:06:58,237
different mathematical equation. It's
is absolutely going to be measured for our

1175
01:06:58,320 --> 01:07:00,637
business. It's going to be measured in
hours. The same thing that's going to be

1176
01:07:00,720 --> 01:07:03,757
done in the engineering space for a lot
of these companies as well. It's a

1177
01:07:03,840 --> 01:07:07,117
measurement of hours. It's are you
making the person more efficient? Are

1178
01:07:07,200 --> 01:07:09,917
they creating more output, you know,
for the amount of time they're spending on

1179
01:07:10,000 --> 01:07:12,717
it, right? So is that lines of code
they're producing or is it quality

1180
01:07:12,800 --> 01:07:16,637
product? Which this is an interesting
thing, right? like I could build a

1181
01:07:16,720 --> 01:07:20,797
larger slide deck or I can write way
more lines of code, let's say, but am I

1182
01:07:20,880 --> 01:07:22,957
actually getting a product at the end
of the day that people are saying, "Oh,

1183
01:07:23,040 --> 01:07:25,917
yeah, that's something I'm willing
to use." And I had that conversation a

1184
01:07:26,000 --> 01:07:29,677
bunch actually at uh at Tech Week in New
York a couple of weeks ago with a number

1185
01:07:29,760 --> 01:07:33,117
of a number of founders and also
a number of the investors right from Andre

1186
01:07:33,200 --> 01:07:36,477
and others who were talking about
the types of companies that they're

1187
01:07:36,560 --> 01:07:38,557
investing in. I think it's really
interesting, right? If you're an

1188
01:07:38,640 --> 01:07:41,837
investor, you say, "No, like you know,
what does your product do, you know, for

1189
01:07:41,920 --> 01:07:44,397
the person who's using, especially if
it's a coding product or something along

1190
01:07:44,480 --> 01:07:47,517
those lines." If you're not indicating
that you're able to help them actually

1191
01:07:47,600 --> 01:07:50,477
build a product, right? Come out with
something a product or service that's

1192
01:07:50,560 --> 01:07:53,277
better that their customer is going
to want to use, the fact that it produces

1193
01:07:53,360 --> 01:07:57,117
more lines of code or a longer slide
deck or a longer pitch deck or a longer

1194
01:07:57,200 --> 01:08:00,957
memo is not better. Right? I think we
we're going to come to a determination,

1195
01:08:01,040 --> 01:08:04,157
Alex, really soon here where we start to
say, "Oh, wait a second. Like, what am I

1196
01:08:04,240 --> 01:08:07,997
really trying to get to here? Is it
more more more more or is it actually a

1197
01:08:08,080 --> 01:08:09,677
better product that I'm trying to get?

1198
01:08:09,760 --> 01:08:11,597
>> Right. But that sort of goes to the way

1199
01:08:11,680 --> 01:08:13,837
that the foundational labs and the cloud

1200
01:08:13,920 --> 01:08:16,957
players are working with you. The things

1201
01:08:17,040 --> 01:08:21,837
that I've heard is that these companies
are making it so that when you plug into

1202
01:08:21,920 --> 01:08:24,717
their systems, you're going
to spend a lot of tokens

1203
01:08:24,800 --> 01:08:28,157
>> and they don't really want you
to be able to measure them. Um, are you

1204
01:08:28,240 --> 01:08:31,357
finding that? So the I think the big

1205
01:08:31,440 --> 01:08:34,637
counter to you know the the um maybe the

1206
01:08:34,720 --> 01:08:38,717
the marketing and sales approach there
is in the control plane that's being

1207
01:08:38,800 --> 01:08:41,517
used to actually help companies make

1208
01:08:41,600 --> 01:08:44,717
decisions on what model's being used or

1209
01:08:44,800 --> 01:08:48,717
what interface is being used to do
specific activities. And this is where

1210
01:08:48,800 --> 01:08:51,437
this concept of central planning
and I say that as someone who's who's

1211
01:08:51,520 --> 01:08:54,557
traveling to China next week. This
concept of centralized planning within

1212
01:08:54,640 --> 01:08:57,917
an enterprise is going to be so important.
It's the you are not allowed

1213
01:08:58,000 --> 01:09:02,717
to check the weather five time five five
times a day using you know clawed 5, you

1214
01:09:02,800 --> 01:09:05,597
know, 5.7 or whatever it might be,
right? Like it's unacceptable, right?

1215
01:09:05,680 --> 01:09:09,757
Mythos is not a good use case
for weather checking. Um even if you're

1216
01:09:09,840 --> 01:09:12,397
worried like I was last night about the
tornadoes going through the Midwest and

1217
01:09:12,480 --> 01:09:15,677
trying to get back here so I wasn't
late for Alex. It's still not a good use,

1218
01:09:15,760 --> 01:09:18,557
right? You can do that still with a
weather app or if you really want to use

1219
01:09:18,640 --> 01:09:21,597
something inexpensive like there's
a lot of those options out there. I think

1220
01:09:21,680 --> 01:09:24,717
what's going to happen, you know,
you the the term control plane is sort of

1221
01:09:24,800 --> 01:09:28,637
out there. It's relatively newish. Um,
it's got governance elements, it's got

1222
01:09:28,720 --> 01:09:32,877
cost control elements, it's got access
to data elements, it's got um, I'll see

1223
01:09:32,960 --> 01:09:35,837
even like the the human interaction
elements like what can you put into it,

1224
01:09:35,920 --> 01:09:37,677
not only what you can get out of it.

1225
01:09:37,760 --> 01:09:42,477
that is going to be the the the you know
the the surface of place so to say where

1226
01:09:42,560 --> 01:09:46,637
people are going to spend a lot of the
time in the energy at the AT&Ts and the

1227
01:09:46,720 --> 01:09:49,997
metas and the services companies and a
lot of these other spaces including in

1228
01:09:50,080 --> 01:09:54,077
the engineering space but also in the
sales in marketing and in in the back

1229
01:09:54,160 --> 01:09:57,597
office as well as they look at other
activities that they will do better but

1230
01:09:57,680 --> 01:10:00,877
they will do better with the right tool.

1231
01:10:00,960 --> 01:10:04,637
quick analogy, right? Like the fact
of the matter is we are simply not best

1232
01:10:04,720 --> 01:10:08,957
suited driving the Lamborghini to go
pick up milk. Those two things just

1233
01:10:09,040 --> 01:10:11,437
don't align, right? Unless you live
in South Beach, in which case it's totally

1234
01:10:11,520 --> 01:10:16,317
okay. Um, but those are the crypto people.
They're not here. So, um, but in

1235
01:10:16,400 --> 01:10:19,197
this case, like that's just not that's
not what we'd want to encourage. And I

1236
01:10:19,280 --> 01:10:22,637
think that that should eventually
proliferate and get to everywhere,

1237
01:10:22,720 --> 01:10:23,997
especially as the costs have gone up.

1238
01:10:24,080 --> 01:10:27,837
And the last time we got together a few
months ago was right at that click point

1239
01:10:27,920 --> 01:10:30,557
of cost. and that you did a great thing
with the audience talking about, hey,

1240
01:10:30,640 --> 01:10:33,997
how much are people willing to spend
which is a super fun exercise which I

1241
01:10:34,080 --> 01:10:36,797
hope maybe you'll do later.
>> We could do we could do it now. Can we

1242
01:10:36,880 --> 01:10:39,197
put let's put the house lights up for a

1243
01:10:39,280 --> 01:10:44,797
second? Um I am curious. Sorry, we're
going to ask you to vote. Um of of the

1244
01:10:44,880 --> 01:10:48,877
folks here, >> would you be
willing to spend double the

1245
01:10:48,960 --> 01:10:50,397
amount that you're spending today for

1246
01:10:50,480 --> 01:10:56,120
the current capabilities
that you have uh with AI?

1247
01:10:56,480 --> 01:10:58,557
That's it. How many people are satisfied

1248
01:10:58,640 --> 01:11:01,677
with the price that you're paying?

1249
01:11:01,760 --> 01:11:05,597
So, if if they really Okay, sorry. I'm
going to go with more more questions. If

1250
01:11:05,680 --> 01:11:10,077
they if let's say your Claude
subscription doubled in price, how many

1251
01:11:10,160 --> 01:11:13,677
people would cancel?
>> Is it fable or not?

1252
01:11:13,760 --> 01:11:16,317
>> Is it fable? Well, right now, no. No.

1253
01:11:16,400 --> 01:11:19,037
>> Yeah. >> I think I
think Okay, we could put the

1254
01:11:19,120 --> 01:11:22,877
lights down again. I think basically
what it what it proves is um we we saw

1255
01:11:22,960 --> 01:11:26,637
about half the room's hands go up that
they would pay double. Um, and it does

1256
01:11:26,720 --> 01:11:29,277
seem like there's a bunch of room for

1257
01:11:29,360 --> 01:11:33,837
the labs to be able to raise prices
and still do well. But Dallas, something

1258
01:11:33,920 --> 01:11:37,597
interesting happened since we spoke last.
So, we spoke at Google Cloud Next

1259
01:11:37,680 --> 01:11:41,597
about this and um, someone asked about
the the margin the I think Dallas, you

1260
01:11:41,680 --> 01:11:45,277
asked whe whether the margin
of the business can be maintained at the

1261
01:11:45,360 --> 01:11:48,477
current prices. And I thought, okay,
well, forget about it because these labs

1262
01:11:48,560 --> 01:11:52,637
have such an economically valuable >>
uh uh tool for us that they'll raise

1263
01:11:52,720 --> 01:11:55,917
prices. But we might be in the moment
where they're going to get into a price

1264
01:11:56,000 --> 01:12:00,717
war because OpenAI is rumored to be
potentially uh dropping prices. And so

1265
01:12:00,800 --> 01:12:03,197
what do you think about that? I >> I
I think we're absolutely right on the

1266
01:12:03,280 --> 01:12:06,957
precipice of that. I think you actually
saw in the audience here by comparison,

1267
01:12:07,040 --> 01:12:10,557
>> we saw every hand go up when you said,
"Would you be willing to pay double when

1268
01:12:10,640 --> 01:12:12,957
we were together 3 months ago in April?"

1269
01:12:13,040 --> 01:12:16,957
>> And when when Alex asked if people
were willing to pay four and five times as

1270
01:12:17,040 --> 01:12:19,437
much, there was still a quarter
of the hands in the room that were up. And

1271
01:12:19,520 --> 01:12:22,237
we're talking a room of about 200 people
roughly, right? So there was a lot of

1272
01:12:22,320 --> 01:12:26,717
people. was a good, you know, good good
tea sample, so to say. Um, I contrast

1273
01:12:26,800 --> 01:12:30,237
that to what we just saw right now, and I
think it's a a fairly, you know, even

1274
01:12:30,320 --> 01:12:32,157
distribution, similar subset of people.

1275
01:12:32,240 --> 01:12:36,477
Um, and the reality is there's there's
more skepticism of value that they're

1276
01:12:36,560 --> 01:12:40,477
getting from it, especially when you
start layering on the access and the

1277
01:12:40,560 --> 01:12:44,477
capabilities associated with some of the
models that are still per seat, as well

1278
01:12:44,560 --> 01:12:47,757
as some of the open models, which you
can get access to and, you know, for

1279
01:12:47,840 --> 01:12:51,437
free. You can do a lot of really cool
things. What would you do if the models

1280
01:12:51,520 --> 01:12:54,717
were half price or tokens were half price?

1281
01:12:54,800 --> 01:12:57,517
>> What would I think would I do something

1282
01:12:57,600 --> 01:12:59,117
differently? Is that the question? Yeah.

1283
01:12:59,200 --> 01:13:03,597
>> Let's say I'm like Open AI or Google
and I come to you and say, "Dallas, we're

1284
01:13:03,680 --> 01:13:05,357
happy that you're using our technology.

1285
01:13:05,440 --> 01:13:08,957
We want to make sure that you don't use
Fable when it's back online. So, we're

1286
01:13:09,040 --> 01:13:11,357
cutting your prices in by 50%."

1287
01:13:11,440 --> 01:13:13,117
>> Yeah. >> Would it change
anything you're doing?

1288
01:13:13,200 --> 01:13:16,957
>> Absolutely. I I think actually like I
know this because I talked to my CEO and

1289
01:13:17,040 --> 01:13:19,117
CIO yesterday and my CIO again this

1290
01:13:19,200 --> 01:13:22,237
morning. Um and we are enterensitive

1291
01:13:22,320 --> 01:13:24,477
environment, right? We have 350,000

1292
01:13:24,560 --> 01:13:27,277
people globally. Um they all have access

1293
01:13:27,360 --> 01:13:31,517
to one tool or another. And so when you
have 350,000 people and they're playing

1294
01:13:31,600 --> 01:13:34,957
with tools, there's a high degree
of price sensitivity just on sheer volume

1295
01:13:35,040 --> 01:13:37,757
alone, right? even if not all of them
are doing super productive things. Just

1296
01:13:37,840 --> 01:13:41,517
to be clear, those aren't all engineers,
but they're all pretty smart people

1297
01:13:41,600 --> 01:13:44,957
doing pretty interesting things and they
have use cases that they're like, "Hey,

1298
01:13:45,040 --> 01:13:47,437
I want to play with this." And we are
encouraging them to do that. We think

1299
01:13:47,520 --> 01:13:51,197
that's great, but we're also price
sensitive, right? There's an elasticity

1300
01:13:51,280 --> 01:13:55,277
there that says the higher the price
goes up, the less we'd want them to use

1301
01:13:55,360 --> 01:13:57,757
that model. We'd want them to use
something cheaper. So, if somebody was

1302
01:13:57,840 --> 01:14:01,517
coming to us with a cheaper model,
100% the direction of travel would go, you

1303
01:14:01,600 --> 01:14:04,237
know, to that. And we've even built
that into some of our control plane

1304
01:14:04,320 --> 01:14:07,517
technology. So there's selectivity
of model and there's also recommendations

1305
01:14:07,600 --> 01:14:10,957
within it as well. So depending on who
you are and what it is that we think

1306
01:14:11,040 --> 01:14:13,997
you're going to do with it,
it automatically configures to have a

1307
01:14:14,080 --> 01:14:19,037
specific model and you have to break the
the the you know the initial setting in

1308
01:14:19,120 --> 01:14:22,077
order to use something different.
So we're already there, but I'm certain we

1309
01:14:22,160 --> 01:14:24,557
would encourage more usage
of cheaper things. There's no doubt.

1310
01:14:24,640 --> 01:14:27,357
>> Have you gotten any of those type
of calls yet or you're waiting for them?

1311
01:14:27,440 --> 01:14:31,357
>> Um I I I won't go too far into it,
but there's definitely conversations going

1312
01:14:31,440 --> 01:14:33,597
right now. Yeah. >> Very interesting.

1313
01:14:33,680 --> 01:14:37,597
>> Yeah. Um, when we spoke recently,
you told me that you're seeing some limits

1314
01:14:37,680 --> 01:14:40,397
with agents. And it seems to me like if

1315
01:14:40,480 --> 01:14:43,277
we're going to see this continue, agents

1316
01:14:43,360 --> 01:14:45,117
can't be limited. They have to be able

1317
01:14:45,200 --> 01:14:50,397
to operate autonomously and spend all
those tokens and be effective. So, what

1318
01:14:50,480 --> 01:14:53,837
are the limits that you're seeing with
agents today? And do you think that the

1319
01:14:53,920 --> 01:14:57,837
like if we could extrapolate a little bit,
it means that we're going to see um

1320
01:14:57,920 --> 01:15:01,917
some more speed bumps as the labs try
to roll this technology out further? Yeah,

1321
01:15:02,000 --> 01:15:04,157
I don't I use you use the term limit. I

1322
01:15:04,240 --> 01:15:07,437
I think it's um I think it's a function

1323
01:15:07,520 --> 01:15:12,477
of both risk tolerance um as well
as cost tolerances and then finally like

1324
01:15:12,560 --> 01:15:15,997
what is it you have an expectations
of these things doing on their own and so

1325
01:15:16,080 --> 01:15:18,877
the limitations are actually in all in all
three of those areas that are coming

1326
01:15:18,960 --> 01:15:22,317
through. um you know from a risk
tolerance point of view I think people

1327
01:15:22,400 --> 01:15:26,797
are saying wait a second I'm worried
that the agent without some level of you

1328
01:15:26,880 --> 01:15:31,037
know call it control or governance
around it could go just about anywhere

1329
01:15:31,120 --> 01:15:34,237
and what does that mean within
my organization depending on what access I

1330
01:15:34,320 --> 01:15:38,317
give to it from a data perspective um
you know I would say from client data or

1331
01:15:38,400 --> 01:15:41,357
whether it's you know it's code itself
and what can it do to change code if you

1332
01:15:41,440 --> 01:15:44,797
ask it to do one thing in one area will
it simply think that it needs to do that

1333
01:15:44,880 --> 01:15:48,157
everywhere else and and there's
a you know we'll say the ability to

1334
01:15:48,240 --> 01:15:51,517
extrapolate on a single point and like
what control exists there. So that's the

1335
01:15:51,600 --> 01:15:56,237
first limit. The second limit is on like
I said on the cost var variance we went

1336
01:15:56,320 --> 01:16:00,157
through there a second ago which is hey
look like there are just things you're

1337
01:16:00,240 --> 01:16:03,437
not going to want it to do because back
to the MIT study there might be things

1338
01:16:03,520 --> 01:16:07,037
that humans can do not only better but
more cheaply especially now depending on

1339
01:16:07,120 --> 01:16:10,077
you know if if you have access to fable
and you're using that at a super high

1340
01:16:10,160 --> 01:16:14,077
premium. Um, and then the third bit,
you know, is is really in that that

1341
01:16:14,160 --> 01:16:17,517
decision-making, we'll call it from an
organizational point of view. And even I

1342
01:16:17,600 --> 01:16:21,997
I even think it was funny. I um uh
of all the of all the places to be here in

1343
01:16:22,080 --> 01:16:25,277
San Francisco. I was actually at a
funeral this morning for my grandmother

1344
01:16:25,360 --> 01:16:28,557
who was born in San Francisco. Actually,
she was literally born right up the

1345
01:16:28,640 --> 01:16:33,517
street here. And it's such a human
thing, right? And I was talking to a

1346
01:16:33,600 --> 01:16:37,197
priest, the priest who was there. He's
actually the priest who wrote the paper

1347
01:16:37,280 --> 01:16:40,637
um along with some of the team over
over at Anthropic. and we're having

1348
01:16:40,720 --> 01:16:43,437
interesting conversation on like
where does ethics play into the broader

1349
01:16:43,520 --> 01:16:47,117
conversation of what's happening
within this area. It's sort of like it's

1350
01:16:47,200 --> 01:16:51,037
workforce planning and then some. And so
there's this like additional, you know,

1351
01:16:51,120 --> 01:16:53,757
component that I don't think we've even
really gotten into because here we are

1352
01:16:53,840 --> 01:16:57,917
at the center of technology. We don't
think about that part of it so much. But

1353
01:16:58,000 --> 01:17:01,597
it is that like benefit side.
The benefit I think has a risk that we

1354
01:17:01,680 --> 01:17:05,357
haven't even really gotten into a ton,
which is, you know, do I want to scare

1355
01:17:05,440 --> 01:17:10,077
all of my people that I'm watching what
they're doing and waiting to replace

1356
01:17:10,160 --> 01:17:13,677
what it is that they're doing? And the
answer is like absolutely not because in

1357
01:17:13,760 --> 01:17:17,117
fact, I met six of our interns on the
plane back from Chicago last night and

1358
01:17:17,200 --> 01:17:20,557
they couldn't be happier to be interns
with us. Uh they just saw something on

1359
01:17:20,640 --> 01:17:22,397
my t-shirt that said >>
you're still hiring interns.

1360
01:17:22,480 --> 01:17:25,437
>> Absolutely. I mean, it it you know,
we're hiring as many people as we did

1361
01:17:25,520 --> 01:17:28,637
last year and we're changing the who
we're hiring. We're actually hiring a

1362
01:17:28,720 --> 01:17:32,397
lot more kids who are pursuing sciences
and you know even in areas that uh that

1363
01:17:32,480 --> 01:17:33,997
go beyond you know engineering.

1364
01:17:34,080 --> 01:17:35,517
>> And where are you not hiring?

1365
01:17:35,600 --> 01:17:38,877
>> Um I think we're hiring a little bit
in the less of just like you know pure

1366
01:17:38,960 --> 01:17:42,157
accounting. Also a function of where
we're doing it. It's not that we're

1367
01:17:42,240 --> 01:17:45,357
hiring less people overall. We're hiring
less people in certain areas. U but

1368
01:17:45,440 --> 01:17:47,917
that's also just a function
of globalization and how we're delivering

1369
01:17:48,000 --> 01:17:51,117
services. Um but it's not really
a function of the fact that we're saying

1370
01:17:51,200 --> 01:17:53,677
oh we don't need these people because
technology will do a lot of it. And that

1371
01:17:53,760 --> 01:17:56,957
then goes back to your original you
know premise of the question which is you

1372
01:17:57,040 --> 01:17:58,637
know what are the limits that are there

1373
01:17:58,720 --> 01:18:01,517
on the agentics themselves. The limits

1374
01:18:01,600 --> 01:18:04,797
are going to be a tolerance for error a

1375
01:18:04,880 --> 01:18:08,717
tolerance for cost and a tolerance again
for like what will be acceptable within

1376
01:18:08,800 --> 01:18:13,037
the organization of having that thing
do versus having you know a human do it.

1377
01:18:13,120 --> 01:18:15,917
And there'll be a cost dynamic to that
part of it as well. But I think there's

1378
01:18:16,000 --> 01:18:20,237
also even just the how do I run
my business and how am I a positive leader

1379
01:18:20,320 --> 01:18:21,997
right within within a given community.

1380
01:18:22,080 --> 01:18:23,757
doesn't matter what type of company it is.

1381
01:18:23,840 --> 01:18:26,557
>> You know, Dallas, I I want to take
a moment just to acknowledge your your

1382
01:18:26,640 --> 01:18:28,237
grandmother, and I'm
sorry about her passing.

1383
01:18:28,320 --> 01:18:30,637
>> I appreciate that. >> She she
spent her life here in San Francisco.

1384
01:18:30,720 --> 01:18:32,317
>> Here in San Francisco. Yeah. Yeah.

1385
01:18:32,400 --> 01:18:35,037
>> Can you tell us a little
bit about her just just briefly?

1386
01:18:35,120 --> 01:18:39,117
>> Uh, sure. I mean, you know, funny uh
you know, do daughter of of an immigrant

1387
01:18:39,200 --> 01:18:42,317
family. Um, people who picked a lot
of things in a lot of places around the

1388
01:18:42,400 --> 01:18:47,037
world, including Argentina and, uh,
and Hawaii, funny enough. Um, and thought,

1389
01:18:47,120 --> 01:18:49,597
you know, picking stuff out of the ground
isn't a good deal. like we should

1390
01:18:49,680 --> 01:18:53,837
move to San Francisco and we're working
in the caneries um up in in Fisherman's

1391
01:18:53,920 --> 01:18:55,277
Wararf which is what her parents did.

1392
01:18:55,360 --> 01:18:57,677
She actually worked in the telco industry.
The first job was actually

1393
01:18:57,760 --> 01:19:00,477
working for company for Bell. It must be

1394
01:19:00,560 --> 01:19:05,277
it must be the second T in the TMT,
right? So for for for grandma Fernandez

1395
01:19:05,360 --> 01:19:07,197
Corteho. Yeah. Like really a cool

1396
01:19:07,280 --> 01:19:12,717
existence and just a a great person and
a great you know um kind of full circle

1397
01:19:12,800 --> 01:19:15,677
story, right? I mean I think this whole
story of technology is such a cool thing

1398
01:19:15,760 --> 01:19:19,037
that all this is happening in one.
I mean, let's be honest, it's all

1399
01:19:19,120 --> 01:19:22,397
happening kind of in one place here in
in in the Bay Area in San Francisco. And

1400
01:19:22,480 --> 01:19:27,917
it's it's really neat to to be able
to like see that thread all the way pulled

1401
01:19:28,000 --> 01:19:31,037
through, right? Like I've been able
to do from a career point of view. Um, and

1402
01:19:31,120 --> 01:19:33,437
also be able to notice that there's
actually a direct connection to the

1403
01:19:33,520 --> 01:19:36,877
things that made this place a great
place 150 years ago or actually still

1404
01:19:36,960 --> 01:19:39,997
the things that make it a great place
because it's still better, right, to

1405
01:19:40,080 --> 01:19:42,957
work in a canery or work for a company
than it is to pick something out of the

1406
01:19:43,040 --> 01:19:47,037
ground, so to say. again as my as my
ancestors you know concluded you know

1407
01:19:47,120 --> 01:19:51,037
back in the 1800s but when you think
about that I I think it does go back

1408
01:19:51,120 --> 01:19:54,637
even to some of the things of like what
do you want to do right like I think you

1409
01:19:54,720 --> 01:19:59,277
want to create technology that makes
people's lives better in much the same

1410
01:19:59,360 --> 01:20:02,317
way that our grandmas make our lives
better right grandma makes everything

1411
01:20:02,400 --> 01:20:04,877
better I think it's the same kind
of concept the things that we talk about

1412
01:20:04,960 --> 01:20:08,157
every day are how do we like build
products you know what are the

1413
01:20:08,240 --> 01:20:10,957
entrepreneurs they're trying to build
products that make their customers you

1414
01:20:11,040 --> 01:20:14,317
know life better or they make something
easier to do whether it's coding or

1415
01:20:14,400 --> 01:20:18,397
whether it's you know customer support
or what have you. I I think we we take

1416
01:20:18,480 --> 01:20:20,957
that thread all the way through. That's
actually where the ROI comes in. It's

1417
01:20:21,040 --> 01:20:23,197
not going to be back to the token maxing.
It's not going to be how much

1418
01:20:23,280 --> 01:20:27,517
you pay and how you pay it and what have
you. It's a whole ecosystem shift in the

1419
01:20:27,600 --> 01:20:31,277
way that we think about the outputs
themselves. The outputs again still are

1420
01:20:31,360 --> 01:20:34,317
am I serving my clients well? Are they
getting you know deal maximization out

1421
01:20:34,400 --> 01:20:37,277
of what uh what they're doing with you
know with any given company frontier

1422
01:20:37,360 --> 01:20:40,557
firm or hyperscaler or you know
Neocloud or whatever it might be.

1423
01:20:40,640 --> 01:20:41,597
>> Well, thank you for sharing with us.

1424
01:20:41,680 --> 01:20:45,597
>> Yeah, thanks for that. Yeah. Um, you
know, interesting San Francisco. One of

1425
01:20:45,680 --> 01:20:49,757
the things that I associate with San
Francisco is it's a city that where

1426
01:20:49,840 --> 01:20:51,437
people are okay losing a little bit of

1427
01:20:51,520 --> 01:20:54,397
control. Um, whether that is building,

1428
01:20:54,480 --> 01:20:59,597
you know, new technologies that do
things a little bit differently than um

1429
01:20:59,680 --> 01:21:01,277
previous generations or the fact that

1430
01:21:01,360 --> 01:21:03,277
this city or not everybody but many

1431
01:21:03,360 --> 01:21:08,797
people love um LSD and mushrooms. Um, and

1432
01:21:08,880 --> 01:21:13,760
>> you said you weren't going
to mention that in our precall. Okay.

1433
01:21:14,160 --> 01:21:17,517
Um, it is interesting with with with

1434
01:21:17,600 --> 01:21:20,237
agents you do you kind of lose control.

1435
01:21:20,320 --> 01:21:24,157
It reminds me a little bit of like of
skydiving in a way where you jump out of

1436
01:21:24,240 --> 01:21:28,397
a plane and you're like whatever happens
but I hope there's a system ready to

1437
01:21:28,480 --> 01:21:30,877
catch me. Um, from your position you're

1438
01:21:30,960 --> 01:21:33,517
know you're deploying agents um in

1439
01:21:33,600 --> 01:21:37,917
pretty high stakes moments. you even if
you have the best governance in place,

1440
01:21:38,000 --> 01:21:42,237
you have to be okay to a degree
of losing control. So, how do you become

1441
01:21:42,320 --> 01:21:44,637
comfortable with that? That's it's

1442
01:21:44,720 --> 01:21:49,757
funny, right? I mean, I think um how
much can you really control like in

1443
01:21:49,840 --> 01:21:53,117
the in the engineering space where you
have the people like doing the code for

1444
01:21:53,200 --> 01:21:56,157
you or in our space where you have the
individuals doing the services, they're

1445
01:21:56,240 --> 01:21:59,597
preparing a you know, an audit or a tax
return or a deal report or what have

1446
01:21:59,680 --> 01:22:02,237
you. We're putting trust in these folks,

1447
01:22:02,320 --> 01:22:04,077
many of you whom you know at like a

1448
01:22:04,160 --> 01:22:07,357
superficial level. Um, when you start

1449
01:22:07,440 --> 01:22:12,317
layering technology into it, because
my view is that it's an augmentation of

1450
01:22:12,400 --> 01:22:16,637
those people to make them better, not
necessarily seeding all the control from

1451
01:22:16,720 --> 01:22:20,877
them and putting into something that I
know less of, that makes me feel a lot

1452
01:22:20,960 --> 01:22:25,837
more comfortable. If you start getting
into the space where a 100% of all the

1453
01:22:25,920 --> 01:22:30,157
things that we do around I don't know
let's say like booking travel becomes

1454
01:22:30,240 --> 01:22:34,797
you knowified right you just put in the
query and the travel gets booked I think

1455
01:22:34,880 --> 01:22:37,757
that's where you do get uncomfortable
like I appreciated that two nights ago

1456
01:22:37,840 --> 01:22:41,197
knowing I had to get back to hang out
with you Alex and and to make grandma

1457
01:22:41,280 --> 01:22:46,237
sing this morning like I can ping my EA
late at night and say hey I I need some

1458
01:22:46,320 --> 01:22:49,677
help like I need to make sure with
tornadoes coming through Chicago there's

1459
01:22:49,760 --> 01:22:54,157
going to be at least a plane on Thursday
morning that takes off at 6 am that

1460
01:22:54,240 --> 01:22:59,117
could get me to the West Coast. Um,
if I'm pushing that into an an agent and

1461
01:22:59,200 --> 01:23:01,677
I'm hopeful that it works and I'm
hopeful that it books a flight, I mean,

1462
01:23:01,760 --> 01:23:04,557
yeah, I might see the outcome, but isn't
it neat like having the human on the

1463
01:23:04,640 --> 01:23:05,917
other line like Liz says, "Hey, I got

1464
01:23:06,000 --> 01:23:08,317
you, bro." Like, like you're handled.

1465
01:23:08,400 --> 01:23:13,437
>> Um, she's augmented. She goes into
our technology and can quickly query

1466
01:23:13,520 --> 01:23:16,477
something and boom, pulls everything
up and she books a flight within like 30

1467
01:23:16,560 --> 01:23:20,957
seconds, right? But having her there
to make that a little bit better like does

1468
01:23:21,040 --> 01:23:23,757
make the whole thing feel better.
And you know candidly by the way as it

1469
01:23:23,840 --> 01:23:27,197
relates to like what you know what my EC
does or my my chief of staff and others

1470
01:23:27,280 --> 01:23:31,517
do like they're able to do like multiple
person's jobs. If you think about like

1471
01:23:31,600 --> 01:23:34,317
what it was before like you know one
person to one person. No it's one to

1472
01:23:34,400 --> 01:23:38,637
like 12. Like we're doing a lot more
with a lot less. So I look at this tech

1473
01:23:38,720 --> 01:23:41,197
is just being an extrapolation on that
point. Yeah there's some things you're

1474
01:23:41,280 --> 01:23:45,197
just not going to stop doing but like
that's totally okay. Um it's it's no

1475
01:23:45,280 --> 01:23:48,477
different than driving, right? It's
like the self-driving dynamic like we are

1476
01:23:48,560 --> 01:23:51,037
going to get really comfortable with that.
I see this you know move to

1477
01:23:51,120 --> 01:23:54,557
Agentic as being you know very very um

1478
01:23:54,640 --> 01:23:58,237
uh similar uh in a parallel run with
you know with that itself. It just so

1479
01:23:58,320 --> 01:23:59,757
happens to be in the physical space.

1480
01:23:59,840 --> 01:24:02,957
>> Yeah it definitely reminds me a lot
of a whimo. You sort of you know you white

1481
01:24:03,040 --> 01:24:06,077
knuckle it in the beginning and then
you start to go on your phone.

1482
01:24:06,160 --> 01:24:09,757
>> Exactly. >> Yeah.

1483
01:24:09,840 --> 01:24:10,877
>> That's what I've done at least.

1484
01:24:10,960 --> 01:24:13,357
>> Yeah. Whole thing.
Uh anyway, don't take

1485
01:24:13,440 --> 01:24:15,757
any uh anything away from those previous

1486
01:24:15,840 --> 01:24:19,517
comments um about the mushrooms.

1487
01:24:19,600 --> 01:24:21,357
>> That's for later. That's happy hour.

1488
01:24:21,440 --> 01:24:22,797
>> Please join us on the roof at 5.

1489
01:24:22,880 --> 01:24:26,717
>> Exactly. >> Uh Dallas,
thank you so much for being

1490
01:24:26,800 --> 01:24:30,797
here with us. Always a pleasure to speak
with you and I do appreciate uh your

1491
01:24:30,880 --> 01:24:32,717
support and PWC support of this event.

1492
01:24:32,800 --> 01:24:33,837
So, thank you. Thank you very much.

1493
01:24:33,920 --> 01:24:35,757
Let's hear it for Dallas. Thank you.

1494
01:24:35,840 --> 01:24:39,320
>> Thank you so much.

1495
01:24:41,920 --> 01:24:45,757
All right. Amazing. Uh, thank you,
Dallas. All right, folks. So, we are

1496
01:24:45,840 --> 01:24:49,357
going to have this next session.
It's going to be an audience participation

1497
01:24:49,440 --> 01:24:52,877
section. So, we definitely want your
questions for us and then we're going to

1498
01:24:52,960 --> 01:24:57,517
go to a coffee break. So, um, though
for many of you, he needs no introduction.

1499
01:24:57,600 --> 01:25:02,157
Let me introduce Ron Roy. I first
started reading Ron Johny's I first

1500
01:25:02,240 --> 01:25:04,397
started reading Ran Johny's market uh,

1501
01:25:04,480 --> 01:25:09,197
writing in 2021. um he had
write written this newsletter

1502
01:25:09,280 --> 01:25:10,797
called margins and I thought it was a

1503
01:25:10,880 --> 01:25:13,357
terrific newsletter. I saved it. I spent

1504
01:25:13,440 --> 01:25:16,397
my winter break reading it. I DM'd him.

1505
01:25:16,480 --> 01:25:19,277
By that January uh we had decided to do

1506
01:25:19,360 --> 01:25:23,437
an emergency podcast about a crazy
financial situation and then Ranj and I

1507
01:25:23,520 --> 01:25:26,077
kept talking more and more. Uh and then

1508
01:25:26,160 --> 01:25:30,797
by January 2023 I wrote to him and said,
"Hey, don't you want to just come and do

1509
01:25:30,880 --> 01:25:35,037
this every week?" And lucky for me
and lucky for us on uh with with anyone

1510
01:25:35,120 --> 01:25:39,117
involved with Big Technology Podcast,
uh Ron John said yes. And so getting a

1511
01:25:39,200 --> 01:25:42,877
chance to speak with Ron John every
Friday is an absolute joy. It's

1512
01:25:42,960 --> 01:25:46,237
definitely one of the highlights
of my week. And today we're thrilled to be

1513
01:25:46,320 --> 01:25:49,917
able to do our Friday show live here
with you with your with your audience

1514
01:25:50,000 --> 01:25:54,477
questions. Um and then we'll just run
it tomorrow like a normal podcast. So I

1515
01:25:54,560 --> 01:25:56,317
hope you're ready. Uh we definitely need

1516
01:25:56,400 --> 01:26:02,920
your participation. And please
join me in welcoming Ron Roy.

1517
01:26:07,040 --> 01:26:11,597
I got them on. I'm going to take these
off though. We'll get into the snap

1518
01:26:11,680 --> 01:26:13,597
spectacles. This is This is a medium

1519
01:26:13,680 --> 01:26:18,717
risk maneuver because I
have this microphone on. So

1520
01:26:18,800 --> 01:26:19,997
>> those are the Snapchat spectacles.

1521
01:26:20,080 --> 01:26:22,077
>> These are the Snapchat spectacles.

1522
01:26:22,160 --> 01:26:25,677
They're the original developer beta
edition though. So, they're not the new

1523
01:26:25,760 --> 01:26:30,797
ones, but I had them in 2021.
>> Now, everyone knows how cool you are.

1524
01:26:30,880 --> 01:26:33,117
>> All right. Should I start
the >> I'm as cool as Evan Spiegel.

1525
01:26:33,200 --> 01:26:35,277
>> That's right. Yeah.
>> Is he still cool, though?

1526
01:26:35,360 --> 01:26:39,997
>> All right. Uh, let's do it.
Um, how would I start it? Um,

1527
01:26:40,080 --> 01:26:41,437
>> I throw Did I throw you off?

1528
01:26:41,520 --> 01:26:44,157
>> Yeah. No, no, no. I didn't even write
this down. Okay. Well, I'll try to do

1529
01:26:44,240 --> 01:26:48,237
it. All right. Snapchat comes out with new
spectacles and we take your audience

1530
01:26:48,320 --> 01:26:51,837
questions. That's coming up right after
this on a Big Technology Podcast Friday

1531
01:26:51,920 --> 01:26:55,597
edition recorded on Thursday. Welcome
to Big Technology Podcast Friday edition

1532
01:26:55,680 --> 01:26:58,557
where we break down the news in our
traditional coolheaded and nuanced

1533
01:26:58,640 --> 01:27:01,197
format. We're joined as always by Ranjan

1534
01:27:01,280 --> 01:27:03,917
Ry who is here with us in not in studio

1535
01:27:04,000 --> 01:27:06,477
live uh with the big technology AI

1536
01:27:06,560 --> 01:27:11,400
summit audience. Audience, let's hear you.

1537
01:27:15,200 --> 01:27:19,357
The way this is going to work is Ranj
and I will break down uh one story and

1538
01:27:19,440 --> 01:27:22,717
then we definitely welcome your questions,
your prompts, your arguments

1539
01:27:22,800 --> 01:27:26,637
with us. Uh if you don't have anything,
we have plenty to do, but um we'd love

1540
01:27:26,720 --> 01:27:30,157
to have your participation. You can line
up at either of the mics there uh to ask

1541
01:27:30,240 --> 01:27:33,757
us a question. Let's start with our top
story this week. I promised myself when

1542
01:27:33,840 --> 01:27:36,557
this uh summit was being planned that I

1543
01:27:36,640 --> 01:27:39,117
would not be mean to Snapchat. However,

1544
01:27:39,200 --> 01:27:40,797
Snapchat has left me with no choice.

1545
01:27:40,880 --> 01:27:45,720
John, let's roll. Image C, please.

1546
01:27:47,280 --> 01:27:51,837
Uh, listeners, if you are listening on
the podcast, you'll see what you're what

1547
01:27:51,920 --> 01:27:53,597
we're looking at at the audience here is

1548
01:27:53,680 --> 01:27:56,957
a image of Evan Spiegel on CNBC wearing

1549
01:27:57,040 --> 01:27:59,437
the latest Snap Specs. The headline is

1550
01:27:59,520 --> 01:28:03,277
Snap Stock Falls after AR Specs debut.

1551
01:28:03,360 --> 01:28:08,717
Um, uh, almost 20 years since the launch
of the iPhone, people are ready to think

1552
01:28:08,800 --> 01:28:12,637
about computing differently, Spiegel
said in an interview with CNBC. The, uh,

1553
01:28:12,720 --> 01:28:15,197
the market reacted differently. Um,

1554
01:28:15,280 --> 01:28:19,677
Rajan, let me just go to you quickly.
Do you think that that, you know, Snapchat

1555
01:28:19,760 --> 01:28:21,677
and Meta and all these other companies

1556
01:28:21,760 --> 01:28:23,517
have been trying to build these AI

1557
01:28:23,600 --> 01:28:27,677
device future for years and this is what

1558
01:28:27,760 --> 01:28:30,797
we're looking at. Um,
>> this is what we're looking at.

1559
01:28:30,880 --> 01:28:32,077
>> You kind of ruined my surprise. Can we

1560
01:28:32,160 --> 01:28:36,717
go to D, please? I mean,

1561
01:28:36,800 --> 01:28:40,077
>> I brought them. I had to bring
them after sending Alex a photo.

1562
01:28:40,160 --> 01:28:42,397
>> Is it time for us
to finally accept that

1563
01:28:42,480 --> 01:28:45,837
we're not going to have an AR device?

1564
01:28:45,920 --> 01:28:49,197
>> Interesting. So, so these again, I got

1565
01:28:49,280 --> 01:28:52,317
these in 2021. It's actually a very cool

1566
01:28:52,400 --> 01:28:55,037
technology. Like, it's No, I'm serious.

1567
01:28:55,120 --> 01:28:59,837
Augmented reality. If you ever used
Magic Leap in the 2010s, like being able

1568
01:28:59,920 --> 01:29:03,997
to paint throughout your room and walk
around that painting, being a being able

1569
01:29:04,080 --> 01:29:07,837
to play games where you're chasing
zombies again, my seven-year-old son

1570
01:29:07,920 --> 01:29:11,677
actually is probably the only fan
of Snap Spectacles in the world right now.

1571
01:29:11,760 --> 01:29:13,837
He still loves using them. He also likes

1572
01:29:13,920 --> 01:29:15,917
to watch YouTube on them, which is kind

1573
01:29:16,000 --> 01:29:18,717
of amazing. But I think that form factor

1574
01:29:18,800 --> 01:29:22,157
and the experience is amazing. Vision

1575
01:29:22,240 --> 01:29:27,517
Pro has not quite captured it. I don't
think what we just saw on Evan Spiegel's

1576
01:29:27,600 --> 01:29:31,037
face is going to capture it. I think
eventually maybe Apple or someone will,

1577
01:29:31,120 --> 01:29:35,597
but I don't think we're there yet, but I
think we will be. I still am betting on

1578
01:29:35,680 --> 01:29:40,397
AR eyeglasses of some sort. >> Okay,
I'm going to go to questions in a

1579
01:29:40,480 --> 01:29:43,917
moment if anyone has one. So, feel
free to to stand up there if you want,

1580
01:29:44,000 --> 01:29:46,397
otherwise we can keep doing this. Um,

1581
01:29:46,480 --> 01:29:51,037
here's some of the social media reaction.
Uh, does anyone anyone that

1582
01:29:51,120 --> 01:29:55,997
works at Snapchat have the guts to tell
leadership that these things are ugly?

1583
01:29:56,080 --> 01:30:00,637
That feeling when your glasses are so
heavy they give you cauliflower ear.

1584
01:30:00,720 --> 01:30:03,837
Snap is the best brand in the world
when you're 16 and the worst brand when

1585
01:30:03,920 --> 01:30:05,517
you're to be associated with when you're

1586
01:30:05,600 --> 01:30:09,117
21. Um, the people who actually buy

1587
01:30:09,200 --> 01:30:13,677
$2,000 AI glasses aren't teenagers. If
you think you really want to wear Always

1588
01:30:13,760 --> 01:30:16,877
on camera around in public, uh,
it should have to look like this with the

1589
01:30:16,960 --> 01:30:20,557
picture of Evan Spiegel. Um, let me make

1590
01:30:20,640 --> 01:30:25,037
the case that this it's over. Um, >> good
because I'm going to take the other side.

1591
01:30:25,120 --> 01:30:28,877
>> I think that the iPhone series
that released that we just saw. So, does

1592
01:30:28,960 --> 01:30:31,517
anyone here listen to Ron John on Fridays?

1593
01:30:31,600 --> 01:30:34,637
>> Do we have listeners? All right. Okay.

1594
01:30:34,720 --> 01:30:37,197
>> So, you'll know which
direction this might go.

1595
01:30:37,280 --> 01:30:40,397
>> What's this guy been begging for for
like since he came on the show the first

1596
01:30:40,480 --> 01:30:42,957
time? Better Siri, >> better Siri.

1597
01:30:43,040 --> 01:30:45,037
>> I think they actually did it. Like the

1598
01:30:45,120 --> 01:30:47,197
new Siri, if you look at the videos,

1599
01:30:47,280 --> 01:30:51,037
looks terrific. And so maybe this idea

1600
01:30:51,120 --> 01:30:54,877
of we have to wear the computing
on our face is something that like kind of

1601
01:30:54,960 --> 01:30:59,197
sounds good in concept, but model after
model. It's not. And the AI device is

1602
01:30:59,280 --> 01:31:03,837
the iPhone. >> Okay,
that's I I that is an interesting

1603
01:31:03,920 --> 01:31:06,157
direction to take it. I still think the

1604
01:31:06,240 --> 01:31:08,557
form factor. Do do any of the audience

1605
01:31:08,640 --> 01:31:11,277
have like meta ray bands or any other

1606
01:31:11,360 --> 01:31:15,677
device like that? I see a few. Like you
start to feel as you're walking around,

1607
01:31:15,760 --> 01:31:19,597
as you're kind of interacting in the real
world, stuff can happen that's not

1608
01:31:19,680 --> 01:31:24,157
just maybe eventually Siri talking
to you and using a traditional AI model to

1609
01:31:24,240 --> 01:31:27,917
actually, you know, give you kind of
information. So, I don't know. I'm still

1610
01:31:28,000 --> 01:31:30,557
I still think AR as a form factor via

1611
01:31:30,640 --> 01:31:33,837
glasses is going to happen. I think uh

1612
01:31:33,920 --> 01:31:38,557
you have met Ray-B bands, you enjoy them.
You know, I I I didn't want to say

1613
01:31:38,640 --> 01:31:40,797
this publicly, but I have not used them.

1614
01:31:40,880 --> 01:31:44,237
I I I I mean, I I have, but I don't I

1615
01:31:44,320 --> 01:31:48,157
used them for a bit. And like I said
on our show recently, I was on a hike. It

1616
01:31:48,240 --> 01:31:51,917
was cold. >> I was ready
to get to the summit and put

1617
01:31:52,000 --> 01:31:55,037
those glasses on and it the battery

1618
01:31:55,120 --> 01:31:57,597
started blinking red and I couldn't use

1619
01:31:57,680 --> 01:31:59,517
them. Had to use my old phone.

1620
01:31:59,600 --> 01:32:03,677
>> And let me say this, like, you know,
they they haven't taken off as a

1621
01:32:03,760 --> 01:32:07,197
mainstream consumer device. And if
you look at the stock of every single

1622
01:32:07,280 --> 01:32:09,837
company that's pursuing
them, it's not good.

1623
01:32:09,920 --> 01:32:13,837
>> Snapchat, like we just said, is is
struggling. Um, Meta, as we know, has

1624
01:32:13,920 --> 01:32:18,237
its problems. Um, no one's looking
at the these Ray-B bands to save Meta.

1625
01:32:18,320 --> 01:32:23,997
>> Well, no, but to me, the Apple Vision
Pro is like the more direct correlated

1626
01:32:24,080 --> 01:32:27,357
product versus >> You want to know what
the best thing about the Vision Pro was?

1627
01:32:27,440 --> 01:32:29,997
>> Find Vision Pro. Yeah.
>> They put the person on Vision Pro on

1628
01:32:30,080 --> 01:32:34,557
Siri and he fixed it.
Mike Rockwell. Oh, >> so

1629
01:32:34,640 --> 01:32:36,077
>> I would not have
guessed that the person

1630
01:32:36,160 --> 01:32:39,997
who made the Vision Pro would be the one
to fix Siri after all these years, but I

1631
01:32:40,080 --> 01:32:42,877
guess if that's the case,
that's exciting. Yeah.

1632
01:32:42,960 --> 01:32:44,237
>> Yeah. >> So, you're
going to still You're going

1633
01:32:44,320 --> 01:32:46,877
to Can you put those
glasses on one more time?

1634
01:32:46,960 --> 01:32:50,237
>> Again, I was told backstage this might
destroy my microphone, but I'm going to

1635
01:32:50,320 --> 01:32:52,157
try. Do I look cool?

1636
01:32:52,240 --> 01:32:54,397
>> No. No.

1637
01:32:54,480 --> 01:32:57,517
>> No. No. I mean, even we were we were

1638
01:32:57,600 --> 01:33:01,117
joking that Evan Spiegel is going to the

1639
01:33:01,200 --> 01:33:05,357
Met Gala with his model wife. Like, this
is like the coolest person in the world

1640
01:33:05,440 --> 01:33:08,877
and how bad they made him look.
That if it was like when Zuck wore Project

1641
01:33:08,960 --> 01:33:11,037
Orion, no one really cared that much.

1642
01:33:11,120 --> 01:33:15,437
But I think because Evan Spiegel is such
a cool looking dude, that's why it looks

1643
01:33:15,520 --> 01:33:19,037
so egregious. That's my take. >> So,
your your answer on the way to make

1644
01:33:19,120 --> 01:33:23,437
those things work
is just be less handsome.

1645
01:33:23,520 --> 01:33:27,677
>> That's it. That's it. We'll we'll
write to Spiegel and let him know.

1646
01:33:27,760 --> 01:33:29,757
>> No questions. Okay. All right. Great.

1647
01:33:29,840 --> 01:33:32,237
>> All right. Yeah. Don't be shy.

1648
01:33:32,320 --> 01:33:35,597
>> Is it on? Yes, it is on. Hey guys,
>> if you're willing, let us know who you

1649
01:33:35,680 --> 01:33:38,397
are. And >> this is Ser Johal.
Actually, um industry

1650
01:33:38,480 --> 01:33:43,677
analyst flew from New York a day
earlier to join you guys. So,

1651
01:33:43,760 --> 01:33:44,717
>> welcome. Let's give him a welcome.

1652
01:33:44,800 --> 01:33:47,880
>> Thanks for

1653
01:33:49,520 --> 01:33:52,557
ask question about um actually it's an

1654
01:33:52,640 --> 01:33:54,477
observation. And I want to get your take

1655
01:33:54,560 --> 01:33:59,117
on this. Um when companies have this gap

1656
01:33:59,200 --> 01:34:01,517
between what people need today and what

1657
01:34:01,600 --> 01:34:06,397
they're working on which is like a in
future it can be two plus years out. So

1658
01:34:06,480 --> 01:34:10,157
they lose that traction from the
investors point of view as well as

1659
01:34:10,240 --> 01:34:13,197
employees and and partners. So do you

1660
01:34:13,280 --> 01:34:15,837
see that that's happening to meta and

1661
01:34:15,920 --> 01:34:18,877
others like it happened to IBM when they

1662
01:34:18,960 --> 01:34:21,357
were living in future with Watson X for

1663
01:34:21,440 --> 01:34:25,277
example. So um what's your take on that?

1664
01:34:25,360 --> 01:34:28,157
It can be B TOC or B2B examples.

1665
01:34:28,240 --> 01:34:29,277
>> Okay, thank you for the question.

1666
01:34:29,360 --> 01:34:33,197
>> Yeah, no that's a great question. I
do think like if we're talking about kind

1667
01:34:33,280 --> 01:34:36,077
of how the interface for how you

1668
01:34:36,160 --> 01:34:38,157
interact with a computer. I have been

1669
01:34:38,240 --> 01:34:43,117
begging for something else other than
me holding my phone looking at it and we

1670
01:34:43,200 --> 01:34:47,437
haven't really gotten anything for a
long time. I mean we Humane tried with

1671
01:34:47,520 --> 01:34:49,997
the pin. Um, I still think maybe some

1672
01:34:50,080 --> 01:34:55,757
kind of pin is going to be around Johnny
Ives pin at OpenAI. Maybe maybe at some

1673
01:34:55,840 --> 01:34:58,237
point. >> Well, Greg Greg
Brockman is coming, so we'll put

1674
01:34:58,320 --> 01:34:59,677
>> we got to ask him about the talk about

1675
01:34:59,760 --> 01:35:02,637
that. But, but I think like being able

1676
01:35:02,720 --> 01:35:05,517
to interact with all of this information

1677
01:35:05,600 --> 01:35:09,677
now, being able to like process
information so much more reliably with

1678
01:35:09,760 --> 01:35:14,397
AI just I don't want to have to keep
looking at my phone and just looking at

1679
01:35:14,480 --> 01:35:18,477
it on there. But even actually on the
phone, I'm guessing do a lot of people

1680
01:35:18,560 --> 01:35:23,517
here dictate more to their phone right
now. Like that's completely changed. In

1681
01:35:23,600 --> 01:35:26,797
the past, like I would have felt weird
just talking to my phone and now I'm

1682
01:35:26,880 --> 01:35:31,277
constantly using whisper flow
and just talking to whatever. So that's

1683
01:35:31,360 --> 01:35:32,557
>> tell a story about you and your wife.

1684
01:35:32,640 --> 01:35:36,957
>> I know this. Okay. I don't know if this
is the most depressing thing or and my

1685
01:35:37,040 --> 01:35:40,237
wife is not here and hopefully won't
kill me if this is being live streamed

1686
01:35:40,320 --> 01:35:42,477
right now, but >> No,
we won't broadcast this. So, I am

1687
01:35:42,560 --> 01:35:46,397
constantly dictating to my computer, to
my phone. And the other day, it was like

1688
01:35:46,480 --> 01:35:49,037
Friday night, we had put our son to bed.

1689
01:35:49,120 --> 01:35:51,197
We're both on our laptops and I'm on one

1690
01:35:51,280 --> 01:35:53,437
side of the couch dictating. And then I

1691
01:35:53,520 --> 01:35:58,397
look over and she's also has her laptop
open and is kind of whispering to her

1692
01:35:58,480 --> 01:36:01,997
computer, too. And I'm like, is this the
future of tech? But it's a new computing

1693
01:36:02,080 --> 01:36:03,677
interface. So, I'm happy about it.

1694
01:36:03,760 --> 01:36:05,197
>> Right. I look,
I think the the gentleman

1695
01:36:05,280 --> 01:36:08,957
brings up a great question, which
is that things are moving so fast. How do

1696
01:36:09,040 --> 01:36:12,877
you plan right now? And honestly,
I don't know how you do it because every

1697
01:36:12,960 --> 01:36:16,157
day there seems to be a new capability,
then the capability is taken off the

1698
01:36:16,240 --> 01:36:20,637
table. Like Fable, for instance, I mean,
the tweets about Fable where like people

1699
01:36:20,720 --> 01:36:23,997
are adding Dave Sachs and they're like,
"Please, I'll do anything for Fable

1700
01:36:24,080 --> 01:36:27,757
back. Just bring it back." And he won't
bring it back. But it's just like I

1701
01:36:27,840 --> 01:36:30,477
don't understand how any company does
that. And I think that actually would be

1702
01:36:30,560 --> 01:36:33,757
a good topic for us to sort
of get into on a future show.

1703
01:36:33,840 --> 01:36:36,317
>> Yep. Okay, let's go this way.

1704
01:36:36,400 --> 01:36:37,837
>> Hi, thank you for taking my question.

1705
01:36:37,920 --> 01:36:40,397
Um, I was curious your views on in the

1706
01:36:40,480 --> 01:36:42,477
next like five years as AR glasses

1707
01:36:42,560 --> 01:36:45,197
evolve whether like the chunky

1708
01:36:45,280 --> 01:36:49,677
spectacles is the way to go or thinner
glasses with like upboard compute like

1709
01:36:49,760 --> 01:36:54,237
either wireless to your iPhone or like
the vision pro like cable down to the

1710
01:36:54,320 --> 01:36:57,437
battery pack and compute. Well, it's
a battery for Vision Pro, but it could

1711
01:36:57,520 --> 01:37:00,397
also have compute on board. So,
I'm curious like the next five years where

1712
01:37:00,480 --> 01:37:04,797
you think consumers will gravitate
towards. No, it's a it's a great

1713
01:37:04,880 --> 01:37:08,637
question because if you ever use
Magic Leap, there was like a >> puck.

1714
01:37:08,720 --> 01:37:11,597
>> Puck. That's what it was.
>> My Here's my hot take. Anything any

1715
01:37:11,680 --> 01:37:14,157
device that requires
a puck >> not working.

1716
01:37:14,240 --> 01:37:15,597
>> Well, how about this though?

1717
01:37:15,680 --> 01:37:19,117
>> Okay. >> What if what
if the puck is your iPhone?

1718
01:37:19,200 --> 01:37:23,197
>> Oh All right. >> Exactly.
So, which gives Apple an

1719
01:37:23,280 --> 01:37:27,197
opportunity that like if the compute
is taking place on the phone in your

1720
01:37:27,280 --> 01:37:31,677
pocket, it allows you to be much slimmer
from a power perspective. you don't want

1721
01:37:31,760 --> 01:37:33,837
like a lightning cable connected to your

1722
01:37:33,920 --> 01:37:36,477
face. But like it's still I think there

1723
01:37:36,560 --> 01:37:39,837
is a it makes me think Apple still has a

1724
01:37:39,920 --> 01:37:43,757
good chance in this space because
the iPhone can do all of the heavy lifting

1725
01:37:43,840 --> 01:37:48,157
versus this thing on that
is really heavy on your head.

1726
01:37:48,240 --> 01:37:50,317
>> Can I can I ask so what's your name?

1727
01:37:50,400 --> 01:37:55,757
>> I'm Kyle. >> Kyle, so uh what do you
want to use a like face computer for?

1728
01:37:55,840 --> 01:37:58,557
>> Oh that's a good question.
>> See I told you Ron John this stuff is

1729
01:37:58,640 --> 01:38:01,757
not happening. No, no, no.
Kyle's got something.

1730
01:38:01,840 --> 01:38:03,917
>> All right, let's give
Kyle an opportunity here.

1731
01:38:04,000 --> 01:38:06,157
>> Let me think for >> I didn't
mean to put you on the spot,

1732
01:38:06,240 --> 01:38:09,597
but thank you for helping
me prove my point.

1733
01:38:09,680 --> 01:38:12,317
>> Oh, I think one
cool thing would be like

1734
01:38:12,400 --> 01:38:16,397
a shared uh like TV or something like

1735
01:38:16,480 --> 01:38:21,117
like imagine Vision Pro and you have
just a shared just like movie theater.

1736
01:38:21,200 --> 01:38:24,557
You're on a plane with your family
or something and you just have like the

1737
01:38:24,640 --> 01:38:28,397
shared experience, but it's just glasses.
I like that. I like that. Like

1738
01:38:28,480 --> 01:38:31,597
for me, one of the coolest things I
always like cuz you know, as you get

1739
01:38:31,680 --> 01:38:35,997
older, you move further away from your
friends and uh it would be cool to like

1740
01:38:36,080 --> 01:38:38,317
be in the vision pro together.

1741
01:38:38,400 --> 01:38:41,997
>> Yeah. Sit sit half court, watch
the Knicks next to Timothy Chalamay, but

1742
01:38:42,080 --> 01:38:45,757
we're just in division pro. But you
know what's interesting? Apple never

1743
01:38:45,840 --> 01:38:48,557
advertised that as a social device. All

1744
01:38:48,640 --> 01:38:50,957
the marketing was just you're sitting

1745
01:38:51,040 --> 01:38:55,597
alone at home and all you're doing
is the Vision Pro. Nobody else exists.

1746
01:38:55,680 --> 01:38:58,797
>> The other big one. Yeah, sorry
to interrupt. I was say the other big one

1747
01:38:58,880 --> 01:39:01,757
for me is just having >> a lot
of monitors around when I'm

1748
01:39:01,840 --> 01:39:05,757
working. Like I even though I have like
an ultra wide or like three monitors,

1749
01:39:05,840 --> 01:39:07,517
feel like sometimes I could have more.

1750
01:39:07,600 --> 01:39:10,317
So just being able to like interact with

1751
01:39:10,400 --> 01:39:14,637
AI to pull up the exact page I'm looking
for out of like I'm one of those people

1752
01:39:14,720 --> 01:39:18,797
has like a thousand tabs. So just being
able to pull up the thing and I just say

1753
01:39:18,880 --> 01:39:21,437
open up that tab and it just
pops up would be pretty cool.

1754
01:39:21,520 --> 01:39:24,317
>> Would 8,000 individual tabs in a giant

1755
01:39:24,400 --> 01:39:29,357
planner space be better
or Kyle's going with this.

1756
01:39:29,440 --> 01:39:33,757
>> No, no, but I do agree like
>> that being able to do more

1757
01:39:33,840 --> 01:39:37,997
>> open scaled work and like look
at different charts and I think there's

1758
01:39:38,080 --> 01:39:42,157
that still and I know like I have friends
who own the vision pro who use it for that

1759
01:39:42,240 --> 01:39:47,037
>> still. >> No, but did >> did
they had a nice time? All right,

1760
01:39:47,120 --> 01:39:49,037
thank you Kyle. Let's go here.

1761
01:39:49,120 --> 01:39:52,797
>> Hey, what's up? This is Sasha from Yo

1762
01:39:52,880 --> 01:39:56,877
University doing research on AI agents
for finance. First of all, love you

1763
01:39:56,960 --> 01:39:59,117
guys. Listening to you guys chat on my

1764
01:39:59,200 --> 01:40:04,077
way to work has been my routine. I think
this is excitement shed by not just me

1765
01:40:04,160 --> 01:40:06,717
but many people in the audience.
So, thank you so much.

1766
01:40:06,800 --> 01:40:09,837
>> Thank you. >> Thank you.
>> Appreciate that. And thank you for

1767
01:40:09,920 --> 01:40:11,837
coming all the Did you fly in?

1768
01:40:11,920 --> 01:40:15,277
>> Uh yes. >> Oh my god. Thank you
for coming. It's great to see you.

1769
01:40:15,360 --> 01:40:19,037
>> It's worth it. Um and I appreciate
your sharp line of uh reasoning and

1770
01:40:19,120 --> 01:40:21,597
questioning. So uh what's your view on

1771
01:40:21,680 --> 01:40:23,517
the meter benchmarks of how long the AI

1772
01:40:23,600 --> 01:40:26,077
and AI agents can work independently

1773
01:40:26,160 --> 01:40:28,797
being saturated? Are we at that point

1774
01:40:28,880 --> 01:40:32,397
that u of them being at infinite work

1775
01:40:32,480 --> 01:40:34,717
number of hours working yet or are we

1776
01:40:34,800 --> 01:40:37,837
reaching that soon and what is the world

1777
01:40:37,920 --> 01:40:41,917
going to be looking like after
we hit that infinite mark?

1778
01:40:42,000 --> 01:40:43,597
>> That is an excellent
question. So, first

1779
01:40:43,680 --> 01:40:47,997
of all, there is some controversy about
the meter monitoring, but I think it's

1780
01:40:48,080 --> 01:40:49,917
kind of directionally accurate, right?

1781
01:40:50,000 --> 01:40:54,077
Like I showed the meter chart at the
very beginning of of our event today

1782
01:40:54,160 --> 01:40:56,317
that like you saw these models, they

1783
01:40:56,400 --> 01:41:01,437
could not code autonomously
for more than 30 minutes in 2023

1784
01:41:01,520 --> 01:41:06,637
and now they can code autonomously
for the equivalent of 18 hours, right? Um,

1785
01:41:06,720 --> 01:41:09,677
so what happens if they just kind
of blow past that limit and then they can

1786
01:41:09,760 --> 01:41:13,597
code all the time? Um that is an
excellent question. Do you have any

1787
01:41:13,680 --> 01:41:16,957
thoughts? >> So I I felt
that with the go mode and

1788
01:41:17,040 --> 01:41:21,437
automation mode uh it practically
felt that they are already doing this

1789
01:41:21,520 --> 01:41:23,357
autonomously forever. >> Right.

1790
01:41:23,440 --> 01:41:27,677
>> Uh I want to want them to send me
reports. My prompt would be if you find

1791
01:41:27,760 --> 01:41:29,357
something interesting send me an email.

1792
01:41:29,440 --> 01:41:33,277
Yes. >> Then I when I see it I will come
back and intervene. I felt that for many

1793
01:41:33,360 --> 01:41:35,917
tasks it's already at that mark.

1794
01:41:36,000 --> 01:41:40,237
>> Yeah. No a perplexity computer will
basically do that. Um, so that's that's

1795
01:41:40,320 --> 01:41:41,437
definitely something that's happening.

1796
01:41:41,520 --> 01:41:44,477
But I'll just say one more thing
and then we'll go to Ron John. Um, Greg

1797
01:41:44,560 --> 01:41:48,397
Brockman who will be here later has this
like idea of a compute powered economy

1798
01:41:48,480 --> 01:41:52,077
where like you know you sort of once you
get these bots working the way that you

1799
01:41:52,160 --> 01:41:56,157
explained you just maybe he'll say this
later um you just kind of throw them at

1800
01:41:56,240 --> 01:41:59,677
any problem and then they just kind
of work autonomously through it. Now we

1801
01:41:59,760 --> 01:42:02,797
don't we've never seen what the world
looks like when you can do something

1802
01:42:02,880 --> 01:42:06,317
like that but I do think given the
progress that we've seen with the models

1803
01:42:06,400 --> 01:42:09,197
you would imagine that something
substantial will come at a certain

1804
01:42:09,280 --> 01:42:12,717
point. So uh my day job I work at a

1805
01:42:12,800 --> 01:42:14,957
company writer. We're an enterprise AI

1806
01:42:15,040 --> 01:42:19,997
company and uh the when and the the goal
mode was asked about recently again the

1807
01:42:20,080 --> 01:42:23,517
idea that you just provide the goal.
The agent will iteratively loop and keep

1808
01:42:23,600 --> 01:42:25,277
doing work until it finds the right

1809
01:42:25,360 --> 01:42:28,557
solution. In the real world that is like

1810
01:42:28,640 --> 01:42:32,877
the benchmarks versus the real world.
I still think there's such a massive gap

1811
01:42:32,960 --> 01:42:35,517
in terms of what does the data look

1812
01:42:35,600 --> 01:42:37,677
like? What what is the actual problem?

1813
01:42:37,760 --> 01:42:40,317
Does the customer or the person actually

1814
01:42:40,400 --> 01:42:42,477
understand the goal in a clear enough

1815
01:42:42,560 --> 01:42:44,717
way that they're able to define it to

1816
01:42:44,800 --> 01:42:47,837
kind of push that loop forward? So I

1817
01:42:47,920 --> 01:42:49,757
think it's an interesting I think with a

1818
01:42:49,840 --> 01:42:52,397
lot of AI again even around the

1819
01:42:52,480 --> 01:42:57,677
benchmarking like again versus real
world understanding of how to use it,

1820
01:42:57,760 --> 01:43:01,597
what's happening and what's available
for it to use. There's still a lot of uh

1821
01:43:01,680 --> 01:43:04,317
work to be done. I mean you you can do

1822
01:43:04,400 --> 01:43:09,277
these goal modes for like
less complicated tasks. I like

1823
01:43:09,360 --> 01:43:12,557
>> what have you goal modeed?
>> Uh so I worked with Perplexity Computer

1824
01:43:12,640 --> 01:43:15,517
recently to um to try to like have it

1825
01:43:15,600 --> 01:43:19,837
find a hotel discount for me.
>> No travel stories.

1826
01:43:19,920 --> 01:43:22,157
I I I've been on a this has been for

1827
01:43:22,240 --> 01:43:25,037
like two years. Every time I remember

1828
01:43:25,120 --> 01:43:27,837
like Sundar actually it was I don't know

1829
01:43:27,920 --> 01:43:32,397
if people remember in 2019 my example.
Wait, wait, no, no. I'm just

1830
01:43:32,480 --> 01:43:34,637
saying. Why does everyone when they talk

1831
01:43:34,720 --> 01:43:37,677
about Agentic talk about travel? I get

1832
01:43:37,760 --> 01:43:39,997
>> cuz travel is such a pain in the ass.

1833
01:43:40,080 --> 01:43:42,797
>> But it's I don't Okay, go on. Go on.

1834
01:43:42,880 --> 01:43:46,797
>> I'm I'm I'm sheepish now. >> No,
no, no. Hotel discount. Hotel discount.

1835
01:43:46,880 --> 01:43:50,157
>> I just looked at the hotel every hour
and when it dropped behind us below a

1836
01:43:50,240 --> 01:43:52,397
certain threshold, emailed me.

1837
01:43:52,480 --> 01:43:54,317
>> That's pretty good. >> Okay,
that's a clear goal. I will give

1838
01:43:54,400 --> 01:43:57,757
you that as a clear goal. >> This is >>
I respect your I respect I think you're

1839
01:43:57,840 --> 01:43:59,917
right that we definitely
need better examples than

1840
01:44:00,000 --> 01:44:04,237
>> It's a pet peeve. It's just
for some reason every Apple the original

1841
01:44:04,320 --> 01:44:07,517
ridiculous Bella Ramsey commercials
around Apple intelligence of core

1842
01:44:07,600 --> 01:44:11,677
everything is flight booking and again
like >> you have killed that Bella Ramsey

1843
01:44:11,760 --> 01:44:15,357
commercial so often they're going to the
creative agency is just going to write

1844
01:44:15,440 --> 01:44:18,557
to you at some point >> and I
love the Last of Us and I love

1845
01:44:18,640 --> 01:44:22,237
Bella Ramsey but those
commercials still irked me. Yeah.

1846
01:44:22,320 --> 01:44:24,477
>> What happened in those commercials?

1847
01:44:24,560 --> 01:44:28,317
uh one of them she's sitting at someone
that she can't remember who that person

1848
01:44:28,400 --> 01:44:32,237
is and in real time asks Apple
intelligence like tell me about this

1849
01:44:32,320 --> 01:44:36,477
person my interactions which again is just
such a weird thing like I'm so much

1850
01:44:36,560 --> 01:44:40,237
better than you that I don't know who
you are and I need to remember who you

1851
01:44:40,320 --> 01:44:43,117
are and have AI tell me but and it

1852
01:44:43,200 --> 01:44:47,357
didn't even come close to working with
Apple intelligence so that was that was

1853
01:44:47,440 --> 01:44:50,957
the worst one >> you know what would be
good for that actually >> what goal mode

1854
01:44:51,040 --> 01:44:55,197
>> AR glasses >> oh AR glasses
is there's the real world use case

1855
01:44:55,280 --> 01:45:00,077
>> or the fact that that commercial was so
bad just again proves my point that okay

1856
01:45:00,160 --> 01:45:03,277
AR glasses aren't going to work. Okay,
sorry. Thank you so much for the question.

1857
01:45:03,360 --> 01:45:06,717
>> Thank you and we'll go over here.

1858
01:45:06,800 --> 01:45:12,557
>> Oh, he was >> okay fine.
We'll go here. Okay. Um, hi.

1859
01:45:12,640 --> 01:45:15,917
Um, >> hey Gerald Harris.
Uh, I'm on the board

1860
01:45:16,000 --> 01:45:19,677
of the Commonwealth Club here and I I
run some uh programs for the club and I

1861
01:45:19,760 --> 01:45:22,877
have the small scenario planning
consultancy. But here's my question that

1862
01:45:22,960 --> 01:45:26,317
I think hasn't come up here. Um,

1863
01:45:26,400 --> 01:45:28,157
what what what should we be concerned

1864
01:45:28,240 --> 01:45:31,917
about in terms of using the AI models

1865
01:45:32,000 --> 01:45:34,237
them building a database on us and then

1866
01:45:34,320 --> 01:45:36,877
turning that into advertising? So the

1867
01:45:36,960 --> 01:45:39,757
advertising potential revenue from some

1868
01:45:39,840 --> 01:45:42,317
from users, do you think the AI

1869
01:45:42,400 --> 01:45:44,477
companies will ever go after that

1870
01:45:44,560 --> 01:45:47,277
revenue or use that information for

1871
01:45:47,360 --> 01:45:49,437
advertising purposes?

1872
01:45:49,520 --> 01:45:51,997
>> Oo, that's a good one.
So, will will AIS

1873
01:45:52,080 --> 01:45:55,357
build an amazing uh sort of profile of

1874
01:45:55,440 --> 01:45:59,677
you based off of all the personal data
and then use that for ads? Most certainly.

1875
01:45:59,760 --> 01:46:02,317
>> So, someone left one of the companies
and wrote about that in the New York

1876
01:46:02,400 --> 01:46:05,197
Times about three months ago.
>> No, definitely. Yeah, that that's

1877
01:46:05,280 --> 01:46:10,157
coming. Um, I I think that um
advertising is obviously going to be one

1878
01:46:10,240 --> 01:46:13,757
use where we're going to start to see
some of the problematic stuff here, but

1879
01:46:13,840 --> 01:46:17,757
actually if you look at the ads that
OpenAI gave you, it's sort of like more

1880
01:46:17,840 --> 01:46:21,757
of a and obviously they they always come
out with the hightouch brand and then

1881
01:46:21,840 --> 01:46:25,677
they will like get you on the direct
response like super targeted stuff um

1882
01:46:25,760 --> 01:46:29,757
once they realize they can't make money
on brand. Um, but like it looks pretty

1883
01:46:29,840 --> 01:46:33,277
good. Like if you're sorry to go with
the travel example again, but if you're

1884
01:46:33,360 --> 01:46:37,357
like researching travel and chat GPT,
like you can go into an advertising chat

1885
01:46:37,440 --> 01:46:39,357
experience and it will help you. But I I

1886
01:46:39,440 --> 01:46:44,637
think that like we have never had
technology ever that collecting this

1887
01:46:44,720 --> 01:46:49,517
much information about us. Has anyone
here um like talk to like chat GPT or

1888
01:46:49,600 --> 01:46:51,357
Claude and say, "Can you psychoanalyze

1889
01:46:51,440 --> 01:46:57,037
me if you're not or give me
any any insight about myself?

1890
01:46:57,120 --> 01:47:00,317
Yeah, >> just like five of you.
The rest of you have done it.

1891
01:47:00,400 --> 01:47:02,877
>> Admit it. >> It's scary
and and we're g and that's

1892
01:47:02,960 --> 01:47:06,557
the like gated stuff and we're
giving this all this information to to

1893
01:47:06,640 --> 01:47:10,637
companies and it's a a trust thing that we
don't really know where it's going to

1894
01:47:10,720 --> 01:47:13,597
go. >> Yeah, I think
it's interesting because

1895
01:47:13,680 --> 01:47:17,517
again, OpenAI originally ads are going
to be a big part of the business. Now

1896
01:47:17,600 --> 01:47:22,157
they've pivoted away from that a bit
but are still releasing ad products. At

1897
01:47:22,240 --> 01:47:24,317
least Anthropic is not I mean I don't

1898
01:47:24,400 --> 01:47:25,677
think they're going in that direction at

1899
01:47:25,760 --> 01:47:29,357
all. Google and Gemini obviously 100%

1900
01:47:29,440 --> 01:47:32,717
will and like you probably will get a

1901
01:47:32,800 --> 01:47:35,677
really good ad experience. I mean the

1902
01:47:35,760 --> 01:47:40,717
more it knows about you the types
of questions you're asking but it is like

1903
01:47:40,800 --> 01:47:42,637
you know like thousands and tens of

1904
01:47:42,720 --> 01:47:44,717
thousands of three to five word Google

1905
01:47:44,800 --> 01:47:48,557
searches are a lot different than entire

1906
01:47:48,640 --> 01:47:51,837
thought partnership therapy like

1907
01:47:51,920 --> 01:47:57,357
research exploration. these models will
know everything about you and it is it's

1908
01:47:57,440 --> 01:47:59,997
terrifying. >> Have you done
the psychoanalyze me prompt?

1909
01:48:00,080 --> 01:48:04,237
>> No, but I did so I use like Claude and

1910
01:48:04,320 --> 01:48:08,957
Chat GPT Gemini. I'll kind of go through
the three of them and cycle around. So

1911
01:48:09,040 --> 01:48:12,317
it is funny. So I have asked like what
do you know about me? Like what would

1912
01:48:12,400 --> 01:48:14,797
you ask back? And they all have very

1913
01:48:14,880 --> 01:48:17,517
different kind of like jagged jarring

1914
01:48:17,600 --> 01:48:20,157
aggregations of who I am. Like I don't

1915
01:48:20,240 --> 01:48:26,077
know. They're uh I I I don't have one
that just knows me through and through.

1916
01:48:26,160 --> 01:48:27,517
>> I I've done it. >> I diversify them.

1917
01:48:27,600 --> 01:48:29,037
>> I've done it with chat GPT.

1918
01:48:29,120 --> 01:48:30,957
>> Yeah, >> it's pretty good. >> Yeah.

1919
01:48:31,040 --> 01:48:33,517
>> And then like it will first give you a

1920
01:48:33,600 --> 01:48:38,237
answer that's like kind of sanitized and
then you say go a little deeper and then

1921
01:48:38,320 --> 01:48:42,157
you say get a little darker. >> Oh dear.

1922
01:48:42,240 --> 01:48:45,197
>> I have not redteamed uh chat TV.

1923
01:48:45,280 --> 01:48:47,677
>> I challenge you all to give it a shot.

1924
01:48:47,760 --> 01:48:49,837
Don't go darker. Don't go darker.

1925
01:48:49,920 --> 01:48:54,477
>> But if you really want to. >> Okay.
Thank you over here. And thank you

1926
01:48:54,560 --> 01:48:56,077
for the question.
>> Let's move on. Let's move on.

1927
01:48:56,160 --> 01:48:58,557
>> Yeah. >> Thank you guys
and really appreciate all

1928
01:48:58,640 --> 01:49:02,877
the work that you do on the podcast
and YouTube. Uh your conversations go so

1929
01:49:02,960 --> 01:49:05,597
deep and still very broad.
So I really appreciate that.

1930
01:49:05,680 --> 01:49:08,157
>> Thank you. We appreciate it.
You're you're a listener or viewer.

1931
01:49:08,240 --> 01:49:10,237
>> Oh yeah. All right.
Both. >> Thank you so much.

1932
01:49:10,320 --> 01:49:12,637
>> Depends on where I am.
>> Amazing. Appreciate it.

1933
01:49:12,720 --> 01:49:17,437
>> Um and you know my name is Si. I am
a adviser for startups in the AI space. Um

1934
01:49:17,520 --> 01:49:19,437
and I've been in product management and

1935
01:49:19,520 --> 01:49:22,477
go to market strategy. Um my question

1936
01:49:22,560 --> 01:49:27,197
for you some somewhat related to the
gentleman before you know there's a lot

1937
01:49:27,280 --> 01:49:29,437
of coverage on agents and what agents

1938
01:49:29,520 --> 01:49:31,917
can do and uh a little bit about the

1939
01:49:32,000 --> 01:49:34,557
governance and cost control. What I

1940
01:49:34,640 --> 01:49:38,157
don't see a lot about um you know is you

1941
01:49:38,240 --> 01:49:40,477
know agents do drift probably why we

1942
01:49:40,560 --> 01:49:43,437
don't have better examples than travel.

1943
01:49:43,520 --> 01:49:45,437
Um and you know over the a period of

1944
01:49:45,520 --> 01:49:47,437
time they don't get better at that

1945
01:49:47,520 --> 01:49:52,637
specific uh you know uh understanding
of the the people and the business context

1946
01:49:52,720 --> 01:49:54,077
and the operating model they're working

1947
01:49:54,160 --> 01:49:58,557
in um and you know learn from it and I

1948
01:49:58,640 --> 01:50:01,197
was wondering is there is there a lot of

1949
01:50:01,280 --> 01:50:02,877
conversation that you guys are seeing

1950
01:50:02,960 --> 01:50:06,237
around uh the learning of uh or the

1951
01:50:06,320 --> 01:50:09,277
learning layer in the agents and is it

1952
01:50:09,360 --> 01:50:12,477
too early or you know um for that to

1953
01:50:12,560 --> 01:50:16,877
happen And the conversation is mostly
around governance and cost and access.

1954
01:50:16,960 --> 01:50:18,477
>> It's actually a good
question for Ronan.

1955
01:50:18,560 --> 01:50:20,957
>> Yeah. So again,
like working in this all

1956
01:50:21,040 --> 01:50:24,557
day and thank you for the question.
I I think what's going to happen and I'm

1957
01:50:24,640 --> 01:50:27,117
already starting to see is like 6 to 12

1958
01:50:27,200 --> 01:50:31,677
months ago the approach was just jam
as much information into whatever system

1959
01:50:31,760 --> 01:50:36,237
you can. And we were kind of promised
that it would just work and AI is just

1960
01:50:36,320 --> 01:50:39,997
going to and like there's you know you'd
get this feeling that oh wait a 100page

1961
01:50:40,080 --> 01:50:44,317
PDF it actually could parse and I can
get information out of but then when

1962
01:50:44,400 --> 01:50:46,397
you're doing that at any kind of agentic

1963
01:50:46,480 --> 01:50:49,357
scale it doesn't and I think already

1964
01:50:49,440 --> 01:50:52,077
like how information gets chunked up and

1965
01:50:52,160 --> 01:50:54,397
like distributed in different ways and

1966
01:50:54,480 --> 01:50:57,197
context windows everyone would see like

1967
01:50:57,280 --> 01:51:02,157
1 million uh token context window but we
didn't really know what it meant and now

1968
01:51:02,240 --> 01:51:05,037
more and more like and I actually think
this is going to be one of the most

1969
01:51:05,120 --> 01:51:07,117
interesting like professional

1970
01:51:07,200 --> 01:51:09,997
opportunities and areas to be an expert

1971
01:51:10,080 --> 01:51:12,637
in going forward is like being able to

1972
01:51:12,720 --> 01:51:17,677
kind of it's funny like it is as much
art as science right now. But I think

1973
01:51:17,760 --> 01:51:21,277
like the more you can start to grab
a hold of how information flows through

1974
01:51:21,360 --> 01:51:25,837
these agents and how you actually get
it reliable, it's going to be a really

1975
01:51:25,920 --> 01:51:30,317
really interesting space.
It's not it's no one job right now.

1976
01:51:30,400 --> 01:51:32,957
>> Thank you. Okay, thank you
for the question. All right, we have seven

1977
01:51:33,040 --> 01:51:35,677
minutes, three questions. Let's see
if we can keep the questions brief and

1978
01:51:35,760 --> 01:51:37,757
we'll keep the answers brief
and we'll get to everybody.

1979
01:51:37,840 --> 01:51:41,117
>> I'll try to be quick.
>> Hey everyone, uh big fan of the pod by

1980
01:51:41,200 --> 01:51:45,437
the way. Flew Chris Auei. I'm one of
the co-founders of a company called Virus

1981
01:51:45,520 --> 01:51:47,757
Watcher. I'm here >> with my colleague.

1982
01:51:47,840 --> 01:51:51,437
>> Flew in from Dallas.
He flew in from Austin. Yes. A big fan.

1983
01:51:51,520 --> 01:51:53,357
>> We got a couple
Dallas folks here today.

1984
01:51:53,440 --> 01:51:54,957
>> Oh, who's Dallas? Shout out Dallas.

1985
01:51:55,040 --> 01:51:57,037
There we go. Okay. Go Matt. >> All right.

1986
01:51:57,120 --> 01:51:59,837
>> Oh, yeah. We're big in Texas.

1987
01:51:59,920 --> 01:52:04,157
Um I'm going to go in a different
direction uh with my question is uh I

1988
01:52:04,240 --> 01:52:06,877
want to know kind of what y'all's
thoughts and honestly if you can ask

1989
01:52:06,960 --> 01:52:08,397
this kind of later on with Greg Brockman

1990
01:52:08,480 --> 01:52:11,597
and others um what is the thought on

1991
01:52:11,680 --> 01:52:15,837
using AI models and AI you know
technology for bi biological

1992
01:52:15,920 --> 01:52:20,397
intelligence bioveurveillance and public
health for emerging risk detection with

1993
01:52:20,480 --> 01:52:23,677
infectious diseases is a very
interesting project that we're working

1994
01:52:23,760 --> 01:52:27,677
on we have a lot of epidemiologists
on our team we were just at the UN two

1995
01:52:27,760 --> 01:52:31,677
weeks ago with the World Health
Organization um and trying to discuss

1996
01:52:31,760 --> 01:52:36,397
these problems and how we can use the
technology to to kind of uh detect these

1997
01:52:36,480 --> 01:52:39,117
things early on. So I kind of want
to get you guys thoughts on that.

1998
01:52:39,200 --> 01:52:41,277
>> Can I ask you
a question back and if you

1999
01:52:41,360 --> 01:52:42,957
could give a brief answer
that would be great.

2000
01:52:43,040 --> 01:52:46,717
>> Okay. >> Are you a believer in the
biocapabilities of these models like

2001
01:52:46,800 --> 01:52:50,877
Fable? Like do you think Anthropic
made the right decision by gating them?

2002
01:52:50,960 --> 01:52:54,797
>> Wait. Biocapabilities meaning I think
you're talking about biological weapons.

2003
01:52:54,880 --> 01:52:56,637
He's talking about like disease correct.

2004
01:52:56,720 --> 01:52:59,597
But I I think that biological his answer

2005
01:52:59,680 --> 01:53:03,277
will give us insight into the

2006
01:53:03,360 --> 01:53:06,477
>> I I mean I think it's a it's a very

2007
01:53:06,560 --> 01:53:09,357
touchy you know it's a there's no right

2008
01:53:09,440 --> 01:53:14,237
answer I don't think um but I do think
it probably did more harm uh you know

2009
01:53:14,320 --> 01:53:17,277
than you know not allowing everyone
to kind of take advantage of the

2010
01:53:17,360 --> 01:53:20,557
>> so you're not you're not basically
worried that today's cutting edge models

2011
01:53:20,640 --> 01:53:25,597
could cause like new bio threats >>
no I I I am but I think there's also a

2012
01:53:25,680 --> 01:53:29,437
lot of positive that come out of it
as well everybody to use it though.

2013
01:53:29,520 --> 01:53:33,197
>> Yeah. I mean >> I I just
you know it doesn't even have

2014
01:53:33,280 --> 01:53:37,677
to be a fable style model. It can be you
can take a frontier model distill a more

2015
01:53:37,760 --> 01:53:41,197
um >> right >> custom model.
Yeah. Safety uh was you

2016
01:53:41,280 --> 01:53:44,317
know safety standards involved
but sorry I don't want to be too long.

2017
01:53:44,400 --> 01:53:47,117
No great question Ron John

2018
01:53:47,200 --> 01:53:52,077
>> I in terms of the first part of it I
do think again like and it's one of the

2019
01:53:52,160 --> 01:53:56,397
underappreciated things and I feel
in the AI industry like disease prevention

2020
01:53:56,480 --> 01:54:01,277
and scenario modeling around vir like
we don't talk about that stuff enough or

2021
01:54:01,360 --> 01:54:05,997
whoever is trying to talk about it it
does not get listened to enough so I

2022
01:54:06,080 --> 01:54:10,637
love the kind of work you're doing
and like those kind of stories would bring

2023
01:54:10,720 --> 01:54:14,637
the AI industry a little bit better
reputation Yeah. No, I I I agree.

2024
01:54:14,720 --> 01:54:15,997
>> You're thinking
about biological weapons.

2025
01:54:16,080 --> 01:54:18,957
>> No, I think that that if Well, if you
can do one, then you could certainly do

2026
01:54:19,040 --> 01:54:23,117
the other. Am I >> I don't know.
I'm not >> Is one like Okay.

2027
01:54:23,200 --> 01:54:27,117
>> I actually have no idea. >> Why are you
giving me this all these problems here?

2028
01:54:27,200 --> 01:54:30,877
>> Uh no, I I I hope that we
can and we know that's a very

2029
01:54:30,960 --> 01:54:33,117
>> No, no, no. I love it.
>> We're going to speak with we'll we'll

2030
01:54:33,200 --> 01:54:35,917
hopefully have some time to talk
about health with Brockman.

2031
01:54:36,000 --> 01:54:37,597
>> Yeah, I would love Yeah, if you can.

2032
01:54:37,680 --> 01:54:39,837
Okay. >> Okay.
Thank you for the question over

2033
01:54:39,920 --> 01:54:43,597
here. Hi. Uh my name is Heidi. I'm your

2034
01:54:43,680 --> 01:54:47,277
linking online friend. >> That's right.
You messaged me yesterday.

2035
01:54:47,360 --> 01:54:51,197
Yes. >> Yeah. Yeah.
Uh during this uh event uh

2036
01:54:51,280 --> 01:54:53,757
see each other in person. Uh my question

2037
01:54:53,840 --> 01:54:56,797
about uh before you with a different

2038
01:54:56,880 --> 01:54:59,437
speaker mention about China. So I'm

2039
01:54:59,520 --> 01:55:02,637
Chinese. I'm curious about uh why you

2040
01:55:02,720 --> 01:55:05,437
think uh Chinese uh China is important

2041
01:55:05,520 --> 01:55:09,757
base partnership with uh uh USA special

2042
01:55:09,840 --> 01:55:12,237
high-tech. Uh second question about R.

2043
01:55:12,320 --> 01:55:16,397
So you mentioned about RM large model.

2044
01:55:16,480 --> 01:55:20,797
So my question about uh Gemini JDP and

2045
01:55:20,880 --> 01:55:24,477
cloudy and an which one you prefer think

2046
01:55:24,560 --> 01:55:28,797
about in future is more stronger
smarter in the future? Thank you.

2047
01:55:28,880 --> 01:55:30,477
>> Thank you. Thank you for the question.

2048
01:55:30,560 --> 01:55:33,117
>> Well, the question
on China is is sorry,

2049
01:55:33,200 --> 01:55:35,197
what's the question on China?

2050
01:55:35,280 --> 01:55:38,317
>> No, no. Why you think it's important uh

2051
01:55:38,400 --> 01:55:42,157
Bis relationship with USA? So, yeah.

2052
01:55:42,240 --> 01:55:44,397
Thank you. >> You know,
all right. I will I will take

2053
01:55:44,480 --> 01:55:49,037
that question and then Ranjan will
answer which model he likes better. Um

2054
01:55:49,120 --> 01:55:52,717
you know, I I think that we're obviously
in this competition. The US and China

2055
01:55:52,800 --> 01:55:57,917
are in this competition and there's like
a lot of suspicion on both ends. And um

2056
01:55:58,000 --> 01:56:00,717
you know I've said in the past that like
it would be great if there was better

2057
01:56:00,800 --> 01:56:04,557
comp cooperation between the two
countries and people have told me that

2058
01:56:04,640 --> 01:56:09,117
I'm naive um but I'm not going to you
know lose that hope. I think that if if

2059
01:56:09,200 --> 01:56:12,237
I mean in some ways competition's good
right because then you'll just have

2060
01:56:12,320 --> 01:56:16,237
everybody striving to build a better
thing. But you know we just talked about

2061
01:56:16,320 --> 01:56:19,997
uh AI for health if the countries were
able to work together maybe on specific

2062
01:56:20,080 --> 01:56:22,077
initiatives um that would be much

2063
01:56:22,160 --> 01:56:25,197
welcome. I I think the most interesting

2064
01:56:25,280 --> 01:56:28,477
part of like US versus or US with China

2065
01:56:28,560 --> 01:56:33,437
in terms of AI in the last few months,
the conversation around moving towards

2066
01:56:33,520 --> 01:56:38,077
deepseek and adding it into your agentic
process or adding Chinese models did not

2067
01:56:38,160 --> 01:56:42,637
exist in any conversation I was in 12
months ago and now is in almost every

2068
01:56:42,720 --> 01:56:45,037
conversation at least as an option

2069
01:56:45,120 --> 01:56:48,397
because cost has become such a larger

2070
01:56:48,480 --> 01:56:52,237
part of the conversation. So I think
I think it's a good thing. I think the

2071
01:56:52,320 --> 01:56:54,877
more integration there is overall across

2072
01:56:54,960 --> 01:57:00,280
these systems, the better.
>> Yeah, but watch out for

2073
01:57:01,760 --> 01:57:04,077
>> the cat holding
the hands with Dario and

2074
01:57:04,160 --> 01:57:06,077
Sam is my favorite meme the whole year.

2075
01:57:06,160 --> 01:57:08,957
Okay, we have one minute left,
but let's see if we can do it in a minute.

2076
01:57:09,040 --> 01:57:10,157
>> All right, I'll go
through this quickly.

2077
01:57:10,240 --> 01:57:13,437
Hi, Maria Sarah Roberts.
I'm in business management consulting.

2078
01:57:13,520 --> 01:57:15,037
>> My question is a little bit different.

2079
01:57:15,120 --> 01:57:18,477
Um, as we are evolving into or entering

2080
01:57:18,560 --> 01:57:23,037
or in a world already where endless
possibilities are available, where and

2081
01:57:23,120 --> 01:57:26,877
how do you see the evolution of the
responsibilities that big tech companies

2082
01:57:26,960 --> 01:57:31,597
have in supporting making this world
better? Not the product and our

2083
01:57:31,680 --> 01:57:33,277
engagement and how we interact with all

2084
01:57:33,360 --> 01:57:35,757
the screens better, but rather like the

2085
01:57:35,840 --> 01:57:38,717
real big problems that the world has.

2086
01:57:38,800 --> 01:57:41,517
And not because they should, but because

2087
01:57:41,600 --> 01:57:46,317
they can. So, how do you think about
like the evolution of that as we step

2088
01:57:46,400 --> 01:57:50,957
into this world? >> I I I'll answer that.
Or do you want to answer? Go ahead.

2089
01:57:51,040 --> 01:57:54,717
>> I just we we had a backstage we had
a long rant because I'd forgotten that

2090
01:57:54,800 --> 01:57:56,877
Anthropic is a PBC public benefit

2091
01:57:56,960 --> 01:57:59,357
corporation, but I still think there is

2092
01:57:59,440 --> 01:58:02,157
a responsibility. I don't actually see

2093
01:58:02,240 --> 01:58:04,077
it happening in any kind of way because

2094
01:58:04,160 --> 01:58:06,637
competition is so fierce now. I wish it

2095
01:58:06,720 --> 01:58:10,157
would, but I don't see it happening. Um,

2096
01:58:10,240 --> 01:58:13,357
yeah. I um that that was a good answer.

2097
01:58:13,440 --> 01:58:16,957
Come on. We have we I want to make
people make sure everybody gets to

2098
01:58:17,040 --> 01:58:20,477
coffee. Oh, no. Screw it.
I'll answer it. Um,

2099
01:58:20,560 --> 01:58:23,117
so so Jeff Bezos so I think they have an

2100
01:58:23,200 --> 01:58:25,917
obligation to do good for the world and

2101
01:58:26,000 --> 01:58:31,037
um I don't know if and I know people
within the companies are you know are

2102
01:58:31,120 --> 01:58:34,637
very serious about that. So they have
their own side projects, but I think you

2103
01:58:34,720 --> 01:58:37,997
know as a society maybe we can't expect
them to do it out of the goodness of

2104
01:58:38,080 --> 01:58:42,477
their own hearts. Um Jeff Bezos was
recently doing an interview and he said

2105
01:58:42,560 --> 01:58:46,077
um he said I think what I do in the
private sector is going to outweigh

2106
01:58:46,160 --> 01:58:50,157
whatever I could do charitably.
Um I disagree with that and I think that

2107
01:58:50,240 --> 01:58:53,917
that's just the mindset of a lot of people
at the top of these companies and

2108
01:58:54,000 --> 01:58:58,797
so like everybody else needs to be aware
of that and not count on it even though

2109
01:58:58,880 --> 01:59:01,837
it would be nice.
So that's my perspective.

2110
01:59:01,920 --> 01:59:04,957
All right. Uh, Ronjan, will you come
back with us after the break and speak

2111
01:59:05,040 --> 01:59:09,997
with Alex Stamos and I about whether um
this mythos and fable stuff is material

2112
01:59:10,080 --> 01:59:11,917
or marketing?
>> You know, I would love to.

2113
01:59:12,000 --> 01:59:13,677
>> You guys want to get a coffee break?

2114
01:59:13,760 --> 01:59:18,360
>> You guys are amazing.
>> Thank you very much.

2115
01:59:18,720 --> 01:59:23,357
>> All right, we'll we'll be back here
3:25. Um, please enjoy the coffee and

2116
01:59:23,440 --> 01:59:27,037
we'll see you in a couple minutes.
So, you may have heard that AI is in some

2117
01:59:27,120 --> 01:59:31,437
trouble in Washington. Um, and that
story has led to a lot of guessing.

2118
01:59:31,520 --> 01:59:33,437
People have guessed, are these models

2119
01:59:33,520 --> 01:59:35,837
actually that dangerous? How much better

2120
01:59:35,920 --> 01:59:40,717
are they than previous generations? And
do they deserve to be restricted by the

2121
01:59:40,800 --> 01:59:44,237
federal government? We've certainly done
a lot of guessing. And I think the only

2122
01:59:44,320 --> 01:59:48,477
way to get to the bottom of this is to
speak with somebody who is deep in the

2123
01:59:48,560 --> 01:59:50,957
weeds on the cyber security aspects of

2124
01:59:51,040 --> 01:59:55,677
AI models, someone who secures them,
and someone who has spent a career working

2125
01:59:55,760 --> 01:59:59,917
in security. And that would be Alex
Stamos. Alex Damos is the former chief

2126
02:00:00,000 --> 02:00:04,157
security officer at Meta and he's
the current chief product officer at

2127
02:00:04,240 --> 02:00:06,397
Corridor. Um Ranjan and I have this

2128
02:00:06,480 --> 02:00:09,037
debate all the time. Is what's going on

2129
02:00:09,120 --> 02:00:13,517
with Mythos marketing or material? Well,
let's get to the bottom of it. Ranjan

2130
02:00:13,600 --> 02:00:17,197
will come out and be my co-er for this
one. So, ladies and gentlemen, please

2131
02:00:17,280 --> 02:00:23,000
welcome Alex Damos
and once again Ranjan Roy.

2132
02:00:25,760 --> 02:00:29,197
>> Thank you. >> Hey, thanks man.

2133
02:00:29,280 --> 02:00:31,597
Alex, boring times in your industry.

2134
02:00:31,680 --> 02:00:34,157
>> Yeah, it's an honor to be here. I am by

2135
02:00:34,240 --> 02:00:36,477
far the poorest person in the afternoon

2136
02:00:36,560 --> 02:00:41,597
lineup. It's like I bring down the the
mean uh net worth by a comma, I believe.

2137
02:00:41,680 --> 02:00:43,117
So, I do appreciate being here.

2138
02:00:43,200 --> 02:00:47,837
>> Well, Alex, um I don't want to quue
you into podcaster finances, but

2139
02:00:47,920 --> 02:00:50,157
>> I think you're good >> with the guests,
I guess. Yeah. Yeah. So

2140
02:00:50,240 --> 02:00:53,357
talk a little bit about um we're
obviously at this moment where the

2141
02:00:53,440 --> 02:00:57,357
government has pulled down or or
basically forced anthropic to pull down

2142
02:00:57,440 --> 02:01:01,197
fable or mythos uh its frontier
model >> right both of them. Yes.

2143
02:01:01,280 --> 02:01:02,477
>> What happened?

2144
02:01:02,560 --> 02:01:05,277
>> Right. So uh just for recap uh for

2145
02:01:05,360 --> 02:01:08,637
everybody uh last Friday uh what

2146
02:01:08,720 --> 02:01:10,237
happened was the white house became

2147
02:01:10,320 --> 02:01:12,877
aware of research that Amazon had did

2148
02:01:12,960 --> 02:01:15,837
had done. Um it is not unreasonable for

2149
02:01:15,920 --> 02:01:17,917
Amazon to do this work right. So uh

2150
02:01:18,000 --> 02:01:21,997
Amazon hosts uh anthropics models in in

2151
02:01:22,080 --> 02:01:25,277
what they call bedrock. Uh this includes

2152
02:01:25,360 --> 02:01:27,357
in all of the classified spaces. So if

2153
02:01:27,440 --> 02:01:29,837
the NSA is using anthropics models, it

2154
02:01:29,920 --> 02:01:32,557
is running in AWS secret cloud or top

2155
02:01:32,640 --> 02:01:34,877
secret cloud. Uh and so Amazon is doing

2156
02:01:34,960 --> 02:01:37,517
security testing. expect this was part

2157
02:01:37,600 --> 02:01:39,757
of getting ready to host Fable and

2158
02:01:39,840 --> 02:01:43,597
Bedrock and uh found some things that

2159
02:01:43,680 --> 02:01:47,837
they wanted to complain about with
Fable's protections that make Fable

2160
02:01:47,920 --> 02:01:49,277
different than Mythos. I guess we can

2161
02:01:49,360 --> 02:01:53,117
talk about the exact ones. Uh Amazon had

2162
02:01:53,200 --> 02:01:55,517
sent over these results to Anthropic.

2163
02:01:55,600 --> 02:01:59,037
There was some disagreement between
Anthropic and Amazon during the week of

2164
02:01:59,120 --> 02:02:03,437
how serious they were. Now, I don't
have the exact details here, but somehow

2165
02:02:03,520 --> 02:02:07,757
apparently Andy Jasse, the CEO of Amazon,
mentions this to somebody at the

2166
02:02:07,840 --> 02:02:09,997
White House, and the White House freaks

2167
02:02:10,080 --> 02:02:14,157
out and uh according to reporting uh

2168
02:02:14,240 --> 02:02:18,077
calls Amazon or Anthropic uh and uh is

2169
02:02:18,160 --> 02:02:21,757
not happy with how quickly they can
get Enthropic on the phone. Anthropic is

2170
02:02:21,840 --> 02:02:23,677
asking for more technical details.

2171
02:02:23,760 --> 02:02:25,997
They're not able to get details from
the White House of what the White House

2172
02:02:26,080 --> 02:02:27,997
wants. So, the White House orders them

2173
02:02:28,080 --> 02:02:31,757
to take Fable down. Uh, Enthropic says,

2174
02:02:31,840 --> 02:02:35,677
"No, we can just fix these issues. Why
don't we work it out together?" And as a

2175
02:02:35,760 --> 02:02:38,317
result, instead of working out together

2176
02:02:38,400 --> 02:02:40,557
on Friday afternoon, something around

2177
02:02:40,640 --> 02:02:42,797
5:00 p p.m. Pacific time, the Commerce

2178
02:02:42,880 --> 02:02:45,677
Department signs an order saying that

2179
02:02:45,760 --> 02:02:50,317
the models are designated export
controlled and that no foreigner,

2180
02:02:50,400 --> 02:02:54,317
including the foreign citizens who
worked and actually built the models

2181
02:02:54,400 --> 02:02:58,397
themselves, can can touch them.
uh whether or not the administration

2182
02:02:58,480 --> 02:03:02,237
actually has that legal power
is disputed, but Anthropic decides to

2183
02:03:02,320 --> 02:03:05,277
actually follow through instead
of immediately getting a temporary

2184
02:03:05,360 --> 02:03:09,677
restraining order. And they because they
have no ability to make sure that no uh

2185
02:03:09,760 --> 02:03:11,757
foreign uh citizen is is seeing the

2186
02:03:11,840 --> 02:03:14,157
models actually pulls them down. Uh both

2187
02:03:14,240 --> 02:03:19,117
Mythos uh which was privately accessible
as well as Fable uh which was publicly

2188
02:03:19,200 --> 02:03:22,077
available. >> Now you've spent
a career working in security.

2189
02:03:22,160 --> 02:03:25,597
>> Yes. Um Ron John and I for months now

2190
02:03:25,680 --> 02:03:28,477
have had this debate about whether all

2191
02:03:28,560 --> 02:03:31,837
of this talk of doom and super powerful

2192
02:03:31,920 --> 02:03:34,477
models from anthropic like for instance

2193
02:03:34,560 --> 02:03:37,437
mythos a model so powerful that you

2194
02:03:37,520 --> 02:03:39,517
can't access it. Um whether that

2195
02:03:39,600 --> 02:03:42,797
represented a real capability increase

2196
02:03:42,880 --> 02:03:45,517
or whether that was just marketing and

2197
02:03:45,600 --> 02:03:48,637
it's a step change uh not not really a

2198
02:03:48,720 --> 02:03:52,557
step change in capabilities. So I
actually want to toss it to Rajan who is

2199
02:03:52,640 --> 02:03:56,317
firmly on the marketing side of this
thing just to make the argument of why

2200
02:03:56,400 --> 02:04:00,397
you think this is marketing and Alex
can address that. Yeah, I think trying to

2201
02:04:00,480 --> 02:04:03,117
understand like what is that capabil new

2202
02:04:03,200 --> 02:04:05,197
capability that is so dangerous? What is

2203
02:04:05,280 --> 02:04:07,597
that step change? I think the reason I

2204
02:04:07,680 --> 02:04:10,557
get so suspect on it is how perfectly

2205
02:04:10,640 --> 02:04:15,197
coordinate coordinated a lot of the
marketing and communications are around

2206
02:04:15,280 --> 02:04:18,877
how they've rolled these out.
Like again, mythos is the most dangerous

2207
02:04:18,960 --> 02:04:22,797
thing. There's someone eating a sandwich
in a park that actually got like where

2208
02:04:22,880 --> 02:04:25,837
the model broke out of containment was

2209
02:04:25,920 --> 02:04:30,397
like a PR release. It was it was a very
coordinated thing. So what is that

2210
02:04:30,480 --> 02:04:34,957
danger that they're trying to kind
of bring to the world? Like what is it

2211
02:04:35,040 --> 02:04:37,837
really? What does it look like, feel like,
or what is it actually going to be

2212
02:04:37,920 --> 02:04:41,517
a problem? >> Uh so how much money
do you guys have writing on this?

2213
02:04:41,600 --> 02:04:43,437
>> Well, how would we even play this?
Like I don't even know.

2214
02:04:43,520 --> 02:04:46,237
>> Oh, like as a bet? >> As a bet?
Like is there a poly market?

2215
02:04:46,320 --> 02:04:50,557
Uh yeah, I'm trying to see what
my angle is here. Apparently I have

2216
02:04:50,640 --> 02:04:54,797
>> Well, well, no, you are the person that
knows Trump action. you know the answers

2217
02:04:54,880 --> 02:04:58,877
here and we have just been kind of like
debating >> just like we just argue on.

2218
02:04:58,960 --> 02:05:00,637
>> So okay so it's
actually it's actually a

2219
02:05:00,720 --> 02:05:02,717
little bit of a complicated answer. So

2220
02:05:02,800 --> 02:05:07,757
um mythos is almost it is the best

2221
02:05:07,840 --> 02:05:11,117
bugfinding model that we know about for

2222
02:05:11,200 --> 02:05:14,557
sure. It is most likely the best bug

2223
02:05:14,640 --> 02:05:19,197
finding model in the world. And I say
most likely because we do not know what

2224
02:05:19,280 --> 02:05:21,037
the Chinese secretly have in their labs.

2225
02:05:21,120 --> 02:05:23,597
But for all the stuff for which we have

2226
02:05:23,680 --> 02:05:27,677
open known security evaluations for it

2227
02:05:27,760 --> 02:05:32,077
scores the best. Now, is it the avenging

2228
02:05:32,160 --> 02:05:38,477
cyber god that anthropic makes it out to
be? I personally don't think so. Yes, it

2229
02:05:38,560 --> 02:05:40,877
is. >> You're not necessar
you're not right yet,

2230
02:05:40,960 --> 02:05:42,957
>> but but it is it is really good. It is

2231
02:05:43,040 --> 02:05:45,677
really good at finding bugs. But from my

2232
02:05:45,760 --> 02:05:51,197
perspective, the like Opus 48, GPT 5.5,
a bunch of other models are here in

2233
02:05:51,280 --> 02:05:53,437
Mythosis here >> at bug finding

2234
02:05:53,520 --> 02:05:56,317
>> at bug finding and exploit development

2235
02:05:56,400 --> 02:06:00,797
and it is not here, which is kind
of where you would think from reading all

2236
02:06:00,880 --> 02:06:04,877
this stuff. The other thing is that
everybody pretends that Mythos was the

2237
02:06:04,960 --> 02:06:07,437
breakthrough, but that is not true. For
people in the security world who've been

2238
02:06:07,520 --> 02:06:09,277
paying attention, this Rubicon was

2239
02:06:09,360 --> 02:06:11,757
crossed well last year

2240
02:06:11,840 --> 02:06:14,637
>> with the Opus 4 series with the GPD5

2241
02:06:14,720 --> 02:06:17,197
series. That is when the models became

2242
02:06:17,280 --> 02:06:22,557
better or as good or or better than
the best human bugfinders and not just as

2243
02:06:22,640 --> 02:06:25,597
good as the best bug finders.
And there's like 50 or 100 of these people

2244
02:06:25,680 --> 02:06:28,477
and I know a couple dozen of them.
The problem is those people are very

2245
02:06:28,560 --> 02:06:31,357
expensive, right? And they don't scale.

2246
02:06:31,440 --> 02:06:35,037
You can't just throw water on them
and make them multiply like gremlins. That

2247
02:06:35,120 --> 02:06:39,837
would be nice. I mean, certainly the NSA
would love that, but models can you can

2248
02:06:39,920 --> 02:06:41,677
just scale them as much as you want with

2249
02:06:41,760 --> 02:06:45,517
money and power and and and GPUs. And so

2250
02:06:45,600 --> 02:06:48,717
that last year was a huge deal because

2251
02:06:48,800 --> 02:06:51,677
all of a sudden you saw these amateur

2252
02:06:51,760 --> 02:06:53,997
bugfinders. Imagine you went to a high

2253
02:06:54,080 --> 02:06:58,797
school track meet and every single kid
is posting Olympic world record times.

2254
02:06:58,880 --> 02:07:02,557
You'd be like, "Man, they're juicing,
right? That something had happened."

2255
02:07:02,640 --> 02:07:04,317
That is what last year was like. like

2256
02:07:04,400 --> 02:07:07,517
mid la m mid to late last year in the

2257
02:07:07,600 --> 02:07:12,077
vulnerability discovery world is the
kids started juicing right and it's

2258
02:07:12,160 --> 02:07:15,757
because the models hit this point where
they could find everything and so yes

2259
02:07:15,840 --> 02:07:18,797
mythos is really good at finding bugs
it's really good at writing exploits

2260
02:07:18,880 --> 02:07:21,437
fable is not and so that is that is what

2261
02:07:21,520 --> 02:07:24,637
the whole point of fable is it is the

2262
02:07:24,720 --> 02:07:28,957
same model weights as mythos
but anthropic put a protection in place to

2263
02:07:29,040 --> 02:07:31,437
keep it from doing the really nasty

2264
02:07:31,520 --> 02:07:35,117
stuff that mythos will do And anth uh

2265
02:07:35,200 --> 02:07:37,277
Amazon has some complaints about that.

2266
02:07:37,360 --> 02:07:41,437
But what happened
is the complaints Amazon has

2267
02:07:41,520 --> 02:07:45,837
uh a number of us have looked at them
and then Enthropic has pushed back even

2268
02:07:45,920 --> 02:07:48,237
with the jailbreaks that Amazon has. You

2269
02:07:48,320 --> 02:07:50,317
cannot get Fable to do things that you

2270
02:07:50,400 --> 02:07:53,357
can't do with the Opus series with GPD5

2271
02:07:53,440 --> 02:07:55,517
and even with a bunch of Chinese models.

2272
02:07:55,600 --> 02:07:58,957
So that is why the the actions by the
Trump administration make absolutely no

2273
02:07:59,040 --> 02:08:01,757
sense because those models will not

2274
02:08:01,840 --> 02:08:06,077
refuse at all. You can just ask GPD5
to go find these bugs and it will go do it

2275
02:08:06,160 --> 02:08:09,437
for you. Um, you can just ask Kimmy,
which is an open source Chinese model.

2276
02:08:09,520 --> 02:08:14,237
It will go write you these exploits. And
so, yes, you can trick Fable into doing

2277
02:08:14,320 --> 02:08:18,317
certain things, but you can't trick it
into doing the really powerful Mythos

2278
02:08:18,400 --> 02:08:22,237
stuff, which is like grind on the Linux
kernel for all day and find a bunch of

2279
02:08:22,320 --> 02:08:25,357
bugs, which will cost you hundreds
of thousands of dollars, it turns out, if

2280
02:08:25,440 --> 02:08:29,757
you pay the full price. Um, and those
have not been jailbroken. And so there's

2281
02:08:29,840 --> 02:08:31,437
no real good reason for what happened.

2282
02:08:31,520 --> 02:08:35,277
So yes, Mythos is really powerful. It
has only been accessible to a very small

2283
02:08:35,360 --> 02:08:37,997
number of companies. It is not magically

2284
02:08:38,080 --> 02:08:39,997
powerful. The other thing you have to

2285
02:08:40,080 --> 02:08:43,357
remember here is like bugs are cheap.

2286
02:08:43,440 --> 02:08:48,317
Exploits are like we're now drowning in
bugs because these models are so good at

2287
02:08:48,400 --> 02:08:51,277
finding and we have so much bad
software. We've been writing bad

2288
02:08:51,360 --> 02:08:53,357
software forever. And it turns out that

2289
02:08:53,440 --> 02:08:56,237
like writing human beings writing code

2290
02:08:56,320 --> 02:08:59,197
in C and C++ especially was a really bad

2291
02:08:59,280 --> 02:09:00,877
idea that like we should not have been

2292
02:09:00,960 --> 02:09:04,077
writing like building our entire lives

2293
02:09:04,160 --> 02:09:09,037
like based upon memory unsafe languages
and then in the presence of these

2294
02:09:09,120 --> 02:09:11,037
superhuman bugfinders not just mythos

2295
02:09:11,120 --> 02:09:12,877
but all of them that was not a good

2296
02:09:12,960 --> 02:09:16,717
idea. And so, uh, if you're a pedestrian

2297
02:09:16,800 --> 02:09:19,677
and you get hit by a F-150 at 80 miles

2298
02:09:19,760 --> 02:09:22,397
per hour or a McLaren at 200 miles per

2299
02:09:22,480 --> 02:09:24,797
hour, you don't really care. You're

2300
02:09:24,880 --> 02:09:27,677
dead, right? And that's the difference

2301
02:09:27,760 --> 02:09:30,797
here is like the mythos is the F1.

2302
02:09:30,880 --> 02:09:32,157
Mythos is the F1.

2303
02:09:32,240 --> 02:09:35,357
>> Yeah, exactly.
And Kimmy or GLM52 is the

2304
02:09:35,440 --> 02:09:39,437
F-150. And so you could take the the the

2305
02:09:39,520 --> 02:09:43,997
McLaren off the market, but all you've
done is actually hurt the defenders who

2306
02:09:44,080 --> 02:09:48,157
want to have full coverage, right? Who
want to be able to find all the bugs. An

2307
02:09:48,240 --> 02:09:52,157
attacker only needs one or two bugs to
string together to actually pull off the

2308
02:09:52,240 --> 02:09:57,117
attack. And so that's why, you know,
so about 150 of us wrote this open letter

2309
02:09:57,200 --> 02:10:00,317
and signed it saying this is really
stupid because all you're doing is

2310
02:10:00,400 --> 02:10:02,957
hurting defenders and you're also really

2311
02:10:03,040 --> 02:10:05,437
hurting the US AI industry. Okay, I

2312
02:10:05,520 --> 02:10:07,677
think Ron John wins the debate.

2313
02:10:07,760 --> 02:10:11,517
>> Well, no, still it's
not all marketing, but it's

2314
02:10:11,600 --> 02:10:15,677
>> it's not all marketing,
but like I I just want to say like

2315
02:10:15,760 --> 02:10:19,517
mythos is really powerful, but doesn't
really matter because there's so many

2316
02:10:19,600 --> 02:10:23,517
bugs out there that you can use any
of these models to find them. I what I'd

2317
02:10:23,600 --> 02:10:26,237
like to do is I'd like to shift
the debate one away from finding bugs

2318
02:10:26,320 --> 02:10:30,237
because the other problem here is like
we cannot set the standard that US AI

2319
02:10:30,320 --> 02:10:34,397
models can't find bugs. That is a
terrible terrible terrible standard. If

2320
02:10:34,480 --> 02:10:38,637
you have a piece if you are writing
software with an LLM, it has to be able

2321
02:10:38,720 --> 02:10:41,357
to understand what a bug looks like. So

2322
02:10:41,440 --> 02:10:44,397
it does not write those bugs. It has to.

2323
02:10:44,480 --> 02:10:48,637
And that is that's the trick actually
Amazon used was they tricked it into

2324
02:10:48,720 --> 02:10:51,037
basically finding bugs in individual

2325
02:10:51,120 --> 02:10:53,117
lines of code because it has to be able

2326
02:10:53,200 --> 02:10:56,477
to do that to write secure code. If if

2327
02:10:56,560 --> 02:11:00,477
our if American LMS know nothing about
security, they will create more security

2328
02:11:00,560 --> 02:11:02,957
flaws. So we cannot create a standard

2329
02:11:03,040 --> 02:11:07,837
where American LMS are dumb about
security. That would be a humongous own

2330
02:11:07,920 --> 02:11:09,597
goal and that is one of the possible

2331
02:11:09,680 --> 02:11:12,877
outcomes here of of this whole brewhaha.

2332
02:11:12,960 --> 02:11:15,917
That would be really stupid. And so uh

2333
02:11:16,000 --> 02:11:20,557
you know my I think yes I I don't think
either one of you wins. I mean as the

2334
02:11:20,640 --> 02:11:23,917
referee here I think Ron John has
a little bit of a you have a point but

2335
02:11:24,000 --> 02:11:26,557
Mythos is really good. It just

2336
02:11:26,640 --> 02:11:29,757
>> it doesn't Anthropic has pleaded up too

2337
02:11:29,840 --> 02:11:32,477
much. Um, OpenAI is rumored to be

2338
02:11:32,560 --> 02:11:33,917
releasing something that's mythos

2339
02:11:34,000 --> 02:11:37,357
quality really soon. Um, and I doubt

2340
02:11:37,440 --> 02:11:41,517
that they're gonna like follow they'll
just give it a number like uh I think

2341
02:11:41,600 --> 02:11:44,797
they've learned from anthropic you don't
call your thing like the cyber gorgon.

2342
02:11:44,880 --> 02:11:47,917
Um, >> uh, >> that actually would sell.

2343
02:11:48,000 --> 02:11:52,557
>> Yeah, yeah, it sells a little too well,
right? Um, right. Maybe as a Greek with

2344
02:11:52,640 --> 02:11:55,597
the Odyssey coming out, I'm getting
a little sensitive about the the marketing

2345
02:11:55,680 --> 02:11:57,597
here, but like like you know, let's

2346
02:11:57,680 --> 02:11:59,837
let's like back off on the naming scheme

2347
02:11:59,920 --> 02:12:05,197
and move back to numbers. Uh uh but like
I I think a a big part of this is we

2348
02:12:05,280 --> 02:12:08,317
just have to like reset the standard
of like not bug finding but the actual

2349
02:12:08,400 --> 02:12:11,517
creating and creation of exploits and then
other kinds of offensive operations

2350
02:12:11,600 --> 02:12:13,917
of which there's a bunch of other
things you can do offensively with these

2351
02:12:14,000 --> 02:12:17,997
models. Those are the things that we
should call off limits not bug finding

2352
02:12:18,080 --> 02:12:21,357
because that is something that we have
to do much more quickly than we're doing

2353
02:12:21,440 --> 02:12:23,517
right now. C >> can I I just
want to try to articulate

2354
02:12:23,600 --> 02:12:26,477
my point which is you know this is all

2355
02:12:26,560 --> 02:12:30,797
right. So when we talk about is mythos
material or is mythos marketing um my

2356
02:12:30,880 --> 02:12:34,397
argument would be like wouldn't if
something was material it would look a

2357
02:12:34,480 --> 02:12:38,157
lot like what we're seeing today right
like at a certain point these models are

2358
02:12:38,240 --> 02:12:42,397
increasing in capabilities you're going
to want to put some safeguards on them

2359
02:12:42,480 --> 02:12:46,157
and so is what anthropic is doing that
different from like where you'd want to

2360
02:12:46,240 --> 02:12:49,357
be if you actually had these concerns

2361
02:12:49,440 --> 02:12:51,677
>> I mean I think both
anthropic and openai

2362
02:12:51,760 --> 02:12:55,757
have a path in which they have like
a KYC model now >> know your customer

2363
02:12:55,840 --> 02:12:59,437
>> a know your customer. Yeah. So, OpenAI
actually has a couple levels. They have

2364
02:12:59,520 --> 02:13:03,117
a public one where you can basically say,
I want access to cyber capabilities

2365
02:13:03,200 --> 02:13:06,557
and then they have a private one.
I believe Anthropic only has the private

2366
02:13:06,640 --> 02:13:09,437
one, right, to get access to mythos. Uh

2367
02:13:09,520 --> 02:13:13,997
but uh I I do think it's reasonable
to have some level of gating, but in the

2368
02:13:14,080 --> 02:13:19,197
case of GlassW, I think it was honestly
it I thought it was not widespread

2369
02:13:19,280 --> 02:13:21,837
enough. I know of like critical
infrastructure companies that don't have

2370
02:13:21,920 --> 02:13:25,837
access to mythos right now.
Um >> but nobody does right now. Well, no,

2371
02:13:25,920 --> 02:13:29,517
right now before last week, like people
who should have had access did not have

2372
02:13:29,600 --> 02:13:31,837
access. Now, seeing what happened here,

2373
02:13:31,920 --> 02:13:34,397
you can see why they were so careful.

2374
02:13:34,480 --> 02:13:37,997
Um, but from my perspective as

2375
02:13:38,080 --> 02:13:40,397
defenders, we are in a race against

2376
02:13:40,480 --> 02:13:45,357
time, right? We have all of these bugs
that need to be fixed. And we also have

2377
02:13:45,440 --> 02:13:49,917
this weird thing where we're like, this
whole conversation is predicated on the

2378
02:13:50,000 --> 02:13:51,437
idea that like the American labs are

2379
02:13:51,520 --> 02:13:54,557
years ahead of our adversaries. And that

2380
02:13:54,640 --> 02:13:57,357
is just not true, right? So while Fable

2381
02:13:57,440 --> 02:14:00,637
was shut down on Tuesday of this week,

2382
02:14:00,720 --> 02:14:03,197
GLM 5.2 was shipped right from Z.AI,

2383
02:14:03,280 --> 02:14:07,837
which is a Chinese lab. This is an
openweight model. It is MIT licensed. So

2384
02:14:07,920 --> 02:14:11,757
any of you can go not just download
the weights from HuggingFace, but you can

2385
02:14:11,840 --> 02:14:13,357
include it in your own product and you

2386
02:14:13,440 --> 02:14:15,917
can retrain it. It is within percentage

2387
02:14:16,000 --> 02:14:20,077
points of the top closed models from

2388
02:14:20,160 --> 02:14:22,477
anthropic and open AI in a bunch of

2389
02:14:22,560 --> 02:14:27,277
things. um a bunch of of evals. We don't
know how good it is at bugf finding yet.

2390
02:14:27,360 --> 02:14:30,317
So there hasn't been any good testing
here. But in coding and a bunch of other

2391
02:14:30,400 --> 02:14:34,317
intelligence tasks, it's almost as good
as the best the United States has. And

2392
02:14:34,400 --> 02:14:39,277
that's the best Chinese model we know
about, right? Unlike American labs, the

2393
02:14:39,360 --> 02:14:43,357
Chinese are not going to announce their
mythos, right? And so the idea that we

2394
02:14:43,440 --> 02:14:47,437
should be doing everything in the United
States to just be playing defense is

2395
02:14:47,520 --> 02:14:51,197
we're playing like a a defensive
model here, that is stupid from my

2396
02:14:51,280 --> 02:14:55,517
perspective. We need to be accelerating
our capability to do these things and

2397
02:14:55,600 --> 02:14:59,117
not just playing defense. >> Well,
now I am sufficiently scared even

2398
02:14:59,200 --> 02:15:02,477
though I've thought that the mythos
was pure marketing. But but what do we do

2399
02:15:02,560 --> 02:15:06,797
about it? Like how do we what
is that like and I I think I share that

2400
02:15:06,880 --> 02:15:10,637
glasswing again why it felt like
marketing to me. It was this kind of

2401
02:15:10,720 --> 02:15:14,397
exclusive club that people were invited
to and I actually spoke with people who

2402
02:15:14,480 --> 02:15:18,237
are using it and they spoke about it
almost like it was a club like oh yeah

2403
02:15:18,320 --> 02:15:22,557
we got access to mythos. We're spending
a ton of money. Like I think what should

2404
02:15:22,640 --> 02:15:26,637
it look like this kind of roll out
to actually >> when you get help us

2405
02:15:26,720 --> 02:15:30,317
>> in Glasswind I mean it they're fixing
real bugs they have found like 10,000

2406
02:15:30,400 --> 02:15:34,317
bugs now one of the problems with
glasswing is they found like 10,000 bugs

2407
02:15:34,400 --> 02:15:37,517
but they've only fixed a thousand right
so like you do not want a 10 to1 ratio

2408
02:15:37,600 --> 02:15:41,357
of the bugs you found to fixed um
we just joined a thing called project

2409
02:15:41,440 --> 02:15:45,517
Athena which chain guard is running
which is all about fixing specifically

2410
02:15:45,600 --> 02:15:50,157
open source bugs right so glasswing is a
lot of that is closed source with inside

2411
02:15:50,240 --> 02:15:54,797
commercial companies. We're part of this
coalition that is using uh Mythos to fix

2412
02:15:54,880 --> 02:15:57,757
open source bugs, which is a huge
challenge because like just finding the

2413
02:15:57,840 --> 02:16:01,677
people who have access to that
that source code uh commit access is

2414
02:16:01,760 --> 02:16:04,637
difficult. Uh so that's going to be like

2415
02:16:04,720 --> 02:16:06,717
a huge push. So it it's not like just

2416
02:16:06,800 --> 02:16:10,157
totally BS. Um I do think that one of

2417
02:16:10,240 --> 02:16:14,717
the things that's going on here is that if
these companies ran any modern AI

2418
02:16:14,800 --> 02:16:17,597
security tool against their codebase,
they'd find 80% of those bugs. They

2419
02:16:17,680 --> 02:16:19,197
don't have to use Mythos, right? Okay.

2420
02:16:19,280 --> 02:16:23,277
So, um, it's just that they've never
looked, right? It's they're like

2421
02:16:23,360 --> 02:16:26,477
>> that's concerning. >> Yeah.
Right. And and so they've they're

2422
02:16:26,560 --> 02:16:29,997
like, "Oh, yeah. I've got this old
crappy code." And they look for the

2423
02:16:30,080 --> 02:16:31,357
first time and they all
of a sudden they find

2424
02:16:31,440 --> 02:16:35,117
>> and they could have done it a year ago
or 6 months ago and found some number or

2425
02:16:35,200 --> 02:16:37,597
some percentage. >> I had
this conversation with a CEO this

2426
02:16:37,680 --> 02:16:41,677
morning who's like, "How do I get into
Mythos, Alex, who can call?" And I said,

2427
02:16:41,760 --> 02:16:45,597
"Don't wait. >> Here you go.
Here's an API key for Opus

2428
02:16:45,680 --> 02:16:49,997
right now. it works today because do
not wait for the Trump administration to

2429
02:16:50,080 --> 02:16:53,197
figure out what they're doing with Mythos.
Use Opus 48 right now and I

2430
02:16:53,280 --> 02:16:55,517
guarantee you'll get 70%
of the bugs you care about.

2431
02:16:55,600 --> 02:16:58,877
>> Alex, you're you're a security
researcher. One of the stipulations

2432
02:16:58,960 --> 02:17:02,317
about getting Fable back. So, by the way,
Anthropic has said they expect uh

2433
02:17:02,400 --> 02:17:06,077
Fable to come back on soon. Um, but one
of the stipulations that's been talked

2434
02:17:06,160 --> 02:17:09,917
about to do that is that they have
to prevent it from being able to be

2435
02:17:10,000 --> 02:17:12,717
jailbroken. >> Is that possible?

2436
02:17:12,800 --> 02:17:16,397
>> Yeah. I saw I saw you did
jailbreak in air quotes. Yeah. No.

2437
02:17:16,480 --> 02:17:19,277
>> So we have we have two minutes left.
So talk a little bit about why that's not

2438
02:17:19,360 --> 02:17:21,197
possible. >> No, there's
actually a paper from NIST.

2439
02:17:21,280 --> 02:17:25,037
It's like a kind of a goal completeness
theorem kind of uh paper that basically

2440
02:17:25,120 --> 02:17:28,397
says it's impossible to make a model
that is completely not jailbreakable.

2441
02:17:28,480 --> 02:17:32,317
Like I said, Fable needs to be able to
understand what security flaws are to be

2442
02:17:32,400 --> 02:17:35,357
able to do its job. I think what what

2443
02:17:35,440 --> 02:17:39,757
Enthropic might what what Enthropic
might the game might they might be

2444
02:17:39,840 --> 02:17:42,237
playing here is redefine what jailbreak

2445
02:17:42,320 --> 02:17:43,917
is back to what their system card says.

2446
02:17:44,000 --> 02:17:46,877
So their system card says we define

2447
02:17:46,960 --> 02:17:51,357
different levels of cyber capability
and finding a couple of bugs and knowing

2448
02:17:51,440 --> 02:17:55,357
what a bug is is okay, but we won't let
you like build exploit chains and and do

2449
02:17:55,440 --> 02:17:59,677
lots of long-term stuff. And so if they
can define back to their system card

2450
02:17:59,760 --> 02:18:02,317
what a jailbreak is, then they're fine,

2451
02:18:02,400 --> 02:18:07,437
right? Um, what would be terrible
for the entire American AI and cyber

2452
02:18:07,520 --> 02:18:10,957
security industries is if the Trump
administration defines a jailbreak as

2453
02:18:11,040 --> 02:18:15,197
having any ability to find any
vulnerability in code because then all

2454
02:18:15,280 --> 02:18:17,837
of us have to go to Chinese models.

2455
02:18:17,920 --> 02:18:22,557
>> That's that's it. Yes. Like that would
be a terrible terrible outcome. And that

2456
02:18:22,640 --> 02:18:24,957
might happen because unfortunately one
the news that broke just in the last

2457
02:18:25,040 --> 02:18:28,397
couple hours is Politico is reporting
that Anthropic and the Trump

2458
02:18:28,480 --> 02:18:32,957
administration are negotiating what the AI
safety standards should be. So like I

2459
02:18:33,040 --> 02:18:36,397
don't know it's like Howard Lutnik
writing an eval right now. Like I find

2460
02:18:36,480 --> 02:18:41,837
that a little terrifying. So um I I am
hoping that Enthropic obviously there's

2461
02:18:41,920 --> 02:18:44,477
very smart people in Enthropic working
on this. I am hoping that they are able

2462
02:18:44,560 --> 02:18:48,157
to come up with something reasonable
that moves stuff towards exploitation

2463
02:18:48,240 --> 02:18:52,637
not bug finding like we have to have
the ability as defenders to find and fix

2464
02:18:52,720 --> 02:18:55,517
flaws. The other thing I just want
because we don't have a lot of time. The

2465
02:18:55,600 --> 02:18:57,837
other real own goal here was that this

2466
02:18:57,920 --> 02:19:00,957
injected a type of political instability

2467
02:19:01,040 --> 02:19:05,437
and political risk into the USI industry
that did not exist. There are a bunch of

2468
02:19:05,520 --> 02:19:09,917
companies that are now going to be have
to use openweight models because at any

2469
02:19:10,000 --> 02:19:12,797
time on a Friday afternoon, it's it's
good thing that Fable was only out for a

2470
02:19:12,880 --> 02:19:16,317
week because people weren't didn't have
time to work into their critical path.

2471
02:19:16,400 --> 02:19:20,637
But if if if Fable had been out for a
month and at 5:00 PM on a Friday all of

2472
02:19:20,720 --> 02:19:24,157
a sudden it got yanked, then pagers
across the country would have gone off

2473
02:19:24,240 --> 02:19:27,837
because every everybody's systems would
have fallen over. And so now this week

2474
02:19:27,920 --> 02:19:33,277
CIOS and CTO's are signing contracts
to have openweight models on different

2475
02:19:33,360 --> 02:19:36,237
hosts on US hosts. They're not using
Chinese hosts, but they're using Chinese

2476
02:19:36,320 --> 02:19:39,437
models on as backups through LM routers

2477
02:19:39,520 --> 02:19:42,237
because political risk is now a risk of

2478
02:19:42,320 --> 02:19:46,637
using AI companies. That is a massive
own goal. The United States of America

2479
02:19:46,720 --> 02:19:51,117
does not need to do that. We should not
do that again. That is incredibly stupid

2480
02:19:51,200 --> 02:19:52,957
at a moment of maximum pressure from the

2481
02:19:53,040 --> 02:19:55,437
People's Republic of China. Um, and so I

2482
02:19:55,520 --> 02:19:58,237
can't stress enough like we we lost a

2483
02:19:58,320 --> 02:20:00,557
war this weekend. Like, let's not lose

2484
02:20:00,640 --> 02:20:06,157
another war by making it unreliable
to utilize the United States for AI.

2485
02:20:06,240 --> 02:20:08,397
>> Alex, will you come on the show
for a longer conversation about this?

2486
02:20:08,480 --> 02:20:09,437
>> Absolutely. I'd love to. Yeah.

2487
02:20:09,520 --> 02:20:10,877
>> Alex Tamos and Ron General.

2488
02:20:10,960 --> 02:20:14,680
>> Thank you. >> Thank you.

2489
02:20:17,680 --> 02:20:21,117
Okay, so Anthropic is not just a massive

2490
02:20:21,200 --> 02:20:24,797
model builder, it's a massive product
builder as well with products like Cloud

2491
02:20:24,880 --> 02:20:27,277
Code and Co-work that have taken off

2492
02:20:27,360 --> 02:20:30,557
like crazy over the past few months. And

2493
02:20:30,640 --> 02:20:32,797
Cloud Code came out of a place that few

2494
02:20:32,880 --> 02:20:35,437
of us know much about called Anthropic

2495
02:20:35,520 --> 02:20:38,957
Labs. And Anthropic Labs is a

2496
02:20:39,040 --> 02:20:41,037
organization within Anthropic that is

2497
02:20:41,120 --> 02:20:43,357
working on building the next level of

2498
02:20:43,440 --> 02:20:46,317
frontier products with AI at the center.

2499
02:20:46,400 --> 02:20:48,717
And so today we are lucky to hear from

2500
02:20:48,800 --> 02:20:51,517
the person running that lab, Mike Kger,

2501
02:20:51,600 --> 02:20:53,757
who is the co-founder of Instagram and

2502
02:20:53,840 --> 02:20:57,517
now the lead of um of Anthropic Labs at

2503
02:20:57,600 --> 02:21:01,757
Anthropic. We're going to welcome him
on stage along with Lauren Good of Wired,

2504
02:21:01,840 --> 02:21:05,037
who will join me as a co-in. Mike
and Lauren, let's uh let's let's hear from

2505
02:21:05,120 --> 02:21:08,520
both of you guys.

2506
02:21:10,000 --> 02:21:14,797
You're here. >> All right.
>> All right, Mike. So, chill times in

2507
02:21:14,880 --> 02:21:17,117
anthropic land. >> Nothing going on.

2508
02:21:17,200 --> 02:21:20,237
>> Slow week. >> Slow week.

2509
02:21:20,320 --> 02:21:24,237
>> You want to start, Lauren? >> Yeah.
Uh first of all I mean we want to

2510
02:21:24,320 --> 02:21:28,477
talk about what you're working on at
labs uh and explain your role to folks

2511
02:21:28,560 --> 02:21:32,557
but I want to ask you first how close
are you right now to the situation with

2512
02:21:32,640 --> 02:21:35,117
the white house >> less
than in my CPO role. So I

2513
02:21:35,200 --> 02:21:39,677
transitioned about like five months ago
into this labs role I think in the CPO

2514
02:21:39,760 --> 02:21:41,197
role I think would have been deep in it.

2515
02:21:41,280 --> 02:21:46,397
Now I'm more you know obviously you know
we want to restore it and and as a like

2516
02:21:46,480 --> 02:21:49,837
uh product person I want to make sure
that that gets access but less close to

2517
02:21:49,920 --> 02:21:53,997
it than uh you know in that sort
of seale role that I had before.

2518
02:21:54,080 --> 02:21:56,877
>> Okay, Alex, do you have a follow-up?
I have like eight follow-ups.

2519
02:21:56,960 --> 02:22:00,797
>> Definitely. Well, we there was a guy
named Ben on X who said, "I will mail

2520
02:22:00,880 --> 02:22:03,997
anthropic and original copy of my long
form birth certificate if they will

2521
02:22:04,080 --> 02:22:08,397
enable fable for me again." I sound like
those lunatics who were obsessed with 40

2522
02:22:08,480 --> 02:22:12,477
now. Will you take Ben's long form?
I don't know uh that will take as long

2523
02:22:12,560 --> 02:22:16,477
form, but it has been interesting.
I mean, Fable was only available um for a

2524
02:22:16,560 --> 02:22:20,637
few days, but I've definitely uh every
time I've tweeted since then, uh they've

2525
02:22:20,720 --> 02:22:23,437
not read whatever I was tweeting
and they've mostly been like bring back

2526
02:22:23,520 --> 02:22:26,717
Fable, which like in Instagram, we got
rid of Gotham. Do you remember Gotham?

2527
02:22:26,800 --> 02:22:29,517
The filter. This is like and then for
the rest like the next eight years, all

2528
02:22:29,600 --> 02:22:34,157
I heard was bring back Gotham. So,
it struck a nerf, but Fable will uh will

2529
02:22:34,240 --> 02:22:37,837
come back before Gotham did. But yeah,
it's it's it clearly the folks that had

2530
02:22:37,920 --> 02:22:40,237
gotten to use it and started
incorporating it. It's actually really

2531
02:22:40,320 --> 02:22:45,277
interesting. Um, I've learned to not
really trust day of or even week of

2532
02:22:45,360 --> 02:22:48,477
model reactions. Like you don't really
know until you've put it through its

2533
02:22:48,560 --> 02:22:52,397
paces. And so like I almost just
completely block out the noise in the

2534
02:22:52,480 --> 02:22:55,517
first couple of days of any new model
release cuz I don't know, everybody has

2535
02:22:55,600 --> 02:22:59,037
maybe their like toy like example thing
that they like to do with a new model.

2536
02:22:59,120 --> 02:23:02,397
But it's hard to actually put it through
its paces until you've actually had real

2537
02:23:02,480 --> 02:23:05,757
work done with it. And I think people
were just starting to do that and then,

2538
02:23:05,840 --> 02:23:08,637
you know, we had to uh uh sort of pull

2539
02:23:08,720 --> 02:23:14,157
back Fable. But um I remember
in December when we put out Opus 46, uh it

2540
02:23:14,240 --> 02:23:16,877
was like this interesting time where
everybody went home for the holidays and

2541
02:23:16,960 --> 02:23:20,077
a lot of people had that week off
between Christmas and New Year's and

2542
02:23:20,160 --> 02:23:23,837
then they came back and were like, "Oh,
I spent a lot of time and I really get

2543
02:23:23,920 --> 02:23:25,517
why Opus is good and I want to do it."

2544
02:23:25,600 --> 02:23:28,877
So I don't think Fable has had
that opportunity yet. But despite that

2545
02:23:28,960 --> 02:23:32,317
though, I mean, this was a pretty big
reaction from the Trump administration.

2546
02:23:32,400 --> 02:23:35,917
And I think everyone here, especially
if you were listening to the Alex Stamosh

2547
02:23:36,000 --> 02:23:39,997
session, uh, understands what's going on.
But this happened within a few days

2548
02:23:40,080 --> 02:23:43,917
of the model release. Um, if I can give
Wired some credit, Wired just reported

2549
02:23:44,000 --> 02:23:47,917
last night that it was due
to Anthropic's uh, relationship with or

2550
02:23:48,000 --> 02:23:51,837
having given uh, access to the model
to SK Telecom that could have raised flags

2551
02:23:51,920 --> 02:23:57,437
within the administration. How surprised
were you by how immediate that that

2552
02:23:57,520 --> 02:23:59,997
backlash was? >> Yeah,
I think the the the sort of

2553
02:24:00,080 --> 02:24:04,317
reaction decision was surprising and we
were sort of immediately engaged with

2554
02:24:04,400 --> 02:24:09,597
them to, you know, restore access as well.
Um, and so at the same time, you

2555
02:24:09,680 --> 02:24:13,037
know, the one thing that we we think
a lot about internally is, you know, there

2556
02:24:13,120 --> 02:24:16,797
used to be a poster on the Facebook
wall when we were that was like every day

2557
02:24:16,880 --> 02:24:20,157
feels like a week. And I think that's
becoming true in AI. And I think a good

2558
02:24:20,240 --> 02:24:24,557
thing to remind ourselves of in general
in the industry is uh we're dealing with

2559
02:24:24,640 --> 02:24:28,717
unprecedented times. We're dealing with
new situations and uh they can develop

2560
02:24:28,800 --> 02:24:32,877
really quickly as well. And so I think
uh also developing the capabilities and

2561
02:24:32,960 --> 02:24:35,757
the connections and and to make sure
those conversations can happen quickly

2562
02:24:35,840 --> 02:24:38,797
is really really important.
>> I have a new motto suggestion for you

2563
02:24:38,880 --> 02:24:42,637
for the wall. Move fast.
Move fast and jailbreak things.

2564
02:24:42,720 --> 02:24:45,357
>> Jailbreak things. I don't think
they're going to use that, Lauren.

2565
02:24:45,440 --> 02:24:48,957
>> Okay. Um, you know,
we just heard from Alex

2566
02:24:49,040 --> 02:24:53,277
that some of these capabilities have
been uh available on, you know, previous

2567
02:24:53,360 --> 02:24:57,197
models or, you know, you can you can
find bugs with other with other models

2568
02:24:57,280 --> 02:25:00,637
that are available today. So, why do you
think anthropic got singled out on this

2569
02:25:00,720 --> 02:25:03,357
front? >> Um, I don't know why like Fable

2570
02:25:03,440 --> 02:25:05,677
specifically was singled out um as you

2571
02:25:05,760 --> 02:25:08,237
know, again, Fable being the non-cyber

2572
02:25:08,320 --> 02:25:11,277
uh intention uh model as well. Um I

2573
02:25:11,360 --> 02:25:15,757
think that the thing that does change
over time is there's uh you know the

2574
02:25:15,840 --> 02:25:18,877
capabilities if you knew what to prompt
and knew what you were looking for and

2575
02:25:18,960 --> 02:25:22,477
then there's the capabilities I think
you know uh I'm not sure I resonate with

2576
02:25:22,560 --> 02:25:26,317
the like juicing high school athletes
metaphor from Alex but like you know the

2577
02:25:26,400 --> 02:25:29,277
uplift that you get like uplift is a
thing that we think about a lot when we

2578
02:25:29,360 --> 02:25:32,957
think about model um safety. So if you
look at our model cards for example, one

2579
02:25:33,040 --> 02:25:37,677
of the ways and we look at uh risk
in the bio domain is comparing the uplift

2580
02:25:37,760 --> 02:25:41,917
uh from sort of lay a lay person using
the model versus you know an expert or

2581
02:25:42,000 --> 02:25:45,197
lay person just using the internet and
and seeing what the comparison there is.

2582
02:25:45,280 --> 02:25:47,677
And so that is one trend that is you

2583
02:25:47,760 --> 02:25:49,997
know has been you know progressing as

2584
02:25:50,080 --> 02:25:55,037
the models get more capable. Um and so
maybe less why Fable is singled out and

2585
02:25:55,120 --> 02:25:57,277
maybe more like what
the overall trajectory is.

2586
02:25:57,360 --> 02:26:01,517
>> Interesting. So, we just had Ara
Karazzian, the uh lead economist on RAMP

2587
02:26:01,600 --> 02:26:04,877
on the podcast and one of the interesting
things that he said was, you

2588
02:26:04,960 --> 02:26:08,397
know, we we had during the Pentagon
situation, there were these headlines,

2589
02:26:08,480 --> 02:26:11,437
okay, this company will not use
anthropic anymore, but actually the data

2590
02:26:11,520 --> 02:26:16,157
from RAMP shows that spending actually
increased uh to anthropic models. It was

2591
02:26:16,240 --> 02:26:20,477
apparently a good publicity moment
for the company. So I I think it sort of

2592
02:26:20,560 --> 02:26:24,077
plays into a debate that we have like
here on the show about like how much of

2593
02:26:24,160 --> 02:26:28,317
this is like real concern for the issues
and how much of it is you know from

2594
02:26:28,400 --> 02:26:32,557
Anthropic is marketing and like we have
somebody from Anthropic here who can

2595
02:26:32,640 --> 02:26:35,197
actually shed some light on that.
We just said Stamos talking a little bit

2596
02:26:35,280 --> 02:26:38,237
about it from his perspective on the
security side but we're lucky to have

2597
02:26:38,320 --> 02:26:42,877
you here today. So so is it material
or is it marketing or some combination? I

2598
02:26:42,960 --> 02:26:46,397
mean I think uh it is one of the hardest
things to like really deeply believe

2599
02:26:46,480 --> 02:26:48,797
that something is true and not marketing

2600
02:26:48,880 --> 02:26:51,437
and then due to not even just anthropic.

2601
02:26:51,520 --> 02:26:55,437
I think like you know uh uh I think

2602
02:26:55,520 --> 02:26:58,957
people are generally right to be
skeptical of any company saying anything

2603
02:26:59,040 --> 02:27:01,837
and you should like put it through
your own filters as well. Um, but for me

2604
02:27:01,920 --> 02:27:05,197
personally, I'm always like, "No, but it's
real." And like we like both deeply

2605
02:27:05,280 --> 02:27:09,117
care about safety and are like to the
extent that we are being vocal about

2606
02:27:09,200 --> 02:27:12,877
anything, it is to either sort of help
paint the picture of what very likely is

2607
02:27:12,960 --> 02:27:16,797
coming or we believe is coming or what
we've already seen and spotted. Um, you

2608
02:27:16,880 --> 02:27:21,197
know, for example, in the in the Mythos
case, really just looking at uh uh like

2609
02:27:21,280 --> 02:27:24,237
vulnerability scanning and bug finding
and doing it in partnership with with

2610
02:27:24,320 --> 02:27:25,997
companies that were in that kind of

2611
02:27:26,080 --> 02:27:27,597
project glasswing initial announcement.

2612
02:27:27,680 --> 02:27:32,317
And so um the technology is awesome not

2613
02:27:32,400 --> 02:27:37,037
in the like in the just you know it is it
is doing really incredible things and

2614
02:27:37,120 --> 02:27:40,957
therefore even calling out what we see
as what is happening I think can seem

2615
02:27:41,040 --> 02:27:44,637
hypy. Uh I wish I could press a button
and was and like I could make everybody

2616
02:27:44,720 --> 02:27:47,357
believe that we are not being hypy.
I realize that's not the reality that we

2617
02:27:47,440 --> 02:27:48,957
operate in but at least from my

2618
02:27:49,040 --> 02:27:51,597
perspective um we try to call it like it

2619
02:27:51,680 --> 02:27:56,557
is. My understanding is that within
labs in particular in research and

2620
02:27:56,640 --> 02:28:01,277
development at anthropic that you're
using the best AI models to actually

2621
02:28:01,360 --> 02:28:04,317
build new products to prototype new
products and sort of test out your

2622
02:28:04,400 --> 02:28:07,837
thesis. So is this uh is this li is this

2623
02:28:07,920 --> 02:28:10,157
ban essentially now limiting your

2624
02:28:10,240 --> 02:28:12,477
ability to do that within labs? Yeah, I

2625
02:28:12,560 --> 02:28:17,757
mean definitely Fable uh is the best
model I've ever used and uh it's not to

2626
02:28:17,840 --> 02:28:20,957
say that like work has stopped but it's
definitely like less good than the other

2627
02:28:21,040 --> 02:28:23,757
models that uh that or sorry we using

2628
02:28:23,840 --> 02:28:26,077
models that are less good than that um

2629
02:28:26,160 --> 02:28:28,557
as well and it is also I mean maybe the

2630
02:28:28,640 --> 02:28:33,357
reverse of the uh distrusting the first
week uh you know response is what

2631
02:28:33,440 --> 02:28:36,557
happens when you don't have fable and
like obviously the Twitter reaction for

2632
02:28:36,640 --> 02:28:39,437
the people that had already kind of like
gotten uh into the model and and were

2633
02:28:39,520 --> 02:28:43,277
using it was strong. Um, but I'd say
even in my personal use, I'm like, "Oh,

2634
02:28:43,360 --> 02:28:46,557
like I'm on Opus 48 and it's good. Like
I'm still productive. I'm doing work."

2635
02:28:46,640 --> 02:28:50,397
But um, and we can go into sort of like
how my work changed with sort of these

2636
02:28:50,480 --> 02:28:55,117
like Fable or like you know uh the models
of that sort of uh family, but it

2637
02:28:55,200 --> 02:28:57,917
is noticeable for sure. >> Yeah,
I think we'd like to know that. I

2638
02:28:58,000 --> 02:28:59,917
mean, you know, we the public had access

2639
02:29:00,000 --> 02:29:03,197
to Fable for like a half a minute. Um,

2640
02:29:03,280 --> 02:29:07,917
groups in this like Project Glass Wing
have had access to Mythos. um we don't

2641
02:29:08,000 --> 02:29:11,437
really know what the difference
is between using a model that you can use

2642
02:29:11,520 --> 02:29:14,397
today is and using one of these you know

2643
02:29:14,480 --> 02:29:19,357
superanthropic models. Yeah. So what
actually could you do differently with a

2644
02:29:19,440 --> 02:29:20,477
fable or a mythos?

2645
02:29:20,560 --> 02:29:22,717
>> I think for me it's the sort of scope

2646
02:29:22,800 --> 02:29:27,837
and scale of delegation. Um and all
these things are really imperfect. Like

2647
02:29:27,920 --> 02:29:31,277
you know people would say like oh
is this now a level five software engineer

2648
02:29:31,360 --> 02:29:35,037
or level six but anybody who's used
these models extensively knows that

2649
02:29:35,120 --> 02:29:38,317
they're still spiky in capabilities
right in some ways they're you know in

2650
02:29:38,400 --> 02:29:43,117
many ways they're a better engineer than
me. Um and in other ways like you know I

2651
02:29:43,200 --> 02:29:46,957
was complaining today that it had missed
a descender like a the G that bottom

2652
02:29:47,040 --> 02:29:49,437
part is called the descender. I'm like,
how did you put that in the UI and it's

2653
02:29:49,520 --> 02:29:52,237
clipped? And of course, like there's
vision capabilities that need to improve

2654
02:29:52,320 --> 02:29:56,477
and there's uh debugging capabilities
and there's sometimes even just sort of

2655
02:29:56,560 --> 02:30:00,157
human common sense that is, you know,
we're way better at than when than the

2656
02:30:00,240 --> 02:30:02,797
models are. But overall, I think the big

2657
02:30:02,880 --> 02:30:07,437
shift for me working um and it was really
interesting because uh it sort of

2658
02:30:07,520 --> 02:30:10,317
coincided with me going back into
a builder role. So I really got to see

2659
02:30:10,400 --> 02:30:14,477
going from you know using these models
as an executive you're trying to do the

2660
02:30:14,560 --> 02:30:18,957
most of them but you know like not going
to have it right all your email and uh

2661
02:30:19,040 --> 02:30:22,797
you know I think strategy still needs
to come from you and then you can use the

2662
02:30:22,880 --> 02:30:25,757
models to sort of pressure test it. Um

2663
02:30:25,840 --> 02:30:29,917
but going back into a builder role
and going from okay I am delegating chunks

2664
02:30:30,000 --> 02:30:33,037
like please fix this bug or I'm thinking
of implementing this feature let's go

2665
02:30:33,120 --> 02:30:36,637
back and forth um to something that ends
up being much more sort of all right

2666
02:30:36,720 --> 02:30:40,797
like I got this bug report from one
of our users or I have this notion of

2667
02:30:40,880 --> 02:30:43,917
something that I want to build like
can you sketch out two or three ways in

2668
02:30:44,000 --> 02:30:48,317
which we could do it right like that
seems plausible often I find actually

2669
02:30:48,400 --> 02:30:52,157
sometimes it'll uh give me the the sort
of explanation or proposal and be like

2670
02:30:52,240 --> 02:30:55,517
okay that actually is over my head like
you are clearly way smarter than me like

2671
02:30:55,600 --> 02:30:59,917
explain it to me like I'm maybe not five
but at least you know uh uh not you and

2672
02:31:00,000 --> 02:31:03,837
it'll sometimes you know explain it
that way but then go build it and get it

2673
02:31:03,920 --> 02:31:08,397
right getting it right like you know
at a very very very high rate and I think

2674
02:31:08,480 --> 02:31:12,317
that starts really changing how you
operate like I I moved much more to

2675
02:31:12,400 --> 02:31:16,477
before going to bed making sure that I
had like queued up for fable like enough

2676
02:31:16,560 --> 02:31:19,117
chunky work to last I would call it
the whole night and then I would check in

2677
02:31:19,200 --> 02:31:21,757
later and it got it done in an hour and
it was like I guess hanging out for the

2678
02:31:21,840 --> 02:31:25,997
next seven hours but like really like
delegating like uh uh much more of a

2679
02:31:26,080 --> 02:31:28,957
goal than just a >> so
one one task for example.

2680
02:31:29,040 --> 02:31:31,917
>> Yeah. >> Like give us an example
of one task you would hand to it.

2681
02:31:32,000 --> 02:31:35,677
>> Uh I mean here's a kind of crazy one
which is uh for the programmers in the

2682
02:31:35,760 --> 02:31:40,797
audience like I uh I had written one
of our labs uh projects in um in Python.

2683
02:31:40,880 --> 02:31:43,837
That's like the language I know all
Instagram was all Python. And for like

2684
02:31:43,920 --> 02:31:47,197
some not super exciting reasons,
we actually needed it to be in Typescript

2685
02:31:47,280 --> 02:31:50,237
to deploy it. And I was like,
"All right, that's going to be like, you

2686
02:31:50,320 --> 02:31:55,117
know, in Instagram, we for years talked
about moving from Python to PHP or hack

2687
02:31:55,200 --> 02:31:58,397
or the the Facebook language after
the acquisition." And at least when I was

2688
02:31:58,480 --> 02:32:02,717
there, never did. Um, but I basically we
have a feature called dynamic workflows

2689
02:32:02,800 --> 02:32:05,197
where you can have it like also
break down the task into like a lot of

2690
02:32:05,280 --> 02:32:09,917
subtasks. And I trusted it to sort
of not just do the individual action, but

2691
02:32:10,000 --> 02:32:13,597
here's like a whole language conversion
of millions of hundreds of thousands of

2692
02:32:13,680 --> 02:32:17,437
lines of code at that point. Uh, go off
and do it. Go plan it. Go execute it. go

2693
02:32:17,520 --> 02:32:21,037
verify the work, double verify the work,
and then I came back to the work being

2694
02:32:21,120 --> 02:32:22,877
complete. So that level of like this is

2695
02:32:22,960 --> 02:32:26,317
a big sort of chunky uh task.

2696
02:32:26,400 --> 02:32:28,077
>> So that was so you're basically saying

2697
02:32:28,160 --> 02:32:33,597
it was faster. Did it in an hour. You're
you're s guessing compared to with fable

2698
02:32:33,680 --> 02:32:36,317
compared to what it would have been
before. I think the main difference is

2699
02:32:36,400 --> 02:32:39,517
in the past you would it would be like
great I did it and you'd be like but did

2700
02:32:39,600 --> 02:32:43,117
you you kind of took a shortcut here
or this is not quite right or I need to go

2701
02:32:43,200 --> 02:32:46,797
verify it or like oh you cut this corner
or >> it's like the managing interns thing

2702
02:32:46,880 --> 02:32:49,757
that everyone's been saying for the past
year which is very offensive to interns

2703
02:32:49,840 --> 02:32:52,717
by the way but yes >> yeah exactly >>
I don't know have you managed an intern?

2704
02:32:52,800 --> 02:32:54,877
Yeah, that's true. And it was and you're

2705
02:32:54,960 --> 02:32:57,037
saying it was more correct. It was so

2706
02:32:57,120 --> 02:33:00,157
it's faster. It's more accurate. Uh more

2707
02:33:00,240 --> 02:33:04,477
reliable and then according to the US
administration dangerous. I think the

2708
02:33:04,560 --> 02:33:08,237
other piece is it has like a a greater
theory of mind is the wrong word but

2709
02:33:08,320 --> 02:33:12,797
sort of like theory of project so that
it's less you know oh I'm going to make

2710
02:33:12,880 --> 02:33:15,757
this change and it'll say great I'll
make this change but really especially

2711
02:33:15,840 --> 02:33:19,757
if you've done like software engineering
at scale the best engineers kind of keep

2712
02:33:19,840 --> 02:33:22,957
in mind all the disperate parts of how
this things and they also see around the

2713
02:33:23,040 --> 02:33:26,877
corners like I can make this change
but if I don't do it in this way then the

2714
02:33:26,960 --> 02:33:30,397
next change is going to be incrementally
harder and I think that's like been a

2715
02:33:30,480 --> 02:33:34,237
significant difference I've seen in
that kind of class of models Okay, cool.

2716
02:33:34,320 --> 02:33:36,237
>> So, I think when
we talk about Anthropic

2717
02:33:36,320 --> 02:33:39,357
Labs, right, people think of Claude Code

2718
02:33:39,440 --> 02:33:41,597
because it is really your breakout

2719
02:33:41,680 --> 02:33:45,517
product and uh and it sounds like you've

2720
02:33:45,600 --> 02:33:48,957
been tasked with basically figuring out
what the next Claude code is. Would you

2721
02:33:49,040 --> 02:33:51,677
say that's an accurate description
of what you're doing at Labs and also why

2722
02:33:51,760 --> 02:33:55,917
does Anthropic need labs? >> Labs.
Yeah, it's also maybe worth

2723
02:33:56,000 --> 02:33:59,597
thinking about why we needed labs in 2024
when I arrived and why we need labs

2724
02:33:59,680 --> 02:34:03,357
today because I think that the answer
kind of shifts. Uh I started the labs

2725
02:34:03,440 --> 02:34:05,757
the original labs team with Ben
man who's one of the co-founders of

2726
02:34:05,840 --> 02:34:09,757
Anthropic. Um in my third week at
Enthropic and it had been something that

2727
02:34:09,840 --> 02:34:13,357
I've been bubbling under and at the time
the reason was really different. It was

2728
02:34:13,440 --> 02:34:16,957
all of our product engineering with team
was 25 people and we didn't have the

2729
02:34:17,040 --> 02:34:20,797
models really like we had claude that
when I joined it was set three like you

2730
02:34:20,880 --> 02:34:24,237
and Opus 3 like those were for the time
good models but you weren't going to

2731
02:34:24,320 --> 02:34:25,997
they were not even interns right they

2732
02:34:26,080 --> 02:34:29,997
weren't even IC3 engineers. Um so if you

2733
02:34:30,080 --> 02:34:34,397
have a team of only 25 or 30 engineers
they are working on like the next you

2734
02:34:34,480 --> 02:34:37,917
know incremental thing and we were
feeling like the models are starting to

2735
02:34:38,000 --> 02:34:41,037
get better but we don't have any
products that sort of show that off like

2736
02:34:41,120 --> 02:34:44,637
a good litmus test for me is when we
get ready to release a model do we have

2737
02:34:44,720 --> 02:34:48,637
either a product or a demo or or you
know some other illustration of

2738
02:34:48,720 --> 02:34:51,677
something that is very different and it
gets harder over time like with with

2739
02:34:51,760 --> 02:34:53,757
Fable you know even like illustrating

2740
02:34:53,840 --> 02:34:56,637
like that weekend task or you know this

2741
02:34:56,720 --> 02:35:01,037
longer amount of work. So really labs
at the time was let's make sure we don't

2742
02:35:01,120 --> 02:35:05,037
like our products don't fall behind
the model exponential that's happening. Um

2743
02:35:05,120 --> 02:35:08,397
and yeah so cloud code came out of that
initial one because nobody in the rest

2744
02:35:08,480 --> 02:35:12,077
of the product or people were thinking
about coding but nobody was sort of had

2745
02:35:12,160 --> 02:35:15,757
the like space to go and think about
well what if we totally change the form

2746
02:35:15,840 --> 02:35:18,957
factor and we embrace the fact that the
models were going to uh evolve in this

2747
02:35:19,040 --> 02:35:23,357
way and a lot of the like the two most
useful thought exercises we do in labs.

2748
02:35:23,440 --> 02:35:27,357
One is like visualize the gap between
what the models can do today and how

2749
02:35:27,440 --> 02:35:31,037
most people use it and can we close that
gap. So that's one. And the other one is

2750
02:35:31,120 --> 02:35:33,677
imagine what the models are bad at now
that they're actually going to be really

2751
02:35:33,760 --> 02:35:36,637
good at in six months and let's make
sure we have a product ready for that by

2752
02:35:36,720 --> 02:35:41,197
then. And I think those are like the two
guiding uh questions for for labs. Um

2753
02:35:41,280 --> 02:35:44,717
and then also out of that first
incarnation came computer use. Computer

2754
02:35:44,800 --> 02:35:47,597
use was different though because when
we built it it was really bad. Like we

2755
02:35:47,680 --> 02:35:51,597
tried a bunch of products with it
and this was around you know silent 35 and

2756
02:35:51,680 --> 02:35:55,357
be like Claude can you help me you know
clean up my desktop and it would like

2757
02:35:55,440 --> 02:35:58,877
click the thing and it delete the file
like this is not not safe for release

2758
02:35:58,960 --> 02:36:02,717
we're definitely not going to go and
and and build this or to ship this but we

2759
02:36:02,800 --> 02:36:06,717
had that product so that every new model
that we we'd release we'd first check it

2760
02:36:06,800 --> 02:36:10,157
internally and say did computer use get
better and we'd tell the research team

2761
02:36:10,240 --> 02:36:13,357
how it gotten better or worse until
the moment where we said it's good enough

2762
02:36:13,440 --> 02:36:16,397
we're actually going to put a product
out around this. So it's it also gives

2763
02:36:16,480 --> 02:36:19,677
you this sort of uh sort of beacon into
the future that then you can kind like

2764
02:36:19,760 --> 02:36:23,837
measure your your future products against.
Um but then compare it to now.

2765
02:36:23,920 --> 02:36:25,517
So we have a driving product team.

2766
02:36:25,600 --> 02:36:30,237
There's co-work there's um you know
cloud code has grown a lot. We have our

2767
02:36:30,320 --> 02:36:34,637
platform. Um and now I think it's
actually much much less about none of

2768
02:36:34,720 --> 02:36:37,757
these product teams are doing this sort
of thinking and I think it's much more

2769
02:36:37,840 --> 02:36:42,157
that the models are advancing really
quickly and even our capability to

2770
02:36:42,240 --> 02:36:46,237
interact with them needs to evolve.
So one of the things we collaborated with

2771
02:36:46,320 --> 02:36:49,837
uh labs and cloud code that we ship
today um is cloud code artifacts. So

2772
02:36:49,920 --> 02:36:53,037
having cloud code not just be able to type
back to you but also sort of draw a

2773
02:36:53,120 --> 02:36:56,477
picture or give you an illustration. And
that partially came from spending a lot

2774
02:36:56,560 --> 02:37:00,717
of time in lab saying just a text box
and a big text response is not going to

2775
02:37:00,800 --> 02:37:03,197
cut it anymore. Like when I mentioned
that the models feel like they're way

2776
02:37:03,280 --> 02:37:05,917
smarter than me when they talk to me
sometimes I'm like can you draw me a

2777
02:37:06,000 --> 02:37:09,837
picture because this is what I actually
need to fully uh fully understand this.

2778
02:37:09,920 --> 02:37:13,357
But it's really what we've been thinking
about is, you know, yes, we have a lot

2779
02:37:13,440 --> 02:37:16,637
more products. You know, we have we
actually have a lot of consolidation to

2780
02:37:16,720 --> 02:37:19,517
do in our products. Like that's another
initiative that we have. But within

2781
02:37:19,600 --> 02:37:23,597
that, we still have an opportunity
to make things much more accessible to a

2782
02:37:23,680 --> 02:37:27,197
person that does not spend all of their
time thinking about prompting and the

2783
02:37:27,280 --> 02:37:29,997
exponential and the difference between
high, low, and medium effort. Like

2784
02:37:30,080 --> 02:37:34,157
there's a lot we can do.
>> But Mike, so there's a it's it puts

2785
02:37:34,240 --> 02:37:38,477
people using anthropic models in an
interesting place, right? Um there you

2786
02:37:38,560 --> 02:37:43,037
know cursor I think just sold for 60
billion to SpaceX and someone put this

2787
02:37:43,120 --> 02:37:46,957
meme on Twitter that like you know
cursor would have sold for 300 billion

2788
02:37:47,040 --> 02:37:51,037
if it wasn't for this guy and it's
a picture of Boris Churnney the person who

2789
02:37:51,120 --> 02:37:53,677
created Claude Code. Um, and so for

2790
02:37:53,760 --> 02:37:58,077
companies that are going to build
on top of Anthropic technology, you know,

2791
02:37:58,160 --> 02:38:02,637
they're going to wonder, do I want
to partner with Anthropic or is Anthropic

2792
02:38:02,720 --> 02:38:05,677
going to go ahead and and build the
product that I'm going to want to build

2793
02:38:05,760 --> 02:38:07,517
potentially even after
partnering with them?

2794
02:38:07,600 --> 02:38:11,037
>> Yeah. And I mean, we'll take the like
agentic coding side and I think the

2795
02:38:11,120 --> 02:38:15,037
broader sort of aspect of of you know,
being both a platform and a product I

2796
02:38:15,120 --> 02:38:17,757
think is is really interesting. um when

2797
02:38:17,840 --> 02:38:20,557
we take on projects it's the goal is

2798
02:38:20,640 --> 02:38:23,357
often to sort of push that area of the

2799
02:38:23,440 --> 02:38:28,157
industry forward. So um you know there
were AI coding editors and some of them

2800
02:38:28,240 --> 02:38:31,837
were really good and you know uh but
nobody was quite thinking about it in as

2801
02:38:31,920 --> 02:38:35,037
sort of free form a way as we got
to think about it with cloud code. And now

2802
02:38:35,120 --> 02:38:39,277
a lot more products have that flavor
than I think would have otherwise. And

2803
02:38:39,360 --> 02:38:42,557
so I think if we're ever you can call
me out on this Alex like if we're ever

2804
02:38:42,640 --> 02:38:44,877
entering an industry where like all
you're doing is the same thing everybody

2805
02:38:44,960 --> 02:38:48,317
else is doing but like you've got the
anthropic brand I feel like that's a bad

2806
02:38:48,400 --> 02:38:51,837
use of our time and a bad use of our
either labs or product team time. Like

2807
02:38:51,920 --> 02:38:55,117
if we're going in somewhere it should
hopefully be to say all right we think

2808
02:38:55,200 --> 02:38:58,637
that the direction of travel is this way
we can build a product of that and then

2809
02:38:58,720 --> 02:39:01,997
by the way there's no world nor should
there be a world where like all the

2810
02:39:02,080 --> 02:39:04,717
products are entropic products that would
be a bad world right so like that

2811
02:39:04,800 --> 02:39:08,717
is hopefully either creating new space
for for companies or sort of showing the

2812
02:39:08,800 --> 02:39:10,797
way where other products
can incorporate that

2813
02:39:10,880 --> 02:39:13,917
>> it would almost be like working
for a tech company that has like social

2814
02:39:14,000 --> 02:39:17,597
messaging video

2815
02:39:17,680 --> 02:39:22,877
>> right Mike yeah okay you did leave >>
well there was question for example when

2816
02:39:22,960 --> 02:39:26,397
you know Anthropic launched a product
that was seen as competitive to Figma

2817
02:39:26,480 --> 02:39:29,997
and you had been on the Figma board
prior to that and I think you stepped

2818
02:39:30,080 --> 02:39:33,597
down is that correct? Yeah. And so um
it's a good question that it's al Alex

2819
02:39:33,680 --> 02:39:36,957
has brought up I think where Silicon
Valley is known for this really healthy

2820
02:39:37,040 --> 02:39:41,197
vibrant risk tolerant startup ecosystem
and when the bigs start coming in with

2821
02:39:41,280 --> 02:39:45,357
tons of venture capital and you know
a lot of resources uh people say well wait

2822
02:39:45,440 --> 02:39:48,317
are they just are they essentially
just going to steal my idea?

2823
02:39:48,400 --> 02:39:52,557
>> Yeah. No, I think I think our dual
existence and it's something that other

2824
02:39:52,640 --> 02:39:55,277
companies have had to navigate.
We talked all talked about Amazon a lot in

2825
02:39:55,360 --> 02:39:58,477
the in the previous panel like they
have to navigate this role where they are

2826
02:39:58,560 --> 02:40:01,917
both infrastructure provider. They
obviously have a very large e-commerce.

2827
02:40:02,000 --> 02:40:06,317
They they do video but they also serve
video and um and then you know by and

2828
02:40:06,400 --> 02:40:09,597
large customers can live in that dual
world of like okay I'm using their

2829
02:40:09,680 --> 02:40:12,637
infrastructure also knowing that they
they are also using their infrastructure

2830
02:40:12,720 --> 02:40:17,197
to do that and I think the uh you can
talk to our customers and see how well

2831
02:40:17,280 --> 02:40:19,197
we're doing at this like the thing
I always try to do is like at least

2832
02:40:19,280 --> 02:40:20,557
approach it with a lot of transparency.

2833
02:40:20,640 --> 02:40:23,037
So the cursor example is an interesting
one where like Michael and I talked a

2834
02:40:23,120 --> 02:40:28,317
lot over the you know time around here's
where things are heading. Um and uh you

2835
02:40:28,400 --> 02:40:31,277
know with similarly with with the other
products that we think about like can we

2836
02:40:31,360 --> 02:40:33,917
I think it's a couple things.
It's transparency and then it's shared

2837
02:40:34,000 --> 02:40:37,837
building blocks like yeah I think um
in general and I actually don't think

2838
02:40:37,920 --> 02:40:41,037
there's any cases where this is even
true like we're trying to build on top

2839
02:40:41,120 --> 02:40:44,877
of the same capabilities that are
available elsewhere. The last time I was

2840
02:40:44,960 --> 02:40:47,677
here in the Commonwealth Club on this
stage was our healthcare day at the

2841
02:40:47,760 --> 02:40:52,397
beginning of the year and we didn't ship
like cloud healthcare only we have it

2842
02:40:52,480 --> 02:40:55,677
like nobody else has it. We shipped
a bunch of like plugins and skills and

2843
02:40:55,760 --> 02:40:57,677
MCPs and like complimentary abilities.

2844
02:40:57,760 --> 02:41:01,597
So that's how I'm not claiming it's easy
or that it's a straightforward thing,

2845
02:41:01,680 --> 02:41:05,117
but it is how we're trying to navigate
what is like admittedly a complicated

2846
02:41:05,200 --> 02:41:08,317
sort of situation. >> Speaking
of startups, Anthropic is still

2847
02:41:08,400 --> 02:41:13,277
technically a startup, but you're worth
a lot of money. I mean, what what's the

2848
02:41:13,360 --> 02:41:15,757
latest valuation? Is it >> 965?

2849
02:41:15,840 --> 02:41:19,277
>> 965 billion dollars
or something like that. So

2850
02:41:19,360 --> 02:41:20,877
>> you sold Instagram
for a billion, right?

2851
02:41:20,960 --> 02:41:25,037
startup in 2010. Yeah. >> Right.
Financials have changed uh quite

2852
02:41:25,120 --> 02:41:27,437
a bit since then. And yet uh Enthropic

2853
02:41:27,520 --> 02:41:31,117
has positioned itself, it is uh you know

2854
02:41:31,200 --> 02:41:33,277
a PBC and it's positioned itself as sort

2855
02:41:33,360 --> 02:41:36,877
of a more um ethical uh company around

2856
02:41:36,960 --> 02:41:38,877
building AI. And I'm wondering if you

2857
02:41:38,960 --> 02:41:43,917
could talk a little bit about how you
see that positioning and anthropics role

2858
02:41:44,000 --> 02:41:47,357
in particular changing the culture
of the valley. Like I know I think back to

2859
02:41:47,440 --> 02:41:51,437
how Google in the beginning of the 2000s
really changed the culture of Silicon

2860
02:41:51,520 --> 02:41:55,197
Valley in so many ways and how do you
see anthropics culture now dictating

2861
02:41:55,280 --> 02:41:57,517
this next era? >> Yeah,
that's a really interesting

2862
02:41:57,600 --> 02:41:59,837
question. Maybe I'll start like
inside and I think there's an external

2863
02:41:59,920 --> 02:42:02,717
component too. I think the reason
I joined in the first place so I was

2864
02:42:02,800 --> 02:42:07,437
winding down my second startup um
and knew I wanted to go work at a frontier

2865
02:42:07,520 --> 02:42:10,557
lab because I had started to use these
models for coding and they were bad at

2866
02:42:10,640 --> 02:42:13,277
coding but I could see that they were
as bad as they were ever going to be at

2867
02:42:13,360 --> 02:42:15,997
coding. they were going to improve and I
had started building on top of these

2868
02:42:16,080 --> 02:42:19,357
APIs. So the startup I was doing
was called artifact and we did sort of AI

2869
02:42:19,440 --> 02:42:23,997
powered uh sort of news recommendations
and um actually read a lot of big

2870
02:42:24,080 --> 02:42:25,677
technology via artifact back in the day.

2871
02:42:25,760 --> 02:42:28,557
It was a lot of things we added.
So um >> but not wired

2872
02:42:28,640 --> 02:42:31,837
>> but not you know you guys had
a really hard payw wall to be honest.

2873
02:42:31,920 --> 02:42:34,877
>> Fair enough.
>> We didn't do very great.

2874
02:42:34,960 --> 02:42:37,517
>> Do you need a discount on my
subscription because I can get one for

2875
02:42:37,600 --> 02:42:40,637
you. Okay. >> It's actually
really funny like the bacon deals.

2876
02:42:40,720 --> 02:42:43,677
>> Yeah, bacon deals. It's the It's the
login cookies. It's like really hard to

2877
02:42:43,760 --> 02:42:46,077
keep people. >> I know.
I know. I just Please escalate

2878
02:42:46,160 --> 02:42:48,637
this to Kai now. I know.

2879
02:42:48,720 --> 02:42:52,077
>> Um uh and email
login very hard to do in

2880
02:42:52,160 --> 02:42:56,077
um but so uh but I was building on top
of the APIs and be like, "Wow, okay,

2881
02:42:56,160 --> 02:42:58,957
they're they're able to do really
interesting things." Um but what

2882
02:42:59,040 --> 02:43:01,117
ultimately made me go to Anthropic was

2883
02:43:01,200 --> 02:43:04,157
like they walk the walk and they really

2884
02:43:04,240 --> 02:43:08,717
uh like deeply believe in trying to make
AI go well for humanity and that is like

2885
02:43:08,800 --> 02:43:13,117
in that's like in the water internally
and I think has been why I think the

2886
02:43:13,200 --> 02:43:16,637
company has remained as cohesive as it
has even as we've grown and I think that

2887
02:43:16,720 --> 02:43:20,157
it's like a testament also to the the
co-founders there on how often they are

2888
02:43:20,240 --> 02:43:23,357
talking about this as well. It was a
a surprise for me coming from a world

2889
02:43:23,440 --> 02:43:28,157
where at Instagram we did a weekly all
hands and we talked about product 95% of

2890
02:43:28,240 --> 02:43:31,277
the time and maybe 5% of the time we
talked about something else that was

2891
02:43:31,360 --> 02:43:34,557
going on in the world around
the company. uh probably maybe I'm

2892
02:43:34,640 --> 02:43:37,517
underelling our like go to market maybe
it was like 8020 but it was definitely

2893
02:43:37,600 --> 02:43:39,517
very very heavy product and I remember

2894
02:43:39,600 --> 02:43:44,557
anthropic about six months in uh myself
um and Kate Jensen who's one of the the

2895
02:43:44,640 --> 02:43:47,117
leaders in the sales organization did
a joint all hands where we talked about

2896
02:43:47,200 --> 02:43:50,157
our like you know how we're doing
product and go to market together and

2897
02:43:50,240 --> 02:43:52,957
people were like this is so great I
finally understand our product strategy

2898
02:43:53,040 --> 02:43:57,277
and like what we've been doing it's
like oh right this is not quote unquote a

2899
02:43:57,360 --> 02:44:01,597
product company you know it is a very
missiondriven AI company with like a

2900
02:44:01,680 --> 02:44:03,997
very strong sense of like why it exists

2901
02:44:04,080 --> 02:44:06,077
in the world. I think in terms of the

2902
02:44:06,160 --> 02:44:11,357
overall impact on the valley remains
to be seen. I think um the positive signs

2903
02:44:11,440 --> 02:44:15,197
that I've seen or interesting signs I've
seen that I've seen are like an renewed

2904
02:44:15,280 --> 02:44:19,117
interest in philanthropy um across the
board and I think that's something that

2905
02:44:19,200 --> 02:44:23,197
um has been written about and I think it
will be an interesting sort of outflow.

2906
02:44:23,280 --> 02:44:26,957
Again, who knows how all of this goes,
but like uh depending on how it goes, it

2907
02:44:27,040 --> 02:44:30,877
could mean a lot of interesting new sort
of philanthropic um uh deployment. And

2908
02:44:30,960 --> 02:44:33,517
then I think the other piece is you know

2909
02:44:33,600 --> 02:44:37,757
uh the conversation around how AI could

2910
02:44:37,840 --> 02:44:41,757
or should go is one that is happening
in real time with the technology versus

2911
02:44:41,840 --> 02:44:44,717
retrospectively which I think has been
the case for other technology waves and

2912
02:44:44,800 --> 02:44:47,597
I think that is a good
thing. >> Interesting.

2913
02:44:47,680 --> 02:44:50,637
>> You know um Mike you know you talked a

2914
02:44:50,720 --> 02:44:54,877
little bit about um anthropic has
this gap that it sees between the

2915
02:44:54,960 --> 02:44:59,037
capabilities of the models and where
everybody is building products. And with

2916
02:44:59,120 --> 02:45:03,517
labs, what you try to do is get ahead
of that so you can show people what AI

2917
02:45:03,600 --> 02:45:07,837
might be able to do now and six months
from now. So please tell us, please tell

2918
02:45:07,920 --> 02:45:11,997
us like what you're building, um, where
you see the where you see the potential

2919
02:45:12,080 --> 02:45:14,077
>> and and what people
should be on the lookout for.

2920
02:45:14,160 --> 02:45:17,277
>> Yeah, it's a road map.
>> Oh, and if I can throw an and

2921
02:45:17,360 --> 02:45:21,437
>> what you're building now, but also
if you had if you have a pie in the sky

2922
02:45:21,520 --> 02:45:25,437
like Elon Musk data centers in space
type ambition, I want to hear about that

2923
02:45:25,520 --> 02:45:27,517
too. Tell us tell us
what tell us everything.

2924
02:45:27,600 --> 02:45:29,677
>> Great. We have 13 minutes left.

2925
02:45:29,760 --> 02:45:34,477
>> Exactly. The rest is the monologue
in my chronic group. Um I think maybe two

2926
02:45:34,560 --> 02:45:37,917
themes I'm really excited about that we've
been exploring a lot. Um the first

2927
02:45:38,000 --> 02:45:41,037
one is giving Claude an environment

2928
02:45:41,120 --> 02:45:46,637
where it has more agency and it also
has more self-nowledge. And I'm going to

2929
02:45:46,720 --> 02:45:50,477
unpack that because that's like a lot
of AI words. But um uh I'll give you an

2930
02:45:50,560 --> 02:45:53,517
example of where we are currently doing
a bad job of this. Like if you are in a

2931
02:45:53,600 --> 02:45:57,517
cloud project and you make a file
with claude, you're like that's great. can

2932
02:45:57,600 --> 02:46:02,477
you add it to our project? Cloud will be
no you have to go download the file and

2933
02:46:02,560 --> 02:46:06,717
go drag and drop into this thing and
you're like what? Um uh until yesterday

2934
02:46:06,800 --> 02:46:09,597
I would have said the same thing about
cloud design and cloud code where if

2935
02:46:09,680 --> 02:46:12,637
you're in cloud code you're like cool
like I need a design for this thing that

2936
02:46:12,720 --> 02:46:15,357
we're building let's or you're in cloud
design and you make a mockup and you

2937
02:46:15,440 --> 02:46:19,277
want to go build it be like cool here's
a zip file and you're like what you know

2938
02:46:19,360 --> 02:46:22,797
and I think um so that's a little bit
of interoperability but in general this

2939
02:46:22,880 --> 02:46:26,557
theme of giving if you give cloud
a lot of notion of its environment I was

2940
02:46:26,640 --> 02:46:30,397
talking to actually a customer like
an API customer um and one of the things

2941
02:46:30,480 --> 02:46:33,597
that they were experimenting with
is actually even giving claude like a

2942
02:46:33,680 --> 02:46:37,117
secure version of their source code
while it's running in the agent loop in

2943
02:46:37,200 --> 02:46:39,917
their product so that if it hits
an issue it doesn't go like I don't know I

2944
02:46:40,000 --> 02:46:42,797
hit an issue it can be like well it's
probably this thing you know at least

2945
02:46:42,880 --> 02:46:46,157
when it's talking to one of the sort
of maintainers of of the software and so

2946
02:46:46,240 --> 02:46:49,277
that overall theme and of course you
have to do it with safeguards and like

2947
02:46:49,360 --> 02:46:53,437
be really careful about what you unlock
with it sounds kind of obvious but it's

2948
02:46:53,520 --> 02:46:57,357
actually night and day in terms of how
expressive these products end up being

2949
02:46:57,440 --> 02:46:59,837
able to be right and you can even see it

2950
02:46:59,920 --> 02:47:04,877
going from like maybe like core chat
or classic chat in cloudi and something

2951
02:47:04,960 --> 02:47:07,917
like co-work where it's got a little
bit more agency and it has a runtime and

2952
02:47:08,000 --> 02:47:10,717
it's able to sort of like understand
a little bit about it environment but I

2953
02:47:10,800 --> 02:47:14,877
think we are at like 10% of the journey
about where we we could go. Um actually

2954
02:47:14,960 --> 02:47:17,677
one of the reasons I think people got
excited about things like openclaw is

2955
02:47:17,760 --> 02:47:22,957
seeing how a you know harness that is
modifiable and you can talk to it about

2956
02:47:23,040 --> 02:47:26,397
things and you don't ever get the sense
of like oh sorry I can't do that you're

2957
02:47:26,480 --> 02:47:30,237
gonna have to go to this like settings
screen and turn it on. It's just a thing

2958
02:47:30,320 --> 02:47:33,597
it has access to and hopefully with like
the right gardening and permissions. So

2959
02:47:33,680 --> 02:47:38,157
that's like theme one that I'm like
extremely excited about. Um and I think

2960
02:47:38,240 --> 02:47:39,837
I if we do it right should actually like

2961
02:47:39,920 --> 02:47:41,437
transform all of our products like from

2962
02:47:41,520 --> 02:47:46,237
head to toe. Um the other piece um is uh

2963
02:47:46,320 --> 02:47:50,077
and I'll maybe like share like the the
the feel not the internal product we're

2964
02:47:50,160 --> 02:47:54,077
working on but like the the phrase I got
as feedback was uh like I think closing

2965
02:47:54,160 --> 02:47:56,717
the gap. I mean I talked about closing
the gap between capabilities and

2966
02:47:56,800 --> 02:47:59,997
reality. I think it's also closing the
gap between how people understand their

2967
02:48:00,080 --> 02:48:04,477
own work and then how the actual
day-to-day is to do that work. Um I was

2968
02:48:04,560 --> 02:48:06,637
talking to somebody uh internally who's

2969
02:48:06,720 --> 02:48:10,157
on our privacy team and to move a ticket

2970
02:48:10,240 --> 02:48:14,557
from like one queue through another one
via the task tracker into another was

2971
02:48:14,640 --> 02:48:16,397
like eight different steps of copy and

2972
02:48:16,480 --> 02:48:18,717
pasting of like manually moving pretty

2973
02:48:18,800 --> 02:48:21,757
you know like uh uh kind of annoying to

2974
02:48:21,840 --> 02:48:25,837
have to do probably errorprone. You have
to like keep spot checking it. and we

2975
02:48:25,920 --> 02:48:29,917
helped her with one of our labs projects
to basically like make that not a pain

2976
02:48:30,000 --> 02:48:32,397
and she's like ah this is the first time
in my career and she's like been working

2977
02:48:32,480 --> 02:48:35,677
for 30 years. We're like what's
in my head and what I am using is like now

2978
02:48:35,760 --> 02:48:40,797
this like it is now closed. Um,
and I want to like bring that feeling to

2979
02:48:40,880 --> 02:48:43,837
everybody who like, you know, of course
cloud unlocked a lot of, you know,

2980
02:48:43,920 --> 02:48:47,197
nontechnical people being able to code,
but like it's we're still asking people

2981
02:48:47,280 --> 02:48:51,357
to understand way too many concepts
of like what is what is, you know, the

2982
02:48:51,440 --> 02:48:55,357
difference between like my sandbox
environment and production or like

2983
02:48:55,440 --> 02:48:58,557
connected MCP is myself or others or how
should I store data and of course you

2984
02:48:58,640 --> 02:49:02,077
can't abstract everything, but like if
you combine both of those themes, if you

2985
02:49:02,160 --> 02:49:04,797
give cloud a lot of self-nowledge and
you're creating an environment where it

2986
02:49:04,880 --> 02:49:09,517
can actually solve complex problems for
people and repeatable ways. Um, I think

2987
02:49:09,600 --> 02:49:11,517
I get like very very excited about those.

2988
02:49:11,600 --> 02:49:12,637
>> And your moonshot. >> Yeah.

2989
02:49:12,720 --> 02:49:14,237
>> I'm not letting you off the hook.
What's your moonshot?

2990
02:49:14,320 --> 02:49:16,957
>> Moonshot. >> Yeah. >> Uh, no.
Nothing in space. Although I

2991
02:49:17,040 --> 02:49:21,357
guess we're, you know, we're in talking
to SpaceX about spacey things. Um, but

2992
02:49:21,440 --> 02:49:22,957
you're talking to space. I mean, those

2993
02:49:23,040 --> 02:49:25,997
are the announcement for compute. Yeah.

2994
02:49:26,080 --> 02:49:30,237
>> Um, it it was exploring extra
extraorbital. What was the phrase?

2995
02:49:30,320 --> 02:49:34,077
Something about exploring
like post orbital world things.

2996
02:49:34,160 --> 02:49:37,517
>> Definitely not my department, but yeah,
there's there's stuff stuff in So the

2997
02:49:37,600 --> 02:49:40,317
labs is isn't working specifically
with the team on compute,

2998
02:49:40,400 --> 02:49:42,397
>> right? Exactly. Separate separate.

2999
02:49:42,480 --> 02:49:44,717
>> Totally separate. Okay.
Okay. So you're moonshot.

3000
02:49:44,800 --> 02:49:47,917
>> Yeah. >> Do you personally
believe in data centers in space?

3001
02:49:48,000 --> 02:49:51,997
>> I had a conversation I uh I by far
from a data center expert, but I talked to

3002
02:49:52,080 --> 02:49:55,837
somebody who is a like a person who
sends things to space who's not Elon

3003
02:49:55,920 --> 02:49:58,557
Musk. Um and >> that's what
that's what you would say.

3004
02:49:58,640 --> 02:50:00,957
>> Yeah.

3005
02:50:01,040 --> 02:50:05,597
Um uh and they were really bullish and I
was like trying to talk about why and

3006
02:50:05,680 --> 02:50:07,197
there was basically like I guess

3007
02:50:07,280 --> 02:50:11,037
effectively infinite um uh uh power if

3008
02:50:11,120 --> 02:50:14,797
you convert it well and you know
infinite land and I was like okay you

3009
02:50:14,880 --> 02:50:17,917
can buy that I mean and I think they
feel good about the shielding you have

3010
02:50:18,000 --> 02:50:21,677
to do again clearly not my area of
expertise but after that talk I was like

3011
02:50:21,760 --> 02:50:26,557
okay I see I see it you know um even if
it's going to take a few years at first I

3012
02:50:26,640 --> 02:50:29,437
>> admittedly thought it was a crazy idea
in general but now I'm like oh actually

3013
02:50:29,520 --> 02:50:31,277
clearly can understand
why this might make sense.

3014
02:50:31,360 --> 02:50:32,717
>> When when you were
talking earlier about

3015
02:50:32,800 --> 02:50:36,477
the ways that uh the work in claude
is going to get compressed and all those

3016
02:50:36,560 --> 02:50:40,877
steps, I couldn't help but think
of tokens and how you know maybe it's good

3017
02:50:40,960 --> 02:50:43,277
for your business model in the short
term if people have to take so many

3018
02:50:43,360 --> 02:50:46,557
steps and use so many tokens. But tokens
have become this unit of economics that

3019
02:50:46,640 --> 02:50:48,237
we're using to describe the industry now

3020
02:50:48,320 --> 02:50:50,717
and people are token maxing and now

3021
02:50:50,800 --> 02:50:54,077
they're token mising and and and uh one

3022
02:50:54,160 --> 02:50:55,997
I want to see I want to hear where you

3023
02:50:56,080 --> 02:50:57,917
sit on that spectrum if you're a token

3024
02:50:58,000 --> 02:51:01,197
maxer and two is there a near future in

3025
02:51:01,280 --> 02:51:03,037
which the industry is not actually

3026
02:51:03,120 --> 02:51:06,237
measured by tokens you know it goes the

3027
02:51:06,320 --> 02:51:10,397
way of MIPS or dialup or some other
you know there's some other unit of

3028
02:51:10,480 --> 02:51:14,557
measurement that actually defines
the economics of this era. Yeah, I think

3029
02:51:14,640 --> 02:51:17,197
both of those are really interesting
questions. It was interesting earlier

3030
02:51:17,280 --> 02:51:19,917
this year when you started hearing about
like companies that have like dashboards

3031
02:51:20,000 --> 02:51:24,077
showing like who used it the most and we
of course have like internal metrics as

3032
02:51:24,160 --> 02:51:28,237
well. And we found that there's not
a lot of correlation between like the

3033
02:51:28,320 --> 02:51:31,117
person who's using the most tokens
and like the person that I like it it's an

3034
02:51:31,200 --> 02:51:33,597
interesting thought I maybe do at your
companies is like write down your 10

3035
02:51:33,680 --> 02:51:36,157
most productive people that you think
are most productive and then like get

3036
02:51:36,240 --> 02:51:39,037
your top 10 token users and see how
closely they correlate. At least for us

3037
02:51:39,120 --> 02:51:43,597
it wasn't that correlated. Does it seem
dangerous to sort of like uh sort of

3038
02:51:43,680 --> 02:51:47,437
purely glorify the like maximum usage
obviously it's like very gameable but

3039
02:51:47,520 --> 02:51:51,197
even beyond that I think it's it's
you know yes you can ask cloud to do 10

3040
02:51:51,280 --> 02:51:53,837
different variants on something but if
you thought about it deeply maybe you

3041
02:51:53,920 --> 02:51:56,397
would do choose two that you thought
were most promising and the third one if

3042
02:51:56,480 --> 02:52:00,157
you then had like some iteration on that
as well. So I would not say like a token

3043
02:52:00,240 --> 02:52:04,557
max. Actually the tokeniest thing was
that conversion uh thing I did was just

3044
02:52:04,640 --> 02:52:07,277
like a couple million tokens. There's
like a lot of tokens that it took to to

3045
02:52:07,360 --> 02:52:11,997
convert the uh the thing from from
Python to TypeScript. Um but I think

3046
02:52:12,080 --> 02:52:16,077
people are being more thoughtful about
um these different pieces and and one of

3047
02:52:16,160 --> 02:52:19,917
the things we look at whenever we look
at a model launch is not just model

3048
02:52:20,000 --> 02:52:22,957
intelligence but we're also really
thinking about model intelligence and

3049
02:52:23,040 --> 02:52:26,397
effort and token efficiency as that
combination. And I think that's a big

3050
02:52:26,480 --> 02:52:29,917
lever we have to improve is how do
we continue to be more and more token

3051
02:52:30,000 --> 02:52:33,277
efficient for a given task so that you
can also hopefully you don't have to

3052
02:52:33,360 --> 02:52:36,077
think very hard about this we can do
this automatically but like we're able

3053
02:52:36,160 --> 02:52:39,277
to tune the solution to the problem
a little bit more. Um, and then to your

3054
02:52:39,360 --> 02:52:42,957
second question, yeah, I, you know,
when I was still in the CPO seat, I I was

3055
02:52:43,040 --> 02:52:45,917
thinking a lot about sort
of outcomebased pricing as something that

3056
02:52:46,000 --> 02:52:48,877
would be really interesting to do if you
could do it. And of course, if you talk

3057
02:52:48,960 --> 02:52:52,877
to, uh, like the Sieras and Fins of
the world that have like a really clear,

3058
02:52:52,960 --> 02:52:56,557
like we kept this, you know, we were
able to solve this customer request and

3059
02:52:56,640 --> 02:53:00,637
not have it go escalated. Like that's
really clear. Um, it gets so much

3060
02:53:00,720 --> 02:53:03,437
fuzzier on these like tasks that we
actually ask Claude these days like I

3061
02:53:03,520 --> 02:53:06,957
had a strategy document. I use Claude to
critique my strategy document. Like what

3062
02:53:07,040 --> 02:53:09,517
was the outcome? It's like well I don't
know it's like tell me how the strategy

3063
02:53:09,600 --> 02:53:12,797
goes six months from now. It feels like
it's going to be very hard to to capture

3064
02:53:12,880 --> 02:53:15,677
that as well. But um I would like to see

3065
02:53:15,760 --> 02:53:17,837
some more experimentation around can you

3066
02:53:17,920 --> 02:53:22,877
better capture like what it's worth you
know to the individual and then or the

3067
02:53:22,960 --> 02:53:26,637
company and then can we find the best
way to to do that as well and I guess

3068
02:53:26,720 --> 02:53:29,997
the the most concrete thing we've moved
towards that and uh we have a product

3069
02:53:30,080 --> 02:53:32,557
product called cloud managed
agents where we'll run all of the

3070
02:53:32,640 --> 02:53:35,997
infrastructure for you in terms of doing
all of the you know agentic harness and

3071
02:53:36,080 --> 02:53:39,437
calling the tools etc and you can either
do it in sort of the normal mode which

3072
02:53:39,520 --> 02:53:42,717
is you give it tasks it you know will go
through tokens it'll tell you when it's

3073
02:53:42,800 --> 02:53:46,157
done or we an outcome based mode where
you can say here's what good looks like

3074
02:53:46,240 --> 02:53:50,237
here's a rubric go and do it and you
know you know it'll go off and and make

3075
02:53:50,320 --> 02:53:53,197
it more outcome so like if everybody had
moved on to that API then I think maybe

3076
02:53:53,280 --> 02:53:56,317
we could have a different sort
of outcome based uh pricing but we'll see

3077
02:53:56,400 --> 02:53:59,197
how that gets adopted >> John
or the guys in the back can do we

3078
02:53:59,280 --> 02:54:03,917
have the random image can we show
the random image >> um if we can great

3079
02:54:04,000 --> 02:54:06,237
>> I'm excited the random
image >> oh here it is

3080
02:54:06,320 --> 02:54:09,437
>> okay it's just because we didn't have
a good label for it so we just called it

3081
02:54:09,520 --> 02:54:12,317
the random image because it might come
up at any point but this is a chart from

3082
02:54:12,400 --> 02:54:16,157
the Financial Times Um,
speaking of utility, where it shows

3083
02:54:16,240 --> 02:54:20,397
the amount of app releases that have
come out, um, which are skyrocketing,

3084
02:54:20,480 --> 02:54:23,517
and then apps with significant usage
that seem to be going down and app

3085
02:54:23,600 --> 02:54:26,317
reviews which seem to be going down. Um,

3086
02:54:26,400 --> 02:54:30,797
so Mike, I'd love to hear you respond to
what we're seeing in the image here. Um,

3087
02:54:30,880 --> 02:54:34,237
is it possible that like everybody's
coding and releasing, but we're not

3088
02:54:34,320 --> 02:54:36,157
really seeing a big boom in productivity?

3089
02:54:36,240 --> 02:54:39,357
>> That's really I mean, I think there's
definitely a parallel in app usage in

3090
02:54:39,440 --> 02:54:43,277
general. It'd be interesting to seeing
if any of those app released became one

3091
02:54:43,360 --> 02:54:47,197
of the apps with significant usage. Um,
>> we could take it we could take it down.

3092
02:54:47,280 --> 02:54:49,837
Yeah, go ahead. >> Um,
I think uh this ties into something

3093
02:54:49,920 --> 02:54:51,837
that I've been thinking a lot about
which obviously my background is in

3094
02:54:51,920 --> 02:54:53,197
consumer and I've been wondering what

3095
02:54:53,280 --> 02:54:55,677
the consumer AI breakouts will end up

3096
02:54:55,760 --> 02:54:59,197
being and we've seen a lot of them yet.

3097
02:54:59,280 --> 02:55:03,917
Um, and I think part of it is, you know,
I I don't know how far back that that

3098
02:55:04,000 --> 02:55:07,117
chart goes, but when we were releasing
Instagram, it still felt a little bit

3099
02:55:07,200 --> 02:55:11,277
wild west in terms of the apps. people
were excited about apps and like two

3100
02:55:11,360 --> 02:55:14,957
kind of random people released an app
and were able to get to like number one

3101
02:55:15,040 --> 02:55:18,477
in photos and video within three months,
right? I think that is much harder now

3102
02:55:18,560 --> 02:55:20,237
when you think about how consolidated

3103
02:55:20,320 --> 02:55:23,597
the top 10 is and how much time spent is

3104
02:55:23,680 --> 02:55:27,517
spent on like the Tik Toks and reals
of the world. It's a lot, right? And so I

3105
02:55:27,600 --> 02:55:30,957
think getting that breakthrough consumer
experience I think is really really

3106
02:55:31,040 --> 02:55:33,117
hard. So I think that is uh as much a

3107
02:55:33,200 --> 02:55:35,677
story about how sort of consolidated

3108
02:55:35,760 --> 02:55:38,157
consumer products are these days. number

3109
02:55:38,240 --> 02:55:41,997
one. Number two, how um entrenched or

3110
02:55:42,080 --> 02:55:44,557
how powerful it is to have uh that sort

3111
02:55:44,640 --> 02:55:49,517
of data uh people call it data gravity
like the data gravity of something like

3112
02:55:49,600 --> 02:55:52,877
your Google Docs are in your Google Docs.
So even if somebody has a like 2x

3113
02:55:52,960 --> 02:55:57,037
better AI powered, you know, doc editor,
you're going to move all your stuff

3114
02:55:57,120 --> 02:56:00,557
maybe, probably not. So I think that it
speaks to, you know, the things that are

3115
02:56:00,640 --> 02:56:03,437
sticky. I think I think about it a lot
is like the hard stuff is still hard

3116
02:56:03,520 --> 02:56:06,797
like making something people want still
really hard. you know, we have amazing

3117
02:56:06,880 --> 02:56:10,717
models internally on top of not all
of our products work, right? And so, um, I

3118
02:56:10,800 --> 02:56:14,557
think that's a bullish sign for like
product people like me because it means

3119
02:56:14,640 --> 02:56:17,357
that I think we hopefully still add value.
But I think that chart is maybe

3120
02:56:17,440 --> 02:56:19,997
another place which is like, uh, it's

3121
02:56:20,080 --> 02:56:22,477
harder in many ways than ever to uh,

3122
02:56:22,560 --> 02:56:27,037
break through even if you can code
more quickly. And could we have done

3123
02:56:27,120 --> 02:56:29,357
Instagram in, you know, a month instead

3124
02:56:29,440 --> 02:56:32,557
of three or four? Probably. But we got

3125
02:56:32,640 --> 02:56:36,797
there after like a long winding
turns and twists and turns process.

3126
02:56:36,880 --> 02:56:39,437
>> You had I think 18 people
at Instagram when you sold it.

3127
02:56:39,520 --> 02:56:41,677
>> 13. >> With these tools,
do you think you would

3128
02:56:41,760 --> 02:56:44,797
have How many people do you
think you would have had?

3129
02:56:44,880 --> 02:56:46,877
>> It's really interesting
because of those 13

3130
02:56:46,960 --> 02:56:48,077
>> give us a number
because like everyone's

3131
02:56:48,160 --> 02:56:50,957
like, "Oh, one billion$1 billion one
person startup." How close could you

3132
02:56:51,040 --> 02:56:55,117
guys have gotten? >> I think
we could have gotten there with

3133
02:56:55,200 --> 02:56:58,317
like four to six, you know? >> Okay.

3134
02:56:58,400 --> 02:57:01,437
>> Yeah. Um, or the thing that we would
have done if we'd grown it, we'd be able

3135
02:57:01,520 --> 02:57:05,117
to do things in more than a single track.
Like Instagram was, if you ever

3136
02:57:05,200 --> 02:57:08,317
watched my 5-year-old play soccer now,
by which I mean like the ball is there

3137
02:57:08,400 --> 02:57:11,437
and every single person runs to the ball.
Like that was our product team. It

3138
02:57:11,520 --> 02:57:15,517
was like video go and everybody like
goes and works on the one thing and like

3139
02:57:15,600 --> 02:57:19,757
we' be able to like play positions.
Like Android we built in about a month for

3140
02:57:19,840 --> 02:57:23,357
Instagram. We could have done it
probably in a week with the models and

3141
02:57:23,440 --> 02:57:26,797
but and to build Android we took
everybody off iOS and we all like

3142
02:57:26,880 --> 02:57:31,037
relearned to code in uh you know code
Android OS and then we went off and do

3143
02:57:31,120 --> 02:57:34,237
that and like for that whole month we
were barely shipping updates on iOS. So

3144
02:57:34,320 --> 02:57:36,317
I think you can be a lot more actually
a really good example there's a labs

3145
02:57:36,400 --> 02:57:41,437
project I have internally um that um
helps accelerate how uh anthropic

3146
02:57:41,520 --> 02:57:46,077
engineers like code and do code
review and that um that project I am

3147
02:57:46,160 --> 02:57:50,637
maintaining an iOS and an Android
version of uh and I basically have the

3148
02:57:50,720 --> 02:57:53,677
cloud that works on the iOS one
basically like ping the Android one and

3149
02:57:53,760 --> 02:57:56,557
be like hey I implemented this sorry
Android users it's still the second one

3150
02:57:56,640 --> 02:58:01,597
even in ALM world sorry um uh and then
the the Android version is like okay I'm

3151
02:58:01,680 --> 02:58:04,157
going to do this oh that doesn't count
because that feature doesn't makes sense

3152
02:58:04,240 --> 02:58:06,957
here. I'm going to drop it. And of course,
we wouldn't have been able to

3153
02:58:07,040 --> 02:58:10,797
delegate all of that on on Instagram,
but we sure could have done a lot by

3154
02:58:10,880 --> 02:58:14,637
having sort of platform parity.
Like this dream of platform like close to

3155
02:58:14,720 --> 02:58:17,357
parody is now actually quite doable.

3156
02:58:17,440 --> 02:58:19,997
>> You're probably going to get calls
now from the remaining six or seven people

3157
02:58:20,080 --> 02:58:24,237
on your Instagram team going, "Cool,
was I did I make the cut in the new era?"

3158
02:58:24,320 --> 02:58:26,877
Also, it sounds like you probably could
bring Gotham back now if you really

3159
02:58:26,960 --> 02:58:29,197
wanted to with all >> I forget
if we eventually I think for

3160
02:58:29,280 --> 02:58:31,277
April Fools, maybe we
brought it back one day. Yeah.

3161
02:58:31,360 --> 02:58:32,397
>> Yeah. Do we have time
for one more question?

3162
02:58:32,480 --> 02:58:33,597
>> Yeah. Yeah. Last question. Yeah. Okay.

3163
02:58:33,680 --> 02:58:36,157
Um I mean my last question for you is um

3164
02:58:36,240 --> 02:58:38,957
you worked on a product that now as it

3165
02:58:39,040 --> 02:58:43,917
has evolved is is um in many ways
ethically fraught because of some of the

3166
02:58:44,000 --> 02:58:46,557
harms that people are concerned about
with children. And when you talk about

3167
02:58:46,640 --> 02:58:50,237
the fact that there hasn't really been
a big breakout consumer app for AI, I

3168
02:58:50,320 --> 02:58:54,477
think there has and it's chat bots,
right? And chat bots have also led to

3169
02:58:54,560 --> 02:58:56,797
some real dangers and harms for young

3170
02:58:56,880 --> 02:58:58,557
people. And so when you are building in

3171
02:58:58,640 --> 02:59:02,557
labs, how are you thinking about the you

3172
02:59:02,640 --> 02:59:05,757
know the potential harms and the risks
that come with just making this

3173
02:59:05,840 --> 02:59:08,637
technology that much better? >> Yeah.
I mean I think there there are

3174
02:59:08,720 --> 02:59:11,757
certainly products that we have either
prototyped or conceptualized and been

3175
02:59:11,840 --> 02:59:16,157
like this product because it sounds so
hypy. I hate this, but like like this

3176
02:59:16,240 --> 02:59:20,397
product if shipped would be bad for the
world or like would nudge people in the

3177
02:59:20,480 --> 02:59:24,317
wrong direction or even if we did
it right, the like wrong or like more

3178
02:59:24,400 --> 02:59:26,077
morally fraught version of this would be

3179
02:59:26,160 --> 02:59:29,517
actively we think bad. And so I think

3180
02:59:29,600 --> 02:59:31,917
>> asking that question
a lot internally um

3181
02:59:32,000 --> 02:59:36,557
makes a difference. Um and having it's
a luxury to have core products and models

3182
02:59:36,640 --> 02:59:39,997
that are doing really really well. So
we don't like that's that in some ways an

3183
02:59:40,080 --> 02:59:43,917
easy decision even if we think it could
get a lot of um a lot of use. But yeah,

3184
02:59:44,000 --> 02:59:47,597
I think going back to an earlier
conversation, I think frontloading it is

3185
02:59:47,680 --> 02:59:51,117
really valuable and really
thinking through like

3186
02:59:51,200 --> 02:59:54,637
it is now more normalized to have people
at a company and definitely anthropic

3187
02:59:54,720 --> 02:59:56,877
does who are like economists thinking
about the impact of the thing that

3188
02:59:56,960 --> 02:59:59,917
you're building on the world and that just
was not the case in along the years

3189
03:00:00,000 --> 03:00:02,317
on on most of social media I think.

3190
03:00:02,400 --> 03:00:05,357
>> Yeah, Mike, it's always great to speak
and we thank you again for bringing your

3191
03:00:05,440 --> 03:00:08,797
insight today and um let's do it again

3192
03:00:08,880 --> 03:00:11,517
soon. Let's hear from Mike and Lauren.

3193
03:00:11,600 --> 03:00:15,117
Thank you. Great job. Thank you. All

3194
03:00:15,200 --> 03:00:20,440
right. Are you guys ready
for our last conversation of the day?

3195
03:00:20,640 --> 03:00:23,677
>> Are you having a good time?

3196
03:00:23,760 --> 03:00:27,517
>> All right. Um, so in 2015, Greg

3197
03:00:27,600 --> 03:00:29,837
Brockman, Elon Musk, and Sam Ultman

3198
03:00:29,920 --> 03:00:33,597
started a nonprofit called Open AI, and

3199
03:00:33,680 --> 03:00:35,597
the plan was to pursue artificial

3200
03:00:35,680 --> 03:00:38,637
general intelligence. the company or the

3201
03:00:38,720 --> 03:00:40,717
nonprofit when it was a nonprofit back

3202
03:00:40,800 --> 03:00:43,037
then began in Greg Brockman's living

3203
03:00:43,120 --> 03:00:46,557
room. And these folks were convinced

3204
03:00:46,640 --> 03:00:48,317
that achieving artificial general

3205
03:00:48,400 --> 03:00:53,917
intelligence or AI on par with human
intelligence was possible. And to be

3206
03:00:54,000 --> 03:00:56,317
honest, most people in the valley
thought that that was an interesting

3207
03:00:56,400 --> 03:00:57,917
side project, but most of the attention

3208
03:00:58,000 --> 03:01:00,717
was on social media at the time. Well,

3209
03:01:00,800 --> 03:01:03,517
fast forward 11 years and here we are.

3210
03:01:03,600 --> 03:01:07,117
OpenAI is probably going to go public
within the next year at a trillion

3211
03:01:07,200 --> 03:01:10,877
dollar valuation. They're going
to announce likely, you know, because

3212
03:01:10,960 --> 03:01:16,397
thirdparty data is showing it a billion
users in chatbt fairly soon. Uh and they

3213
03:01:16,480 --> 03:01:17,757
of course raised the largest venture

3214
03:01:17,840 --> 03:01:20,637
capital round in history at 122 billion.

3215
03:01:20,720 --> 03:01:23,517
So they are at the the leading edge of a

3216
03:01:23,600 --> 03:01:28,237
technology that has captured all of our
attention and is changing the world. Uh,

3217
03:01:28,320 --> 03:01:31,757
and so when you listen to Greg Brockman,
one of the things that you can see even

3218
03:01:31,840 --> 03:01:36,317
from his conversations all the way
in the past is a clear conviction and an

3219
03:01:36,400 --> 03:01:39,597
understanding of where this technology
would lead and where the products would

3220
03:01:39,680 --> 03:01:43,597
go. It's amazing. You listen to the
podcasts from pre-Chat GPT or really in

3221
03:01:43,680 --> 03:01:45,837
the early days of the GPT models. Uh,

3222
03:01:45,920 --> 03:01:48,157
and you can hear Greg speaking with

3223
03:01:48,240 --> 03:01:52,077
absolute clarity about where the models

3224
03:01:52,160 --> 03:01:54,637
and applications would go today. So I

3225
03:01:54,720 --> 03:01:59,357
think to to close our day, let's take
a look into the future of where open AI

3226
03:01:59,440 --> 03:02:01,437
and the frontier is going in a

3227
03:02:01,520 --> 03:02:06,600
conversation with Greg Brockman.
Let's welcome Greg.

3228
03:02:10,400 --> 03:02:13,357
>> Great to see you, Greg.
>> Thank you for having me.

3229
03:02:13,440 --> 03:02:15,757
Um, you know, Greg, this is our fourth

3230
03:02:15,840 --> 03:02:18,637
time speaking and we've spoken every

3231
03:02:18,720 --> 03:02:21,997
time about OpenAI's product direction

3232
03:02:22,080 --> 03:02:25,757
and I think I'm starting to get it. um

3233
03:02:25,840 --> 03:02:27,917
you know there was this conversation

3234
03:02:28,000 --> 03:02:31,437
that a super app was the wrong term for

3235
03:02:31,520 --> 03:02:35,037
what you were doing with um the app that

3236
03:02:35,120 --> 03:02:37,517
you're building bringing codeex which is

3237
03:02:37,600 --> 03:02:40,717
coding side of openi's product browser

3238
03:02:40,800 --> 03:02:45,437
and chat gpt together and when you use
the word super app people would be like

3239
03:02:45,520 --> 03:02:48,797
no a super app is actually something
that you can just use every other app

3240
03:02:48,880 --> 03:02:53,597
within um and now as we've
seen these products

3241
03:02:53,680 --> 03:02:57,357
come together actually super app might
be the correct term. You know, at least

3242
03:02:57,440 --> 03:03:01,037
for us on the outside, we're starting
to see it that when you need to do

3243
03:03:01,120 --> 03:03:03,357
anything, um you it will start with a

3244
03:03:03,440 --> 03:03:06,637
prompt in chat GPT and then OpenAI's

3245
03:03:06,720 --> 03:03:11,277
technology will use either your browser
or your computer to get that done for

3246
03:03:11,360 --> 03:03:13,677
you. Is that the right
way to think about it?

3247
03:03:13,760 --> 03:03:16,797
>> I I think that's a pretty good
perspective, right? And I think to

3248
03:03:16,880 --> 03:03:21,837
really zoom out, the thing we're
actually trying to build is in AGI,

3249
03:03:21,920 --> 03:03:25,357
right? that if you think about what
people have been using since chat GBT

3250
03:03:25,440 --> 03:03:29,437
it's a language model right
there's a big gap between these

3251
03:03:29,520 --> 03:03:34,557
it's amazing you can talk to chat talks
back to you great wonderful but when we

3252
03:03:34,640 --> 03:03:37,917
launched in 2022 there was no memory
right it's not hooked up to any tools

3253
03:03:38,000 --> 03:03:40,557
has no context and so it really is that

3254
03:03:40,640 --> 03:03:45,517
this conversational intelligence is only
one part of what people really need to

3255
03:03:45,600 --> 03:03:49,037
get work done to be able to achieve
their goals and where we're going is to

3256
03:03:49,120 --> 03:03:52,557
have an AI that's really looking out
for you right that who can provide the

3257
03:03:52,640 --> 03:03:56,317
goals, the directions, that it's
constantly thinking about what can I do

3258
03:03:56,400 --> 03:04:00,237
for Alex today. Uh that it's able to go
and solve super hard problems, very

3259
03:04:00,320 --> 03:04:03,197
mundane problems. You wake up, your inbox
is organized, but also if there's

3260
03:04:03,280 --> 03:04:07,837
like a health plan that you are thinking
about that it can help you help you

3261
03:04:07,920 --> 03:04:11,037
achieve that, figure out medical
treatments, uh or you know, sort of back

3262
03:04:11,120 --> 03:04:15,037
and forth provide you with that kind
of information at least. And I think that

3263
03:04:15,120 --> 03:04:18,397
the question of what's the interface you
want, what is the product that you want

3264
03:04:18,480 --> 03:04:21,837
is what we spend a lot of time thinking
about. The answer is you want almost no

3265
03:04:21,920 --> 03:04:23,757
interface. You want no product, right?

3266
03:04:23,840 --> 03:04:26,637
You want this to be like what's
the interface between you and me, right?

3267
03:04:26,720 --> 03:04:29,437
Just being able to talk to a persistent

3268
03:04:29,520 --> 03:04:33,997
entity of some form that's able to go
and accomplish goals for you. And so

3269
03:04:34,080 --> 03:04:37,997
building that is hard. It will take time,
but we have a lot of the pieces,

3270
03:04:38,080 --> 03:04:41,037
right? We're increasingly bringing
together the product layer, trying to

3271
03:04:41,120 --> 03:04:45,277
make the models better, trying to make
the whole system just so there's less

3272
03:04:45,360 --> 03:04:50,157
like clicking buttons and toggles and
changing modes and all these things. Not

3273
03:04:50,240 --> 03:04:53,197
to say that there won't be some
of those along the way, but the long-term

3274
03:04:53,280 --> 03:04:55,997
trajectory is towards
simplification, unification.

3275
03:04:56,080 --> 03:04:57,277
>> Yeah, it's very
interesting that you say

3276
03:04:57,360 --> 03:05:00,397
the interface uh will melt away. And so

3277
03:05:00,480 --> 03:05:04,317
to go a little bit deeper with my
question, um many of us who use products

3278
03:05:04,400 --> 03:05:08,797
like chatbt today, we'll see that the
bot will make a suggestion at the end.

3279
03:05:08,880 --> 03:05:11,997
You know, you ask it about nutrition and
it says should I make a health plan for

3280
03:05:12,080 --> 03:05:15,997
you or make a diet plan for you? You ask
it, sorry Ron John, we just talked about

3281
03:05:16,080 --> 03:05:17,597
travel, but you ask it about travel and

3282
03:05:17,680 --> 03:05:19,197
then it will give you an agenda for

3283
03:05:19,280 --> 03:05:23,037
instance. And so am I hearing you right

3284
03:05:23,120 --> 03:05:27,677
that what's going to happen within chat
GPT just to give an example is you talk

3285
03:05:27,760 --> 03:05:32,637
to it about your health decisions and it
might say you know you probably need to

3286
03:05:32,720 --> 03:05:36,157
you know go to this specialist let me
make an appointment for you and then it

3287
03:05:36,240 --> 03:05:40,477
will go and actually take that action
on your behalf. So it goes from simply a

3288
03:05:40,560 --> 03:05:44,157
conversation interface to actually
understanding your intent and then going

3289
03:05:44,240 --> 03:05:48,157
out and accomplishing that for you. that
that I that's exactly right and I think

3290
03:05:48,240 --> 03:05:51,037
that if you've used codecs and by the
way how many people in the room have

3291
03:05:51,120 --> 03:05:53,997
used have used codecs >> and people

3292
03:05:54,080 --> 03:05:55,997
>> yeah decent number
of people um and that

3293
03:05:56,080 --> 03:05:58,637
our goal is to really bring the power of

3294
03:05:58,720 --> 03:06:00,797
codecs to everyone right to bring agents

3295
03:06:00,880 --> 03:06:03,437
to everyone you that technology exists

3296
03:06:03,520 --> 03:06:05,677
right now right you can hook up like I

3297
03:06:05,760 --> 03:06:08,957
hook up my codeex to slack to my Gmail

3298
03:06:09,040 --> 03:06:13,117
to my calendar and there are many people
within openai nontechnical users you

3299
03:06:13,200 --> 03:06:16,477
know it's got code in the name but it's
not really about code. It's really about

3300
03:06:16,560 --> 03:06:21,597
having this general purpose tool using
harness an agent and that the kinds of

3301
03:06:21,680 --> 03:06:25,757
things for example someone on our comm's
team does is um she was organizing an

3302
03:06:25,840 --> 03:06:28,477
event and it would just ask all of the

3303
03:06:28,560 --> 03:06:30,797
uh event attendees for their dietary

3304
03:06:30,880 --> 03:06:36,317
preferences set up a whole seating chart
kind of did all of that work uh so that

3305
03:06:36,400 --> 03:06:39,437
she could focus on the parts that she
wanted to and and really thinking about

3306
03:06:39,520 --> 03:06:42,637
the vision of what she wanted to achieve
and I think that we're going to see this

3307
03:06:42,720 --> 03:06:46,797
across the board. So the it's not sci-fi
anymore to think about an AI that's

3308
03:06:46,880 --> 03:06:49,597
hooked up to these tools. And I remember

3309
03:06:49,680 --> 03:06:51,517
with you know our very first attempt at

3310
03:06:51,600 --> 03:06:55,197
tool use in chatbt was 2023. I think in

3311
03:06:55,280 --> 03:06:59,277
like March or April or something we
released plugins. Do people remember

3312
03:06:59,360 --> 03:07:01,677
plugins back in early chat days?

3313
03:07:01,760 --> 03:07:05,677
>> That didn't work. >> It didn't
work at all because the models

3314
03:07:05,760 --> 03:07:08,877
the models weren't ready. Right. The form
factor is correct. Obviously you're

3315
03:07:08,960 --> 03:07:12,637
going to have an AI that's able to like
talk to your Gmail like no question. But

3316
03:07:12,720 --> 03:07:16,157
we could only like have three different
connectors exposed to the model at a

3317
03:07:16,240 --> 03:07:20,397
time or start forgetting. You know, we
had like 2K maybe 4K token context. Like

3318
03:07:20,480 --> 03:07:23,597
there's just no memory, right? It's kind
of like when you had early computers in

3319
03:07:23,680 --> 03:07:27,357
the 60s or 70s or something, right? You
know, that you had tiny little me memory

3320
03:07:27,440 --> 03:07:30,477
banks and today you have your phone
that's like better than any

3321
03:07:30,560 --> 03:07:33,357
supercomputer from that era. And I
think that's where we're going with these

3322
03:07:33,440 --> 03:07:36,157
models, right? That the rate
of improvement has been so steep. So now

3323
03:07:36,240 --> 03:07:39,997
you can have hundreds of different tools
accessible that we have the ability to

3324
03:07:40,080 --> 03:07:43,197
hook them up to whole file system.
So you can almost have the full power of

3325
03:07:43,280 --> 03:07:46,637
the internet and like almost any
application you want at the model's

3326
03:07:46,720 --> 03:07:49,117
fingertips. And it's smart, right? It's

3327
03:07:49,200 --> 03:07:52,957
got 52 million token context, depends

3328
03:07:53,040 --> 03:07:55,837
how you squint on it. And the capability

3329
03:07:55,920 --> 03:08:00,237
level is also getting so so powerful,
right? These models are now solving

3330
03:08:00,320 --> 03:08:03,037
unsolved math problems and physics
problems, right? And really helping

3331
03:08:03,120 --> 03:08:06,317
people be able to achieve things they
couldn't otherwise. like we are on the

3332
03:08:06,400 --> 03:08:11,357
era on the edge of this era of agents
really transforming how we all operate

3333
03:08:11,440 --> 03:08:15,597
whether it's in software engineering
finance legal sales and in our personal

3334
03:08:15,680 --> 03:08:19,677
lives too. So just to unpack that
example that you were giving there one

3335
03:08:19,760 --> 03:08:24,317
of your colleagues is chatting with Chad
GPT about an event and then suggest hey

3336
03:08:24,400 --> 03:08:28,797
you know uh how should we you know
contact event attendees about something

3337
03:08:28,880 --> 03:08:32,157
and instead of like saying okay I
have to do that and going into an event

3338
03:08:32,240 --> 03:08:37,197
program basically what happens is the
the interface will take over from there

3339
03:08:37,280 --> 03:08:41,117
once it says it's a good idea and you
agree and then hook into whatever tools

3340
03:08:41,200 --> 03:08:42,797
you're using and then do it for you.

3341
03:08:42,880 --> 03:08:47,597
>> Exactly. So it uses its you know Gmail
connector searches through your inbox to

3342
03:08:47,680 --> 03:08:49,757
find all the people who are attending

3343
03:08:49,840 --> 03:08:53,037
and then if you're on the like what what

3344
03:08:53,120 --> 03:08:57,357
is in dietary restriction sees oh these
people I already have their dietary

3345
03:08:57,440 --> 03:08:59,517
restrictions these people I do not um

3346
03:08:59,600 --> 03:09:02,397
drafts an email depending on exactly how

3347
03:09:02,480 --> 03:09:07,677
you have things set up it might say hey
I drafted these emails can I send them

3348
03:09:07,760 --> 03:09:10,477
if you have a connector that doesn't
even let it send emails said I drafted

3349
03:09:10,560 --> 03:09:15,197
it you need to send them And uh in a
different world, you could also imagine

3350
03:09:15,280 --> 03:09:18,397
that it's you've built enough trust with
the system where it says I drafted the

3351
03:09:18,480 --> 03:09:23,037
emails and I actually sent them.
And I think that this actually points to a

3352
03:09:23,120 --> 03:09:27,117
really important aspect of the agentic
era, which is trust, right? That we need

3353
03:09:27,200 --> 03:09:30,397
to really learn how to build trust with
these systems, where they're good, where

3354
03:09:30,480 --> 03:09:34,957
they're not. Figure out what you want
to delegate to them and how you want to

3355
03:09:35,040 --> 03:09:38,077
entrust them with responsibility.
And that's something we view as earned,

3356
03:09:38,160 --> 03:09:41,437
right? It's not something that that we
can we can grant but by providing lots

3357
03:09:41,520 --> 03:09:44,077
of tools and control and oversight and

3358
03:09:44,160 --> 03:09:48,477
supervision to the operator to the person
who that this AI is operating on

3359
03:09:48,560 --> 03:09:51,837
behalf of like we think that that is
going to be such an important thing. So

3360
03:09:51,920 --> 03:09:54,477
that's a key product
feature and differentiator.

3361
03:09:54,560 --> 03:09:57,677
>> Yeah. And when you go back to some
of the early attempts at this, there was

3362
03:09:57,760 --> 03:10:02,797
this like move that OpenAI had to let
you call an Uber within Chat GPT. And it

3363
03:10:02,880 --> 03:10:05,997
it followed a long line of companies
that have tried to get you to take

3364
03:10:06,080 --> 03:10:08,477
action within chat, but it never really

3365
03:10:08,560 --> 03:10:11,517
took off. And the difference here might

3366
03:10:11,600 --> 03:10:14,797
be that the chatbot can take control of

3367
03:10:14,880 --> 03:10:19,437
your browser or take control of your
computer and then you don't necessarily

3368
03:10:19,520 --> 03:10:23,277
have to worry about like is this plug-in
going to work? it goes and accomplishes

3369
03:10:23,360 --> 03:10:25,437
that for you by taking over your

3370
03:10:25,520 --> 03:10:28,557
machine. So I wonder you know if you

3371
03:10:28,640 --> 03:10:31,677
expect a fight from the user interfaces

3372
03:10:31,760 --> 03:10:34,077
that we have today aka like all the

3373
03:10:34,160 --> 03:10:36,637
other apps all the software where to be

3374
03:10:36,720 --> 03:10:39,837
truly useful chat GPT will have to not

3375
03:10:39,920 --> 03:10:43,837
be blocked to be able to go out
and execute these actions on behalf of a

3376
03:10:43,920 --> 03:10:48,557
user. Well, look f first of all, I'd
say that it's this is not theoretical at

3377
03:10:48,640 --> 03:10:51,277
this point, right? That people have
been using Kodak. So, it's a separate

3378
03:10:51,360 --> 03:10:54,157
product, separate app, you have
to install it separately. Really starting

3379
03:10:54,240 --> 03:10:59,037
to focus on software engineering. But
the amount of non-software work that has

3380
03:10:59,120 --> 03:11:01,917
been happening in Codex has been
absolutely exploding, right? It's been

3381
03:11:02,000 --> 03:11:06,077
this like incredible exponential curve,
exactly the thing that you you would

3382
03:11:06,160 --> 03:11:09,677
expect. And within OpenAI, we basically
have the same level of penetration now

3383
03:11:09,760 --> 03:11:13,197
in usage as Slack, right? It's like
everyone and OpenAI is like an entirely

3384
03:11:13,280 --> 03:11:16,477
Slack based company. We do not use email
for the most part. It's like really like

3385
03:11:16,560 --> 03:11:19,197
if you're not on Slack, you're not going
to do any work. Um, and it's kind of

3386
03:11:19,280 --> 03:11:23,357
feeling that way now with Codeex app
as well. And that everyone's Codeex is

3387
03:11:23,440 --> 03:11:28,317
hooked up to all of these tools. Um, how
the EOS ecosystem evolves. I think it's

3388
03:11:28,400 --> 03:11:31,677
going to be a very nuanced thing because
I think one thing that is very important

3389
03:11:31,760 --> 03:11:36,557
is that we believe that there should be
an ecosystem that gets to be vibrant and

3390
03:11:36,640 --> 03:11:40,797
thriving and that people can really
build and see the benefits. And so we've

3391
03:11:40,880 --> 03:11:44,797
actually seen this from uh partner
companies where uh you know that we I

3392
03:11:44,880 --> 03:11:47,597
remember there's a couple different
partners where we said hey we really

3393
03:11:47,680 --> 03:11:50,557
want to train our AI to be really good
at using your software and we didn't

3394
03:11:50,640 --> 03:11:53,197
know what they would say and actually

3395
03:11:53,280 --> 03:11:57,677
the response we got is this is the most
partner friendly outreach we've ever had

3396
03:11:57,760 --> 03:12:01,917
right that the idea that you will make
your AI specifically good at using our

3397
03:12:02,000 --> 03:12:06,077
tool and they just see the opportunity
because their tool will be used just so

3398
03:12:06,160 --> 03:12:10,797
much more as a result and that everyone
is trying to think about how do they not

3399
03:12:10,880 --> 03:12:14,877
just survive as a company into the AI
era but thrive? like how do you really

3400
03:12:14,960 --> 03:12:18,557
get the advantages of the fact there's
going to be so much more activity and if

3401
03:12:18,640 --> 03:12:22,557
you don't have AI in there if you shut
it out then you're actually going to be

3402
03:12:22,640 --> 03:12:27,357
declining not thriving right this this
kind of makes open AI puts open AI so

3403
03:12:27,440 --> 03:12:31,917
first of all you're going to bring you
talk about people using codeex so one of

3404
03:12:32,000 --> 03:12:35,437
your colleagues shared and I think
you've talked about this too that you've

3405
03:12:35,520 --> 03:12:37,837
brought chat GPT into codeex so you can

3406
03:12:37,920 --> 03:12:42,477
bring codeex into chat GPT which
is basically like if we're users of chat

3407
03:12:42,560 --> 03:12:45,037
GPT this experience
that we talked about of

3408
03:12:45,120 --> 03:12:49,437
chatt not only suggesting what you might
want to do next but going to do it for

3409
03:12:49,520 --> 03:12:53,197
you that's going to happen and so
it makes you effectively an operating

3410
03:12:53,280 --> 03:12:57,757
system don't you think but not
the operating system like an iOS where you

3411
03:12:57,840 --> 03:13:02,477
would like go open up the your phone
and then tap different apps it's almost as

3412
03:13:02,560 --> 03:13:06,637
if all interaction with all apps will
happen through this interface is that

3413
03:13:06,720 --> 03:13:11,277
the ambition >> I think
that you could describe it that

3414
03:13:11,360 --> 03:13:13,517
way but I think of it a differently.

3415
03:13:13,600 --> 03:13:15,357
Like the way that I think about this is

3416
03:13:15,440 --> 03:13:18,157
that what is the ideal interface to an

3417
03:13:18,240 --> 03:13:20,957
AGI or we call it kind of a personal AGI

3418
03:13:21,040 --> 03:13:25,437
and I think that it's again the same
interface that you and I are using right

3419
03:13:25,520 --> 03:13:29,277
now you just want to talk to an
assistant right you want to talk to

3420
03:13:29,360 --> 03:13:31,117
something that can go and and and work

3421
03:13:31,200 --> 03:13:34,077
and and operate on your behalf and so

3422
03:13:34,160 --> 03:13:37,597
that yes like that agent that AGI that

3423
03:13:37,680 --> 03:13:41,597
AI will have its own computer right it
will have its own access to things that

3424
03:13:41,680 --> 03:13:45,037
maybe And you know like ideal co-orker
would be they can come over and type

3425
03:13:45,120 --> 03:13:47,197
things on your computer too. So some

3426
03:13:47,280 --> 03:13:50,557
access some delegated access to your own

3427
03:13:50,640 --> 03:13:55,597
own system and you know maybe you
delegate access to your inbox sometimes

3428
03:13:55,680 --> 03:13:59,197
maybe it has its own inbox with some
sort of uh some sort of you know window

3429
03:13:59,280 --> 03:14:04,157
into into the things that it needs.
You forward emails to it. Th these are not

3430
03:14:04,240 --> 03:14:07,277
actually if you if you think about this
is not unprecedented, right? It's like

3431
03:14:07,360 --> 03:14:10,557
the way that you work with an assistant

3432
03:14:10,640 --> 03:14:14,797
who's a person that we've we've actually
or any co-orker really, we've spent a

3433
03:14:14,880 --> 03:14:18,477
lot of time really thinking about how
do you build these trust boundaries and

3434
03:14:18,560 --> 03:14:22,237
make sure that you're able to operate
together. And so I think of it as just a

3435
03:14:22,320 --> 03:14:25,597
different thing. It's not it you could
think of it as an operating system, but

3436
03:14:25,680 --> 03:14:29,357
an operating system is almost something
from a different time, right? It's a

3437
03:14:29,440 --> 03:14:33,197
different layer of the stack. This is
really more about how do you interface

3438
03:14:33,280 --> 03:14:35,357
with technology broadly. And I think

3439
03:14:35,440 --> 03:14:37,917
that the beautiful thing about AI is

3440
03:14:38,000 --> 03:14:41,037
it's really about bringing the machine

3441
03:14:41,120 --> 03:14:45,597
closer to the human rather than us
having to contort ourselves into like

3442
03:14:45,680 --> 03:14:49,837
files and folders and like all these
details that somehow are not natural,

3443
03:14:49,920 --> 03:14:52,797
right? That are more about how the machine
operates rather than how we operate.

3444
03:14:52,880 --> 03:14:56,557
>> Yeah. Talking about a personal
intelligence, it sort of um I don't did

3445
03:14:56,640 --> 03:14:58,317
you watch WWDC last week?

3446
03:14:58,400 --> 03:15:00,317
>> Uh no, no, I missed it. I was I was

3447
03:15:00,400 --> 03:15:04,477
banned, but um I watched it on TV. Um

3448
03:15:04,560 --> 03:15:09,117
come on, Apple. Anyway, it does look
like you and and and Siri, the new Siri

3449
03:15:09,200 --> 03:15:12,797
are going to come kind of into
competition, right? Because they're an

3450
03:15:12,880 --> 03:15:15,917
app that's going to sit or an intelligence
that will sit on top of all

3451
03:15:16,000 --> 03:15:18,397
of your apps and let you take action.

3452
03:15:18,480 --> 03:15:22,237
And Chacht will be an app on the iPhone.

3453
03:15:22,320 --> 03:15:26,237
So then talk a little bit about
whether that positioning is going to be

3454
03:15:26,320 --> 03:15:29,597
difficult for open AI and how you're
thinking about that strategically.

3455
03:15:29,680 --> 03:15:32,957
>> Well, I just think again think of it
a little differently. Like I think that

3456
03:15:33,040 --> 03:15:38,077
we're in the beginning of this new
agentic era and the way that this has

3457
03:15:38,160 --> 03:15:42,157
always gone in AI is that when you have
a new level of capability, it means you

3458
03:15:42,240 --> 03:15:45,917
have an opportunity to rethink
everything, right? Rethink how

3459
03:15:46,000 --> 03:15:48,477
>> people interface, how like what the

3460
03:15:48,560 --> 03:15:53,517
techn is capable of. And I think that
this is no no different, right? In my

3461
03:15:53,600 --> 03:15:55,357
mind, like the kinds of things that I

3462
03:15:55,440 --> 03:15:57,357
see on the horizon,

3463
03:15:57,440 --> 03:16:01,037
for example, AI for solving

3464
03:16:01,120 --> 03:16:04,717
scientific problems, right? And I think
we're starting to see the inklings of

3465
03:16:04,800 --> 03:16:08,637
this. Like for example, today we announced

3466
03:16:08,720 --> 03:16:13,517
we have in in uh peer-reviewed
literature people doctors who are using

3467
03:16:13,600 --> 03:16:17,757
03. Remember 03? >> Yep.
>> That was like forever ago now, right?

3468
03:16:17,840 --> 03:16:19,037
That was like one of our earliest

3469
03:16:19,120 --> 03:16:22,557
reasoning models using that to find

3470
03:16:22,640 --> 03:16:27,997
diagnosis for people who I had no
no answers from doctors for many many

3471
03:16:28,080 --> 03:16:31,277
years. You know, there's an example
of someone who had spent 20 years with a

3472
03:16:31,360 --> 03:16:34,957
mysterious ailment. Finally, it's
been diagnosed through the use of this

3473
03:16:35,040 --> 03:16:38,477
technology. And if you're like, okay,
you've got models that can do that. They

3474
03:16:38,560 --> 03:16:41,757
can do that. And then it's really
about like, you know, the same like

3475
03:16:41,840 --> 03:16:45,037
distribution and like, you know, can
you get access to an app? You know, to me

3476
03:16:45,120 --> 03:16:47,997
it's it doesn't type check. It's like we
have something fundamentally new. And so

3477
03:16:48,080 --> 03:16:50,717
that's not to say that there won't be
competition. I actually think that there

3478
03:16:50,800 --> 03:16:53,757
will be and it's going to be great for
everyone. But I just think that the ways

3479
03:16:53,840 --> 03:16:56,237
in which you're going to use this
technology, the things it will be

3480
03:16:56,320 --> 03:16:59,757
capable of and what it'll make you
capable of doing are just totally

3481
03:16:59,840 --> 03:17:01,597
different from anything we've seen before.

3482
03:17:01,680 --> 03:17:05,357
>> You know, I was going to ask you um
well, does it mean that you'll have to

3483
03:17:05,440 --> 03:17:09,917
um you know, create your own device
assuming that like my concept is is you

3484
03:17:10,000 --> 03:17:13,277
know that you're going to have to go
through Apple to get to the user. Um

3485
03:17:13,360 --> 03:17:17,517
assuming that's somewhat valid. But the
answer is you already are right. You're

3486
03:17:17,600 --> 03:17:19,837
so OpenAI is working
on a device right now.

3487
03:17:19,920 --> 03:17:23,757
>> It certainly has
been publicly reported.

3488
03:17:23,840 --> 03:17:26,237
>> I I was in your office in December and

3489
03:17:26,320 --> 03:17:30,797
Sam told me that this is happening.
It's it's multiple devices. Um so if you

3490
03:17:30,880 --> 03:17:35,277
think about the way that again you're
going to interface with um with these

3491
03:17:35,360 --> 03:17:40,637
AIs, how does that device play in or
series of devices? Well, look, I I think

3492
03:17:40,720 --> 03:17:42,077
again I I would just step back and say

3493
03:17:42,160 --> 03:17:43,837
that I think this is the beginning of

3494
03:17:43,920 --> 03:17:48,157
something very new and that I think

3495
03:17:48,240 --> 03:17:52,237
about the way that

3496
03:17:52,320 --> 03:17:55,597
I think just say like I think the biggest
shift that has happened in terms

3497
03:17:55,680 --> 03:17:58,397
of interface again it's not even about

3498
03:17:58,480 --> 03:18:00,397
devices and and and things like that.

3499
03:18:00,480 --> 03:18:04,157
really about the shift from conversational
intelligence like kind of

3500
03:18:04,240 --> 03:18:08,157
the chat paradigm where it's like kind
of you have an AI that's personalized

3501
03:18:08,240 --> 03:18:11,997
enough to you that it's worth reading
its output right you ask it a question

3502
03:18:12,080 --> 03:18:16,557
you get an answer it's something that's
useful to you to agents where they're

3503
03:18:16,640 --> 03:18:21,837
capable enough to actually do things
for you like that is a big shift and that

3504
03:18:21,920 --> 03:18:26,317
that implies a difference
in how you want to

3505
03:18:26,400 --> 03:18:27,917
interact and so you kind of are just

3506
03:18:28,000 --> 03:18:30,637
going to want a single agent that has

3507
03:18:30,720 --> 03:18:33,917
access to your context and this will be

3508
03:18:34,000 --> 03:18:35,757
true in personal life. This will be true

3509
03:18:35,840 --> 03:18:37,677
in a business context, right? You

3510
03:18:37,760 --> 03:18:41,277
imagine for example having a uh you know

3511
03:18:41,360 --> 03:18:43,517
imagine you have a PhD in every field

3512
03:18:43,600 --> 03:18:48,397
co-orker you know Nobel prizes multiple
of them and you hire one of these you

3513
03:18:48,480 --> 03:18:51,837
hire a hundred of them and you don't
invite them to any meetings they're not

3514
03:18:51,920 --> 03:18:53,997
going to be very useful and so there's

3515
03:18:54,080 --> 03:18:56,077
something about how do you get context

3516
03:18:56,160 --> 03:19:00,237
into the AI and not just statically but

3517
03:19:00,320 --> 03:19:03,437
dynamically right as context evolves as

3518
03:19:03,520 --> 03:19:07,597
your business processes evolve how
do you have a context layer that is

3519
03:19:07,680 --> 03:19:11,197
accessible to an AI that lets the AI
operate to the extent of that raw

3520
03:19:11,280 --> 03:19:14,477
intelligence. And so finding ways to

3521
03:19:14,560 --> 03:19:19,837
make that AI be accessible so available
in your meetings to make it very

3522
03:19:19,920 --> 03:19:23,997
ergonomics. It's very easy to get access
to. Um I think all of that's going to

3523
03:19:24,080 --> 03:19:25,837
require a rethink. But I think again

3524
03:19:25,920 --> 03:19:29,357
it's just the core for me starts from

3525
03:19:29,440 --> 03:19:33,757
thinking about the agentic form factor
and then working backwards to how do you

3526
03:19:33,840 --> 03:19:37,677
just make this have the context it needs.
And again, the trust is going to

3527
03:19:37,760 --> 03:19:41,117
be such a core part of making
this whole equation work.

3528
03:19:41,200 --> 03:19:43,597
>> So, kind of like
having this this device

3529
03:19:43,680 --> 03:19:46,797
with you at all times and being like,
I need to get that done and it goes and

3530
03:19:46,880 --> 03:19:49,677
does it for you. >> And I
think that that will be part of

3531
03:19:49,760 --> 03:19:51,917
it, but I almost even think if you don't

3532
03:19:52,000 --> 03:19:56,717
have a device like that, it's not like
you're going to be out of the game,

3533
03:19:56,800 --> 03:19:59,677
right? Because it's this a this AI.
It's not because there's one thing there's

3534
03:19:59,760 --> 03:20:02,637
one version of it where you think of it
where it's like the device is the AI and

3535
03:20:02,720 --> 03:20:05,277
you want your phone to be the AI. you
want that, you know, whatever whatever,

3536
03:20:05,360 --> 03:20:07,917
you know, custom device you're you're
thinking about to be the AI, but it's

3537
03:20:08,000 --> 03:20:10,797
not going to be like that. It's going
to be more like an interface, like no more

3538
03:20:10,880 --> 03:20:14,397
than your phone is you, right? It's
an interface to you. It's a way that I can

3539
03:20:14,480 --> 03:20:17,357
sort of, you know, call you up whenever
I need you, uh, whenever I want to ask

3540
03:20:17,440 --> 03:20:20,877
you a question. And there's different
ways of accessing, right? There's like

3541
03:20:20,960 --> 03:20:24,397
synchronous phone call, I can text you,
I can email you. And I think that we're

3542
03:20:24,480 --> 03:20:27,357
going to be much the same with how
we interact with our agents.

3543
03:20:27,440 --> 03:20:30,317
>> Um, there's been some reports
that OpenAI is working on these like

3544
03:20:30,400 --> 03:20:34,317
birectional voice models. I think we've
talked about that in the past that like

3545
03:20:34,400 --> 03:20:37,037
the goal is to have like an AI that you

3546
03:20:37,120 --> 03:20:41,437
can you can speak with and we'll be able
to process that and speak back with you

3547
03:20:41,520 --> 03:20:43,517
in a much more natural way. Can you

3548
03:20:43,600 --> 03:20:45,597
share anything about that?

3549
03:20:45,680 --> 03:20:49,037
>> No. But no, more seriously, um I mean

3550
03:20:49,120 --> 03:20:52,717
look, I think that the that the general
shape of the of the technology like the

3551
03:20:52,800 --> 03:20:57,277
way that like we had we had we've
had voice models um you know kind of a

3552
03:20:57,360 --> 03:21:01,837
really cool voice experience for you
know year and a half two years now. Um,

3553
03:21:01,920 --> 03:21:03,997
you know, we first demoed it back in

3554
03:21:04,080 --> 03:21:06,957
March, April of 2024. Um, brought it to

3555
03:21:07,040 --> 03:21:11,597
market, um, you know, maybe late
that late that year. And the way that it

3556
03:21:11,680 --> 03:21:15,117
works and the way that everyone's
models work is that you basically chain

3557
03:21:15,200 --> 03:21:18,797
together, um, well, the original way
that these things worked was that you

3558
03:21:18,880 --> 03:21:21,677
would chain together a text to or a

3559
03:21:21,760 --> 03:21:25,837
speechtoext model, then you do
a texttoext model, and then you would do a

3560
03:21:25,920 --> 03:21:27,757
texttospech model. Horribleness, right?

3561
03:21:27,840 --> 03:21:31,357
Like these three things chained together.
Um it still has been the case

3562
03:21:31,440 --> 03:21:35,357
that even if you have one unified model
that's able to kind of take in input and

3563
03:21:35,440 --> 03:21:38,797
then you know able to output a response
you still have this problem of turn

3564
03:21:38,880 --> 03:21:42,637
taking right imagine that like we have
this like you cannot overlap you cannot

3565
03:21:42,720 --> 03:21:47,117
interrupt it's just like once you you
you speak to me in a turn and then you

3566
03:21:47,200 --> 03:21:50,317
got to wait for me to finish my whole
response. That is not how human

3567
03:21:50,400 --> 03:21:54,797
conversation works. And so that we we
basically have like a hack where we have

3568
03:21:54,880 --> 03:21:58,717
these models that determine oh it seems
like the turn has ended and oh it seems

3569
03:21:58,800 --> 03:22:01,997
like the turn has started and we're
like why are we talking about turns right

3570
03:22:02,080 --> 03:22:05,757
like turns again are so unnatural.
This is the humans contorting ourselves to

3571
03:22:05,840 --> 03:22:09,277
the machine and its limitations.
And so the obvious thing that you want to

3572
03:22:09,360 --> 03:22:13,757
accomplish is a model in AI that works
much more like you and I do right that's

3573
03:22:13,840 --> 03:22:18,797
able to process input at the same time
it's processing output. And all of that

3574
03:22:18,880 --> 03:22:21,597
is of course something that many people
in this field are trying to to run

3575
03:22:21,680 --> 03:22:23,277
towards. Um I think it's going to be

3576
03:22:23,360 --> 03:22:25,197
very very exciting as you move to these

3577
03:22:25,280 --> 03:22:30,317
natural very human fluid like
conversational interfaces. No one's seen

3578
03:22:30,400 --> 03:22:34,077
anything like it. Like one thing that
that I think about is the the current

3579
03:22:34,160 --> 03:22:37,357
interaction with you know chat GBT voice.
In many ways it's magical right?

3580
03:22:37,440 --> 03:22:41,517
So many people use it on their commute
able to ask all these questions. But it

3581
03:22:41,600 --> 03:22:44,557
also is so frustrating, right? Whenever
it breaks the magic because it's like

3582
03:22:44,640 --> 03:22:48,557
you realize, oh, I want to like add some
followup and it keeps talking over you

3583
03:22:48,640 --> 03:22:53,277
and it didn't it's just like that is
just it doesn't make sense. And so I

3584
03:22:53,360 --> 03:22:57,757
think that part of what we need part of
like the whole point of this AI is to be

3585
03:22:57,840 --> 03:23:01,357
something that you can interact with
interact with fluidly and naturally. And

3586
03:23:01,440 --> 03:23:02,717
by the way, I think it's not just going

3587
03:23:02,800 --> 03:23:07,197
to be about the sort of use case. Like

3588
03:23:07,280 --> 03:23:10,157
we kind of think about the the personal
use case, but it's also really the work

3589
03:23:10,240 --> 03:23:13,597
use case. And I think some of the most
magical experiences that that I've had

3590
03:23:13,680 --> 03:23:18,077
with codecs have been when operating it
through voice. Like many people, we have

3591
03:23:18,160 --> 03:23:20,077
a voice built in. Some people use third

3592
03:23:20,160 --> 03:23:25,117
party apps for it. Um, and that you just
get a very different experience when you

3593
03:23:25,200 --> 03:23:28,877
start to realize that like typing
a quick message to give some feedback

3594
03:23:28,960 --> 03:23:32,397
easy. but like writing out a whole
paragraph and everything you want.

3595
03:23:32,480 --> 03:23:35,277
Horrible. No one wants to do that,
right? You just want to be like saying

3596
03:23:35,360 --> 03:23:38,637
things and you want the real-time
feedback loop and all of that is going

3597
03:23:38,720 --> 03:23:40,797
to happen and it's going to be amazing.

3598
03:23:40,880 --> 03:23:45,517
So, let's talk about model improvement
briefly. Um, so there was a discussion a

3599
03:23:45,600 --> 03:23:47,517
couple years ago that large language

3600
03:23:47,600 --> 03:23:49,597
models were about to hit a wall. Um,

3601
03:23:49,680 --> 03:23:53,437
that was wrong. And um, something you

3602
03:23:53,520 --> 03:23:57,277
know that I'm thinking about is I think
we're all thinking about it is how much

3603
03:23:57,360 --> 03:23:59,437
better can these models get? And when

3604
03:23:59,520 --> 03:24:03,037
will the improvement stop? Any thoughts?

3605
03:24:03,120 --> 03:24:06,957
>> Well, I think that
this is a place where

3606
03:24:07,040 --> 03:24:10,957
when you're kind of building these
models, you get kind of a sense and an

3607
03:24:11,040 --> 03:24:15,917
intuition that I think is harder to get
from the outside because we see all the

3608
03:24:16,000 --> 03:24:19,357
data points and we see also the work
that goes into these improvements. And

3609
03:24:19,440 --> 03:24:21,277
so that there's two parts to the answer.

3610
03:24:21,360 --> 03:24:26,600
One is I think that the
fundamental science is

3611
03:24:35,280 --> 03:24:40,157
one of the most mysterious and important

3612
03:24:40,240 --> 03:24:44,397
just scientific discoveries
and empirical observations that that I can

3613
03:24:44,480 --> 03:24:47,677
that I that I'm aware of that I can
imagine right that we are able to

3614
03:24:47,760 --> 03:24:50,157
actually build these models and that the

3615
03:24:50,240 --> 03:24:55,037
scaling laws continue right that it
just is the case that you can just keep

3616
03:24:55,120 --> 03:24:58,557
training these models more data,

3617
03:24:58,640 --> 03:25:02,397
more compute, better architectures,
and there's a lot of improvements that go

3618
03:25:02,480 --> 03:25:05,917
in. But every time we've kind of run
into a like, h this isn't quite scaling

3619
03:25:06,000 --> 03:25:10,477
the way we expect, it's we have a problem.
We have a bug that our math

3620
03:25:10,560 --> 03:25:13,837
wasn't quite right. That oh,
our implementation isn't isn't isn't quite

3621
03:25:13,920 --> 03:25:18,877
matching the math, whatever the thing is.
And that is, I think, a very

3622
03:25:18,960 --> 03:25:22,477
important thing to to sort of internalize.
And actually if you we've

3623
03:25:22,560 --> 03:25:25,757
done studies where you go back to the
beginning of the field right that neural

3624
03:25:25,840 --> 03:25:28,077
nets themselves were designed in like

3625
03:25:28,160 --> 03:25:31,117
1940s right before computers right as a

3626
03:25:31,200 --> 03:25:35,437
model of maybe this is maybe this how
the brain processes information first

3627
03:25:35,520 --> 03:25:38,957
hardware implementation was 1959
with the perceptron and if you look at

3628
03:25:39,040 --> 03:25:41,837
landmark results in the field that the

3629
03:25:41,920 --> 03:25:46,317
landmark results follow this incredibly
smooth deterministic path of more

3630
03:25:46,400 --> 03:25:50,557
compute being poured into them and so
70 years of people maybe 80 years now of

3631
03:25:50,640 --> 03:25:53,277
people saying this stuff is never going
to work, never going to scale, going to

3632
03:25:53,360 --> 03:25:55,677
hit the wall. Hasn't hit the wall yet.

3633
03:25:55,760 --> 03:25:59,997
There's still no wall in sight. And so
I think that the fundamentals allow it.

3634
03:26:00,080 --> 03:26:02,477
Now the practicality is hard, right?

3635
03:26:02,560 --> 03:26:06,157
Actually building these massive
supercomputers. It's hard. It's

3636
03:26:06,240 --> 03:26:10,157
expensive. It's not easy, right? That we
have teams that just like work so hard

3637
03:26:10,240 --> 03:26:14,477
to solve these incredibly hard technical
problems. We have our own network

3638
03:26:14,560 --> 03:26:18,317
protocol that we've had to design.
um that we have people who look at every

3639
03:26:18,400 --> 03:26:21,837
single layer of the stack that there's
weird wiggles in the graph and you the

3640
03:26:21,920 --> 03:26:24,477
way to think about these neural
nets is that there's like there's no

3641
03:26:24,560 --> 03:26:27,997
abstractions, right? It's almost like
any little piece that's wrong can have a

3642
03:26:28,080 --> 03:26:31,357
ripple effect that only shows up down
there. And so you need people who deeply

3643
03:26:31,440 --> 03:26:34,957
understand all of it. And yet if you get

3644
03:26:35,040 --> 03:26:36,717
the right team together, put the right

3645
03:26:36,800 --> 03:26:39,277
mission in front of people and people do

3646
03:26:39,360 --> 03:26:44,397
that grind, the outcome, it's worth it,
right? And it's achievable and it's

3647
03:26:44,480 --> 03:26:47,837
possible. And so I think that for those
reasons the progress will continue.

3648
03:26:47,920 --> 03:26:51,597
>> So then I'd love to hear your
perspective if if models can basically

3649
03:26:51,680 --> 03:26:53,917
progress much further from where they

3650
03:26:54,000 --> 03:26:57,117
are today. Um let's say let's say open

3651
03:26:57,200 --> 03:27:01,517
AI builds the best model and it
the equivalent of like something with like

3652
03:27:01,600 --> 03:27:04,077
15 PhDs with excellent emotional

3653
03:27:04,160 --> 03:27:06,237
intelligence that doesn't complain and

3654
03:27:06,320 --> 03:27:09,277
goes out and does stuff for you. Um, and

3655
03:27:09,360 --> 03:27:13,117
then the the next model maker will build

3656
03:27:13,200 --> 03:27:15,757
a a less good, but it has 13 PhDs and

3657
03:27:15,840 --> 03:27:20,237
it's like pretty good uh, you know,
EQ and we'll still go and do things for

3658
03:27:20,320 --> 03:27:24,397
you. So, where does the differentiation
come in when you get to that level of

3659
03:27:24,480 --> 03:27:28,637
intelligence? Because we've seen the
model makers kind of move in lock step.

3660
03:27:28,720 --> 03:27:32,237
One makes an advance, the next one comes
in and makes the advance. So, they all

3661
03:27:32,320 --> 03:27:36,717
become that smart. Do they is it
possible to differentiate? Well, I think

3662
03:27:36,800 --> 03:27:41,437
there are several dimensions to the
answer. Um, number one is I do think

3663
03:27:41,520 --> 03:27:45,597
there's a bit of an attractor state
where just like from a business model

3664
03:27:45,680 --> 03:27:49,517
perspective, every provider
sells out all their compute.

3665
03:27:49,600 --> 03:27:52,477
Okay. Right. I think that is just like
the world that we're heading towards

3666
03:27:52,560 --> 03:27:55,677
where there just is not going to be
enough compute to serve all the demand.

3667
03:27:55,760 --> 03:27:58,477
Right? That we're heading to this compute
powered economy that everyone's

3668
03:27:58,560 --> 03:28:02,157
going to be using these models all
the time to be able to accomplish tasks of

3669
03:28:02,240 --> 03:28:06,557
interest. And we just see it. It's like
right now we're talking about compute

3670
03:28:06,640 --> 03:28:08,877
constraints and like like the number of

3671
03:28:08,960 --> 03:28:11,517
people using these agents is like order

3672
03:28:11,600 --> 03:28:13,677
of 10 million 20 million maybe you know

3673
03:28:13,760 --> 03:28:18,797
it's like we're not at planet scale
chatbt is like a billion users right but

3674
03:28:18,880 --> 03:28:21,277
we haven't brought the agentic power
there yet so you're just looking at

3675
03:28:21,360 --> 03:28:25,277
these factors and the depth of usage
is also tiny compared to where we're going

3676
03:28:25,360 --> 03:28:28,957
and so I think that we're just going
to be in a world where even if you have

3677
03:28:29,040 --> 03:28:33,357
different vendors different capability
level open source models all these

3678
03:28:33,440 --> 03:28:37,197
things these neoclouds like I think that
compute is just going to be this scarce

3679
03:28:37,280 --> 03:28:41,997
resource and I think that it's going
to go go to use. Um so to some extent I

3680
03:28:42,080 --> 03:28:45,757
think that the the like is this a good
business to be in and for new entrance

3681
03:28:45,840 --> 03:28:48,557
to come into and things like that.
My answer is actually yes. I think that

3682
03:28:48,640 --> 03:28:53,117
there is like a huge market um that we
are just not going to be able to address

3683
03:28:53,200 --> 03:28:57,677
and we need much more energy momentum
there. Um but a second thing is that it

3684
03:28:57,760 --> 03:28:59,357
also misses the fact that intelligence

3685
03:28:59,440 --> 03:29:02,157
is not a undimensional thing right that

3686
03:29:02,240 --> 03:29:07,597
that If you really zoom in, being good
at different domains is something where

3687
03:29:07,680 --> 03:29:11,037
even if you you have a lot of raw
intelligence, getting good if you've

3688
03:29:11,120 --> 03:29:15,277
never practiced like you've never
actually done a pitch or something like

3689
03:29:15,360 --> 03:29:18,477
you're not going to be good at it your
first time, right? And that there's lots

3690
03:29:18,560 --> 03:29:21,277
of different you've never operated
a spreadsheet, right? You're not going to

3691
03:29:21,360 --> 03:29:25,997
be able to succeed at doing some complex
modeling. And so I think that there is

3692
03:29:26,080 --> 03:29:29,197
something that we have been
internalizing which is that we look

3693
03:29:29,280 --> 03:29:31,597
across different industries
and different domains and we have to

3694
03:29:31,680 --> 03:29:36,797
prioritize. We can't possibly be great
at every single area at once. There is

3695
03:29:36,880 --> 03:29:40,157
definitely a lot of like, hey, you
just get the general intelligence up and

3696
03:29:40,240 --> 03:29:43,837
it'll experience a lot of these things,
but to really become a domain expert, to

3697
03:29:43,920 --> 03:29:47,917
really be that PhD and to really be
something that can help push forward the

3698
03:29:48,000 --> 03:29:50,637
ambition of a field, like that's hard.

3699
03:29:50,720 --> 03:29:53,757
And and by the way, one thing I
also want to say is that I think

3700
03:29:53,840 --> 03:29:57,037
understanding what happens when you
successfully do that. I think that

3701
03:29:57,120 --> 03:29:59,357
having a good mental model of that's
important, which is you look at

3702
03:29:59,440 --> 03:30:01,837
something like rewind to AlphaGo, right?

3703
03:30:01,920 --> 03:30:03,997
you remember move 37, this move that

3704
03:30:04,080 --> 03:30:09,197
like changed people's understanding
of the game and then now more people play

3705
03:30:09,280 --> 03:30:12,717
go than ever, right? And that it
actually inspired people to do even

3706
03:30:12,800 --> 03:30:16,157
more. I think we're just going to see
that. And so I think that the depth is

3707
03:30:16,240 --> 03:30:20,557
never going to stop, right? How deep can
you go on science, right? Like I I think

3708
03:30:20,640 --> 03:30:23,917
that people have thought sometimes that,
hey, we found out all the physics, it's

3709
03:30:24,000 --> 03:30:27,837
all good. We're all done. And I don't
think that that's the future we're

3710
03:30:27,920 --> 03:30:30,557
signed up for. I think we're signed
up for one where we're got to keep find

3711
03:30:30,640 --> 03:30:33,917
every time you unlock one mystery,
>> every every time you solve one mystery

3712
03:30:34,000 --> 03:30:36,877
unlocks like 10 more, right? So, I think
that there's just going to be so much

3713
03:30:36,960 --> 03:30:39,917
more to do and tons of room
for differentiation across different

3714
03:30:40,000 --> 03:30:41,517
companies. >> So, I think
I'm reading you right and

3715
03:30:41,600 --> 03:30:46,557
that your belief is um maybe there's
a way that everybody can scale up these

3716
03:30:46,640 --> 03:30:50,877
models, but ultimately the company
with the most compute is going to win. And

3717
03:30:50,960 --> 03:30:54,877
you know we spoke a couple months ago
and you had mentioned that like you were

3718
03:30:54,960 --> 03:30:59,357
asked internally how much compute should
we buy and you said all of it and they

3719
03:30:59,440 --> 03:31:03,117
said no really how much should we buy
and you said no buy all of it and and

3720
03:31:03,200 --> 03:31:06,077
OpenAI is definitely the leader in in

3721
03:31:06,160 --> 03:31:10,397
buying compute. I mean we see the the
the money going out obviously a lot of

3722
03:31:10,480 --> 03:31:14,237
money coming in through investment
and and now you you're you've built a

3723
03:31:14,320 --> 03:31:17,997
business with customers but there's
a lot of money going out. Do you ever

3724
03:31:18,080 --> 03:31:22,317
wonder, hey, are like,
do you ever wonder maybe we're

3725
03:31:22,400 --> 03:31:24,637
not going to be able to pay all
this money back because it's a brand new

3726
03:31:24,720 --> 03:31:29,117
category? Well, the way that I look
at it is on the fundamentals, right? You

3727
03:31:29,200 --> 03:31:31,357
you need to really look at the fact that

3728
03:31:31,440 --> 03:31:36,717
the way that compute goes is that it's
multiple years out before compute

3729
03:31:36,800 --> 03:31:39,517
actually arrives, right? Depending
on exactly what you're doing. For example,

3730
03:31:39,600 --> 03:31:44,637
we've been investing in our own chip
program now for multiple years and super

3731
03:31:44,720 --> 03:31:46,477
exciting progress like you know we'll

3732
03:31:46,560 --> 03:31:48,877
we'll we'll have more to announce uh

3733
03:31:48,960 --> 03:31:53,997
actually pretty soon. Um but the fact
that we're able to do that is something

3734
03:31:54,080 --> 03:31:56,637
very unique, right? Really thinking
about the full vertical integration of

3735
03:31:56,720 --> 03:31:59,837
the supply chain. And I think that the
world we're heading towards is one where

3736
03:31:59,920 --> 03:32:03,757
again there's just not going to be
enough compute in the world to satisfy

3737
03:32:03,840 --> 03:32:06,637
all the demand. And we see this very
concretely like you look at the

3738
03:32:06,720 --> 03:32:10,237
exponential I mean rewind to the
exponential of chatbt. Um look at the

3739
03:32:10,320 --> 03:32:12,157
exponentials we're on now. Uh you think

3740
03:32:12,240 --> 03:32:17,517
about the problems that we that
we are able to solve. You know,

3741
03:32:17,600 --> 03:32:21,437
it's it's actually kind of interesting
that yeah, we just yesterday announced

3742
03:32:21,520 --> 03:32:24,157
um it was two days ago announced a new

3743
03:32:24,240 --> 03:32:27,837
result in uh basically chemistry and

3744
03:32:27,920 --> 03:32:31,597
being able to synthesize, you know,
new new improved reaction. And all of this

3745
03:32:31,680 --> 03:32:35,037
is without much attention. You know, the
thing I just said of if you go deep in a

3746
03:32:35,120 --> 03:32:38,317
domain, you can really transform it.
And we're not even scratching the surface

3747
03:32:38,400 --> 03:32:42,397
yet. And so the way to think about is
the economy is so massive, right? And we

3748
03:32:42,480 --> 03:32:45,517
see it very concretely in terms of our
own growth, in terms of what people are

3749
03:32:45,600 --> 03:32:49,437
willing to pay and kind of the size and
growth of this whole industry. And so I

3750
03:32:49,520 --> 03:32:54,077
think that the thing that I think about
the most is how do we meet the demand?

3751
03:32:54,160 --> 03:32:57,357
How do you actually have something
that can help support all of the work that

3752
03:32:57,440 --> 03:33:00,397
people want to do in the economy? And I
think that is such a vast thing. I don't

3753
03:33:00,480 --> 03:33:03,997
think any of us have internalized it yet.
Yeah. But if I may, there is a

3754
03:33:04,080 --> 03:33:07,437
price war brewing. I mean, at least
that's according to the report. It's

3755
03:33:07,520 --> 03:33:08,717
great to have you here to talk about it.

3756
03:33:08,800 --> 03:33:10,797
The Wall Street Journal recently had a

3757
03:33:10,880 --> 03:33:13,437
report that an upcoming OpenAI model

3758
03:33:13,520 --> 03:33:16,317
might have significant price cuts. And

3759
03:33:16,400 --> 03:33:19,437
so again, like how can you know if it

3760
03:33:19,520 --> 03:33:22,717
requires so much resources to serve this

3761
03:33:22,800 --> 03:33:27,037
demand and it is growing demand um in
an environment where there might be price

3762
03:33:27,120 --> 03:33:29,277
cuts, how do you make that math work?

3763
03:33:29,360 --> 03:33:33,437
>> Well, again, I look at it from a
different angle. So if you look at the

3764
03:33:33,520 --> 03:33:37,357
whole history of what we've done,
we actually have been increasing the

3765
03:33:37,440 --> 03:33:41,837
intelligence cutting price right for a
fixed amount of intelligence and people

3766
03:33:41,920 --> 03:33:45,037
somehow just like like the Jeffans
paradox just keeps happening and so I

3767
03:33:45,120 --> 03:33:48,397
think frontier intelligence will always
be something that is going to be you

3768
03:33:48,480 --> 03:33:50,957
know it's always going to be
the priciest thing but I think that a year

3769
03:33:51,040 --> 03:33:54,077
from now that level of intelligence
is going to feel pretty mundane and like

3770
03:33:54,160 --> 03:33:58,637
you know going to be much more available.
Um, and I think that that the

3771
03:33:58,720 --> 03:34:02,077
the world that we're in is one where
people are starting to really think

3772
03:34:02,160 --> 03:34:05,037
about value. And it's actually been a

3773
03:34:05,120 --> 03:34:09,757
very interesting shift where over the
past, you know, first quarter, maybe up

3774
03:34:09,840 --> 03:34:13,677
until now, people have just been like
this AI agent stuff, it's all new. We

3775
03:34:13,760 --> 03:34:16,397
need to bring into our enterprise like
we don't want to be left behind. How do

3776
03:34:16,480 --> 03:34:20,237
we be part of this this future? And now
people are like, okay, like let's make

3777
03:34:20,320 --> 03:34:23,277
sure this is actually delivering ROI
and value. And I actually think that's a

3778
03:34:23,360 --> 03:34:26,477
great place to be, right? because people
are asking the right questions. And I

3779
03:34:26,560 --> 03:34:29,437
hear this I I had some customer meetings
today where people were saying exactly

3780
03:34:29,520 --> 03:34:33,277
this. They were like, "How can we have
even just like good spend controls? How

3781
03:34:33,360 --> 03:34:37,757
can we have observability?" And I I
think we literally today just released

3782
03:34:37,840 --> 03:34:40,237
spend controls. So,
you know, it's like Okay.

3783
03:34:40,320 --> 03:34:43,677
>> Exactly. Yeah. We're we are really
investing hard in enterprise readiness

3784
03:34:43,760 --> 03:34:45,757
and the tools that our customers are

3785
03:34:45,840 --> 03:34:47,757
telling us that they need. And I think

3786
03:34:47,840 --> 03:34:53,037
that that for me is the shift that we've
also been going through as a company is

3787
03:34:53,120 --> 03:34:55,677
really not just thinking about hey we're
just going to release models and you

3788
03:34:55,760 --> 03:34:58,957
know have a model really thinking about
the end to end of the business. How do

3789
03:34:59,040 --> 03:35:01,037
we bring this into solving real problems

3790
03:35:01,120 --> 03:35:04,557
for real customers and that is happening

3791
03:35:04,640 --> 03:35:07,117
so quickly across every single industry

3792
03:35:07,200 --> 03:35:12,317
and the number of different companies
that still feel like they're wrapping

3793
03:35:12,400 --> 03:35:15,277
their mind around how to best make use
of these models we're learning at the

3794
03:35:15,360 --> 03:35:19,997
same time. I think it's just so early
in this whole game to me the the like the

3795
03:35:20,080 --> 03:35:23,917
absolute size of the market growing so
quickly, our own revenue ramp growing so

3796
03:35:24,000 --> 03:35:27,277
quickly. I think it's still just like
none of us are anticipating how steep

3797
03:35:27,360 --> 03:35:30,797
that's all going to go.
>> Are you going to cut prices?

3798
03:35:30,880 --> 03:35:35,517
>> I So again, the answer is always yes,
right? But it's about like I think that

3799
03:35:35,600 --> 03:35:38,797
what's going to keep happening is that
we're going to have frontier models. Um

3800
03:35:38,880 --> 03:35:42,157
I don't think there's going to be like
a massive shift in in the short term. I I

3801
03:35:42,240 --> 03:35:44,397
don't think that that is the kind
of thing that's going to happen. But I

3802
03:35:44,480 --> 03:35:49,197
think the thing you should anticipate
is that over a year-long time horizon to

3803
03:35:49,280 --> 03:35:52,637
get to today's level of intelligence
that feels very premier, it's going to

3804
03:35:52,720 --> 03:35:56,397
be much cheaper. But there's going to be
a new thing that is going to be so much

3805
03:35:56,480 --> 03:35:58,957
better and you're going to be like, why
would I ever use this other one, right?

3806
03:35:59,040 --> 03:36:01,517
It's just it's just how
it's always going to be.

3807
03:36:01,600 --> 03:36:03,517
>> So Sadia Nadell has
had some interesting

3808
03:36:03,600 --> 03:36:07,677
tweets and interviews recently.
He recently said the model is becoming a

3809
03:36:07,760 --> 03:36:11,197
commodity and the valuable asset
is company or this might be a paraphrase

3810
03:36:11,280 --> 03:36:13,037
the valuable asset is a company specific

3811
03:36:13,120 --> 03:36:15,597
AI system that continually learns from

3812
03:36:15,680 --> 03:36:20,477
your data. Um what do you think about
that and is it weird to be competing

3813
03:36:20,560 --> 03:36:21,997
with Microsoft now?

3814
03:36:22,080 --> 03:36:25,597
>> Well look I don't
think that there's any

3815
03:36:25,680 --> 03:36:29,517
layer of the stack here that is going
to just kind of be removed from the value

3816
03:36:29,600 --> 03:36:33,277
chain. I think that these things
multiplied together and if you think

3817
03:36:33,360 --> 03:36:36,237
about the most the base layer of compute

3818
03:36:36,320 --> 03:36:40,797
right that is something where
it's just like no compute no AI

3819
03:36:40,880 --> 03:36:43,677
and to some extent you can say oh comput
is commoditized it's just flops who

3820
03:36:43,760 --> 03:36:47,597
cares about it but in reality like you
look at you look at today's chip stocks

3821
03:36:47,680 --> 03:36:50,957
you look at the uh you know people who
are selling compute kind of what what is

3822
03:36:51,040 --> 03:36:54,397
what the market is valuing people
at and they see that there's a fundamental

3823
03:36:54,480 --> 03:36:58,717
asset here that is just so critical and I
think that is because it is a revenue

3824
03:36:58,800 --> 03:37:03,277
center it is something that anyone
who's building AI has to rely on and that

3825
03:37:03,360 --> 03:37:06,477
there's a bunch of very interesting
dynamics in terms of the efficiencies

3826
03:37:06,560 --> 03:37:08,717
that you can squeeze out and the
margins, all these things. But

3827
03:37:08,800 --> 03:37:11,677
fundamentally, even though it's like
you can kind of squint at it and say it's

3828
03:37:11,760 --> 03:37:15,837
commoditized, it's not. It's it's it's
not that that the value goes away. It's

3829
03:37:15,920 --> 03:37:19,357
not that the margins go away. It's like
something the market will reward because

3830
03:37:19,440 --> 03:37:23,197
it has fundamental value and that the
importance of it's going to go up over

3831
03:37:23,280 --> 03:37:26,397
time. You can see that with some of the
prices that people are paying for H100s,

3832
03:37:26,480 --> 03:37:30,877
right? hoppers are, you know, kind of,
you know, not not obsolete, right? But

3833
03:37:30,960 --> 03:37:36,477
they're they're they're a a previous gen
chip and u nor in any normal situation

3834
03:37:36,560 --> 03:37:38,957
where you're not totally supply
constrained, no one would be buying

3835
03:37:39,040 --> 03:37:42,557
them. But instead, the market prices
are up relative to to where they were

3836
03:37:42,640 --> 03:37:45,517
before. So there's this inversion that's
happening and again I think it's going

3837
03:37:45,600 --> 03:37:48,717
to keep happening where because

3838
03:37:48,800 --> 03:37:53,117
everyone has this avalanche of demand
that you're going to see prices and

3839
03:37:53,200 --> 03:37:57,117
margins and all of these things AC
continuing to increase at various levels

3840
03:37:57,200 --> 03:37:59,277
of the stack. I think the same kind of

3841
03:37:59,360 --> 03:38:01,997
applies for models where the models

3842
03:38:02,080 --> 03:38:07,197
themselves are also again they're not
>> there's there's a lot of competition

3843
03:38:07,280 --> 03:38:09,997
there and I think that's very good.
I think it's good for the enterprise. I

3844
03:38:10,080 --> 03:38:11,917
think it's good for customers consumers.

3845
03:38:12,000 --> 03:38:16,077
Um, but I think that there's a lot
of areas where, for example, our models

3846
03:38:16,160 --> 03:38:19,917
have always been the sort of smartest
ones, right? The ones that are able to

3847
03:38:20,000 --> 03:38:22,957
solve these incredibly hard problems.
I think we're just starting to reach a

3848
03:38:23,040 --> 03:38:26,637
phase where you're going to see the
transformative impact from that, right?

3849
03:38:26,720 --> 03:38:31,357
It's like if we're really able to speed
up science through models, the smarter

3850
03:38:31,440 --> 03:38:33,277
the model, the faster it's going to go.

3851
03:38:33,360 --> 03:38:36,237
And it's very, very different
from a model that has a conversational

3852
03:38:36,320 --> 03:38:39,597
interface that you're able to, you know,
is able to book your travel, right? Or

3853
03:38:39,680 --> 03:38:42,877
organize your calendar. So that's also
a dimension I think we're going to do a

3854
03:38:42,960 --> 03:38:46,957
very good job in. But I'm just saying
it's it's a different area. Um and then

3855
03:38:47,040 --> 03:38:48,957
I think that the question of well how do

3856
03:38:49,040 --> 03:38:53,997
you actually connect the intelligence
to your own customers right to real value

3857
03:38:54,080 --> 03:38:58,157
to you have all these enterprises
that have built incredible businesses in

3858
03:38:58,240 --> 03:39:01,677
different domains and it's a huge thing
and it's not something where if you

3859
03:39:01,760 --> 03:39:04,557
don't have domain expertise that you're
just going to be able to do right and

3860
03:39:04,640 --> 03:39:07,437
part of it is that you need you think
about regulated industries you think

3861
03:39:07,520 --> 03:39:11,517
about any area where there's like you
know think about education where you

3862
03:39:11,600 --> 03:39:14,397
have a parent you have a teacher you
have a student you have these different

3863
03:39:14,480 --> 03:39:18,797
parties that need to interact in very
thoughtful ways for all of these areas,

3864
03:39:18,880 --> 03:39:23,357
all of these domains that there's a lot
of value to be built by being in that

3865
03:39:23,440 --> 03:39:27,117
area and thinking about how the workflow
should work, how these models should be

3866
03:39:27,200 --> 03:39:30,877
orchestrated. And so I I really think
that there's more than enough to go

3867
03:39:30,960 --> 03:39:34,957
around. And I think that we have to work
together as a whole ecosystem in order

3868
03:39:35,040 --> 03:39:37,917
to deliver the kind of value that I
think is possible from these systems.

3869
03:39:38,000 --> 03:39:42,157
>> Okay, just to go back to the SIA point
one more time, uh he's called models a

3870
03:39:42,240 --> 03:39:45,997
commodity. He's trying to build his
own frontier intelligence. He's telling

3871
03:39:46,080 --> 03:39:49,197
potentially your customers, hey, you got
to come work with us because we're going

3872
03:39:49,280 --> 03:39:53,437
to help build these loops that will
learn from your data. Um, he's got

3873
03:39:53,520 --> 03:39:55,997
access to your IP, I think, till 2032.

3874
03:39:56,080 --> 03:39:59,277
So, how does it make you feel
to hear this coming from SAI?

3875
03:39:59,360 --> 03:40:01,197
>> Look, I think that the most important

3876
03:40:01,280 --> 03:40:06,477
thing that is happening right now is
the usage of AI in the economy to really

3877
03:40:06,560 --> 03:40:10,077
transform the economy and to uplift
everyone. And so, I think that that is

3878
03:40:10,160 --> 03:40:11,437
something that I'm really focused on.

3879
03:40:11,520 --> 03:40:15,197
And the more that people are trying
to make that happen, I think that that's

3880
03:40:15,280 --> 03:40:16,877
better for everyone.

3881
03:40:16,960 --> 03:40:20,557
>> Um, GPT 5.6 is rumored
to be on its way.

3882
03:40:20,640 --> 03:40:24,477
Supposed to be this just
a Twitter rumor, but I'm going

3883
03:40:24,560 --> 03:40:26,797
to read it to you. Um, always the best

3884
03:40:26,880 --> 03:40:29,437
rumors. Three times cheaper than Fable

3885
03:40:29,520 --> 03:40:32,557
up to 1.5 million token context. Uh,

3886
03:40:32,640 --> 03:40:35,757
stronger agentic coding workflows. Um,

3887
03:40:35,840 --> 03:40:37,517
how much of that is true? What should we

3888
03:40:37,600 --> 03:40:41,757
expect for GPT 5.6?

3889
03:40:41,840 --> 03:40:46,957
I mean, look, you should always expect
better, faster, smarter, the whole

3890
03:40:47,040 --> 03:40:52,440
thing. >> So, everything confirmed.

3891
03:40:52,720 --> 03:40:56,717
>> Definitely believe everything
you read on Twitter. Yeah, maybe not.

3892
03:40:56,800 --> 03:40:57,997
>> That has actually been a source of

3893
03:40:58,080 --> 03:41:01,437
problems in my personal life. Um, okay.

3894
03:41:01,520 --> 03:41:04,957
So, uh, I want to end on health. Um,
you brought it up a couple times. You

3895
03:41:05,040 --> 03:41:08,717
actually had a question in the audience
about it earlier. Um, you know,

3896
03:41:08,800 --> 03:41:10,797
sometimes there's a story and you read

3897
03:41:10,880 --> 03:41:14,157
it and you say to yourself, I know this

3898
03:41:14,240 --> 03:41:16,397
person is speaking to the media and I

3899
03:41:16,480 --> 03:41:18,477
know that what they're saying sounds

3900
03:41:18,560 --> 03:41:21,437
like maybe it's true. Um, but there's

3901
03:41:21,520 --> 03:41:25,677
something wrong with the story and we're
not going to see more of it. Um, and

3902
03:41:25,760 --> 03:41:27,677
I've read a couple of those recently.

3903
03:41:27,760 --> 03:41:29,757
Um, one is I think is it your friend the

3904
03:41:29,840 --> 03:41:34,317
GitLab CEO um, Sid Sberandage? He had um

3905
03:41:34,400 --> 03:41:37,917
he he got cancer and used uh he got all

3906
03:41:38,000 --> 03:41:40,637
the diagnostic testing uh he could have.

3907
03:41:40,720 --> 03:41:42,797
So just went out and tested like crazy

3908
03:41:42,880 --> 03:41:45,677
and fed that data into ChachiPT with the

3909
03:41:45,760 --> 03:41:50,077
assistance of some people who had built
purpose-built application for it and was

3910
03:41:50,160 --> 03:41:52,637
able I don't know if cure is the right

3911
03:41:52,720 --> 03:41:57,197
word but to beat back the cancer to a
degree. There was also this dog Rosie

3912
03:41:57,280 --> 03:42:02,077
the dog in Australia. You guys heard
of Rosie? like the craziest story where

3913
03:42:02,160 --> 03:42:03,677
this guy I'm gonna get some detail

3914
03:42:03,760 --> 03:42:07,917
wrong, but a guy um biopsied his dog

3915
03:42:08,000 --> 03:42:11,197
which had cancer um ran the mutations

3916
03:42:11,280 --> 03:42:14,157
across across Alphafold and then was

3917
03:42:14,240 --> 03:42:16,717
able to design an mRNA vaccine that he

3918
03:42:16,800 --> 03:42:19,517
injected into the dog um with the

3919
03:42:19,600 --> 03:42:24,077
assistance of chatbots to build this
thing which ended up being able to jump

3920
03:42:24,160 --> 03:42:26,397
over tables again and the tumor shrunk.

3921
03:42:26,480 --> 03:42:29,357
Um, when we think about the future of AI

3922
03:42:29,440 --> 03:42:32,317
and health, um, help us sort out the

3923
03:42:32,400 --> 03:42:36,637
truth with this question. Are these
a couple of outliers that made good

3924
03:42:36,720 --> 03:42:40,397
headlines, but there was something about
the story we weren't hearing, or is this

3925
03:42:40,480 --> 03:42:42,477
going to become standard in the future?

3926
03:42:42,560 --> 03:42:44,237
>> Absolutely going to become standard.

3927
03:42:44,320 --> 03:42:46,477
Absolutely. And it's I I personally have

3928
03:42:46,560 --> 03:42:49,677
a number of friends who have done very

3929
03:42:49,760 --> 03:42:53,677
similar things of get the data right

3930
03:42:53,760 --> 03:42:55,997
your health diagnostics

3931
03:42:56,080 --> 03:43:01,357
and use codecs right use these models
to get insights from them and I think that

3932
03:43:01,440 --> 03:43:02,957
there are many people like I think that

3933
03:43:03,040 --> 03:43:06,397
there's about 230 million people each

3934
03:43:06,480 --> 03:43:10,637
week who use chat GBT for health queries

3935
03:43:10,720 --> 03:43:13,037
right and that's been that's that's like

3936
03:43:13,120 --> 03:43:15,597
a staggering scale Right. And this is pe

3937
03:43:15,680 --> 03:43:17,677
these are people sometimes you upload a

3938
03:43:17,760 --> 03:43:22,797
scan. Sometimes you have doctors who are
telling you conflicting information. And

3939
03:43:22,880 --> 03:43:26,237
I think that we've been in a world
where patients are not empowered, right?

3940
03:43:26,320 --> 03:43:27,997
Patients have to be the doctor, right?

3941
03:43:28,080 --> 03:43:31,917
You're the decider. You are accountable,
right? You know, doctor makes a mistake

3942
03:43:32,000 --> 03:43:35,917
and you're going to be paying the price
for the rest of your life. Like it's

3943
03:43:36,000 --> 03:43:40,077
just it's a very different kind of
incentive. And this is very personal for

3944
03:43:40,160 --> 03:43:42,717
me. you know, my wife has a number of
health conditions and I think that we've

3945
03:43:42,800 --> 03:43:45,917
just been we've not like I don't even
know how we'd be able to manage many of

3946
03:43:46,000 --> 03:43:49,757
her conditions right now without the use
of of chat. And I think we're just at

3947
03:43:49,840 --> 03:43:51,037
the beginning of this journey, right?

3948
03:43:51,120 --> 03:43:54,877
That I think that the degree to which

3949
03:43:54,960 --> 03:43:59,437
even if you have the best medical team,
the best acts as the best best experts,

3950
03:43:59,520 --> 03:44:04,317
there's only so much that can be done,
right? You think about the things that

3951
03:44:04,400 --> 03:44:06,957
are just outside of the reach
of humanity or even just sometimes it's

3952
03:44:07,040 --> 03:44:11,037
like someone didn't even read the chart,
right? kind of missed a detail. All of

3953
03:44:11,120 --> 03:44:14,717
that we should be able to improve
massively through these tools. And so I

3954
03:44:14,800 --> 03:44:17,517
think that the personalized medicine and

3955
03:44:17,600 --> 03:44:22,637
sometimes it's going to be about drugs
and drug discovery that are of you know

3956
03:44:22,720 --> 03:44:27,117
for for mass market but sometimes it'll
be even for the kind of NF1 things like

3957
03:44:27,200 --> 03:44:29,517
the the disease um diagnoses that I

3958
03:44:29,600 --> 03:44:31,597
mentioned earlier today. Sometimes it

3959
03:44:31,680 --> 03:44:34,477
will be for just like trying to

3960
03:44:34,560 --> 03:44:38,957
understand conditions and trying to come
up with with new potential therapeutics.

3961
03:44:39,040 --> 03:44:42,717
All of that we're seeing it happening
right now in front of our eyes. It's not

3962
03:44:42,800 --> 03:44:46,797
theoretical. It's really happening. And so
one of the most I think it's like one

3963
03:44:46,880 --> 03:44:50,717
of the most astounding possibilities
of AI is how much it can improve our

3964
03:44:50,800 --> 03:44:53,037
health. And you think about the ripple

3965
03:44:53,120 --> 03:44:54,957
effects of the system, right? Where so

3966
03:44:55,040 --> 03:44:58,557
much spending on the health care system

3967
03:44:58,640 --> 03:45:00,477
happens right now. that's a massive part

3968
03:45:00,560 --> 03:45:02,637
of the economy and that if you're

3969
03:45:02,720 --> 03:45:07,837
actually able to help people prevent
issues, right, to to get ahead of

3970
03:45:07,920 --> 03:45:11,517
potential, you know,
health health problems, that's

3971
03:45:11,600 --> 03:45:14,877
something that actually then alleviates
a lot of burden, a lot of strain. And

3972
03:45:14,960 --> 03:45:17,837
we're in a world where doctors are
burned out, nurses are burned out, like

3973
03:45:17,920 --> 03:45:21,517
there's like a real crisis that's
happening in front of us. And I think AI

3974
03:45:21,600 --> 03:45:23,997
will be able to help with all of that.

3975
03:45:24,080 --> 03:45:25,997
Like we have that potential if we deploy

3976
03:45:26,080 --> 03:45:29,277
it and use it wisely and well. And so I

3977
03:45:29,360 --> 03:45:32,477
think that applying AI to medicine, like

3978
03:45:32,560 --> 03:45:36,317
that's something that I is really
a personal motivation for me in in

3979
03:45:36,400 --> 03:45:38,637
thinking about this whole journey of of
what we're building, what we're trying

3980
03:45:38,720 --> 03:45:42,237
to do with open AI. And I'm hopeful that
we as a as a world and a community can

3981
03:45:42,320 --> 03:45:44,877
make the most of that. Let's hope. I

3982
03:45:44,960 --> 03:45:47,437
think we will. I'm very very confident.

3983
03:45:47,520 --> 03:45:51,837
>> Greg, thank you so much. Thank you.

3984
03:45:51,920 --> 03:45:55,640
Thank you. Thank you so much.

3985
03:46:00,320 --> 03:46:05,400
Oh my god. Thank you everyone.
You have a good time today.

3986
03:46:08,560 --> 03:46:12,637
>> Thank you.
Should we do it again next year?

3987
03:46:12,720 --> 03:46:17,597
>> Going to come. >> All right.
U let me uh give you a a

3988
03:46:17,680 --> 03:46:20,077
perspective about what we're doing
for the rest of the day. First of all, if

3989
03:46:20,160 --> 03:46:21,757
you're backstage, Cali and John, can you

3990
03:46:21,840 --> 03:46:24,077
come out here for a second?

3991
03:46:24,160 --> 03:46:27,997
Um, as I as I bring them out, as I bring

3992
03:46:28,080 --> 03:46:32,077
them out, let me let me uh explain
that we're going to go from here to the

3993
03:46:32,160 --> 03:46:33,997
rooftop. We have some drinks and food

3994
03:46:34,080 --> 03:46:36,477
ready for you guys. We also have a space

3995
03:46:36,560 --> 03:46:39,677
directly outside. Uh, so that is a place

3996
03:46:39,760 --> 03:46:44,477
that you can hang out. Um,
figure out what's what's busier and try

3997
03:46:44,560 --> 03:46:48,237
to spread yourself a little bit so we'll
have room. Uh, John, come up here. Cali,

3998
03:46:48,320 --> 03:46:50,477
come up here. Um, guys, this would not

3999
03:46:50,560 --> 03:46:56,280
have happened without John
Bashu and Calli Win. Um,

4000
03:46:59,600 --> 03:47:01,437
>> thank you, Cali.
>> Thank you very much.

4001
03:47:01,520 --> 03:47:04,477
>> So, uh, true story, I announced this

4002
03:47:04,560 --> 03:47:08,717
show on a podcast live and I thought
we would get a lot of people signing up.

4003
03:47:08,800 --> 03:47:12,397
One person signed up and I called I
called both of them and I said, you

4004
03:47:12,480 --> 03:47:15,517
know, it was great brainstorming this idea
with you, but we're not going to do

4005
03:47:15,600 --> 03:47:20,317
it. Um, so we'll see you next year. And
both of them said, let's do it and let's

4006
03:47:20,400 --> 03:47:22,317
do it at the Commonwealth Club. and we

4007
03:47:22,400 --> 03:47:27,480
did it. So, thank you
guys and thank you all.

4008
03:47:29,120 --> 03:47:33,037
Before we leave, I have to thank our
sponsors at PWC, Dallas, and Arita. You

4009
03:47:33,120 --> 03:47:38,120
guys are amazing.
And thanks to the PWC crew.

4010
03:47:38,640 --> 03:47:42,317
Um, just a great group and amazing
people to work with. And of course,

4011
03:47:42,400 --> 03:47:43,597
thank you to the Commonwealth Club here

4012
03:47:43,680 --> 03:47:46,717
for collaborating with us on this event.

4013
03:47:46,800 --> 03:47:51,117
Wow. uh six years after I showed up here
and there was literally nobody in the

4014
03:47:51,200 --> 03:47:53,117
audience and we kind of have social

4015
03:47:53,200 --> 03:47:55,037
distanced and probably would have gone

4016
03:47:55,120 --> 03:47:58,397
to jail if London Breed found out. Um

4017
03:47:58,480 --> 03:48:02,637
we're all here together and it means
the world to me. So, thank you all. I'm

4018
03:48:02,720 --> 03:48:05,997
getting emotional here. Um thank you
for helping me live the dream and let's go

4019
03:48:06,080 --> 03:48:11,240
party. >> Thank you.