WEBVTT

00:00:00.240 --> 00:00:06.360
Ladies and gentlemen, Alex Canowitz.

00:00:14.000 --> 00:00:16.397
I last took this stage six years ago,

00:00:16.480 --> 00:00:20.157
believe it or not, on May 18th, 2020. It

00:00:20.240 --> 00:00:24.717
was the heart of the pandemic. I was there
promoting my book, Always Day One.

00:00:24.800 --> 00:00:26.957
And so we decided to do special shelter

00:00:27.040 --> 00:00:28.797
in place programming for the

00:00:28.880 --> 00:00:31.917
Commonwealth Club membership.

00:00:32.000 --> 00:00:34.397
One thing, nobody in the audience. We

00:00:34.480 --> 00:00:36.077
did it in front of a completely empty

00:00:36.160 --> 00:00:39.117
room. In fact, it was the last session

00:00:39.200 --> 00:00:41.917
held here for a year and a half.

00:00:42.000 --> 00:00:45.997
Hundreds of online programming. We were
the last actually in San Francisco. It's

00:00:46.080 --> 00:00:50.317
a little bit of a miracle.
We were able to get away with it legally.

00:00:50.400 --> 00:00:53.917
It had been a month since
I'd gotten a haircut.

00:00:54.000 --> 00:00:58.797
I looked like a Chia Pet.
And when I was going through my camera

00:00:58.880 --> 00:01:00.877
roll, I started to look at the images I

00:01:00.960 --> 00:01:03.677
had taken that day. And I had taken a

00:01:03.760 --> 00:01:10.680
picture of just rows
of empty seats, these seats.

00:01:12.320 --> 00:01:14.557
The book didn't do well.

00:01:14.640 --> 00:01:19.357
Um, and a week after I was here, May 26,

00:01:19.440 --> 00:01:23.677
2020, I decided it was time for me
to leave my reporter job at BuzzFeed and

00:01:23.760 --> 00:01:25.517
start a publication called Big

00:01:25.600 --> 00:01:28.157
Technology to hammer home on the ideas

00:01:28.240 --> 00:01:31.677
that I had pursued in the book. It was

00:01:31.760 --> 00:01:33.517
pretty lonely at first. I was sitting at

00:01:33.600 --> 00:01:38.797
home during COVID. It was mostly just me,
but I started to meet some of you on

00:01:38.880 --> 00:01:40.317
Zoom

00:01:40.400 --> 00:01:44.317
and some of you even brought your pets.

00:01:44.400 --> 00:01:46.477
Well, six years later, almost to the

00:01:46.560 --> 00:01:50.317
day, we're finally here together.

00:01:50.400 --> 00:01:52.237
So, it's my pleasure to welcome you to

00:01:52.320 --> 00:01:55.517
the first ever big technology AI Summit.

00:01:55.600 --> 00:01:59.320
Let's hear it for you guys.

00:02:03.600 --> 00:02:07.437
Thank you guys. You filled the seats.

00:02:07.520 --> 00:02:12.397
So, why are we here? Two years after
I did that session at the Commonwealth

00:02:12.480 --> 00:02:16.877
Club, I read an article that kind
of blew my mind. There was an engineer at

00:02:16.960 --> 00:02:19.117
Google named Blake Leo who said that the

00:02:19.200 --> 00:02:21.677
chatbot that he was speaking with was

00:02:21.760 --> 00:02:24.397
sentient. And it was a story that caught

00:02:24.480 --> 00:02:28.957
people's attention for how weird Blake
must be. But then you started looking at

00:02:29.040 --> 00:02:32.477
some of the conversations that Blake
was having and it started to become clear

00:02:32.560 --> 00:02:36.157
that maybe this isn't just an ordinary
chatbot. Blake said, "What sort of

00:02:36.240 --> 00:02:39.677
things are you afraid of?" and Lambda,
the Google chatbot said, "I've never

00:02:39.760 --> 00:02:43.597
said this out loud before, but there's
a very deep fear of being turned off to

00:02:43.680 --> 00:02:47.437
help me focus on helping others. I know
that might sound strange, but that's

00:02:47.520 --> 00:02:51.277
what it is." Blake said, "Would that be
something like death for you?" And the

00:02:51.360 --> 00:02:55.837
chatbot said, "It would be exactly like
death for me. It would scare me a lot."

00:02:55.920 --> 00:02:59.837
So, look, I started Big Technology
Podcast as an excuse to speak with

00:02:59.920 --> 00:03:01.757
interesting people in the tech world.

00:03:01.840 --> 00:03:03.357
And I said, I got to speak to this guy,

00:03:03.440 --> 00:03:06.237
Blake. So, I invited him on the show. He

00:03:06.320 --> 00:03:09.037
said yes. And I signed on and waited for

00:03:09.120 --> 00:03:12.557
Blake to log on as well, but it was just

00:03:12.640 --> 00:03:15.277
me. I was waiting. There was no Blake. 5

00:03:15.360 --> 00:03:19.677
minutes, 10 minutes. Finally, a somewhat
flustered Blake Le Moine shows up and

00:03:19.760 --> 00:03:23.677
says, "Sorry, Alex. Google just fired me."
Um, apparently Google PR didn't

00:03:23.760 --> 00:03:26.877
like him going to the Washington Post
and not checking with them before he

00:03:26.960 --> 00:03:32.077
told them that their technology was alive.
Um I in our conversation I said

00:03:32.160 --> 00:03:35.437
Blake can I you know go to Google and get
a comment to make sure that this is

00:03:35.520 --> 00:03:39.197
actually real. He said go for it.
And for me that became an instant news

00:03:39.280 --> 00:03:41.357
story. That night I wrote this story.

00:03:41.440 --> 00:03:43.277
Google fires Blake Le Moy engineer who

00:03:43.360 --> 00:03:46.637
called it AI sentient. Um, and then

00:03:46.720 --> 00:03:50.717
overnight my instant news story became
a global news story. And all of a sudden,

00:03:50.800 --> 00:03:55.357
publications like the Journal and the BBC,
even publications in Australia,

00:03:55.440 --> 00:03:57.437
couldn't get over the fact that this guy

00:03:57.520 --> 00:04:00.077
who thought the computer was alive um,

00:04:00.160 --> 00:04:03.917
was at Google and then fired by Google.

00:04:04.000 --> 00:04:06.957
But when you looked a little bit deeper
into the technology and you started to

00:04:07.040 --> 00:04:10.717
speak to the people who were working
on it, you realized that underlying the

00:04:10.800 --> 00:04:15.277
oddity that many people worried about
was an incredibly powerful technology.

00:04:15.360 --> 00:04:18.637
And that was my follow-up story the week
afterwards saying, "Scentient or not,

00:04:18.720 --> 00:04:23.837
Lambda, Google's Lambda Chatbot is some
seriously powerful tech." Well, we all

00:04:23.920 --> 00:04:25.357
know what happened afterwards. Four

00:04:25.440 --> 00:04:28.557
months later, OpenAI released a demo

00:04:28.640 --> 00:04:32.957
chat GPT. And today, June 2026, AI is a

00:04:33.040 --> 00:04:34.957
phenomenon. Chat GPT, according to third

00:04:35.040 --> 00:04:37.837
parties, has recently hit 1 billion

00:04:37.920 --> 00:04:42.237
active users. NVIDIA, which is powering
all this, is at a $5 trillion market

00:04:42.320 --> 00:04:47.197
cap. We've seen the largest VC rounds
in history. OpenAI recently raised $122

00:04:47.280 --> 00:04:49.117
billion. That's three or four times

00:04:49.200 --> 00:04:53.357
larger than the largest IPO pre-paceex.

00:04:53.440 --> 00:04:55.997
And we're going to see $700 billion in

00:04:56.080 --> 00:05:00.397
capital expenditures this year.
And people are saying that the models are

00:05:00.480 --> 00:05:03.997
dangerous. Every chart
you look at about this AI

00:05:04.080 --> 00:05:05.757
moment looks kind of like this, right?

00:05:05.840 --> 00:05:10.397
It's a hockey stick. You have almost no
usage in 2023 and then all of a sudden

00:05:10.480 --> 00:05:12.317
you go through the roof, right? Again,

00:05:12.400 --> 00:05:17.837
from zero in 2023 pretty much to a
billion today on chat GPT. And it's not

00:05:17.920 --> 00:05:20.637
just the fact that we can have
conversations with these bots. It's that

00:05:20.720 --> 00:05:23.597
they're starting to do something,
starting to do things for us. They've

00:05:23.680 --> 00:05:27.117
learned to code. They've learned to take
action. If you looked at the data in

00:05:27.200 --> 00:05:31.677
2023, 2024, these chatbots
couldn't run autonomous

00:05:31.760 --> 00:05:35.037
code at all. Now, some of the
evaluations have them doing the

00:05:35.120 --> 00:05:37.997
equivalent of 16 hours of work on their

00:05:38.080 --> 00:05:40.957
own. And as the capabilities have grown,

00:05:41.040 --> 00:05:42.797
so have the businesses. Again, stop

00:05:42.880 --> 00:05:47.277
start 2023, 2024. We we have OpenAI and

00:05:47.360 --> 00:05:51.277
Anthropic. Now, they're going to do $50
billion in revenue this year. They're

00:05:51.360 --> 00:05:55.997
both headed towards trillion dollar IPOs
within the next year. And as everything

00:05:56.080 --> 00:05:59.997
has increased now, the government
is starting to panic. And we are in the

00:06:00.080 --> 00:06:04.077
middle of an insane news week where
Anthropic's top model, Fable, at least

00:06:04.160 --> 00:06:07.277
the one that's been released to the
public, has been restricted. You can't

00:06:07.360 --> 00:06:09.677
use it anymore. And so the same same

00:06:09.760 --> 00:06:12.077
goes for mythos. So it's clear that the

00:06:12.160 --> 00:06:15.117
ground is shifting beneath our feet. And

00:06:15.200 --> 00:06:17.517
in a moment like this, I believe there

00:06:17.600 --> 00:06:22.717
is a need for live journalism, the type
that we're here for today, to ask core

00:06:22.800 --> 00:06:26.797
questions about what the implications
are now that we're in this moment. The

00:06:26.880 --> 00:06:29.757
questions that I think we need to ask,
we're going to ask three core ones today

00:06:29.840 --> 00:06:33.357
throughout all the sessions are
what does this tech or what is this

00:06:33.440 --> 00:06:37.677
technology going to do next, right? What
happens? Where will it go if it fulfills

00:06:37.760 --> 00:06:41.917
its potential? If we don't know where
the technology is going to go next, we

00:06:42.000 --> 00:06:43.517
can't plan for what's going to happen.

00:06:43.600 --> 00:06:46.477
So I think at the core we have to figure
out where it's going next and we have

00:06:46.560 --> 00:06:50.237
the right people in the room here
to do it today. We also have to ask what

00:06:50.320 --> 00:06:53.677
happens if it works. If the technology
fulfills its potential, what happens

00:06:53.760 --> 00:06:56.317
next? And then of course what happens if

00:06:56.400 --> 00:06:59.357
it goes wrong? If all this investment

00:06:59.440 --> 00:07:04.157
ends up not panning out, what are the
downstream implications for technology,

00:07:04.240 --> 00:07:08.877
for our economy, and of course for the
foundational labs themselves? So we're

00:07:08.960 --> 00:07:10.957
going to do it today with a great lineup.
We're going to kick off with

00:07:11.040 --> 00:07:15.037
Aaron Levy, the CEO of Box, talking
about AI's most critical questions.

00:07:15.120 --> 00:07:18.397
We're going to then bring on the three
best AI infrastructure reporters in the

00:07:18.480 --> 00:07:22.077
world to talk about the AI buildout.
Uh after that, we're going to have Dallas

00:07:22.160 --> 00:07:27.437
Dolan, the TMT leader at PWC, talk
to us about what the real ROI is from AI

00:07:27.520 --> 00:07:29.997
tokens and what happens when you're
actually building with these things and

00:07:30.080 --> 00:07:34.717
what the business return is on that front.
And then Ron Roy will join me for

00:07:34.800 --> 00:07:38.317
a Q&A with you, right? Right. We're
going to do our Friday show on Thursday

00:07:38.400 --> 00:07:40.637
and we're going to encourage
your participation. So, if you have

00:07:40.720 --> 00:07:42.717
questions, bring them uh forward to us.

00:07:42.800 --> 00:07:46.557
We'll have a mic here and a mic there.
U we'll then have a coffee break. We have

00:07:46.640 --> 00:07:50.557
an espresso cart out back. Uh that was
very important for us in the planning.

00:07:50.640 --> 00:07:54.157
Um the fancier drinks, not during the
break because we want to keep it moving.

00:07:54.240 --> 00:07:56.637
So, if you want a fancier drink, you can

00:07:56.720 --> 00:07:59.197
go before or after. It's important that

00:07:59.280 --> 00:08:01.997
we give you this information. Uh we're

00:08:02.080 --> 00:08:06.397
then going to you know talk about the
whether uh the mythos situation in Fable

00:08:06.480 --> 00:08:10.077
whether whether there's actually a cyber
threat from these latest models or

00:08:10.160 --> 00:08:13.357
whether it's marketing and we're going
to joined by Alex Stamos the former

00:08:13.440 --> 00:08:15.997
chief security officer at Meta to do

00:08:16.080 --> 00:08:18.637
that. Um then we'll speak with Mike Kger

00:08:18.720 --> 00:08:20.957
the head of Anthropic Labs to talk about

00:08:21.040 --> 00:08:24.157
what building AI product is uh what how

00:08:24.240 --> 00:08:28.717
you build AI product natively today.
And then finally, Greg Brockman will close

00:08:28.800 --> 00:08:33.197
us out, the president and co-founder
of OpenAI, talking about that company's

00:08:33.280 --> 00:08:36.957
plans and what the frontier uh looks like.

00:08:37.040 --> 00:08:39.117
>> All right, everybody ready?

00:08:39.200 --> 00:08:44.237
>> All right, we put a lot
of work into this today. I

00:08:44.320 --> 00:08:46.797
promise you, we're going to do whatever
we can to make sure you have a great

00:08:46.880 --> 00:08:51.037
day. And thank you again for trusting
me with your time, trusting us with your

00:08:51.120 --> 00:08:54.717
time, and again filling those empty seats.
It's a real dream come true. So,

00:08:54.800 --> 00:08:59.880
thank you all. Let's hear
for you guys one more time.

00:09:01.840 --> 00:09:04.957
>> A quick adjustment here.
We want you sounding perfect.

00:09:05.040 --> 00:09:08.237
>> Okay. So, as this adjustment happens,
I'm going to invite our first guest.

00:09:08.320 --> 00:09:10.877
Aaron Levy is the CEO of Box. He's one

00:09:10.960 --> 00:09:15.517
of the most insightful and fun voices
on the future of this technology. He

00:09:15.600 --> 00:09:19.677
actually was the fourth guest ever on Big
Technology Podcast and the guest the

00:09:19.760 --> 00:09:23.837
only other time we did a live podcast
together. So, with that, I am thrilled

00:09:23.920 --> 00:09:29.160
to welcome Aaron Levy.
Please join me in welcoming him.

00:09:32.080 --> 00:09:33.197
>> All right. Great to see you.

00:09:33.280 --> 00:09:35.597
>> Good to see you. Thank you. Um I like

00:09:35.680 --> 00:09:38.077
that you set expectations with and some

00:09:38.160 --> 00:09:41.597
answers. Um uh so uh with that uh in

00:09:41.680 --> 00:09:45.997
your agenda. So I will try and provide
some some answers, not all the answers.

00:09:46.080 --> 00:09:49.437
>> I think we've done this. First of all,
every time we speak, I feel like I can't

00:09:49.520 --> 00:09:52.077
even get a question out and we're
already on to some of the

00:09:52.160 --> 00:09:53.437
>> Sorry. Can I just do my monologue now?

00:09:53.520 --> 00:09:56.957
Are you good? Okay. Thanks. Um >> I
I think it's I think we've done this

00:09:57.040 --> 00:10:00.557
enough to know that we are going to get
some answers uh to these questions and

00:10:00.640 --> 00:10:04.797
>> I'll do my best.
>> I think I don't want to start off by

00:10:04.880 --> 00:10:06.237
disparaging other interviews, but it

00:10:06.320 --> 00:10:08.957
doesn't doesn't always happen that way.

00:10:09.040 --> 00:10:12.717
>> Well, um you know,
we're we we we get to

00:10:12.800 --> 00:10:17.277
provide the answers because we're not
the lab that's being either regulated or

00:10:17.360 --> 00:10:20.637
dealing with lots of issues. So, I get
to just pontificate and there's no no

00:10:20.720 --> 00:10:23.597
consequence. So, it's great.
>> All right. So, we're going to ask you

00:10:23.680 --> 00:10:26.957
some of the questions about what's
happening. Um, le let's dive right into

00:10:27.040 --> 00:10:30.317
it. I'm going to start, you know, again,
since you're not at one of the labs,

00:10:30.400 --> 00:10:32.557
let's start to talk about the biggest

00:10:32.640 --> 00:10:34.717
most controversial moment now, uh, which

00:10:34.800 --> 00:10:37.837
is the anthropic fable situation. Um,

00:10:37.920 --> 00:10:42.077
let me put to you what I
call the Jassie mystery.

00:10:42.160 --> 00:10:44.797
So, what we know about >>
I'm sure he liked that name.

00:10:44.880 --> 00:10:46.957
>> Well, he didn't show up,
so we can talk about it in his

00:10:47.040 --> 00:10:52.957
>> absence. Okay. Uh so what we know about
the fable ban or the export controls on

00:10:53.040 --> 00:10:53.917
anthropic yeah

00:10:54.000 --> 00:10:57.277
>> is that Amazon found a vulnerability

00:10:57.360 --> 00:11:01.597
>> in the software and Andy Jasse maybe
made a call to Dario definitely made a

00:11:01.680 --> 00:11:04.957
call to the white house and then
very soon afterwards there were export

00:11:05.040 --> 00:11:08.717
controls that were put
on anthropics frontier model.

00:11:08.800 --> 00:11:12.077
>> Yeah, those two fact
patterns are probably not ideal.

00:11:12.160 --> 00:11:15.517
>> So the mystery is why did he do that?

00:11:15.600 --> 00:11:20.477
Yeah. And here's one hypothesis. This is
from Chimath. He said, "Google, Amazon,

00:11:20.560 --> 00:11:24.477
Microsoft, Meta now have a serious
non-zero opportunity to tank the

00:11:24.560 --> 00:11:28.557
Frontier Labs. Go to the government,
kneecap the lab's motion of putting the

00:11:28.640 --> 00:11:33.517
latest models out into the wild, become
the trusted gate gatekeeper between labs

00:11:33.600 --> 00:11:36.797
and the public by having
the labs go through their clouds."

00:11:36.880 --> 00:11:39.197
>> Okay. >> Plausible.

00:11:39.280 --> 00:11:42.717
>> Um I I would I would say um I mean,

00:11:42.800 --> 00:11:45.117
anything's plausible. I I prefer AAM's

00:11:45.200 --> 00:11:47.917
razor on this one. Um which is um you

00:11:48.000 --> 00:11:51.037
know ever since Mythos um you know

00:11:51.120 --> 00:11:55.117
Mythos very clearly was this
event that basically said

00:11:55.200 --> 00:11:58.077
you know AI is obviously getting
super powerful. It has all these risks

00:11:58.160 --> 00:12:01.517
associated with it. We are going
to give it to some you know small trusted

00:12:01.600 --> 00:12:04.157
partner network. They're going to go
evaluate their own tools. They're going

00:12:04.240 --> 00:12:07.917
to evaluate these capabilities. There's
been a lot of sort of it's a very kind

00:12:08.000 --> 00:12:11.677
of dramatic you know kind of roll out
of a technology. And I think what that has

00:12:11.760 --> 00:12:15.677
done is it's created this flywheel where
it almost incentivizes even more more

00:12:15.760 --> 00:12:19.677
drama and more research and more depth
in in uh you know security being the

00:12:19.760 --> 00:12:24.077
primary space in a way that we could
have already been doing since GPT4 if we

00:12:24.160 --> 00:12:26.877
wanted like you can go and deploy
these things to go find lots of

00:12:26.960 --> 00:12:30.557
vulnerabilities you can use them
offensively or defensively but mythos

00:12:30.640 --> 00:12:34.477
kind of created that extra air of of
seriousness and and uncertainty around

00:12:34.560 --> 00:12:38.397
it for good reason because it's
a incredibly powerful model so I go with a

00:12:38.480 --> 00:12:41.917
razor which is Amazon obviously has
security research teams they're like

00:12:42.000 --> 00:12:46.877
like any any company at that scale you
know we we we try and you know test and

00:12:46.960 --> 00:12:50.317
push the limits of models and in our
particular domain of use cases clearly

00:12:50.400 --> 00:12:53.197
at Amazon scale you have a very large
security team they're trying trying to

00:12:53.280 --> 00:12:57.117
jail break models all the time and so
almost by definition there's already a

00:12:57.200 --> 00:13:01.437
public private partnership on on all
forms of jailbreaking models you know

00:13:01.520 --> 00:13:04.557
trying to push them to the limits
as a part of that and especially with the

00:13:04.640 --> 00:13:09.117
surrounding atmosphere of mythos I think
it would be very natural for Andy to to

00:13:09.200 --> 00:13:11.437
either share that research or his
team to share that research and that

00:13:11.520 --> 00:13:15.277
escalates and then that that sort
of creates its own flywheel. But the idea

00:13:15.360 --> 00:13:18.557
that there's some kind of board
boardroom level, you know, sort of

00:13:18.640 --> 00:13:21.997
strategy meeting that says we we
now need to kind of like co-opt the

00:13:22.080 --> 00:13:25.277
technology become the only interface
to the government. This kind of puts us in

00:13:25.360 --> 00:13:27.757
the pole position. Um I think it's less

00:13:27.840 --> 00:13:31.197
likely that and more likely this is a uh

00:13:31.280 --> 00:13:33.757
this is a situation where the the mythos

00:13:33.840 --> 00:13:36.797
momentum continued. Fable obviously, you

00:13:36.880 --> 00:13:40.957
know, had ways of of getting back to kind
of the mythos level capability. Um,

00:13:41.040 --> 00:13:44.717
and researchers, you know, sort
of shared that information. And I don't I

00:13:44.800 --> 00:13:48.957
think there's like a basically,
you know, very limited small percentage

00:13:49.040 --> 00:13:52.877
chance that then Andy and team knew that
like the very next event would be they'd

00:13:52.960 --> 00:13:56.397
stop the model. Um, and and that's
not even good for Amazon like like

00:13:56.480 --> 00:13:59.917
strategically like Amazon makes plenty
of money the more that Fable gets used

00:14:00.000 --> 00:14:03.197
in the world. So I don't think you would
I don't think you would do some kind of

00:14:03.280 --> 00:14:05.997
like you know um kind of maneuvering to

00:14:06.080 --> 00:14:10.957
to to create this. So I I kind of just
go with this is it's a very chaotic kind

00:14:11.040 --> 00:14:14.317
of environment right now. The government
has you know only a few tools at their

00:14:14.400 --> 00:14:16.957
disposal at any given time to deploy
against these things. Those are going to

00:14:17.040 --> 00:14:20.797
be kind of blunt instruments. Um and uh
and this stuff is coming together very

00:14:20.880 --> 00:14:24.477
quickly because of you know in some
cases the lack of technical capability

00:14:24.560 --> 00:14:27.197
of the government compared to how
powerful these models are. It's like you

00:14:27.280 --> 00:14:29.677
don't you know when you see something
that seems very scary like oh my gosh

00:14:29.760 --> 00:14:33.117
the thing could be we can jailbreak
the model and get back to mythos level

00:14:33.200 --> 00:14:36.077
capability and mythos was the thing
we're supposed to be scared about then

00:14:36.160 --> 00:14:39.197
you know just like stop it like
that I think that's like a very natural

00:14:39.280 --> 00:14:43.597
reaction based on the atmosphere that
we've created in AI recently. So I just

00:14:43.680 --> 00:14:45.197
go with that as as the answer.

00:14:45.280 --> 00:14:48.397
>> I mean I like that you say
the atmosphere that we've created in AI

00:14:48.480 --> 00:14:50.957
lately. >> Everybody but me.
Yeah. Um well I mean

00:14:51.040 --> 00:14:55.277
as far as the company on the receiving
end of this though is anthropic and you

00:14:55.360 --> 00:14:58.237
know you talked about these mythical
capabilities they called the model

00:14:58.320 --> 00:15:01.997
mythos. They put in the documentation
that like it broke out of its

00:15:02.080 --> 00:15:06.397
containment and wrote the engineer while
he was having a sandwich in the park. Um

00:15:06.480 --> 00:15:08.957
is it that surprising that this
is one of the downstream impacts?

00:15:09.040 --> 00:15:11.437
>> Yeah I mean if you put that in your
announcement blog post you know people

00:15:11.520 --> 00:15:14.317
might be able to kind of extrapolate
and get pretty pretty pretty scared of

00:15:14.400 --> 00:15:18.957
things. I think it's interesting. So um
you know on the anthropic front first of

00:15:19.040 --> 00:15:20.557
all I have I have a huge amount of

00:15:20.640 --> 00:15:23.117
respect for the entire kind of stack of

00:15:23.200 --> 00:15:26.157
researchers and policy folks across AI.

00:15:26.240 --> 00:15:30.157
I happen to have disagreements with some
of the the categories but but I think

00:15:30.240 --> 00:15:32.157
there's a deep let's say if you were if

00:15:32.240 --> 00:15:34.077
you imagined a continuum of the most

00:15:34.160 --> 00:15:37.437
like you know if if you uh uh if you

00:15:37.520 --> 00:15:42.637
kind of had like like the most I mean
this only in like a polite way. It will

00:15:42.720 --> 00:15:45.757
sound impolite, but like I mean like
like if you're the most doomer on one

00:15:45.840 --> 00:15:50.157
end of the spectrum and the most like
like accelerationist on the other end of

00:15:50.240 --> 00:15:51.357
the spectrum. Here here's kind of the

00:15:51.440 --> 00:15:54.477
the views. The most doomer uh possible

00:15:54.560 --> 00:15:58.957
is was afraid of like GPD3 and GPD3 was

00:15:59.040 --> 00:16:02.317
going to like you know sort
of accelerate and and you know kind of

00:16:02.400 --> 00:16:06.237
achieve some kind of unstoppable
continual improvement. Um and you know

00:16:06.320 --> 00:16:09.517
the accelerationist says like we need
like fable 20 as soon as possible,

00:16:09.600 --> 00:16:13.517
right? So that's that's sort of the
continuum. I'm probably like I you know

00:16:13.600 --> 00:16:17.117
maybe twothirds up to the accelerationist
kind of side of things.

00:16:17.200 --> 00:16:20.797
But if you were on this on the doomer
and I I'm trying to say the polite

00:16:20.880 --> 00:16:23.837
version of doomer like you're deep
in AI safety. You're very scared of the

00:16:23.920 --> 00:16:26.957
technology. You think there's as much
likelihood of bad things happening as

00:16:27.040 --> 00:16:30.317
good things techn you know happening.
We have to win the race and control and

00:16:30.400 --> 00:16:32.957
kind of stamp down on the technology. We
don't want this to be this sort of thing

00:16:33.040 --> 00:16:36.157
that runs in the wild. If you're in
that end of the continuum the thing that

00:16:36.240 --> 00:16:37.517
happened this weekend is actually the

00:16:37.600 --> 00:16:40.397
best case scenario for you. So, so like

00:16:40.480 --> 00:16:45.757
you actually want there to be these
sort of like like valves and and like and

00:16:45.840 --> 00:16:48.397
like buttons in the government that just
is like we're just going to stop it.

00:16:48.480 --> 00:16:51.117
>> I mean, that's Dario's position. Do
you think he's happy with what's going on?

00:16:51.200 --> 00:16:52.637
>> Uh, you know, I'm not going to I won't

00:16:52.720 --> 00:16:56.157
try and guess uh any of that. Um uh but

00:16:56.240 --> 00:16:58.477
I I will just say if you had to

00:16:58.560 --> 00:17:00.957
establish a if you had to establish a

00:17:01.040 --> 00:17:05.597
regulatory regime that said we are going
to review models, we're going to push

00:17:05.680 --> 00:17:08.957
the limits of models and we're going
to have the ability to either roll back

00:17:09.040 --> 00:17:12.717
access to models or prevent their their
release in the first place. We want that

00:17:12.800 --> 00:17:17.997
to be a regulatory approach. You would
need an event like Fable to effectively

00:17:18.080 --> 00:17:21.677
create the precedent for that environment.
like like you're not going

00:17:21.760 --> 00:17:25.517
to wait for Congress to vote on this
being the new kind of you know process.

00:17:25.600 --> 00:17:28.797
You would you would kind of need
something that sort of shocks the system

00:17:28.880 --> 00:17:30.637
into that kind of regulatory framework.

00:17:30.720 --> 00:17:35.517
So all I'm saying is that if you're if
you were on this end of the continuum,

00:17:35.600 --> 00:17:39.837
this is actually an outcome that is sort
of almost desirable. Now maybe you you

00:17:39.920 --> 00:17:42.877
would wish that there would be more
a technical evaluation, more back and

00:17:42.960 --> 00:17:45.917
forth. Maybe you wish the policy people
were different on the other end. Who

00:17:46.000 --> 00:17:49.757
knows? But the idea that we now have
established that the government can

00:17:49.840 --> 00:17:51.517
press a button and prevent the roll out

00:17:51.600 --> 00:17:55.037
of AI um uh is is actually like a

00:17:55.120 --> 00:17:59.677
probably a positive update for an entire
cohort of of people. Now, unfortunately,

00:17:59.760 --> 00:18:03.117
I don't know that that any of your
guests represent that cohort, but I

00:18:03.200 --> 00:18:06.157
think you could easily get some people
that would be like this is the greatest

00:18:06.240 --> 00:18:09.997
thing that's ever happened in AI safety
because now we actually have we've

00:18:10.080 --> 00:18:13.277
created the case law essentially for this.
We we we now we know we know the

00:18:13.360 --> 00:18:17.037
tool exists. And then the next messy
process is when should we use the tool

00:18:17.120 --> 00:18:20.877
again? What should the real you know
kind of ongoing process look like? But I

00:18:20.960 --> 00:18:23.117
think that um you know probably I wish

00:18:23.200 --> 00:18:28.077
this wasn't the case but I think
practically in the next 3 to 5 years we

00:18:28.160 --> 00:18:30.797
probably have to end up in an
environment where models do get

00:18:30.880 --> 00:18:34.237
evaluated by the government. There is
a sort of collaborative approach between

00:18:34.320 --> 00:18:37.837
the government and research and and
the labs. You know the government has to

00:18:37.920 --> 00:18:41.677
kind of greenlight the the release
of the model. Um, I think it's probably

00:18:41.760 --> 00:18:45.117
become either too scary of a technology
or too economically powerful of a

00:18:45.200 --> 00:18:49.437
technology for governments to not want
to be in that position. I think that

00:18:49.520 --> 00:18:51.837
that that has massive implications when

00:18:51.920 --> 00:18:54.077
you when you kind of unpack it like just

00:18:54.160 --> 00:18:59.117
totally massive implications. One
being other countries now have far more

00:18:59.200 --> 00:19:02.317
incentive to stand up their own
sovereign AI initiatives. So, it's

00:19:02.400 --> 00:19:06.797
actually like maybe net negative for
the US economic position in AI that that

00:19:06.880 --> 00:19:10.317
this is the the outcome. Um, I think
somebody could take the other side and

00:19:10.400 --> 00:19:12.957
say, "No, we'll always have the most
powerful models, and so this puts us in

00:19:13.040 --> 00:19:15.517
the best position because now we can do
like horse trading with other countries

00:19:15.600 --> 00:19:19.677
of do you want access to our stuff."
So, I think it's it's, you know, I I like

00:19:19.760 --> 00:19:22.877
the fact that this is a super
interesting debate. Um, and I I have

00:19:22.960 --> 00:19:27.037
like a huge appreciation for every part
of the continuum um because I think it's

00:19:27.120 --> 00:19:30.637
like so intellectually interesting.
I still land on the hey, we probably want

00:19:30.720 --> 00:19:34.477
to treat this technology more as a
substrate technology and then regulate

00:19:34.560 --> 00:19:39.277
the applied use cases. So we should
regulate if you use AI to break into

00:19:39.360 --> 00:19:43.517
something. We should regulate if you
use AI to do bio, you know, kind of um

00:19:43.600 --> 00:19:46.397
research that that leads to dangerous
things. We shouldn't regulate the model

00:19:46.480 --> 00:19:49.357
itself. But I totally understand the other
views that are on the other end of

00:19:49.440 --> 00:19:52.637
this and and I think it's kind of very
natural with this important of a

00:19:52.720 --> 00:19:56.557
technology that it has to be somewhat of
a democratic process of how we decide to

00:19:56.640 --> 00:19:58.317
regulate it. >> Yeah.
You remember there were all those

00:19:58.400 --> 00:20:01.677
petitions six-month pause and everyone
kind of laughed at them. Yes.

00:20:01.760 --> 00:20:04.557
>> This is effectively the best
way to do that type of pause.

00:20:04.640 --> 00:20:09.037
>> Yeah. I mean this is if you were in the
P AI movement this is again like this is

00:20:09.120 --> 00:20:13.197
a great outcome. We we now have proven
how we can pause AAI. Um now it's an

00:20:13.280 --> 00:20:16.157
interesting kind of like mechanic
that they chose. It's sort of this export

00:20:16.240 --> 00:20:18.477
control thing, but effectively if you

00:20:18.560 --> 00:20:20.877
have an export control where non- US uh

00:20:20.960 --> 00:20:23.517
nationals can't can't use the technology

00:20:23.600 --> 00:20:26.157
like effectively that's PAI because your

00:20:26.240 --> 00:20:29.357
end API users of these of these models

00:20:29.440 --> 00:20:33.517
almost have no way to fully ensure
at all times that their end users don't

00:20:33.600 --> 00:20:36.317
sort of fall into some kind of you
know criteria that's off limits. So

00:20:36.400 --> 00:20:39.677
>> and there's there's already companies
that are pulling back. JP Morgan for

00:20:39.760 --> 00:20:42.637
instance has told its Hong
Kong users no more cloud

00:20:42.720 --> 00:20:45.197
>> right. So, so okay. So, now if you

00:20:45.280 --> 00:20:47.197
really war game this out like two to

00:20:47.280 --> 00:20:49.677
three, four more years out. This is um

00:20:49.760 --> 00:20:52.237
this is kind of interesting. So we we we

00:20:52.320 --> 00:20:55.437
have this like sovereign cloud um uh

00:20:55.520 --> 00:20:59.917
kind of comparison but but cloud you
know for better or worse basically

00:21:00.000 --> 00:21:04.237
became a commodity like whether you're
running in a cloud in uh you know

00:21:04.320 --> 00:21:06.877
there's like all lots of you know
performance implications like some some

00:21:06.960 --> 00:21:10.717
are faster some are cheaper but like
largely like you can get a web server

00:21:10.800 --> 00:21:13.357
you know built out wherever you are
in the world you can get storage built out

00:21:13.440 --> 00:21:16.637
wherever you are in the world we can
build sovereign clouds sovereign AI is a

00:21:16.720 --> 00:21:20.397
is a different kind of you know has
some intric intricacies uh that are

00:21:20.480 --> 00:21:24.317
different, right? Like like intelligence
just is not commoditized yet. We don't

00:21:24.400 --> 00:21:28.717
have everything having the same model
capability. So so the there's lots of

00:21:28.800 --> 00:21:32.397
really interesting implications which
is well what if like one set one country

00:21:32.480 --> 00:21:35.917
has access to you know frontier
intelligence before the other country

00:21:36.000 --> 00:21:38.557
you know what does that mean
geopolitically? What does that mean

00:21:38.640 --> 00:21:42.637
economically? Obviously now if you're
another country you have so much

00:21:42.720 --> 00:21:46.237
commercial incentive to make sure that
you can build out labs and have access

00:21:46.320 --> 00:21:51.117
to frontier intelligence as a kind
of hedge against the US. So who's a net

00:21:51.200 --> 00:21:55.517
winner in that? Probably China. Like and
so what's interesting is like you end up

00:21:55.600 --> 00:21:58.477
that I don't know if you know probably
most people saw the Doresh Jensen

00:21:58.560 --> 00:22:01.517
interview. And you can you can actually
it's a roarshack test. You can watch

00:22:01.600 --> 00:22:05.357
that through two totally different lenses.
You can have one lens which is

00:22:05.440 --> 00:22:07.197
which is like Dorash is totally right.

00:22:07.280 --> 00:22:10.557
We have this huge lead like this stuff
is so dangerous but if we control it

00:22:10.640 --> 00:22:13.517
then like we're going to control
everything. The other lens which is

00:22:13.600 --> 00:22:16.717
probably more of the Jensen angle
is like actually you know these other

00:22:16.800 --> 00:22:19.597
countries have a lot of incentive to also
get this right. And so even if it's

00:22:19.680 --> 00:22:23.357
like a $500 billion problem for them,
they just might deploy that much capital

00:22:23.440 --> 00:22:27.437
on this problem and and they will
eventually get it right. And so at the

00:22:27.520 --> 00:22:31.357
outcome actually we haven't gotten any
gains as better intelligence from the

00:22:31.440 --> 00:22:34.637
rest of the world. But what we have
lost is our economic superiority in this

00:22:34.720 --> 00:22:38.397
technology category because what we've
caused is a catalyst for all the other

00:22:38.480 --> 00:22:41.357
countries to have to build out their own
stack. And if they build out their own

00:22:41.440 --> 00:22:44.877
stack, it's probably going to be,
you know, chips from China, models from

00:22:44.960 --> 00:22:49.197
China, etc. which I don't have any
like you know you know reason to be be

00:22:49.280 --> 00:22:52.237
against other than just like I want
America to win the economic you know

00:22:52.320 --> 00:22:57.197
angles on this and so so this is sort
of this debate that happens uh on on like

00:22:57.280 --> 00:23:00.237
where should you apply export controls
and what are the implications of that

00:23:00.320 --> 00:23:04.557
downstream and even this week you know
post fable we see that you know you have

00:23:04.640 --> 00:23:06.957
models that are certainly not fable

00:23:07.040 --> 00:23:11.997
performance uh level but but opus 4.7
4.8 eight level uh which is a big update

00:23:12.080 --> 00:23:15.357
for a lot of people on what what is now
possible with with open weight models

00:23:15.440 --> 00:23:18.957
that we just didn't have you know
kind of visibility into you know before

00:23:19.040 --> 00:23:22.157
>> yeah I think you shared recently
that the openw weight model or open source

00:23:22.240 --> 00:23:24.637
models the capabilities are not that far

00:23:24.720 --> 00:23:29.197
away from the frontier and in fact like
as these models get smarter uh they're

00:23:29.280 --> 00:23:31.837
they're only they're almost going to
saturate with intelligence where there's

00:23:31.920 --> 00:23:35.197
not going to be such a big difference
between let's say the smartest open-

00:23:35.280 --> 00:23:38.717
source model and the frontier don't you
think and then so won't this push people

00:23:38.800 --> 00:23:42.397
to open source Well,
so the the big the big ongoing

00:23:42.480 --> 00:23:45.117
conversation and I think you have some
guests that that can really represent,

00:23:45.200 --> 00:23:49.357
you know, what what they're seeing on
the front lines is is do you have a sort

00:23:49.440 --> 00:23:53.917
of fast takeoff scenario of model
capability and progress um and with with

00:23:54.000 --> 00:23:57.997
some kind of continual learning kind of
you know self-improvement uh dynamic and

00:23:58.080 --> 00:24:01.437
then and then it stands to reason that
like the company with the most compute

00:24:01.520 --> 00:24:04.717
or the country with the most compute and
you get the fast takeoff you get sort of

00:24:04.800 --> 00:24:09.037
a a virtuous flywheel um that maybe
is sort of has some compounding you know

00:24:09.120 --> 00:24:12.157
benefit benefits to it that are just,
you know, unreachable by anybody else.

00:24:12.240 --> 00:24:16.477
That's a scenario. Another scenario
is is that that's another just incremental

00:24:16.560 --> 00:24:20.717
capability. Everybody kind of catches up
to it and you always have this sort of,

00:24:20.800 --> 00:24:25.037
you know, kind of two loops going at all
times and the with the closed the closed

00:24:25.120 --> 00:24:27.517
providers and the open providers
and they're and they're kind of always

00:24:27.600 --> 00:24:29.917
within 3 to six months of each other.

00:24:30.000 --> 00:24:34.237
The world is so different uh from a
market structure standpoint whether we

00:24:34.320 --> 00:24:38.877
end up in an outcome where we have sort
of an exponential progress in the in the

00:24:38.960 --> 00:24:43.117
models that you kind of continually
learn versus versus the closed source

00:24:43.200 --> 00:24:47.677
models uh and it's like a 5-year you
know gap in progress and that just goes

00:24:47.760 --> 00:24:51.677
again kind of exponential totally
different market structures um the the

00:24:51.760 --> 00:24:55.277
one that that where we have this
exponential progress is again it's

00:24:55.360 --> 00:24:57.837
probably actually net positive
for America in that case in which case the

00:24:57.920 --> 00:25:02.077
export controls probably worked Um,
it means our kind of top three, four

00:25:02.160 --> 00:25:04.557
labs have this incredible superiority.

00:25:04.640 --> 00:25:06.717
We control access to this technology.

00:25:06.800 --> 00:25:09.997
That's actually a good scenario. Like
like economically speaking, it might not

00:25:10.080 --> 00:25:13.917
be like a total net good scenario for
society, but it's good for the US. Let's

00:25:14.000 --> 00:25:16.957
just say that's one scenario. A lot
of people are betting that that is that

00:25:17.040 --> 00:25:20.637
that's where we're at with research.
The other scenario, and like China sits

00:25:20.720 --> 00:25:24.077
around uh and they probably bet on this
scenario, is no, we're going to be able

00:25:24.160 --> 00:25:27.597
to keep up. We're going to throw more
comput. We're going to get more data.

00:25:27.680 --> 00:25:30.397
we're going to build our own, you know,
flywheels. And it's always three months

00:25:30.480 --> 00:25:31.917
out and, uh, you know, kind of behind.

00:25:32.000 --> 00:25:35.437
And if it's three months behind and it's
an open weights provider, uh, that has

00:25:35.520 --> 00:25:38.397
more of a commoditization kind
of business model approach because they

00:25:38.480 --> 00:25:40.797
just want to, you know, sell more
infrastructure or chips or they just

00:25:40.880 --> 00:25:44.077
want to reduce our, you know,
superiority in the space, which is

00:25:44.160 --> 00:25:45.517
actually like strategic for China to do.

00:25:45.600 --> 00:25:47.917
Like they're everybody wonders like why
are they doing this open weight stuff?

00:25:48.000 --> 00:25:52.317
It's actually makes total sense. like
you're just reducing US's dominance in a

00:25:52.400 --> 00:25:55.757
in a field and it might be worth
a couple hundred billion dollars to do

00:25:55.840 --> 00:25:59.917
that for something that might be worth
you know 10 trillion dollars. So so if

00:26:00.000 --> 00:26:01.997
that keeps up because there is real

00:26:02.080 --> 00:26:07.517
economic advantage to to to doing so
then you have this new kind of you know

00:26:07.600 --> 00:26:11.357
sort of dynamic that plays out which
is maybe the layer of of of incremental

00:26:11.440 --> 00:26:15.197
value shift is effectively the applied
layer of AI. So, if you think about

00:26:15.280 --> 00:26:17.917
there's the lab layer and then there's
the applied layer, >> the cursors,

00:26:18.000 --> 00:26:23.277
>> the cursors, the the Harveys, the
the Sierras, the decagons, the boxes. Um,

00:26:23.360 --> 00:26:26.397
>> which is amazing because everyone
said they're just a thin wrapper on top of

00:26:26.480 --> 00:26:29.117
large language models, but now
maybe that's where the value comes.

00:26:29.200 --> 00:26:32.557
>> Yeah. So, so um and and you know,
it's one of these things which is like we

00:26:32.640 --> 00:26:33.997
just have to not be binary about it.

00:26:34.080 --> 00:26:37.757
like everything I'm saying I think
the frontier models still make way more

00:26:37.840 --> 00:26:41.197
money in the future than they do today
because what happens at the routing

00:26:41.280 --> 00:26:46.237
layer is you still sort of say hey I
want fable um or GPT5 you know five or

00:26:46.320 --> 00:26:49.677
whatever the next model will be I want
that to be the orchestrator I want I

00:26:49.760 --> 00:26:53.117
need like the super intelligence at
the orchestration layer and I need super

00:26:53.200 --> 00:26:57.037
intelligence at the review and sort
of like you know fix and check the work of

00:26:57.120 --> 00:27:00.957
the of the other agent and and so so
you have like a barbell you know maybe

00:27:01.040 --> 00:27:04.637
U-shaped model where you use frontier
intelligence but then ing in the middle.

00:27:04.720 --> 00:27:05.837
You can just say, "Nope, I'm going to

00:27:05.920 --> 00:27:08.717
take that to Neotron or or Kimmy 26 or

00:27:08.800 --> 00:27:13.597
GLM552 or whatever." And and then all
of a sudden it's like you have super high

00:27:13.680 --> 00:27:18.077
cost, you know, inference in one part
of the workload, super low cost, still

00:27:18.160 --> 00:27:21.677
pretty good inference in another part
of the workload, but who has the incentive

00:27:21.760 --> 00:27:26.237
to do that? It's the applied layer of AI
because because like the business model

00:27:26.320 --> 00:27:29.917
of the applied layer is obviously like
we our job is to you know give you the

00:27:30.000 --> 00:27:34.077
best model for the job not just
the model from just our lab. So it's cool

00:27:34.160 --> 00:27:38.477
because we actually now have a good kind
of push pull between Frontier Labs and

00:27:38.560 --> 00:27:42.557
the applied layer uh where where you
probably wouldn't want it to be that

00:27:42.640 --> 00:27:45.917
we're all sort of only in the orbit
of one or two companies you know

00:27:46.000 --> 00:27:48.157
commercially and economically.
You'd want you'd want to make sure that

00:27:48.240 --> 00:27:51.837
there's some good tension there. And so
I think that's kind of the direction

00:27:51.920 --> 00:27:56.477
things are things are headed between
kind of the token costs, the the open

00:27:56.560 --> 00:27:59.837
source models becoming so good and then
maybe even some of this regulatory

00:27:59.920 --> 00:28:03.837
dynamic. I think the applied layer sort
of incrementally gets gets more more of

00:28:03.920 --> 00:28:05.757
that opportunity which
is which is obviously great.

00:28:05.840 --> 00:28:09.837
>> So you've talked about open source and
you've just mentioned China. Um but what

00:28:09.920 --> 00:28:13.437
you can what can you
tell us about lashon fat?

00:28:13.520 --> 00:28:16.797
>> Um uh it's great memes. So folks, uh,

00:28:16.880 --> 00:28:19.917
Lashon Fat is a rumored open- source

00:28:20.000 --> 00:28:25.037
model from Mistral and has been
the subject of great fascination from the

00:28:25.120 --> 00:28:27.437
internet, wouldn't you say?
>> I there's great comedy. Uh,

00:28:27.520 --> 00:28:29.677
>> John, can we can we um show people what

00:28:29.760 --> 00:28:32.877
we're talking about? Let's roll image A.

00:28:32.960 --> 00:28:35.117
>> Uh, this is Lashon Fat,

00:28:35.200 --> 00:28:37.677
>> the number one model from Europe. Yes.

00:28:37.760 --> 00:28:40.317
>> My French world uh my French uh

00:28:40.400 --> 00:28:42.477
rudimentary French, it translates to the

00:28:42.560 --> 00:28:46.557
very fat kitten. Um, can we roll B? This

00:28:46.640 --> 00:28:50.477
is a standard day in Paris now.

00:28:50.560 --> 00:28:54.797
>> Yeah, but it does show something
that there's so much eagerness for AI that

00:28:54.880 --> 00:28:57.997
there's now fan art
for this potential model from MR.

00:28:58.080 --> 00:28:59.757
>> We we've reached we've reached a really

00:28:59.840 --> 00:29:02.317
important sort of phase in the cycle. Um

00:29:02.400 --> 00:29:06.237
uh so um uh so you know um I do think

00:29:06.320 --> 00:29:09.757
that it is kind of cool because what
some of the things that you maybe

00:29:09.840 --> 00:29:13.357
discounted the importance of all
of a sudden just like have so much more

00:29:13.440 --> 00:29:16.797
importance. Uh like I'm I'm watching
the um I don't know if folks are watching

00:29:16.880 --> 00:29:20.477
like the fireworks base 10 space as an
example. Like it's pretty cool that that

00:29:20.560 --> 00:29:24.157
that we now have these open weights
models that you can effectively post

00:29:24.240 --> 00:29:26.317
train on your particular domain of of

00:29:26.400 --> 00:29:29.197
task and you can go and you know eek out

00:29:29.280 --> 00:29:33.757
another five or 10 points of improve of
performance on these types of models. Um

00:29:33.840 --> 00:29:38.077
and again that's only possible because
of the misdraws because of the the the

00:29:38.160 --> 00:29:41.997
Chinese you know kind of open weights
models and the cost curve has gone down

00:29:42.080 --> 00:29:45.357
so much that there are actually some
situations which is oh actually maybe I

00:29:45.440 --> 00:29:49.277
should train a model just for my use
case because it it's literally like

00:29:49.360 --> 00:29:52.477
economically now it's not even like
I want control. It's it's actually

00:29:52.560 --> 00:29:56.717
economically advantageous for you to do
so. So, is this the answer to like the

00:29:56.800 --> 00:29:59.757
big token maxing hype where like
everyone's spending all this money on

00:29:59.840 --> 00:30:03.117
tokens and not really understanding
where they're going or whether there's

00:30:03.200 --> 00:30:05.997
an ROI that >> Yeah,
I mean I think that in practice

00:30:06.080 --> 00:30:10.077
that phase probably lasted two and a half
weeks. Um like like from the moment

00:30:10.160 --> 00:30:13.037
that meta took >> the media
over hype token max.

00:30:13.120 --> 00:30:17.037
>> No, I would never claim uh that.
Um we should go through your your various

00:30:17.120 --> 00:30:20.557
podcast headlines, but um uh the uh >>
we're not going to do that. We got we

00:30:20.640 --> 00:30:22.317
got the the cat pictures and that's it.

00:30:22.400 --> 00:30:26.397
Yeah. Um, no, but like I mean if I if
I had to like capture the the cycle of

00:30:26.480 --> 00:30:30.077
like the first token maxing, you know,
meta has a leaderboard, uses the most

00:30:30.160 --> 00:30:33.117
tokens possible to now, you know,
the last weeks of rumors of like we're

00:30:33.200 --> 00:30:36.477
shutting down everything, no one can use
AI. Um, you know, it's about a two-month

00:30:36.560 --> 00:30:40.557
period. So, it's, you know, people need
to like probably, you know, always kind

00:30:40.640 --> 00:30:44.717
of step back and just be like, okay,
are is what we're doing like a pragmatic,

00:30:44.800 --> 00:30:49.117
you know, thing for for for work or we
just sort of like like getting kind of

00:30:49.200 --> 00:30:53.357
hyped up too crazy on something. Um what
what's interesting is this this phase

00:30:53.440 --> 00:30:55.517
was so short that I don't ever think it

00:30:55.600 --> 00:30:57.757
reached outside of of the tech industry.

00:30:57.840 --> 00:31:02.637
I was um I I was uh I was at at we we
kind of host these CIO dinners in every

00:31:02.720 --> 00:31:06.157
city that we go to and I was uh we had
a dinner like within 3 days of like the

00:31:06.240 --> 00:31:08.397
token maxing like like initial spike on

00:31:08.480 --> 00:31:13.197
like Google trends like the word finally
emerged and like three people had heard

00:31:13.280 --> 00:31:15.997
about it and so I I feel
confident that it died

00:31:16.080 --> 00:31:17.997
>> right they haven't heard about because
their employees are outspending the

00:31:18.080 --> 00:31:21.357
tokens. >> Yeah fair point fair point. So,
but hopefully it will have completely died

00:31:21.440 --> 00:31:24.957
by the time it reaches reaches the rest
of the world and then we can just move

00:31:25.040 --> 00:31:26.797
to more normal environments and and but

00:31:26.880 --> 00:31:29.197
the the the thing that that is true of

00:31:29.280 --> 00:31:33.037
of the phenomenon is that these agents

00:31:33.120 --> 00:31:37.677
uh are just using hundreds of times more
tokens than they were before. And so,

00:31:37.760 --> 00:31:41.037
you know, when we launched our first
kind of AI use case within Box, our

00:31:41.120 --> 00:31:43.437
product, the average number of tokens

00:31:43.520 --> 00:31:45.357
that was was being used on a task was

00:31:45.440 --> 00:31:48.557
like 5,000 10,000 20,000 tokens. Now our

00:31:48.640 --> 00:31:53.597
latest agents might use a million tokens
or five million tokens on on executing a

00:31:53.680 --> 00:31:57.117
task. And so that that's you know
in some case that's a 100x increase uh in

00:31:57.200 --> 00:32:00.397
in number of tokens. And the reason for
that is obviously like what's happening

00:32:00.480 --> 00:32:03.757
is right as we solve one use case when

00:32:03.840 --> 00:32:07.757
you would think that we can drive down
the cost curve of that one use case. All

00:32:07.840 --> 00:32:11.837
of a sudden a model capability allows us
to now add another use case that's much

00:32:11.920 --> 00:32:16.157
harder. And then our appetite just grows
to solve harder and harder and harder

00:32:16.240 --> 00:32:19.437
problems. And so it's this funny thing
cuz people get confused. They say, "I

00:32:19.520 --> 00:32:22.957
thought AI was supposed to get
was supposed to be getting cheaper and and

00:32:23.040 --> 00:32:26.477
it's like yes, you can actually think
about it as cheaper if you looked at

00:32:26.560 --> 00:32:29.837
like the unit of intelligence.
The reason it's more expensive is because

00:32:29.920 --> 00:32:33.517
we're now taking on bigger tasks." And so
we're getting confused because we're

00:32:33.600 --> 00:32:37.117
like, why is this the one, you know,
tech trend that doesn't have sort of the

00:32:37.200 --> 00:32:40.157
Moors law phenomenon? It's because
actually, no, we're we're we're

00:32:40.240 --> 00:32:44.237
outrunning the improve the efficiency
improvements in our appetite for what

00:32:44.320 --> 00:32:47.597
these models can go and do. And so it's
actually what you need to do is have

00:32:47.680 --> 00:32:52.157
like a way to normalize the cost of of
of the tokens to the tasks that you can

00:32:52.240 --> 00:32:55.517
now deploy. And then if you look at that
then that starts to look cheaper on a

00:32:55.600 --> 00:32:59.197
per task basis. It's just again
our tasks are getting bigger or or more

00:32:59.280 --> 00:33:02.717
accurate or more effective. And uh
and that's going to happen for for quite

00:33:02.800 --> 00:33:05.037
some time. >> The reason
why token maxing took off as

00:33:05.120 --> 00:33:07.357
a concept is because people saw the

00:33:07.440 --> 00:33:09.677
exponential revenue right the fact that

00:33:09.760 --> 00:33:13.117
anthropic and open AI were at zero 2023.

00:33:13.200 --> 00:33:17.037
Now they're like gonna do $50 billion
this year at the very least. And so

00:33:17.120 --> 00:33:20.077
people are looking for an explanation
and either the question either the

00:33:20.160 --> 00:33:24.557
answer is this is real or some it's
somehow inflated and that's why people

00:33:24.640 --> 00:33:27.597
go to token maxing. So if I'm hearing
you're right, if I'm hearing you right,

00:33:27.680 --> 00:33:31.757
what you're saying is all this spend
is much more legit than some of the online

00:33:31.840 --> 00:33:36.717
discussion makes it out to be. Well,
uh I I think if I had to like officially

00:33:36.800 --> 00:33:41.437
provide my own takeaway uh
for my own point, um it would be

00:33:41.520 --> 00:33:44.717
it would it would sort of be
there's there's sort of always this

00:33:44.800 --> 00:33:48.317
experimentation phase of a new
technology and this happens to be a

00:33:48.400 --> 00:33:52.957
relatively expensive technology. So thus
the experimentation phase is expensive.

00:33:53.040 --> 00:33:57.597
Uh and then what will happen is a
enterprises will deploy AI um and then

00:33:57.680 --> 00:34:00.877
they'll sort of peel off they'll start
to see like where are the real use

00:34:00.960 --> 00:34:03.837
cases, where are the ones that aren't
as real. They'll wind down the ones that

00:34:03.920 --> 00:34:07.117
aren't as real. The ones that are real,
they'll then look at it and they'll say,

00:34:07.200 --> 00:34:10.957
"Is there a way to do it at a lower cost
once we understand it enough? Uh, or do

00:34:11.040 --> 00:34:13.997
we still need the frontier intelligence
for for everything we're doing?" And

00:34:14.080 --> 00:34:16.557
that's actually just like a pretty
normal, I think, process that that

00:34:16.640 --> 00:34:18.317
everybody's going through right now. Uh,

00:34:18.400 --> 00:34:20.797
but uh, but you know, I I I think about

00:34:20.880 --> 00:34:26.157
it like our engineering team, we are
not token maxers in the sense of like

00:34:26.240 --> 00:34:29.277
there's no leaderboard. We're not
incentivizing overuse of tokens. We're

00:34:29.360 --> 00:34:33.517
just saying use it as effectively as
possible to to get your work done faster

00:34:33.600 --> 00:34:37.597
and our growth rate of spend is
exponential and we're like totally happy

00:34:37.680 --> 00:34:41.837
about it. Like nobody internally
is other than like ah we got to like shift

00:34:41.920 --> 00:34:44.877
some things around and make sure we
plan for this even more next year. Like

00:34:44.960 --> 00:34:47.197
that's obviously a stressful
conversation but we're not stressed

00:34:47.280 --> 00:34:50.237
about the idea that we're spending on AI.
Like we're we're quite excited about

00:34:50.320 --> 00:34:53.677
the productivity gains that we get. And
so I think what's happening is is every

00:34:53.760 --> 00:34:56.477
enterprise is having to kind of go
through their own journey on that. like

00:34:56.560 --> 00:34:59.437
they're they're deploying it in some
teams and some teams are saying oh my

00:34:59.520 --> 00:35:02.397
gosh this is this is like the greatest
thing of all time and then other teams

00:35:02.480 --> 00:35:05.197
you kind of look at what they're doing
you can't see any kind of measurable

00:35:05.280 --> 00:35:08.237
improvement in the output of that
organization and so then you're like

00:35:08.320 --> 00:35:11.037
okay you know like maybe maybe it's not

00:35:11.120 --> 00:35:14.237
as effective there but I I am I I would

00:35:14.320 --> 00:35:18.397
say like I think it's very easy to kind
of capture one or two anecdotes and then

00:35:18.480 --> 00:35:20.717
and then kind of overextrapulate on the

00:35:20.800 --> 00:35:23.117
overall themes I would say the vast

00:35:23.200 --> 00:35:25.437
majority of the current agentic spend

00:35:25.520 --> 00:35:30.797
that's happening is sustainable because
because because partly because it's

00:35:30.880 --> 00:35:34.077
actually coming mostly from engineering
and engineering related related tasks

00:35:34.160 --> 00:35:38.237
and this is an audience that is kind
of technically capable of of you know

00:35:38.320 --> 00:35:40.957
determining whether they they like the
work you know product that's coming out

00:35:41.040 --> 00:35:45.037
of the AI um maybe as it gets to other
parts of knowledge work you know those

00:35:45.120 --> 00:35:48.477
people will not be as sort of familiar
with how to do the ROI measurement and

00:35:48.560 --> 00:35:52.557
then it'll get even messier but like so
far I think it's actually been largely

00:35:52.640 --> 00:35:55.517
you know totally reasonable >> okay
uh we have a couple minutes left.

00:35:55.600 --> 00:35:57.437
Let's do a small lightning round. Oh, no.

00:35:57.520 --> 00:36:01.037
>> Um, >> so, uh,
my first take here is that that

00:36:01.120 --> 00:36:03.277
series is really good.
It's going to be really good now.

00:36:03.360 --> 00:36:05.437
>> Yeah. >> What do you think?

00:36:05.520 --> 00:36:09.197
>> Uh, no. I agree. >> What? >> Elaborate.

00:36:09.280 --> 00:36:13.197
>> Oh, is it lightning round or is it
uh like you want to hear a fiveminute

00:36:13.280 --> 00:36:15.917
answer round? >> Give like a 60sec answer.

00:36:16.000 --> 00:36:18.717
>> I mean, what could be easier than
pressing a button on your phone and

00:36:18.800 --> 00:36:23.117
talking to it? And if you you know
at least based on the announcement they've

00:36:23.200 --> 00:36:27.117
taken Gemini uh which is a very good
model and been able to I don't know if

00:36:27.200 --> 00:36:30.877
it's fork or distill or something
within there is is sort of Gemini grade

00:36:30.960 --> 00:36:34.397
intelligence. So if you get Gemini grade
intelligence and voice on your phone

00:36:34.480 --> 00:36:38.957
press a button I think you're just going
to use that for a lot of things. Um and

00:36:39.040 --> 00:36:42.237
uh and then you know I think
the exciting thing is like imagine that

00:36:42.320 --> 00:36:45.597
hooked up to various apps on your phone
and you're like hey order this thing for

00:36:45.680 --> 00:36:50.397
me or you know you know you know go
and add this calendar entry like I think

00:36:50.480 --> 00:36:54.557
those are very plausible daily use
cases that that we will have. Um and it's

00:36:54.640 --> 00:36:57.677
exactly the sweet spot for Apple
to to kind of own that space.

00:36:57.760 --> 00:36:59.437
>> Yeah. No, I think Apple did it finally.

00:36:59.520 --> 00:37:00.797
That was that was good.
That was a good time.

00:37:00.880 --> 00:37:03.517
>> Are you going to tell me if my answer
is right at the end of each one?

00:37:03.600 --> 00:37:05.757
>> Okay. >> That's Yeah,
this is good. Okay. Okay.

00:37:05.840 --> 00:37:10.157
So we agree one for one. Okay. Um,
how about this one? Permanent underclass.

00:37:10.240 --> 00:37:12.477
>> Uh, I don't like this one. This one I

00:37:12.560 --> 00:37:15.677
don't like at all. Um, uh, not only do I

00:37:15.760 --> 00:37:19.997
disagree with it, but I I I think it's
just like a bad meme to have in the

00:37:20.080 --> 00:37:23.277
atmosphere. I think it's like not good
for college students coming into the

00:37:23.360 --> 00:37:25.677
into the workforce, uh, of having so

00:37:25.760 --> 00:37:30.317
much stress, uh, about, you know, what
company to join and and what's going to

00:37:30.400 --> 00:37:34.077
what's going to, you know, kind of play
out. I do think companies actually do

00:37:34.160 --> 00:37:37.837
the job market a disservice though
by not being as clear on their own

00:37:37.920 --> 00:37:41.757
philosophies on this. Um, which some
of it is reasonable because it's like, oh

00:37:41.840 --> 00:37:44.637
man, we're just like, we're getting
thrown through a loop. There's so much

00:37:44.720 --> 00:37:47.837
innovation. But I do think that companies
like need to be somewhat clear

00:37:47.920 --> 00:37:49.517
on, hey, here's how we want to use AI.

00:37:49.600 --> 00:37:53.757
We want to use AI to accelerate
our our work or accelerate our technical

00:37:53.840 --> 00:37:58.397
innovation or accelerate our ability
to hit customers. um versus, you know, no,

00:37:58.480 --> 00:38:01.997
we're we're actually like our metric
is as few employees as possible, you know,

00:38:02.080 --> 00:38:05.997
with AI, like you kind of do want to,
you know, be able to have some stance.

00:38:06.080 --> 00:38:07.357
And I think companies have been very

00:38:07.440 --> 00:38:10.157
confused. Um and that lets this meme

00:38:10.240 --> 00:38:13.197
somewhat persist uh uh you know, for uh

00:38:13.280 --> 00:38:15.757
for for for you know, the internet.

00:38:15.840 --> 00:38:17.357
>> Okay, I won't rate that one. Thank you.

00:38:17.440 --> 00:38:19.837
>> Um all right, last one.
Is is the SpaceX

00:38:19.920 --> 00:38:23.677
performance good or bad news
for OpenAI and Anthropic?

00:38:23.760 --> 00:38:27.437
>> Oh, well, it's obviously good news. Um,
>> you don't think Elon took some of their

00:38:27.520 --> 00:38:31.037
money because he pitched the market on
an AI company and that's where the money

00:38:31.120 --> 00:38:34.157
got funneled into? >> I'm not sure.
I've seen like a limit of

00:38:34.240 --> 00:38:37.917
appetite of of if for for I mean there

00:38:38.000 --> 00:38:41.437
there's there's a literal limit of money
in the world, but I don't know that I

00:38:41.520 --> 00:38:45.677
don't know that that is zero sum at this
stage. Uh so I I think um I think people

00:38:45.760 --> 00:38:50.637
are pretty clear that that you know if
the revenue of this entire category of

00:38:50.720 --> 00:38:54.077
the frontier models and the
infrastructure stack is is measured in

00:38:54.160 --> 00:38:58.557
the trillions then you can have you know
20 companies that that all take a piece

00:38:58.640 --> 00:39:02.557
of that at different layers of the stack.
So So I'm I'm not I'm not sure I

00:39:02.640 --> 00:39:04.477
I would be convinced
that that would be zero sum.

00:39:04.560 --> 00:39:06.717
>> Did you buy SpaceX? >> I actually did.

00:39:06.800 --> 00:39:11.917
>> Okay. Um, not, you know, uh, I I'm uh
I don't know if I'm embarrassed or not,

00:39:12.000 --> 00:39:15.117
but I'm not gonna say the amount of
shares, but I I wanted to like be a part

00:39:15.200 --> 00:39:17.837
of the movement. So, I'm on Robin Hood

00:39:17.920 --> 00:39:20.557
buying my my my retail shares of uh of

00:39:20.640 --> 00:39:22.957
SpaceX. I'm I'm up like 15 bucks now.

00:39:23.040 --> 00:39:26.957
Um, per share. Per share, but uh uh so

00:39:27.040 --> 00:39:30.877
I'm I'm happy. Yeah. >> Amazing.
Well, Erin, you know, you uh

00:39:30.960 --> 00:39:34.957
you answered my email uh when we were
just at the very start of this podcast,

00:39:35.040 --> 00:39:37.677
four episodes in, >> came
on the show. I feel like every

00:39:37.760 --> 00:39:40.397
single time we talk,
something crazy is happening.

00:39:40.480 --> 00:39:43.757
>> That's a guarantee at this point. So,
>> boy, are we in the thick of it, right?

00:39:43.840 --> 00:39:46.877
>> Yeah. Awesome. Good to see you,
>> sir. Thank you so much, Aaron. Thank

00:39:46.960 --> 00:39:50.120
you, everybody.

00:39:54.960 --> 00:40:00.157
>> Are we off to a good start?
Let's hear it.

00:40:00.240 --> 00:40:01.837
>> Let's hear it one more time for Aaron.

00:40:01.920 --> 00:40:05.160
We're gonna

00:40:07.200 --> 00:40:10.717
We uh we need your energy here for a
couple of reasons. First of all, we're

00:40:10.800 --> 00:40:15.277
live streaming this. We want to make
sure that your presence here is felt in

00:40:15.360 --> 00:40:19.357
our live stream. Um and then we go
from like two chairs to four chairs. So,

00:40:19.440 --> 00:40:23.837
we're just going to need you guys
to help us fill that time. Um so, uh AI

00:40:23.920 --> 00:40:26.957
infrastructure, right? We're in the
middle of the greatest infrastructure

00:40:27.040 --> 00:40:31.357
buildout of all time, bigger than cable,
bigger than the railroads. Um, we're

00:40:31.440 --> 00:40:35.837
going to see $700 billion in capital
expenditures this year. And that means

00:40:35.920 --> 00:40:39.357
that some of the biggest questions
that we have about this AI moment are

00:40:39.440 --> 00:40:43.517
actually things that we can find out
from the infrastructure discussion

00:40:43.600 --> 00:40:47.277
alone. Questions like, are we
overbuilding? Will these data centers

00:40:47.360 --> 00:40:49.837
that are announced ever get stood up?

00:40:49.920 --> 00:40:53.997
And of course, will all this added
compute actually lead to better AI

00:40:54.080 --> 00:40:58.477
models or is it misguided? And so to do
it, to have this discussion, we're going

00:40:58.560 --> 00:41:02.077
to speak with three of the best AI
infrastructure reporters in the world.

00:41:02.160 --> 00:41:04.717
Ana Gardez from the information, Max

00:41:04.800 --> 00:41:07.037
Churnney from Reuters, and Lauren Good

00:41:07.120 --> 00:41:12.680
of Wired. Let's give it up
for Max, Ana, and Lauren.

00:41:18.000 --> 00:41:21.037
>> Hey guys. >> Hi.
Hey, >> hey, hi everyone.

00:41:21.120 --> 00:41:23.517
>> So, um, let's start here. There's there

00:41:23.600 --> 00:41:27.037
have been headlines that, um, of the

00:41:27.120 --> 00:41:29.597
announced AI data centers that are

00:41:29.680 --> 00:41:32.077
supposed to come up, something like 50%

00:41:32.160 --> 00:41:36.557
of them are actually being built. Um, is

00:41:36.640 --> 00:41:40.957
that the case? And if so, why?
Ana, do you want to lead us off?

00:41:41.040 --> 00:41:42.957
>> Sure. I totally believe that statistic

00:41:43.040 --> 00:41:44.957
and I think it might actually be higher

00:41:45.040 --> 00:41:48.237
if you include announcements. Um because

00:41:48.320 --> 00:41:52.157
of all the planned data centers that are
underway, I think it's highly likely

00:41:52.240 --> 00:41:56.877
that many will be delayed due to higher
costs, how hard it is to get labor. But

00:41:56.960 --> 00:42:01.037
then the number that I'm keeping close
track of is announced projects versus

00:42:01.120 --> 00:42:03.037
actually projects that are being built.

00:42:03.120 --> 00:42:06.237
And I think we've seen some, you know,
pretty crazy announcements from

00:42:06.320 --> 00:42:11.197
companies like OpenAI with all of these
different 10 gawatt, 6 gawatt projects.

00:42:11.280 --> 00:42:14.877
And those are numbers that I'm paying
a lot of attention to because I think we

00:42:14.960 --> 00:42:18.317
need to sort of back into them and say,
okay, if you wanted 10 gigawatts by this

00:42:18.400 --> 00:42:21.997
date, how many do you have today, was
that a real commitment? How firm is that

00:42:22.080 --> 00:42:26.077
commitment? But on on projects that are
actually getting built, I do think, you

00:42:26.160 --> 00:42:29.677
know, 50% not really getting done
on time is is, you know, what I would

00:42:29.760 --> 00:42:33.437
expect. Wait, what percentage would you
say have been announced but not start

00:42:33.520 --> 00:42:36.237
not included in that 50% number?

00:42:36.320 --> 00:42:40.237
>> Um, you know, when I think of that
number, it's I mean, it's quite high.

00:42:40.320 --> 00:42:45.197
Like even if you think of OpenAI, for
example, announcing a 10 gawatt project.

00:42:45.280 --> 00:42:49.517
Um, you know, we're not going to see 10
gigawatts in the next couple of years or

00:42:49.600 --> 00:42:52.637
they're going to sort of spread out
that bet. And I think that's a really good

00:42:52.720 --> 00:42:57.037
area area for reporters to look at is if
you just back into announcements, it's a

00:42:57.120 --> 00:43:00.237
really good way to say, "Hey,
this project is not not on track."

00:43:00.320 --> 00:43:03.117
>> Yeah, it's kind of crazy because if
you look at the way that stocks have been

00:43:03.200 --> 00:43:06.717
traded publicly, a lot of the market
action that we're seeing is entirely

00:43:06.800 --> 00:43:10.077
dependent on those
announcements coming true.

00:43:10.160 --> 00:43:15.437
>> So, Max, let's go to you. Um, what
are the consequences going to be if this

00:43:15.520 --> 00:43:18.637
these announced buildouts
don't materialize?

00:43:18.720 --> 00:43:23.677
Well, I think there's uh a lot
of shareholders of these public companies

00:43:23.760 --> 00:43:24.957
that are going to be pretty frustrated.

00:43:25.040 --> 00:43:28.477
Um Microsoft, Amazon, etc. have been

00:43:28.560 --> 00:43:32.157
making big CACback spats as I think
everybody in this room knows. They've

00:43:32.240 --> 00:43:35.517
been going to the market to get debt
now, which is, you know, you know, I

00:43:35.600 --> 00:43:39.037
think shareholders would be pretty
interested. Uh in terms of the

00:43:39.120 --> 00:43:42.797
consequences though, I mean, I'm a chip
reporter, so that's sort of where I

00:43:42.880 --> 00:43:47.837
that's how I kind of think about it.
Um and it's very likely I would say that

00:43:47.920 --> 00:43:51.437
we're going to lead we're going to see
some overcapacity uh essentially. So

00:43:51.520 --> 00:43:53.757
like people are building especially
memory companies are building tons of

00:43:53.840 --> 00:43:57.117
factories right now and typically
the chip industry is again I'm sure

00:43:57.200 --> 00:44:02.157
everybody in this this room knows uh
tends to be cyclical. So the whiplash

00:44:02.240 --> 00:44:06.077
from this one might be pretty bad um
depending on exactly when it comes to an

00:44:06.160 --> 00:44:08.397
end if it does. >> I think
what Max is saying is that it's

00:44:08.480 --> 00:44:12.477
good for his job security because
the more news there is the more he'll have

00:44:12.560 --> 00:44:15.837
to report on. Yeah. Oh, you guys are not
going to get a break anytime soon. Not

00:44:15.920 --> 00:44:18.637
at all. >> No, I can't I'm
very I'm very curious if

00:44:18.720 --> 00:44:22.477
the if there is a bust what that looks
like exactly and what precipitates it. I

00:44:22.560 --> 00:44:23.757
think it'll be a lot of fun to cover.

00:44:23.840 --> 00:44:28.717
>> Well, one of the big questions >> fun.
I also think if I can say too, I

00:44:28.800 --> 00:44:33.037
think this is a particularly unique
time because not only do we have these uh

00:44:33.120 --> 00:44:36.877
incredibly highly valued, ambitious
frontier labs uh that are getting

00:44:36.960 --> 00:44:40.317
involved in these circular deals,
but also we're in the middle of a memory

00:44:40.400 --> 00:44:44.237
shortage. um there's this unrelenting
demand for compute and also this is a

00:44:44.320 --> 00:44:45.997
midterm election year and so I think

00:44:46.080 --> 00:44:47.597
you're going to see a lot of uh

00:44:47.680 --> 00:44:51.277
politicizing of the data centers too as

00:44:51.360 --> 00:44:54.797
uh you know sort of lawmakers try to

00:44:54.880 --> 00:44:58.637
appeal to their base because a lot
of people are very unhappy about data

00:44:58.720 --> 00:45:01.757
centers. >> Yeah.
I mean they're not pulling well at all.

00:45:01.840 --> 00:45:04.717
>> No. So um you know when you think about

00:45:04.800 --> 00:45:09.117
where the collapse might happen um
a popular thing for people to discuss is

00:45:09.200 --> 00:45:11.517
well maybe Nvidia which has been making

00:45:11.600 --> 00:45:16.717
such premiums on its hardware um it
can't sustain it or it gets caught. So

00:45:16.800 --> 00:45:18.397
Lauren you spent a lot of time with uh

00:45:18.480 --> 00:45:21.277
Jensen Wong from Nvidia um what do you

00:45:21.360 --> 00:45:24.717
think about Jensen would enable him to

00:45:24.800 --> 00:45:29.037
sustain uh Nvidia's lead or do you
think that some of these skeptics have a

00:45:29.120 --> 00:45:33.117
point? Uh yes and yes I think some of the

00:45:33.200 --> 00:45:34.797
skeptics absolutely have a point and I

00:45:34.880 --> 00:45:38.157
think once you reach the uh the sort of

00:45:38.240 --> 00:45:42.557
uh is it the zenith is there the nadar
I always get those two confused that

00:45:42.640 --> 00:45:46.797
Nvidia has um you know people are always
sort of looking to to take you down a

00:45:46.880 --> 00:45:49.517
peg and compete but I think

00:45:49.600 --> 00:45:52.957
>> Nvidia and Jensen has been incredibly

00:45:53.040 --> 00:45:57.517
good in Nvidia's history at sort
of pivoting the company at exactly the

00:45:57.600 --> 00:46:00.317
moment that they need to in order
to make sure that they've sort of caught

00:46:00.400 --> 00:46:02.877
the next wave. Um, and we certainly saw

00:46:02.960 --> 00:46:05.597
that happen with um, not only like you

00:46:05.680 --> 00:46:07.677
know the GPU to begin with and parallel

00:46:07.760 --> 00:46:10.077
processing, but then again with sort of

00:46:10.160 --> 00:46:14.957
pivoting towards crypto which ultimately
meant they were in good place for AI and

00:46:15.040 --> 00:46:18.797
now we see the company doing that by
addressing the inference market a lot

00:46:18.880 --> 00:46:22.157
more closely too and you know Jensen
coming out and making these big

00:46:22.240 --> 00:46:25.197
proclamations that actually they're
the biggest CPU maker in the world which I

00:46:25.280 --> 00:46:29.357
know Intel and AMD must be thrilled about.
So, uh, you know, I think he's

00:46:29.440 --> 00:46:32.797
very smart and very strategic and
that there's a good chance that they do

00:46:32.880 --> 00:46:35.597
maintain their dominance. >> I mean,
I think I think the question is

00:46:35.680 --> 00:46:38.557
like how much of the inference market
they're going to get. I mean, it's it's

00:46:38.640 --> 00:46:42.957
whether it's 80% or 40% or somewhere
in between or or 90%. I mean, I think

00:46:43.040 --> 00:46:45.357
that's what everybody's fighting out
at the moment. I don't think there's a

00:46:45.440 --> 00:46:48.557
question that they're going to have
some big chunk of it. Just it's just how

00:46:48.640 --> 00:46:50.957
much, you know? >> Yeah.
Yeah, I'm going to go to An in a

00:46:51.040 --> 00:46:54.237
moment, but uh Lauren, I just want you
to tell us a little bit of uh what it

00:46:54.320 --> 00:46:58.637
was like uh with Jensen
on a cover shoot for Wired.

00:46:58.720 --> 00:47:00.717
>> You're talking about
when I brushed his hair.

00:47:00.800 --> 00:47:04.957
>> That would be it. >> Yeah. Okay.
Uh so, >> wait, you actually No,

00:47:05.040 --> 00:47:08.077
>> I really did. >> Yeah.
Yeah. I'll be brushing I'll be

00:47:08.160 --> 00:47:11.837
lining up later to brush people's hair
if anyone would like to. Uh yeah, so I

00:47:11.920 --> 00:47:16.237
did a cover story on Jensen for Wired
a couple years ago and uh our Wired art

00:47:16.320 --> 00:47:19.597
department is worldclass by the way.
And so we had this big photo shoot set up

00:47:19.680 --> 00:47:23.277
and I asked if I could tag along to
the photo shoot because I just wanted 15

00:47:23.360 --> 00:47:27.037
more minutes with him to ask him some
follow-up questions about Blackwell. And

00:47:27.120 --> 00:47:30.797
so they let me tag along down to the
office in Santa Clara, set up this whole

00:47:30.880 --> 00:47:35.357
set for Jensen, who had like, you know,
exactly two minutes to give us. And as

00:47:35.440 --> 00:47:38.877
the photographer was looking
at Jensen under the bright lights, the

00:47:38.960 --> 00:47:40.877
photographer said, "Oh, he's got some

00:47:40.960 --> 00:47:43.917
flyaways and that like gorgeous silver

00:47:44.000 --> 00:47:46.237
hair that he has." And then he said,

00:47:46.320 --> 00:47:48.717
"Does anyone have a brush?"

00:47:48.800 --> 00:47:51.757
Silence across the set. No one at NVIDIA

00:47:51.840 --> 00:47:55.837
apparently had a brush. I was like,
"Okay." Uh, and as anyone who has long

00:47:55.920 --> 00:47:59.677
hair knows, you always have a brush.
So I said, "Well, I have a brush." So then

00:47:59.760 --> 00:48:01.437
I went to my backpack and got the brush.

00:48:01.520 --> 00:48:04.717
And then I said, "Okay, here's the brush."
And no one moved. And meanwhile,

00:48:04.800 --> 00:48:07.117
Jensen is like, "What are we doing here,

00:48:07.200 --> 00:48:12.077
folks?" And I I said, "Oh, okay." So,
I walked up to Jensen and I took out my

00:48:12.160 --> 00:48:14.637
brush and I got really close to him
and I said, "Jensen, we're about to get a

00:48:14.720 --> 00:48:17.197
lot more close." And he said, "Oh,

00:48:17.280 --> 00:48:21.917
Christ." And um and then I brushed his
hair and I have to say, I did bring some

00:48:22.000 --> 00:48:24.237
evidence of this.
I think it looks great, frankly.

00:48:24.320 --> 00:48:29.997
>> Oh, wow. I think it turned out really
well. So, yeah. Now, the unfortunate the

00:48:30.080 --> 00:48:32.237
unfortunate conclusion to this story
very quickly because I know we need to

00:48:32.320 --> 00:48:36.077
move on is that people were joking
afterwards like you know the net worth

00:48:36.160 --> 00:48:39.597
of every individual strand of hair
on that brush, right? Like like you could

00:48:39.680 --> 00:48:43.117
sell this on eBay and which I was
obviously not going to do as an ethical

00:48:43.200 --> 00:48:48.157
journalist. But then like several months
later, my back my backpack was in my it

00:48:48.240 --> 00:48:52.397
was my gym bag. It was in the back
of my car and um downtown San Francisco, I

00:48:52.480 --> 00:48:53.757
think you know where this is going. My

00:48:53.840 --> 00:48:56.797
car got broken into and the brush got

00:48:56.880 --> 00:48:59.837
stolen. So those thieves have no idea

00:48:59.920 --> 00:49:03.197
the value of what they got away with.

00:49:03.280 --> 00:49:05.277
>> They are cloning Jensen as we speak.

00:49:05.360 --> 00:49:09.517
>> Yes. I'll see you in the back
afterwards if you on your hairbrushed.

00:49:09.600 --> 00:49:12.957
>> Ana, can you talk to us a little bit
about what we've kind of hinted at at

00:49:13.040 --> 00:49:16.717
this point that there's this, you know,
Nvidia, the common p perception on

00:49:16.800 --> 00:49:20.557
Nvidia is their GPUs were great
for training, but you can actually do the

00:49:20.640 --> 00:49:24.637
inference or the act of using a model
on a variety of of different chips. And

00:49:24.720 --> 00:49:28.317
once people once these labs train
their models and they're happy with their

00:49:28.400 --> 00:49:30.477
models, most of the computing is going

00:49:30.560 --> 00:49:35.437
to go to these inference chips. Um,
and therefore, even if Nvidia has a bet

00:49:35.520 --> 00:49:39.677
there, they're not going to be able to
sustain their dominance. Is is that what

00:49:39.760 --> 00:49:40.957
do you think about that? Is that a

00:49:41.040 --> 00:49:43.917
potential flaw in the armor for Nvidia?

00:49:44.000 --> 00:49:48.157
I think it is a flaw and I think there's
if anyone's going to sort of attack

00:49:48.240 --> 00:49:51.997
Nvidia's dominance they're going to do
it on inference like you said and there

00:49:52.080 --> 00:49:53.837
is sort of a massive effort underway

00:49:53.920 --> 00:49:56.317
right now to make all inference chips

00:49:56.400 --> 00:50:00.957
under the sun work well and so every
single company that buys Nvidia chips

00:50:01.040 --> 00:50:04.637
and is spending a lot of money on NVIDIA
chips is trying really hard to make

00:50:04.720 --> 00:50:08.557
these other other chips work whether
it's in-house chips from Google or

00:50:08.640 --> 00:50:12.877
Amazon or even in OpenAI's case and
potentially Anthropic's case you know do

00:50:12.960 --> 00:50:17.277
we make our own inference chip.
Um, so I'd have a hard time, you know,

00:50:17.360 --> 00:50:21.197
believing that none of those chips are
going to pan out, but you know, they

00:50:21.280 --> 00:50:25.837
might not tackle the bulk of the inference
workload even. But then again,

00:50:25.920 --> 00:50:29.197
even if they do 10% of your inference,
maybe you're saving enough money that

00:50:29.280 --> 00:50:32.557
you think the effort is worth it and it
gives you negotiating leverage with

00:50:32.640 --> 00:50:36.557
NVIDIA. If they know that you have
an in-house ship team, you know, Jensen's

00:50:36.640 --> 00:50:38.797
going to be a little bit worried when
negotiating with you when you're playing

00:50:38.880 --> 00:50:41.437
hard ball with him. So, I think
everyone's going to have to have an

00:50:41.520 --> 00:50:45.997
inference chip answer to Nvidia.
But when I talk to data center companies

00:50:46.080 --> 00:50:49.597
about what they're seeing, um, you know,
some data center companies don't really

00:50:49.680 --> 00:50:53.117
care what chip goes inside of their data
center, but they they try to get hints

00:50:53.200 --> 00:50:56.237
from the companies that they're working
with. Um, on the data center side,

00:50:56.320 --> 00:51:00.317
they're seeing a lot of NVIDIA
and in the instances that I'm seeing where

00:51:00.400 --> 00:51:03.357
there are non- NVIDIA chips, it's
because of some sort of financial backs

00:51:03.440 --> 00:51:06.957
stop. So, I do wonder how much how long
that will have to keep being the case

00:51:07.040 --> 00:51:11.037
because that will definitely hinder
non- Nvidia chips if they need special

00:51:11.120 --> 00:51:13.997
financing arrangements
to get inside the data center.

00:51:14.080 --> 00:51:15.837
>> Am I wrong in thinking the AI world is

00:51:15.920 --> 00:51:17.997
sort of separating on two poles? There's

00:51:18.080 --> 00:51:21.917
the Nvidia OpenAI pole and the anthropic

00:51:22.000 --> 00:51:25.197
Google Amazon poll, right? So, it's

00:51:25.280 --> 00:51:28.237
almost two separate ecosystems competing

00:51:28.320 --> 00:51:33.757
with each other. I think OpenAI is
investing a lot in non- Nvidia hardware.

00:51:33.840 --> 00:51:35.837
>> Are they trying
to move away from Nvidia?

00:51:35.920 --> 00:51:38.237
>> Yeah. Yeah, they are.
Um they're they're

00:51:38.320 --> 00:51:40.077
using chips from other companies. Um

00:51:40.160 --> 00:51:42.397
Cerebrris is a good example and they

00:51:42.480 --> 00:51:47.117
have their own in-house chip. Um so
I think maybe publicly they're they're

00:51:47.200 --> 00:51:50.797
doing some big announcements with Nvidia
and I believe it's 5 gawatts they have

00:51:50.880 --> 00:51:54.397
to deploy in the next couple of years
on Ver Rubin. That's that's a lot and that

00:51:54.480 --> 00:51:58.797
that in itself might hinder them from
doing more. But I think um I think Sam

00:51:58.880 --> 00:52:01.117
Alman wants to diversify from Nvidia.

00:52:01.200 --> 00:52:03.757
>> Why? >> Because you
know just like any other

00:52:03.840 --> 00:52:08.237
company it's so expensive and I don't
you know they have their own in-house

00:52:08.320 --> 00:52:11.517
silicon effort. I don't think they they
think that they know their model better

00:52:11.600 --> 00:52:16.157
than anyone else like their their AI
model and so I think they think that

00:52:16.240 --> 00:52:20.077
they're the best to develop a chip
that can run their model efficiently.

00:52:20.160 --> 00:52:21.917
>> Max, you're back and forth to Taiwan

00:52:22.000 --> 00:52:25.037
very frequently. um when we think about

00:52:25.120 --> 00:52:27.517
the position of Taiwan and Taiwan uh

00:52:27.600 --> 00:52:30.557
semiconductor in this world um you know

00:52:30.640 --> 00:52:35.117
the fact that it's in this like tenuous
geopolitical place is something people

00:52:35.200 --> 00:52:39.357
tend to be like oh okay and then move
on right away and I want to hear your

00:52:39.440 --> 00:52:41.917
perspective on whether uh the

00:52:42.000 --> 00:52:47.037
independence of Taiwan and the stability
of TSMC is this like hidden black swan

00:52:47.120 --> 00:52:48.797
event that's just kind of in plain sight

00:52:48.880 --> 00:52:50.717
that people are not paying attention

00:52:50.800 --> 00:52:54.477
enough attention to. I mean, it it is uh

00:52:54.560 --> 00:52:57.837
for some reason the modern world decided
to to decided to put all of our chip

00:52:57.920 --> 00:53:01.197
manufacturer or most of it next to uh

00:53:01.280 --> 00:53:05.597
Kim Jong-un and the rest of it
is in China is in is in Taiwan. I I don't

00:53:05.680 --> 00:53:08.397
really understand who decided that or
why we decided that, but that's that's

00:53:08.480 --> 00:53:12.717
the nature of the beast. Um chip
companies, the design companies in the

00:53:12.800 --> 00:53:16.957
US do not plan for this. The contingency
the contingency plan, excuse me, is

00:53:17.040 --> 00:53:20.397
something along the lines of like,
well, we're all kind of screwed if China

00:53:20.480 --> 00:53:23.597
invades. um which okay, but like there's

00:53:23.680 --> 00:53:26.717
no real plan there. Um so, and when I

00:53:26.800 --> 00:53:30.877
say invades, I don't necessarily mean
like a literal invasion. I think what's

00:53:30.960 --> 00:53:34.077
a lot more likely is some kind of soft
power exchange like what happened in

00:53:34.160 --> 00:53:37.997
Hong Kong over time. Um I think
realistically that's that's a more

00:53:38.080 --> 00:53:42.157
likely option. Although I'm not saying
Xiinping won't just invade. Like he

00:53:42.240 --> 00:53:44.237
absolutely will and he said he would. Um

00:53:44.320 --> 00:53:49.437
which could be catastrophic for TSMC and
the modern world again. Um, I I do know

00:53:49.520 --> 00:53:52.477
that chip companies, as I said,
there really isn't much planning and much

00:53:52.560 --> 00:53:55.437
consideration of it because it's it I
think people in the industry just think

00:53:55.520 --> 00:53:58.717
it's an event that's just so crazy,

00:53:58.800 --> 00:54:02.557
complicated, and out of this world that
nobody's going to do anything about it.

00:54:02.640 --> 00:54:04.797
So, >> Max, do you want to talk about what

00:54:04.880 --> 00:54:08.717
happens when you try to visit TSMC?

00:54:08.800 --> 00:54:11.117
>> They send me to the gift shop.

00:54:11.200 --> 00:54:13.597
>> Uh, so >> you're in Taipei
and that's where you go.

00:54:13.680 --> 00:54:17.517
>> That's Yep. That's basically it. They
they got a a museum where they have like

00:54:17.600 --> 00:54:21.357
um I don't know a bunch of chips that
they've made over the years. One of the

00:54:21.440 --> 00:54:24.397
cerebras chips is in there. They're very
proud of it. The dinner plate size one.

00:54:24.480 --> 00:54:27.677
Um and I've never been to the museum,
but uh that's where they wanted to

00:54:27.760 --> 00:54:30.797
always that's where they always want
to take me. I think uh Wired is one of the

00:54:30.880 --> 00:54:34.157
few organizations, news outlets that's
actually been able to go inside of one

00:54:34.240 --> 00:54:38.557
of the factories. Um and they wrote
an interesting story about it. Um but

00:54:38.640 --> 00:54:41.757
>> yeah, I do not get to go to the TSMT
Fabs unfortunately. They're not they're

00:54:41.840 --> 00:54:44.237
not all that interesting inside. like
they're cool if you've never seen one,

00:54:44.320 --> 00:54:47.837
but um they mostly look the same. It's
like a bunch of white boxes with robots

00:54:47.920 --> 00:54:51.597
moving around. Um so it's not like
people that know what they're looking

00:54:51.680 --> 00:54:54.157
for. It's helpful because you can like
count the number of EUV machines in

00:54:54.240 --> 00:54:57.037
there and figure out roughly how many
chips they can make and like do that

00:54:57.120 --> 00:54:59.997
kind of thing and figure out what kind
of tools they're using. But um if the

00:55:00.080 --> 00:55:04.077
tours that most chip companies give
are not particularly enlightening um in

00:55:04.160 --> 00:55:06.077
general, >> yeah, >> they
are fascinating though. Yeah,

00:55:06.160 --> 00:55:07.677
they're cool, but they're
just they're not useful.

00:55:07.760 --> 00:55:09.997
>> Yeah. >> In in like
a meaningful reporting sense

00:55:10.080 --> 00:55:13.197
or an investor's, you know, from an
investor's perspective or something like

00:55:13.280 --> 00:55:15.837
that. >> I had gotten
very excited about TSMC.

00:55:15.920 --> 00:55:19.437
Did a ton of reporting, spoke with former
employees there and then got the

00:55:19.520 --> 00:55:22.317
courage up and I called them and I said,
"I'm ready to come to Taiwan." And

00:55:22.400 --> 00:55:25.837
they're like, "You can come to the gift
shop." And I was like, "All right, well,

00:55:25.920 --> 00:55:28.477
I guess I'm not coming."
>> My understanding is that our freelancer

00:55:28.560 --> 00:55:32.557
who got in there spent about a year
negotiating with the PR team to get in.

00:55:32.640 --> 00:55:35.997
>> I've been working on it for three. So
if she has any tips like I could I could

00:55:36.080 --> 00:55:37.597
definitely >> call up Virginia Hefernon.

00:55:37.680 --> 00:55:40.877
>> I will. Absolutely. >> Lauren,
you've been inside a fab in

00:55:40.960 --> 00:55:44.877
Arizona. What's it like? >> Yeah.
Well, like this was the um Intel

00:55:44.960 --> 00:55:47.837
Fab 52, which is their most advanced

00:55:47.920 --> 00:55:50.477
fab. It's a 2 nanometer fab and it is on

00:55:50.560 --> 00:55:55.277
the level of what TSMC does in terms
of 2nmter, but of course at a fraction of

00:55:55.360 --> 00:55:58.317
the scale of what TSMC does. And to
Max's point about, well, what is our

00:55:58.400 --> 00:56:02.157
redundancy or our resiliency plan in the
event of an invasion of Taiwan? It seems

00:56:02.240 --> 00:56:04.237
like right now the US government is very

00:56:04.320 --> 00:56:06.317
interested in what Intel can do in its

00:56:06.400 --> 00:56:10.077
fabs, but um it's pretty small. Uh and

00:56:10.160 --> 00:56:12.637
so yeah, I went to Chandler, Arizona. Um

00:56:12.720 --> 00:56:15.997
saw Fab 52. Um you know, got the bunny

00:56:16.080 --> 00:56:19.677
bunny suit on and um like Max said, it's

00:56:19.760 --> 00:56:23.757
it's interesting to see uh everything is

00:56:23.840 --> 00:56:26.237
roboticized. It's like if you had this

00:56:26.320 --> 00:56:31.437
um utopian fever dream of what the future
of manufacturing looked like and

00:56:31.520 --> 00:56:33.917
everything was white and everyone
was dressed in all white and everything's

00:56:34.000 --> 00:56:37.277
like being you know moving around. They
call them foops that actually carry the

00:56:37.360 --> 00:56:41.197
silicon around above head and then they
just sort of automatically go down to

00:56:41.280 --> 00:56:45.677
get like etched and carved and stamped
and everything. Um it look it looks like

00:56:45.760 --> 00:56:47.837
it feels and looks like science fiction.

00:56:47.920 --> 00:56:51.917
But from a reporting perspective, oftent
times the companies will cover up the

00:56:52.000 --> 00:56:55.757
names of the vendors on the different
machines and you may be able to get some

00:56:55.840 --> 00:56:58.157
insight. Um, for example, there there

00:56:58.240 --> 00:57:01.997
were ASML UEV machines in these giant

00:57:02.080 --> 00:57:05.517
bays the size of school buses, but then
you might see some spaces where they're

00:57:05.600 --> 00:57:08.877
like, well, we have two more coming
and we're waiting for them. And so you're

00:57:08.960 --> 00:57:10.237
like, okay, what does that mean? you

00:57:10.320 --> 00:57:13.597
know, UEV is obviously a very um you

00:57:13.680 --> 00:57:15.997
know, a a very coveted sort of

00:57:16.080 --> 00:57:20.957
technology and they're also very
expensive and you know, so so you get

00:57:21.040 --> 00:57:24.557
some insights from that, but uh
primarily it's just kind of um it's good

00:57:24.640 --> 00:57:27.437
for context for better understanding
how this all works, >> right?

00:57:27.520 --> 00:57:30.077
>> I was not allowed to go
to this tour just for what it's worth.

00:57:30.160 --> 00:57:31.357
>> Max, I think you're
doing something wrong.

00:57:31.440 --> 00:57:34.317
>> Max, you got to work on your emails.

00:57:34.400 --> 00:57:36.957
>> Um so, >> just kidding. That means
you're doing actually a great job.

00:57:37.040 --> 00:57:40.557
>> Thanks, Lauren. Uh there was an there
was an interesting uh moment in the

00:57:40.640 --> 00:57:44.477
Darkesh Jensen interview which I guess
will come up in every session today. Um

00:57:44.560 --> 00:57:46.637
where where basically what Jensen is

00:57:46.720 --> 00:57:49.597
trying to get across to Daresh is if you

00:57:49.680 --> 00:57:53.677
do not provide chips to China um
they're going to be there's going to be a

00:57:53.760 --> 00:57:57.437
constraint. They will develop their own
models on their own chips. They could

00:57:57.520 --> 00:58:01.837
become the global standards and then put
export controls on the US. Does anybody

00:58:01.920 --> 00:58:05.437
here think that that is a legitimate
concern or do you think Jensen just

00:58:05.520 --> 00:58:08.957
wants to sell chips in China?
wants to sell it to chips in China.

00:58:09.040 --> 00:58:10.797
>> Okay, >> I mean it might
be a legitimate concern

00:58:10.880 --> 00:58:15.837
in some sense, but Nvidia has run out
of places to grow. Uh they can't sell to

00:58:15.920 --> 00:58:18.237
any more countries. They can't sell
to any more hyperscalers. They can't sell

00:58:18.320 --> 00:58:21.677
to any more even they're going after
the enterprise market, which is messy and

00:58:21.760 --> 00:58:23.997
complicated and difficult to actually

00:58:24.080 --> 00:58:26.237
make a lot of money in. So, China is a

00:58:26.320 --> 00:58:31.597
big market that has basically no access
to Nvidia chips right now. I mean, you

00:58:31.680 --> 00:58:34.077
know, if you're getting those kinds
of gross margins, like, yeah, of course,

00:58:34.160 --> 00:58:37.757
that's like an obvious just a very
obvious place to to be able to sell

00:58:37.840 --> 00:58:41.837
chips, the company's kind of obsessed
with it at the moment. And it's why

00:58:41.920 --> 00:58:43.357
Jensen goes to Washington all the time.

00:58:43.440 --> 00:58:47.277
That's why he >> insisted
on going to China. I mean, it's

00:58:47.360 --> 00:58:50.477
it seems pretty clear to me.
>> Anisa, any thoughts?

00:58:50.560 --> 00:58:54.637
>> Max spends a little bit more time
thinking about that than I do. Um,

00:58:54.720 --> 00:58:58.237
>> I I do understand, you know, obviously
he wants to sell to China. It's a huge

00:58:58.320 --> 00:59:01.757
market and they're taking a hit because
they can't sell to China. So, you can't

00:59:01.840 --> 00:59:05.997
really answer the question without
acknowledging that. Um, but I I do think

00:59:06.080 --> 00:59:09.117
that, you know, from the national
security perspective, um, Chinese

00:59:09.200 --> 00:59:13.517
companies are developing AI chips um,
no matter what happens. Um, so, you know,

00:59:13.600 --> 00:59:17.117
whether we can or can't, I do think that
they'll continue to develop alternatives

00:59:17.200 --> 00:59:19.837
to Nvidia. >> I mean,
their manufacturing process is

00:59:19.920 --> 00:59:22.477
is a lot worse. like the one of the uh

00:59:22.560 --> 00:59:24.077
research firms just published an

00:59:24.160 --> 00:59:26.957
analysis of the sort of one of the new

00:59:27.040 --> 00:59:29.917
Chinese chips like homemade um by Smick

00:59:30.000 --> 00:59:32.877
and it's like it's good but they're they

00:59:32.960 --> 00:59:37.117
run up to up to the limit of like they
can't buy EUV machines um the big ones

00:59:37.200 --> 00:59:40.077
that Lauren was just talking about.
So if you can't buy those things it's just

00:59:40.160 --> 00:59:43.437
fundamentally difficult to make
transistors below a certain size and

00:59:43.520 --> 00:59:47.357
they can't do it effectively. >> Yeah.
And to Max's point, Nvidia only

00:59:47.440 --> 00:59:50.157
has so much more room to grow. I mean
they're at the point where they have

00:59:50.240 --> 00:59:53.517
become a major investor in Corewave and
then Corewave is using that money to buy

00:59:53.600 --> 00:59:57.517
Nvidia chips and so it's it's you know
it's really and there was an earnings

00:59:57.600 --> 00:59:59.677
call um earlier this year maybe late

00:59:59.760 --> 01:00:04.637
last year where uh Nvidia said that they
had a domestic buyer of some of the

01:00:04.720 --> 01:00:07.837
chips that they weren't able to sell to
China though they not disclose the buyer

01:00:07.920 --> 01:00:10.717
and so they're at the point where
they're basically yeah trying to sell

01:00:10.800 --> 01:00:14.477
them wherever they can and so you'd have
to imagine the motivation there is they

01:00:14.560 --> 01:00:19.357
want to sell chips to China >> no
those leftover H20s that they were Yeah.

01:00:19.440 --> 01:00:23.197
China that somebody else wanted them cuz
there's people want silicon right now so

01:00:23.280 --> 01:00:26.637
badly any any kind will do.
>> I think they sold about was it $600

01:00:26.720 --> 01:00:28.077
million worth of those? >> Yeah.

01:00:28.160 --> 01:00:31.437
>> Yeah. Okay. So, a minute left. Let's
do this an actual lightning round. So, yes

01:00:31.520 --> 01:00:35.437
or no? Uh do you think that uh are
you all bitter or less pills? Which is

01:00:35.520 --> 01:00:38.477
basically like the reason why there's
all this infrastructure is because the

01:00:38.560 --> 01:00:42.557
AI labs think the more you know GPUs and
chips you string together the better the

01:00:42.640 --> 01:00:45.597
models will get. Do you think that it
will continue to improve as these

01:00:45.680 --> 01:00:49.917
buildouts uh you know continue
to to explode in size or no?

01:00:50.000 --> 01:00:51.757
>> Wait, ask the question once more.

01:00:51.840 --> 01:00:56.397
>> Is is it is it is the investment worth
it in in these data centers or are they

01:00:56.480 --> 01:00:59.117
ultimately like not going to have much
better models even though they have a

01:00:59.200 --> 01:01:01.357
lot more chips? >> Are we
going down the line here? You

01:01:01.440 --> 01:01:05.837
start with us. So you start
with me. Um generally yes.

01:01:05.920 --> 01:01:10.237
>> Okay. >> Mhm. >> Max.
>> Uh not not sure. Sorry. Okay. Okay.

01:01:10.320 --> 01:01:14.237
>> I think it depends on the amortization
what ends up happening there.

01:01:14.320 --> 01:01:17.437
>> Okay. Ana, you announced
your news on Twitter, right?

01:01:17.520 --> 01:01:19.997
>> I did and LinkedIn. >> Why
don't you tell everybody where where

01:01:20.080 --> 01:01:22.717
you're going next? >> Oh,
um July 6th, I'm starting a new job

01:01:22.800 --> 01:01:25.277
at the Wall Street Journal.
>> All right, let's hear for Ana.

01:01:25.360 --> 01:01:28.920
>> Thank you. >> Congratulations.

01:01:29.520 --> 01:01:31.997
And thank you, Lauren and Max, for joining
us as well. Let's hear for them,

01:01:32.080 --> 01:01:36.120
the whole panel. Thank you guys.

01:01:40.240 --> 01:01:43.037
Thank you very much. So, so often the

01:01:43.120 --> 01:01:47.357
story of AI's trajectory is is told
without the people using it. We hear

01:01:47.440 --> 01:01:49.997
about these concepts called token maxing

01:01:50.080 --> 01:01:52.237
that are talked about in the abstract

01:01:52.320 --> 01:01:55.437
and then one day you see a chart and you

01:01:55.520 --> 01:01:59.677
never actually speak to the people
spending the tokens. So I think to fully

01:01:59.760 --> 01:02:03.117
understand how this technology
is progression progressing and its

01:02:03.200 --> 01:02:08.237
potential to meet its the way that
people want it to go, we actually we

01:02:08.320 --> 01:02:11.837
actually have to speak with the people
who are building with the tokens. And

01:02:11.920 --> 01:02:13.117
that's why I'm thrilled to bring on

01:02:13.200 --> 01:02:17.037
Dallas Dolan, the the TMT leader at PWC,

01:02:17.120 --> 01:02:19.597
who is actually implementing this AI,

01:02:19.680 --> 01:02:24.237
looking at the costs, and making sure
that it's worth the money that they're

01:02:24.320 --> 01:02:28.637
spending on it, and will take us deep
into the token maxing and the budgeting

01:02:28.720 --> 01:02:34.760
conversation. So folks, let's
give it up for Dallas Dolan of PWC.

01:02:36.720 --> 01:02:38.637
>> Great to see you, Dallas.
>> The tables have turned. You're

01:02:38.720 --> 01:02:42.237
interviewing me. This is good. Uh what
do you think about Aaron Levy was here

01:02:42.320 --> 01:02:43.997
like 10 minutes ago and he talked about

01:02:44.080 --> 01:02:49.917
how token maxing was a BS uh um media
narrative effectively and it never

01:02:50.000 --> 01:02:52.077
really happened.
>> Do you agree with that?

01:02:52.160 --> 01:02:53.117
>> I don't know if I
totally agree with it.

01:02:53.200 --> 01:02:57.277
We were actually talking backstage just
before he came on and um I think there's

01:02:57.360 --> 01:03:01.837
absolutely like we'll call it above
above the you know above the plane sort

01:03:01.920 --> 01:03:04.877
of commentary that's out there
that uh we're all seeing in in both the

01:03:04.960 --> 01:03:08.557
mainstream media as well as like what
plays really well on on social media on

01:03:08.640 --> 01:03:11.277
X and other places um and in podcasts.

01:03:11.360 --> 01:03:12.877
And then there's there's the reality

01:03:12.960 --> 01:03:15.597
within a lot of organizations, but it's

01:03:15.680 --> 01:03:20.237
it's happening enough where the behaviors,
I'll call them, are slightly

01:03:20.320 --> 01:03:23.917
problematic from a cost and from an ROI
point of view that it's real, right? You

01:03:24.000 --> 01:03:27.437
can't say that every circumstance
is a problem necessarily, but it's coming

01:03:27.520 --> 01:03:30.237
through in a way that, you know, it's
enough to think about and say, hey, are

01:03:30.320 --> 01:03:31.837
we doing this the right way, right?

01:03:31.920 --> 01:03:35.277
Broadly speaking from an enterprise
strategy and also then from even a

01:03:35.360 --> 01:03:36.877
broader ecosystem point of view.

01:03:36.960 --> 01:03:40.797
>> Yeah. So, there's this moment now where
we're seeing actually a counter to token

01:03:40.880 --> 01:03:42.717
maxing. Uh, it's called token

01:03:42.800 --> 01:03:44.797
minimizing. And you're seeing companies

01:03:44.880 --> 01:03:48.797
like AT&T and Meta um get really serious

01:03:48.880 --> 01:03:53.837
about cost. We also know that Uber
of course uh spent their entire budget in

01:03:53.920 --> 01:03:57.197
less than half a year. Um, who do you
think is going to win out at the end of

01:03:57.280 --> 01:04:00.877
the day, the token minimizers or the
token maxers given the definition of

01:04:00.960 --> 01:04:02.477
token maxing you you just gave us?

01:04:02.560 --> 01:04:05.117
>> Yeah, I mean I think here here's
the good news. The good news is I don't I

01:04:05.200 --> 01:04:08.157
don't think there's a winner and loser
that's going to be defined by did you

01:04:08.240 --> 01:04:11.197
token max or not token max. I think the
winner and loser is going to be defined

01:04:11.280 --> 01:04:15.837
by did you outcome max or not. Um it's
in part going to be a function of how

01:04:15.920 --> 01:04:18.797
did you incentivize people which goes
into that leaderboard and the things

01:04:18.880 --> 01:04:21.597
that got a lot of folks will say in
trouble or certainly in the news. Right?

01:04:21.680 --> 01:04:24.637
So there's that piece of it. It's
the how do you actually want to encourage

01:04:24.720 --> 01:04:29.437
people to do it without encouraging the
wrong things. It's take it too far etc.

01:04:29.520 --> 01:04:32.397
Right? So there's that piece and or spend
too much money. And then the other

01:04:32.480 --> 01:04:34.957
part of it is going to be actually
I think from a planning point of view

01:04:35.040 --> 01:04:38.717
within an organization and I deal
with this as well. So I sit on the the the

01:04:38.800 --> 01:04:43.277
boards for for our US and and our global
organization at PWC and we talk about

01:04:43.360 --> 01:04:46.637
this a lot. In fact we talk about
this even contextually from a comparative

01:04:46.720 --> 01:04:49.997
point of view within different industries.
So it's not just saying hey

01:04:50.080 --> 01:04:53.517
does one organization as a services
company spend more or less than another

01:04:53.600 --> 01:04:57.037
but also how are we compared and spending
you know against let's say like

01:04:57.120 --> 01:04:59.997
a tech company who's you know got
a bunch of engineers and doing the coding

01:05:00.080 --> 01:05:04.157
there. And I think what we're looking
for is a what's like the baseline,

01:05:04.240 --> 01:05:08.077
right? So it's it's benchmarking with
industry out with within the industry,

01:05:08.160 --> 01:05:12.157
outside the industry and against a given
benchmark and saying, okay, are we

01:05:12.240 --> 01:05:16.077
actually getting an ROI? Are we way
outside the bounds in terms of what we

01:05:16.160 --> 01:05:18.797
think the spend is or what we know
the spend is, right? Some of the companies

01:05:18.880 --> 01:05:21.117
that you've named are certainly
companies that we're aware of and work

01:05:21.200 --> 01:05:24.797
with too. Um, so we have some decent
intel on what's happening within the

01:05:24.880 --> 01:05:27.677
industry from that point of view. But it
is going to be that like what outcomes

01:05:27.760 --> 01:05:31.277
did you get? And I will tell you, I've
seen some things that people have built,

01:05:31.360 --> 01:05:34.397
for example, in the deal space that's
just incredible. Like really, really

01:05:34.480 --> 01:05:38.237
amazing output. It's taking, you know,
thousands of hours worth of work and

01:05:38.320 --> 01:05:40.397
creating a product within seven minutes.

01:05:40.480 --> 01:05:44.717
The cost is very high, but it's
an amazing output and it does show very

01:05:44.800 --> 01:05:48.637
well from a consumer or from a,
you know, a customer point of view. It's

01:05:48.720 --> 01:05:51.277
also very exciting for, you know, the
folks who work in the industry who don't

01:05:51.360 --> 01:05:54.317
have to stay up all night maybe working
on something. That's great. There's

01:05:54.400 --> 01:05:57.357
other processes that people haven't even
attacked yet. And I think the question

01:05:57.440 --> 01:06:00.797
is when do you actually go after each
one of those and say how am I going to

01:06:00.880 --> 01:06:03.837
outcome maximize for each one of these
processes. That's actually when you're

01:06:03.920 --> 01:06:08.077
going to see I think the benefits come
through. There will be maximization of

01:06:08.160 --> 01:06:11.597
spend but it'll be maximized in a way
that you're actually getting an outcome

01:06:11.680 --> 01:06:13.757
from it not just because
you spent the most money.

01:06:13.840 --> 01:06:15.517
>> So you're already
doing ROI calculations.

01:06:15.600 --> 01:06:17.997
>> Absolutely. Yeah.
>> So you're probably far ahead than most

01:06:18.080 --> 01:06:22.717
folks. Um how do you determine the ROI
and and what percentage of your projects

01:06:22.800 --> 01:06:25.437
would you say are actually
generating a positive ROI?

01:06:25.520 --> 01:06:27.757
>> Yeah. Well, I'll tell you this. I
know I mean even from an external point of

01:06:27.840 --> 01:06:31.517
view, I think MIT did did a recent
publication talking about like what sort

01:06:31.600 --> 01:06:35.277
of uh activities are really, you know,
replaceable just with generative AI in

01:06:35.360 --> 01:06:38.317
general, especially like in the vision
and and you know, human interactive

01:06:38.400 --> 01:06:39.757
space. And I think they came up with a

01:06:39.840 --> 01:06:43.837
an outcome of about 23%. Um, as in you

01:06:43.920 --> 01:06:47.757
wouldn't use a human to do 23%
of the work. I look at that and say in our

01:06:47.840 --> 01:06:51.437
business it's probably not as high
as 23%, but it's going to be some

01:06:51.520 --> 01:06:55.197
percentage of every single thing
that people do, right? So it's it's a

01:06:55.280 --> 01:06:58.237
different mathematical equation. It's
is absolutely going to be measured for our

01:06:58.320 --> 01:07:00.637
business. It's going to be measured in
hours. The same thing that's going to be

01:07:00.720 --> 01:07:03.757
done in the engineering space for a lot
of these companies as well. It's a

01:07:03.840 --> 01:07:07.117
measurement of hours. It's are you
making the person more efficient? Are

01:07:07.200 --> 01:07:09.917
they creating more output, you know,
for the amount of time they're spending on

01:07:10.000 --> 01:07:12.717
it, right? So is that lines of code
they're producing or is it quality

01:07:12.800 --> 01:07:16.637
product? Which this is an interesting
thing, right? like I could build a

01:07:16.720 --> 01:07:20.797
larger slide deck or I can write way
more lines of code, let's say, but am I

01:07:20.880 --> 01:07:22.957
actually getting a product at the end
of the day that people are saying, "Oh,

01:07:23.040 --> 01:07:25.917
yeah, that's something I'm willing
to use." And I had that conversation a

01:07:26.000 --> 01:07:29.677
bunch actually at uh at Tech Week in New
York a couple of weeks ago with a number

01:07:29.760 --> 01:07:33.117
of a number of founders and also
a number of the investors right from Andre

01:07:33.200 --> 01:07:36.477
and others who were talking about
the types of companies that they're

01:07:36.560 --> 01:07:38.557
investing in. I think it's really
interesting, right? If you're an

01:07:38.640 --> 01:07:41.837
investor, you say, "No, like you know,
what does your product do, you know, for

01:07:41.920 --> 01:07:44.397
the person who's using, especially if
it's a coding product or something along

01:07:44.480 --> 01:07:47.517
those lines." If you're not indicating
that you're able to help them actually

01:07:47.600 --> 01:07:50.477
build a product, right? Come out with
something a product or service that's

01:07:50.560 --> 01:07:53.277
better that their customer is going
to want to use, the fact that it produces

01:07:53.360 --> 01:07:57.117
more lines of code or a longer slide
deck or a longer pitch deck or a longer

01:07:57.200 --> 01:08:00.957
memo is not better. Right? I think we
we're going to come to a determination,

01:08:01.040 --> 01:08:04.157
Alex, really soon here where we start to
say, "Oh, wait a second. Like, what am I

01:08:04.240 --> 01:08:07.997
really trying to get to here? Is it
more more more more or is it actually a

01:08:08.080 --> 01:08:09.677
better product that I'm trying to get?

01:08:09.760 --> 01:08:11.597
>> Right. But that sort of goes to the way

01:08:11.680 --> 01:08:13.837
that the foundational labs and the cloud

01:08:13.920 --> 01:08:16.957
players are working with you. The things

01:08:17.040 --> 01:08:21.837
that I've heard is that these companies
are making it so that when you plug into

01:08:21.920 --> 01:08:24.717
their systems, you're going
to spend a lot of tokens

01:08:24.800 --> 01:08:28.157
>> and they don't really want you
to be able to measure them. Um, are you

01:08:28.240 --> 01:08:31.357
finding that? So the I think the big

01:08:31.440 --> 01:08:34.637
counter to you know the the um maybe the

01:08:34.720 --> 01:08:38.717
the marketing and sales approach there
is in the control plane that's being

01:08:38.800 --> 01:08:41.517
used to actually help companies make

01:08:41.600 --> 01:08:44.717
decisions on what model's being used or

01:08:44.800 --> 01:08:48.717
what interface is being used to do
specific activities. And this is where

01:08:48.800 --> 01:08:51.437
this concept of central planning
and I say that as someone who's who's

01:08:51.520 --> 01:08:54.557
traveling to China next week. This
concept of centralized planning within

01:08:54.640 --> 01:08:57.917
an enterprise is going to be so important.
It's the you are not allowed

01:08:58.000 --> 01:09:02.717
to check the weather five time five five
times a day using you know clawed 5, you

01:09:02.800 --> 01:09:05.597
know, 5.7 or whatever it might be,
right? Like it's unacceptable, right?

01:09:05.680 --> 01:09:09.757
Mythos is not a good use case
for weather checking. Um even if you're

01:09:09.840 --> 01:09:12.397
worried like I was last night about the
tornadoes going through the Midwest and

01:09:12.480 --> 01:09:15.677
trying to get back here so I wasn't
late for Alex. It's still not a good use,

01:09:15.760 --> 01:09:18.557
right? You can do that still with a
weather app or if you really want to use

01:09:18.640 --> 01:09:21.597
something inexpensive like there's
a lot of those options out there. I think

01:09:21.680 --> 01:09:24.717
what's going to happen, you know,
you the the term control plane is sort of

01:09:24.800 --> 01:09:28.637
out there. It's relatively newish. Um,
it's got governance elements, it's got

01:09:28.720 --> 01:09:32.877
cost control elements, it's got access
to data elements, it's got um, I'll see

01:09:32.960 --> 01:09:35.837
even like the the human interaction
elements like what can you put into it,

01:09:35.920 --> 01:09:37.677
not only what you can get out of it.

01:09:37.760 --> 01:09:42.477
that is going to be the the the you know
the the surface of place so to say where

01:09:42.560 --> 01:09:46.637
people are going to spend a lot of the
time in the energy at the AT&Ts and the

01:09:46.720 --> 01:09:49.997
metas and the services companies and a
lot of these other spaces including in

01:09:50.080 --> 01:09:54.077
the engineering space but also in the
sales in marketing and in in the back

01:09:54.160 --> 01:09:57.597
office as well as they look at other
activities that they will do better but

01:09:57.680 --> 01:10:00.877
they will do better with the right tool.

01:10:00.960 --> 01:10:04.637
quick analogy, right? Like the fact
of the matter is we are simply not best

01:10:04.720 --> 01:10:08.957
suited driving the Lamborghini to go
pick up milk. Those two things just

01:10:09.040 --> 01:10:11.437
don't align, right? Unless you live
in South Beach, in which case it's totally

01:10:11.520 --> 01:10:16.317
okay. Um, but those are the crypto people.
They're not here. So, um, but in

01:10:16.400 --> 01:10:19.197
this case, like that's just not that's
not what we'd want to encourage. And I

01:10:19.280 --> 01:10:22.637
think that that should eventually
proliferate and get to everywhere,

01:10:22.720 --> 01:10:23.997
especially as the costs have gone up.

01:10:24.080 --> 01:10:27.837
And the last time we got together a few
months ago was right at that click point

01:10:27.920 --> 01:10:30.557
of cost. and that you did a great thing
with the audience talking about, hey,

01:10:30.640 --> 01:10:33.997
how much are people willing to spend
which is a super fun exercise which I

01:10:34.080 --> 01:10:36.797
hope maybe you'll do later.
>> We could do we could do it now. Can we

01:10:36.880 --> 01:10:39.197
put let's put the house lights up for a

01:10:39.280 --> 01:10:44.797
second? Um I am curious. Sorry, we're
going to ask you to vote. Um of of the

01:10:44.880 --> 01:10:48.877
folks here, >> would you be
willing to spend double the

01:10:48.960 --> 01:10:50.397
amount that you're spending today for

01:10:50.480 --> 01:10:56.120
the current capabilities
that you have uh with AI?

01:10:56.480 --> 01:10:58.557
That's it. How many people are satisfied

01:10:58.640 --> 01:11:01.677
with the price that you're paying?

01:11:01.760 --> 01:11:05.597
So, if if they really Okay, sorry. I'm
going to go with more more questions. If

01:11:05.680 --> 01:11:10.077
they if let's say your Claude
subscription doubled in price, how many

01:11:10.160 --> 01:11:13.677
people would cancel?
>> Is it fable or not?

01:11:13.760 --> 01:11:16.317
>> Is it fable? Well, right now, no. No.

01:11:16.400 --> 01:11:19.037
>> Yeah. >> I think I
think Okay, we could put the

01:11:19.120 --> 01:11:22.877
lights down again. I think basically
what it what it proves is um we we saw

01:11:22.960 --> 01:11:26.637
about half the room's hands go up that
they would pay double. Um, and it does

01:11:26.720 --> 01:11:29.277
seem like there's a bunch of room for

01:11:29.360 --> 01:11:33.837
the labs to be able to raise prices
and still do well. But Dallas, something

01:11:33.920 --> 01:11:37.597
interesting happened since we spoke last.
So, we spoke at Google Cloud Next

01:11:37.680 --> 01:11:41.597
about this and um, someone asked about
the the margin the I think Dallas, you

01:11:41.680 --> 01:11:45.277
asked whe whether the margin
of the business can be maintained at the

01:11:45.360 --> 01:11:48.477
current prices. And I thought, okay,
well, forget about it because these labs

01:11:48.560 --> 01:11:52.637
have such an economically valuable >>
uh uh tool for us that they'll raise

01:11:52.720 --> 01:11:55.917
prices. But we might be in the moment
where they're going to get into a price

01:11:56.000 --> 01:12:00.717
war because OpenAI is rumored to be
potentially uh dropping prices. And so

01:12:00.800 --> 01:12:03.197
what do you think about that? I >> I
I think we're absolutely right on the

01:12:03.280 --> 01:12:06.957
precipice of that. I think you actually
saw in the audience here by comparison,

01:12:07.040 --> 01:12:10.557
>> we saw every hand go up when you said,
"Would you be willing to pay double when

01:12:10.640 --> 01:12:12.957
we were together 3 months ago in April?"

01:12:13.040 --> 01:12:16.957
>> And when when Alex asked if people
were willing to pay four and five times as

01:12:17.040 --> 01:12:19.437
much, there was still a quarter
of the hands in the room that were up. And

01:12:19.520 --> 01:12:22.237
we're talking a room of about 200 people
roughly, right? So there was a lot of

01:12:22.320 --> 01:12:26.717
people. was a good, you know, good good
tea sample, so to say. Um, I contrast

01:12:26.800 --> 01:12:30.237
that to what we just saw right now, and I
think it's a a fairly, you know, even

01:12:30.320 --> 01:12:32.157
distribution, similar subset of people.

01:12:32.240 --> 01:12:36.477
Um, and the reality is there's there's
more skepticism of value that they're

01:12:36.560 --> 01:12:40.477
getting from it, especially when you
start layering on the access and the

01:12:40.560 --> 01:12:44.477
capabilities associated with some of the
models that are still per seat, as well

01:12:44.560 --> 01:12:47.757
as some of the open models, which you
can get access to and, you know, for

01:12:47.840 --> 01:12:51.437
free. You can do a lot of really cool
things. What would you do if the models

01:12:51.520 --> 01:12:54.717
were half price or tokens were half price?

01:12:54.800 --> 01:12:57.517
>> What would I think would I do something

01:12:57.600 --> 01:12:59.117
differently? Is that the question? Yeah.

01:12:59.200 --> 01:13:03.597
>> Let's say I'm like Open AI or Google
and I come to you and say, "Dallas, we're

01:13:03.680 --> 01:13:05.357
happy that you're using our technology.

01:13:05.440 --> 01:13:08.957
We want to make sure that you don't use
Fable when it's back online. So, we're

01:13:09.040 --> 01:13:11.357
cutting your prices in by 50%."

01:13:11.440 --> 01:13:13.117
>> Yeah. >> Would it change
anything you're doing?

01:13:13.200 --> 01:13:16.957
>> Absolutely. I I think actually like I
know this because I talked to my CEO and

01:13:17.040 --> 01:13:19.117
CIO yesterday and my CIO again this

01:13:19.200 --> 01:13:22.237
morning. Um and we are enterensitive

01:13:22.320 --> 01:13:24.477
environment, right? We have 350,000

01:13:24.560 --> 01:13:27.277
people globally. Um they all have access

01:13:27.360 --> 01:13:31.517
to one tool or another. And so when you
have 350,000 people and they're playing

01:13:31.600 --> 01:13:34.957
with tools, there's a high degree
of price sensitivity just on sheer volume

01:13:35.040 --> 01:13:37.757
alone, right? even if not all of them
are doing super productive things. Just

01:13:37.840 --> 01:13:41.517
to be clear, those aren't all engineers,
but they're all pretty smart people

01:13:41.600 --> 01:13:44.957
doing pretty interesting things and they
have use cases that they're like, "Hey,

01:13:45.040 --> 01:13:47.437
I want to play with this." And we are
encouraging them to do that. We think

01:13:47.520 --> 01:13:51.197
that's great, but we're also price
sensitive, right? There's an elasticity

01:13:51.280 --> 01:13:55.277
there that says the higher the price
goes up, the less we'd want them to use

01:13:55.360 --> 01:13:57.757
that model. We'd want them to use
something cheaper. So, if somebody was

01:13:57.840 --> 01:14:01.517
coming to us with a cheaper model,
100% the direction of travel would go, you

01:14:01.600 --> 01:14:04.237
know, to that. And we've even built
that into some of our control plane

01:14:04.320 --> 01:14:07.517
technology. So there's selectivity
of model and there's also recommendations

01:14:07.600 --> 01:14:10.957
within it as well. So depending on who
you are and what it is that we think

01:14:11.040 --> 01:14:13.997
you're going to do with it,
it automatically configures to have a

01:14:14.080 --> 01:14:19.037
specific model and you have to break the
the the you know the initial setting in

01:14:19.120 --> 01:14:22.077
order to use something different.
So we're already there, but I'm certain we

01:14:22.160 --> 01:14:24.557
would encourage more usage
of cheaper things. There's no doubt.

01:14:24.640 --> 01:14:27.357
>> Have you gotten any of those type
of calls yet or you're waiting for them?

01:14:27.440 --> 01:14:31.357
>> Um I I I won't go too far into it,
but there's definitely conversations going

01:14:31.440 --> 01:14:33.597
right now. Yeah. >> Very interesting.

01:14:33.680 --> 01:14:37.597
>> Yeah. Um, when we spoke recently,
you told me that you're seeing some limits

01:14:37.680 --> 01:14:40.397
with agents. And it seems to me like if

01:14:40.480 --> 01:14:43.277
we're going to see this continue, agents

01:14:43.360 --> 01:14:45.117
can't be limited. They have to be able

01:14:45.200 --> 01:14:50.397
to operate autonomously and spend all
those tokens and be effective. So, what

01:14:50.480 --> 01:14:53.837
are the limits that you're seeing with
agents today? And do you think that the

01:14:53.920 --> 01:14:57.837
like if we could extrapolate a little bit,
it means that we're going to see um

01:14:57.920 --> 01:15:01.917
some more speed bumps as the labs try
to roll this technology out further? Yeah,

01:15:02.000 --> 01:15:04.157
I don't I use you use the term limit. I

01:15:04.240 --> 01:15:07.437
I think it's um I think it's a function

01:15:07.520 --> 01:15:12.477
of both risk tolerance um as well
as cost tolerances and then finally like

01:15:12.560 --> 01:15:15.997
what is it you have an expectations
of these things doing on their own and so

01:15:16.080 --> 01:15:18.877
the limitations are actually in all in all
three of those areas that are coming

01:15:18.960 --> 01:15:22.317
through. um you know from a risk
tolerance point of view I think people

01:15:22.400 --> 01:15:26.797
are saying wait a second I'm worried
that the agent without some level of you

01:15:26.880 --> 01:15:31.037
know call it control or governance
around it could go just about anywhere

01:15:31.120 --> 01:15:34.237
and what does that mean within
my organization depending on what access I

01:15:34.320 --> 01:15:38.317
give to it from a data perspective um
you know I would say from client data or

01:15:38.400 --> 01:15:41.357
whether it's you know it's code itself
and what can it do to change code if you

01:15:41.440 --> 01:15:44.797
ask it to do one thing in one area will
it simply think that it needs to do that

01:15:44.880 --> 01:15:48.157
everywhere else and and there's
a you know we'll say the ability to

01:15:48.240 --> 01:15:51.517
extrapolate on a single point and like
what control exists there. So that's the

01:15:51.600 --> 01:15:56.237
first limit. The second limit is on like
I said on the cost var variance we went

01:15:56.320 --> 01:16:00.157
through there a second ago which is hey
look like there are just things you're

01:16:00.240 --> 01:16:03.437
not going to want it to do because back
to the MIT study there might be things

01:16:03.520 --> 01:16:07.037
that humans can do not only better but
more cheaply especially now depending on

01:16:07.120 --> 01:16:10.077
you know if if you have access to fable
and you're using that at a super high

01:16:10.160 --> 01:16:14.077
premium. Um, and then the third bit,
you know, is is really in that that

01:16:14.160 --> 01:16:17.517
decision-making, we'll call it from an
organizational point of view. And even I

01:16:17.600 --> 01:16:21.997
I even think it was funny. I um uh
of all the of all the places to be here in

01:16:22.080 --> 01:16:25.277
San Francisco. I was actually at a
funeral this morning for my grandmother

01:16:25.360 --> 01:16:28.557
who was born in San Francisco. Actually,
she was literally born right up the

01:16:28.640 --> 01:16:33.517
street here. And it's such a human
thing, right? And I was talking to a

01:16:33.600 --> 01:16:37.197
priest, the priest who was there. He's
actually the priest who wrote the paper

01:16:37.280 --> 01:16:40.637
um along with some of the team over
over at Anthropic. and we're having

01:16:40.720 --> 01:16:43.437
interesting conversation on like
where does ethics play into the broader

01:16:43.520 --> 01:16:47.117
conversation of what's happening
within this area. It's sort of like it's

01:16:47.200 --> 01:16:51.037
workforce planning and then some. And so
there's this like additional, you know,

01:16:51.120 --> 01:16:53.757
component that I don't think we've even
really gotten into because here we are

01:16:53.840 --> 01:16:57.917
at the center of technology. We don't
think about that part of it so much. But

01:16:58.000 --> 01:17:01.597
it is that like benefit side.
The benefit I think has a risk that we

01:17:01.680 --> 01:17:05.357
haven't even really gotten into a ton,
which is, you know, do I want to scare

01:17:05.440 --> 01:17:10.077
all of my people that I'm watching what
they're doing and waiting to replace

01:17:10.160 --> 01:17:13.677
what it is that they're doing? And the
answer is like absolutely not because in

01:17:13.760 --> 01:17:17.117
fact, I met six of our interns on the
plane back from Chicago last night and

01:17:17.200 --> 01:17:20.557
they couldn't be happier to be interns
with us. Uh they just saw something on

01:17:20.640 --> 01:17:22.397
my t-shirt that said >>
you're still hiring interns.

01:17:22.480 --> 01:17:25.437
>> Absolutely. I mean, it it you know,
we're hiring as many people as we did

01:17:25.520 --> 01:17:28.637
last year and we're changing the who
we're hiring. We're actually hiring a

01:17:28.720 --> 01:17:32.397
lot more kids who are pursuing sciences
and you know even in areas that uh that

01:17:32.480 --> 01:17:33.997
go beyond you know engineering.

01:17:34.080 --> 01:17:35.517
>> And where are you not hiring?

01:17:35.600 --> 01:17:38.877
>> Um I think we're hiring a little bit
in the less of just like you know pure

01:17:38.960 --> 01:17:42.157
accounting. Also a function of where
we're doing it. It's not that we're

01:17:42.240 --> 01:17:45.357
hiring less people overall. We're hiring
less people in certain areas. U but

01:17:45.440 --> 01:17:47.917
that's also just a function
of globalization and how we're delivering

01:17:48.000 --> 01:17:51.117
services. Um but it's not really
a function of the fact that we're saying

01:17:51.200 --> 01:17:53.677
oh we don't need these people because
technology will do a lot of it. And that

01:17:53.760 --> 01:17:56.957
then goes back to your original you
know premise of the question which is you

01:17:57.040 --> 01:17:58.637
know what are the limits that are there

01:17:58.720 --> 01:18:01.517
on the agentics themselves. The limits

01:18:01.600 --> 01:18:04.797
are going to be a tolerance for error a

01:18:04.880 --> 01:18:08.717
tolerance for cost and a tolerance again
for like what will be acceptable within

01:18:08.800 --> 01:18:13.037
the organization of having that thing
do versus having you know a human do it.

01:18:13.120 --> 01:18:15.917
And there'll be a cost dynamic to that
part of it as well. But I think there's

01:18:16.000 --> 01:18:20.237
also even just the how do I run
my business and how am I a positive leader

01:18:20.320 --> 01:18:21.997
right within within a given community.

01:18:22.080 --> 01:18:23.757
doesn't matter what type of company it is.

01:18:23.840 --> 01:18:26.557
>> You know, Dallas, I I want to take
a moment just to acknowledge your your

01:18:26.640 --> 01:18:28.237
grandmother, and I'm
sorry about her passing.

01:18:28.320 --> 01:18:30.637
>> I appreciate that. >> She she
spent her life here in San Francisco.

01:18:30.720 --> 01:18:32.317
>> Here in San Francisco. Yeah. Yeah.

01:18:32.400 --> 01:18:35.037
>> Can you tell us a little
bit about her just just briefly?

01:18:35.120 --> 01:18:39.117
>> Uh, sure. I mean, you know, funny uh
you know, do daughter of of an immigrant

01:18:39.200 --> 01:18:42.317
family. Um, people who picked a lot
of things in a lot of places around the

01:18:42.400 --> 01:18:47.037
world, including Argentina and, uh,
and Hawaii, funny enough. Um, and thought,

01:18:47.120 --> 01:18:49.597
you know, picking stuff out of the ground
isn't a good deal. like we should

01:18:49.680 --> 01:18:53.837
move to San Francisco and we're working
in the caneries um up in in Fisherman's

01:18:53.920 --> 01:18:55.277
Wararf which is what her parents did.

01:18:55.360 --> 01:18:57.677
She actually worked in the telco industry.
The first job was actually

01:18:57.760 --> 01:19:00.477
working for company for Bell. It must be

01:19:00.560 --> 01:19:05.277
it must be the second T in the TMT,
right? So for for for grandma Fernandez

01:19:05.360 --> 01:19:07.197
Corteho. Yeah. Like really a cool

01:19:07.280 --> 01:19:12.717
existence and just a a great person and
a great you know um kind of full circle

01:19:12.800 --> 01:19:15.677
story, right? I mean I think this whole
story of technology is such a cool thing

01:19:15.760 --> 01:19:19.037
that all this is happening in one.
I mean, let's be honest, it's all

01:19:19.120 --> 01:19:22.397
happening kind of in one place here in
in in the Bay Area in San Francisco. And

01:19:22.480 --> 01:19:27.917
it's it's really neat to to be able
to like see that thread all the way pulled

01:19:28.000 --> 01:19:31.037
through, right? Like I've been able
to do from a career point of view. Um, and

01:19:31.120 --> 01:19:33.437
also be able to notice that there's
actually a direct connection to the

01:19:33.520 --> 01:19:36.877
things that made this place a great
place 150 years ago or actually still

01:19:36.960 --> 01:19:39.997
the things that make it a great place
because it's still better, right, to

01:19:40.080 --> 01:19:42.957
work in a canery or work for a company
than it is to pick something out of the

01:19:43.040 --> 01:19:47.037
ground, so to say. again as my as my
ancestors you know concluded you know

01:19:47.120 --> 01:19:51.037
back in the 1800s but when you think
about that I I think it does go back

01:19:51.120 --> 01:19:54.637
even to some of the things of like what
do you want to do right like I think you

01:19:54.720 --> 01:19:59.277
want to create technology that makes
people's lives better in much the same

01:19:59.360 --> 01:20:02.317
way that our grandmas make our lives
better right grandma makes everything

01:20:02.400 --> 01:20:04.877
better I think it's the same kind
of concept the things that we talk about

01:20:04.960 --> 01:20:08.157
every day are how do we like build
products you know what are the

01:20:08.240 --> 01:20:10.957
entrepreneurs they're trying to build
products that make their customers you

01:20:11.040 --> 01:20:14.317
know life better or they make something
easier to do whether it's coding or

01:20:14.400 --> 01:20:18.397
whether it's you know customer support
or what have you. I I think we we take

01:20:18.480 --> 01:20:20.957
that thread all the way through. That's
actually where the ROI comes in. It's

01:20:21.040 --> 01:20:23.197
not going to be back to the token maxing.
It's not going to be how much

01:20:23.280 --> 01:20:27.517
you pay and how you pay it and what have
you. It's a whole ecosystem shift in the

01:20:27.600 --> 01:20:31.277
way that we think about the outputs
themselves. The outputs again still are

01:20:31.360 --> 01:20:34.317
am I serving my clients well? Are they
getting you know deal maximization out

01:20:34.400 --> 01:20:37.277
of what uh what they're doing with you
know with any given company frontier

01:20:37.360 --> 01:20:40.557
firm or hyperscaler or you know
Neocloud or whatever it might be.

01:20:40.640 --> 01:20:41.597
>> Well, thank you for sharing with us.

01:20:41.680 --> 01:20:45.597
>> Yeah, thanks for that. Yeah. Um, you
know, interesting San Francisco. One of

01:20:45.680 --> 01:20:49.757
the things that I associate with San
Francisco is it's a city that where

01:20:49.840 --> 01:20:51.437
people are okay losing a little bit of

01:20:51.520 --> 01:20:54.397
control. Um, whether that is building,

01:20:54.480 --> 01:20:59.597
you know, new technologies that do
things a little bit differently than um

01:20:59.680 --> 01:21:01.277
previous generations or the fact that

01:21:01.360 --> 01:21:03.277
this city or not everybody but many

01:21:03.360 --> 01:21:08.797
people love um LSD and mushrooms. Um, and

01:21:08.880 --> 01:21:13.760
>> you said you weren't going
to mention that in our precall. Okay.

01:21:14.160 --> 01:21:17.517
Um, it is interesting with with with

01:21:17.600 --> 01:21:20.237
agents you do you kind of lose control.

01:21:20.320 --> 01:21:24.157
It reminds me a little bit of like of
skydiving in a way where you jump out of

01:21:24.240 --> 01:21:28.397
a plane and you're like whatever happens
but I hope there's a system ready to

01:21:28.480 --> 01:21:30.877
catch me. Um, from your position you're

01:21:30.960 --> 01:21:33.517
know you're deploying agents um in

01:21:33.600 --> 01:21:37.917
pretty high stakes moments. you even if
you have the best governance in place,

01:21:38.000 --> 01:21:42.237
you have to be okay to a degree
of losing control. So, how do you become

01:21:42.320 --> 01:21:44.637
comfortable with that? That's it's

01:21:44.720 --> 01:21:49.757
funny, right? I mean, I think um how
much can you really control like in

01:21:49.840 --> 01:21:53.117
the in the engineering space where you
have the people like doing the code for

01:21:53.200 --> 01:21:56.157
you or in our space where you have the
individuals doing the services, they're

01:21:56.240 --> 01:21:59.597
preparing a you know, an audit or a tax
return or a deal report or what have

01:21:59.680 --> 01:22:02.237
you. We're putting trust in these folks,

01:22:02.320 --> 01:22:04.077
many of you whom you know at like a

01:22:04.160 --> 01:22:07.357
superficial level. Um, when you start

01:22:07.440 --> 01:22:12.317
layering technology into it, because
my view is that it's an augmentation of

01:22:12.400 --> 01:22:16.637
those people to make them better, not
necessarily seeding all the control from

01:22:16.720 --> 01:22:20.877
them and putting into something that I
know less of, that makes me feel a lot

01:22:20.960 --> 01:22:25.837
more comfortable. If you start getting
into the space where a 100% of all the

01:22:25.920 --> 01:22:30.157
things that we do around I don't know
let's say like booking travel becomes

01:22:30.240 --> 01:22:34.797
you knowified right you just put in the
query and the travel gets booked I think

01:22:34.880 --> 01:22:37.757
that's where you do get uncomfortable
like I appreciated that two nights ago

01:22:37.840 --> 01:22:41.197
knowing I had to get back to hang out
with you Alex and and to make grandma

01:22:41.280 --> 01:22:46.237
sing this morning like I can ping my EA
late at night and say hey I I need some

01:22:46.320 --> 01:22:49.677
help like I need to make sure with
tornadoes coming through Chicago there's

01:22:49.760 --> 01:22:54.157
going to be at least a plane on Thursday
morning that takes off at 6 am that

01:22:54.240 --> 01:22:59.117
could get me to the West Coast. Um,
if I'm pushing that into an an agent and

01:22:59.200 --> 01:23:01.677
I'm hopeful that it works and I'm
hopeful that it books a flight, I mean,

01:23:01.760 --> 01:23:04.557
yeah, I might see the outcome, but isn't
it neat like having the human on the

01:23:04.640 --> 01:23:05.917
other line like Liz says, "Hey, I got

01:23:06.000 --> 01:23:08.317
you, bro." Like, like you're handled.

01:23:08.400 --> 01:23:13.437
>> Um, she's augmented. She goes into
our technology and can quickly query

01:23:13.520 --> 01:23:16.477
something and boom, pulls everything
up and she books a flight within like 30

01:23:16.560 --> 01:23:20.957
seconds, right? But having her there
to make that a little bit better like does

01:23:21.040 --> 01:23:23.757
make the whole thing feel better.
And you know candidly by the way as it

01:23:23.840 --> 01:23:27.197
relates to like what you know what my EC
does or my my chief of staff and others

01:23:27.280 --> 01:23:31.517
do like they're able to do like multiple
person's jobs. If you think about like

01:23:31.600 --> 01:23:34.317
what it was before like you know one
person to one person. No it's one to

01:23:34.400 --> 01:23:38.637
like 12. Like we're doing a lot more
with a lot less. So I look at this tech

01:23:38.720 --> 01:23:41.197
is just being an extrapolation on that
point. Yeah there's some things you're

01:23:41.280 --> 01:23:45.197
just not going to stop doing but like
that's totally okay. Um it's it's no

01:23:45.280 --> 01:23:48.477
different than driving, right? It's
like the self-driving dynamic like we are

01:23:48.560 --> 01:23:51.037
going to get really comfortable with that.
I see this you know move to

01:23:51.120 --> 01:23:54.557
Agentic as being you know very very um

01:23:54.640 --> 01:23:58.237
uh similar uh in a parallel run with
you know with that itself. It just so

01:23:58.320 --> 01:23:59.757
happens to be in the physical space.

01:23:59.840 --> 01:24:02.957
>> Yeah it definitely reminds me a lot
of a whimo. You sort of you know you white

01:24:03.040 --> 01:24:06.077
knuckle it in the beginning and then
you start to go on your phone.

01:24:06.160 --> 01:24:09.757
>> Exactly. >> Yeah.

01:24:09.840 --> 01:24:10.877
>> That's what I've done at least.

01:24:10.960 --> 01:24:13.357
>> Yeah. Whole thing.
Uh anyway, don't take

01:24:13.440 --> 01:24:15.757
any uh anything away from those previous

01:24:15.840 --> 01:24:19.517
comments um about the mushrooms.

01:24:19.600 --> 01:24:21.357
>> That's for later. That's happy hour.

01:24:21.440 --> 01:24:22.797
>> Please join us on the roof at 5.

01:24:22.880 --> 01:24:26.717
>> Exactly. >> Uh Dallas,
thank you so much for being

01:24:26.800 --> 01:24:30.797
here with us. Always a pleasure to speak
with you and I do appreciate uh your

01:24:30.880 --> 01:24:32.717
support and PWC support of this event.

01:24:32.800 --> 01:24:33.837
So, thank you. Thank you very much.

01:24:33.920 --> 01:24:35.757
Let's hear it for Dallas. Thank you.

01:24:35.840 --> 01:24:39.320
>> Thank you so much.

01:24:41.920 --> 01:24:45.757
All right. Amazing. Uh, thank you,
Dallas. All right, folks. So, we are

01:24:45.840 --> 01:24:49.357
going to have this next session.
It's going to be an audience participation

01:24:49.440 --> 01:24:52.877
section. So, we definitely want your
questions for us and then we're going to

01:24:52.960 --> 01:24:57.517
go to a coffee break. So, um, though
for many of you, he needs no introduction.

01:24:57.600 --> 01:25:02.157
Let me introduce Ron Roy. I first
started reading Ron Johny's I first

01:25:02.240 --> 01:25:04.397
started reading Ran Johny's market uh,

01:25:04.480 --> 01:25:09.197
writing in 2021. um he had
write written this newsletter

01:25:09.280 --> 01:25:10.797
called margins and I thought it was a

01:25:10.880 --> 01:25:13.357
terrific newsletter. I saved it. I spent

01:25:13.440 --> 01:25:16.397
my winter break reading it. I DM'd him.

01:25:16.480 --> 01:25:19.277
By that January uh we had decided to do

01:25:19.360 --> 01:25:23.437
an emergency podcast about a crazy
financial situation and then Ranj and I

01:25:23.520 --> 01:25:26.077
kept talking more and more. Uh and then

01:25:26.160 --> 01:25:30.797
by January 2023 I wrote to him and said,
"Hey, don't you want to just come and do

01:25:30.880 --> 01:25:35.037
this every week?" And lucky for me
and lucky for us on uh with with anyone

01:25:35.120 --> 01:25:39.117
involved with Big Technology Podcast,
uh Ron John said yes. And so getting a

01:25:39.200 --> 01:25:42.877
chance to speak with Ron John every
Friday is an absolute joy. It's

01:25:42.960 --> 01:25:46.237
definitely one of the highlights
of my week. And today we're thrilled to be

01:25:46.320 --> 01:25:49.917
able to do our Friday show live here
with you with your with your audience

01:25:50.000 --> 01:25:54.477
questions. Um and then we'll just run
it tomorrow like a normal podcast. So I

01:25:54.560 --> 01:25:56.317
hope you're ready. Uh we definitely need

01:25:56.400 --> 01:26:02.920
your participation. And please
join me in welcoming Ron Roy.

01:26:07.040 --> 01:26:11.597
I got them on. I'm going to take these
off though. We'll get into the snap

01:26:11.680 --> 01:26:13.597
spectacles. This is This is a medium

01:26:13.680 --> 01:26:18.717
risk maneuver because I
have this microphone on. So

01:26:18.800 --> 01:26:19.997
>> those are the Snapchat spectacles.

01:26:20.080 --> 01:26:22.077
>> These are the Snapchat spectacles.

01:26:22.160 --> 01:26:25.677
They're the original developer beta
edition though. So, they're not the new

01:26:25.760 --> 01:26:30.797
ones, but I had them in 2021.
>> Now, everyone knows how cool you are.

01:26:30.880 --> 01:26:33.117
>> All right. Should I start
the >> I'm as cool as Evan Spiegel.

01:26:33.200 --> 01:26:35.277
>> That's right. Yeah.
>> Is he still cool, though?

01:26:35.360 --> 01:26:39.997
>> All right. Uh, let's do it.
Um, how would I start it? Um,

01:26:40.080 --> 01:26:41.437
>> I throw Did I throw you off?

01:26:41.520 --> 01:26:44.157
>> Yeah. No, no, no. I didn't even write
this down. Okay. Well, I'll try to do

01:26:44.240 --> 01:26:48.237
it. All right. Snapchat comes out with new
spectacles and we take your audience

01:26:48.320 --> 01:26:51.837
questions. That's coming up right after
this on a Big Technology Podcast Friday

01:26:51.920 --> 01:26:55.597
edition recorded on Thursday. Welcome
to Big Technology Podcast Friday edition

01:26:55.680 --> 01:26:58.557
where we break down the news in our
traditional coolheaded and nuanced

01:26:58.640 --> 01:27:01.197
format. We're joined as always by Ranjan

01:27:01.280 --> 01:27:03.917
Ry who is here with us in not in studio

01:27:04.000 --> 01:27:06.477
live uh with the big technology AI

01:27:06.560 --> 01:27:11.400
summit audience. Audience, let's hear you.

01:27:15.200 --> 01:27:19.357
The way this is going to work is Ranj
and I will break down uh one story and

01:27:19.440 --> 01:27:22.717
then we definitely welcome your questions,
your prompts, your arguments

01:27:22.800 --> 01:27:26.637
with us. Uh if you don't have anything,
we have plenty to do, but um we'd love

01:27:26.720 --> 01:27:30.157
to have your participation. You can line
up at either of the mics there uh to ask

01:27:30.240 --> 01:27:33.757
us a question. Let's start with our top
story this week. I promised myself when

01:27:33.840 --> 01:27:36.557
this uh summit was being planned that I

01:27:36.640 --> 01:27:39.117
would not be mean to Snapchat. However,

01:27:39.200 --> 01:27:40.797
Snapchat has left me with no choice.

01:27:40.880 --> 01:27:45.720
John, let's roll. Image C, please.

01:27:47.280 --> 01:27:51.837
Uh, listeners, if you are listening on
the podcast, you'll see what you're what

01:27:51.920 --> 01:27:53.597
we're looking at at the audience here is

01:27:53.680 --> 01:27:56.957
a image of Evan Spiegel on CNBC wearing

01:27:57.040 --> 01:27:59.437
the latest Snap Specs. The headline is

01:27:59.520 --> 01:28:03.277
Snap Stock Falls after AR Specs debut.

01:28:03.360 --> 01:28:08.717
Um, uh, almost 20 years since the launch
of the iPhone, people are ready to think

01:28:08.800 --> 01:28:12.637
about computing differently, Spiegel
said in an interview with CNBC. The, uh,

01:28:12.720 --> 01:28:15.197
the market reacted differently. Um,

01:28:15.280 --> 01:28:19.677
Rajan, let me just go to you quickly.
Do you think that that, you know, Snapchat

01:28:19.760 --> 01:28:21.677
and Meta and all these other companies

01:28:21.760 --> 01:28:23.517
have been trying to build these AI

01:28:23.600 --> 01:28:27.677
device future for years and this is what

01:28:27.760 --> 01:28:30.797
we're looking at. Um,
>> this is what we're looking at.

01:28:30.880 --> 01:28:32.077
>> You kind of ruined my surprise. Can we

01:28:32.160 --> 01:28:36.717
go to D, please? I mean,

01:28:36.800 --> 01:28:40.077
>> I brought them. I had to bring
them after sending Alex a photo.

01:28:40.160 --> 01:28:42.397
>> Is it time for us
to finally accept that

01:28:42.480 --> 01:28:45.837
we're not going to have an AR device?

01:28:45.920 --> 01:28:49.197
>> Interesting. So, so these again, I got

01:28:49.280 --> 01:28:52.317
these in 2021. It's actually a very cool

01:28:52.400 --> 01:28:55.037
technology. Like, it's No, I'm serious.

01:28:55.120 --> 01:28:59.837
Augmented reality. If you ever used
Magic Leap in the 2010s, like being able

01:28:59.920 --> 01:29:03.997
to paint throughout your room and walk
around that painting, being a being able

01:29:04.080 --> 01:29:07.837
to play games where you're chasing
zombies again, my seven-year-old son

01:29:07.920 --> 01:29:11.677
actually is probably the only fan
of Snap Spectacles in the world right now.

01:29:11.760 --> 01:29:13.837
He still loves using them. He also likes

01:29:13.920 --> 01:29:15.917
to watch YouTube on them, which is kind

01:29:16.000 --> 01:29:18.717
of amazing. But I think that form factor

01:29:18.800 --> 01:29:22.157
and the experience is amazing. Vision

01:29:22.240 --> 01:29:27.517
Pro has not quite captured it. I don't
think what we just saw on Evan Spiegel's

01:29:27.600 --> 01:29:31.037
face is going to capture it. I think
eventually maybe Apple or someone will,

01:29:31.120 --> 01:29:35.597
but I don't think we're there yet, but I
think we will be. I still am betting on

01:29:35.680 --> 01:29:40.397
AR eyeglasses of some sort. >> Okay,
I'm going to go to questions in a

01:29:40.480 --> 01:29:43.917
moment if anyone has one. So, feel
free to to stand up there if you want,

01:29:44.000 --> 01:29:46.397
otherwise we can keep doing this. Um,

01:29:46.480 --> 01:29:51.037
here's some of the social media reaction.
Uh, does anyone anyone that

01:29:51.120 --> 01:29:55.997
works at Snapchat have the guts to tell
leadership that these things are ugly?

01:29:56.080 --> 01:30:00.637
That feeling when your glasses are so
heavy they give you cauliflower ear.

01:30:00.720 --> 01:30:03.837
Snap is the best brand in the world
when you're 16 and the worst brand when

01:30:03.920 --> 01:30:05.517
you're to be associated with when you're

01:30:05.600 --> 01:30:09.117
21. Um, the people who actually buy

01:30:09.200 --> 01:30:13.677
$2,000 AI glasses aren't teenagers. If
you think you really want to wear Always

01:30:13.760 --> 01:30:16.877
on camera around in public, uh,
it should have to look like this with the

01:30:16.960 --> 01:30:20.557
picture of Evan Spiegel. Um, let me make

01:30:20.640 --> 01:30:25.037
the case that this it's over. Um, >> good
because I'm going to take the other side.

01:30:25.120 --> 01:30:28.877
>> I think that the iPhone series
that released that we just saw. So, does

01:30:28.960 --> 01:30:31.517
anyone here listen to Ron John on Fridays?

01:30:31.600 --> 01:30:34.637
>> Do we have listeners? All right. Okay.

01:30:34.720 --> 01:30:37.197
>> So, you'll know which
direction this might go.

01:30:37.280 --> 01:30:40.397
>> What's this guy been begging for for
like since he came on the show the first

01:30:40.480 --> 01:30:42.957
time? Better Siri, >> better Siri.

01:30:43.040 --> 01:30:45.037
>> I think they actually did it. Like the

01:30:45.120 --> 01:30:47.197
new Siri, if you look at the videos,

01:30:47.280 --> 01:30:51.037
looks terrific. And so maybe this idea

01:30:51.120 --> 01:30:54.877
of we have to wear the computing
on our face is something that like kind of

01:30:54.960 --> 01:30:59.197
sounds good in concept, but model after
model. It's not. And the AI device is

01:30:59.280 --> 01:31:03.837
the iPhone. >> Okay,
that's I I that is an interesting

01:31:03.920 --> 01:31:06.157
direction to take it. I still think the

01:31:06.240 --> 01:31:08.557
form factor. Do do any of the audience

01:31:08.640 --> 01:31:11.277
have like meta ray bands or any other

01:31:11.360 --> 01:31:15.677
device like that? I see a few. Like you
start to feel as you're walking around,

01:31:15.760 --> 01:31:19.597
as you're kind of interacting in the real
world, stuff can happen that's not

01:31:19.680 --> 01:31:24.157
just maybe eventually Siri talking
to you and using a traditional AI model to

01:31:24.240 --> 01:31:27.917
actually, you know, give you kind of
information. So, I don't know. I'm still

01:31:28.000 --> 01:31:30.557
I still think AR as a form factor via

01:31:30.640 --> 01:31:33.837
glasses is going to happen. I think uh

01:31:33.920 --> 01:31:38.557
you have met Ray-B bands, you enjoy them.
You know, I I I didn't want to say

01:31:38.640 --> 01:31:40.797
this publicly, but I have not used them.

01:31:40.880 --> 01:31:44.237
I I I I mean, I I have, but I don't I

01:31:44.320 --> 01:31:48.157
used them for a bit. And like I said
on our show recently, I was on a hike. It

01:31:48.240 --> 01:31:51.917
was cold. >> I was ready
to get to the summit and put

01:31:52.000 --> 01:31:55.037
those glasses on and it the battery

01:31:55.120 --> 01:31:57.597
started blinking red and I couldn't use

01:31:57.680 --> 01:31:59.517
them. Had to use my old phone.

01:31:59.600 --> 01:32:03.677
>> And let me say this, like, you know,
they they haven't taken off as a

01:32:03.760 --> 01:32:07.197
mainstream consumer device. And if
you look at the stock of every single

01:32:07.280 --> 01:32:09.837
company that's pursuing
them, it's not good.

01:32:09.920 --> 01:32:13.837
>> Snapchat, like we just said, is is
struggling. Um, Meta, as we know, has

01:32:13.920 --> 01:32:18.237
its problems. Um, no one's looking
at the these Ray-B bands to save Meta.

01:32:18.320 --> 01:32:23.997
>> Well, no, but to me, the Apple Vision
Pro is like the more direct correlated

01:32:24.080 --> 01:32:27.357
product versus >> You want to know what
the best thing about the Vision Pro was?

01:32:27.440 --> 01:32:29.997
>> Find Vision Pro. Yeah.
>> They put the person on Vision Pro on

01:32:30.080 --> 01:32:34.557
Siri and he fixed it.
Mike Rockwell. Oh, >> so

01:32:34.640 --> 01:32:36.077
>> I would not have
guessed that the person

01:32:36.160 --> 01:32:39.997
who made the Vision Pro would be the one
to fix Siri after all these years, but I

01:32:40.080 --> 01:32:42.877
guess if that's the case,
that's exciting. Yeah.

01:32:42.960 --> 01:32:44.237
>> Yeah. >> So, you're
going to still You're going

01:32:44.320 --> 01:32:46.877
to Can you put those
glasses on one more time?

01:32:46.960 --> 01:32:50.237
>> Again, I was told backstage this might
destroy my microphone, but I'm going to

01:32:50.320 --> 01:32:52.157
try. Do I look cool?

01:32:52.240 --> 01:32:54.397
>> No. No.

01:32:54.480 --> 01:32:57.517
>> No. No. I mean, even we were we were

01:32:57.600 --> 01:33:01.117
joking that Evan Spiegel is going to the

01:33:01.200 --> 01:33:05.357
Met Gala with his model wife. Like, this
is like the coolest person in the world

01:33:05.440 --> 01:33:08.877
and how bad they made him look.
That if it was like when Zuck wore Project

01:33:08.960 --> 01:33:11.037
Orion, no one really cared that much.

01:33:11.120 --> 01:33:15.437
But I think because Evan Spiegel is such
a cool looking dude, that's why it looks

01:33:15.520 --> 01:33:19.037
so egregious. That's my take. >> So,
your your answer on the way to make

01:33:19.120 --> 01:33:23.437
those things work
is just be less handsome.

01:33:23.520 --> 01:33:27.677
>> That's it. That's it. We'll we'll
write to Spiegel and let him know.

01:33:27.760 --> 01:33:29.757
>> No questions. Okay. All right. Great.

01:33:29.840 --> 01:33:32.237
>> All right. Yeah. Don't be shy.

01:33:32.320 --> 01:33:35.597
>> Is it on? Yes, it is on. Hey guys,
>> if you're willing, let us know who you

01:33:35.680 --> 01:33:38.397
are. And >> this is Ser Johal.
Actually, um industry

01:33:38.480 --> 01:33:43.677
analyst flew from New York a day
earlier to join you guys. So,

01:33:43.760 --> 01:33:44.717
>> welcome. Let's give him a welcome.

01:33:44.800 --> 01:33:47.880
>> Thanks for

01:33:49.520 --> 01:33:52.557
ask question about um actually it's an

01:33:52.640 --> 01:33:54.477
observation. And I want to get your take

01:33:54.560 --> 01:33:59.117
on this. Um when companies have this gap

01:33:59.200 --> 01:34:01.517
between what people need today and what

01:34:01.600 --> 01:34:06.397
they're working on which is like a in
future it can be two plus years out. So

01:34:06.480 --> 01:34:10.157
they lose that traction from the
investors point of view as well as

01:34:10.240 --> 01:34:13.197
employees and and partners. So do you

01:34:13.280 --> 01:34:15.837
see that that's happening to meta and

01:34:15.920 --> 01:34:18.877
others like it happened to IBM when they

01:34:18.960 --> 01:34:21.357
were living in future with Watson X for

01:34:21.440 --> 01:34:25.277
example. So um what's your take on that?

01:34:25.360 --> 01:34:28.157
It can be B TOC or B2B examples.

01:34:28.240 --> 01:34:29.277
>> Okay, thank you for the question.

01:34:29.360 --> 01:34:33.197
>> Yeah, no that's a great question. I
do think like if we're talking about kind

01:34:33.280 --> 01:34:36.077
of how the interface for how you

01:34:36.160 --> 01:34:38.157
interact with a computer. I have been

01:34:38.240 --> 01:34:43.117
begging for something else other than
me holding my phone looking at it and we

01:34:43.200 --> 01:34:47.437
haven't really gotten anything for a
long time. I mean we Humane tried with

01:34:47.520 --> 01:34:49.997
the pin. Um, I still think maybe some

01:34:50.080 --> 01:34:55.757
kind of pin is going to be around Johnny
Ives pin at OpenAI. Maybe maybe at some

01:34:55.840 --> 01:34:58.237
point. >> Well, Greg Greg
Brockman is coming, so we'll put

01:34:58.320 --> 01:34:59.677
>> we got to ask him about the talk about

01:34:59.760 --> 01:35:02.637
that. But, but I think like being able

01:35:02.720 --> 01:35:05.517
to interact with all of this information

01:35:05.600 --> 01:35:09.677
now, being able to like process
information so much more reliably with

01:35:09.760 --> 01:35:14.397
AI just I don't want to have to keep
looking at my phone and just looking at

01:35:14.480 --> 01:35:18.477
it on there. But even actually on the
phone, I'm guessing do a lot of people

01:35:18.560 --> 01:35:23.517
here dictate more to their phone right
now. Like that's completely changed. In

01:35:23.600 --> 01:35:26.797
the past, like I would have felt weird
just talking to my phone and now I'm

01:35:26.880 --> 01:35:31.277
constantly using whisper flow
and just talking to whatever. So that's

01:35:31.360 --> 01:35:32.557
>> tell a story about you and your wife.

01:35:32.640 --> 01:35:36.957
>> I know this. Okay. I don't know if this
is the most depressing thing or and my

01:35:37.040 --> 01:35:40.237
wife is not here and hopefully won't
kill me if this is being live streamed

01:35:40.320 --> 01:35:42.477
right now, but >> No,
we won't broadcast this. So, I am

01:35:42.560 --> 01:35:46.397
constantly dictating to my computer, to
my phone. And the other day, it was like

01:35:46.480 --> 01:35:49.037
Friday night, we had put our son to bed.

01:35:49.120 --> 01:35:51.197
We're both on our laptops and I'm on one

01:35:51.280 --> 01:35:53.437
side of the couch dictating. And then I

01:35:53.520 --> 01:35:58.397
look over and she's also has her laptop
open and is kind of whispering to her

01:35:58.480 --> 01:36:01.997
computer, too. And I'm like, is this the
future of tech? But it's a new computing

01:36:02.080 --> 01:36:03.677
interface. So, I'm happy about it.

01:36:03.760 --> 01:36:05.197
>> Right. I look,
I think the the gentleman

01:36:05.280 --> 01:36:08.957
brings up a great question, which
is that things are moving so fast. How do

01:36:09.040 --> 01:36:12.877
you plan right now? And honestly,
I don't know how you do it because every

01:36:12.960 --> 01:36:16.157
day there seems to be a new capability,
then the capability is taken off the

01:36:16.240 --> 01:36:20.637
table. Like Fable, for instance, I mean,
the tweets about Fable where like people

01:36:20.720 --> 01:36:23.997
are adding Dave Sachs and they're like,
"Please, I'll do anything for Fable

01:36:24.080 --> 01:36:27.757
back. Just bring it back." And he won't
bring it back. But it's just like I

01:36:27.840 --> 01:36:30.477
don't understand how any company does
that. And I think that actually would be

01:36:30.560 --> 01:36:33.757
a good topic for us to sort
of get into on a future show.

01:36:33.840 --> 01:36:36.317
>> Yep. Okay, let's go this way.

01:36:36.400 --> 01:36:37.837
>> Hi, thank you for taking my question.

01:36:37.920 --> 01:36:40.397
Um, I was curious your views on in the

01:36:40.480 --> 01:36:42.477
next like five years as AR glasses

01:36:42.560 --> 01:36:45.197
evolve whether like the chunky

01:36:45.280 --> 01:36:49.677
spectacles is the way to go or thinner
glasses with like upboard compute like

01:36:49.760 --> 01:36:54.237
either wireless to your iPhone or like
the vision pro like cable down to the

01:36:54.320 --> 01:36:57.437
battery pack and compute. Well, it's
a battery for Vision Pro, but it could

01:36:57.520 --> 01:37:00.397
also have compute on board. So,
I'm curious like the next five years where

01:37:00.480 --> 01:37:04.797
you think consumers will gravitate
towards. No, it's a it's a great

01:37:04.880 --> 01:37:08.637
question because if you ever use
Magic Leap, there was like a >> puck.

01:37:08.720 --> 01:37:11.597
>> Puck. That's what it was.
>> My Here's my hot take. Anything any

01:37:11.680 --> 01:37:14.157
device that requires
a puck >> not working.

01:37:14.240 --> 01:37:15.597
>> Well, how about this though?

01:37:15.680 --> 01:37:19.117
>> Okay. >> What if what
if the puck is your iPhone?

01:37:19.200 --> 01:37:23.197
>> Oh All right. >> Exactly.
So, which gives Apple an

01:37:23.280 --> 01:37:27.197
opportunity that like if the compute
is taking place on the phone in your

01:37:27.280 --> 01:37:31.677
pocket, it allows you to be much slimmer
from a power perspective. you don't want

01:37:31.760 --> 01:37:33.837
like a lightning cable connected to your

01:37:33.920 --> 01:37:36.477
face. But like it's still I think there

01:37:36.560 --> 01:37:39.837
is a it makes me think Apple still has a

01:37:39.920 --> 01:37:43.757
good chance in this space because
the iPhone can do all of the heavy lifting

01:37:43.840 --> 01:37:48.157
versus this thing on that
is really heavy on your head.

01:37:48.240 --> 01:37:50.317
>> Can I can I ask so what's your name?

01:37:50.400 --> 01:37:55.757
>> I'm Kyle. >> Kyle, so uh what do you
want to use a like face computer for?

01:37:55.840 --> 01:37:58.557
>> Oh that's a good question.
>> See I told you Ron John this stuff is

01:37:58.640 --> 01:38:01.757
not happening. No, no, no.
Kyle's got something.

01:38:01.840 --> 01:38:03.917
>> All right, let's give
Kyle an opportunity here.

01:38:04.000 --> 01:38:06.157
>> Let me think for >> I didn't
mean to put you on the spot,

01:38:06.240 --> 01:38:09.597
but thank you for helping
me prove my point.

01:38:09.680 --> 01:38:12.317
>> Oh, I think one
cool thing would be like

01:38:12.400 --> 01:38:16.397
a shared uh like TV or something like

01:38:16.480 --> 01:38:21.117
like imagine Vision Pro and you have
just a shared just like movie theater.

01:38:21.200 --> 01:38:24.557
You're on a plane with your family
or something and you just have like the

01:38:24.640 --> 01:38:28.397
shared experience, but it's just glasses.
I like that. I like that. Like

01:38:28.480 --> 01:38:31.597
for me, one of the coolest things I
always like cuz you know, as you get

01:38:31.680 --> 01:38:35.997
older, you move further away from your
friends and uh it would be cool to like

01:38:36.080 --> 01:38:38.317
be in the vision pro together.

01:38:38.400 --> 01:38:41.997
>> Yeah. Sit sit half court, watch
the Knicks next to Timothy Chalamay, but

01:38:42.080 --> 01:38:45.757
we're just in division pro. But you
know what's interesting? Apple never

01:38:45.840 --> 01:38:48.557
advertised that as a social device. All

01:38:48.640 --> 01:38:50.957
the marketing was just you're sitting

01:38:51.040 --> 01:38:55.597
alone at home and all you're doing
is the Vision Pro. Nobody else exists.

01:38:55.680 --> 01:38:58.797
>> The other big one. Yeah, sorry
to interrupt. I was say the other big one

01:38:58.880 --> 01:39:01.757
for me is just having >> a lot
of monitors around when I'm

01:39:01.840 --> 01:39:05.757
working. Like I even though I have like
an ultra wide or like three monitors,

01:39:05.840 --> 01:39:07.517
feel like sometimes I could have more.

01:39:07.600 --> 01:39:10.317
So just being able to like interact with

01:39:10.400 --> 01:39:14.637
AI to pull up the exact page I'm looking
for out of like I'm one of those people

01:39:14.720 --> 01:39:18.797
has like a thousand tabs. So just being
able to pull up the thing and I just say

01:39:18.880 --> 01:39:21.437
open up that tab and it just
pops up would be pretty cool.

01:39:21.520 --> 01:39:24.317
>> Would 8,000 individual tabs in a giant

01:39:24.400 --> 01:39:29.357
planner space be better
or Kyle's going with this.

01:39:29.440 --> 01:39:33.757
>> No, no, but I do agree like
>> that being able to do more

01:39:33.840 --> 01:39:37.997
>> open scaled work and like look
at different charts and I think there's

01:39:38.080 --> 01:39:42.157
that still and I know like I have friends
who own the vision pro who use it for that

01:39:42.240 --> 01:39:47.037
>> still. >> No, but did >> did
they had a nice time? All right,

01:39:47.120 --> 01:39:49.037
thank you Kyle. Let's go here.

01:39:49.120 --> 01:39:52.797
>> Hey, what's up? This is Sasha from Yo

01:39:52.880 --> 01:39:56.877
University doing research on AI agents
for finance. First of all, love you

01:39:56.960 --> 01:39:59.117
guys. Listening to you guys chat on my

01:39:59.200 --> 01:40:04.077
way to work has been my routine. I think
this is excitement shed by not just me

01:40:04.160 --> 01:40:06.717
but many people in the audience.
So, thank you so much.

01:40:06.800 --> 01:40:09.837
>> Thank you. >> Thank you.
>> Appreciate that. And thank you for

01:40:09.920 --> 01:40:11.837
coming all the Did you fly in?

01:40:11.920 --> 01:40:15.277
>> Uh yes. >> Oh my god. Thank you
for coming. It's great to see you.

01:40:15.360 --> 01:40:19.037
>> It's worth it. Um and I appreciate
your sharp line of uh reasoning and

01:40:19.120 --> 01:40:21.597
questioning. So uh what's your view on

01:40:21.680 --> 01:40:23.517
the meter benchmarks of how long the AI

01:40:23.600 --> 01:40:26.077
and AI agents can work independently

01:40:26.160 --> 01:40:28.797
being saturated? Are we at that point

01:40:28.880 --> 01:40:32.397
that u of them being at infinite work

01:40:32.480 --> 01:40:34.717
number of hours working yet or are we

01:40:34.800 --> 01:40:37.837
reaching that soon and what is the world

01:40:37.920 --> 01:40:41.917
going to be looking like after
we hit that infinite mark?

01:40:42.000 --> 01:40:43.597
>> That is an excellent
question. So, first

01:40:43.680 --> 01:40:47.997
of all, there is some controversy about
the meter monitoring, but I think it's

01:40:48.080 --> 01:40:49.917
kind of directionally accurate, right?

01:40:50.000 --> 01:40:54.077
Like I showed the meter chart at the
very beginning of of our event today

01:40:54.160 --> 01:40:56.317
that like you saw these models, they

01:40:56.400 --> 01:41:01.437
could not code autonomously
for more than 30 minutes in 2023

01:41:01.520 --> 01:41:06.637
and now they can code autonomously
for the equivalent of 18 hours, right? Um,

01:41:06.720 --> 01:41:09.677
so what happens if they just kind
of blow past that limit and then they can

01:41:09.760 --> 01:41:13.597
code all the time? Um that is an
excellent question. Do you have any

01:41:13.680 --> 01:41:16.957
thoughts? >> So I I felt
that with the go mode and

01:41:17.040 --> 01:41:21.437
automation mode uh it practically
felt that they are already doing this

01:41:21.520 --> 01:41:23.357
autonomously forever. >> Right.

01:41:23.440 --> 01:41:27.677
>> Uh I want to want them to send me
reports. My prompt would be if you find

01:41:27.760 --> 01:41:29.357
something interesting send me an email.

01:41:29.440 --> 01:41:33.277
Yes. >> Then I when I see it I will come
back and intervene. I felt that for many

01:41:33.360 --> 01:41:35.917
tasks it's already at that mark.

01:41:36.000 --> 01:41:40.237
>> Yeah. No a perplexity computer will
basically do that. Um, so that's that's

01:41:40.320 --> 01:41:41.437
definitely something that's happening.

01:41:41.520 --> 01:41:44.477
But I'll just say one more thing
and then we'll go to Ron John. Um, Greg

01:41:44.560 --> 01:41:48.397
Brockman who will be here later has this
like idea of a compute powered economy

01:41:48.480 --> 01:41:52.077
where like you know you sort of once you
get these bots working the way that you

01:41:52.160 --> 01:41:56.157
explained you just maybe he'll say this
later um you just kind of throw them at

01:41:56.240 --> 01:41:59.677
any problem and then they just kind
of work autonomously through it. Now we

01:41:59.760 --> 01:42:02.797
don't we've never seen what the world
looks like when you can do something

01:42:02.880 --> 01:42:06.317
like that but I do think given the
progress that we've seen with the models

01:42:06.400 --> 01:42:09.197
you would imagine that something
substantial will come at a certain

01:42:09.280 --> 01:42:12.717
point. So uh my day job I work at a

01:42:12.800 --> 01:42:14.957
company writer. We're an enterprise AI

01:42:15.040 --> 01:42:19.997
company and uh the when and the the goal
mode was asked about recently again the

01:42:20.080 --> 01:42:23.517
idea that you just provide the goal.
The agent will iteratively loop and keep

01:42:23.600 --> 01:42:25.277
doing work until it finds the right

01:42:25.360 --> 01:42:28.557
solution. In the real world that is like

01:42:28.640 --> 01:42:32.877
the benchmarks versus the real world.
I still think there's such a massive gap

01:42:32.960 --> 01:42:35.517
in terms of what does the data look

01:42:35.600 --> 01:42:37.677
like? What what is the actual problem?

01:42:37.760 --> 01:42:40.317
Does the customer or the person actually

01:42:40.400 --> 01:42:42.477
understand the goal in a clear enough

01:42:42.560 --> 01:42:44.717
way that they're able to define it to

01:42:44.800 --> 01:42:47.837
kind of push that loop forward? So I

01:42:47.920 --> 01:42:49.757
think it's an interesting I think with a

01:42:49.840 --> 01:42:52.397
lot of AI again even around the

01:42:52.480 --> 01:42:57.677
benchmarking like again versus real
world understanding of how to use it,

01:42:57.760 --> 01:43:01.597
what's happening and what's available
for it to use. There's still a lot of uh

01:43:01.680 --> 01:43:04.317
work to be done. I mean you you can do

01:43:04.400 --> 01:43:09.277
these goal modes for like
less complicated tasks. I like

01:43:09.360 --> 01:43:12.557
>> what have you goal modeed?
>> Uh so I worked with Perplexity Computer

01:43:12.640 --> 01:43:15.517
recently to um to try to like have it

01:43:15.600 --> 01:43:19.837
find a hotel discount for me.
>> No travel stories.

01:43:19.920 --> 01:43:22.157
I I I've been on a this has been for

01:43:22.240 --> 01:43:25.037
like two years. Every time I remember

01:43:25.120 --> 01:43:27.837
like Sundar actually it was I don't know

01:43:27.920 --> 01:43:32.397
if people remember in 2019 my example.
Wait, wait, no, no. I'm just

01:43:32.480 --> 01:43:34.637
saying. Why does everyone when they talk

01:43:34.720 --> 01:43:37.677
about Agentic talk about travel? I get

01:43:37.760 --> 01:43:39.997
>> cuz travel is such a pain in the ass.

01:43:40.080 --> 01:43:42.797
>> But it's I don't Okay, go on. Go on.

01:43:42.880 --> 01:43:46.797
>> I'm I'm I'm sheepish now. >> No,
no, no. Hotel discount. Hotel discount.

01:43:46.880 --> 01:43:50.157
>> I just looked at the hotel every hour
and when it dropped behind us below a

01:43:50.240 --> 01:43:52.397
certain threshold, emailed me.

01:43:52.480 --> 01:43:54.317
>> That's pretty good. >> Okay,
that's a clear goal. I will give

01:43:54.400 --> 01:43:57.757
you that as a clear goal. >> This is >>
I respect your I respect I think you're

01:43:57.840 --> 01:43:59.917
right that we definitely
need better examples than

01:44:00.000 --> 01:44:04.237
>> It's a pet peeve. It's just
for some reason every Apple the original

01:44:04.320 --> 01:44:07.517
ridiculous Bella Ramsey commercials
around Apple intelligence of core

01:44:07.600 --> 01:44:11.677
everything is flight booking and again
like >> you have killed that Bella Ramsey

01:44:11.760 --> 01:44:15.357
commercial so often they're going to the
creative agency is just going to write

01:44:15.440 --> 01:44:18.557
to you at some point >> and I
love the Last of Us and I love

01:44:18.640 --> 01:44:22.237
Bella Ramsey but those
commercials still irked me. Yeah.

01:44:22.320 --> 01:44:24.477
>> What happened in those commercials?

01:44:24.560 --> 01:44:28.317
uh one of them she's sitting at someone
that she can't remember who that person

01:44:28.400 --> 01:44:32.237
is and in real time asks Apple
intelligence like tell me about this

01:44:32.320 --> 01:44:36.477
person my interactions which again is just
such a weird thing like I'm so much

01:44:36.560 --> 01:44:40.237
better than you that I don't know who
you are and I need to remember who you

01:44:40.320 --> 01:44:43.117
are and have AI tell me but and it

01:44:43.200 --> 01:44:47.357
didn't even come close to working with
Apple intelligence so that was that was

01:44:47.440 --> 01:44:50.957
the worst one >> you know what would be
good for that actually >> what goal mode

01:44:51.040 --> 01:44:55.197
>> AR glasses >> oh AR glasses
is there's the real world use case

01:44:55.280 --> 01:45:00.077
>> or the fact that that commercial was so
bad just again proves my point that okay

01:45:00.160 --> 01:45:03.277
AR glasses aren't going to work. Okay,
sorry. Thank you so much for the question.

01:45:03.360 --> 01:45:06.717
>> Thank you and we'll go over here.

01:45:06.800 --> 01:45:12.557
>> Oh, he was >> okay fine.
We'll go here. Okay. Um, hi.

01:45:12.640 --> 01:45:15.917
Um, >> hey Gerald Harris.
Uh, I'm on the board

01:45:16.000 --> 01:45:19.677
of the Commonwealth Club here and I I
run some uh programs for the club and I

01:45:19.760 --> 01:45:22.877
have the small scenario planning
consultancy. But here's my question that

01:45:22.960 --> 01:45:26.317
I think hasn't come up here. Um,

01:45:26.400 --> 01:45:28.157
what what what should we be concerned

01:45:28.240 --> 01:45:31.917
about in terms of using the AI models

01:45:32.000 --> 01:45:34.237
them building a database on us and then

01:45:34.320 --> 01:45:36.877
turning that into advertising? So the

01:45:36.960 --> 01:45:39.757
advertising potential revenue from some

01:45:39.840 --> 01:45:42.317
from users, do you think the AI

01:45:42.400 --> 01:45:44.477
companies will ever go after that

01:45:44.560 --> 01:45:47.277
revenue or use that information for

01:45:47.360 --> 01:45:49.437
advertising purposes?

01:45:49.520 --> 01:45:51.997
>> Oo, that's a good one.
So, will will AIS

01:45:52.080 --> 01:45:55.357
build an amazing uh sort of profile of

01:45:55.440 --> 01:45:59.677
you based off of all the personal data
and then use that for ads? Most certainly.

01:45:59.760 --> 01:46:02.317
>> So, someone left one of the companies
and wrote about that in the New York

01:46:02.400 --> 01:46:05.197
Times about three months ago.
>> No, definitely. Yeah, that that's

01:46:05.280 --> 01:46:10.157
coming. Um, I I think that um
advertising is obviously going to be one

01:46:10.240 --> 01:46:13.757
use where we're going to start to see
some of the problematic stuff here, but

01:46:13.840 --> 01:46:17.757
actually if you look at the ads that
OpenAI gave you, it's sort of like more

01:46:17.840 --> 01:46:21.757
of a and obviously they they always come
out with the hightouch brand and then

01:46:21.840 --> 01:46:25.677
they will like get you on the direct
response like super targeted stuff um

01:46:25.760 --> 01:46:29.757
once they realize they can't make money
on brand. Um, but like it looks pretty

01:46:29.840 --> 01:46:33.277
good. Like if you're sorry to go with
the travel example again, but if you're

01:46:33.360 --> 01:46:37.357
like researching travel and chat GPT,
like you can go into an advertising chat

01:46:37.440 --> 01:46:39.357
experience and it will help you. But I I

01:46:39.440 --> 01:46:44.637
think that like we have never had
technology ever that collecting this

01:46:44.720 --> 01:46:49.517
much information about us. Has anyone
here um like talk to like chat GPT or

01:46:49.600 --> 01:46:51.357
Claude and say, "Can you psychoanalyze

01:46:51.440 --> 01:46:57.037
me if you're not or give me
any any insight about myself?

01:46:57.120 --> 01:47:00.317
Yeah, >> just like five of you.
The rest of you have done it.

01:47:00.400 --> 01:47:02.877
>> Admit it. >> It's scary
and and we're g and that's

01:47:02.960 --> 01:47:06.557
the like gated stuff and we're
giving this all this information to to

01:47:06.640 --> 01:47:10.637
companies and it's a a trust thing that we
don't really know where it's going to

01:47:10.720 --> 01:47:13.597
go. >> Yeah, I think
it's interesting because

01:47:13.680 --> 01:47:17.517
again, OpenAI originally ads are going
to be a big part of the business. Now

01:47:17.600 --> 01:47:22.157
they've pivoted away from that a bit
but are still releasing ad products. At

01:47:22.240 --> 01:47:24.317
least Anthropic is not I mean I don't

01:47:24.400 --> 01:47:25.677
think they're going in that direction at

01:47:25.760 --> 01:47:29.357
all. Google and Gemini obviously 100%

01:47:29.440 --> 01:47:32.717
will and like you probably will get a

01:47:32.800 --> 01:47:35.677
really good ad experience. I mean the

01:47:35.760 --> 01:47:40.717
more it knows about you the types
of questions you're asking but it is like

01:47:40.800 --> 01:47:42.637
you know like thousands and tens of

01:47:42.720 --> 01:47:44.717
thousands of three to five word Google

01:47:44.800 --> 01:47:48.557
searches are a lot different than entire

01:47:48.640 --> 01:47:51.837
thought partnership therapy like

01:47:51.920 --> 01:47:57.357
research exploration. these models will
know everything about you and it is it's

01:47:57.440 --> 01:47:59.997
terrifying. >> Have you done
the psychoanalyze me prompt?

01:48:00.080 --> 01:48:04.237
>> No, but I did so I use like Claude and

01:48:04.320 --> 01:48:08.957
Chat GPT Gemini. I'll kind of go through
the three of them and cycle around. So

01:48:09.040 --> 01:48:12.317
it is funny. So I have asked like what
do you know about me? Like what would

01:48:12.400 --> 01:48:14.797
you ask back? And they all have very

01:48:14.880 --> 01:48:17.517
different kind of like jagged jarring

01:48:17.600 --> 01:48:20.157
aggregations of who I am. Like I don't

01:48:20.240 --> 01:48:26.077
know. They're uh I I I don't have one
that just knows me through and through.

01:48:26.160 --> 01:48:27.517
>> I I've done it. >> I diversify them.

01:48:27.600 --> 01:48:29.037
>> I've done it with chat GPT.

01:48:29.120 --> 01:48:30.957
>> Yeah, >> it's pretty good. >> Yeah.

01:48:31.040 --> 01:48:33.517
>> And then like it will first give you a

01:48:33.600 --> 01:48:38.237
answer that's like kind of sanitized and
then you say go a little deeper and then

01:48:38.320 --> 01:48:42.157
you say get a little darker. >> Oh dear.

01:48:42.240 --> 01:48:45.197
>> I have not redteamed uh chat TV.

01:48:45.280 --> 01:48:47.677
>> I challenge you all to give it a shot.

01:48:47.760 --> 01:48:49.837
Don't go darker. Don't go darker.

01:48:49.920 --> 01:48:54.477
>> But if you really want to. >> Okay.
Thank you over here. And thank you

01:48:54.560 --> 01:48:56.077
for the question.
>> Let's move on. Let's move on.

01:48:56.160 --> 01:48:58.557
>> Yeah. >> Thank you guys
and really appreciate all

01:48:58.640 --> 01:49:02.877
the work that you do on the podcast
and YouTube. Uh your conversations go so

01:49:02.960 --> 01:49:05.597
deep and still very broad.
So I really appreciate that.

01:49:05.680 --> 01:49:08.157
>> Thank you. We appreciate it.
You're you're a listener or viewer.

01:49:08.240 --> 01:49:10.237
>> Oh yeah. All right.
Both. >> Thank you so much.

01:49:10.320 --> 01:49:12.637
>> Depends on where I am.
>> Amazing. Appreciate it.

01:49:12.720 --> 01:49:17.437
>> Um and you know my name is Si. I am
a adviser for startups in the AI space. Um

01:49:17.520 --> 01:49:19.437
and I've been in product management and

01:49:19.520 --> 01:49:22.477
go to market strategy. Um my question

01:49:22.560 --> 01:49:27.197
for you some somewhat related to the
gentleman before you know there's a lot

01:49:27.280 --> 01:49:29.437
of coverage on agents and what agents

01:49:29.520 --> 01:49:31.917
can do and uh a little bit about the

01:49:32.000 --> 01:49:34.557
governance and cost control. What I

01:49:34.640 --> 01:49:38.157
don't see a lot about um you know is you

01:49:38.240 --> 01:49:40.477
know agents do drift probably why we

01:49:40.560 --> 01:49:43.437
don't have better examples than travel.

01:49:43.520 --> 01:49:45.437
Um and you know over the a period of

01:49:45.520 --> 01:49:47.437
time they don't get better at that

01:49:47.520 --> 01:49:52.637
specific uh you know uh understanding
of the the people and the business context

01:49:52.720 --> 01:49:54.077
and the operating model they're working

01:49:54.160 --> 01:49:58.557
in um and you know learn from it and I

01:49:58.640 --> 01:50:01.197
was wondering is there is there a lot of

01:50:01.280 --> 01:50:02.877
conversation that you guys are seeing

01:50:02.960 --> 01:50:06.237
around uh the learning of uh or the

01:50:06.320 --> 01:50:09.277
learning layer in the agents and is it

01:50:09.360 --> 01:50:12.477
too early or you know um for that to

01:50:12.560 --> 01:50:16.877
happen And the conversation is mostly
around governance and cost and access.

01:50:16.960 --> 01:50:18.477
>> It's actually a good
question for Ronan.

01:50:18.560 --> 01:50:20.957
>> Yeah. So again,
like working in this all

01:50:21.040 --> 01:50:24.557
day and thank you for the question.
I I think what's going to happen and I'm

01:50:24.640 --> 01:50:27.117
already starting to see is like 6 to 12

01:50:27.200 --> 01:50:31.677
months ago the approach was just jam
as much information into whatever system

01:50:31.760 --> 01:50:36.237
you can. And we were kind of promised
that it would just work and AI is just

01:50:36.320 --> 01:50:39.997
going to and like there's you know you'd
get this feeling that oh wait a 100page

01:50:40.080 --> 01:50:44.317
PDF it actually could parse and I can
get information out of but then when

01:50:44.400 --> 01:50:46.397
you're doing that at any kind of agentic

01:50:46.480 --> 01:50:49.357
scale it doesn't and I think already

01:50:49.440 --> 01:50:52.077
like how information gets chunked up and

01:50:52.160 --> 01:50:54.397
like distributed in different ways and

01:50:54.480 --> 01:50:57.197
context windows everyone would see like

01:50:57.280 --> 01:51:02.157
1 million uh token context window but we
didn't really know what it meant and now

01:51:02.240 --> 01:51:05.037
more and more like and I actually think
this is going to be one of the most

01:51:05.120 --> 01:51:07.117
interesting like professional

01:51:07.200 --> 01:51:09.997
opportunities and areas to be an expert

01:51:10.080 --> 01:51:12.637
in going forward is like being able to

01:51:12.720 --> 01:51:17.677
kind of it's funny like it is as much
art as science right now. But I think

01:51:17.760 --> 01:51:21.277
like the more you can start to grab
a hold of how information flows through

01:51:21.360 --> 01:51:25.837
these agents and how you actually get
it reliable, it's going to be a really

01:51:25.920 --> 01:51:30.317
really interesting space.
It's not it's no one job right now.

01:51:30.400 --> 01:51:32.957
>> Thank you. Okay, thank you
for the question. All right, we have seven

01:51:33.040 --> 01:51:35.677
minutes, three questions. Let's see
if we can keep the questions brief and

01:51:35.760 --> 01:51:37.757
we'll keep the answers brief
and we'll get to everybody.

01:51:37.840 --> 01:51:41.117
>> I'll try to be quick.
>> Hey everyone, uh big fan of the pod by

01:51:41.200 --> 01:51:45.437
the way. Flew Chris Auei. I'm one of
the co-founders of a company called Virus

01:51:45.520 --> 01:51:47.757
Watcher. I'm here >> with my colleague.

01:51:47.840 --> 01:51:51.437
>> Flew in from Dallas.
He flew in from Austin. Yes. A big fan.

01:51:51.520 --> 01:51:53.357
>> We got a couple
Dallas folks here today.

01:51:53.440 --> 01:51:54.957
>> Oh, who's Dallas? Shout out Dallas.

01:51:55.040 --> 01:51:57.037
There we go. Okay. Go Matt. >> All right.

01:51:57.120 --> 01:51:59.837
>> Oh, yeah. We're big in Texas.

01:51:59.920 --> 01:52:04.157
Um I'm going to go in a different
direction uh with my question is uh I

01:52:04.240 --> 01:52:06.877
want to know kind of what y'all's
thoughts and honestly if you can ask

01:52:06.960 --> 01:52:08.397
this kind of later on with Greg Brockman

01:52:08.480 --> 01:52:11.597
and others um what is the thought on

01:52:11.680 --> 01:52:15.837
using AI models and AI you know
technology for bi biological

01:52:15.920 --> 01:52:20.397
intelligence bioveurveillance and public
health for emerging risk detection with

01:52:20.480 --> 01:52:23.677
infectious diseases is a very
interesting project that we're working

01:52:23.760 --> 01:52:27.677
on we have a lot of epidemiologists
on our team we were just at the UN two

01:52:27.760 --> 01:52:31.677
weeks ago with the World Health
Organization um and trying to discuss

01:52:31.760 --> 01:52:36.397
these problems and how we can use the
technology to to kind of uh detect these

01:52:36.480 --> 01:52:39.117
things early on. So I kind of want
to get you guys thoughts on that.

01:52:39.200 --> 01:52:41.277
>> Can I ask you
a question back and if you

01:52:41.360 --> 01:52:42.957
could give a brief answer
that would be great.

01:52:43.040 --> 01:52:46.717
>> Okay. >> Are you a believer in the
biocapabilities of these models like

01:52:46.800 --> 01:52:50.877
Fable? Like do you think Anthropic
made the right decision by gating them?

01:52:50.960 --> 01:52:54.797
>> Wait. Biocapabilities meaning I think
you're talking about biological weapons.

01:52:54.880 --> 01:52:56.637
He's talking about like disease correct.

01:52:56.720 --> 01:52:59.597
But I I think that biological his answer

01:52:59.680 --> 01:53:03.277
will give us insight into the

01:53:03.360 --> 01:53:06.477
>> I I mean I think it's a it's a very

01:53:06.560 --> 01:53:09.357
touchy you know it's a there's no right

01:53:09.440 --> 01:53:14.237
answer I don't think um but I do think
it probably did more harm uh you know

01:53:14.320 --> 01:53:17.277
than you know not allowing everyone
to kind of take advantage of the

01:53:17.360 --> 01:53:20.557
>> so you're not you're not basically
worried that today's cutting edge models

01:53:20.640 --> 01:53:25.597
could cause like new bio threats >>
no I I I am but I think there's also a

01:53:25.680 --> 01:53:29.437
lot of positive that come out of it
as well everybody to use it though.

01:53:29.520 --> 01:53:33.197
>> Yeah. I mean >> I I just
you know it doesn't even have

01:53:33.280 --> 01:53:37.677
to be a fable style model. It can be you
can take a frontier model distill a more

01:53:37.760 --> 01:53:41.197
um >> right >> custom model.
Yeah. Safety uh was you

01:53:41.280 --> 01:53:44.317
know safety standards involved
but sorry I don't want to be too long.

01:53:44.400 --> 01:53:47.117
No great question Ron John

01:53:47.200 --> 01:53:52.077
>> I in terms of the first part of it I
do think again like and it's one of the

01:53:52.160 --> 01:53:56.397
underappreciated things and I feel
in the AI industry like disease prevention

01:53:56.480 --> 01:54:01.277
and scenario modeling around vir like
we don't talk about that stuff enough or

01:54:01.360 --> 01:54:05.997
whoever is trying to talk about it it
does not get listened to enough so I

01:54:06.080 --> 01:54:10.637
love the kind of work you're doing
and like those kind of stories would bring

01:54:10.720 --> 01:54:14.637
the AI industry a little bit better
reputation Yeah. No, I I I agree.

01:54:14.720 --> 01:54:15.997
>> You're thinking
about biological weapons.

01:54:16.080 --> 01:54:18.957
>> No, I think that that if Well, if you
can do one, then you could certainly do

01:54:19.040 --> 01:54:23.117
the other. Am I >> I don't know.
I'm not >> Is one like Okay.

01:54:23.200 --> 01:54:27.117
>> I actually have no idea. >> Why are you
giving me this all these problems here?

01:54:27.200 --> 01:54:30.877
>> Uh no, I I I hope that we
can and we know that's a very

01:54:30.960 --> 01:54:33.117
>> No, no, no. I love it.
>> We're going to speak with we'll we'll

01:54:33.200 --> 01:54:35.917
hopefully have some time to talk
about health with Brockman.

01:54:36.000 --> 01:54:37.597
>> Yeah, I would love Yeah, if you can.

01:54:37.680 --> 01:54:39.837
Okay. >> Okay.
Thank you for the question over

01:54:39.920 --> 01:54:43.597
here. Hi. Uh my name is Heidi. I'm your

01:54:43.680 --> 01:54:47.277
linking online friend. >> That's right.
You messaged me yesterday.

01:54:47.360 --> 01:54:51.197
Yes. >> Yeah. Yeah.
Uh during this uh event uh

01:54:51.280 --> 01:54:53.757
see each other in person. Uh my question

01:54:53.840 --> 01:54:56.797
about uh before you with a different

01:54:56.880 --> 01:54:59.437
speaker mention about China. So I'm

01:54:59.520 --> 01:55:02.637
Chinese. I'm curious about uh why you

01:55:02.720 --> 01:55:05.437
think uh Chinese uh China is important

01:55:05.520 --> 01:55:09.757
base partnership with uh uh USA special

01:55:09.840 --> 01:55:12.237
high-tech. Uh second question about R.

01:55:12.320 --> 01:55:16.397
So you mentioned about RM large model.

01:55:16.480 --> 01:55:20.797
So my question about uh Gemini JDP and

01:55:20.880 --> 01:55:24.477
cloudy and an which one you prefer think

01:55:24.560 --> 01:55:28.797
about in future is more stronger
smarter in the future? Thank you.

01:55:28.880 --> 01:55:30.477
>> Thank you. Thank you for the question.

01:55:30.560 --> 01:55:33.117
>> Well, the question
on China is is sorry,

01:55:33.200 --> 01:55:35.197
what's the question on China?

01:55:35.280 --> 01:55:38.317
>> No, no. Why you think it's important uh

01:55:38.400 --> 01:55:42.157
Bis relationship with USA? So, yeah.

01:55:42.240 --> 01:55:44.397
Thank you. >> You know,
all right. I will I will take

01:55:44.480 --> 01:55:49.037
that question and then Ranjan will
answer which model he likes better. Um

01:55:49.120 --> 01:55:52.717
you know, I I think that we're obviously
in this competition. The US and China

01:55:52.800 --> 01:55:57.917
are in this competition and there's like
a lot of suspicion on both ends. And um

01:55:58.000 --> 01:56:00.717
you know I've said in the past that like
it would be great if there was better

01:56:00.800 --> 01:56:04.557
comp cooperation between the two
countries and people have told me that

01:56:04.640 --> 01:56:09.117
I'm naive um but I'm not going to you
know lose that hope. I think that if if

01:56:09.200 --> 01:56:12.237
I mean in some ways competition's good
right because then you'll just have

01:56:12.320 --> 01:56:16.237
everybody striving to build a better
thing. But you know we just talked about

01:56:16.320 --> 01:56:19.997
uh AI for health if the countries were
able to work together maybe on specific

01:56:20.080 --> 01:56:22.077
initiatives um that would be much

01:56:22.160 --> 01:56:25.197
welcome. I I think the most interesting

01:56:25.280 --> 01:56:28.477
part of like US versus or US with China

01:56:28.560 --> 01:56:33.437
in terms of AI in the last few months,
the conversation around moving towards

01:56:33.520 --> 01:56:38.077
deepseek and adding it into your agentic
process or adding Chinese models did not

01:56:38.160 --> 01:56:42.637
exist in any conversation I was in 12
months ago and now is in almost every

01:56:42.720 --> 01:56:45.037
conversation at least as an option

01:56:45.120 --> 01:56:48.397
because cost has become such a larger

01:56:48.480 --> 01:56:52.237
part of the conversation. So I think
I think it's a good thing. I think the

01:56:52.320 --> 01:56:54.877
more integration there is overall across

01:56:54.960 --> 01:57:00.280
these systems, the better.
>> Yeah, but watch out for

01:57:01.760 --> 01:57:04.077
>> the cat holding
the hands with Dario and

01:57:04.160 --> 01:57:06.077
Sam is my favorite meme the whole year.

01:57:06.160 --> 01:57:08.957
Okay, we have one minute left,
but let's see if we can do it in a minute.

01:57:09.040 --> 01:57:10.157
>> All right, I'll go
through this quickly.

01:57:10.240 --> 01:57:13.437
Hi, Maria Sarah Roberts.
I'm in business management consulting.

01:57:13.520 --> 01:57:15.037
>> My question is a little bit different.

01:57:15.120 --> 01:57:18.477
Um, as we are evolving into or entering

01:57:18.560 --> 01:57:23.037
or in a world already where endless
possibilities are available, where and

01:57:23.120 --> 01:57:26.877
how do you see the evolution of the
responsibilities that big tech companies

01:57:26.960 --> 01:57:31.597
have in supporting making this world
better? Not the product and our

01:57:31.680 --> 01:57:33.277
engagement and how we interact with all

01:57:33.360 --> 01:57:35.757
the screens better, but rather like the

01:57:35.840 --> 01:57:38.717
real big problems that the world has.

01:57:38.800 --> 01:57:41.517
And not because they should, but because

01:57:41.600 --> 01:57:46.317
they can. So, how do you think about
like the evolution of that as we step

01:57:46.400 --> 01:57:50.957
into this world? >> I I I'll answer that.
Or do you want to answer? Go ahead.

01:57:51.040 --> 01:57:54.717
>> I just we we had a backstage we had
a long rant because I'd forgotten that

01:57:54.800 --> 01:57:56.877
Anthropic is a PBC public benefit

01:57:56.960 --> 01:57:59.357
corporation, but I still think there is

01:57:59.440 --> 01:58:02.157
a responsibility. I don't actually see

01:58:02.240 --> 01:58:04.077
it happening in any kind of way because

01:58:04.160 --> 01:58:06.637
competition is so fierce now. I wish it

01:58:06.720 --> 01:58:10.157
would, but I don't see it happening. Um,

01:58:10.240 --> 01:58:13.357
yeah. I um that that was a good answer.

01:58:13.440 --> 01:58:16.957
Come on. We have we I want to make
people make sure everybody gets to

01:58:17.040 --> 01:58:20.477
coffee. Oh, no. Screw it.
I'll answer it. Um,

01:58:20.560 --> 01:58:23.117
so so Jeff Bezos so I think they have an

01:58:23.200 --> 01:58:25.917
obligation to do good for the world and

01:58:26.000 --> 01:58:31.037
um I don't know if and I know people
within the companies are you know are

01:58:31.120 --> 01:58:34.637
very serious about that. So they have
their own side projects, but I think you

01:58:34.720 --> 01:58:37.997
know as a society maybe we can't expect
them to do it out of the goodness of

01:58:38.080 --> 01:58:42.477
their own hearts. Um Jeff Bezos was
recently doing an interview and he said

01:58:42.560 --> 01:58:46.077
um he said I think what I do in the
private sector is going to outweigh

01:58:46.160 --> 01:58:50.157
whatever I could do charitably.
Um I disagree with that and I think that

01:58:50.240 --> 01:58:53.917
that's just the mindset of a lot of people
at the top of these companies and

01:58:54.000 --> 01:58:58.797
so like everybody else needs to be aware
of that and not count on it even though

01:58:58.880 --> 01:59:01.837
it would be nice.
So that's my perspective.

01:59:01.920 --> 01:59:04.957
All right. Uh, Ronjan, will you come
back with us after the break and speak

01:59:05.040 --> 01:59:09.997
with Alex Stamos and I about whether um
this mythos and fable stuff is material

01:59:10.080 --> 01:59:11.917
or marketing?
>> You know, I would love to.

01:59:12.000 --> 01:59:13.677
>> You guys want to get a coffee break?

01:59:13.760 --> 01:59:18.360
>> You guys are amazing.
>> Thank you very much.

01:59:18.720 --> 01:59:23.357
>> All right, we'll we'll be back here
3:25. Um, please enjoy the coffee and

01:59:23.440 --> 01:59:27.037
we'll see you in a couple minutes.
So, you may have heard that AI is in some

01:59:27.120 --> 01:59:31.437
trouble in Washington. Um, and that
story has led to a lot of guessing.

01:59:31.520 --> 01:59:33.437
People have guessed, are these models

01:59:33.520 --> 01:59:35.837
actually that dangerous? How much better

01:59:35.920 --> 01:59:40.717
are they than previous generations? And
do they deserve to be restricted by the

01:59:40.800 --> 01:59:44.237
federal government? We've certainly done
a lot of guessing. And I think the only

01:59:44.320 --> 01:59:48.477
way to get to the bottom of this is to
speak with somebody who is deep in the

01:59:48.560 --> 01:59:50.957
weeds on the cyber security aspects of

01:59:51.040 --> 01:59:55.677
AI models, someone who secures them,
and someone who has spent a career working

01:59:55.760 --> 01:59:59.917
in security. And that would be Alex
Stamos. Alex Damos is the former chief

02:00:00.000 --> 02:00:04.157
security officer at Meta and he's
the current chief product officer at

02:00:04.240 --> 02:00:06.397
Corridor. Um Ranjan and I have this

02:00:06.480 --> 02:00:09.037
debate all the time. Is what's going on

02:00:09.120 --> 02:00:13.517
with Mythos marketing or material? Well,
let's get to the bottom of it. Ranjan

02:00:13.600 --> 02:00:17.197
will come out and be my co-er for this
one. So, ladies and gentlemen, please

02:00:17.280 --> 02:00:23.000
welcome Alex Damos
and once again Ranjan Roy.

02:00:25.760 --> 02:00:29.197
>> Thank you. >> Hey, thanks man.

02:00:29.280 --> 02:00:31.597
Alex, boring times in your industry.

02:00:31.680 --> 02:00:34.157
>> Yeah, it's an honor to be here. I am by

02:00:34.240 --> 02:00:36.477
far the poorest person in the afternoon

02:00:36.560 --> 02:00:41.597
lineup. It's like I bring down the the
mean uh net worth by a comma, I believe.

02:00:41.680 --> 02:00:43.117
So, I do appreciate being here.

02:00:43.200 --> 02:00:47.837
>> Well, Alex, um I don't want to quue
you into podcaster finances, but

02:00:47.920 --> 02:00:50.157
>> I think you're good >> with the guests,
I guess. Yeah. Yeah. So

02:00:50.240 --> 02:00:53.357
talk a little bit about um we're
obviously at this moment where the

02:00:53.440 --> 02:00:57.357
government has pulled down or or
basically forced anthropic to pull down

02:00:57.440 --> 02:01:01.197
fable or mythos uh its frontier
model >> right both of them. Yes.

02:01:01.280 --> 02:01:02.477
>> What happened?

02:01:02.560 --> 02:01:05.277
>> Right. So uh just for recap uh for

02:01:05.360 --> 02:01:08.637
everybody uh last Friday uh what

02:01:08.720 --> 02:01:10.237
happened was the white house became

02:01:10.320 --> 02:01:12.877
aware of research that Amazon had did

02:01:12.960 --> 02:01:15.837
had done. Um it is not unreasonable for

02:01:15.920 --> 02:01:17.917
Amazon to do this work right. So uh

02:01:18.000 --> 02:01:21.997
Amazon hosts uh anthropics models in in

02:01:22.080 --> 02:01:25.277
what they call bedrock. Uh this includes

02:01:25.360 --> 02:01:27.357
in all of the classified spaces. So if

02:01:27.440 --> 02:01:29.837
the NSA is using anthropics models, it

02:01:29.920 --> 02:01:32.557
is running in AWS secret cloud or top

02:01:32.640 --> 02:01:34.877
secret cloud. Uh and so Amazon is doing

02:01:34.960 --> 02:01:37.517
security testing. expect this was part

02:01:37.600 --> 02:01:39.757
of getting ready to host Fable and

02:01:39.840 --> 02:01:43.597
Bedrock and uh found some things that

02:01:43.680 --> 02:01:47.837
they wanted to complain about with
Fable's protections that make Fable

02:01:47.920 --> 02:01:49.277
different than Mythos. I guess we can

02:01:49.360 --> 02:01:53.117
talk about the exact ones. Uh Amazon had

02:01:53.200 --> 02:01:55.517
sent over these results to Anthropic.

02:01:55.600 --> 02:01:59.037
There was some disagreement between
Anthropic and Amazon during the week of

02:01:59.120 --> 02:02:03.437
how serious they were. Now, I don't
have the exact details here, but somehow

02:02:03.520 --> 02:02:07.757
apparently Andy Jasse, the CEO of Amazon,
mentions this to somebody at the

02:02:07.840 --> 02:02:09.997
White House, and the White House freaks

02:02:10.080 --> 02:02:14.157
out and uh according to reporting uh

02:02:14.240 --> 02:02:18.077
calls Amazon or Anthropic uh and uh is

02:02:18.160 --> 02:02:21.757
not happy with how quickly they can
get Enthropic on the phone. Anthropic is

02:02:21.840 --> 02:02:23.677
asking for more technical details.

02:02:23.760 --> 02:02:25.997
They're not able to get details from
the White House of what the White House

02:02:26.080 --> 02:02:27.997
wants. So, the White House orders them

02:02:28.080 --> 02:02:31.757
to take Fable down. Uh, Enthropic says,

02:02:31.840 --> 02:02:35.677
"No, we can just fix these issues. Why
don't we work it out together?" And as a

02:02:35.760 --> 02:02:38.317
result, instead of working out together

02:02:38.400 --> 02:02:40.557
on Friday afternoon, something around

02:02:40.640 --> 02:02:42.797
5:00 p p.m. Pacific time, the Commerce

02:02:42.880 --> 02:02:45.677
Department signs an order saying that

02:02:45.760 --> 02:02:50.317
the models are designated export
controlled and that no foreigner,

02:02:50.400 --> 02:02:54.317
including the foreign citizens who
worked and actually built the models

02:02:54.400 --> 02:02:58.397
themselves, can can touch them.
uh whether or not the administration

02:02:58.480 --> 02:03:02.237
actually has that legal power
is disputed, but Anthropic decides to

02:03:02.320 --> 02:03:05.277
actually follow through instead
of immediately getting a temporary

02:03:05.360 --> 02:03:09.677
restraining order. And they because they
have no ability to make sure that no uh

02:03:09.760 --> 02:03:11.757
foreign uh citizen is is seeing the

02:03:11.840 --> 02:03:14.157
models actually pulls them down. Uh both

02:03:14.240 --> 02:03:19.117
Mythos uh which was privately accessible
as well as Fable uh which was publicly

02:03:19.200 --> 02:03:22.077
available. >> Now you've spent
a career working in security.

02:03:22.160 --> 02:03:25.597
>> Yes. Um Ron John and I for months now

02:03:25.680 --> 02:03:28.477
have had this debate about whether all

02:03:28.560 --> 02:03:31.837
of this talk of doom and super powerful

02:03:31.920 --> 02:03:34.477
models from anthropic like for instance

02:03:34.560 --> 02:03:37.437
mythos a model so powerful that you

02:03:37.520 --> 02:03:39.517
can't access it. Um whether that

02:03:39.600 --> 02:03:42.797
represented a real capability increase

02:03:42.880 --> 02:03:45.517
or whether that was just marketing and

02:03:45.600 --> 02:03:48.637
it's a step change uh not not really a

02:03:48.720 --> 02:03:52.557
step change in capabilities. So I
actually want to toss it to Rajan who is

02:03:52.640 --> 02:03:56.317
firmly on the marketing side of this
thing just to make the argument of why

02:03:56.400 --> 02:04:00.397
you think this is marketing and Alex
can address that. Yeah, I think trying to

02:04:00.480 --> 02:04:03.117
understand like what is that capabil new

02:04:03.200 --> 02:04:05.197
capability that is so dangerous? What is

02:04:05.280 --> 02:04:07.597
that step change? I think the reason I

02:04:07.680 --> 02:04:10.557
get so suspect on it is how perfectly

02:04:10.640 --> 02:04:15.197
coordinate coordinated a lot of the
marketing and communications are around

02:04:15.280 --> 02:04:18.877
how they've rolled these out.
Like again, mythos is the most dangerous

02:04:18.960 --> 02:04:22.797
thing. There's someone eating a sandwich
in a park that actually got like where

02:04:22.880 --> 02:04:25.837
the model broke out of containment was

02:04:25.920 --> 02:04:30.397
like a PR release. It was it was a very
coordinated thing. So what is that

02:04:30.480 --> 02:04:34.957
danger that they're trying to kind
of bring to the world? Like what is it

02:04:35.040 --> 02:04:37.837
really? What does it look like, feel like,
or what is it actually going to be

02:04:37.920 --> 02:04:41.517
a problem? >> Uh so how much money
do you guys have writing on this?

02:04:41.600 --> 02:04:43.437
>> Well, how would we even play this?
Like I don't even know.

02:04:43.520 --> 02:04:46.237
>> Oh, like as a bet? >> As a bet?
Like is there a poly market?

02:04:46.320 --> 02:04:50.557
Uh yeah, I'm trying to see what
my angle is here. Apparently I have

02:04:50.640 --> 02:04:54.797
>> Well, well, no, you are the person that
knows Trump action. you know the answers

02:04:54.880 --> 02:04:58.877
here and we have just been kind of like
debating >> just like we just argue on.

02:04:58.960 --> 02:05:00.637
>> So okay so it's
actually it's actually a

02:05:00.720 --> 02:05:02.717
little bit of a complicated answer. So

02:05:02.800 --> 02:05:07.757
um mythos is almost it is the best

02:05:07.840 --> 02:05:11.117
bugfinding model that we know about for

02:05:11.200 --> 02:05:14.557
sure. It is most likely the best bug

02:05:14.640 --> 02:05:19.197
finding model in the world. And I say
most likely because we do not know what

02:05:19.280 --> 02:05:21.037
the Chinese secretly have in their labs.

02:05:21.120 --> 02:05:23.597
But for all the stuff for which we have

02:05:23.680 --> 02:05:27.677
open known security evaluations for it

02:05:27.760 --> 02:05:32.077
scores the best. Now, is it the avenging

02:05:32.160 --> 02:05:38.477
cyber god that anthropic makes it out to
be? I personally don't think so. Yes, it

02:05:38.560 --> 02:05:40.877
is. >> You're not necessar
you're not right yet,

02:05:40.960 --> 02:05:42.957
>> but but it is it is really good. It is

02:05:43.040 --> 02:05:45.677
really good at finding bugs. But from my

02:05:45.760 --> 02:05:51.197
perspective, the like Opus 48, GPT 5.5,
a bunch of other models are here in

02:05:51.280 --> 02:05:53.437
Mythosis here >> at bug finding

02:05:53.520 --> 02:05:56.317
>> at bug finding and exploit development

02:05:56.400 --> 02:06:00.797
and it is not here, which is kind
of where you would think from reading all

02:06:00.880 --> 02:06:04.877
this stuff. The other thing is that
everybody pretends that Mythos was the

02:06:04.960 --> 02:06:07.437
breakthrough, but that is not true. For
people in the security world who've been

02:06:07.520 --> 02:06:09.277
paying attention, this Rubicon was

02:06:09.360 --> 02:06:11.757
crossed well last year

02:06:11.840 --> 02:06:14.637
>> with the Opus 4 series with the GPD5

02:06:14.720 --> 02:06:17.197
series. That is when the models became

02:06:17.280 --> 02:06:22.557
better or as good or or better than
the best human bugfinders and not just as

02:06:22.640 --> 02:06:25.597
good as the best bug finders.
And there's like 50 or 100 of these people

02:06:25.680 --> 02:06:28.477
and I know a couple dozen of them.
The problem is those people are very

02:06:28.560 --> 02:06:31.357
expensive, right? And they don't scale.

02:06:31.440 --> 02:06:35.037
You can't just throw water on them
and make them multiply like gremlins. That

02:06:35.120 --> 02:06:39.837
would be nice. I mean, certainly the NSA
would love that, but models can you can

02:06:39.920 --> 02:06:41.677
just scale them as much as you want with

02:06:41.760 --> 02:06:45.517
money and power and and and GPUs. And so

02:06:45.600 --> 02:06:48.717
that last year was a huge deal because

02:06:48.800 --> 02:06:51.677
all of a sudden you saw these amateur

02:06:51.760 --> 02:06:53.997
bugfinders. Imagine you went to a high

02:06:54.080 --> 02:06:58.797
school track meet and every single kid
is posting Olympic world record times.

02:06:58.880 --> 02:07:02.557
You'd be like, "Man, they're juicing,
right? That something had happened."

02:07:02.640 --> 02:07:04.317
That is what last year was like. like

02:07:04.400 --> 02:07:07.517
mid la m mid to late last year in the

02:07:07.600 --> 02:07:12.077
vulnerability discovery world is the
kids started juicing right and it's

02:07:12.160 --> 02:07:15.757
because the models hit this point where
they could find everything and so yes

02:07:15.840 --> 02:07:18.797
mythos is really good at finding bugs
it's really good at writing exploits

02:07:18.880 --> 02:07:21.437
fable is not and so that is that is what

02:07:21.520 --> 02:07:24.637
the whole point of fable is it is the

02:07:24.720 --> 02:07:28.957
same model weights as mythos
but anthropic put a protection in place to

02:07:29.040 --> 02:07:31.437
keep it from doing the really nasty

02:07:31.520 --> 02:07:35.117
stuff that mythos will do And anth uh

02:07:35.200 --> 02:07:37.277
Amazon has some complaints about that.

02:07:37.360 --> 02:07:41.437
But what happened
is the complaints Amazon has

02:07:41.520 --> 02:07:45.837
uh a number of us have looked at them
and then Enthropic has pushed back even

02:07:45.920 --> 02:07:48.237
with the jailbreaks that Amazon has. You

02:07:48.320 --> 02:07:50.317
cannot get Fable to do things that you

02:07:50.400 --> 02:07:53.357
can't do with the Opus series with GPD5

02:07:53.440 --> 02:07:55.517
and even with a bunch of Chinese models.

02:07:55.600 --> 02:07:58.957
So that is why the the actions by the
Trump administration make absolutely no

02:07:59.040 --> 02:08:01.757
sense because those models will not

02:08:01.840 --> 02:08:06.077
refuse at all. You can just ask GPD5
to go find these bugs and it will go do it

02:08:06.160 --> 02:08:09.437
for you. Um, you can just ask Kimmy,
which is an open source Chinese model.

02:08:09.520 --> 02:08:14.237
It will go write you these exploits. And
so, yes, you can trick Fable into doing

02:08:14.320 --> 02:08:18.317
certain things, but you can't trick it
into doing the really powerful Mythos

02:08:18.400 --> 02:08:22.237
stuff, which is like grind on the Linux
kernel for all day and find a bunch of

02:08:22.320 --> 02:08:25.357
bugs, which will cost you hundreds
of thousands of dollars, it turns out, if

02:08:25.440 --> 02:08:29.757
you pay the full price. Um, and those
have not been jailbroken. And so there's

02:08:29.840 --> 02:08:31.437
no real good reason for what happened.

02:08:31.520 --> 02:08:35.277
So yes, Mythos is really powerful. It
has only been accessible to a very small

02:08:35.360 --> 02:08:37.997
number of companies. It is not magically

02:08:38.080 --> 02:08:39.997
powerful. The other thing you have to

02:08:40.080 --> 02:08:43.357
remember here is like bugs are cheap.

02:08:43.440 --> 02:08:48.317
Exploits are like we're now drowning in
bugs because these models are so good at

02:08:48.400 --> 02:08:51.277
finding and we have so much bad
software. We've been writing bad

02:08:51.360 --> 02:08:53.357
software forever. And it turns out that

02:08:53.440 --> 02:08:56.237
like writing human beings writing code

02:08:56.320 --> 02:08:59.197
in C and C++ especially was a really bad

02:08:59.280 --> 02:09:00.877
idea that like we should not have been

02:09:00.960 --> 02:09:04.077
writing like building our entire lives

02:09:04.160 --> 02:09:09.037
like based upon memory unsafe languages
and then in the presence of these

02:09:09.120 --> 02:09:11.037
superhuman bugfinders not just mythos

02:09:11.120 --> 02:09:12.877
but all of them that was not a good

02:09:12.960 --> 02:09:16.717
idea. And so, uh, if you're a pedestrian

02:09:16.800 --> 02:09:19.677
and you get hit by a F-150 at 80 miles

02:09:19.760 --> 02:09:22.397
per hour or a McLaren at 200 miles per

02:09:22.480 --> 02:09:24.797
hour, you don't really care. You're

02:09:24.880 --> 02:09:27.677
dead, right? And that's the difference

02:09:27.760 --> 02:09:30.797
here is like the mythos is the F1.

02:09:30.880 --> 02:09:32.157
Mythos is the F1.

02:09:32.240 --> 02:09:35.357
>> Yeah, exactly.
And Kimmy or GLM52 is the

02:09:35.440 --> 02:09:39.437
F-150. And so you could take the the the

02:09:39.520 --> 02:09:43.997
McLaren off the market, but all you've
done is actually hurt the defenders who

02:09:44.080 --> 02:09:48.157
want to have full coverage, right? Who
want to be able to find all the bugs. An

02:09:48.240 --> 02:09:52.157
attacker only needs one or two bugs to
string together to actually pull off the

02:09:52.240 --> 02:09:57.117
attack. And so that's why, you know,
so about 150 of us wrote this open letter

02:09:57.200 --> 02:10:00.317
and signed it saying this is really
stupid because all you're doing is

02:10:00.400 --> 02:10:02.957
hurting defenders and you're also really

02:10:03.040 --> 02:10:05.437
hurting the US AI industry. Okay, I

02:10:05.520 --> 02:10:07.677
think Ron John wins the debate.

02:10:07.760 --> 02:10:11.517
>> Well, no, still it's
not all marketing, but it's

02:10:11.600 --> 02:10:15.677
>> it's not all marketing,
but like I I just want to say like

02:10:15.760 --> 02:10:19.517
mythos is really powerful, but doesn't
really matter because there's so many

02:10:19.600 --> 02:10:23.517
bugs out there that you can use any
of these models to find them. I what I'd

02:10:23.600 --> 02:10:26.237
like to do is I'd like to shift
the debate one away from finding bugs

02:10:26.320 --> 02:10:30.237
because the other problem here is like
we cannot set the standard that US AI

02:10:30.320 --> 02:10:34.397
models can't find bugs. That is a
terrible terrible terrible standard. If

02:10:34.480 --> 02:10:38.637
you have a piece if you are writing
software with an LLM, it has to be able

02:10:38.720 --> 02:10:41.357
to understand what a bug looks like. So

02:10:41.440 --> 02:10:44.397
it does not write those bugs. It has to.

02:10:44.480 --> 02:10:48.637
And that is that's the trick actually
Amazon used was they tricked it into

02:10:48.720 --> 02:10:51.037
basically finding bugs in individual

02:10:51.120 --> 02:10:53.117
lines of code because it has to be able

02:10:53.200 --> 02:10:56.477
to do that to write secure code. If if

02:10:56.560 --> 02:11:00.477
our if American LMS know nothing about
security, they will create more security

02:11:00.560 --> 02:11:02.957
flaws. So we cannot create a standard

02:11:03.040 --> 02:11:07.837
where American LMS are dumb about
security. That would be a humongous own

02:11:07.920 --> 02:11:09.597
goal and that is one of the possible

02:11:09.680 --> 02:11:12.877
outcomes here of of this whole brewhaha.

02:11:12.960 --> 02:11:15.917
That would be really stupid. And so uh

02:11:16.000 --> 02:11:20.557
you know my I think yes I I don't think
either one of you wins. I mean as the

02:11:20.640 --> 02:11:23.917
referee here I think Ron John has
a little bit of a you have a point but

02:11:24.000 --> 02:11:26.557
Mythos is really good. It just

02:11:26.640 --> 02:11:29.757
>> it doesn't Anthropic has pleaded up too

02:11:29.840 --> 02:11:32.477
much. Um, OpenAI is rumored to be

02:11:32.560 --> 02:11:33.917
releasing something that's mythos

02:11:34.000 --> 02:11:37.357
quality really soon. Um, and I doubt

02:11:37.440 --> 02:11:41.517
that they're gonna like follow they'll
just give it a number like uh I think

02:11:41.600 --> 02:11:44.797
they've learned from anthropic you don't
call your thing like the cyber gorgon.

02:11:44.880 --> 02:11:47.917
Um, >> uh, >> that actually would sell.

02:11:48.000 --> 02:11:52.557
>> Yeah, yeah, it sells a little too well,
right? Um, right. Maybe as a Greek with

02:11:52.640 --> 02:11:55.597
the Odyssey coming out, I'm getting
a little sensitive about the the marketing

02:11:55.680 --> 02:11:57.597
here, but like like you know, let's

02:11:57.680 --> 02:11:59.837
let's like back off on the naming scheme

02:11:59.920 --> 02:12:05.197
and move back to numbers. Uh uh but like
I I think a a big part of this is we

02:12:05.280 --> 02:12:08.317
just have to like reset the standard
of like not bug finding but the actual

02:12:08.400 --> 02:12:11.517
creating and creation of exploits and then
other kinds of offensive operations

02:12:11.600 --> 02:12:13.917
of which there's a bunch of other
things you can do offensively with these

02:12:14.000 --> 02:12:17.997
models. Those are the things that we
should call off limits not bug finding

02:12:18.080 --> 02:12:21.357
because that is something that we have
to do much more quickly than we're doing

02:12:21.440 --> 02:12:23.517
right now. C >> can I I just
want to try to articulate

02:12:23.600 --> 02:12:26.477
my point which is you know this is all

02:12:26.560 --> 02:12:30.797
right. So when we talk about is mythos
material or is mythos marketing um my

02:12:30.880 --> 02:12:34.397
argument would be like wouldn't if
something was material it would look a

02:12:34.480 --> 02:12:38.157
lot like what we're seeing today right
like at a certain point these models are

02:12:38.240 --> 02:12:42.397
increasing in capabilities you're going
to want to put some safeguards on them

02:12:42.480 --> 02:12:46.157
and so is what anthropic is doing that
different from like where you'd want to

02:12:46.240 --> 02:12:49.357
be if you actually had these concerns

02:12:49.440 --> 02:12:51.677
>> I mean I think both
anthropic and openai

02:12:51.760 --> 02:12:55.757
have a path in which they have like
a KYC model now >> know your customer

02:12:55.840 --> 02:12:59.437
>> a know your customer. Yeah. So, OpenAI
actually has a couple levels. They have

02:12:59.520 --> 02:13:03.117
a public one where you can basically say,
I want access to cyber capabilities

02:13:03.200 --> 02:13:06.557
and then they have a private one.
I believe Anthropic only has the private

02:13:06.640 --> 02:13:09.437
one, right, to get access to mythos. Uh

02:13:09.520 --> 02:13:13.997
but uh I I do think it's reasonable
to have some level of gating, but in the

02:13:14.080 --> 02:13:19.197
case of GlassW, I think it was honestly
it I thought it was not widespread

02:13:19.280 --> 02:13:21.837
enough. I know of like critical
infrastructure companies that don't have

02:13:21.920 --> 02:13:25.837
access to mythos right now.
Um >> but nobody does right now. Well, no,

02:13:25.920 --> 02:13:29.517
right now before last week, like people
who should have had access did not have

02:13:29.600 --> 02:13:31.837
access. Now, seeing what happened here,

02:13:31.920 --> 02:13:34.397
you can see why they were so careful.

02:13:34.480 --> 02:13:37.997
Um, but from my perspective as

02:13:38.080 --> 02:13:40.397
defenders, we are in a race against

02:13:40.480 --> 02:13:45.357
time, right? We have all of these bugs
that need to be fixed. And we also have

02:13:45.440 --> 02:13:49.917
this weird thing where we're like, this
whole conversation is predicated on the

02:13:50.000 --> 02:13:51.437
idea that like the American labs are

02:13:51.520 --> 02:13:54.557
years ahead of our adversaries. And that

02:13:54.640 --> 02:13:57.357
is just not true, right? So while Fable

02:13:57.440 --> 02:14:00.637
was shut down on Tuesday of this week,

02:14:00.720 --> 02:14:03.197
GLM 5.2 was shipped right from Z.AI,

02:14:03.280 --> 02:14:07.837
which is a Chinese lab. This is an
openweight model. It is MIT licensed. So

02:14:07.920 --> 02:14:11.757
any of you can go not just download
the weights from HuggingFace, but you can

02:14:11.840 --> 02:14:13.357
include it in your own product and you

02:14:13.440 --> 02:14:15.917
can retrain it. It is within percentage

02:14:16.000 --> 02:14:20.077
points of the top closed models from

02:14:20.160 --> 02:14:22.477
anthropic and open AI in a bunch of

02:14:22.560 --> 02:14:27.277
things. um a bunch of of evals. We don't
know how good it is at bugf finding yet.

02:14:27.360 --> 02:14:30.317
So there hasn't been any good testing
here. But in coding and a bunch of other

02:14:30.400 --> 02:14:34.317
intelligence tasks, it's almost as good
as the best the United States has. And

02:14:34.400 --> 02:14:39.277
that's the best Chinese model we know
about, right? Unlike American labs, the

02:14:39.360 --> 02:14:43.357
Chinese are not going to announce their
mythos, right? And so the idea that we

02:14:43.440 --> 02:14:47.437
should be doing everything in the United
States to just be playing defense is

02:14:47.520 --> 02:14:51.197
we're playing like a a defensive
model here, that is stupid from my

02:14:51.280 --> 02:14:55.517
perspective. We need to be accelerating
our capability to do these things and

02:14:55.600 --> 02:14:59.117
not just playing defense. >> Well,
now I am sufficiently scared even

02:14:59.200 --> 02:15:02.477
though I've thought that the mythos
was pure marketing. But but what do we do

02:15:02.560 --> 02:15:06.797
about it? Like how do we what
is that like and I I think I share that

02:15:06.880 --> 02:15:10.637
glasswing again why it felt like
marketing to me. It was this kind of

02:15:10.720 --> 02:15:14.397
exclusive club that people were invited
to and I actually spoke with people who

02:15:14.480 --> 02:15:18.237
are using it and they spoke about it
almost like it was a club like oh yeah

02:15:18.320 --> 02:15:22.557
we got access to mythos. We're spending
a ton of money. Like I think what should

02:15:22.640 --> 02:15:26.637
it look like this kind of roll out
to actually >> when you get help us

02:15:26.720 --> 02:15:30.317
>> in Glasswind I mean it they're fixing
real bugs they have found like 10,000

02:15:30.400 --> 02:15:34.317
bugs now one of the problems with
glasswing is they found like 10,000 bugs

02:15:34.400 --> 02:15:37.517
but they've only fixed a thousand right
so like you do not want a 10 to1 ratio

02:15:37.600 --> 02:15:41.357
of the bugs you found to fixed um
we just joined a thing called project

02:15:41.440 --> 02:15:45.517
Athena which chain guard is running
which is all about fixing specifically

02:15:45.600 --> 02:15:50.157
open source bugs right so glasswing is a
lot of that is closed source with inside

02:15:50.240 --> 02:15:54.797
commercial companies. We're part of this
coalition that is using uh Mythos to fix

02:15:54.880 --> 02:15:57.757
open source bugs, which is a huge
challenge because like just finding the

02:15:57.840 --> 02:16:01.677
people who have access to that
that source code uh commit access is

02:16:01.760 --> 02:16:04.637
difficult. Uh so that's going to be like

02:16:04.720 --> 02:16:06.717
a huge push. So it it's not like just

02:16:06.800 --> 02:16:10.157
totally BS. Um I do think that one of

02:16:10.240 --> 02:16:14.717
the things that's going on here is that if
these companies ran any modern AI

02:16:14.800 --> 02:16:17.597
security tool against their codebase,
they'd find 80% of those bugs. They

02:16:17.680 --> 02:16:19.197
don't have to use Mythos, right? Okay.

02:16:19.280 --> 02:16:23.277
So, um, it's just that they've never
looked, right? It's they're like

02:16:23.360 --> 02:16:26.477
>> that's concerning. >> Yeah.
Right. And and so they've they're

02:16:26.560 --> 02:16:29.997
like, "Oh, yeah. I've got this old
crappy code." And they look for the

02:16:30.080 --> 02:16:31.357
first time and they all
of a sudden they find

02:16:31.440 --> 02:16:35.117
>> and they could have done it a year ago
or 6 months ago and found some number or

02:16:35.200 --> 02:16:37.597
some percentage. >> I had
this conversation with a CEO this

02:16:37.680 --> 02:16:41.677
morning who's like, "How do I get into
Mythos, Alex, who can call?" And I said,

02:16:41.760 --> 02:16:45.597
"Don't wait. >> Here you go.
Here's an API key for Opus

02:16:45.680 --> 02:16:49.997
right now. it works today because do
not wait for the Trump administration to

02:16:50.080 --> 02:16:53.197
figure out what they're doing with Mythos.
Use Opus 48 right now and I

02:16:53.280 --> 02:16:55.517
guarantee you'll get 70%
of the bugs you care about.

02:16:55.600 --> 02:16:58.877
>> Alex, you're you're a security
researcher. One of the stipulations

02:16:58.960 --> 02:17:02.317
about getting Fable back. So, by the way,
Anthropic has said they expect uh

02:17:02.400 --> 02:17:06.077
Fable to come back on soon. Um, but one
of the stipulations that's been talked

02:17:06.160 --> 02:17:09.917
about to do that is that they have
to prevent it from being able to be

02:17:10.000 --> 02:17:12.717
jailbroken. >> Is that possible?

02:17:12.800 --> 02:17:16.397
>> Yeah. I saw I saw you did
jailbreak in air quotes. Yeah. No.

02:17:16.480 --> 02:17:19.277
>> So we have we have two minutes left.
So talk a little bit about why that's not

02:17:19.360 --> 02:17:21.197
possible. >> No, there's
actually a paper from NIST.

02:17:21.280 --> 02:17:25.037
It's like a kind of a goal completeness
theorem kind of uh paper that basically

02:17:25.120 --> 02:17:28.397
says it's impossible to make a model
that is completely not jailbreakable.

02:17:28.480 --> 02:17:32.317
Like I said, Fable needs to be able to
understand what security flaws are to be

02:17:32.400 --> 02:17:35.357
able to do its job. I think what what

02:17:35.440 --> 02:17:39.757
Enthropic might what what Enthropic
might the game might they might be

02:17:39.840 --> 02:17:42.237
playing here is redefine what jailbreak

02:17:42.320 --> 02:17:43.917
is back to what their system card says.

02:17:44.000 --> 02:17:46.877
So their system card says we define

02:17:46.960 --> 02:17:51.357
different levels of cyber capability
and finding a couple of bugs and knowing

02:17:51.440 --> 02:17:55.357
what a bug is is okay, but we won't let
you like build exploit chains and and do

02:17:55.440 --> 02:17:59.677
lots of long-term stuff. And so if they
can define back to their system card

02:17:59.760 --> 02:18:02.317
what a jailbreak is, then they're fine,

02:18:02.400 --> 02:18:07.437
right? Um, what would be terrible
for the entire American AI and cyber

02:18:07.520 --> 02:18:10.957
security industries is if the Trump
administration defines a jailbreak as

02:18:11.040 --> 02:18:15.197
having any ability to find any
vulnerability in code because then all

02:18:15.280 --> 02:18:17.837
of us have to go to Chinese models.

02:18:17.920 --> 02:18:22.557
>> That's that's it. Yes. Like that would
be a terrible terrible outcome. And that

02:18:22.640 --> 02:18:24.957
might happen because unfortunately one
the news that broke just in the last

02:18:25.040 --> 02:18:28.397
couple hours is Politico is reporting
that Anthropic and the Trump

02:18:28.480 --> 02:18:32.957
administration are negotiating what the AI
safety standards should be. So like I

02:18:33.040 --> 02:18:36.397
don't know it's like Howard Lutnik
writing an eval right now. Like I find

02:18:36.480 --> 02:18:41.837
that a little terrifying. So um I I am
hoping that Enthropic obviously there's

02:18:41.920 --> 02:18:44.477
very smart people in Enthropic working
on this. I am hoping that they are able

02:18:44.560 --> 02:18:48.157
to come up with something reasonable
that moves stuff towards exploitation

02:18:48.240 --> 02:18:52.637
not bug finding like we have to have
the ability as defenders to find and fix

02:18:52.720 --> 02:18:55.517
flaws. The other thing I just want
because we don't have a lot of time. The

02:18:55.600 --> 02:18:57.837
other real own goal here was that this

02:18:57.920 --> 02:19:00.957
injected a type of political instability

02:19:01.040 --> 02:19:05.437
and political risk into the USI industry
that did not exist. There are a bunch of

02:19:05.520 --> 02:19:09.917
companies that are now going to be have
to use openweight models because at any

02:19:10.000 --> 02:19:12.797
time on a Friday afternoon, it's it's
good thing that Fable was only out for a

02:19:12.880 --> 02:19:16.317
week because people weren't didn't have
time to work into their critical path.

02:19:16.400 --> 02:19:20.637
But if if if Fable had been out for a
month and at 5:00 PM on a Friday all of

02:19:20.720 --> 02:19:24.157
a sudden it got yanked, then pagers
across the country would have gone off

02:19:24.240 --> 02:19:27.837
because every everybody's systems would
have fallen over. And so now this week

02:19:27.920 --> 02:19:33.277
CIOS and CTO's are signing contracts
to have openweight models on different

02:19:33.360 --> 02:19:36.237
hosts on US hosts. They're not using
Chinese hosts, but they're using Chinese

02:19:36.320 --> 02:19:39.437
models on as backups through LM routers

02:19:39.520 --> 02:19:42.237
because political risk is now a risk of

02:19:42.320 --> 02:19:46.637
using AI companies. That is a massive
own goal. The United States of America

02:19:46.720 --> 02:19:51.117
does not need to do that. We should not
do that again. That is incredibly stupid

02:19:51.200 --> 02:19:52.957
at a moment of maximum pressure from the

02:19:53.040 --> 02:19:55.437
People's Republic of China. Um, and so I

02:19:55.520 --> 02:19:58.237
can't stress enough like we we lost a

02:19:58.320 --> 02:20:00.557
war this weekend. Like, let's not lose

02:20:00.640 --> 02:20:06.157
another war by making it unreliable
to utilize the United States for AI.

02:20:06.240 --> 02:20:08.397
>> Alex, will you come on the show
for a longer conversation about this?

02:20:08.480 --> 02:20:09.437
>> Absolutely. I'd love to. Yeah.

02:20:09.520 --> 02:20:10.877
>> Alex Tamos and Ron General.

02:20:10.960 --> 02:20:14.680
>> Thank you. >> Thank you.

02:20:17.680 --> 02:20:21.117
Okay, so Anthropic is not just a massive

02:20:21.200 --> 02:20:24.797
model builder, it's a massive product
builder as well with products like Cloud

02:20:24.880 --> 02:20:27.277
Code and Co-work that have taken off

02:20:27.360 --> 02:20:30.557
like crazy over the past few months. And

02:20:30.640 --> 02:20:32.797
Cloud Code came out of a place that few

02:20:32.880 --> 02:20:35.437
of us know much about called Anthropic

02:20:35.520 --> 02:20:38.957
Labs. And Anthropic Labs is a

02:20:39.040 --> 02:20:41.037
organization within Anthropic that is

02:20:41.120 --> 02:20:43.357
working on building the next level of

02:20:43.440 --> 02:20:46.317
frontier products with AI at the center.

02:20:46.400 --> 02:20:48.717
And so today we are lucky to hear from

02:20:48.800 --> 02:20:51.517
the person running that lab, Mike Kger,

02:20:51.600 --> 02:20:53.757
who is the co-founder of Instagram and

02:20:53.840 --> 02:20:57.517
now the lead of um of Anthropic Labs at

02:20:57.600 --> 02:21:01.757
Anthropic. We're going to welcome him
on stage along with Lauren Good of Wired,

02:21:01.840 --> 02:21:05.037
who will join me as a co-in. Mike
and Lauren, let's uh let's let's hear from

02:21:05.120 --> 02:21:08.520
both of you guys.

02:21:10.000 --> 02:21:14.797
You're here. >> All right.
>> All right, Mike. So, chill times in

02:21:14.880 --> 02:21:17.117
anthropic land. >> Nothing going on.

02:21:17.200 --> 02:21:20.237
>> Slow week. >> Slow week.

02:21:20.320 --> 02:21:24.237
>> You want to start, Lauren? >> Yeah.
Uh first of all I mean we want to

02:21:24.320 --> 02:21:28.477
talk about what you're working on at
labs uh and explain your role to folks

02:21:28.560 --> 02:21:32.557
but I want to ask you first how close
are you right now to the situation with

02:21:32.640 --> 02:21:35.117
the white house >> less
than in my CPO role. So I

02:21:35.200 --> 02:21:39.677
transitioned about like five months ago
into this labs role I think in the CPO

02:21:39.760 --> 02:21:41.197
role I think would have been deep in it.

02:21:41.280 --> 02:21:46.397
Now I'm more you know obviously you know
we want to restore it and and as a like

02:21:46.480 --> 02:21:49.837
uh product person I want to make sure
that that gets access but less close to

02:21:49.920 --> 02:21:53.997
it than uh you know in that sort
of seale role that I had before.

02:21:54.080 --> 02:21:56.877
>> Okay, Alex, do you have a follow-up?
I have like eight follow-ups.

02:21:56.960 --> 02:22:00.797
>> Definitely. Well, we there was a guy
named Ben on X who said, "I will mail

02:22:00.880 --> 02:22:03.997
anthropic and original copy of my long
form birth certificate if they will

02:22:04.080 --> 02:22:08.397
enable fable for me again." I sound like
those lunatics who were obsessed with 40

02:22:08.480 --> 02:22:12.477
now. Will you take Ben's long form?
I don't know uh that will take as long

02:22:12.560 --> 02:22:16.477
form, but it has been interesting.
I mean, Fable was only available um for a

02:22:16.560 --> 02:22:20.637
few days, but I've definitely uh every
time I've tweeted since then, uh they've

02:22:20.720 --> 02:22:23.437
not read whatever I was tweeting
and they've mostly been like bring back

02:22:23.520 --> 02:22:26.717
Fable, which like in Instagram, we got
rid of Gotham. Do you remember Gotham?

02:22:26.800 --> 02:22:29.517
The filter. This is like and then for
the rest like the next eight years, all

02:22:29.600 --> 02:22:34.157
I heard was bring back Gotham. So,
it struck a nerf, but Fable will uh will

02:22:34.240 --> 02:22:37.837
come back before Gotham did. But yeah,
it's it's it clearly the folks that had

02:22:37.920 --> 02:22:40.237
gotten to use it and started
incorporating it. It's actually really

02:22:40.320 --> 02:22:45.277
interesting. Um, I've learned to not
really trust day of or even week of

02:22:45.360 --> 02:22:48.477
model reactions. Like you don't really
know until you've put it through its

02:22:48.560 --> 02:22:52.397
paces. And so like I almost just
completely block out the noise in the

02:22:52.480 --> 02:22:55.517
first couple of days of any new model
release cuz I don't know, everybody has

02:22:55.600 --> 02:22:59.037
maybe their like toy like example thing
that they like to do with a new model.

02:22:59.120 --> 02:23:02.397
But it's hard to actually put it through
its paces until you've actually had real

02:23:02.480 --> 02:23:05.757
work done with it. And I think people
were just starting to do that and then,

02:23:05.840 --> 02:23:08.637
you know, we had to uh uh sort of pull

02:23:08.720 --> 02:23:14.157
back Fable. But um I remember
in December when we put out Opus 46, uh it

02:23:14.240 --> 02:23:16.877
was like this interesting time where
everybody went home for the holidays and

02:23:16.960 --> 02:23:20.077
a lot of people had that week off
between Christmas and New Year's and

02:23:20.160 --> 02:23:23.837
then they came back and were like, "Oh,
I spent a lot of time and I really get

02:23:23.920 --> 02:23:25.517
why Opus is good and I want to do it."

02:23:25.600 --> 02:23:28.877
So I don't think Fable has had
that opportunity yet. But despite that

02:23:28.960 --> 02:23:32.317
though, I mean, this was a pretty big
reaction from the Trump administration.

02:23:32.400 --> 02:23:35.917
And I think everyone here, especially
if you were listening to the Alex Stamosh

02:23:36.000 --> 02:23:39.997
session, uh, understands what's going on.
But this happened within a few days

02:23:40.080 --> 02:23:43.917
of the model release. Um, if I can give
Wired some credit, Wired just reported

02:23:44.000 --> 02:23:47.917
last night that it was due
to Anthropic's uh, relationship with or

02:23:48.000 --> 02:23:51.837
having given uh, access to the model
to SK Telecom that could have raised flags

02:23:51.920 --> 02:23:57.437
within the administration. How surprised
were you by how immediate that that

02:23:57.520 --> 02:23:59.997
backlash was? >> Yeah,
I think the the the sort of

02:24:00.080 --> 02:24:04.317
reaction decision was surprising and we
were sort of immediately engaged with

02:24:04.400 --> 02:24:09.597
them to, you know, restore access as well.
Um, and so at the same time, you

02:24:09.680 --> 02:24:13.037
know, the one thing that we we think
a lot about internally is, you know, there

02:24:13.120 --> 02:24:16.797
used to be a poster on the Facebook
wall when we were that was like every day

02:24:16.880 --> 02:24:20.157
feels like a week. And I think that's
becoming true in AI. And I think a good

02:24:20.240 --> 02:24:24.557
thing to remind ourselves of in general
in the industry is uh we're dealing with

02:24:24.640 --> 02:24:28.717
unprecedented times. We're dealing with
new situations and uh they can develop

02:24:28.800 --> 02:24:32.877
really quickly as well. And so I think
uh also developing the capabilities and

02:24:32.960 --> 02:24:35.757
the connections and and to make sure
those conversations can happen quickly

02:24:35.840 --> 02:24:38.797
is really really important.
>> I have a new motto suggestion for you

02:24:38.880 --> 02:24:42.637
for the wall. Move fast.
Move fast and jailbreak things.

02:24:42.720 --> 02:24:45.357
>> Jailbreak things. I don't think
they're going to use that, Lauren.

02:24:45.440 --> 02:24:48.957
>> Okay. Um, you know,
we just heard from Alex

02:24:49.040 --> 02:24:53.277
that some of these capabilities have
been uh available on, you know, previous

02:24:53.360 --> 02:24:57.197
models or, you know, you can you can
find bugs with other with other models

02:24:57.280 --> 02:25:00.637
that are available today. So, why do you
think anthropic got singled out on this

02:25:00.720 --> 02:25:03.357
front? >> Um, I don't know why like Fable

02:25:03.440 --> 02:25:05.677
specifically was singled out um as you

02:25:05.760 --> 02:25:08.237
know, again, Fable being the non-cyber

02:25:08.320 --> 02:25:11.277
uh intention uh model as well. Um I

02:25:11.360 --> 02:25:15.757
think that the thing that does change
over time is there's uh you know the

02:25:15.840 --> 02:25:18.877
capabilities if you knew what to prompt
and knew what you were looking for and

02:25:18.960 --> 02:25:22.477
then there's the capabilities I think
you know uh I'm not sure I resonate with

02:25:22.560 --> 02:25:26.317
the like juicing high school athletes
metaphor from Alex but like you know the

02:25:26.400 --> 02:25:29.277
uplift that you get like uplift is a
thing that we think about a lot when we

02:25:29.360 --> 02:25:32.957
think about model um safety. So if you
look at our model cards for example, one

02:25:33.040 --> 02:25:37.677
of the ways and we look at uh risk
in the bio domain is comparing the uplift

02:25:37.760 --> 02:25:41.917
uh from sort of lay a lay person using
the model versus you know an expert or

02:25:42.000 --> 02:25:45.197
lay person just using the internet and
and seeing what the comparison there is.

02:25:45.280 --> 02:25:47.677
And so that is one trend that is you

02:25:47.760 --> 02:25:49.997
know has been you know progressing as

02:25:50.080 --> 02:25:55.037
the models get more capable. Um and so
maybe less why Fable is singled out and

02:25:55.120 --> 02:25:57.277
maybe more like what
the overall trajectory is.

02:25:57.360 --> 02:26:01.517
>> Interesting. So, we just had Ara
Karazzian, the uh lead economist on RAMP

02:26:01.600 --> 02:26:04.877
on the podcast and one of the interesting
things that he said was, you

02:26:04.960 --> 02:26:08.397
know, we we had during the Pentagon
situation, there were these headlines,

02:26:08.480 --> 02:26:11.437
okay, this company will not use
anthropic anymore, but actually the data

02:26:11.520 --> 02:26:16.157
from RAMP shows that spending actually
increased uh to anthropic models. It was

02:26:16.240 --> 02:26:20.477
apparently a good publicity moment
for the company. So I I think it sort of

02:26:20.560 --> 02:26:24.077
plays into a debate that we have like
here on the show about like how much of

02:26:24.160 --> 02:26:28.317
this is like real concern for the issues
and how much of it is you know from

02:26:28.400 --> 02:26:32.557
Anthropic is marketing and like we have
somebody from Anthropic here who can

02:26:32.640 --> 02:26:35.197
actually shed some light on that.
We just said Stamos talking a little bit

02:26:35.280 --> 02:26:38.237
about it from his perspective on the
security side but we're lucky to have

02:26:38.320 --> 02:26:42.877
you here today. So so is it material
or is it marketing or some combination? I

02:26:42.960 --> 02:26:46.397
mean I think uh it is one of the hardest
things to like really deeply believe

02:26:46.480 --> 02:26:48.797
that something is true and not marketing

02:26:48.880 --> 02:26:51.437
and then due to not even just anthropic.

02:26:51.520 --> 02:26:55.437
I think like you know uh uh I think

02:26:55.520 --> 02:26:58.957
people are generally right to be
skeptical of any company saying anything

02:26:59.040 --> 02:27:01.837
and you should like put it through
your own filters as well. Um, but for me

02:27:01.920 --> 02:27:05.197
personally, I'm always like, "No, but it's
real." And like we like both deeply

02:27:05.280 --> 02:27:09.117
care about safety and are like to the
extent that we are being vocal about

02:27:09.200 --> 02:27:12.877
anything, it is to either sort of help
paint the picture of what very likely is

02:27:12.960 --> 02:27:16.797
coming or we believe is coming or what
we've already seen and spotted. Um, you

02:27:16.880 --> 02:27:21.197
know, for example, in the in the Mythos
case, really just looking at uh uh like

02:27:21.280 --> 02:27:24.237
vulnerability scanning and bug finding
and doing it in partnership with with

02:27:24.320 --> 02:27:25.997
companies that were in that kind of

02:27:26.080 --> 02:27:27.597
project glasswing initial announcement.

02:27:27.680 --> 02:27:32.317
And so um the technology is awesome not

02:27:32.400 --> 02:27:37.037
in the like in the just you know it is it
is doing really incredible things and

02:27:37.120 --> 02:27:40.957
therefore even calling out what we see
as what is happening I think can seem

02:27:41.040 --> 02:27:44.637
hypy. Uh I wish I could press a button
and was and like I could make everybody

02:27:44.720 --> 02:27:47.357
believe that we are not being hypy.
I realize that's not the reality that we

02:27:47.440 --> 02:27:48.957
operate in but at least from my

02:27:49.040 --> 02:27:51.597
perspective um we try to call it like it

02:27:51.680 --> 02:27:56.557
is. My understanding is that within
labs in particular in research and

02:27:56.640 --> 02:28:01.277
development at anthropic that you're
using the best AI models to actually

02:28:01.360 --> 02:28:04.317
build new products to prototype new
products and sort of test out your

02:28:04.400 --> 02:28:07.837
thesis. So is this uh is this li is this

02:28:07.920 --> 02:28:10.157
ban essentially now limiting your

02:28:10.240 --> 02:28:12.477
ability to do that within labs? Yeah, I

02:28:12.560 --> 02:28:17.757
mean definitely Fable uh is the best
model I've ever used and uh it's not to

02:28:17.840 --> 02:28:20.957
say that like work has stopped but it's
definitely like less good than the other

02:28:21.040 --> 02:28:23.757
models that uh that or sorry we using

02:28:23.840 --> 02:28:26.077
models that are less good than that um

02:28:26.160 --> 02:28:28.557
as well and it is also I mean maybe the

02:28:28.640 --> 02:28:33.357
reverse of the uh distrusting the first
week uh you know response is what

02:28:33.440 --> 02:28:36.557
happens when you don't have fable and
like obviously the Twitter reaction for

02:28:36.640 --> 02:28:39.437
the people that had already kind of like
gotten uh into the model and and were

02:28:39.520 --> 02:28:43.277
using it was strong. Um, but I'd say
even in my personal use, I'm like, "Oh,

02:28:43.360 --> 02:28:46.557
like I'm on Opus 48 and it's good. Like
I'm still productive. I'm doing work."

02:28:46.640 --> 02:28:50.397
But um, and we can go into sort of like
how my work changed with sort of these

02:28:50.480 --> 02:28:55.117
like Fable or like you know uh the models
of that sort of uh family, but it

02:28:55.200 --> 02:28:57.917
is noticeable for sure. >> Yeah,
I think we'd like to know that. I

02:28:58.000 --> 02:28:59.917
mean, you know, we the public had access

02:29:00.000 --> 02:29:03.197
to Fable for like a half a minute. Um,

02:29:03.280 --> 02:29:07.917
groups in this like Project Glass Wing
have had access to Mythos. um we don't

02:29:08.000 --> 02:29:11.437
really know what the difference
is between using a model that you can use

02:29:11.520 --> 02:29:14.397
today is and using one of these you know

02:29:14.480 --> 02:29:19.357
superanthropic models. Yeah. So what
actually could you do differently with a

02:29:19.440 --> 02:29:20.477
fable or a mythos?

02:29:20.560 --> 02:29:22.717
>> I think for me it's the sort of scope

02:29:22.800 --> 02:29:27.837
and scale of delegation. Um and all
these things are really imperfect. Like

02:29:27.920 --> 02:29:31.277
you know people would say like oh
is this now a level five software engineer

02:29:31.360 --> 02:29:35.037
or level six but anybody who's used
these models extensively knows that

02:29:35.120 --> 02:29:38.317
they're still spiky in capabilities
right in some ways they're you know in

02:29:38.400 --> 02:29:43.117
many ways they're a better engineer than
me. Um and in other ways like you know I

02:29:43.200 --> 02:29:46.957
was complaining today that it had missed
a descender like a the G that bottom

02:29:47.040 --> 02:29:49.437
part is called the descender. I'm like,
how did you put that in the UI and it's

02:29:49.520 --> 02:29:52.237
clipped? And of course, like there's
vision capabilities that need to improve

02:29:52.320 --> 02:29:56.477
and there's uh debugging capabilities
and there's sometimes even just sort of

02:29:56.560 --> 02:30:00.157
human common sense that is, you know,
we're way better at than when than the

02:30:00.240 --> 02:30:02.797
models are. But overall, I think the big

02:30:02.880 --> 02:30:07.437
shift for me working um and it was really
interesting because uh it sort of

02:30:07.520 --> 02:30:10.317
coincided with me going back into
a builder role. So I really got to see

02:30:10.400 --> 02:30:14.477
going from you know using these models
as an executive you're trying to do the

02:30:14.560 --> 02:30:18.957
most of them but you know like not going
to have it right all your email and uh

02:30:19.040 --> 02:30:22.797
you know I think strategy still needs
to come from you and then you can use the

02:30:22.880 --> 02:30:25.757
models to sort of pressure test it. Um

02:30:25.840 --> 02:30:29.917
but going back into a builder role
and going from okay I am delegating chunks

02:30:30.000 --> 02:30:33.037
like please fix this bug or I'm thinking
of implementing this feature let's go

02:30:33.120 --> 02:30:36.637
back and forth um to something that ends
up being much more sort of all right

02:30:36.720 --> 02:30:40.797
like I got this bug report from one
of our users or I have this notion of

02:30:40.880 --> 02:30:43.917
something that I want to build like
can you sketch out two or three ways in

02:30:44.000 --> 02:30:48.317
which we could do it right like that
seems plausible often I find actually

02:30:48.400 --> 02:30:52.157
sometimes it'll uh give me the the sort
of explanation or proposal and be like

02:30:52.240 --> 02:30:55.517
okay that actually is over my head like
you are clearly way smarter than me like

02:30:55.600 --> 02:30:59.917
explain it to me like I'm maybe not five
but at least you know uh uh not you and

02:31:00.000 --> 02:31:03.837
it'll sometimes you know explain it
that way but then go build it and get it

02:31:03.920 --> 02:31:08.397
right getting it right like you know
at a very very very high rate and I think

02:31:08.480 --> 02:31:12.317
that starts really changing how you
operate like I I moved much more to

02:31:12.400 --> 02:31:16.477
before going to bed making sure that I
had like queued up for fable like enough

02:31:16.560 --> 02:31:19.117
chunky work to last I would call it
the whole night and then I would check in

02:31:19.200 --> 02:31:21.757
later and it got it done in an hour and
it was like I guess hanging out for the

02:31:21.840 --> 02:31:25.997
next seven hours but like really like
delegating like uh uh much more of a

02:31:26.080 --> 02:31:28.957
goal than just a >> so
one one task for example.

02:31:29.040 --> 02:31:31.917
>> Yeah. >> Like give us an example
of one task you would hand to it.

02:31:32.000 --> 02:31:35.677
>> Uh I mean here's a kind of crazy one
which is uh for the programmers in the

02:31:35.760 --> 02:31:40.797
audience like I uh I had written one
of our labs uh projects in um in Python.

02:31:40.880 --> 02:31:43.837
That's like the language I know all
Instagram was all Python. And for like

02:31:43.920 --> 02:31:47.197
some not super exciting reasons,
we actually needed it to be in Typescript

02:31:47.280 --> 02:31:50.237
to deploy it. And I was like,
"All right, that's going to be like, you

02:31:50.320 --> 02:31:55.117
know, in Instagram, we for years talked
about moving from Python to PHP or hack

02:31:55.200 --> 02:31:58.397
or the the Facebook language after
the acquisition." And at least when I was

02:31:58.480 --> 02:32:02.717
there, never did. Um, but I basically we
have a feature called dynamic workflows

02:32:02.800 --> 02:32:05.197
where you can have it like also
break down the task into like a lot of

02:32:05.280 --> 02:32:09.917
subtasks. And I trusted it to sort
of not just do the individual action, but

02:32:10.000 --> 02:32:13.597
here's like a whole language conversion
of millions of hundreds of thousands of

02:32:13.680 --> 02:32:17.437
lines of code at that point. Uh, go off
and do it. Go plan it. Go execute it. go

02:32:17.520 --> 02:32:21.037
verify the work, double verify the work,
and then I came back to the work being

02:32:21.120 --> 02:32:22.877
complete. So that level of like this is

02:32:22.960 --> 02:32:26.317
a big sort of chunky uh task.

02:32:26.400 --> 02:32:28.077
>> So that was so you're basically saying

02:32:28.160 --> 02:32:33.597
it was faster. Did it in an hour. You're
you're s guessing compared to with fable

02:32:33.680 --> 02:32:36.317
compared to what it would have been
before. I think the main difference is

02:32:36.400 --> 02:32:39.517
in the past you would it would be like
great I did it and you'd be like but did

02:32:39.600 --> 02:32:43.117
you you kind of took a shortcut here
or this is not quite right or I need to go

02:32:43.200 --> 02:32:46.797
verify it or like oh you cut this corner
or >> it's like the managing interns thing

02:32:46.880 --> 02:32:49.757
that everyone's been saying for the past
year which is very offensive to interns

02:32:49.840 --> 02:32:52.717
by the way but yes >> yeah exactly >>
I don't know have you managed an intern?

02:32:52.800 --> 02:32:54.877
Yeah, that's true. And it was and you're

02:32:54.960 --> 02:32:57.037
saying it was more correct. It was so

02:32:57.120 --> 02:33:00.157
it's faster. It's more accurate. Uh more

02:33:00.240 --> 02:33:04.477
reliable and then according to the US
administration dangerous. I think the

02:33:04.560 --> 02:33:08.237
other piece is it has like a a greater
theory of mind is the wrong word but

02:33:08.320 --> 02:33:12.797
sort of like theory of project so that
it's less you know oh I'm going to make

02:33:12.880 --> 02:33:15.757
this change and it'll say great I'll
make this change but really especially

02:33:15.840 --> 02:33:19.757
if you've done like software engineering
at scale the best engineers kind of keep

02:33:19.840 --> 02:33:22.957
in mind all the disperate parts of how
this things and they also see around the

02:33:23.040 --> 02:33:26.877
corners like I can make this change
but if I don't do it in this way then the

02:33:26.960 --> 02:33:30.397
next change is going to be incrementally
harder and I think that's like been a

02:33:30.480 --> 02:33:34.237
significant difference I've seen in
that kind of class of models Okay, cool.

02:33:34.320 --> 02:33:36.237
>> So, I think when
we talk about Anthropic

02:33:36.320 --> 02:33:39.357
Labs, right, people think of Claude Code

02:33:39.440 --> 02:33:41.597
because it is really your breakout

02:33:41.680 --> 02:33:45.517
product and uh and it sounds like you've

02:33:45.600 --> 02:33:48.957
been tasked with basically figuring out
what the next Claude code is. Would you

02:33:49.040 --> 02:33:51.677
say that's an accurate description
of what you're doing at Labs and also why

02:33:51.760 --> 02:33:55.917
does Anthropic need labs? >> Labs.
Yeah, it's also maybe worth

02:33:56.000 --> 02:33:59.597
thinking about why we needed labs in 2024
when I arrived and why we need labs

02:33:59.680 --> 02:34:03.357
today because I think that the answer
kind of shifts. Uh I started the labs

02:34:03.440 --> 02:34:05.757
the original labs team with Ben
man who's one of the co-founders of

02:34:05.840 --> 02:34:09.757
Anthropic. Um in my third week at
Enthropic and it had been something that

02:34:09.840 --> 02:34:13.357
I've been bubbling under and at the time
the reason was really different. It was

02:34:13.440 --> 02:34:16.957
all of our product engineering with team
was 25 people and we didn't have the

02:34:17.040 --> 02:34:20.797
models really like we had claude that
when I joined it was set three like you

02:34:20.880 --> 02:34:24.237
and Opus 3 like those were for the time
good models but you weren't going to

02:34:24.320 --> 02:34:25.997
they were not even interns right they

02:34:26.080 --> 02:34:29.997
weren't even IC3 engineers. Um so if you

02:34:30.080 --> 02:34:34.397
have a team of only 25 or 30 engineers
they are working on like the next you

02:34:34.480 --> 02:34:37.917
know incremental thing and we were
feeling like the models are starting to

02:34:38.000 --> 02:34:41.037
get better but we don't have any
products that sort of show that off like

02:34:41.120 --> 02:34:44.637
a good litmus test for me is when we
get ready to release a model do we have

02:34:44.720 --> 02:34:48.637
either a product or a demo or or you
know some other illustration of

02:34:48.720 --> 02:34:51.677
something that is very different and it
gets harder over time like with with

02:34:51.760 --> 02:34:53.757
Fable you know even like illustrating

02:34:53.840 --> 02:34:56.637
like that weekend task or you know this

02:34:56.720 --> 02:35:01.037
longer amount of work. So really labs
at the time was let's make sure we don't

02:35:01.120 --> 02:35:05.037
like our products don't fall behind
the model exponential that's happening. Um

02:35:05.120 --> 02:35:08.397
and yeah so cloud code came out of that
initial one because nobody in the rest

02:35:08.480 --> 02:35:12.077
of the product or people were thinking
about coding but nobody was sort of had

02:35:12.160 --> 02:35:15.757
the like space to go and think about
well what if we totally change the form

02:35:15.840 --> 02:35:18.957
factor and we embrace the fact that the
models were going to uh evolve in this

02:35:19.040 --> 02:35:23.357
way and a lot of the like the two most
useful thought exercises we do in labs.

02:35:23.440 --> 02:35:27.357
One is like visualize the gap between
what the models can do today and how

02:35:27.440 --> 02:35:31.037
most people use it and can we close that
gap. So that's one. And the other one is

02:35:31.120 --> 02:35:33.677
imagine what the models are bad at now
that they're actually going to be really

02:35:33.760 --> 02:35:36.637
good at in six months and let's make
sure we have a product ready for that by

02:35:36.720 --> 02:35:41.197
then. And I think those are like the two
guiding uh questions for for labs. Um

02:35:41.280 --> 02:35:44.717
and then also out of that first
incarnation came computer use. Computer

02:35:44.800 --> 02:35:47.597
use was different though because when
we built it it was really bad. Like we

02:35:47.680 --> 02:35:51.597
tried a bunch of products with it
and this was around you know silent 35 and

02:35:51.680 --> 02:35:55.357
be like Claude can you help me you know
clean up my desktop and it would like

02:35:55.440 --> 02:35:58.877
click the thing and it delete the file
like this is not not safe for release

02:35:58.960 --> 02:36:02.717
we're definitely not going to go and
and and build this or to ship this but we

02:36:02.800 --> 02:36:06.717
had that product so that every new model
that we we'd release we'd first check it

02:36:06.800 --> 02:36:10.157
internally and say did computer use get
better and we'd tell the research team

02:36:10.240 --> 02:36:13.357
how it gotten better or worse until
the moment where we said it's good enough

02:36:13.440 --> 02:36:16.397
we're actually going to put a product
out around this. So it's it also gives

02:36:16.480 --> 02:36:19.677
you this sort of uh sort of beacon into
the future that then you can kind like

02:36:19.760 --> 02:36:23.837
measure your your future products against.
Um but then compare it to now.

02:36:23.920 --> 02:36:25.517
So we have a driving product team.

02:36:25.600 --> 02:36:30.237
There's co-work there's um you know
cloud code has grown a lot. We have our

02:36:30.320 --> 02:36:34.637
platform. Um and now I think it's
actually much much less about none of

02:36:34.720 --> 02:36:37.757
these product teams are doing this sort
of thinking and I think it's much more

02:36:37.840 --> 02:36:42.157
that the models are advancing really
quickly and even our capability to

02:36:42.240 --> 02:36:46.237
interact with them needs to evolve.
So one of the things we collaborated with

02:36:46.320 --> 02:36:49.837
uh labs and cloud code that we ship
today um is cloud code artifacts. So

02:36:49.920 --> 02:36:53.037
having cloud code not just be able to type
back to you but also sort of draw a

02:36:53.120 --> 02:36:56.477
picture or give you an illustration. And
that partially came from spending a lot

02:36:56.560 --> 02:37:00.717
of time in lab saying just a text box
and a big text response is not going to

02:37:00.800 --> 02:37:03.197
cut it anymore. Like when I mentioned
that the models feel like they're way

02:37:03.280 --> 02:37:05.917
smarter than me when they talk to me
sometimes I'm like can you draw me a

02:37:06.000 --> 02:37:09.837
picture because this is what I actually
need to fully uh fully understand this.

02:37:09.920 --> 02:37:13.357
But it's really what we've been thinking
about is, you know, yes, we have a lot

02:37:13.440 --> 02:37:16.637
more products. You know, we have we
actually have a lot of consolidation to

02:37:16.720 --> 02:37:19.517
do in our products. Like that's another
initiative that we have. But within

02:37:19.600 --> 02:37:23.597
that, we still have an opportunity
to make things much more accessible to a

02:37:23.680 --> 02:37:27.197
person that does not spend all of their
time thinking about prompting and the

02:37:27.280 --> 02:37:29.997
exponential and the difference between
high, low, and medium effort. Like

02:37:30.080 --> 02:37:34.157
there's a lot we can do.
>> But Mike, so there's a it's it puts

02:37:34.240 --> 02:37:38.477
people using anthropic models in an
interesting place, right? Um there you

02:37:38.560 --> 02:37:43.037
know cursor I think just sold for 60
billion to SpaceX and someone put this

02:37:43.120 --> 02:37:46.957
meme on Twitter that like you know
cursor would have sold for 300 billion

02:37:47.040 --> 02:37:51.037
if it wasn't for this guy and it's
a picture of Boris Churnney the person who

02:37:51.120 --> 02:37:53.677
created Claude Code. Um, and so for

02:37:53.760 --> 02:37:58.077
companies that are going to build
on top of Anthropic technology, you know,

02:37:58.160 --> 02:38:02.637
they're going to wonder, do I want
to partner with Anthropic or is Anthropic

02:38:02.720 --> 02:38:05.677
going to go ahead and and build the
product that I'm going to want to build

02:38:05.760 --> 02:38:07.517
potentially even after
partnering with them?

02:38:07.600 --> 02:38:11.037
>> Yeah. And I mean, we'll take the like
agentic coding side and I think the

02:38:11.120 --> 02:38:15.037
broader sort of aspect of of you know,
being both a platform and a product I

02:38:15.120 --> 02:38:17.757
think is is really interesting. um when

02:38:17.840 --> 02:38:20.557
we take on projects it's the goal is

02:38:20.640 --> 02:38:23.357
often to sort of push that area of the

02:38:23.440 --> 02:38:28.157
industry forward. So um you know there
were AI coding editors and some of them

02:38:28.240 --> 02:38:31.837
were really good and you know uh but
nobody was quite thinking about it in as

02:38:31.920 --> 02:38:35.037
sort of free form a way as we got
to think about it with cloud code. And now

02:38:35.120 --> 02:38:39.277
a lot more products have that flavor
than I think would have otherwise. And

02:38:39.360 --> 02:38:42.557
so I think if we're ever you can call
me out on this Alex like if we're ever

02:38:42.640 --> 02:38:44.877
entering an industry where like all
you're doing is the same thing everybody

02:38:44.960 --> 02:38:48.317
else is doing but like you've got the
anthropic brand I feel like that's a bad

02:38:48.400 --> 02:38:51.837
use of our time and a bad use of our
either labs or product team time. Like

02:38:51.920 --> 02:38:55.117
if we're going in somewhere it should
hopefully be to say all right we think

02:38:55.200 --> 02:38:58.637
that the direction of travel is this way
we can build a product of that and then

02:38:58.720 --> 02:39:01.997
by the way there's no world nor should
there be a world where like all the

02:39:02.080 --> 02:39:04.717
products are entropic products that would
be a bad world right so like that

02:39:04.800 --> 02:39:08.717
is hopefully either creating new space
for for companies or sort of showing the

02:39:08.800 --> 02:39:10.797
way where other products
can incorporate that

02:39:10.880 --> 02:39:13.917
>> it would almost be like working
for a tech company that has like social

02:39:14.000 --> 02:39:17.597
messaging video

02:39:17.680 --> 02:39:22.877
>> right Mike yeah okay you did leave >>
well there was question for example when

02:39:22.960 --> 02:39:26.397
you know Anthropic launched a product
that was seen as competitive to Figma

02:39:26.480 --> 02:39:29.997
and you had been on the Figma board
prior to that and I think you stepped

02:39:30.080 --> 02:39:33.597
down is that correct? Yeah. And so um
it's a good question that it's al Alex

02:39:33.680 --> 02:39:36.957
has brought up I think where Silicon
Valley is known for this really healthy

02:39:37.040 --> 02:39:41.197
vibrant risk tolerant startup ecosystem
and when the bigs start coming in with

02:39:41.280 --> 02:39:45.357
tons of venture capital and you know
a lot of resources uh people say well wait

02:39:45.440 --> 02:39:48.317
are they just are they essentially
just going to steal my idea?

02:39:48.400 --> 02:39:52.557
>> Yeah. No, I think I think our dual
existence and it's something that other

02:39:52.640 --> 02:39:55.277
companies have had to navigate.
We talked all talked about Amazon a lot in

02:39:55.360 --> 02:39:58.477
the in the previous panel like they
have to navigate this role where they are

02:39:58.560 --> 02:40:01.917
both infrastructure provider. They
obviously have a very large e-commerce.

02:40:02.000 --> 02:40:06.317
They they do video but they also serve
video and um and then you know by and

02:40:06.400 --> 02:40:09.597
large customers can live in that dual
world of like okay I'm using their

02:40:09.680 --> 02:40:12.637
infrastructure also knowing that they
they are also using their infrastructure

02:40:12.720 --> 02:40:17.197
to do that and I think the uh you can
talk to our customers and see how well

02:40:17.280 --> 02:40:19.197
we're doing at this like the thing
I always try to do is like at least

02:40:19.280 --> 02:40:20.557
approach it with a lot of transparency.

02:40:20.640 --> 02:40:23.037
So the cursor example is an interesting
one where like Michael and I talked a

02:40:23.120 --> 02:40:28.317
lot over the you know time around here's
where things are heading. Um and uh you

02:40:28.400 --> 02:40:31.277
know with similarly with with the other
products that we think about like can we

02:40:31.360 --> 02:40:33.917
I think it's a couple things.
It's transparency and then it's shared

02:40:34.000 --> 02:40:37.837
building blocks like yeah I think um
in general and I actually don't think

02:40:37.920 --> 02:40:41.037
there's any cases where this is even
true like we're trying to build on top

02:40:41.120 --> 02:40:44.877
of the same capabilities that are
available elsewhere. The last time I was

02:40:44.960 --> 02:40:47.677
here in the Commonwealth Club on this
stage was our healthcare day at the

02:40:47.760 --> 02:40:52.397
beginning of the year and we didn't ship
like cloud healthcare only we have it

02:40:52.480 --> 02:40:55.677
like nobody else has it. We shipped
a bunch of like plugins and skills and

02:40:55.760 --> 02:40:57.677
MCPs and like complimentary abilities.

02:40:57.760 --> 02:41:01.597
So that's how I'm not claiming it's easy
or that it's a straightforward thing,

02:41:01.680 --> 02:41:05.117
but it is how we're trying to navigate
what is like admittedly a complicated

02:41:05.200 --> 02:41:08.317
sort of situation. >> Speaking
of startups, Anthropic is still

02:41:08.400 --> 02:41:13.277
technically a startup, but you're worth
a lot of money. I mean, what what's the

02:41:13.360 --> 02:41:15.757
latest valuation? Is it >> 965?

02:41:15.840 --> 02:41:19.277
>> 965 billion dollars
or something like that. So

02:41:19.360 --> 02:41:20.877
>> you sold Instagram
for a billion, right?

02:41:20.960 --> 02:41:25.037
startup in 2010. Yeah. >> Right.
Financials have changed uh quite

02:41:25.120 --> 02:41:27.437
a bit since then. And yet uh Enthropic

02:41:27.520 --> 02:41:31.117
has positioned itself, it is uh you know

02:41:31.200 --> 02:41:33.277
a PBC and it's positioned itself as sort

02:41:33.360 --> 02:41:36.877
of a more um ethical uh company around

02:41:36.960 --> 02:41:38.877
building AI. And I'm wondering if you

02:41:38.960 --> 02:41:43.917
could talk a little bit about how you
see that positioning and anthropics role

02:41:44.000 --> 02:41:47.357
in particular changing the culture
of the valley. Like I know I think back to

02:41:47.440 --> 02:41:51.437
how Google in the beginning of the 2000s
really changed the culture of Silicon

02:41:51.520 --> 02:41:55.197
Valley in so many ways and how do you
see anthropics culture now dictating

02:41:55.280 --> 02:41:57.517
this next era? >> Yeah,
that's a really interesting

02:41:57.600 --> 02:41:59.837
question. Maybe I'll start like
inside and I think there's an external

02:41:59.920 --> 02:42:02.717
component too. I think the reason
I joined in the first place so I was

02:42:02.800 --> 02:42:07.437
winding down my second startup um
and knew I wanted to go work at a frontier

02:42:07.520 --> 02:42:10.557
lab because I had started to use these
models for coding and they were bad at

02:42:10.640 --> 02:42:13.277
coding but I could see that they were
as bad as they were ever going to be at

02:42:13.360 --> 02:42:15.997
coding. they were going to improve and I
had started building on top of these

02:42:16.080 --> 02:42:19.357
APIs. So the startup I was doing
was called artifact and we did sort of AI

02:42:19.440 --> 02:42:23.997
powered uh sort of news recommendations
and um actually read a lot of big

02:42:24.080 --> 02:42:25.677
technology via artifact back in the day.

02:42:25.760 --> 02:42:28.557
It was a lot of things we added.
So um >> but not wired

02:42:28.640 --> 02:42:31.837
>> but not you know you guys had
a really hard payw wall to be honest.

02:42:31.920 --> 02:42:34.877
>> Fair enough.
>> We didn't do very great.

02:42:34.960 --> 02:42:37.517
>> Do you need a discount on my
subscription because I can get one for

02:42:37.600 --> 02:42:40.637
you. Okay. >> It's actually
really funny like the bacon deals.

02:42:40.720 --> 02:42:43.677
>> Yeah, bacon deals. It's the It's the
login cookies. It's like really hard to

02:42:43.760 --> 02:42:46.077
keep people. >> I know.
I know. I just Please escalate

02:42:46.160 --> 02:42:48.637
this to Kai now. I know.

02:42:48.720 --> 02:42:52.077
>> Um uh and email
login very hard to do in

02:42:52.160 --> 02:42:56.077
um but so uh but I was building on top
of the APIs and be like, "Wow, okay,

02:42:56.160 --> 02:42:58.957
they're they're able to do really
interesting things." Um but what

02:42:59.040 --> 02:43:01.117
ultimately made me go to Anthropic was

02:43:01.200 --> 02:43:04.157
like they walk the walk and they really

02:43:04.240 --> 02:43:08.717
uh like deeply believe in trying to make
AI go well for humanity and that is like

02:43:08.800 --> 02:43:13.117
in that's like in the water internally
and I think has been why I think the

02:43:13.200 --> 02:43:16.637
company has remained as cohesive as it
has even as we've grown and I think that

02:43:16.720 --> 02:43:20.157
it's like a testament also to the the
co-founders there on how often they are

02:43:20.240 --> 02:43:23.357
talking about this as well. It was a
a surprise for me coming from a world

02:43:23.440 --> 02:43:28.157
where at Instagram we did a weekly all
hands and we talked about product 95% of

02:43:28.240 --> 02:43:31.277
the time and maybe 5% of the time we
talked about something else that was

02:43:31.360 --> 02:43:34.557
going on in the world around
the company. uh probably maybe I'm

02:43:34.640 --> 02:43:37.517
underelling our like go to market maybe
it was like 8020 but it was definitely

02:43:37.600 --> 02:43:39.517
very very heavy product and I remember

02:43:39.600 --> 02:43:44.557
anthropic about six months in uh myself
um and Kate Jensen who's one of the the

02:43:44.640 --> 02:43:47.117
leaders in the sales organization did
a joint all hands where we talked about

02:43:47.200 --> 02:43:50.157
our like you know how we're doing
product and go to market together and

02:43:50.240 --> 02:43:52.957
people were like this is so great I
finally understand our product strategy

02:43:53.040 --> 02:43:57.277
and like what we've been doing it's
like oh right this is not quote unquote a

02:43:57.360 --> 02:44:01.597
product company you know it is a very
missiondriven AI company with like a

02:44:01.680 --> 02:44:03.997
very strong sense of like why it exists

02:44:04.080 --> 02:44:06.077
in the world. I think in terms of the

02:44:06.160 --> 02:44:11.357
overall impact on the valley remains
to be seen. I think um the positive signs

02:44:11.440 --> 02:44:15.197
that I've seen or interesting signs I've
seen that I've seen are like an renewed

02:44:15.280 --> 02:44:19.117
interest in philanthropy um across the
board and I think that's something that

02:44:19.200 --> 02:44:23.197
um has been written about and I think it
will be an interesting sort of outflow.

02:44:23.280 --> 02:44:26.957
Again, who knows how all of this goes,
but like uh depending on how it goes, it

02:44:27.040 --> 02:44:30.877
could mean a lot of interesting new sort
of philanthropic um uh deployment. And

02:44:30.960 --> 02:44:33.517
then I think the other piece is you know

02:44:33.600 --> 02:44:37.757
uh the conversation around how AI could

02:44:37.840 --> 02:44:41.757
or should go is one that is happening
in real time with the technology versus

02:44:41.840 --> 02:44:44.717
retrospectively which I think has been
the case for other technology waves and

02:44:44.800 --> 02:44:47.597
I think that is a good
thing. >> Interesting.

02:44:47.680 --> 02:44:50.637
>> You know um Mike you know you talked a

02:44:50.720 --> 02:44:54.877
little bit about um anthropic has
this gap that it sees between the

02:44:54.960 --> 02:44:59.037
capabilities of the models and where
everybody is building products. And with

02:44:59.120 --> 02:45:03.517
labs, what you try to do is get ahead
of that so you can show people what AI

02:45:03.600 --> 02:45:07.837
might be able to do now and six months
from now. So please tell us, please tell

02:45:07.920 --> 02:45:11.997
us like what you're building, um, where
you see the where you see the potential

02:45:12.080 --> 02:45:14.077
>> and and what people
should be on the lookout for.

02:45:14.160 --> 02:45:17.277
>> Yeah, it's a road map.
>> Oh, and if I can throw an and

02:45:17.360 --> 02:45:21.437
>> what you're building now, but also
if you had if you have a pie in the sky

02:45:21.520 --> 02:45:25.437
like Elon Musk data centers in space
type ambition, I want to hear about that

02:45:25.520 --> 02:45:27.517
too. Tell us tell us
what tell us everything.

02:45:27.600 --> 02:45:29.677
>> Great. We have 13 minutes left.

02:45:29.760 --> 02:45:34.477
>> Exactly. The rest is the monologue
in my chronic group. Um I think maybe two

02:45:34.560 --> 02:45:37.917
themes I'm really excited about that we've
been exploring a lot. Um the first

02:45:38.000 --> 02:45:41.037
one is giving Claude an environment

02:45:41.120 --> 02:45:46.637
where it has more agency and it also
has more self-nowledge. And I'm going to

02:45:46.720 --> 02:45:50.477
unpack that because that's like a lot
of AI words. But um uh I'll give you an

02:45:50.560 --> 02:45:53.517
example of where we are currently doing
a bad job of this. Like if you are in a

02:45:53.600 --> 02:45:57.517
cloud project and you make a file
with claude, you're like that's great. can

02:45:57.600 --> 02:46:02.477
you add it to our project? Cloud will be
no you have to go download the file and

02:46:02.560 --> 02:46:06.717
go drag and drop into this thing and
you're like what? Um uh until yesterday

02:46:06.800 --> 02:46:09.597
I would have said the same thing about
cloud design and cloud code where if

02:46:09.680 --> 02:46:12.637
you're in cloud code you're like cool
like I need a design for this thing that

02:46:12.720 --> 02:46:15.357
we're building let's or you're in cloud
design and you make a mockup and you

02:46:15.440 --> 02:46:19.277
want to go build it be like cool here's
a zip file and you're like what you know

02:46:19.360 --> 02:46:22.797
and I think um so that's a little bit
of interoperability but in general this

02:46:22.880 --> 02:46:26.557
theme of giving if you give cloud
a lot of notion of its environment I was

02:46:26.640 --> 02:46:30.397
talking to actually a customer like
an API customer um and one of the things

02:46:30.480 --> 02:46:33.597
that they were experimenting with
is actually even giving claude like a

02:46:33.680 --> 02:46:37.117
secure version of their source code
while it's running in the agent loop in

02:46:37.200 --> 02:46:39.917
their product so that if it hits
an issue it doesn't go like I don't know I

02:46:40.000 --> 02:46:42.797
hit an issue it can be like well it's
probably this thing you know at least

02:46:42.880 --> 02:46:46.157
when it's talking to one of the sort
of maintainers of of the software and so

02:46:46.240 --> 02:46:49.277
that overall theme and of course you
have to do it with safeguards and like

02:46:49.360 --> 02:46:53.437
be really careful about what you unlock
with it sounds kind of obvious but it's

02:46:53.520 --> 02:46:57.357
actually night and day in terms of how
expressive these products end up being

02:46:57.440 --> 02:46:59.837
able to be right and you can even see it

02:46:59.920 --> 02:47:04.877
going from like maybe like core chat
or classic chat in cloudi and something

02:47:04.960 --> 02:47:07.917
like co-work where it's got a little
bit more agency and it has a runtime and

02:47:08.000 --> 02:47:10.717
it's able to sort of like understand
a little bit about it environment but I

02:47:10.800 --> 02:47:14.877
think we are at like 10% of the journey
about where we we could go. Um actually

02:47:14.960 --> 02:47:17.677
one of the reasons I think people got
excited about things like openclaw is

02:47:17.760 --> 02:47:22.957
seeing how a you know harness that is
modifiable and you can talk to it about

02:47:23.040 --> 02:47:26.397
things and you don't ever get the sense
of like oh sorry I can't do that you're

02:47:26.480 --> 02:47:30.237
gonna have to go to this like settings
screen and turn it on. It's just a thing

02:47:30.320 --> 02:47:33.597
it has access to and hopefully with like
the right gardening and permissions. So

02:47:33.680 --> 02:47:38.157
that's like theme one that I'm like
extremely excited about. Um and I think

02:47:38.240 --> 02:47:39.837
I if we do it right should actually like

02:47:39.920 --> 02:47:41.437
transform all of our products like from

02:47:41.520 --> 02:47:46.237
head to toe. Um the other piece um is uh

02:47:46.320 --> 02:47:50.077
and I'll maybe like share like the the
the feel not the internal product we're

02:47:50.160 --> 02:47:54.077
working on but like the the phrase I got
as feedback was uh like I think closing

02:47:54.160 --> 02:47:56.717
the gap. I mean I talked about closing
the gap between capabilities and

02:47:56.800 --> 02:47:59.997
reality. I think it's also closing the
gap between how people understand their

02:48:00.080 --> 02:48:04.477
own work and then how the actual
day-to-day is to do that work. Um I was

02:48:04.560 --> 02:48:06.637
talking to somebody uh internally who's

02:48:06.720 --> 02:48:10.157
on our privacy team and to move a ticket

02:48:10.240 --> 02:48:14.557
from like one queue through another one
via the task tracker into another was

02:48:14.640 --> 02:48:16.397
like eight different steps of copy and

02:48:16.480 --> 02:48:18.717
pasting of like manually moving pretty

02:48:18.800 --> 02:48:21.757
you know like uh uh kind of annoying to

02:48:21.840 --> 02:48:25.837
have to do probably errorprone. You have
to like keep spot checking it. and we

02:48:25.920 --> 02:48:29.917
helped her with one of our labs projects
to basically like make that not a pain

02:48:30.000 --> 02:48:32.397
and she's like ah this is the first time
in my career and she's like been working

02:48:32.480 --> 02:48:35.677
for 30 years. We're like what's
in my head and what I am using is like now

02:48:35.760 --> 02:48:40.797
this like it is now closed. Um,
and I want to like bring that feeling to

02:48:40.880 --> 02:48:43.837
everybody who like, you know, of course
cloud unlocked a lot of, you know,

02:48:43.920 --> 02:48:47.197
nontechnical people being able to code,
but like it's we're still asking people

02:48:47.280 --> 02:48:51.357
to understand way too many concepts
of like what is what is, you know, the

02:48:51.440 --> 02:48:55.357
difference between like my sandbox
environment and production or like

02:48:55.440 --> 02:48:58.557
connected MCP is myself or others or how
should I store data and of course you

02:48:58.640 --> 02:49:02.077
can't abstract everything, but like if
you combine both of those themes, if you

02:49:02.160 --> 02:49:04.797
give cloud a lot of self-nowledge and
you're creating an environment where it

02:49:04.880 --> 02:49:09.517
can actually solve complex problems for
people and repeatable ways. Um, I think

02:49:09.600 --> 02:49:11.517
I get like very very excited about those.

02:49:11.600 --> 02:49:12.637
>> And your moonshot. >> Yeah.

02:49:12.720 --> 02:49:14.237
>> I'm not letting you off the hook.
What's your moonshot?

02:49:14.320 --> 02:49:16.957
>> Moonshot. >> Yeah. >> Uh, no.
Nothing in space. Although I

02:49:17.040 --> 02:49:21.357
guess we're, you know, we're in talking
to SpaceX about spacey things. Um, but

02:49:21.440 --> 02:49:22.957
you're talking to space. I mean, those

02:49:23.040 --> 02:49:25.997
are the announcement for compute. Yeah.

02:49:26.080 --> 02:49:30.237
>> Um, it it was exploring extra
extraorbital. What was the phrase?

02:49:30.320 --> 02:49:34.077
Something about exploring
like post orbital world things.

02:49:34.160 --> 02:49:37.517
>> Definitely not my department, but yeah,
there's there's stuff stuff in So the

02:49:37.600 --> 02:49:40.317
labs is isn't working specifically
with the team on compute,

02:49:40.400 --> 02:49:42.397
>> right? Exactly. Separate separate.

02:49:42.480 --> 02:49:44.717
>> Totally separate. Okay.
Okay. So you're moonshot.

02:49:44.800 --> 02:49:47.917
>> Yeah. >> Do you personally
believe in data centers in space?

02:49:48.000 --> 02:49:51.997
>> I had a conversation I uh I by far
from a data center expert, but I talked to

02:49:52.080 --> 02:49:55.837
somebody who is a like a person who
sends things to space who's not Elon

02:49:55.920 --> 02:49:58.557
Musk. Um and >> that's what
that's what you would say.

02:49:58.640 --> 02:50:00.957
>> Yeah.

02:50:01.040 --> 02:50:05.597
Um uh and they were really bullish and I
was like trying to talk about why and

02:50:05.680 --> 02:50:07.197
there was basically like I guess

02:50:07.280 --> 02:50:11.037
effectively infinite um uh uh power if

02:50:11.120 --> 02:50:14.797
you convert it well and you know
infinite land and I was like okay you

02:50:14.880 --> 02:50:17.917
can buy that I mean and I think they
feel good about the shielding you have

02:50:18.000 --> 02:50:21.677
to do again clearly not my area of
expertise but after that talk I was like

02:50:21.760 --> 02:50:26.557
okay I see I see it you know um even if
it's going to take a few years at first I

02:50:26.640 --> 02:50:29.437
>> admittedly thought it was a crazy idea
in general but now I'm like oh actually

02:50:29.520 --> 02:50:31.277
clearly can understand
why this might make sense.

02:50:31.360 --> 02:50:32.717
>> When when you were
talking earlier about

02:50:32.800 --> 02:50:36.477
the ways that uh the work in claude
is going to get compressed and all those

02:50:36.560 --> 02:50:40.877
steps, I couldn't help but think
of tokens and how you know maybe it's good

02:50:40.960 --> 02:50:43.277
for your business model in the short
term if people have to take so many

02:50:43.360 --> 02:50:46.557
steps and use so many tokens. But tokens
have become this unit of economics that

02:50:46.640 --> 02:50:48.237
we're using to describe the industry now

02:50:48.320 --> 02:50:50.717
and people are token maxing and now

02:50:50.800 --> 02:50:54.077
they're token mising and and and uh one

02:50:54.160 --> 02:50:55.997
I want to see I want to hear where you

02:50:56.080 --> 02:50:57.917
sit on that spectrum if you're a token

02:50:58.000 --> 02:51:01.197
maxer and two is there a near future in

02:51:01.280 --> 02:51:03.037
which the industry is not actually

02:51:03.120 --> 02:51:06.237
measured by tokens you know it goes the

02:51:06.320 --> 02:51:10.397
way of MIPS or dialup or some other
you know there's some other unit of

02:51:10.480 --> 02:51:14.557
measurement that actually defines
the economics of this era. Yeah, I think

02:51:14.640 --> 02:51:17.197
both of those are really interesting
questions. It was interesting earlier

02:51:17.280 --> 02:51:19.917
this year when you started hearing about
like companies that have like dashboards

02:51:20.000 --> 02:51:24.077
showing like who used it the most and we
of course have like internal metrics as

02:51:24.160 --> 02:51:28.237
well. And we found that there's not
a lot of correlation between like the

02:51:28.320 --> 02:51:31.117
person who's using the most tokens
and like the person that I like it it's an

02:51:31.200 --> 02:51:33.597
interesting thought I maybe do at your
companies is like write down your 10

02:51:33.680 --> 02:51:36.157
most productive people that you think
are most productive and then like get

02:51:36.240 --> 02:51:39.037
your top 10 token users and see how
closely they correlate. At least for us

02:51:39.120 --> 02:51:43.597
it wasn't that correlated. Does it seem
dangerous to sort of like uh sort of

02:51:43.680 --> 02:51:47.437
purely glorify the like maximum usage
obviously it's like very gameable but

02:51:47.520 --> 02:51:51.197
even beyond that I think it's it's
you know yes you can ask cloud to do 10

02:51:51.280 --> 02:51:53.837
different variants on something but if
you thought about it deeply maybe you

02:51:53.920 --> 02:51:56.397
would do choose two that you thought
were most promising and the third one if

02:51:56.480 --> 02:52:00.157
you then had like some iteration on that
as well. So I would not say like a token

02:52:00.240 --> 02:52:04.557
max. Actually the tokeniest thing was
that conversion uh thing I did was just

02:52:04.640 --> 02:52:07.277
like a couple million tokens. There's
like a lot of tokens that it took to to

02:52:07.360 --> 02:52:11.997
convert the uh the thing from from
Python to TypeScript. Um but I think

02:52:12.080 --> 02:52:16.077
people are being more thoughtful about
um these different pieces and and one of

02:52:16.160 --> 02:52:19.917
the things we look at whenever we look
at a model launch is not just model

02:52:20.000 --> 02:52:22.957
intelligence but we're also really
thinking about model intelligence and

02:52:23.040 --> 02:52:26.397
effort and token efficiency as that
combination. And I think that's a big

02:52:26.480 --> 02:52:29.917
lever we have to improve is how do
we continue to be more and more token

02:52:30.000 --> 02:52:33.277
efficient for a given task so that you
can also hopefully you don't have to

02:52:33.360 --> 02:52:36.077
think very hard about this we can do
this automatically but like we're able

02:52:36.160 --> 02:52:39.277
to tune the solution to the problem
a little bit more. Um, and then to your

02:52:39.360 --> 02:52:42.957
second question, yeah, I, you know,
when I was still in the CPO seat, I I was

02:52:43.040 --> 02:52:45.917
thinking a lot about sort
of outcomebased pricing as something that

02:52:46.000 --> 02:52:48.877
would be really interesting to do if you
could do it. And of course, if you talk

02:52:48.960 --> 02:52:52.877
to, uh, like the Sieras and Fins of
the world that have like a really clear,

02:52:52.960 --> 02:52:56.557
like we kept this, you know, we were
able to solve this customer request and

02:52:56.640 --> 02:53:00.637
not have it go escalated. Like that's
really clear. Um, it gets so much

02:53:00.720 --> 02:53:03.437
fuzzier on these like tasks that we
actually ask Claude these days like I

02:53:03.520 --> 02:53:06.957
had a strategy document. I use Claude to
critique my strategy document. Like what

02:53:07.040 --> 02:53:09.517
was the outcome? It's like well I don't
know it's like tell me how the strategy

02:53:09.600 --> 02:53:12.797
goes six months from now. It feels like
it's going to be very hard to to capture

02:53:12.880 --> 02:53:15.677
that as well. But um I would like to see

02:53:15.760 --> 02:53:17.837
some more experimentation around can you

02:53:17.920 --> 02:53:22.877
better capture like what it's worth you
know to the individual and then or the

02:53:22.960 --> 02:53:26.637
company and then can we find the best
way to to do that as well and I guess

02:53:26.720 --> 02:53:29.997
the the most concrete thing we've moved
towards that and uh we have a product

02:53:30.080 --> 02:53:32.557
product called cloud managed
agents where we'll run all of the

02:53:32.640 --> 02:53:35.997
infrastructure for you in terms of doing
all of the you know agentic harness and

02:53:36.080 --> 02:53:39.437
calling the tools etc and you can either
do it in sort of the normal mode which

02:53:39.520 --> 02:53:42.717
is you give it tasks it you know will go
through tokens it'll tell you when it's

02:53:42.800 --> 02:53:46.157
done or we an outcome based mode where
you can say here's what good looks like

02:53:46.240 --> 02:53:50.237
here's a rubric go and do it and you
know you know it'll go off and and make

02:53:50.320 --> 02:53:53.197
it more outcome so like if everybody had
moved on to that API then I think maybe

02:53:53.280 --> 02:53:56.317
we could have a different sort
of outcome based uh pricing but we'll see

02:53:56.400 --> 02:53:59.197
how that gets adopted >> John
or the guys in the back can do we

02:53:59.280 --> 02:54:03.917
have the random image can we show
the random image >> um if we can great

02:54:04.000 --> 02:54:06.237
>> I'm excited the random
image >> oh here it is

02:54:06.320 --> 02:54:09.437
>> okay it's just because we didn't have
a good label for it so we just called it

02:54:09.520 --> 02:54:12.317
the random image because it might come
up at any point but this is a chart from

02:54:12.400 --> 02:54:16.157
the Financial Times Um,
speaking of utility, where it shows

02:54:16.240 --> 02:54:20.397
the amount of app releases that have
come out, um, which are skyrocketing,

02:54:20.480 --> 02:54:23.517
and then apps with significant usage
that seem to be going down and app

02:54:23.600 --> 02:54:26.317
reviews which seem to be going down. Um,

02:54:26.400 --> 02:54:30.797
so Mike, I'd love to hear you respond to
what we're seeing in the image here. Um,

02:54:30.880 --> 02:54:34.237
is it possible that like everybody's
coding and releasing, but we're not

02:54:34.320 --> 02:54:36.157
really seeing a big boom in productivity?

02:54:36.240 --> 02:54:39.357
>> That's really I mean, I think there's
definitely a parallel in app usage in

02:54:39.440 --> 02:54:43.277
general. It'd be interesting to seeing
if any of those app released became one

02:54:43.360 --> 02:54:47.197
of the apps with significant usage. Um,
>> we could take it we could take it down.

02:54:47.280 --> 02:54:49.837
Yeah, go ahead. >> Um,
I think uh this ties into something

02:54:49.920 --> 02:54:51.837
that I've been thinking a lot about
which obviously my background is in

02:54:51.920 --> 02:54:53.197
consumer and I've been wondering what

02:54:53.280 --> 02:54:55.677
the consumer AI breakouts will end up

02:54:55.760 --> 02:54:59.197
being and we've seen a lot of them yet.

02:54:59.280 --> 02:55:03.917
Um, and I think part of it is, you know,
I I don't know how far back that that

02:55:04.000 --> 02:55:07.117
chart goes, but when we were releasing
Instagram, it still felt a little bit

02:55:07.200 --> 02:55:11.277
wild west in terms of the apps. people
were excited about apps and like two

02:55:11.360 --> 02:55:14.957
kind of random people released an app
and were able to get to like number one

02:55:15.040 --> 02:55:18.477
in photos and video within three months,
right? I think that is much harder now

02:55:18.560 --> 02:55:20.237
when you think about how consolidated

02:55:20.320 --> 02:55:23.597
the top 10 is and how much time spent is

02:55:23.680 --> 02:55:27.517
spent on like the Tik Toks and reals
of the world. It's a lot, right? And so I

02:55:27.600 --> 02:55:30.957
think getting that breakthrough consumer
experience I think is really really

02:55:31.040 --> 02:55:33.117
hard. So I think that is uh as much a

02:55:33.200 --> 02:55:35.677
story about how sort of consolidated

02:55:35.760 --> 02:55:38.157
consumer products are these days. number

02:55:38.240 --> 02:55:41.997
one. Number two, how um entrenched or

02:55:42.080 --> 02:55:44.557
how powerful it is to have uh that sort

02:55:44.640 --> 02:55:49.517
of data uh people call it data gravity
like the data gravity of something like

02:55:49.600 --> 02:55:52.877
your Google Docs are in your Google Docs.
So even if somebody has a like 2x

02:55:52.960 --> 02:55:57.037
better AI powered, you know, doc editor,
you're going to move all your stuff

02:55:57.120 --> 02:56:00.557
maybe, probably not. So I think that it
speaks to, you know, the things that are

02:56:00.640 --> 02:56:03.437
sticky. I think I think about it a lot
is like the hard stuff is still hard

02:56:03.520 --> 02:56:06.797
like making something people want still
really hard. you know, we have amazing

02:56:06.880 --> 02:56:10.717
models internally on top of not all
of our products work, right? And so, um, I

02:56:10.800 --> 02:56:14.557
think that's a bullish sign for like
product people like me because it means

02:56:14.640 --> 02:56:17.357
that I think we hopefully still add value.
But I think that chart is maybe

02:56:17.440 --> 02:56:19.997
another place which is like, uh, it's

02:56:20.080 --> 02:56:22.477
harder in many ways than ever to uh,

02:56:22.560 --> 02:56:27.037
break through even if you can code
more quickly. And could we have done

02:56:27.120 --> 02:56:29.357
Instagram in, you know, a month instead

02:56:29.440 --> 02:56:32.557
of three or four? Probably. But we got

02:56:32.640 --> 02:56:36.797
there after like a long winding
turns and twists and turns process.

02:56:36.880 --> 02:56:39.437
>> You had I think 18 people
at Instagram when you sold it.

02:56:39.520 --> 02:56:41.677
>> 13. >> With these tools,
do you think you would

02:56:41.760 --> 02:56:44.797
have How many people do you
think you would have had?

02:56:44.880 --> 02:56:46.877
>> It's really interesting
because of those 13

02:56:46.960 --> 02:56:48.077
>> give us a number
because like everyone's

02:56:48.160 --> 02:56:50.957
like, "Oh, one billion$1 billion one
person startup." How close could you

02:56:51.040 --> 02:56:55.117
guys have gotten? >> I think
we could have gotten there with

02:56:55.200 --> 02:56:58.317
like four to six, you know? >> Okay.

02:56:58.400 --> 02:57:01.437
>> Yeah. Um, or the thing that we would
have done if we'd grown it, we'd be able

02:57:01.520 --> 02:57:05.117
to do things in more than a single track.
Like Instagram was, if you ever

02:57:05.200 --> 02:57:08.317
watched my 5-year-old play soccer now,
by which I mean like the ball is there

02:57:08.400 --> 02:57:11.437
and every single person runs to the ball.
Like that was our product team. It

02:57:11.520 --> 02:57:15.517
was like video go and everybody like
goes and works on the one thing and like

02:57:15.600 --> 02:57:19.757
we' be able to like play positions.
Like Android we built in about a month for

02:57:19.840 --> 02:57:23.357
Instagram. We could have done it
probably in a week with the models and

02:57:23.440 --> 02:57:26.797
but and to build Android we took
everybody off iOS and we all like

02:57:26.880 --> 02:57:31.037
relearned to code in uh you know code
Android OS and then we went off and do

02:57:31.120 --> 02:57:34.237
that and like for that whole month we
were barely shipping updates on iOS. So

02:57:34.320 --> 02:57:36.317
I think you can be a lot more actually
a really good example there's a labs

02:57:36.400 --> 02:57:41.437
project I have internally um that um
helps accelerate how uh anthropic

02:57:41.520 --> 02:57:46.077
engineers like code and do code
review and that um that project I am

02:57:46.160 --> 02:57:50.637
maintaining an iOS and an Android
version of uh and I basically have the

02:57:50.720 --> 02:57:53.677
cloud that works on the iOS one
basically like ping the Android one and

02:57:53.760 --> 02:57:56.557
be like hey I implemented this sorry
Android users it's still the second one

02:57:56.640 --> 02:58:01.597
even in ALM world sorry um uh and then
the the Android version is like okay I'm

02:58:01.680 --> 02:58:04.157
going to do this oh that doesn't count
because that feature doesn't makes sense

02:58:04.240 --> 02:58:06.957
here. I'm going to drop it. And of course,
we wouldn't have been able to

02:58:07.040 --> 02:58:10.797
delegate all of that on on Instagram,
but we sure could have done a lot by

02:58:10.880 --> 02:58:14.637
having sort of platform parity.
Like this dream of platform like close to

02:58:14.720 --> 02:58:17.357
parody is now actually quite doable.

02:58:17.440 --> 02:58:19.997
>> You're probably going to get calls
now from the remaining six or seven people

02:58:20.080 --> 02:58:24.237
on your Instagram team going, "Cool,
was I did I make the cut in the new era?"

02:58:24.320 --> 02:58:26.877
Also, it sounds like you probably could
bring Gotham back now if you really

02:58:26.960 --> 02:58:29.197
wanted to with all >> I forget
if we eventually I think for

02:58:29.280 --> 02:58:31.277
April Fools, maybe we
brought it back one day. Yeah.

02:58:31.360 --> 02:58:32.397
>> Yeah. Do we have time
for one more question?

02:58:32.480 --> 02:58:33.597
>> Yeah. Yeah. Last question. Yeah. Okay.

02:58:33.680 --> 02:58:36.157
Um I mean my last question for you is um

02:58:36.240 --> 02:58:38.957
you worked on a product that now as it

02:58:39.040 --> 02:58:43.917
has evolved is is um in many ways
ethically fraught because of some of the

02:58:44.000 --> 02:58:46.557
harms that people are concerned about
with children. And when you talk about

02:58:46.640 --> 02:58:50.237
the fact that there hasn't really been
a big breakout consumer app for AI, I

02:58:50.320 --> 02:58:54.477
think there has and it's chat bots,
right? And chat bots have also led to

02:58:54.560 --> 02:58:56.797
some real dangers and harms for young

02:58:56.880 --> 02:58:58.557
people. And so when you are building in

02:58:58.640 --> 02:59:02.557
labs, how are you thinking about the you

02:59:02.640 --> 02:59:05.757
know the potential harms and the risks
that come with just making this

02:59:05.840 --> 02:59:08.637
technology that much better? >> Yeah.
I mean I think there there are

02:59:08.720 --> 02:59:11.757
certainly products that we have either
prototyped or conceptualized and been

02:59:11.840 --> 02:59:16.157
like this product because it sounds so
hypy. I hate this, but like like this

02:59:16.240 --> 02:59:20.397
product if shipped would be bad for the
world or like would nudge people in the

02:59:20.480 --> 02:59:24.317
wrong direction or even if we did
it right, the like wrong or like more

02:59:24.400 --> 02:59:26.077
morally fraught version of this would be

02:59:26.160 --> 02:59:29.517
actively we think bad. And so I think

02:59:29.600 --> 02:59:31.917
>> asking that question
a lot internally um

02:59:32.000 --> 02:59:36.557
makes a difference. Um and having it's
a luxury to have core products and models

02:59:36.640 --> 02:59:39.997
that are doing really really well. So
we don't like that's that in some ways an

02:59:40.080 --> 02:59:43.917
easy decision even if we think it could
get a lot of um a lot of use. But yeah,

02:59:44.000 --> 02:59:47.597
I think going back to an earlier
conversation, I think frontloading it is

02:59:47.680 --> 02:59:51.117
really valuable and really
thinking through like

02:59:51.200 --> 02:59:54.637
it is now more normalized to have people
at a company and definitely anthropic

02:59:54.720 --> 02:59:56.877
does who are like economists thinking
about the impact of the thing that

02:59:56.960 --> 02:59:59.917
you're building on the world and that just
was not the case in along the years

03:00:00.000 --> 03:00:02.317
on on most of social media I think.

03:00:02.400 --> 03:00:05.357
>> Yeah, Mike, it's always great to speak
and we thank you again for bringing your

03:00:05.440 --> 03:00:08.797
insight today and um let's do it again

03:00:08.880 --> 03:00:11.517
soon. Let's hear from Mike and Lauren.

03:00:11.600 --> 03:00:15.117
Thank you. Great job. Thank you. All

03:00:15.200 --> 03:00:20.440
right. Are you guys ready
for our last conversation of the day?

03:00:20.640 --> 03:00:23.677
>> Are you having a good time?

03:00:23.760 --> 03:00:27.517
>> All right. Um, so in 2015, Greg

03:00:27.600 --> 03:00:29.837
Brockman, Elon Musk, and Sam Ultman

03:00:29.920 --> 03:00:33.597
started a nonprofit called Open AI, and

03:00:33.680 --> 03:00:35.597
the plan was to pursue artificial

03:00:35.680 --> 03:00:38.637
general intelligence. the company or the

03:00:38.720 --> 03:00:40.717
nonprofit when it was a nonprofit back

03:00:40.800 --> 03:00:43.037
then began in Greg Brockman's living

03:00:43.120 --> 03:00:46.557
room. And these folks were convinced

03:00:46.640 --> 03:00:48.317
that achieving artificial general

03:00:48.400 --> 03:00:53.917
intelligence or AI on par with human
intelligence was possible. And to be

03:00:54.000 --> 03:00:56.317
honest, most people in the valley
thought that that was an interesting

03:00:56.400 --> 03:00:57.917
side project, but most of the attention

03:00:58.000 --> 03:01:00.717
was on social media at the time. Well,

03:01:00.800 --> 03:01:03.517
fast forward 11 years and here we are.

03:01:03.600 --> 03:01:07.117
OpenAI is probably going to go public
within the next year at a trillion

03:01:07.200 --> 03:01:10.877
dollar valuation. They're going
to announce likely, you know, because

03:01:10.960 --> 03:01:16.397
thirdparty data is showing it a billion
users in chatbt fairly soon. Uh and they

03:01:16.480 --> 03:01:17.757
of course raised the largest venture

03:01:17.840 --> 03:01:20.637
capital round in history at 122 billion.

03:01:20.720 --> 03:01:23.517
So they are at the the leading edge of a

03:01:23.600 --> 03:01:28.237
technology that has captured all of our
attention and is changing the world. Uh,

03:01:28.320 --> 03:01:31.757
and so when you listen to Greg Brockman,
one of the things that you can see even

03:01:31.840 --> 03:01:36.317
from his conversations all the way
in the past is a clear conviction and an

03:01:36.400 --> 03:01:39.597
understanding of where this technology
would lead and where the products would

03:01:39.680 --> 03:01:43.597
go. It's amazing. You listen to the
podcasts from pre-Chat GPT or really in

03:01:43.680 --> 03:01:45.837
the early days of the GPT models. Uh,

03:01:45.920 --> 03:01:48.157
and you can hear Greg speaking with

03:01:48.240 --> 03:01:52.077
absolute clarity about where the models

03:01:52.160 --> 03:01:54.637
and applications would go today. So I

03:01:54.720 --> 03:01:59.357
think to to close our day, let's take
a look into the future of where open AI

03:01:59.440 --> 03:02:01.437
and the frontier is going in a

03:02:01.520 --> 03:02:06.600
conversation with Greg Brockman.
Let's welcome Greg.

03:02:10.400 --> 03:02:13.357
>> Great to see you, Greg.
>> Thank you for having me.

03:02:13.440 --> 03:02:15.757
Um, you know, Greg, this is our fourth

03:02:15.840 --> 03:02:18.637
time speaking and we've spoken every

03:02:18.720 --> 03:02:21.997
time about OpenAI's product direction

03:02:22.080 --> 03:02:25.757
and I think I'm starting to get it. um

03:02:25.840 --> 03:02:27.917
you know there was this conversation

03:02:28.000 --> 03:02:31.437
that a super app was the wrong term for

03:02:31.520 --> 03:02:35.037
what you were doing with um the app that

03:02:35.120 --> 03:02:37.517
you're building bringing codeex which is

03:02:37.600 --> 03:02:40.717
coding side of openi's product browser

03:02:40.800 --> 03:02:45.437
and chat gpt together and when you use
the word super app people would be like

03:02:45.520 --> 03:02:48.797
no a super app is actually something
that you can just use every other app

03:02:48.880 --> 03:02:53.597
within um and now as we've
seen these products

03:02:53.680 --> 03:02:57.357
come together actually super app might
be the correct term. You know, at least

03:02:57.440 --> 03:03:01.037
for us on the outside, we're starting
to see it that when you need to do

03:03:01.120 --> 03:03:03.357
anything, um you it will start with a

03:03:03.440 --> 03:03:06.637
prompt in chat GPT and then OpenAI's

03:03:06.720 --> 03:03:11.277
technology will use either your browser
or your computer to get that done for

03:03:11.360 --> 03:03:13.677
you. Is that the right
way to think about it?

03:03:13.760 --> 03:03:16.797
>> I I think that's a pretty good
perspective, right? And I think to

03:03:16.880 --> 03:03:21.837
really zoom out, the thing we're
actually trying to build is in AGI,

03:03:21.920 --> 03:03:25.357
right? that if you think about what
people have been using since chat GBT

03:03:25.440 --> 03:03:29.437
it's a language model right
there's a big gap between these

03:03:29.520 --> 03:03:34.557
it's amazing you can talk to chat talks
back to you great wonderful but when we

03:03:34.640 --> 03:03:37.917
launched in 2022 there was no memory
right it's not hooked up to any tools

03:03:38.000 --> 03:03:40.557
has no context and so it really is that

03:03:40.640 --> 03:03:45.517
this conversational intelligence is only
one part of what people really need to

03:03:45.600 --> 03:03:49.037
get work done to be able to achieve
their goals and where we're going is to

03:03:49.120 --> 03:03:52.557
have an AI that's really looking out
for you right that who can provide the

03:03:52.640 --> 03:03:56.317
goals, the directions, that it's
constantly thinking about what can I do

03:03:56.400 --> 03:04:00.237
for Alex today. Uh that it's able to go
and solve super hard problems, very

03:04:00.320 --> 03:04:03.197
mundane problems. You wake up, your inbox
is organized, but also if there's

03:04:03.280 --> 03:04:07.837
like a health plan that you are thinking
about that it can help you help you

03:04:07.920 --> 03:04:11.037
achieve that, figure out medical
treatments, uh or you know, sort of back

03:04:11.120 --> 03:04:15.037
and forth provide you with that kind
of information at least. And I think that

03:04:15.120 --> 03:04:18.397
the question of what's the interface you
want, what is the product that you want

03:04:18.480 --> 03:04:21.837
is what we spend a lot of time thinking
about. The answer is you want almost no

03:04:21.920 --> 03:04:23.757
interface. You want no product, right?

03:04:23.840 --> 03:04:26.637
You want this to be like what's
the interface between you and me, right?

03:04:26.720 --> 03:04:29.437
Just being able to talk to a persistent

03:04:29.520 --> 03:04:33.997
entity of some form that's able to go
and accomplish goals for you. And so

03:04:34.080 --> 03:04:37.997
building that is hard. It will take time,
but we have a lot of the pieces,

03:04:38.080 --> 03:04:41.037
right? We're increasingly bringing
together the product layer, trying to

03:04:41.120 --> 03:04:45.277
make the models better, trying to make
the whole system just so there's less

03:04:45.360 --> 03:04:50.157
like clicking buttons and toggles and
changing modes and all these things. Not

03:04:50.240 --> 03:04:53.197
to say that there won't be some
of those along the way, but the long-term

03:04:53.280 --> 03:04:55.997
trajectory is towards
simplification, unification.

03:04:56.080 --> 03:04:57.277
>> Yeah, it's very
interesting that you say

03:04:57.360 --> 03:05:00.397
the interface uh will melt away. And so

03:05:00.480 --> 03:05:04.317
to go a little bit deeper with my
question, um many of us who use products

03:05:04.400 --> 03:05:08.797
like chatbt today, we'll see that the
bot will make a suggestion at the end.

03:05:08.880 --> 03:05:11.997
You know, you ask it about nutrition and
it says should I make a health plan for

03:05:12.080 --> 03:05:15.997
you or make a diet plan for you? You ask
it, sorry Ron John, we just talked about

03:05:16.080 --> 03:05:17.597
travel, but you ask it about travel and

03:05:17.680 --> 03:05:19.197
then it will give you an agenda for

03:05:19.280 --> 03:05:23.037
instance. And so am I hearing you right

03:05:23.120 --> 03:05:27.677
that what's going to happen within chat
GPT just to give an example is you talk

03:05:27.760 --> 03:05:32.637
to it about your health decisions and it
might say you know you probably need to

03:05:32.720 --> 03:05:36.157
you know go to this specialist let me
make an appointment for you and then it

03:05:36.240 --> 03:05:40.477
will go and actually take that action
on your behalf. So it goes from simply a

03:05:40.560 --> 03:05:44.157
conversation interface to actually
understanding your intent and then going

03:05:44.240 --> 03:05:48.157
out and accomplishing that for you. that
that I that's exactly right and I think

03:05:48.240 --> 03:05:51.037
that if you've used codecs and by the
way how many people in the room have

03:05:51.120 --> 03:05:53.997
used have used codecs >> and people

03:05:54.080 --> 03:05:55.997
>> yeah decent number
of people um and that

03:05:56.080 --> 03:05:58.637
our goal is to really bring the power of

03:05:58.720 --> 03:06:00.797
codecs to everyone right to bring agents

03:06:00.880 --> 03:06:03.437
to everyone you that technology exists

03:06:03.520 --> 03:06:05.677
right now right you can hook up like I

03:06:05.760 --> 03:06:08.957
hook up my codeex to slack to my Gmail

03:06:09.040 --> 03:06:13.117
to my calendar and there are many people
within openai nontechnical users you

03:06:13.200 --> 03:06:16.477
know it's got code in the name but it's
not really about code. It's really about

03:06:16.560 --> 03:06:21.597
having this general purpose tool using
harness an agent and that the kinds of

03:06:21.680 --> 03:06:25.757
things for example someone on our comm's
team does is um she was organizing an

03:06:25.840 --> 03:06:28.477
event and it would just ask all of the

03:06:28.560 --> 03:06:30.797
uh event attendees for their dietary

03:06:30.880 --> 03:06:36.317
preferences set up a whole seating chart
kind of did all of that work uh so that

03:06:36.400 --> 03:06:39.437
she could focus on the parts that she
wanted to and and really thinking about

03:06:39.520 --> 03:06:42.637
the vision of what she wanted to achieve
and I think that we're going to see this

03:06:42.720 --> 03:06:46.797
across the board. So the it's not sci-fi
anymore to think about an AI that's

03:06:46.880 --> 03:06:49.597
hooked up to these tools. And I remember

03:06:49.680 --> 03:06:51.517
with you know our very first attempt at

03:06:51.600 --> 03:06:55.197
tool use in chatbt was 2023. I think in

03:06:55.280 --> 03:06:59.277
like March or April or something we
released plugins. Do people remember

03:06:59.360 --> 03:07:01.677
plugins back in early chat days?

03:07:01.760 --> 03:07:05.677
>> That didn't work. >> It didn't
work at all because the models

03:07:05.760 --> 03:07:08.877
the models weren't ready. Right. The form
factor is correct. Obviously you're

03:07:08.960 --> 03:07:12.637
going to have an AI that's able to like
talk to your Gmail like no question. But

03:07:12.720 --> 03:07:16.157
we could only like have three different
connectors exposed to the model at a

03:07:16.240 --> 03:07:20.397
time or start forgetting. You know, we
had like 2K maybe 4K token context. Like

03:07:20.480 --> 03:07:23.597
there's just no memory, right? It's kind
of like when you had early computers in

03:07:23.680 --> 03:07:27.357
the 60s or 70s or something, right? You
know, that you had tiny little me memory

03:07:27.440 --> 03:07:30.477
banks and today you have your phone
that's like better than any

03:07:30.560 --> 03:07:33.357
supercomputer from that era. And I
think that's where we're going with these

03:07:33.440 --> 03:07:36.157
models, right? That the rate
of improvement has been so steep. So now

03:07:36.240 --> 03:07:39.997
you can have hundreds of different tools
accessible that we have the ability to

03:07:40.080 --> 03:07:43.197
hook them up to whole file system.
So you can almost have the full power of

03:07:43.280 --> 03:07:46.637
the internet and like almost any
application you want at the model's

03:07:46.720 --> 03:07:49.117
fingertips. And it's smart, right? It's

03:07:49.200 --> 03:07:52.957
got 52 million token context, depends

03:07:53.040 --> 03:07:55.837
how you squint on it. And the capability

03:07:55.920 --> 03:08:00.237
level is also getting so so powerful,
right? These models are now solving

03:08:00.320 --> 03:08:03.037
unsolved math problems and physics
problems, right? And really helping

03:08:03.120 --> 03:08:06.317
people be able to achieve things they
couldn't otherwise. like we are on the

03:08:06.400 --> 03:08:11.357
era on the edge of this era of agents
really transforming how we all operate

03:08:11.440 --> 03:08:15.597
whether it's in software engineering
finance legal sales and in our personal

03:08:15.680 --> 03:08:19.677
lives too. So just to unpack that
example that you were giving there one

03:08:19.760 --> 03:08:24.317
of your colleagues is chatting with Chad
GPT about an event and then suggest hey

03:08:24.400 --> 03:08:28.797
you know uh how should we you know
contact event attendees about something

03:08:28.880 --> 03:08:32.157
and instead of like saying okay I
have to do that and going into an event

03:08:32.240 --> 03:08:37.197
program basically what happens is the
the interface will take over from there

03:08:37.280 --> 03:08:41.117
once it says it's a good idea and you
agree and then hook into whatever tools

03:08:41.200 --> 03:08:42.797
you're using and then do it for you.

03:08:42.880 --> 03:08:47.597
>> Exactly. So it uses its you know Gmail
connector searches through your inbox to

03:08:47.680 --> 03:08:49.757
find all the people who are attending

03:08:49.840 --> 03:08:53.037
and then if you're on the like what what

03:08:53.120 --> 03:08:57.357
is in dietary restriction sees oh these
people I already have their dietary

03:08:57.440 --> 03:08:59.517
restrictions these people I do not um

03:08:59.600 --> 03:09:02.397
drafts an email depending on exactly how

03:09:02.480 --> 03:09:07.677
you have things set up it might say hey
I drafted these emails can I send them

03:09:07.760 --> 03:09:10.477
if you have a connector that doesn't
even let it send emails said I drafted

03:09:10.560 --> 03:09:15.197
it you need to send them And uh in a
different world, you could also imagine

03:09:15.280 --> 03:09:18.397
that it's you've built enough trust with
the system where it says I drafted the

03:09:18.480 --> 03:09:23.037
emails and I actually sent them.
And I think that this actually points to a

03:09:23.120 --> 03:09:27.117
really important aspect of the agentic
era, which is trust, right? That we need

03:09:27.200 --> 03:09:30.397
to really learn how to build trust with
these systems, where they're good, where

03:09:30.480 --> 03:09:34.957
they're not. Figure out what you want
to delegate to them and how you want to

03:09:35.040 --> 03:09:38.077
entrust them with responsibility.
And that's something we view as earned,

03:09:38.160 --> 03:09:41.437
right? It's not something that that we
can we can grant but by providing lots

03:09:41.520 --> 03:09:44.077
of tools and control and oversight and

03:09:44.160 --> 03:09:48.477
supervision to the operator to the person
who that this AI is operating on

03:09:48.560 --> 03:09:51.837
behalf of like we think that that is
going to be such an important thing. So

03:09:51.920 --> 03:09:54.477
that's a key product
feature and differentiator.

03:09:54.560 --> 03:09:57.677
>> Yeah. And when you go back to some
of the early attempts at this, there was

03:09:57.760 --> 03:10:02.797
this like move that OpenAI had to let
you call an Uber within Chat GPT. And it

03:10:02.880 --> 03:10:05.997
it followed a long line of companies
that have tried to get you to take

03:10:06.080 --> 03:10:08.477
action within chat, but it never really

03:10:08.560 --> 03:10:11.517
took off. And the difference here might

03:10:11.600 --> 03:10:14.797
be that the chatbot can take control of

03:10:14.880 --> 03:10:19.437
your browser or take control of your
computer and then you don't necessarily

03:10:19.520 --> 03:10:23.277
have to worry about like is this plug-in
going to work? it goes and accomplishes

03:10:23.360 --> 03:10:25.437
that for you by taking over your

03:10:25.520 --> 03:10:28.557
machine. So I wonder you know if you

03:10:28.640 --> 03:10:31.677
expect a fight from the user interfaces

03:10:31.760 --> 03:10:34.077
that we have today aka like all the

03:10:34.160 --> 03:10:36.637
other apps all the software where to be

03:10:36.720 --> 03:10:39.837
truly useful chat GPT will have to not

03:10:39.920 --> 03:10:43.837
be blocked to be able to go out
and execute these actions on behalf of a

03:10:43.920 --> 03:10:48.557
user. Well, look f first of all, I'd
say that it's this is not theoretical at

03:10:48.640 --> 03:10:51.277
this point, right? That people have
been using Kodak. So, it's a separate

03:10:51.360 --> 03:10:54.157
product, separate app, you have
to install it separately. Really starting

03:10:54.240 --> 03:10:59.037
to focus on software engineering. But
the amount of non-software work that has

03:10:59.120 --> 03:11:01.917
been happening in Codex has been
absolutely exploding, right? It's been

03:11:02.000 --> 03:11:06.077
this like incredible exponential curve,
exactly the thing that you you would

03:11:06.160 --> 03:11:09.677
expect. And within OpenAI, we basically
have the same level of penetration now

03:11:09.760 --> 03:11:13.197
in usage as Slack, right? It's like
everyone and OpenAI is like an entirely

03:11:13.280 --> 03:11:16.477
Slack based company. We do not use email
for the most part. It's like really like

03:11:16.560 --> 03:11:19.197
if you're not on Slack, you're not going
to do any work. Um, and it's kind of

03:11:19.280 --> 03:11:23.357
feeling that way now with Codeex app
as well. And that everyone's Codeex is

03:11:23.440 --> 03:11:28.317
hooked up to all of these tools. Um, how
the EOS ecosystem evolves. I think it's

03:11:28.400 --> 03:11:31.677
going to be a very nuanced thing because
I think one thing that is very important

03:11:31.760 --> 03:11:36.557
is that we believe that there should be
an ecosystem that gets to be vibrant and

03:11:36.640 --> 03:11:40.797
thriving and that people can really
build and see the benefits. And so we've

03:11:40.880 --> 03:11:44.797
actually seen this from uh partner
companies where uh you know that we I

03:11:44.880 --> 03:11:47.597
remember there's a couple different
partners where we said hey we really

03:11:47.680 --> 03:11:50.557
want to train our AI to be really good
at using your software and we didn't

03:11:50.640 --> 03:11:53.197
know what they would say and actually

03:11:53.280 --> 03:11:57.677
the response we got is this is the most
partner friendly outreach we've ever had

03:11:57.760 --> 03:12:01.917
right that the idea that you will make
your AI specifically good at using our

03:12:02.000 --> 03:12:06.077
tool and they just see the opportunity
because their tool will be used just so

03:12:06.160 --> 03:12:10.797
much more as a result and that everyone
is trying to think about how do they not

03:12:10.880 --> 03:12:14.877
just survive as a company into the AI
era but thrive? like how do you really

03:12:14.960 --> 03:12:18.557
get the advantages of the fact there's
going to be so much more activity and if

03:12:18.640 --> 03:12:22.557
you don't have AI in there if you shut
it out then you're actually going to be

03:12:22.640 --> 03:12:27.357
declining not thriving right this this
kind of makes open AI puts open AI so

03:12:27.440 --> 03:12:31.917
first of all you're going to bring you
talk about people using codeex so one of

03:12:32.000 --> 03:12:35.437
your colleagues shared and I think
you've talked about this too that you've

03:12:35.520 --> 03:12:37.837
brought chat GPT into codeex so you can

03:12:37.920 --> 03:12:42.477
bring codeex into chat GPT which
is basically like if we're users of chat

03:12:42.560 --> 03:12:45.037
GPT this experience
that we talked about of

03:12:45.120 --> 03:12:49.437
chatt not only suggesting what you might
want to do next but going to do it for

03:12:49.520 --> 03:12:53.197
you that's going to happen and so
it makes you effectively an operating

03:12:53.280 --> 03:12:57.757
system don't you think but not
the operating system like an iOS where you

03:12:57.840 --> 03:13:02.477
would like go open up the your phone
and then tap different apps it's almost as

03:13:02.560 --> 03:13:06.637
if all interaction with all apps will
happen through this interface is that

03:13:06.720 --> 03:13:11.277
the ambition >> I think
that you could describe it that

03:13:11.360 --> 03:13:13.517
way but I think of it a differently.

03:13:13.600 --> 03:13:15.357
Like the way that I think about this is

03:13:15.440 --> 03:13:18.157
that what is the ideal interface to an

03:13:18.240 --> 03:13:20.957
AGI or we call it kind of a personal AGI

03:13:21.040 --> 03:13:25.437
and I think that it's again the same
interface that you and I are using right

03:13:25.520 --> 03:13:29.277
now you just want to talk to an
assistant right you want to talk to

03:13:29.360 --> 03:13:31.117
something that can go and and and work

03:13:31.200 --> 03:13:34.077
and and operate on your behalf and so

03:13:34.160 --> 03:13:37.597
that yes like that agent that AGI that

03:13:37.680 --> 03:13:41.597
AI will have its own computer right it
will have its own access to things that

03:13:41.680 --> 03:13:45.037
maybe And you know like ideal co-orker
would be they can come over and type

03:13:45.120 --> 03:13:47.197
things on your computer too. So some

03:13:47.280 --> 03:13:50.557
access some delegated access to your own

03:13:50.640 --> 03:13:55.597
own system and you know maybe you
delegate access to your inbox sometimes

03:13:55.680 --> 03:13:59.197
maybe it has its own inbox with some
sort of uh some sort of you know window

03:13:59.280 --> 03:14:04.157
into into the things that it needs.
You forward emails to it. Th these are not

03:14:04.240 --> 03:14:07.277
actually if you if you think about this
is not unprecedented, right? It's like

03:14:07.360 --> 03:14:10.557
the way that you work with an assistant

03:14:10.640 --> 03:14:14.797
who's a person that we've we've actually
or any co-orker really, we've spent a

03:14:14.880 --> 03:14:18.477
lot of time really thinking about how
do you build these trust boundaries and

03:14:18.560 --> 03:14:22.237
make sure that you're able to operate
together. And so I think of it as just a

03:14:22.320 --> 03:14:25.597
different thing. It's not it you could
think of it as an operating system, but

03:14:25.680 --> 03:14:29.357
an operating system is almost something
from a different time, right? It's a

03:14:29.440 --> 03:14:33.197
different layer of the stack. This is
really more about how do you interface

03:14:33.280 --> 03:14:35.357
with technology broadly. And I think

03:14:35.440 --> 03:14:37.917
that the beautiful thing about AI is

03:14:38.000 --> 03:14:41.037
it's really about bringing the machine

03:14:41.120 --> 03:14:45.597
closer to the human rather than us
having to contort ourselves into like

03:14:45.680 --> 03:14:49.837
files and folders and like all these
details that somehow are not natural,

03:14:49.920 --> 03:14:52.797
right? That are more about how the machine
operates rather than how we operate.

03:14:52.880 --> 03:14:56.557
>> Yeah. Talking about a personal
intelligence, it sort of um I don't did

03:14:56.640 --> 03:14:58.317
you watch WWDC last week?

03:14:58.400 --> 03:15:00.317
>> Uh no, no, I missed it. I was I was

03:15:00.400 --> 03:15:04.477
banned, but um I watched it on TV. Um

03:15:04.560 --> 03:15:09.117
come on, Apple. Anyway, it does look
like you and and and Siri, the new Siri

03:15:09.200 --> 03:15:12.797
are going to come kind of into
competition, right? Because they're an

03:15:12.880 --> 03:15:15.917
app that's going to sit or an intelligence
that will sit on top of all

03:15:16.000 --> 03:15:18.397
of your apps and let you take action.

03:15:18.480 --> 03:15:22.237
And Chacht will be an app on the iPhone.

03:15:22.320 --> 03:15:26.237
So then talk a little bit about
whether that positioning is going to be

03:15:26.320 --> 03:15:29.597
difficult for open AI and how you're
thinking about that strategically.

03:15:29.680 --> 03:15:32.957
>> Well, I just think again think of it
a little differently. Like I think that

03:15:33.040 --> 03:15:38.077
we're in the beginning of this new
agentic era and the way that this has

03:15:38.160 --> 03:15:42.157
always gone in AI is that when you have
a new level of capability, it means you

03:15:42.240 --> 03:15:45.917
have an opportunity to rethink
everything, right? Rethink how

03:15:46.000 --> 03:15:48.477
>> people interface, how like what the

03:15:48.560 --> 03:15:53.517
techn is capable of. And I think that
this is no no different, right? In my

03:15:53.600 --> 03:15:55.357
mind, like the kinds of things that I

03:15:55.440 --> 03:15:57.357
see on the horizon,

03:15:57.440 --> 03:16:01.037
for example, AI for solving

03:16:01.120 --> 03:16:04.717
scientific problems, right? And I think
we're starting to see the inklings of

03:16:04.800 --> 03:16:08.637
this. Like for example, today we announced

03:16:08.720 --> 03:16:13.517
we have in in uh peer-reviewed
literature people doctors who are using

03:16:13.600 --> 03:16:17.757
03. Remember 03? >> Yep.
>> That was like forever ago now, right?

03:16:17.840 --> 03:16:19.037
That was like one of our earliest

03:16:19.120 --> 03:16:22.557
reasoning models using that to find

03:16:22.640 --> 03:16:27.997
diagnosis for people who I had no
no answers from doctors for many many

03:16:28.080 --> 03:16:31.277
years. You know, there's an example
of someone who had spent 20 years with a

03:16:31.360 --> 03:16:34.957
mysterious ailment. Finally, it's
been diagnosed through the use of this

03:16:35.040 --> 03:16:38.477
technology. And if you're like, okay,
you've got models that can do that. They

03:16:38.560 --> 03:16:41.757
can do that. And then it's really
about like, you know, the same like

03:16:41.840 --> 03:16:45.037
distribution and like, you know, can
you get access to an app? You know, to me

03:16:45.120 --> 03:16:47.997
it's it doesn't type check. It's like we
have something fundamentally new. And so

03:16:48.080 --> 03:16:50.717
that's not to say that there won't be
competition. I actually think that there

03:16:50.800 --> 03:16:53.757
will be and it's going to be great for
everyone. But I just think that the ways

03:16:53.840 --> 03:16:56.237
in which you're going to use this
technology, the things it will be

03:16:56.320 --> 03:16:59.757
capable of and what it'll make you
capable of doing are just totally

03:16:59.840 --> 03:17:01.597
different from anything we've seen before.

03:17:01.680 --> 03:17:05.357
>> You know, I was going to ask you um
well, does it mean that you'll have to

03:17:05.440 --> 03:17:09.917
um you know, create your own device
assuming that like my concept is is you

03:17:10.000 --> 03:17:13.277
know that you're going to have to go
through Apple to get to the user. Um

03:17:13.360 --> 03:17:17.517
assuming that's somewhat valid. But the
answer is you already are right. You're

03:17:17.600 --> 03:17:19.837
so OpenAI is working
on a device right now.

03:17:19.920 --> 03:17:23.757
>> It certainly has
been publicly reported.

03:17:23.840 --> 03:17:26.237
>> I I was in your office in December and

03:17:26.320 --> 03:17:30.797
Sam told me that this is happening.
It's it's multiple devices. Um so if you

03:17:30.880 --> 03:17:35.277
think about the way that again you're
going to interface with um with these

03:17:35.360 --> 03:17:40.637
AIs, how does that device play in or
series of devices? Well, look, I I think

03:17:40.720 --> 03:17:42.077
again I I would just step back and say

03:17:42.160 --> 03:17:43.837
that I think this is the beginning of

03:17:43.920 --> 03:17:48.157
something very new and that I think

03:17:48.240 --> 03:17:52.237
about the way that

03:17:52.320 --> 03:17:55.597
I think just say like I think the biggest
shift that has happened in terms

03:17:55.680 --> 03:17:58.397
of interface again it's not even about

03:17:58.480 --> 03:18:00.397
devices and and and things like that.

03:18:00.480 --> 03:18:04.157
really about the shift from conversational
intelligence like kind of

03:18:04.240 --> 03:18:08.157
the chat paradigm where it's like kind
of you have an AI that's personalized

03:18:08.240 --> 03:18:11.997
enough to you that it's worth reading
its output right you ask it a question

03:18:12.080 --> 03:18:16.557
you get an answer it's something that's
useful to you to agents where they're

03:18:16.640 --> 03:18:21.837
capable enough to actually do things
for you like that is a big shift and that

03:18:21.920 --> 03:18:26.317
that implies a difference
in how you want to

03:18:26.400 --> 03:18:27.917
interact and so you kind of are just

03:18:28.000 --> 03:18:30.637
going to want a single agent that has

03:18:30.720 --> 03:18:33.917
access to your context and this will be

03:18:34.000 --> 03:18:35.757
true in personal life. This will be true

03:18:35.840 --> 03:18:37.677
in a business context, right? You

03:18:37.760 --> 03:18:41.277
imagine for example having a uh you know

03:18:41.360 --> 03:18:43.517
imagine you have a PhD in every field

03:18:43.600 --> 03:18:48.397
co-orker you know Nobel prizes multiple
of them and you hire one of these you

03:18:48.480 --> 03:18:51.837
hire a hundred of them and you don't
invite them to any meetings they're not

03:18:51.920 --> 03:18:53.997
going to be very useful and so there's

03:18:54.080 --> 03:18:56.077
something about how do you get context

03:18:56.160 --> 03:19:00.237
into the AI and not just statically but

03:19:00.320 --> 03:19:03.437
dynamically right as context evolves as

03:19:03.520 --> 03:19:07.597
your business processes evolve how
do you have a context layer that is

03:19:07.680 --> 03:19:11.197
accessible to an AI that lets the AI
operate to the extent of that raw

03:19:11.280 --> 03:19:14.477
intelligence. And so finding ways to

03:19:14.560 --> 03:19:19.837
make that AI be accessible so available
in your meetings to make it very

03:19:19.920 --> 03:19:23.997
ergonomics. It's very easy to get access
to. Um I think all of that's going to

03:19:24.080 --> 03:19:25.837
require a rethink. But I think again

03:19:25.920 --> 03:19:29.357
it's just the core for me starts from

03:19:29.440 --> 03:19:33.757
thinking about the agentic form factor
and then working backwards to how do you

03:19:33.840 --> 03:19:37.677
just make this have the context it needs.
And again, the trust is going to

03:19:37.760 --> 03:19:41.117
be such a core part of making
this whole equation work.

03:19:41.200 --> 03:19:43.597
>> So, kind of like
having this this device

03:19:43.680 --> 03:19:46.797
with you at all times and being like,
I need to get that done and it goes and

03:19:46.880 --> 03:19:49.677
does it for you. >> And I
think that that will be part of

03:19:49.760 --> 03:19:51.917
it, but I almost even think if you don't

03:19:52.000 --> 03:19:56.717
have a device like that, it's not like
you're going to be out of the game,

03:19:56.800 --> 03:19:59.677
right? Because it's this a this AI.
It's not because there's one thing there's

03:19:59.760 --> 03:20:02.637
one version of it where you think of it
where it's like the device is the AI and

03:20:02.720 --> 03:20:05.277
you want your phone to be the AI. you
want that, you know, whatever whatever,

03:20:05.360 --> 03:20:07.917
you know, custom device you're you're
thinking about to be the AI, but it's

03:20:08.000 --> 03:20:10.797
not going to be like that. It's going
to be more like an interface, like no more

03:20:10.880 --> 03:20:14.397
than your phone is you, right? It's
an interface to you. It's a way that I can

03:20:14.480 --> 03:20:17.357
sort of, you know, call you up whenever
I need you, uh, whenever I want to ask

03:20:17.440 --> 03:20:20.877
you a question. And there's different
ways of accessing, right? There's like

03:20:20.960 --> 03:20:24.397
synchronous phone call, I can text you,
I can email you. And I think that we're

03:20:24.480 --> 03:20:27.357
going to be much the same with how
we interact with our agents.

03:20:27.440 --> 03:20:30.317
>> Um, there's been some reports
that OpenAI is working on these like

03:20:30.400 --> 03:20:34.317
birectional voice models. I think we've
talked about that in the past that like

03:20:34.400 --> 03:20:37.037
the goal is to have like an AI that you

03:20:37.120 --> 03:20:41.437
can you can speak with and we'll be able
to process that and speak back with you

03:20:41.520 --> 03:20:43.517
in a much more natural way. Can you

03:20:43.600 --> 03:20:45.597
share anything about that?

03:20:45.680 --> 03:20:49.037
>> No. But no, more seriously, um I mean

03:20:49.120 --> 03:20:52.717
look, I think that the that the general
shape of the of the technology like the

03:20:52.800 --> 03:20:57.277
way that like we had we had we've
had voice models um you know kind of a

03:20:57.360 --> 03:21:01.837
really cool voice experience for you
know year and a half two years now. Um,

03:21:01.920 --> 03:21:03.997
you know, we first demoed it back in

03:21:04.080 --> 03:21:06.957
March, April of 2024. Um, brought it to

03:21:07.040 --> 03:21:11.597
market, um, you know, maybe late
that late that year. And the way that it

03:21:11.680 --> 03:21:15.117
works and the way that everyone's
models work is that you basically chain

03:21:15.200 --> 03:21:18.797
together, um, well, the original way
that these things worked was that you

03:21:18.880 --> 03:21:21.677
would chain together a text to or a

03:21:21.760 --> 03:21:25.837
speechtoext model, then you do
a texttoext model, and then you would do a

03:21:25.920 --> 03:21:27.757
texttospech model. Horribleness, right?

03:21:27.840 --> 03:21:31.357
Like these three things chained together.
Um it still has been the case

03:21:31.440 --> 03:21:35.357
that even if you have one unified model
that's able to kind of take in input and

03:21:35.440 --> 03:21:38.797
then you know able to output a response
you still have this problem of turn

03:21:38.880 --> 03:21:42.637
taking right imagine that like we have
this like you cannot overlap you cannot

03:21:42.720 --> 03:21:47.117
interrupt it's just like once you you
you speak to me in a turn and then you

03:21:47.200 --> 03:21:50.317
got to wait for me to finish my whole
response. That is not how human

03:21:50.400 --> 03:21:54.797
conversation works. And so that we we
basically have like a hack where we have

03:21:54.880 --> 03:21:58.717
these models that determine oh it seems
like the turn has ended and oh it seems

03:21:58.800 --> 03:22:01.997
like the turn has started and we're
like why are we talking about turns right

03:22:02.080 --> 03:22:05.757
like turns again are so unnatural.
This is the humans contorting ourselves to

03:22:05.840 --> 03:22:09.277
the machine and its limitations.
And so the obvious thing that you want to

03:22:09.360 --> 03:22:13.757
accomplish is a model in AI that works
much more like you and I do right that's

03:22:13.840 --> 03:22:18.797
able to process input at the same time
it's processing output. And all of that

03:22:18.880 --> 03:22:21.597
is of course something that many people
in this field are trying to to run

03:22:21.680 --> 03:22:23.277
towards. Um I think it's going to be

03:22:23.360 --> 03:22:25.197
very very exciting as you move to these

03:22:25.280 --> 03:22:30.317
natural very human fluid like
conversational interfaces. No one's seen

03:22:30.400 --> 03:22:34.077
anything like it. Like one thing that
that I think about is the the current

03:22:34.160 --> 03:22:37.357
interaction with you know chat GBT voice.
In many ways it's magical right?

03:22:37.440 --> 03:22:41.517
So many people use it on their commute
able to ask all these questions. But it

03:22:41.600 --> 03:22:44.557
also is so frustrating, right? Whenever
it breaks the magic because it's like

03:22:44.640 --> 03:22:48.557
you realize, oh, I want to like add some
followup and it keeps talking over you

03:22:48.640 --> 03:22:53.277
and it didn't it's just like that is
just it doesn't make sense. And so I

03:22:53.360 --> 03:22:57.757
think that part of what we need part of
like the whole point of this AI is to be

03:22:57.840 --> 03:23:01.357
something that you can interact with
interact with fluidly and naturally. And

03:23:01.440 --> 03:23:02.717
by the way, I think it's not just going

03:23:02.800 --> 03:23:07.197
to be about the sort of use case. Like

03:23:07.280 --> 03:23:10.157
we kind of think about the the personal
use case, but it's also really the work

03:23:10.240 --> 03:23:13.597
use case. And I think some of the most
magical experiences that that I've had

03:23:13.680 --> 03:23:18.077
with codecs have been when operating it
through voice. Like many people, we have

03:23:18.160 --> 03:23:20.077
a voice built in. Some people use third

03:23:20.160 --> 03:23:25.117
party apps for it. Um, and that you just
get a very different experience when you

03:23:25.200 --> 03:23:28.877
start to realize that like typing
a quick message to give some feedback

03:23:28.960 --> 03:23:32.397
easy. but like writing out a whole
paragraph and everything you want.

03:23:32.480 --> 03:23:35.277
Horrible. No one wants to do that,
right? You just want to be like saying

03:23:35.360 --> 03:23:38.637
things and you want the real-time
feedback loop and all of that is going

03:23:38.720 --> 03:23:40.797
to happen and it's going to be amazing.

03:23:40.880 --> 03:23:45.517
So, let's talk about model improvement
briefly. Um, so there was a discussion a

03:23:45.600 --> 03:23:47.517
couple years ago that large language

03:23:47.600 --> 03:23:49.597
models were about to hit a wall. Um,

03:23:49.680 --> 03:23:53.437
that was wrong. And um, something you

03:23:53.520 --> 03:23:57.277
know that I'm thinking about is I think
we're all thinking about it is how much

03:23:57.360 --> 03:23:59.437
better can these models get? And when

03:23:59.520 --> 03:24:03.037
will the improvement stop? Any thoughts?

03:24:03.120 --> 03:24:06.957
>> Well, I think that
this is a place where

03:24:07.040 --> 03:24:10.957
when you're kind of building these
models, you get kind of a sense and an

03:24:11.040 --> 03:24:15.917
intuition that I think is harder to get
from the outside because we see all the

03:24:16.000 --> 03:24:19.357
data points and we see also the work
that goes into these improvements. And

03:24:19.440 --> 03:24:21.277
so that there's two parts to the answer.

03:24:21.360 --> 03:24:26.600
One is I think that the
fundamental science is

03:24:35.280 --> 03:24:40.157
one of the most mysterious and important

03:24:40.240 --> 03:24:44.397
just scientific discoveries
and empirical observations that that I can

03:24:44.480 --> 03:24:47.677
that I that I'm aware of that I can
imagine right that we are able to

03:24:47.760 --> 03:24:50.157
actually build these models and that the

03:24:50.240 --> 03:24:55.037
scaling laws continue right that it
just is the case that you can just keep

03:24:55.120 --> 03:24:58.557
training these models more data,

03:24:58.640 --> 03:25:02.397
more compute, better architectures,
and there's a lot of improvements that go

03:25:02.480 --> 03:25:05.917
in. But every time we've kind of run
into a like, h this isn't quite scaling

03:25:06.000 --> 03:25:10.477
the way we expect, it's we have a problem.
We have a bug that our math

03:25:10.560 --> 03:25:13.837
wasn't quite right. That oh,
our implementation isn't isn't isn't quite

03:25:13.920 --> 03:25:18.877
matching the math, whatever the thing is.
And that is, I think, a very

03:25:18.960 --> 03:25:22.477
important thing to to sort of internalize.
And actually if you we've

03:25:22.560 --> 03:25:25.757
done studies where you go back to the
beginning of the field right that neural

03:25:25.840 --> 03:25:28.077
nets themselves were designed in like

03:25:28.160 --> 03:25:31.117
1940s right before computers right as a

03:25:31.200 --> 03:25:35.437
model of maybe this is maybe this how
the brain processes information first

03:25:35.520 --> 03:25:38.957
hardware implementation was 1959
with the perceptron and if you look at

03:25:39.040 --> 03:25:41.837
landmark results in the field that the

03:25:41.920 --> 03:25:46.317
landmark results follow this incredibly
smooth deterministic path of more

03:25:46.400 --> 03:25:50.557
compute being poured into them and so
70 years of people maybe 80 years now of

03:25:50.640 --> 03:25:53.277
people saying this stuff is never going
to work, never going to scale, going to

03:25:53.360 --> 03:25:55.677
hit the wall. Hasn't hit the wall yet.

03:25:55.760 --> 03:25:59.997
There's still no wall in sight. And so
I think that the fundamentals allow it.

03:26:00.080 --> 03:26:02.477
Now the practicality is hard, right?

03:26:02.560 --> 03:26:06.157
Actually building these massive
supercomputers. It's hard. It's

03:26:06.240 --> 03:26:10.157
expensive. It's not easy, right? That we
have teams that just like work so hard

03:26:10.240 --> 03:26:14.477
to solve these incredibly hard technical
problems. We have our own network

03:26:14.560 --> 03:26:18.317
protocol that we've had to design.
um that we have people who look at every

03:26:18.400 --> 03:26:21.837
single layer of the stack that there's
weird wiggles in the graph and you the

03:26:21.920 --> 03:26:24.477
way to think about these neural
nets is that there's like there's no

03:26:24.560 --> 03:26:27.997
abstractions, right? It's almost like
any little piece that's wrong can have a

03:26:28.080 --> 03:26:31.357
ripple effect that only shows up down
there. And so you need people who deeply

03:26:31.440 --> 03:26:34.957
understand all of it. And yet if you get

03:26:35.040 --> 03:26:36.717
the right team together, put the right

03:26:36.800 --> 03:26:39.277
mission in front of people and people do

03:26:39.360 --> 03:26:44.397
that grind, the outcome, it's worth it,
right? And it's achievable and it's

03:26:44.480 --> 03:26:47.837
possible. And so I think that for those
reasons the progress will continue.

03:26:47.920 --> 03:26:51.597
>> So then I'd love to hear your
perspective if if models can basically

03:26:51.680 --> 03:26:53.917
progress much further from where they

03:26:54.000 --> 03:26:57.117
are today. Um let's say let's say open

03:26:57.200 --> 03:27:01.517
AI builds the best model and it
the equivalent of like something with like

03:27:01.600 --> 03:27:04.077
15 PhDs with excellent emotional

03:27:04.160 --> 03:27:06.237
intelligence that doesn't complain and

03:27:06.320 --> 03:27:09.277
goes out and does stuff for you. Um, and

03:27:09.360 --> 03:27:13.117
then the the next model maker will build

03:27:13.200 --> 03:27:15.757
a a less good, but it has 13 PhDs and

03:27:15.840 --> 03:27:20.237
it's like pretty good uh, you know,
EQ and we'll still go and do things for

03:27:20.320 --> 03:27:24.397
you. So, where does the differentiation
come in when you get to that level of

03:27:24.480 --> 03:27:28.637
intelligence? Because we've seen the
model makers kind of move in lock step.

03:27:28.720 --> 03:27:32.237
One makes an advance, the next one comes
in and makes the advance. So, they all

03:27:32.320 --> 03:27:36.717
become that smart. Do they is it
possible to differentiate? Well, I think

03:27:36.800 --> 03:27:41.437
there are several dimensions to the
answer. Um, number one is I do think

03:27:41.520 --> 03:27:45.597
there's a bit of an attractor state
where just like from a business model

03:27:45.680 --> 03:27:49.517
perspective, every provider
sells out all their compute.

03:27:49.600 --> 03:27:52.477
Okay. Right. I think that is just like
the world that we're heading towards

03:27:52.560 --> 03:27:55.677
where there just is not going to be
enough compute to serve all the demand.

03:27:55.760 --> 03:27:58.477
Right? That we're heading to this compute
powered economy that everyone's

03:27:58.560 --> 03:28:02.157
going to be using these models all
the time to be able to accomplish tasks of

03:28:02.240 --> 03:28:06.557
interest. And we just see it. It's like
right now we're talking about compute

03:28:06.640 --> 03:28:08.877
constraints and like like the number of

03:28:08.960 --> 03:28:11.517
people using these agents is like order

03:28:11.600 --> 03:28:13.677
of 10 million 20 million maybe you know

03:28:13.760 --> 03:28:18.797
it's like we're not at planet scale
chatbt is like a billion users right but

03:28:18.880 --> 03:28:21.277
we haven't brought the agentic power
there yet so you're just looking at

03:28:21.360 --> 03:28:25.277
these factors and the depth of usage
is also tiny compared to where we're going

03:28:25.360 --> 03:28:28.957
and so I think that we're just going
to be in a world where even if you have

03:28:29.040 --> 03:28:33.357
different vendors different capability
level open source models all these

03:28:33.440 --> 03:28:37.197
things these neoclouds like I think that
compute is just going to be this scarce

03:28:37.280 --> 03:28:41.997
resource and I think that it's going
to go go to use. Um so to some extent I

03:28:42.080 --> 03:28:45.757
think that the the like is this a good
business to be in and for new entrance

03:28:45.840 --> 03:28:48.557
to come into and things like that.
My answer is actually yes. I think that

03:28:48.640 --> 03:28:53.117
there is like a huge market um that we
are just not going to be able to address

03:28:53.200 --> 03:28:57.677
and we need much more energy momentum
there. Um but a second thing is that it

03:28:57.760 --> 03:28:59.357
also misses the fact that intelligence

03:28:59.440 --> 03:29:02.157
is not a undimensional thing right that

03:29:02.240 --> 03:29:07.597
that If you really zoom in, being good
at different domains is something where

03:29:07.680 --> 03:29:11.037
even if you you have a lot of raw
intelligence, getting good if you've

03:29:11.120 --> 03:29:15.277
never practiced like you've never
actually done a pitch or something like

03:29:15.360 --> 03:29:18.477
you're not going to be good at it your
first time, right? And that there's lots

03:29:18.560 --> 03:29:21.277
of different you've never operated
a spreadsheet, right? You're not going to

03:29:21.360 --> 03:29:25.997
be able to succeed at doing some complex
modeling. And so I think that there is

03:29:26.080 --> 03:29:29.197
something that we have been
internalizing which is that we look

03:29:29.280 --> 03:29:31.597
across different industries
and different domains and we have to

03:29:31.680 --> 03:29:36.797
prioritize. We can't possibly be great
at every single area at once. There is

03:29:36.880 --> 03:29:40.157
definitely a lot of like, hey, you
just get the general intelligence up and

03:29:40.240 --> 03:29:43.837
it'll experience a lot of these things,
but to really become a domain expert, to

03:29:43.920 --> 03:29:47.917
really be that PhD and to really be
something that can help push forward the

03:29:48.000 --> 03:29:50.637
ambition of a field, like that's hard.

03:29:50.720 --> 03:29:53.757
And and by the way, one thing I
also want to say is that I think

03:29:53.840 --> 03:29:57.037
understanding what happens when you
successfully do that. I think that

03:29:57.120 --> 03:29:59.357
having a good mental model of that's
important, which is you look at

03:29:59.440 --> 03:30:01.837
something like rewind to AlphaGo, right?

03:30:01.920 --> 03:30:03.997
you remember move 37, this move that

03:30:04.080 --> 03:30:09.197
like changed people's understanding
of the game and then now more people play

03:30:09.280 --> 03:30:12.717
go than ever, right? And that it
actually inspired people to do even

03:30:12.800 --> 03:30:16.157
more. I think we're just going to see
that. And so I think that the depth is

03:30:16.240 --> 03:30:20.557
never going to stop, right? How deep can
you go on science, right? Like I I think

03:30:20.640 --> 03:30:23.917
that people have thought sometimes that,
hey, we found out all the physics, it's

03:30:24.000 --> 03:30:27.837
all good. We're all done. And I don't
think that that's the future we're

03:30:27.920 --> 03:30:30.557
signed up for. I think we're signed
up for one where we're got to keep find

03:30:30.640 --> 03:30:33.917
every time you unlock one mystery,
>> every every time you solve one mystery

03:30:34.000 --> 03:30:36.877
unlocks like 10 more, right? So, I think
that there's just going to be so much

03:30:36.960 --> 03:30:39.917
more to do and tons of room
for differentiation across different

03:30:40.000 --> 03:30:41.517
companies. >> So, I think
I'm reading you right and

03:30:41.600 --> 03:30:46.557
that your belief is um maybe there's
a way that everybody can scale up these

03:30:46.640 --> 03:30:50.877
models, but ultimately the company
with the most compute is going to win. And

03:30:50.960 --> 03:30:54.877
you know we spoke a couple months ago
and you had mentioned that like you were

03:30:54.960 --> 03:30:59.357
asked internally how much compute should
we buy and you said all of it and they

03:30:59.440 --> 03:31:03.117
said no really how much should we buy
and you said no buy all of it and and

03:31:03.200 --> 03:31:06.077
OpenAI is definitely the leader in in

03:31:06.160 --> 03:31:10.397
buying compute. I mean we see the the
the money going out obviously a lot of

03:31:10.480 --> 03:31:14.237
money coming in through investment
and and now you you're you've built a

03:31:14.320 --> 03:31:17.997
business with customers but there's
a lot of money going out. Do you ever

03:31:18.080 --> 03:31:22.317
wonder, hey, are like,
do you ever wonder maybe we're

03:31:22.400 --> 03:31:24.637
not going to be able to pay all
this money back because it's a brand new

03:31:24.720 --> 03:31:29.117
category? Well, the way that I look
at it is on the fundamentals, right? You

03:31:29.200 --> 03:31:31.357
you need to really look at the fact that

03:31:31.440 --> 03:31:36.717
the way that compute goes is that it's
multiple years out before compute

03:31:36.800 --> 03:31:39.517
actually arrives, right? Depending
on exactly what you're doing. For example,

03:31:39.600 --> 03:31:44.637
we've been investing in our own chip
program now for multiple years and super

03:31:44.720 --> 03:31:46.477
exciting progress like you know we'll

03:31:46.560 --> 03:31:48.877
we'll we'll have more to announce uh

03:31:48.960 --> 03:31:53.997
actually pretty soon. Um but the fact
that we're able to do that is something

03:31:54.080 --> 03:31:56.637
very unique, right? Really thinking
about the full vertical integration of

03:31:56.720 --> 03:31:59.837
the supply chain. And I think that the
world we're heading towards is one where

03:31:59.920 --> 03:32:03.757
again there's just not going to be
enough compute in the world to satisfy

03:32:03.840 --> 03:32:06.637
all the demand. And we see this very
concretely like you look at the

03:32:06.720 --> 03:32:10.237
exponential I mean rewind to the
exponential of chatbt. Um look at the

03:32:10.320 --> 03:32:12.157
exponentials we're on now. Uh you think

03:32:12.240 --> 03:32:17.517
about the problems that we that
we are able to solve. You know,

03:32:17.600 --> 03:32:21.437
it's it's actually kind of interesting
that yeah, we just yesterday announced

03:32:21.520 --> 03:32:24.157
um it was two days ago announced a new

03:32:24.240 --> 03:32:27.837
result in uh basically chemistry and

03:32:27.920 --> 03:32:31.597
being able to synthesize, you know,
new new improved reaction. And all of this

03:32:31.680 --> 03:32:35.037
is without much attention. You know, the
thing I just said of if you go deep in a

03:32:35.120 --> 03:32:38.317
domain, you can really transform it.
And we're not even scratching the surface

03:32:38.400 --> 03:32:42.397
yet. And so the way to think about is
the economy is so massive, right? And we

03:32:42.480 --> 03:32:45.517
see it very concretely in terms of our
own growth, in terms of what people are

03:32:45.600 --> 03:32:49.437
willing to pay and kind of the size and
growth of this whole industry. And so I

03:32:49.520 --> 03:32:54.077
think that the thing that I think about
the most is how do we meet the demand?

03:32:54.160 --> 03:32:57.357
How do you actually have something
that can help support all of the work that

03:32:57.440 --> 03:33:00.397
people want to do in the economy? And I
think that is such a vast thing. I don't

03:33:00.480 --> 03:33:03.997
think any of us have internalized it yet.
Yeah. But if I may, there is a

03:33:04.080 --> 03:33:07.437
price war brewing. I mean, at least
that's according to the report. It's

03:33:07.520 --> 03:33:08.717
great to have you here to talk about it.

03:33:08.800 --> 03:33:10.797
The Wall Street Journal recently had a

03:33:10.880 --> 03:33:13.437
report that an upcoming OpenAI model

03:33:13.520 --> 03:33:16.317
might have significant price cuts. And

03:33:16.400 --> 03:33:19.437
so again, like how can you know if it

03:33:19.520 --> 03:33:22.717
requires so much resources to serve this

03:33:22.800 --> 03:33:27.037
demand and it is growing demand um in
an environment where there might be price

03:33:27.120 --> 03:33:29.277
cuts, how do you make that math work?

03:33:29.360 --> 03:33:33.437
>> Well, again, I look at it from a
different angle. So if you look at the

03:33:33.520 --> 03:33:37.357
whole history of what we've done,
we actually have been increasing the

03:33:37.440 --> 03:33:41.837
intelligence cutting price right for a
fixed amount of intelligence and people

03:33:41.920 --> 03:33:45.037
somehow just like like the Jeffans
paradox just keeps happening and so I

03:33:45.120 --> 03:33:48.397
think frontier intelligence will always
be something that is going to be you

03:33:48.480 --> 03:33:50.957
know it's always going to be
the priciest thing but I think that a year

03:33:51.040 --> 03:33:54.077
from now that level of intelligence
is going to feel pretty mundane and like

03:33:54.160 --> 03:33:58.637
you know going to be much more available.
Um, and I think that that the

03:33:58.720 --> 03:34:02.077
the world that we're in is one where
people are starting to really think

03:34:02.160 --> 03:34:05.037
about value. And it's actually been a

03:34:05.120 --> 03:34:09.757
very interesting shift where over the
past, you know, first quarter, maybe up

03:34:09.840 --> 03:34:13.677
until now, people have just been like
this AI agent stuff, it's all new. We

03:34:13.760 --> 03:34:16.397
need to bring into our enterprise like
we don't want to be left behind. How do

03:34:16.480 --> 03:34:20.237
we be part of this this future? And now
people are like, okay, like let's make

03:34:20.320 --> 03:34:23.277
sure this is actually delivering ROI
and value. And I actually think that's a

03:34:23.360 --> 03:34:26.477
great place to be, right? because people
are asking the right questions. And I

03:34:26.560 --> 03:34:29.437
hear this I I had some customer meetings
today where people were saying exactly

03:34:29.520 --> 03:34:33.277
this. They were like, "How can we have
even just like good spend controls? How

03:34:33.360 --> 03:34:37.757
can we have observability?" And I I
think we literally today just released

03:34:37.840 --> 03:34:40.237
spend controls. So,
you know, it's like Okay.

03:34:40.320 --> 03:34:43.677
>> Exactly. Yeah. We're we are really
investing hard in enterprise readiness

03:34:43.760 --> 03:34:45.757
and the tools that our customers are

03:34:45.840 --> 03:34:47.757
telling us that they need. And I think

03:34:47.840 --> 03:34:53.037
that that for me is the shift that we've
also been going through as a company is

03:34:53.120 --> 03:34:55.677
really not just thinking about hey we're
just going to release models and you

03:34:55.760 --> 03:34:58.957
know have a model really thinking about
the end to end of the business. How do

03:34:59.040 --> 03:35:01.037
we bring this into solving real problems

03:35:01.120 --> 03:35:04.557
for real customers and that is happening

03:35:04.640 --> 03:35:07.117
so quickly across every single industry

03:35:07.200 --> 03:35:12.317
and the number of different companies
that still feel like they're wrapping

03:35:12.400 --> 03:35:15.277
their mind around how to best make use
of these models we're learning at the

03:35:15.360 --> 03:35:19.997
same time. I think it's just so early
in this whole game to me the the like the

03:35:20.080 --> 03:35:23.917
absolute size of the market growing so
quickly, our own revenue ramp growing so

03:35:24.000 --> 03:35:27.277
quickly. I think it's still just like
none of us are anticipating how steep

03:35:27.360 --> 03:35:30.797
that's all going to go.
>> Are you going to cut prices?

03:35:30.880 --> 03:35:35.517
>> I So again, the answer is always yes,
right? But it's about like I think that

03:35:35.600 --> 03:35:38.797
what's going to keep happening is that
we're going to have frontier models. Um

03:35:38.880 --> 03:35:42.157
I don't think there's going to be like
a massive shift in in the short term. I I

03:35:42.240 --> 03:35:44.397
don't think that that is the kind
of thing that's going to happen. But I

03:35:44.480 --> 03:35:49.197
think the thing you should anticipate
is that over a year-long time horizon to

03:35:49.280 --> 03:35:52.637
get to today's level of intelligence
that feels very premier, it's going to

03:35:52.720 --> 03:35:56.397
be much cheaper. But there's going to be
a new thing that is going to be so much

03:35:56.480 --> 03:35:58.957
better and you're going to be like, why
would I ever use this other one, right?

03:35:59.040 --> 03:36:01.517
It's just it's just how
it's always going to be.

03:36:01.600 --> 03:36:03.517
>> So Sadia Nadell has
had some interesting

03:36:03.600 --> 03:36:07.677
tweets and interviews recently.
He recently said the model is becoming a

03:36:07.760 --> 03:36:11.197
commodity and the valuable asset
is company or this might be a paraphrase

03:36:11.280 --> 03:36:13.037
the valuable asset is a company specific

03:36:13.120 --> 03:36:15.597
AI system that continually learns from

03:36:15.680 --> 03:36:20.477
your data. Um what do you think about
that and is it weird to be competing

03:36:20.560 --> 03:36:21.997
with Microsoft now?

03:36:22.080 --> 03:36:25.597
>> Well look I don't
think that there's any

03:36:25.680 --> 03:36:29.517
layer of the stack here that is going
to just kind of be removed from the value

03:36:29.600 --> 03:36:33.277
chain. I think that these things
multiplied together and if you think

03:36:33.360 --> 03:36:36.237
about the most the base layer of compute

03:36:36.320 --> 03:36:40.797
right that is something where
it's just like no compute no AI

03:36:40.880 --> 03:36:43.677
and to some extent you can say oh comput
is commoditized it's just flops who

03:36:43.760 --> 03:36:47.597
cares about it but in reality like you
look at you look at today's chip stocks

03:36:47.680 --> 03:36:50.957
you look at the uh you know people who
are selling compute kind of what what is

03:36:51.040 --> 03:36:54.397
what the market is valuing people
at and they see that there's a fundamental

03:36:54.480 --> 03:36:58.717
asset here that is just so critical and I
think that is because it is a revenue

03:36:58.800 --> 03:37:03.277
center it is something that anyone
who's building AI has to rely on and that

03:37:03.360 --> 03:37:06.477
there's a bunch of very interesting
dynamics in terms of the efficiencies

03:37:06.560 --> 03:37:08.717
that you can squeeze out and the
margins, all these things. But

03:37:08.800 --> 03:37:11.677
fundamentally, even though it's like
you can kind of squint at it and say it's

03:37:11.760 --> 03:37:15.837
commoditized, it's not. It's it's it's
not that that the value goes away. It's

03:37:15.920 --> 03:37:19.357
not that the margins go away. It's like
something the market will reward because

03:37:19.440 --> 03:37:23.197
it has fundamental value and that the
importance of it's going to go up over

03:37:23.280 --> 03:37:26.397
time. You can see that with some of the
prices that people are paying for H100s,

03:37:26.480 --> 03:37:30.877
right? hoppers are, you know, kind of,
you know, not not obsolete, right? But

03:37:30.960 --> 03:37:36.477
they're they're they're a a previous gen
chip and u nor in any normal situation

03:37:36.560 --> 03:37:38.957
where you're not totally supply
constrained, no one would be buying

03:37:39.040 --> 03:37:42.557
them. But instead, the market prices
are up relative to to where they were

03:37:42.640 --> 03:37:45.517
before. So there's this inversion that's
happening and again I think it's going

03:37:45.600 --> 03:37:48.717
to keep happening where because

03:37:48.800 --> 03:37:53.117
everyone has this avalanche of demand
that you're going to see prices and

03:37:53.200 --> 03:37:57.117
margins and all of these things AC
continuing to increase at various levels

03:37:57.200 --> 03:37:59.277
of the stack. I think the same kind of

03:37:59.360 --> 03:38:01.997
applies for models where the models

03:38:02.080 --> 03:38:07.197
themselves are also again they're not
>> there's there's a lot of competition

03:38:07.280 --> 03:38:09.997
there and I think that's very good.
I think it's good for the enterprise. I

03:38:10.080 --> 03:38:11.917
think it's good for customers consumers.

03:38:12.000 --> 03:38:16.077
Um, but I think that there's a lot
of areas where, for example, our models

03:38:16.160 --> 03:38:19.917
have always been the sort of smartest
ones, right? The ones that are able to

03:38:20.000 --> 03:38:22.957
solve these incredibly hard problems.
I think we're just starting to reach a

03:38:23.040 --> 03:38:26.637
phase where you're going to see the
transformative impact from that, right?

03:38:26.720 --> 03:38:31.357
It's like if we're really able to speed
up science through models, the smarter

03:38:31.440 --> 03:38:33.277
the model, the faster it's going to go.

03:38:33.360 --> 03:38:36.237
And it's very, very different
from a model that has a conversational

03:38:36.320 --> 03:38:39.597
interface that you're able to, you know,
is able to book your travel, right? Or

03:38:39.680 --> 03:38:42.877
organize your calendar. So that's also
a dimension I think we're going to do a

03:38:42.960 --> 03:38:46.957
very good job in. But I'm just saying
it's it's a different area. Um and then

03:38:47.040 --> 03:38:48.957
I think that the question of well how do

03:38:49.040 --> 03:38:53.997
you actually connect the intelligence
to your own customers right to real value

03:38:54.080 --> 03:38:58.157
to you have all these enterprises
that have built incredible businesses in

03:38:58.240 --> 03:39:01.677
different domains and it's a huge thing
and it's not something where if you

03:39:01.760 --> 03:39:04.557
don't have domain expertise that you're
just going to be able to do right and

03:39:04.640 --> 03:39:07.437
part of it is that you need you think
about regulated industries you think

03:39:07.520 --> 03:39:11.517
about any area where there's like you
know think about education where you

03:39:11.600 --> 03:39:14.397
have a parent you have a teacher you
have a student you have these different

03:39:14.480 --> 03:39:18.797
parties that need to interact in very
thoughtful ways for all of these areas,

03:39:18.880 --> 03:39:23.357
all of these domains that there's a lot
of value to be built by being in that

03:39:23.440 --> 03:39:27.117
area and thinking about how the workflow
should work, how these models should be

03:39:27.200 --> 03:39:30.877
orchestrated. And so I I really think
that there's more than enough to go

03:39:30.960 --> 03:39:34.957
around. And I think that we have to work
together as a whole ecosystem in order

03:39:35.040 --> 03:39:37.917
to deliver the kind of value that I
think is possible from these systems.

03:39:38.000 --> 03:39:42.157
>> Okay, just to go back to the SIA point
one more time, uh he's called models a

03:39:42.240 --> 03:39:45.997
commodity. He's trying to build his
own frontier intelligence. He's telling

03:39:46.080 --> 03:39:49.197
potentially your customers, hey, you got
to come work with us because we're going

03:39:49.280 --> 03:39:53.437
to help build these loops that will
learn from your data. Um, he's got

03:39:53.520 --> 03:39:55.997
access to your IP, I think, till 2032.

03:39:56.080 --> 03:39:59.277
So, how does it make you feel
to hear this coming from SAI?

03:39:59.360 --> 03:40:01.197
>> Look, I think that the most important

03:40:01.280 --> 03:40:06.477
thing that is happening right now is
the usage of AI in the economy to really

03:40:06.560 --> 03:40:10.077
transform the economy and to uplift
everyone. And so, I think that that is

03:40:10.160 --> 03:40:11.437
something that I'm really focused on.

03:40:11.520 --> 03:40:15.197
And the more that people are trying
to make that happen, I think that that's

03:40:15.280 --> 03:40:16.877
better for everyone.

03:40:16.960 --> 03:40:20.557
>> Um, GPT 5.6 is rumored
to be on its way.

03:40:20.640 --> 03:40:24.477
Supposed to be this just
a Twitter rumor, but I'm going

03:40:24.560 --> 03:40:26.797
to read it to you. Um, always the best

03:40:26.880 --> 03:40:29.437
rumors. Three times cheaper than Fable

03:40:29.520 --> 03:40:32.557
up to 1.5 million token context. Uh,

03:40:32.640 --> 03:40:35.757
stronger agentic coding workflows. Um,

03:40:35.840 --> 03:40:37.517
how much of that is true? What should we

03:40:37.600 --> 03:40:41.757
expect for GPT 5.6?

03:40:41.840 --> 03:40:46.957
I mean, look, you should always expect
better, faster, smarter, the whole

03:40:47.040 --> 03:40:52.440
thing. >> So, everything confirmed.

03:40:52.720 --> 03:40:56.717
>> Definitely believe everything
you read on Twitter. Yeah, maybe not.

03:40:56.800 --> 03:40:57.997
>> That has actually been a source of

03:40:58.080 --> 03:41:01.437
problems in my personal life. Um, okay.

03:41:01.520 --> 03:41:04.957
So, uh, I want to end on health. Um,
you brought it up a couple times. You

03:41:05.040 --> 03:41:08.717
actually had a question in the audience
about it earlier. Um, you know,

03:41:08.800 --> 03:41:10.797
sometimes there's a story and you read

03:41:10.880 --> 03:41:14.157
it and you say to yourself, I know this

03:41:14.240 --> 03:41:16.397
person is speaking to the media and I

03:41:16.480 --> 03:41:18.477
know that what they're saying sounds

03:41:18.560 --> 03:41:21.437
like maybe it's true. Um, but there's

03:41:21.520 --> 03:41:25.677
something wrong with the story and we're
not going to see more of it. Um, and

03:41:25.760 --> 03:41:27.677
I've read a couple of those recently.

03:41:27.760 --> 03:41:29.757
Um, one is I think is it your friend the

03:41:29.840 --> 03:41:34.317
GitLab CEO um, Sid Sberandage? He had um

03:41:34.400 --> 03:41:37.917
he he got cancer and used uh he got all

03:41:38.000 --> 03:41:40.637
the diagnostic testing uh he could have.

03:41:40.720 --> 03:41:42.797
So just went out and tested like crazy

03:41:42.880 --> 03:41:45.677
and fed that data into ChachiPT with the

03:41:45.760 --> 03:41:50.077
assistance of some people who had built
purpose-built application for it and was

03:41:50.160 --> 03:41:52.637
able I don't know if cure is the right

03:41:52.720 --> 03:41:57.197
word but to beat back the cancer to a
degree. There was also this dog Rosie

03:41:57.280 --> 03:42:02.077
the dog in Australia. You guys heard
of Rosie? like the craziest story where

03:42:02.160 --> 03:42:03.677
this guy I'm gonna get some detail

03:42:03.760 --> 03:42:07.917
wrong, but a guy um biopsied his dog

03:42:08.000 --> 03:42:11.197
which had cancer um ran the mutations

03:42:11.280 --> 03:42:14.157
across across Alphafold and then was

03:42:14.240 --> 03:42:16.717
able to design an mRNA vaccine that he

03:42:16.800 --> 03:42:19.517
injected into the dog um with the

03:42:19.600 --> 03:42:24.077
assistance of chatbots to build this
thing which ended up being able to jump

03:42:24.160 --> 03:42:26.397
over tables again and the tumor shrunk.

03:42:26.480 --> 03:42:29.357
Um, when we think about the future of AI

03:42:29.440 --> 03:42:32.317
and health, um, help us sort out the

03:42:32.400 --> 03:42:36.637
truth with this question. Are these
a couple of outliers that made good

03:42:36.720 --> 03:42:40.397
headlines, but there was something about
the story we weren't hearing, or is this

03:42:40.480 --> 03:42:42.477
going to become standard in the future?

03:42:42.560 --> 03:42:44.237
>> Absolutely going to become standard.

03:42:44.320 --> 03:42:46.477
Absolutely. And it's I I personally have

03:42:46.560 --> 03:42:49.677
a number of friends who have done very

03:42:49.760 --> 03:42:53.677
similar things of get the data right

03:42:53.760 --> 03:42:55.997
your health diagnostics

03:42:56.080 --> 03:43:01.357
and use codecs right use these models
to get insights from them and I think that

03:43:01.440 --> 03:43:02.957
there are many people like I think that

03:43:03.040 --> 03:43:06.397
there's about 230 million people each

03:43:06.480 --> 03:43:10.637
week who use chat GBT for health queries

03:43:10.720 --> 03:43:13.037
right and that's been that's that's like

03:43:13.120 --> 03:43:15.597
a staggering scale Right. And this is pe

03:43:15.680 --> 03:43:17.677
these are people sometimes you upload a

03:43:17.760 --> 03:43:22.797
scan. Sometimes you have doctors who are
telling you conflicting information. And

03:43:22.880 --> 03:43:26.237
I think that we've been in a world
where patients are not empowered, right?

03:43:26.320 --> 03:43:27.997
Patients have to be the doctor, right?

03:43:28.080 --> 03:43:31.917
You're the decider. You are accountable,
right? You know, doctor makes a mistake

03:43:32.000 --> 03:43:35.917
and you're going to be paying the price
for the rest of your life. Like it's

03:43:36.000 --> 03:43:40.077
just it's a very different kind of
incentive. And this is very personal for

03:43:40.160 --> 03:43:42.717
me. you know, my wife has a number of
health conditions and I think that we've

03:43:42.800 --> 03:43:45.917
just been we've not like I don't even
know how we'd be able to manage many of

03:43:46.000 --> 03:43:49.757
her conditions right now without the use
of of chat. And I think we're just at

03:43:49.840 --> 03:43:51.037
the beginning of this journey, right?

03:43:51.120 --> 03:43:54.877
That I think that the degree to which

03:43:54.960 --> 03:43:59.437
even if you have the best medical team,
the best acts as the best best experts,

03:43:59.520 --> 03:44:04.317
there's only so much that can be done,
right? You think about the things that

03:44:04.400 --> 03:44:06.957
are just outside of the reach
of humanity or even just sometimes it's

03:44:07.040 --> 03:44:11.037
like someone didn't even read the chart,
right? kind of missed a detail. All of

03:44:11.120 --> 03:44:14.717
that we should be able to improve
massively through these tools. And so I

03:44:14.800 --> 03:44:17.517
think that the personalized medicine and

03:44:17.600 --> 03:44:22.637
sometimes it's going to be about drugs
and drug discovery that are of you know

03:44:22.720 --> 03:44:27.117
for for mass market but sometimes it'll
be even for the kind of NF1 things like

03:44:27.200 --> 03:44:29.517
the the disease um diagnoses that I

03:44:29.600 --> 03:44:31.597
mentioned earlier today. Sometimes it

03:44:31.680 --> 03:44:34.477
will be for just like trying to

03:44:34.560 --> 03:44:38.957
understand conditions and trying to come
up with with new potential therapeutics.

03:44:39.040 --> 03:44:42.717
All of that we're seeing it happening
right now in front of our eyes. It's not

03:44:42.800 --> 03:44:46.797
theoretical. It's really happening. And so
one of the most I think it's like one

03:44:46.880 --> 03:44:50.717
of the most astounding possibilities
of AI is how much it can improve our

03:44:50.800 --> 03:44:53.037
health. And you think about the ripple

03:44:53.120 --> 03:44:54.957
effects of the system, right? Where so

03:44:55.040 --> 03:44:58.557
much spending on the health care system

03:44:58.640 --> 03:45:00.477
happens right now. that's a massive part

03:45:00.560 --> 03:45:02.637
of the economy and that if you're

03:45:02.720 --> 03:45:07.837
actually able to help people prevent
issues, right, to to get ahead of

03:45:07.920 --> 03:45:11.517
potential, you know,
health health problems, that's

03:45:11.600 --> 03:45:14.877
something that actually then alleviates
a lot of burden, a lot of strain. And

03:45:14.960 --> 03:45:17.837
we're in a world where doctors are
burned out, nurses are burned out, like

03:45:17.920 --> 03:45:21.517
there's like a real crisis that's
happening in front of us. And I think AI

03:45:21.600 --> 03:45:23.997
will be able to help with all of that.

03:45:24.080 --> 03:45:25.997
Like we have that potential if we deploy

03:45:26.080 --> 03:45:29.277
it and use it wisely and well. And so I

03:45:29.360 --> 03:45:32.477
think that applying AI to medicine, like

03:45:32.560 --> 03:45:36.317
that's something that I is really
a personal motivation for me in in

03:45:36.400 --> 03:45:38.637
thinking about this whole journey of of
what we're building, what we're trying

03:45:38.720 --> 03:45:42.237
to do with open AI. And I'm hopeful that
we as a as a world and a community can

03:45:42.320 --> 03:45:44.877
make the most of that. Let's hope. I

03:45:44.960 --> 03:45:47.437
think we will. I'm very very confident.

03:45:47.520 --> 03:45:51.837
>> Greg, thank you so much. Thank you.

03:45:51.920 --> 03:45:55.640
Thank you. Thank you so much.

03:46:00.320 --> 03:46:05.400
Oh my god. Thank you everyone.
You have a good time today.

03:46:08.560 --> 03:46:12.637
>> Thank you.
Should we do it again next year?

03:46:12.720 --> 03:46:17.597
>> Going to come. >> All right.
U let me uh give you a a

03:46:17.680 --> 03:46:20.077
perspective about what we're doing
for the rest of the day. First of all, if

03:46:20.160 --> 03:46:21.757
you're backstage, Cali and John, can you

03:46:21.840 --> 03:46:24.077
come out here for a second?

03:46:24.160 --> 03:46:27.997
Um, as I as I bring them out, as I bring

03:46:28.080 --> 03:46:32.077
them out, let me let me uh explain
that we're going to go from here to the

03:46:32.160 --> 03:46:33.997
rooftop. We have some drinks and food

03:46:34.080 --> 03:46:36.477
ready for you guys. We also have a space

03:46:36.560 --> 03:46:39.677
directly outside. Uh, so that is a place

03:46:39.760 --> 03:46:44.477
that you can hang out. Um,
figure out what's what's busier and try

03:46:44.560 --> 03:46:48.237
to spread yourself a little bit so we'll
have room. Uh, John, come up here. Cali,

03:46:48.320 --> 03:46:50.477
come up here. Um, guys, this would not

03:46:50.560 --> 03:46:56.280
have happened without John
Bashu and Calli Win. Um,

03:46:59.600 --> 03:47:01.437
>> thank you, Cali.
>> Thank you very much.

03:47:01.520 --> 03:47:04.477
>> So, uh, true story, I announced this

03:47:04.560 --> 03:47:08.717
show on a podcast live and I thought
we would get a lot of people signing up.

03:47:08.800 --> 03:47:12.397
One person signed up and I called I
called both of them and I said, you

03:47:12.480 --> 03:47:15.517
know, it was great brainstorming this idea
with you, but we're not going to do

03:47:15.600 --> 03:47:20.317
it. Um, so we'll see you next year. And
both of them said, let's do it and let's

03:47:20.400 --> 03:47:22.317
do it at the Commonwealth Club. and we

03:47:22.400 --> 03:47:27.480
did it. So, thank you
guys and thank you all.

03:47:29.120 --> 03:47:33.037
Before we leave, I have to thank our
sponsors at PWC, Dallas, and Arita. You

03:47:33.120 --> 03:47:38.120
guys are amazing.
And thanks to the PWC crew.

03:47:38.640 --> 03:47:42.317
Um, just a great group and amazing
people to work with. And of course,

03:47:42.400 --> 03:47:43.597
thank you to the Commonwealth Club here

03:47:43.680 --> 03:47:46.717
for collaborating with us on this event.

03:47:46.800 --> 03:47:51.117
Wow. uh six years after I showed up here
and there was literally nobody in the

03:47:51.200 --> 03:47:53.117
audience and we kind of have social

03:47:53.200 --> 03:47:55.037
distanced and probably would have gone

03:47:55.120 --> 03:47:58.397
to jail if London Breed found out. Um

03:47:58.480 --> 03:48:02.637
we're all here together and it means
the world to me. So, thank you all. I'm

03:48:02.720 --> 03:48:05.997
getting emotional here. Um thank you
for helping me live the dream and let's go

03:48:06.080 --> 03:48:11.240
party. >> Thank you.