Public transcript

The Real Surprise of the Last 3 Years in AI – Dario Amodei

By: Dwarkesh Clips
Length: 6 min
Language: English
Text from: the video's captions
Transcribed: 12 October 2026

Read the full transcript Open on YouTube

Notes in English.

Summary

Dario Amodei reflects on the last three years of AI progress, saying the underlying exponential has developed roughly as he expected, from smart high school student to college student to PhD-level capabilities, with code going beyond that. The biggest surprise for him is the lack of public recognition of how close we are to the end of the exponential. He restates his 2017

Key points

Dario Amodei says the underlying technology exponential has gone roughly as he expected over the last 3 years, plus or minus a year or two.
The most surprising thing to him has been the lack of public recognition of how close we are to the end of the exponential.
He has held the same hypothesis since 2017, written in a doc called the big blob of compute hypothesis, similar to Rich Sutton's bitter lesson.
The hypothesis lists about seven things that matter, including raw compute, quantity of data, quality and distribution of data, training duration, and an objective function that can scale to the moon.
Pre-training scaling laws have continued to give gains, and the same log-linear scaling is now being seen with RL on top of pre-training.
RL scaling has been shown publicly by other companies on math contests like AIME, and Anthropic sees the same scaling across a wide variety of RL tasks.

Questions it answers

00:00The last 3 years

What has been the biggest update in AI over the last 3 years?

Amodei says the underlying technology exponential has gone roughly as he expected, plus or minus a year or two, with models marching from smart high school student to smart college student to PhD and professional level, and beyond that in code. The biggest surprise is the lack of public recognition of how close we are to the end of the exponential.

The exponential has tracked his expectations, though he wouldn't have predicted the specific direction of code.
The frontier is a little bit uneven.
He finds it wild that people keep discussing the same tired political issues near the end of the exponential.

01:17What is the scaling hypothesis now?

With RL scaling and no publicly known scaling law, what is the current scaling hypothesis?

Amodei says he has the same hypothesis he had back in 2017, written in a doc called the big blob of compute hypothesis. It wasn't specific to language models and was written when GPT-1 had just come out, alongside work in robotics, reasoning, and RL scaling like AlphaGo, Dota, and AlphaStar.

The hypothesis predates language model scaling laws.
It is essentially the same as Rich Sutton's bitter lesson, which came out a couple years later.
It says clever techniques don't matter much; only a few things do.

02:17The seven factors

What are the few things that actually matter according to the hypothesis?

He listed seven things: raw compute, quantity of data, quality and broad distribution of data, how long you train, an objective function that can scale to the moon (pre-training or RL, with objective rewards like math and coding and subjective rewards like RL from human feedback), and two factors around normalization or conditioning for numerical stability so the compute flows in a laminar way.

Objective rewards appear in math and coding; subjective rewards in RL from human feedback.
Numerical stability keeps the big blob of compute flowing in a laminar way.
He still holds this hypothesis and hasn't seen much that contradicts it.

04:24RL scaling

Is the same scaling now being seen with RL as with pre-training?

Yes. Pre-training continues to give gains, and now an RL phase on top of pre-training shows the same log-linear scaling. Other companies have published results showing math contest performance like AIME is log-linear in training time, and Anthropic sees the same across a wide variety of RL tasks.

We now see a pre-training phase followed by an RL phase.
RL scaling is log-linear in training duration.
The scaling holds beyond math contests, across many RL tasks.

Notes

Overview

Dario Amodei discusses what has and hasn't surprised him about AI progress over the last 3 years.
The technology exponential has gone roughly as expected, with plus or minus a year or two on specific timelines.

The Biggest Surprise

The most surprising thing is the lack of public recognition of how close we are to the end of the exponential.
People inside and outside the AI bubble keep discussing the same tired old hot button political issues.

The Big Blob of Compute Hypothesis

Written by Amodei in 2017, around when GPT-1 had just come out.
It was a general document, not specific to language models — it also covered robotics, separate reasoning work, and RL scaling like AlphaGo, Dota at OpenAI, and AlphaStar at DeepMind.
Similar to Rich Sutton's bitter lesson, published a couple years later.
Core idea: clever techniques and new methods don't matter very much; only a few things matter, and he listed seven of them.

The Seven Factors

How much raw compute you have.
The quantity of data.
The quality and distribution of data — it needs to be a broad distribution.
How long you train for.
An objective function that can scale to the moon — the pre-training objective is one; the RL objective of reaching a goal is another, with objective rewards (math, coding) and subjective rewards (RL from human feedback).
Two factors around normalization or conditioning — numerical stability so the big blob of compute flows in a laminar way.

Scaling Today

Amodei still holds this hypothesis and hasn't seen much that contradicts it.
Pre-training scaling laws have continued, and it's widely reported that pre-training is continuing to give gains.
The same scaling is now seen with RL on top of pre-training: other companies have published results showing performance on math contests like AIME is log-linear in how long the model is trained.
Anthropic sees the same RL scaling across a wide variety of tasks, not just math contests.

Text from the video's own captions. Summary and notes written by AI from the transcript, so check anything important against the recording.