# AGI Might Not Need Continual Learning – Dario Amodei *Dwarkesh Clips* · https://www.youtube.com/watch?v=Z0x99Uu4rJc ## Summary Dario Amodei argues that a million-token context windows plus other existing-paradigm capabilities may already be enough to reach powerful AI and generate trillions of dollars in revenue, even without continual learning. He believes continual learning—models learning on the job—may be solved within the next year or two. When pressed on why context lengths have stalled and degrade at scale, he frames long context as an engineering and inference problem rather than a fundamental research barrier, noting the distinction between the context length a model is trained at versus served at. ## Key points - A million tokens of context can represent days of human learning, which Amodei sees as a large amount of information. - Amodei believes the existing paradigm may be enough to get 'countries of genesis in a data center' and generate trillions of dollars of revenue. - He thinks continual learning is being worked on and has a good chance of being solved within the next year or two. - Amodei says the national security and safety implications he wrote about could happen even without continual learning. - One idea for continual learning is simply making context longer, which he calls an engineering problem of training and serving at longer context. - Serving long context requires storing the entire KV cache and juggling GPU memory, which is difficult but not a research problem. - Amodei suggests degradation at long context may come from training at a small context length while serving at a longer one. - Context length grew from about 2,000 tokens (GPT-3) to 128K (GPT-4 Turbo) between 2020 and 2023, but has stayed in the same ballpark since. ## Notes ### Context Windows and Human Learning - Amodei notes that a million tokens is a lot—it can represent days of human learning, since reading a million words would take a person days or weeks. - He believes these capabilities within the existing paradigm may be enough to get 'the countries of genesis in a data center,' though he is not certain and there may be gaps. - Even as things are, he believes this is enough to generate trillions of dollars of revenue. ### Continual Learning - Continual learning refers to a single model learning on the job. - Amodei says Anthropic is working on it and thinks there is a good chance it is solved within the next year or two. - He argues you get most of the way there without it: the trillions-of-dollars-a-year market, national security implications, and safety implications he wrote about in his essay could happen without continual learning. ### Longer Context as a Path - One idea is simply making context longer—Amodei says nothing prevents longer context from working. - It requires training at longer context and learning to serve it at inference; both are engineering problems that Anthropic and presumably others are working on. ### Why Context Lengths Have Stalled - The interviewer notes context grew from roughly 2,000 tokens (GPT-3) to 128K (GPT-4 Turbo) between 2020 and 2023, but has stayed in the same ballpark since. - People report qualitative degradation when models consider very long context. - Amodei frames serving long context as an engineering and inference problem: you must store the entire KV cache and juggle memory across GPUs. - He admits he can no longer follow the low-level details, though he knew them in the GPT-3 era; things have changed with MoE models. ### Training vs. Serving Context - Amodei distinguishes between the context length you train at and the context length you serve at. - If you train at a small context length and serve at a long one, you may get these degradations—it is better than nothing, but degraded. ## Transcript [00:00] A million tokens is a lot. That's that's, you know, that can be days of human learning, right? You know, if you think about the model, you know, you know, kind of read reading reading a million words, you know, it it you know, it takes me how long would it take me to read a million? I mean, you know, like days or weeks at least. Um so, you have these two things and and I think these two these two things within the existing paradigm may just be enough to get you the countries of genesis in a data center. I don't know for sure, but I think they're going to get you a large fraction of it. There may be gaps, but I I certainly think just as things are, this I believe is enough to generate trillions of dollars of revenue. That's [00:37] one. That's all one. Two is this idea of continual learning, this idea of a single model learning on the job. Um I think we're working on that, too, and I think there's a good chance that in the next year or two we also make we also solve that. Um I I again, I I I I you know, I think you get most of the way there without it. I think the trillions of dollars of of, you know, the the I think the trillions of dollars a year market may be all of the national security implications and the safety implications that I wrote about in adolescence of technology can happen without it, but I I I also think we and I imagine others are working on it, and I think there's a good chance that that, you know, that we get there [01:23] within the next year or two. There are a bunch of ideas. I won't go into all of them in detail, but um you know, one is just make the context longer. There's There's nothing preventing longer context from working. You just have to train at longer context and then learn to to serve them at inference, and both of those are engineering problems that we are working on and that I would assume others are working on as well. >> Yeah, so this context length increase It seemed like there was a period from 2020 to 2023 where from GBD3 to GBD4 Turbo there was an increase from like 2,000 context length to 128K. I feel like for the next for the two-ish years since then, we've been in the same-ish ballpark. [01:58] >> Yeah. And when model context lines get much longer than that, people report qualitative degradation in the ability of the model to consider that full context. Um So, I'm curious what you're internally seeing that makes you think like, "Oh, 10 million contacts, 100 million contacts." To get human-like 6-month learning, billion billion contacts. >> a research problem. This is a This is an engineering and inference problem, right? If you want to serve long context, you have to like store your entire KV cache. You have to, you know, uh you know, it's it's it's it's difficult to store all of the memory in the GPUs, to juggle the memory around. I don't even know the detail, you know, at at at this point this is at a level of detail [02:37] that that that that I'm no longer able to follow, although, you know, I I knew it in the GPT-3 era of like, you know, these are the weights, these are the activations you have to store. Um but, you know, you know, these days the whole thing has flipped cuz we have MoE models and and and kind of all of that. But, um and and this degradation you're talking about, like again, without getting too specific, like a question I would ask is like there's two things. There's the context length you train at, and there's a context length that you serve at. If you train at a small context length and then try to serve at a long context length, like maybe you get these degradations. It's better than nothing, [03:12] you might still offer it, but you get these degradations. And maybe it's harder to train at a long context length. Yeah, I so, you know, there's there's a lot. >> If you enjoyed this clip, you can watch the full episode here and subscribe for more clips. Thanks.