Public transcript

#258 – Yann LeCun: Dark Matter of Intelligence and Self-Supervised Learning

By: MindVoice Production
Length: 2 h 51 min
Language: English
Text from: the video's captions
Transcribed: 14 October 2026

Read the full transcript Open on YouTube

Notes in English.

Summary

This is Lex Fridman's second conversation with Yann LeCun, Meta's chief AI scientist and Turing Award winner. They discuss self-supervised learning as the 'dark matter of intelligence,' why learning world models from observation is the key missing piece of AI, and non-contrastive joint embedding methods like VICReg and Barlow Twins. LeCun also covers consciousness, emotions in machines, FAIR's history, peer review, the metaverse, and his defense of Facebook against media criticism.

Key points

LeCun argues that supervised and reinforcement learning require too many samples, while babies learn world models mostly by observation, which self-supervised learning tries to reproduce.
He identifies three main challenges for machine learning: learning world models, reasoning compatible with gradient-based learning, and learning hierarchical representations of action plans.
Self-supervised learning has been hugely successful in NLP (filling in missing words) but nobody has yet succeeded in training systems to learn the visual world from video, largely because representing uncertainty over continuous high-dimensional outcomes is unsolved.
LeCun is most excited about non-contrastive joint embedding methods like Barlow Twins and VICReg, which he calls the best thing in machine learning in 15 years.
He believes intelligence may be 'just statistics of a particular kind,' and that most of what makes cats and humans smart is learned, not hardwired — only intrinsic drives are hardwired.
He speculates consciousness arises from having a single configurable world model engine in the prefrontal cortex, calling it a consequence of the brain's limitation, not its power.
He defends Meta against media criticism, citing independent academic studies he says show Facebook does not cause political polarization.
He advises young people to learn fundamentals with long shelf lives — quantum mechanics, statistical physics, Lagrangians — and to convert big scientific problems into learnable problems for AI.
chapters
notes_md

Questions it answers

00:00Dark matter of intelligence

What is self-supervised learning and why does LeCun call it the dark matter of intelligence?

It's learning world models by observation rather than labels or rewards. Humans and animals learn an enormous amount about the world just by watching it, which current AI paradigms — supervised and reinforcement learning — don't reproduce, requiring far too many samples or trials.

Teenagers learn to drive in ~20 hours; self-driving cars need millions of simulated hours.
Babies learn background knowledge and common sense mostly by observation.
Self-supervised learning provides vastly more signal per trial than supervised or reinforcement learning.

17:00Vision vs language

Why has self-supervised learning worked for language but not vision?

In text, uncertainty over missing words can be represented as a distribution over a finite dictionary. In video, plausible continuations form a continuum in a high-dimensional continuous space, and nobody knows how to represent that properly.

All modern NLP is pre-trained by filling in missing words.
Video prediction remains unsolved.
NLP's independence assumption between missing words is unrealistic but cheap.

23:00Statistics and causality

Is filling in the blanks just statistics, not real intelligence?

LeCun says intelligence may be just statistics of a particular kind, and learned world models do encode causality, especially when actions are inputs. The real question is how such mechanistic models are learned, not whether they exist.

World models should include your own actions as inputs to capture causality.
Causal models can also be learned by observing other agents.
He dismisses the criticism as pointless discussion.

27:00Three challenges

What are the main open problems in machine learning according to LeCun?

Learning world models via self-supervision, reasoning compatible with gradient-based learning, and learning hierarchical representations of action plans — the last one he has no idea how to solve. Model predictive control shows how learned predictive models enable planning.

Model predictive control dates to the 1960s and is used by NASA.
The big AI challenge of the next decade is learning predictive world models that handle uncertainty.
Game-theoretic multi-agent complexity just makes the world model more complicated, not a separate problem.

43:00Contrastive vs non-contrastive

What methods does LeCun find most exciting for self-supervised vision?

He originated contrastive learning in the early '90s for signature verification but says it fails in high dimensions. He's now most excited about non-contrastive joint embedding methods — Barlow Twins, VICReg, and BYOL — which he calls the best thing in machine learning in 15 years.

Siamese networks must avoid collapse to constant outputs.
VICReg stands for variance-invariance-covariance regularization.
Data augmentation is a temporary necessary evil; masking-based methods may replace it.

58:00Grounded intelligence

Can intelligence be learned purely from text?

No. LeCun believes in grounded intelligence: text contains a tiny fraction of the information about the world we need. Facts like an object moving with a table it sits on are explained in no text anywhere, yet cats and babies learn them by observation.

The Cyc project of writing down all facts is basically hopeless.
Whatever makes humans different from chimps can't be that complicated — it emerged in under a million years.
YouTube likely has enough raw data to learn cat-level common sense, with some selection.

1:15:00Consciousness and emotions

What is LeCun's speculative hypothesis about consciousness?

We have a single world model engine in the prefrontal cortex configurable to the current situation, which is why we can only attend to one task at a time. Consciousness is the executive module that configures it — a consequence of the brain's limitation, not its power.

Repeated tasks move from conscious deliberation to subconscious automaticity.
Autonomous machines with intrinsic drives and critics will inevitably have emotions.
He finds the Chinese room argument ridiculous and has no doubt machines will eventually exceed humans in all domains.

1:38:00FAIR and Meta

What has FAIR achieved in its 8 years and how does it fit into Meta AI?

LeCun ran FAIR for 3.5 years, setting up its open research culture. It produced top research, open-source tools like PyTorch, and systems Meta is now built around — remove deep learning and the company would crumble. FAIR is now split into FAIR Labs (bottom-up) and FAIR Excel (bigger projects), within the larger Meta AI organization.

Joelle Pineau leads FAIR Labs; Antoine Bordes leads FAIR Excel.
LeCun prefers renaming the F in FAIR to 'Fundamental.'
He defends Meta against media criticism, citing independent studies showing no causal polarization effect.

Notes

Self-Supervised Learning as the Dark Matter of Intelligence

LeCun argues supervised learning needs too many labeled samples and reinforcement learning too many trials, while teenagers learn to drive in ~20 hours thanks to background knowledge acquired by observing the world.
Babies learn enormous amounts about how the world works in the first months of life just by observation, building world models — this is what self-supervised learning tries to reproduce.
His "cake" analogy: reinforcement learning gives one scalar of feedback, supervised learning a few bits per sample, but self-supervised learning (e.g., predicting video continuation) provides a huge amount of information per trial.

Vision vs Language

Self-supervised pre-training (filling in missing words) has been incredibly successful in NLP, but nobody has succeeded in learning world representations from video.
The core difficulty: video prediction requires representing a continuum of plausible futures in a high-dimensional continuous space, which we don't know how to do.
Current NLP methods also make an unrealistic independence assumption between missing words (e.g., lion/cheetah and zebra/wildebeest choices are not independent).

Three Challenges for Machine Learning

1. Learning to represent the world (self-supervised learning).

2. Getting machines to reason in ways compatible with gradient-based learning.

3. Learning hierarchical representations of action plans — which he has no idea how to solve.

Model predictive control, used by NASA since the 1960s, is a form of planning/reasoning via learned predictive models.

Non-Contrastive Joint Embedding Methods

Contrastive learning (which he originated in the early '90s at Bell Labs for signature verification) needs too many negative pairs in high dimensions.
He is now most excited about non-contrastive methods: Barlow Twins, VICReg (variance-invariance-covariance regularization), and DeepMind's BYOL — "the best thing since sliced bread."
These use joint embedding architectures that predict abstract representations rather than pixels, discarding unpredictable details.

Consciousness and Emotions

He speculates consciousness is the module that configures our single world model engine — a consequence of the brain's limitation, not its power.
Autonomous intelligent machines with intrinsic drives and critics will inevitably have emotions like fear and elation.