This transcript comes from automatic captions and may contain small errors.
0:00 IntroWe have built machines that can write, translate, code, and reason.
But open one up and you do not find rules or explanations.
You find billions of numbers.
Nobody wrote those numbers by hand.
They were learned.
So, even the people who build these models cannot fully say how they work.
Interpretability is the science of looking inside to understand what a model is actually doing.
And in the last few years, it has started to work.
A neural network is layers of simple units, often called neurons, connected by weights.
When a model reads text, each neuron lights up a little or a lot.
These patterns of activity carry everything the model knows about the text so far.
0:45 Neurons and superpositionThe natural idea was to study one neuron at a time and ask what it means.
But many neurons turned out to be confusing.
In one small language model, a single neuron fired for academic citations, English dialogue, web requests, and Korean text.
In 2022, researchers at Anthropic proposed an explanation called superposition.
A model needs to represent far more concepts than it has neurons.
So, it packs them in as overlapping directions, the way you might fit many arrows into a small space.
Each concept is spread across many neurons, and each neuron takes part in many concepts.
That is why single neurons look like a mess.
This was shown clearly in small toy models.
1:30 Sparse autoencodersIn large models, it remains a well-supported idea rather than a proof.
If the concepts are packed together, can we unpack them?
A sparse autoencoder is a second, simpler network trained to rewrite the model's activity as a short list of concepts out of a much larger dictionary, where only a few are active at a time.
In 2023, this worked on a small model with only 512 neurons.
Out came tens of thousands of features, each much easier to name.
DNA, a computer encoding called base 64, Arabic script.
As the researchers put it, "Just 512 neurons can represent tens of thousands of features."
2:15 Features in large modelsIn 2024, the same approach was scaled up to a production model, Claude 3 Sonnet, with dictionaries of up to 34 million features.
Many features were multilingual, firing for the same idea in different languages.
Some were multimodal.
The Golden Gate Bridge feature fired for descriptions of the bridge and for pictures of it.
Some features related to safety, such as deception, flattery, or dangerous information.
The researchers cautioned that finding such a feature does not by itself mean a model is dangerous.
Others followed.
OpenAI trained a 16 million feature dictionary on GPT-4, and Google DeepMind released open dictionaries for its Gemma models, so anyone can explore them.
3:00 Steering and circuitsFeatures are not just labels.
You can turn them up.
For 24 hours in May 2024, Anthropic let the public talk to Golden Gate Claude, a version with the bridge feature amplified.
Asked about its physical form, it answered that it was the Golden Gate Bridge itself.
It brought up the bridge in almost every conversation.
It was playful, but it showed something important.
These features are not just correlations.
They cause what the model says.
In 2025, researchers began tracing how features connect, step by step, into something like a circuit diagram of a single answer.
Ask, "What is the capital of the state containing Dallas?"
3:45 Planning and hallucinationInside the model, you can see an intermediate step where it represents Texas before it answers Austin.
Ask it to write a rhyming poem and before starting a line, it already has candidate rhymes in mind, planning ahead rather than improvising word by word.
And for mental arithmetic, it runs two paths in parallel, one rough estimate and one precise last digit.
Yet when you ask how it did it, it describes carrying the one the way it was taught in school.
Tracing also shed light on hallucinations.
By default, the model leans toward saying it cannot answer.
When it recognizes something it knows, like a famous athlete, a feature switches that default off.
4:30 Languages share circuitsA hallucination can happen when something feels familiar, but the model does not actually know it.
The switch fires by mistake.
One finding matters especially for Europe.
When the model thinks about a concept, like small or opposite, much of the circuitry is shared across languages, with language-specific parts at the input and the output.
And a larger model shared more than twice the proportion of its features between languages compared with a smaller one.
That suggests something learned in one language can carry over to others, which is good news for every language with less training data.
Why does this matter for safety?
Because looking inside can reveal things that behavior alone does not.
5:15 Safety usesIn one experiment, a model was secretly trained with a hidden objective and four teams tried to find it.
Three succeeded.
The team using interpretability tools found it fastest in about an hour and a half, though searching the training data also worked.
Researchers have found single directions inside models that control refusal and persona vectors for traits like flattery, which can be monitored and even used during training to stop a model picking up the trait.
And in 2025, for the first time, Anthropic safety testing of a new model before release included techniques from interpretability.
It is important to be honest about how far this has come.
6:00 Limits todayEven in the best studies, these methods capture only a small fraction of what the model computes on short prompts after hours of careful human work.
The dictionaries leave dark matter unexplained, and some interventions can look like they work for the wrong reasons.
Google DeepMind even reported that simpler probes sometimes beat sparse autoencoders on a practical safety task.
Think of it as the first maps of a new continent, real, useful, and still mostly blank.
The goal many researchers share is something like an MRI for AI.
A scan that could reliably reveal problems inside a model before it is deployed.
Anthropic's chief executive has called it a race between interpretability and model intelligence, and set a goal for 2027.
6:45 ClosingIt is an aspiration, not an achievement.
The encouraging part is that this science is increasingly open.
Dictionaries, circuit tracing tools, and interactive maps are published, and on platforms like Neuroipedia, anyone can explore the features inside real models.
We trust what we understand.
Interpretability is how we can come to understand the machines we build one feature at a time.