0:00 IntroRight now, the best open-weights model scores forty-six on the Artificial Analysis Intelligence Index.
The best closed model scores fifty-eight.
Ten open models now sit between twenty-five and forty-six.
In this video we open them up: the architecture, the data, the training recipes, and what seems to matter most for performance.
0:25 How strength is measuredFirst, how do we measure strength?
The Artificial Analysis index averages ten evaluations in four equal groups: agents, coding, general capability, and scientific reasoning.
It is hard on purpose, and it gets harder with each version. Nemotron 3 Ultra scored nearly forty-eight at launch in June, and twenty-three on today's version.
So scores only compare inside one version. We use version four point three point two.
The second scoreboard is Arena, where people vote between two anonymous answers. That measures human preference, not agent skill.
1:06 The top tenHere are the ten strongest open models on the index.
Xiaomi's MiMo V2.6 Pro leads with forty-six.
DeepSeek V4.1 Flash and Z dot AI's GLM 5.3 follow, at forty-five.
StepFun's Step 5 Preview scores forty-four, and Meta's Muse Spark Open thirty-nine.
Then Qwen 3.8 at twenty-seven billion parameters, K2 Horizon, Beam, NVIDIA's Nemotron, and Inkling.
Notice the sizes. Most of these models have hundreds of billions of parameters, but only a small share is active at a time.
1:47 MiMo and DeepSeekLet's meet the leaders, starting with MiMo V2.6 Pro from Xiaomi.
Seventy layers, a hidden width of six thousand one hundred forty-four, and three hundred eighty-four experts with eight active for each token.
It was pretrained on thirty trillion tokens, with context stretched to one million, and its hidden weights switch to a Muon variant in mid-training.
Then comes agentic reinforcement learning on coding, tool use, visual design and cybersecurity.
Next, DeepSeek V4. Sixty-one layers, one point six trillion parameters, forty-nine billion active, and thirty-three trillion training tokens.
Its Flash sibling, V4.1, has five hundred fifty-two billion parameters and forty-five trillion tokens, with a causal encoder-decoder split: eight billion active reading the prompt, sixteen billion writing the answer.
2:45 GLM and StepGLM comes from Z dot AI. GLM 5 has seven hundred forty-four billion parameters, forty billion active, and eighty layers.
It adds DeepSeek-style sparse attention after pretraining, with only twenty billion tokens of adaptation.
Pretraining used twenty-eight point five trillion tokens, then context extension to two hundred thousand, and about a hundred sixty billion tokens of software engineering data.
GLM 5.3, the version on the leaderboard, reportedly keeps that base and improves through post-training alone.
StepFun's Step 5 Preview has ninety-two layers, deep and narrow, with six hundred billion parameters and twenty-seven billion active.
Its sparse attention merges eight tokens per block, and covers nearly the same tokens as a finer scheme: seventy-eight percent against seventy-nine.
3:46 Muse Spark, Kimi, QwenMeta's Muse Spark Open is a classic big mixture of experts: eight hundred sixteen billion parameters, forty-two active, sixty-two layers, and two hundred fifty-six experts per layer.
It takes text, images, audio and video, and its thinking effort cannot be turned off.
Kimi K3 from Moonshot is the largest of all: two point eight trillion parameters over ninety-three layers.
It mixes linear and full attention three to one, adds attention residuals between blocks, and claims two and a half times the scaling efficiency of its predecessor.
Qwen 3.8 at twenty-seven billion is the odd one out: a dense model, not a mixture of experts.
It still uses the hybrid trick: sixteen blocks, each with three Gated DeltaNet layers and one gated attention layer.
4:41 The rest of the listLower on the list, the recipes look familiar.
K2 Horizon from MBZUAI has three hundred seventy-five billion parameters, and reportedly trained on twenty trillion tokens, about half of them synthetic.
Reflection's Beam has five hundred one billion parameters and fifty-two layers, and has not disclosed its training data.
Thinking Machines' Inkling has nine hundred seventy-five billion parameters and saw forty-five trillion tokens, yet scores twenty-five.
And NVIDIA's best entry, Nemotron 3.5 Super VL, has one hundred twenty billion parameters, twelve active, and scores twenty-six.
The next sections look at what separates the top of the list from the bottom.
5:30 Sparse expertsThey all share one idea: a sparse mixture of experts.
Each layer holds hundreds of expert networks, but a router picks only a few for every token.
MiMo Pro has one point zero two trillion parameters, and uses forty-two billion for each token.
DeepSeek V4.1 Flash has five hundred fifty-two billion, and activates just sixteen billion.
Kimi K3 goes furthest: eight hundred ninety-six routed experts, sixteen active, and two point eight trillion parameters in total.
So the model stores a lot of knowledge, but each token costs far less compute.
6:12 The KV cache problemThe next problem is memory. Every token the model reads leaves behind a key and a value in the KV cache.
At a million tokens of context, that cache becomes the bottleneck, and long reasoning chains are what agents are made of.
DeepSeek V4 attacks it with two kinds of attention.
Compressed sparse attention squeezes every four tokens into one entry, then lets each query pick only the top few.
Heavily compressed attention squeezes every one hundred twenty-eight tokens into one, and keeps dense attention over those.
At one million tokens, V4 Pro needs twenty-seven percent of the compute and ten percent of the cache of DeepSeek V3.2.
V4.1 Flash pushes the cache down to eight hundred ninety bytes per token in fast memory.
7:06 Hybrid attentionOther labs reached for the same goal differently: mix cheap layers with a few expensive ones.
MiMo uses sliding-window attention, where a token sees only the last one hundred twenty-eight, in sixty of its seventy layers, and global attention in ten.
Step 5 Preview alternates three gated sliding-window layers with one sparse full-attention layer.
Kimi K3 and GLM 5.3 Flash use linear attention called Kimi Delta Attention, three layers for every full attention layer.
Qwen 3.8 does the same with Gated DeltaNet.
The pattern is converging: a cheap-to-expensive ratio near three to one, or six to one.
7:54 Residual stream fixesA quieter change is in the residual stream, the highway that carries information between layers.
Hyper-connections widen that highway into several parallel lanes that mix with each other.
That is powerful, but it can blow up when stacked deep.
DeepSeek's fix, manifold-constrained hyper-connections, forces the mixing matrix to be doubly stochastic: every row and every column sums to one.
A few Sinkhorn steps enforce it, so the signal can no longer grow layer after layer.
GLM 5.3 Flash adopts it too, with four streams.
8:34 Training stabilityTraining a trillion-parameter model is mostly an exercise in not blowing up.
DeepSeek, MiMo and Kimi train most weights with Muon, which orthogonalizes each update with a few Newton-Schulz steps.
DeepSeek found that loss spikes came from outliers in the expert layers, with the routing making them worse.
Their fixes: compute routing from slightly older parameters, and clamp the SwiGLU activations to a fixed range.
StepFun keeps hidden states and norm scales in thirty-two bit precision, even when training in FP8.
And MiMo found that during reinforcement learning the router drifts and experts collapse, so they freeze it.
9:19 DataNow the data. Almost every lab trains on between thirty and forty-eight trillion tokens.
MiMo Pro thirty trillion, DeepSeek V4 Pro thirty-three, GLM 5 twenty-eight point five, Inkling forty-five, and MiMo Flash forty-eight.
The mixes are only partly disclosed: web, books, papers, code, math, and now images, audio, and video.
DeepSeek filters out templated and auto-generated web text to avoid model collapse, and leans on long documents like papers and technical reports.
Agentic data, such as ten million software issue and pull-request pairs in GLM 5, is added in mid-training.
Context grows in stages. DeepSeek goes from four thousand tokens to sixteen thousand, sixty-four thousand, and finally a million.
10:14 Post-trainingThen post-training, where most of today's gap is made.
The recipe is converging: a short supervised stage, then reinforcement learning on tasks that can be checked.
DeepSeek trains separate specialists for math, code, agents and instruction following, each with GRPO.
Then a single student learns from all of them with on-policy distillation, matching each teacher's token probabilities on its own samples.
MiMo, Kimi and GLM use close variants: multi-teacher distillation, or distilling earlier stages into later ones.
The old mixed reinforcement learning stage is gone.
10:58 Reinforcement learning at scaleReinforcement learning at this scale is an industrial process.
MiMo Pro runs fifteen hundred sixty-eight prompts times sixteen rollouts every step: twenty-five thousand trajectories, and up to three point seven billion tokens.
Sixty-eight percent of its tasks are coding and agents. The reported cost for Pro is two point six million dollars.
Almost half of that is just generating rollouts.
On the DeepSWE benchmark, that reinforcement learning lifts MiMo Pro from fifty-eight to seventy-three.
GLM 5 trains on over ten thousand verifiable software environments. StepFun says it builds a million.
And everyone fights reward hacking. MiMo logs fewer than two percent confirmed hacks.
11:50 What predicts the scoreSo what actually predicts a model's score?
Plot the ten models by active parameters, and the relationship is weak.
DeepSeek Flash activates sixteen billion and scores forty-five. Inkling activates forty-one billion and scores twenty-five.
Training tokens do not sort them either. Inkling saw forty-five trillion. MiMo Pro saw thirty.
What separates them is post-training, especially agentic reinforcement learning, and the efficiency that makes that training affordable.
With ten data points and no ablations, this is our reading, not a proof.
12:30 The Arena viewArena tells a different story.
In the top fifteen on Arena, only one open model appears: MiMo V2.6 Pro, ranked eleventh at fourteen ninety-one.
The leader, Gemini 4 Argon, is thirty-five Elo points higher.
The index rewards agentic work. Human voters reward style, helpfulness and polish, which is plausibly where closed labs still lead.
These numbers come from an aggregator, so treat the exact order with care.
13:04 ClosingHere is the picture.
Sparse experts make models cheap to run. Hybrid attention and compressed caches make long contexts affordable. Stability tricks make trillion-parameter training possible.
And all of that exists to buy one thing: more, and better, reinforcement learning on verifiable agentic tasks.
That is where the points come from today.
For anyone building on open models, the base matters, but the post-training recipe matters more.
Every number here comes from papers and model cards, listed in the research file.