Renaissance LabsLearn

Models · How a model is trained

Post-training a European model, stage by stage

What worked, what broke and how we found out

11 minAdvanced10 chapters

The film follows the post-training of Open Euro LLM, a 9-billion-parameter model, on the Leonardo supercomputer. It covers instruction tuning, a thinking mode, tool use, preference training and reinforcement learning, and shows what each stage broke, how it was caught by evaluation, and how it was fixed, including by merging weights.

What to take away

  • Post-training is a pipeline of stages, and the same evaluations run between stages so each step's gains and damage are visible.
  • The film reports that unclosed tool calls came from nearly identical output weights on new tag tokens, fixed by refitting eight rows.
  • Data teaches exactly what it contains: the speaker found that 99% of strict-format training answers ignored the format.
  • Reinforcement learning can only reinforce what the model already does sometimes, so rewards need partial credit the model can climb.
  • Averaging the weights of two models from the same parent can restore a skill lost in a later stage.

Go deeper

Read the transcript152 lines, click any line to jump

This transcript comes from automatic captions and may contain small errors.

Keep watching