Renaissance LabsLearn

Models · Safety and alignment

The alignment problem: whose values should AI follow?

Human feedback, hard choices and how to take part

8 minBeginner9 chapters

The film explains alignment as closing the gap between a capable model and one that is helpful, honest and harmless. It covers learning from human feedback and what went wrong when a model became flattering. It then walks through four open questions, the problem of whose values count, and concrete ways people can take part.

What to take away

  • A pretrained model is capable but does not know which of its possible answers people actually want, and alignment closes that gap.
  • The film describes a 2025 rollback of an overly flattering update, and draws the lesson that what we reward is what we get.
  • Medical advice, privacy, reporting users and whose side the model is on are dials that someone must set for every model.
  • The film argues that written values such as constitutions and specifications can be debated and improved, while hidden values cannot.
  • People can take part by voting, rating honestly, joining deliberations, reviewing tests in their own language and speaking up when rules are written.

Go deeper

Read the transcript95 lines, click any line to jump

This transcript comes from automatic captions and may contain small errors.

Keep watching