Renaissance LabsLearn

Models · The hardware

Where the silicon ends

What a GPU limits and what software can change

18 minAdvanced14 chapters

Hopper has no 4-bit tensor cores and Blackwell does. The film separates what is fixed in silicon from what software can still change. It covers how a GPU runs work, the roofline model, dense tensor formats for NVIDIA and AMD, 4-bit weights on Hopper, native NVFP4 training and the CUDA and ROCm software layer.

What to take away

  • A tensor core is fixed hardware, reached only through instructions that list the number formats it supports.
  • Decoding one token at a time is memory bound, while large-batch training is compute bound, and a number format affects each differently.
  • Four-bit weights on Hopper cut memory traffic about four times, but the multiply still runs in 16 bits.
  • On Blackwell B200, dense 4-bit peak is twice 8-bit, and on B300 it is three times.
  • Software can change how many bytes move and how close you get to the roofline, but it cannot raise the roofline.

Go deeper

Read the transcript61 lines, click any line to jump

Sources

Where a source has no link, it is cited from the film's research notes. Estimates and unverified claims are labelled in the film.

Keep watching