/learn
The families of AI silicon
One chapter per family of chips, each opening with an animation. Every frame is computed by this site's model (checked against its Python reference), and every specification links its source. Toggle layers (Concept / Maths / Code) inside any chapter to choose how deep to go. What happens inside one GPU is on GPU Kernels Explained; for slides on the same hardware, see the NVIDIA GPU and Google TPU series.
Where the weights live, who decides when data moves, and how big the engines are: a map of the families of AI chips, and the decoder layer every chapter runs.
HBM, tensor cores and the roofline: why one decode step on a GPU is a memory read, and how batching turns it into a compute problem.
TPUs and Trainium: large weight-stationary arrays fed from HBM by a compiler, and why decode needs a batch as big as the array to fill it.
A whole wafer as one chip: the weights in on-chip SRAM, models split across wafers at layer boundaries, and how defects are survived rather than avoided.
No caches and no DRAM: a static schedule over SRAM-only chips chained into a pipeline, why one token comes back fast, and what holding a model in SRAM costs.
SambaNova's RDU and Graphcore's IPU: a layer fused into one on-chip pipeline instead of a kernel per operation, three tiers of memory, and bulk-synchronous tiles.
Tenstorrent's Blackhole: many small cores with their own SRAM on a network-on-chip, GDDR6 beside them, and Ethernet built into the chip to join cards.
A few watts and a phone's memory: why one user's decode is a bandwidth problem, how narrower weights help, and where FPGAs win on latency and custom numerics.
Weights that never move: a decoder layer on phase-change crossbars, and what it costs in capacity, ADCs and precision.
- 10Choosing
The same decoder model on every family, side by side: decode and prefill bounds, energy, chips and utilisation, where each wins and loses, and what they are converging on.