AI Accelerators Explained
The families of AI silicon, on the same work.
An interactive comparison of the families of AI chips, from NVIDIA and AMD GPUs and TPUs to wafer-scale, compiler-scheduled and FPGA designs, running the same decoder.
- NVIDIA H100 SXMFP81 × 80 GB HBM
- Google TPU7x (Ironwood)FP81 × 192 GiB HBM
- Google TPU v6e (Trillium)BF165 × 32 GB HBM
- Cerebras WSE-3 (CS-3)BF164 × 44 GB SRAM
- NVIDIA Groq 3 LPU (announced)FP8142 × 500 MB SRAM
- GroqChipINT8308 × 230 MB SRAM
How this is checkedEvery specification carries its source and the date it was read, vendor claims are never used as inputs, and the model is checked against published capacity, throughput and latency figures. How it is tested →
The finding: once weights are in SRAM, a layer costs tens of microseconds
Groq's chain of chips, Cerebras's wafers and SambaNova's SN40L are built very differently, yet each pays 12.5 µs to 38.2 µs per layer beyond what reading its weights costs. On a wafer the weight reads take nanoseconds; the per-layer critical path (synchronisation, collectives, crossing the mesh) is the token. A single per-layer floor, fitted on three of the machines, predicts the fourth within 1.84× where the bandwidth bound alone misses by up to 771×. Chapter 10 animates it. The GPUs, with prefill counted, reach about a third of their tensor peak in MLPerf: 33% for eight H100s and 31% for four GB200s.
What you'll learn
Every AI chip multiplies activations by weights. What separates a GPU from a TPU, a wafer-scale engine or an LPU is where the weights live, who decides when data moves, and how big the engines are. An H100 reads its HBM at 3.35 TB/s; a Cerebras wafer reads its SRAM at 21 PB/s, but holds 44 GB, so a 70B model needs 4 wafers where it needs one GPU in 8-bit form.
Each chapter runs the same decoder on one family, with an animation whose every frame comes from a small model tested against a Python reference. Every specification carries its source and the date it was read, announced parts are marked, vendor claims are quoted but never used as inputs, and the model is checked against published figures (its validation band).
Every figure the model uses, with its source and access date: the specifications.
Built by a hardware engineer
The author designs hardware: the RTL, SoC and signal-integrity work behind this track is public too.
- A transformer decoder in RTL: SystemVerilog with 84 passing tests (a Python golden model and cocotb), synthesised in Vivado 2025.2 for an Artix-7: its 4 × 4 systolic array is 455 LUTs, 901 flip-flops and 16 DSPs at 156.8 MHz (the README's synthesis table).
- The decoder, module by module: A walkthrough of every RTL module, the verification and the synthesis numbers.
- RTL challenges for LLM accelerators: Systolic array, mixed-precision MAC, softmax pipeline, KV cache and more, each simulated (two run inside Systolic Arrays Explained).
- FPGA design challenges: FPGA architecture, synthesis, timing closure and the Vivado and Quartus flows, with RTL exercises.
- A five-stage RISC-V SoC: BRV32P: a pipelined RV32IMC core with caches and an AXI4-Lite fabric in Verilog, and a MESI cache controller (Cache_Controller_MESI).
- Modern SoC design: Slide decks from chiplets and advanced packaging to on-chip interconnects and silicon implementation.
- Equalisation in high-speed serial links: One 28 GBd backplane channel end to end: S-parameters, CTLE, FFE and DFE, a noise budget.
- Signal integrity: Seventeen decks on getting a signal from one chip to another intact, every number computed by a model.
- All the hardware decks: RISC-V, Arm, AMBA, simulators, photonics and more.
Part of the companion sites' Hardware track, with GPU Kernels Explained (how a GPU executes the maths), Systolic Arrays Explained (inside a matrix engine) and AI Racks Explained (the machines these chips go into). The number formats they compute in: Numerics. How this site was built, and how to check it: about.