About this site

AI Accelerators Explained compares the families of AI chips on one piece of work: a Llama decoder's prefill and decode steps. The question is the same for every family: where do the weights and activations live, what moves on each step, and who decides when it moves. It sits in the companion sites' Hardware track beside GPU Kernels Explained, Systolic Arrays Explained and AI Racks Explained, and leans on Numerics Explained for number formats. Chapters link the matching slides of the NVIDIA GPU and Google TPU series. It picks one or two representative parts per family; it is not a catalogue, and naming a product is not an endorsement.

The model

accel_model/ is a teaching model, not a simulator of any product. For each part it computes where a model's weights and KV cache must live (HBM when the part has it, SRAM when it does not) and the fewest chips that hold them; the bound on one decode step and one prefill, the larger of bytes over bandwidth and flops over the peak rate; and the chip-to-chip latency the layout forces, where the vendor gives it. Families differ explicitly, not through fudge factors: GPUs and systolic chips split every layer across their chips (tensor layout); wafers and LPUs cut the model at layer boundaries into a pipeline, so one token's time is the sum of the stages' times plus a hop between each, and a static schedule reads every weight from SRAM exactly once. It also tiles a matmul on a weight-stationary array (chapter 3) and counts defects on a wafer (chapter 4).

• Every bound assumes the full stated bandwidth and the dense peak rate; kernel launches, synchronisation and achieved-versus-peak efficiency are not modelled.
• Where a vendor gives only a sparse peak (Cerebras), the model uses it and marks the result “sparse”: a compute term from it is a floor.
• Latency between chips is included only where the sources give it (Groq's 722 ns per hop, the CS-4's 2 µs between wafers); NVLink, ICI and NeuronLink latencies are not disclosed, so the GPU and systolic bounds have no collective time.
• Chip counts are capacity only: weights plus the KV cache, with no room for activations, runtime buffers or spares.
• Energy per token is the chips' stated power over the bound's throughput; where a vendor gives only system power (CS-3) it is a system share, and where it gives none it is not shown. The B200's is Blackwell's maximum board power (TGP), the figure AI Racks Explained uses too.
• The defect density and spare share in chapter 4 are illustrative.

How figures are sourced

Every figure is a field with a value, a status and a source, read on 2026-10-15 (the full list). Vendor datasheets, documentation and product pages come first, then papers (checked by arXiv ID) and benchmark logs. A figure only a third party reports is marked reported. Where the sources used do not give a figure, the site says “not disclosed” rather than estimating it. Announced parts (the CS-4's WSE-3 Turbo, the Groq 3 LPU) are labelled, and a retired one (the first-generation GroqChip) is kept because published results used it. Vendor claims (“up to N×”) are quoted and attributed, never used as model inputs.

Validation band

The model is checked against 9 published figures it does not take as inputs, of three kinds, and shows one more outside the band. Capacity: how many chips a model needs. Cerebras's “as few as four systems” for a 16-bit 70B model and one system for 20B match the model exactly; Groq's Llama 2 70B demonstration used 1.91 times the model's minimum, which is a floor (no room for activations, and whole nodes of eight). Throughput: MLPerf Offline runs against a compute ceiling that counts everything a sample costs (the prefill of a 222-token mean prompt, measured from the benchmark's dataset, the 294 output tokens and attention): eight H100s reached 33% of it and four GB200s (MLPerf v5.1, FP4) 31%. Latency: one user's published tokens per second against the bound. The SN40L, reading its weights from HBM, reached 58% of it. The SRAM designs reached 0.13% to 8.2%. That miss is the model's main limit, and the chapters say so: once the weights are in SRAM, bandwidth stops being what limits one user, and the per-layer latency the sources do not break down takes over. The published speeds imply 17.3 µs to 38.2 µs per layer beyond the bound. The model now adds that as one fitted term, a floor per layer of 24 µs on average: fitted on three systems and tested on the fourth, it predicts every SRAM system within 1.84× in time per token (the bound alone misses by up to 771×). It is four points, so the term is an order of magnitude, not a law. The row marked speculative (the SN40L paper's 70B figure, 2.25 times the bound) is kept out of the band: speculative decoding emits several tokens per read of the weights, which a one-token-per-step bound does not model.

How it is checked

• Reference tests. tests/python/test_accel_model.py checks every field has a known source, status and date, that every source is cited, the parameter counts against the model names, the placement sums, the decode terms, the tile model against its per-tile steps, the defect generator, the chain against the decode bound, and the validation checks.
• Exact parity. The TypeScript port, src/lib/accel/model.ts, repeats every function with the same operations in the same order; scripts/make_fixtures.py writes the reference's results over every parameter the widgets offer, and the unit tests require the port to reproduce all of them exactly (the two exponentials of the yield model to 1e-12). CI fails if the fixtures, the site data or results.md are out of date.
• Animations from the model. Every animation draws a sequence of states the model computes; a frame is a pure function of one state. The end-to-end tests set chosen frames of every animation and require the caption to match the caption built from the Python reference's state for that frame.
• Numbers in the prose are printed from the model when the page is built, not typed.

The animations

Every animation has play and pause, step back and forward, a scrub bar, speeds from 0.25× to 4× and reset; with the animation focused, Space plays or pauses and the arrow keys step. Each step has a one-line caption, also announced to screen readers. With reduce motion set in your system, nothing plays by itself. Animations pause when scrolled out of view. Colours come from Okabe and Ito's colour-blind-safe palette, the same in light and dark mode and on every companion site: off-chip memory (HBM) purple, on-chip SRAM green, tiered memory sky blue, compute in the memory array orange, compute yellow; waiting (a chip-to-chip hop, a defect) is always a hatched pattern or a mark as well as a warning colour.

Source

The code, the model and the tests are on GitHub (MIT licence). The design system is copied from the companion sites; the README records where each piece came from.