AI accelerators by FP16 performance
The silicon agents ultimately run on, top accelerators by FP16 throughput (Epoch AI, CC-BY). Bandwidth & power in the full dataset.
Why are AI chips so expensive?
Because supply is tight and they are hard to make. Frontier accelerators are built on the newest process node at a handful of factories, each one carries scarce high-bandwidth memory, and the packaging that bonds the two together is itself a bottleneck. Demand from every cloud provider competes for the same output.
What is the difference between a GPU and a TPU?
A GPU is a general-purpose parallel processor that was adapted for AI. A TPU is a chip built only for the tensor maths that neural networks use. In practice the choice comes down to throughput, on-chip memory, and whether your model and serving stack run on that silicon at all.
What is FP16 and why is it used to compare AI chips?
FP16 is half-precision floating point, a 16-bit number format. AI models mostly run in this kind of reduced precision, so FP16 operations per second tracks real workload speed better than a headline peak-FLOP figure does. The figures ranked here come from an open accelerator dataset (Epoch AI).
Does Nvidia make its own chips?
No. Nvidia designs its accelerators and TSMC manufactures them. Almost every AI chip company is fabless, meaning it owns the design but not the factory, which is why foundry capacity limits how many chips can exist.
How much memory does an AI chip need?
Enough to hold the model, or it has to shuffle data in and out and slows right down. Frontier accelerators ship with tens to hundreds of gigabytes of high-bandwidth memory on the package. Bandwidth matters as much as capacity, because the compute units sit idle waiting for memory to feed them.