Around 2013, Google did the math that scared Mountain View: if every Android user used Search’s voice command for three minutes a day on x86 CPUs, the company would have to double its data centers just for inference. Consumer GPUs were still adapted gaming cards. The answer was an ASIC dedicated to linear algebra: the TPU (Tensor Processing Unit). This text maps the silicon — systolic MXU, bfloat16, OCS — and the generations through TPU7x Ironwood, generally available since March 31, 2026. The side-by-side GPU vs TPU comparison is in the post GPU vs TPU.
Why an ASIC, not a GPU
A CPU spends transistors on branch prediction, cache, and decoding. A GPU spends them on general-purpose SIMT. The TPU dedicates its die to one operation: MAC (multiply-accumulate). Almost every Google service — Search, Translate, Photos, YouTube, Cloud — and the training of Transformer, BERT, AlphaFold, PaLM, and Gemini passed through this silicon.
HBM <=> VMEM (SRAM on-die)
│
┌────────┴────────┐
v v
MXU sistólica VPU
(MatMul pura) (softmax, GELU, LayerNorm)
128×128 MACs
(16.384 células)
In newer generations, SparseCore handles sparse embeddings (ads, YouTube, Search). v6e and Ironwood use an MXU of 256×256.
Systolic array: data does not return to HBM
In a conventional processor, each intermediate MAC reads and writes registers/cache — the memory wall from the post about graphics cards. In the MXU:
- Weight-stationary: the layer’s weights leave HBM once and park in the cell.
- Systole: the input vector enters from the left and moves one column per cycle.
- Accumulation: each cell multiplies input × local weight, adds what came from above, and pushes it downward.
- Output: the resulting matrix’s column flows out through the bottom.
The number pulses from neighbor to neighbor. Energy becomes computation, not transit. It is the opposite of the bloated KV cache that DeepSeek’s MLA tries to shrink on GPUs.
The side effect shows up in the efficiency calculation: in a 128×128 matrix, a value entering at the edge participates in 128 multiplications before leaving, without touching any memory. On a CPU, each of those 128 operations would be a load/store pair. That is where the difference in watts per operation comes from — not clock speed. TPUs never beat GPUs on frequency.
bfloat16: FP32’s range in half the bits
FP32 consumes bandwidth. IEEE FP16 cuts the exponent to 5 bits and large-network training breaks (overflow/underflow). Google invented bfloat16:
FP32 1 sinal | 8 expoente | 23 mantissa
FP16 1 sinal | 5 expoente | 10 mantissa (faixa estreita)
bfloat16 1 sinal | 8 expoente | 7 mantissa (mesma faixa do FP32)
Half the traffic, a smaller multiplier, training without loss-scaling gymnastics. NVIDIA, Intel, AMD, and ARM adopted it. Today it is the default in JAX, PyTorch, and LLMs.
The insight is understanding what a neural network actually needs. A gradient that is too small becomes zero in FP16 and training stalls; what almost never matters is a weight’s third decimal place. bfloat16 discards precision (mantissa) and preserves range (exponent) — exactly in the order the network tolerates the loss. Ironwood takes the logic further with FP8, where inference accepts even fewer bits.
VPU, SparseCore, ICI, and OCS
Dense MatMul is not the whole model. The VPU handles GELU, ReLU, softmax, LayerNorm/RMSNorm. The SparseCore handles sparse embeddings too wide for the MXU — hardware embedding support appeared in v4, and Cloud documentation lists the block as SparseCore in v5p, v6e, and TPU7x.
The superpower is not the chip: it is the Pod.
chip <== ICI (Inter-Chip Interconnect) ==> chip
malha 2D ou 3D torus
OCS: espelhos MEMS desviam o laser
topologia reconfigurável em software
ICI connects neighbors without TCP/IP. 3D torus closes the edges. OCS (v4, v5p, Ironwood): instead of an electrical switch converting light↔electrons, a micromirror points the beam. It isolates dead chips and changes the partition (data / tensor / pipeline parallel) in milliseconds.
It is worth emphasizing OCS because it has no equivalent on the other side: in a GPU cluster, changing topology means recabling racks. In the pod, it is a command to a mirror. A chip failing midway through weeks of training does not take down the job — the pod reconfigures around the gap.
Generations (official Cloud TPU figures, 2026)
| Generation | Year | Focus | Peak/chip | HBM | Pod |
|---|---|---|---|---|---|
| v1 | 2015 | INT8 inference only | ~92 TOPS | 8 GB DDR3 | Standalone PCIe |
| v2 | 2017 | training + inference, BF16 | 45 TFLOPS | 16 GB HBM | 256 chips (11,5 PFLOPS) |
| v3 | 2018 | liquid cooling, scale | ~123 TFLOPS | 32 GB HBM2 | 1.024 chips |
| v4 | 2021 | OCS, 3D torus | 275 TFLOPS | 32 GB HBM2 | 4.096 (1,1 EFLOPS) |
| v5e | 2023 | cost/watt | 197 TFLOPS BF16 | 16 GB HBM2e | 256 chips |
| v5p | 2023 | LLM (Gemini) | 459 TFLOPS BF16 | 95 GB HBM3 | 8.960 chips |
| v6e Trillium | 2024 | MXU 256×256 | 918 TFLOPS BF16 | 32 GB, 1.638 GB/s | 256 chips |
| TPU7x Ironwood | 2025 (GA 2026) | two chiplets, FP8 | 2.307 BF16 / 4.614 FP8 | 192 GB, 7.380 GB/s | 9.216 chips |
Ironwood is the first TPU designed with inference at its center: 2 TensorCores, 4 SparseCores per chip, bidirectional ICI of 1.200 GB/s. A superpod of 9.216 chips reaches 42,5 EFLOPS in FP8. It was announced on April 9, 2025, entered preview in Cloud on November 24, 2025, and reached general availability on March 31, 2026. In Cloud there is a single VM configuration, the tpu7x-standard-4t: 4 chips, 768 GiB of HBM, 224 vCPUs, and 960 GB of host RAM. v6e remains the value-for-money option for Gemma, Llama, and diffusion.
The eighth generation has already been announced
In April 22, 2026, at Cloud Next ’26, Google introduced the next generation — and, for the first time, split into two parts with separate purposes:
- TPU 8t, for training. It uses a new generation of ICI to scale to 9.600 chips and 2 PB of shared high-bandwidth memory in a single superpod. Google claims 3× Ironwood’s processing power and up to 2× more performance per watt.
- TPU 8i, for inference. It introduces the topology Boardfly, which directly connects 1.152 chips in a pod, and brings 3× more on-chip SRAM — enough to keep entire KV caches in silicon, without going to HBM — plus a dedicated collective operations engine. The promise is 80% more inference performance per dollar than the previous generation.
Notice what 8i reveals: the bottleneck the entire industry is now chasing is the KV cache, the same one DeepSeek attacked through software with MLA. One solves it with model architecture, the other with SRAM. As of this article’s completion, neither appeared in the release notes for Cloud TPU — an announcement is not availability.
Software: XLA and JAX
Without a compiler, the ASIC is a brick. XLA (Accelerated Linear Algebra) reads the graph, fuses operations into a single kernel, allocates VMEM, and schedules the systole. JAX is DeepMind’s language: Python → XLA, with pmap and shard_map. PyTorch/XLA covers those who already have a PyTorch model.
There is a detail that catches everyone the first time: XLA compiles for a fixed shape. Change the shape of the tensor and it recompiles — and recompilation takes seconds. Code that accepts variable batches on a GPU without complaint becomes a recompilation loop on a TPU, eating the entire gain. The rule of thumb is padding to a handful of fixed sizes.
Getting your hands on a TPU
You cannot buy a TPU and bolt it into your case: they only exist inside Google. What you can do is rent one — and there is a free route.
The free route: Google Colab and Kaggle Notebooks offer TPU runtimes on the free plan (the generation offered changes over time). In Colab, it is Runtime → Change runtime type → TPU. It lets you get a feel for the hardware without a credit card.
The serious route, a TPU VM on Google Cloud:
# cria a VM com 8 chips v6e
gcloud compute tpus tpu-vm create tpu-teste
--zone=us-central1-b
--accelerator-type=v6e-8
--version=v2-alpha-tpuv6e
# entra nela (é uma VM Linux comum)
gcloud compute tpus tpu-vm ssh tpu-teste --zone=us-central1-b
Inside the VM, install JAX with the TPU extra and confirm the runtime can see the silicon:
pip install -U "jax[tpu]"
python3 -c "import jax; print(jax.devices()); print(jax.device_count())"
# [TpuDevice(id=0, ...), ...] 8
A large MatMul in bfloat16, which is literally the MXU’s job:
import jax, jax.numpy as jnp
x = jnp.ones((8192, 8192), dtype=jnp.bfloat16)
mm = jax.jit(lambda a: a @ a.T)
y = mm(x).block_until_ready()
print(y.shape, y.dtype)
print(jax.devices()[0].device_kind)
Two caveats about zone and generation. v6e does not exist in every region: the documentation lists us-central1-b, us-east1-d, us-east5-a, us-east5-b, europe-west4-a, asia-northeast1-b and — what matters to people in Brazil — southamerica-west1-a, in Santiago. Requesting the wrong zone makes create fail. And the workflow tpu-vm above applies to generations through v6e; TPU7x is provisioned as a Compute Engine instance, with a different command.
The block_until_ready() is not decoration: JAX is asynchronous, and without it, you time dispatch, not computation. Those who already have a PyTorch model use PyTorch/XLA:
pip install torch torch_xla[tpu]
python3 -c "import torch_xla; print(torch_xla.device())"
And the post’s most important warning: An idle TPU is still billed. Finished the test? Destroy it:
gcloud compute tpus tpu-vm delete tpu-teste --zone=us-central1-b
gcloud compute tpus tpu-vm list --zone=us-central1-b # confira que não sobrou nada
And Linux in all this
Every TPU VM is a Linux machine. You get Debian with the chips attached to the bus, and access is the usual: SSH, systemd, apt, pip. Inside it:
lspci | grep -i google # os aceleradores no barramento
ls /dev/accel* 2>/dev/null; ls /dev/vfio/ 2>/dev/null # o nó de dispositivo muda por geração
python3 -c "import jax; print(jax.local_device_count())"
The proprietary layer is libtpu, the runtime library XLA loads to talk to the chip — it comes bundled with jax[tpu] and is not open source. The upper layer, by contrast, is entirely free: JAX and OpenXLA under Apache 2.0, and PyTorch/XLA under BSD 3-Clause.
As for the only TPU you could have on your desk — the Coral Edge TPU, that US$ 60 dongle that ran INT8-quantized MobileNet and became a craze in Frigate and Raspberry Pi projects — it is over. The repository google-coral/edgetpu was archived in April 2026, the libraries stopped at TensorFlow 2.16, and the driver gasket for PCIe versions no longer compiles on recent kernels without patches. If you have one working, great; buying one in 2026 means buying technical debt. There was no formal end-of-life announcement — the website simply changed the subject: coral.ai now redirects to a new platform, CoralNPU, an NPU with an architecture that is open RISC-V with an MLIR-based toolchain promising to accept models from PyTorch, JAX, or LiteRT. The repositories tell the same story: the entire Edge TPU ecosystem is archived, and the only ones with recent commits are coralnpu and coralnpu-compiler. Edge TPU is over; the Coral brand is not. For local inference today, the route is a GPU with Ollama, llama.cpp or an NPU integrated into the processor.
On your home Linux machine, then, a TPU is not an option. Local GPUs remain CUDA/ROCm — the post about a refurbished server is the other side of the fence.
What was born on this silicon
2017 Transformer ("Attention Is All You Need") — 8 GPUs NVIDIA P100
2018 BERT 16 chips de Cloud TPU (Base) / 64 chips (Large)
2021 AlphaFold 2 128 cores de TPU v3
2022 PaLM 540 B dois pods v4, via Pathways
2023– Gemini v4 e v5p; contexto de 1 a 2 M de tokens
It is worth dispelling a myth this blog helped repeat: the Transformer was not born on a TPU. The 2017 paper is explicit — the base model was trained on a single machine with eight NVIDIA P100 GPUs. What TPUs did was enable everything that came afterward, when the same design needed scale.
BERT is already a TPU story: 16 Cloud TPU chips for Base, 64 for Large — the paper does not name the generation. AlphaFold 2, published in Nature in 2021 (CASP14 was in 2020), trained on 128 TPU v3 cores and solved the protein-folding problem. PaLM demonstrated 540 B on two v4 pods geographically connected through Pathways. And Gemini only opens million-token windows because HBM3 + OCS split the tensor without turning the bus into a queue — the same problem that DeepSeek’s MLA addresses in the cache, on the other side of the industry.
Who is buying TPUs in 2026
For a decade, TPU was internal-use silicon with a slice rented through Cloud. That changed. On October 23, 2025, Google and Anthropic announced a deal worth tens of billions of dollars giving Anthropic access to up to one million TPUs — the ceiling for the entire contracted Google Cloud expansion — and, in the announcement’s words, “well over a gigawatt” of capacity in 2026. Months later, with Broadcom involved, the two companies extended the deal for multiple additional gigawatts starting in 2027.
The strategically important detail: Anthropic runs the Claude family on three different kinds of silicon — Google TPU, AWS Trainium and NVIDIA GPUs — placing each workload type wherever it costs less. It is practical proof that the CUDA duopoly is no longer the only way to train frontier models, and that TPU stopped being a Mountain View curiosity and became a product competitors buy.
Timeline
- 2013 — the voice-command calculation shows x86 inference would require doubling data centers.
- 2015 — TPU v1 enters internal production, INT8 inference only.
- 18/05/2016 — Google publicly reveals TPU’s existence on Google I/O’s opening day.
- 2017 — v2 brings training and bfloat16; the paper is published: In-Datacenter Performance Analysis; the Transformer is born on v2 pods.
- 2018 — v3 with liquid cooling and a pod of 1.024 chips; BERT.
- 2020 — AlphaFold 2 solves protein folding on v3 pods.
- 2021 — v4 introduces OCS and the 3D torus; a pod of 4.096 chips exceeds 1 EFLOPS.
- 2022 — PaLM 540 B trains on two v4 pods connected through Pathways.
- 2023 — v5e (cost/watt) and v5p (LLM); Gemini 1.0.
- 2024 — v6e Trillium, MXU 256×256, and 918 TFLOPS BF16 per chip.
- Oct/2025 — Anthropic secures access to up to 1 million TPUs and 1 GW of capacity.
- 09/04/2025 — TPU7x Ironwood is announced: 4.614 TFLOPS FP8, 192 GB of HBM3E, a superpod of 9.216 chips, and 42,5 EFLOPS per pod.
- 24/11/2025 — TPU7x enters preview on Google Cloud.
- 31/03/2026 — Ironwood reaches general availability.
- 19/04/2026 — the repository
google-coral/edgetpuis archived, ending the Edge TPU cycle (the dismantling had begun in July 2025, withpycoral). - 22/04/2026 — at Cloud Next ’26, Google announces the eighth generation, TPU 8t and TPU 8i.
References
- TPU7x (Ironwood) — official specifications
- TPU v6e (Trillium)
- Cloud TPU — release notes (preview and GA dates)
- Cloud TPU — regions and zones
- CoralNPU — the RISC-V NPU that replaced Edge TPU
- Run a calculation on a Cloud TPU VM with JAX
- In-Datacenter Performance Analysis of a Tensor Processing Unit (v1, Jouppi et al.)
- Anthropic — expanded partnership with Google and Broadcom
- google-coral/edgetpu (archived in April 2026)
- GPU vs TPU: how AI chips work
- How DeepSeek rewrote the Transformer
- How graphics cards work
- What is a neural network?
While the market fights over H100 and Blackwell, Google reaps a decade of ASICs, its own numeric format, lasers in the pod, and compilers. TPU is not “Google’s GPU.” It is proof that, in tensor training and inference, specialized silicon still beats generic brute force — and now that competitors rent pods by the gigawatt, it is also proof that this bet can be sold.