Google open-sources TorchTPU, a native PyTorch stack for TPUs

Google Cloud open-sourced TorchTPU on 20 October 2026. It is a native PyTorch stack for TPUs, co-developed with Meta’s PyTorch team, so existing PyTorch code can run on a TPU with the usual APIs. A developer can debug in eager mode, then compile with torch.compile. Tensors stay ordinary PyTorch tensors on a TPU device, and collective operations go through torch.distributed.

Google’s developers blog announced native PyTorch backends for vLLM, with Inferact, and for SGLang, with RadixArk. Both keep the upstream scheduler and API server and add TPU kernels underneath. TPU Sync moves the KV cache, the memory of an ongoing conversation, between chips without an extra copy on the host.

Zongwei Zhou, PyTorch-TPU lead at Meta, says SGLang-TorchTPU is running in production at Meta. In Hugging Face Transformers v5, Google says a model is sent to a TPU with device_map set to tpu.

The Cloud blog says TorchTitan training runs were validated on up to 8000 Ironwood TPUs and, later in the same post, on up to 8,192. The developers blog cites a public InferenceX benchmark for the speed and cost claims, rather than printing that chart.

Key numbers

• 2.01 cents per million tokens for Qwen 3.5 on Ironwood, in Google’s InferenceX result
• 52% Model Flops Utilization on Qwen 3 8B in the TorchTitan test
• More than 700 tokens per second for one user on the Kimi K3 decode kernel

The SGLang-TorchTPU plugin is Apache 2.0. Google has not stated a licence for every other repository.