On 20 October, Google open-sourced TorchTPU, its native PyTorch backend for TPUs, at PyTorch Conference. The same day, AWS put TorchNeuron, its native PyTorch backend for Trainium, into public beta. A week earlier, on 13 October, NVIDIA had opened its Run:ai platform.
Three chip makers moved their software into the open within a week. The contest is no longer only about silicon. It is about which hardware developers can reach without rewriting their code.
What did each company release?
Each release sits at a different layer of the stack, so the three are not direct copies. Here is what each one covers:
| NVIDIA Run:ai | Google TorchTPU | AWS TorchNeuron | |
| Date | 13 October | 20 October | 20 October |
| Layer | GPU scheduling and sharing on Kubernetes | Native PyTorch backend | Native PyTorch backend |
| Hardware | NVIDIA GPUs | Google TPUs | Trainium2 and Trainium3 |
| Status | Open source | Open source, v0.2.0, Apache 2.0 | Public beta, Apache 2.0 |
| What changes in code | Cluster setup, not model code | Device set to tpu | Device changed from cuda to neuron |
Why did two rivals build the same kind of backend?
Google and AWS reached the same design from opposite sides. Both register their chip as a native PyTorch device through PrivateUse1, PyTorch’s official extension point for custom hardware. Both also replace an older route built on XLA.
Google says TorchTPU was co-developed with the PyTorch team at Meta. Its design, first described in April, is built on PrivateUse1, and Google recommends it over PyTorch/XLA for new workloads. AWS’s backend uses PrivateUse1 and moves away from the XLA-based torch-neuronx.
The pitch is the same too: change the device string. Google says Hugging Face Transformers v5 deploys to TPUs with device_map=”tpu”, and TorchNeuron turns .to(‘cuda’) into .to(‘neuron’).
What do the new backends not change?
Changing the device string gets code running, not running fast. The TorchTPU team says the change alone will not reach peak TPU performance. AWS’s own guide lists three levels of control, from eager mode to torch.compile to custom NKI kernels.
Three limits stand out:
- Custom CUDA code: kernels and extensions written for NVIDIA hardware still need porting
- Maturity: TorchNeuron is a beta, and AWS’s TorchTitan integration supports static shapes only under torch.compile
- Evidence: the 52% model FLOPs utilization on Qwen 3 8B, scaled across up to 8,192 Ironwood TPUs, comes from Google’s own tests, and no independent benchmark has been published
NVIDIA’s release has its own tie to NVIDIA hardware. The Run:ai scheduler is open, but its GPU fractioning relies on a CUDA 13.4 driver feature.
What does TPU time cost?
Google publishes TPU rates per chip-hour, and the newest chip costs far more than the older one. Google’s Cloud TPU pricing page lists these on-demand rates:
- Ironwood: $12.00 in Iowa and $13.20 in London, falling to $5.40 on a three-year commitment in Iowa
- Trillium: $3.24 in Tokyo, and $2.70 in US East regions
Per-hour prices compare unlike chips. Ironwood carries 192 GB of memory, and Google says it delivers about four times Trillium’s per-chip performance. The fairer test is cost per finished task, which no one has published for TorchTPU yet.
Where can teams in India run these chips?
Google’s location tables list GPUs, not TPUs, for its Mumbai and Delhi regions. TPU models appear for regions such as Iowa and Tokyo. Teams in India testing TorchTPU would therefore usually run it outside the country.
That matters for regulated data. RBI requires payment system data to be stored only in systems in India, and SEBI’s cloud framework requires data, including logs, to stay within India’s legal boundaries. Under DPDP, transfers abroad are allowed except to countries the government restricts, though the cross-border provisions are not yet operational.
Access is also widening. Alphabet’s June-quarter filing says Google Cloud has signed agreements to supply TPU systems to customers with on-premises infrastructure, with most of the revenue due in 2027.
What this means going forward
The three releases point the same way. Each chip maker now competes where developers work, inside open frameworks and open schedulers, and each one lowers the first step of trying its hardware. None has yet shown that the second step, running at full speed, is as easy.
The next signals are independent benchmarks, cost per task on real workloads, and how quickly TorchNeuron leaves beta. For teams choosing hardware, the question is shifting from which chip is fastest to which one a codebase can move to and back.
