Google AI chips are now running inside Meta's systems

The landscape for artificial intelligence hardware is shifting as major tech giants find new ways to diversify their computing power. Google has announced that its custom AI chips, known as tensor processing units, are now handling live production workloads for Meta. This development was revealed through the release of a new version of the SGLang inference engine that works natively with PyTorch, a popular framework for building AI applications. A similar open source update for vLLM, another key inference engine, was also released to the public.

Until now, running these specific engines on Google hardware required the use of JAX, Google's own AI framework. The new software versions change that dynamic by making Google's chips a direct and native option for PyTorch developers. This update means that teams can use the hardware without needing to rewrite large portions of their existing code. The goal is to make the transition smoother for companies that have invested heavily in the PyTorch ecosystem.

Zongwei Zhou, who leads the PyTorch TPU efforts at Meta, confirmed that the system is already active. He stated that the setup is running in production at Meta today. Zhou highlighted that this achievement came from a close three-way collaboration between Meta, Google, and the team behind SGLang. That team operates out of RadixArk, a startup founded by the original creators of the engine. According to Zhou, the new stack provides Meta with a remarkably mature and native serving environment for its models.

Jana van Greunen, the director of PyTorch engineering at Meta, emphasized the importance of flexibility in the company's infrastructure. She noted that Meta cannot be constrained by a single hardware architecture. The new TorchTPU tool allows the company to deploy its PyTorch workloads directly onto Google's hardware. Google confirmed that it co-developed this specific tool with Meta's internal engineering team to ensure compatibility and performance.

This is not the first time Meta has experimented with Google's silicon. An older software bridge called PyTorch/XLA previously allowed for this connection, though Google now refers to it as a predecessor. Reports indicated that Meta research teams used these chips through that older method between 2020 and 2023. Those initial contracts were eventually canceled. The current push seems designed specifically to address the needs of large companies like Meta that did not want to migrate their entire infrastructure to JAX.

The physical location of these production chips remains somewhat unclear, but recent reports suggest a significant partnership is in motion. Discussions last year reportedly involved Google renting out its cloud TPU capacity for 2026, with actual purchases planned for 2027. Meta is said to have signed a multi-year rental agreement as part of this arrangement. This move signals a long-term commitment to integrating Google's hardware into their broader AI operations.

The timing of this announcement is notable because it came just one day after Amazon made a similar move. Amazon announced it was open-sourcing its own native PyTorch backend, called TorchNeuron, for its Trainium chips. Both of these developments appear to be aimed at loosening the software grip that Nvidia currently holds over the AI industry. By providing native support for PyTorch, these companies are making it easier for developers to use alternatives without significant friction.

Performance metrics suggest the new setup is competitive. A tuned vLLM configuration serving a massive 2.8-trillion-parameter model reportedly reached over 700 tokens per second for a single user. This test was conducted on seventh-generation Ironwood chips. The results come from Inferact, a startup that helped build the new tool. The test used 16 chips with a specific decoding method to speed up processing. In comparison, a published setup using 16 Nvidia GB200 chips reached 452 tokens per second. For smaller batch sizes, the new kernels deliver roughly 1.4 to 2 times the throughput of the Nvidia baseline.

The team behind SGLang plans to continue adding support for more models, starting with DeepSeek V4.1 Flash and Reflection Beam. They are also working toward providing support for new open models on their release day, a goal they hope to achieve by the first quarter of 2027. More details are expected to be shared at an upcoming summit in San Francisco in November, where the three partners will discuss the future of this technology.

For readers in New England and elsewhere, this shift could mean more competition in the AI market. As major companies like Meta and Amazon push to break away from a single hardware provider, the cost and availability of AI services may become more varied. The integration of Google chips into Meta's production systems marks a significant step in this diversification process.