Google announced Tuesday that its long-awaited version of PyTorch, TorchTPU, is now available as an open source project. This will see Google join Amazon in trying to break Nvidia’s stranglehold on the AI industry. In the AI industry, most AI apps are written to run on Nvidia chips, or GPUs, by default.
PyTorch is a popular open-source toolkit used by AI app developers and startups to build, train, and run AI models. By some estimates, it’s used in 85% of AI apps. PyTorch was originally developed by Meta (then Facebook) in 2016, long before the world had a hunch that LLM would change the world. At the time, Nvidia’s GPUs were already the standard hardware for AI research, so many of PyTorch’s features were built specifically to work with GPUs. In 2022, Meta turned PyTorch over to the Linux Foundation, a nonprofit organization that manages many important open source projects.
Google’s version of TorchTPU allows you to run AI apps on Google’s AI chips called Tensor Processing Units (TPUs). It also makes it easy to migrate existing PyTorch apps to TPU, and thus to Google Cloud. Developers can do this with simple configuration changes without having to rewrite their apps.
Amazon made a similar announcement on Monday when it released its own version of PyTorch called TorchNeuron. TorchNeuron allows you to port your AI apps to Amazon’s own AI chip Trainium and AWS without rewriting them. Amazon also said it would further its efforts to loosen Nvidia’s grip on PyTorch. We’ve sent code to the main PyTorch project that decouples various features from Nvidia’s chips, making it more open to any hardware by default.
However, Google may be better than Amazon in some ways. It says it has worked directly with Meta, Hugging Face, Inferact (maker of vLLM) RadixArk (maker of SGLang) and others to ensure that TorchTPU’s features and code libraries work well with these popular AI development tools, just like the original PyTorch.
Additionally, the search giant has launched the TPU Agent Suite, a dedicated tool that allows developers to migrate apps to Google’s chips and cloud and fix errors along the way. The popular LiteRT tool for running AI on phones, laptops, and embedded devices has also been updated to better integrate with the PyTorch AI app.
This doesn’t mean AI app developers will automatically ditch Nvidia’s hardware all together, just because it’s easier to move to alternative chips or the cloud. But Amazon and Google are clearly working to end Nvidia’s dominance.
If you make a purchase through links in our articles, we may earn a small commission. This does not affect editorial independence.
