Amazon Web Services on Monday released a tool called TorchNeuron that makes it easier for AI startups to migrate their AI apps to Amazon’s AI chip Trainium. In doing so, Amazon is also directly chipping away at the software dominance that helped Nvidia become the default hardware choice for AI apps and the AI hardware provider of choice.
TorchNeuron works with PyTorch, a free, open-source toolkit that developers use to build, train, and run AI models. According to Amazon, you can now run AI apps created with PyTorch on Trainium with a single line of configuration change and no code rewrites.
According to Amazon, approximately 85% of AI apps are written in PyTorch. It was originally developed by Facebook, but since 2022 it has become part of the Linux Foundation, which maintains open source projects. Nvidia has spent years ensuring that its CUDA software (the software infrastructure that allows apps to run on Nvidia GPUs and other hardware) seamlessly supports AI apps created with PyTorch. As a result, Nvidia’s hardware is now deeply integrated into the toolkit, essentially by default.
TorchNeuron is based on PyTorch, so developers who know PyTorch already know how to use it. Amazon isn’t the first company to try to easily bring its chips onto its platform. PyTorch users can also port their apps to AMD GPUs and Apple chips with simple configuration changes.
But Amazon claims that porting to Trainium is not only easy, it also gives AI startups access to Amazon’s entire cloud. Unsurprisingly, AWS claims this is a better choice for startups than using Neocloud, much of which is tied to Nvidia and its hardware (CoreWeave, Nebius, Firmus, etc.).
“You’re building on AWS. You’re not betting on a single-purpose compute provider and rushing to build everything else around it,” Colin Brace, vice president of technology software at AWS Annapurna Labs (Amazon’s division that makes custom chips), told TechCrunch via email. “AWS already has the infrastructure to go from prototype to production, allowing founders to quickly migrate and scale as needed,” he continued.
AWS says its cloud can save money, even if AWS doesn’t offer the lowest price per token, because the same chip handles both training (teaching the AI model) and inference (putting the completed model to work, such as when answering user questions) and is tightly integrated with the cloud infrastructure startups need.
“The value of Trainium, in addition to its price performance, is the fact that it is a single platform for both training and inference, which improves economics across workloads,” said Brace. “Look at Deepgram. They took native PyTorch support and migrated to Trainium to process 70% more audio per dollar with production-grade accuracy.” Deepgram is a provider of AI voice apps and a competitor to ElevenLabs.
But Amazon isn’t content with just giving PyTorch app developers an easy option to migrate to its cloud.
Amazon also says it is pushing for PyTorch not to automatically become Nvidia-centric and will contribute the code to do so to the main PyTorch project. In a blog post announcing the new tool, the company wrote:
Amazon’s approach extends beyond its own hardware. The company is delivering changes upstream to PyTorch itself and working with the PyTorch community to remove code that ties the framework to specific chips. The goal is to make PyTorch more hardware independent, not just for Trainium users, but for the entire developer community.
While such tactics are certainly good for companies selling hardware and cloud services that compete with Nvidia, part of the point of PyTorch is to hide all the pesky difficulties of running AI code on hardware. We’ll have to wait and see if pushing Nvidia that far will give developers more choice and more affordable options, as Amazon promises, or just create more configuration headaches.
In any case, if Amazon succeeds in separating PyTorch from Nvidia completely, it could have far greater implications for the GPU giant than simply making a deal with a chip competitor. Brace said: “The real story is removing the lock-in that was preventing developers from having any real choice at all.”
If you make a purchase through links in our articles, we may earn a small commission. This does not affect editorial independence.
