Microsoft unveils powerful new chip for AI inference

Microsoft has announced the launch of its latest chip, the Maia 200. The company describes it as a silicon flagship product designed to scale AI inference.

The 200 follows the company’s Maia 100, released in 2023, and is technologically equipped to run powerful AI models faster and more efficiently, the company said. Maia has over 100 billion transistors and delivers performance of over 10 petaflops with 4-bit accuracy and approximately 5 petaflops with 8-bit accuracy. This is a significant improvement over the previous model.

Inference refers to the computational process that runs a model, as opposed to the computations required to train the model. As AI companies mature, inference costs become an increasingly important part of overall operating costs, leading to renewed interest in how to optimize processes.

Microsoft wants Maia 200 to be part of that optimization, allowing AI businesses to run with fewer interruptions and less power usage. “Practically speaking, a single Maia 200 node comfortably runs our largest models today, with plenty of headroom for even larger models in the future,” the company said.

Microsoft’s new chip is also part of a growing trend of tech giants turning to chips they design in-house as a way to reduce dependence on Nvidia, whose cutting-edge GPUs have become increasingly important to AI companies’ success. For example, Google has TPUs (tensor processing units), which are not sold as chips, but as computational power that can be accessed through the cloud. And Amazon Trainium, the e-commerce giant’s proprietary AI accelerator chip, just launched its latest version, Trainium3, in December. In either case, the TPU allows you to offload some of the computations assigned to the Nvidia GPU, reducing overall hardware costs.

With Maia, Microsoft is positioned to compete with these alternatives. The company said in a press release on Monday that Maia delivers three times the FP4 performance of 3rd generation Amazon Trainium chips and FP8 performance over Google’s 7th generation TPUs.

Microsoft says Maia is already hard at work developing the company’s AI models with its superintelligence team. It also supports the operation of Copilot, the company’s chatbot. As of Monday, the company announced that it had invited a variety of stakeholders, including developers, academics, and Frontier AI Institute, to use the Maia 200 software development kit in their workloads.