The company expects each Maia 300 data center rack to contain 96 chips, twice the number used with Maia 200. Microsoft says the new configuration can process as much as seven times more AI tokens per rack and deliver 40% more tokens per megawatt than GPU-based systems.
“We need new architectures that will give us essentially better performance, better use of the power that we have,” Microsoft General Manager Alistair Speirs said. “You can think of power translating pretty directly into operational cost for something like a data center as well.”
The shift matters most for inference, the process of running AI models to respond to user requests. Microsoft said Maia will increasingly handle that work across its own products, while Nvidia and AMD hardware will remain part of the company’s infrastructure, including for model training.
Microsoft already uses the Maia 200 for production workloads that include Microsoft 365 Copilot, OpenAI models, Microsoft’s own MAI models and open-source systems. CEO Satya Nadella said in July that Maia 200 delivered “30% better performance per dollar than the latest generation hardware in our fleet.”
The company also expects parts of Microsoft 365 Copilot to run faster with Maia 300, according to technical fellow Jim Kleewein.
Microsoft’s broader AI business still depends heavily on Azure, which hosts models for customers, along with services including Microsoft 365 Copilot and GitHub Copilot. Its data centers currently contain large numbers of Nvidia GPUs, which are also used for models from OpenAI and Anthropic.
That dependence has pushed Microsoft toward a wider mix of processors. In addition to its own chips, the company uses hardware from Nvidia, AMD and Intel. Finance chief Amy Hood said last month that Microsoft’s use of internally developed silicon has contributed to “margin improvement over the course of a decade.”
Microsoft is not providing a firm timeline for reaching its planned Maia deployment scale.
“We reserve a little bit of optionality there, because this is an industry that’s changing very fast, and we want to deploy the right silicon for the right models that we’re using at the right time as well,” Speirs said.
The company’s Maia strategy still differs from the approaches taken by some other cloud providers. Microsoft does not currently rent Maia accelerators by the hour to outside customers, unlike Google with its tensor processing units and Amazon with its Inferentia and Trainium chips.
“Nobody has tested them outside of Microsoft yet,” said Patrick Moorhead, CEO of Moor Insights and Strategy.
Nvidia will continue to play a major role in Microsoft’s hardware plans. Microsoft said it still works with both Nvidia and AMD for model training, even as Maia takes on a larger share of inference inside Microsoft’s own services.
This analysis is based on reporting from CNBC.
Image courtesy of Microsoft.
This article was generated with AI assistance and reviewed for accuracy and quality.
