Microsoft Unveils Custom AI Chips to Power Azure Cloud
At its Ignite conference, Microsoft unveiled two new custom-designed chips meant to accelerate artificial intelligence workloads and boost the performance of its Azure cloud computing platform.
The chips - called Azure Maia and Azure Cobalt - are Microsoft's first datacenter server processors built entirely in-house. They represent the company's push to optimize every layer of its cloud infrastructure for AI, from silicon to servers and datacenter cooling systems.
"The Microsoft Azure Maia AI Accelerator, optimized for artificial intelligence (AI) tasks and generative AI, and the Microsoft Azure Cobalt CPU, an Arm-based processor tailored to run general purpose compute workloads on the Microsoft Cloud."
The Maia AI accelerator is purpose-built to speed up training and inference of large AI models like those used for natural language processing. Microsoft says Maia delivers up to 40% higher performance for AI workloads compared to its existing Azure servers,
Meanwhile, the Cobalt CPU is an Arm-based processor tailored for general cloud computing tasks like running Azure services, Microsoft 365, and SQL Server, with plans to make virtual machines available to customers next year for a variety of workloads.
The Azure Cobalt CPU, named after the blue pigment, is a 128-core chip that’s built on an Arm Neoverse CSS design and customized for Microsoft. It’s designed to power general cloud services on Azure.
By relying less on partners like AMD, Intel, and Nvidia, Microsoft aims to reduce costs and accelerate the delivery of new AI capabilities across its cloud portfolio. The company has already tested Maia with OpenAI using models like GPT-3.5 that powers ChatGPT.
Both the Maia and Cobalt chips will become available early next year to Microsoft’s data centers, initially powering the company’s services such as Microsoft Copilot or Azure OpenAI Service. To enable rapid deployment, Microsoft has developed new server and rack designs optimized for liquid cooling and high density.
The customized hardware is meant to work hand-in-hand with Microsoft's software, with both layers co-designed to maximize performance. The company will also share some of its learnings with the industry. For example, Microsoft plans to open source its rack designs through the Open Compute Project.
Microsoft has a long history of chip design, having worked on silicon for Xbox, Surface, and more. But Maia and Cobalt represent its first datacenter processors meant for wide customer use.
Even with its own chips, Microsoft says partners like AMD, Intel, and Nvidia remain critical to provide choice for Azure customers. The company announced expanded partnerships at Ignite, including new Azure instances powered by Nvidia's latest H100 and H200 GPUs.
By controlling more of the stack in Azure, from datacenter cooling systems down to the core silicon, Microsoft aims to steer the cloud ecosystem toward greater adoption of AI. More efficient, optimized infrastructure also supports the company's sustainability goals.
Microsoft said it is already working on second-gen versions of both its Maia AI accelerator and Cobalt CPU. With the new chips, Microsoft is betting optimized cloud infrastructure will be key to delivering next-generation AI capabilities to enterprises.