Microsoft is teaming up with Nvidia to create an AI supercomputer powered by Azure and tens of thousands of Nvidia GPUs. The project will be a multi-year collaboration, and it will allow organizations to train language models and a broad range of AI applications and services at scale.

“Our collaboration with NVIDIA unlocks the world’s most scalable supercomputer platform, which delivers state-of-the-art AI capabilities for every enterprise on Microsoft Azure,” said Scott Guthrie, executive vice president of the Cloud + AI Group at Microsoft.

Currently, the Azure virtual machines on this supercomputer are using Nvidia A100 GPUs and Quantum 200Gb/s InfiniBand networking. In the future, Microsoft and Nvidia will also offer AI-optimized Azure VMs with Nvidia H100 GPUs and Quantum-2 400Gb/s InfiniBand networking. Azure instances will support Nvidia’s AI Enterprise software suite, and the two companies will also work on optimizing Microsoft’s DeepSpeed deep learning optimization software

This new multi-year AI supercomputer project with Nvidia follows a similar partnership between Microsoft and Open AI. Two years ago, the company announced that they were building a supercomputer hosted in Azure to help developers train extremely large artificial intelligence models.