This analysis details the technical specifications and strategic positioning of Microsoft's second-generation custom silicon, the Maia 200 AI inference accelerator. Built on TSMC's 3nm process, the monolithic die features over 100 billion transistors, 216 GB of HBM3E memory (7 TB/s bandwidth), and 272 MB of on-chip SRAM. The Maia 200 is specifically designed for low-precision inference (native FP4 and FP8 tensor cores), claiming 10 PetaOPS of FP4 performance and five PetaOPS of FP8 performance at an 880W TDP. Microsoft explicitly pitches the Maia 200 as three times more efficient in FP4 performance than Amazon's third-generation Trainium and offering a 30% better performance-per-dollar ratio over existing systems. Deployment utilizes a rack-scale architecture reminiscent of NVIDIA's NVL72, featuring blades containing four Maia 200 chips connected via 28 custom 400 Gbps Ethernet ports per chip. The host processor is the companion Cobalt 200 ARM-based CPU.
Exploring Microsoft's Custom AI Silicon: Maia 200 and Cobalt 200 Deep Dive
0:43 Hyperscaler Silicon Strategy: Microsoft is noted as the latest hyperscaler, alongside Meta (MTIA), Amazon (Trainium/Inferentia), and Google (TPU), to develop proprietary silicon aimed at increasing data center efficiency.
2:47 Microsoft's Custom Silicon: Microsoft's custom chip lineage includes the Maia (AI accelerator) and Cobalt (ARM-based host CPU).
3:20 Cobalt 200 CPU: The Cobalt 200, announced in November 2025, is an ARM-based host CPU featuring a maximum configuration of 132 tweaked ARMv9 (V3) cores spread across two chiplets.
3:44 Maia 200 Core Specifications: The Maia 200 is classified as an inference powerhouse, fabricated on TSMC’s 3nm process (assumed N3E) and incorporating native FP4 and FP8 tensor cores.
4:16 Memory and Transistor Count: The monolithic compute die utilizes six HBM3E stacks, totaling 216 GB of memory with a bandwidth of 7 TB/s. The chip features over 100 billion transistors and 272 MB of on-chip SRAM.
6:41 Die Area Estimation: The main compute die area is estimated at approximately 727 mm², with the 272 MB of SRAM consuming 10-12% of the available silicon area.
7:27 Performance and Power: The Maia 200 delivers 10 PetaOPS (FP4 precision) and 5 PetaOPS (FP8 precision) at a total chip TDP of 880W (a 20W increase over the Maia 100).
5:34 Competitive Benchmarking: Microsoft claims the Maia 200 provides three times the FP4 performance of Amazon’s third-generation Trainium and is the most efficient inference system they have deployed, offering a 30% improvement in performance-per-dollar over existing systems (implying comparison against NVIDIA and AMD).
8:54 Networking and Connectivity: The chip supports 28 400 Gbps Ethernet ports per device. This scale-up utilizes a custom, proprietary Azure protocol layered over standard Ethernet, designed specifically to optimize data movement for low-bit precision data types.
10:12 Rack-Scale Deployment: The chips are deployed in liquid-cooled blades, with four Maia 200 chips per blade. A full rack configuration consists of 18 compute blades, totaling 72 chips per rack (mirroring the NVL72 form factor), with an estimated rack power consumption of 65 kW.
11:54 Software Ecosystem: The SDK includes Triton compiler support for PyTorch, low-level programming via NPL, and a simulator/cost calculator to aid customer migration and planning.
13:02 Monolithic Design Choice: The Maia 200’s use of a monolithic die contrasts with the chiplet approach adopted by competitors like Amazon, indicating reliance on the large-scale integration capability of TSMC's N3P process.
6:00 Deployment Timeline: The Maia 200 is available immediately in US Central, with US West 3 (Phoenix, AZ) slated as the next region.