NVIDIA Announces the Blackwell GPU Architecture

NVIDIA announced the Blackwell GPU architecture at GTC in March 2024. The flagship B200 combined 208 billion transistors across two tightly connected dies and targeted the rapidly growing cost of training and serving large AI models.

Blackwell introduced second-generation Transformer Engine technology with support for very low-precision formats including FP4. Lower precision can increase throughput and reduce memory traffic when model accuracy can be preserved. Fifth-generation NVLink also increased bandwidth between accelerators, critical for models spread across many GPUs.

The architecture was designed as part of rack-scale systems such as GB200 NVL72 rather than only as a standalone card. AI hardware competition had moved from “which GPU is fastest?” toward the design of entire clusters, interconnects, power systems, and cooling.