Nvidia's Rubin Architecture: Networking is the Key to AI Performance

Nvidia's unveiling of the Rubin architecture at CES 2026 signals a paradigm shift in AI infrastructure. While the Rubin GPU itself boasts impressive specifications, including 50 petaFLOPS of 4-bit computation for transformer-based inference, the true innovation lies in Nvidia's 'extreme co-design' approach, integrating six new chips that harmonize compute and networking. This holistic strategy, as explained by Gilad Shainer, senior vice president of networking at Nvidia, unlocks unprecedented performance gains by optimizing data flow and minimizing bottlenecks across the entire system.

Scale-Up Revolution: In-Network Compute Takes Center Stage

The Rubin architecture prioritizes efficient communication between GPUs within a single rack. The NVLink6 switch, a key component of the 'scale-up network,' doubles the bandwidth to 3,600 GB/s for GPU-to-GPU connections, compared to its predecessor. Crucially, Nvidia is expanding 'in-network compute,' offloading specific calculations from the GPUs to the network itself.

This approach addresses a critical bottleneck in distributed AI workloads. Offloading tasks like the 'all-reduce' operation during AI training—where gradients computed by each GPU need to be averaged—saves significant computational time and power. "The scale-up network is not really the network itself," Shainer explains. "It's computing infrastructure, and some of the computing operations are done on the network...on the switch."

Scaling Out and Across: Connecting the AI Ecosystem

Beyond the rack, the Rubin architecture addresses the challenges of connecting multiple racks within a data center through its 'scale-out network'. Key components include the ConnectX-9 networking interface card, the BlueField-4 data processing unit (DPU), and the Spectrum-6 Ethernet switch. The Spectrum-6 leverages co-packaged optics to double the bandwidth while minimizing jitter—the variation in arrival times of information packets. Jitter, according to Shainer, directly translates to lost revenue, as idle GPUs waste valuable processing time waiting for delayed data.

While the Rubin architecture doesn't currently include dedicated chips for connecting between data centers ('scale-across'), Nvidia acknowledges this as the next frontier. The demand for ever-larger GPU clusters, exceeding 100,000 GPUs for certain workloads, necessitates seamless inter-data center communication. The increasing size and complexity of AI models means that we need to start thinking about security concerns across multiple data centers as well.

Nvidia's Rubin architecture represents a strategic move towards tightly integrated, networking-centric AI infrastructure. By optimizing data flow, minimizing latency, and expanding in-network compute capabilities, Nvidia is not just building faster GPUs but architecting an entirely new paradigm for distributed AI. This approach will be crucial for training and deploying increasingly complex AI models and maintaining security across these complex systems, and the emphasis on networking will define the next generation of AI hardware.