AI Data Center Network Design and Optimization
Free NVIDIA-Certified Professional: AI Networking practice — 6 questions on AI Data Center Network Design and Optimization, with explanations. No sign-up. Full 12-question mixed test →
A rail-optimized fat-tree fabric is being designed for a 128-node DGX cluster with 8 GPU NICs (rails) per node, giving 1024 total NIC endpoints. Each leaf switch has 32 ports and is configured non-blocking 1:1 (16 downlinks to compute nodes, 16 uplinks to spine). How many leaf switches are required in total across all 8 rails?
Each rail needs its own dedicated set of leaf switches. With 16 downlink ports per leaf, 128 nodes require ceil(128/16) = 8 leaf switches per rail. Across 8 rails, 8 x 8 = 64 leaf switches total.
An AI factory customer requires the lowest possible tail latency for all-reduce collective operations across a 2,000+ GPU training cluster and specifically wants in-network aggregation (SHARP) offload to reduce data movement during gradient reduction. Which fabric type should be recommended?
SHARP (Scalable Hierarchical Aggregation and Reduction Protocol) in-network computing is an InfiniBand-specific capability available on Quantum-2 switches, offloading collective reduction operations into the fabric and directly reducing tail latency for large-scale all-reduce.
In an AI data center design, why is a dedicated storage fabric recommended instead of merging NVMe-oF storage traffic onto the same rail-optimized GPU compute fabric used for NCCL collectives?
Storage I/O generates bursty, incast traffic patterns that differ fundamentally from the tightly synchronized GPU-to-GPU collective traffic. Sharing the fabric introduces congestion and jitter that directly degrades all-reduce completion times and training throughput, which is why compute and storage fabrics are architecturally separated.
A network engineer implementing a rail-optimized topology connects both GPU0 and GPU1 NICs from every node to the same leaf switch (Leaf-0) to conserve switch ports, while GPU2 through GPU7 remain on their own dedicated per-rail leaf switches. What problem does this introduce?
Rail-optimized design assumes exactly one dedicated leaf switch group per GPU rail. Doubling up two rails onto one leaf switch doubles the downlink traffic that switch must carry without adding uplink capacity, breaking the non-blocking 1:1 ratio and creating a hotspot for GPU0/GPU1 traffic specifically.
In the context of a Clos-based fat-tree fabric for AI training, what does it mean for the design to be 'non-blocking'?
Non-blocking in a Clos/fat-tree design means full bisection bandwidth is available: any communication pattern between node sets can occur at full NIC line rate simultaneously, which is achieved by maintaining a 1:1 ratio of downlink to uplink capacity at each tier.
A cluster has 128 nodes, each with 8x400 Gb/s Spectrum-X NICs in a rail-optimized layout (1024 NICs total at 400 Gb/s each). For a fully non-blocking any-to-any design, what is the minimum required bisection bandwidth if the cluster is logically split into two equal halves of 64 nodes each?
Bisection bandwidth must equal the total aggregate NIC bandwidth of the smaller half in the worst case where all traffic crosses the cut: 64 nodes x 8 NICs x 400 Gb/s = 204.8 Tb/s, ensuring no oversubscription across the cut for any-to-any traffic.
Ready for the real thing?
The full course has two full-length practice tests, video lessons for every exam domain, hands-on labs and detailed answer explanations.
Start my full course on Udemy →