Figure: bandwidth aggregation across a CXL 3.0 switch topology. A single x16 PCIe 5.0 endpoint is capped at 64 GB/s, so a B200 GPU is fanned out through a CXL 3.0 switch to four endpoints of 64 GB/s each, a theoretical aggregate of 256 GB/s and a combined 1 TB of DRAM plus 16 TB of flash. Against a Llama-70B per-layer transfer of 2.22 GB — attention 302 MB, FFN 1.41 GB, KV slice 512 MB — inside a 5 to 10 millisecond layer compute budget, the required bandwidth is 222 to 445 GB/s: one endpoint takes 34.8 ms and stalls the GPU, two endpoints take 17.4 ms and still stall, four endpoints take 8.7 ms and fit a 10 ms budget. The counterpoint is that steady-state decode demand is only 11.4 GB/s for 16 users at 10 tokens per second with a 3 percent miss tail, which is 4.5 percent fabric utilization, so the four endpoints are chosen for capacity, 1 TB DRAM and 16 TB flash, not for the 256 GB/s ceiling. The 140 GB of FP16 weights stay HBM-resident and never cross CXL. All values are an analytical model, not measured.

Section 5

Bandwidth Aggregation

Linear scaling through CXL 3.0 switch topology

Single Endpoint Limitation

CXL bandwidth per endpoint is constrained by PCIe lane count. A single x16 Gen5 endpoint maxes out regardless of internal capability.

64 GB/s
x16 PCIe 5.0 limit
CXL 3.0 Switch Aggregation
GPU
B200
CXL 3.0
CXL 3.0 Switch
Multi-port fabric
Theoretical Aggregate Bandwidth
4 endpoints × 64 GB/s =
256 GB/s
Theoretical aggregate of the link budget — a supply ceiling, not a measured or achieved rate. Each endpoint contributes 256 GB DRAM and 4 TB flash → 1 TB DRAM + 16 TB flash total.
Layer Prefetch: Overlap Execution & Transfer
Execution Trace Predictor
Observes GPU execution → predicts next layer → triggers prefetch
0 ms
5 ms
10 ms
15 ms
20 ms
GPU Compute
Prefetch (k=1)
HBM State
Perfect Overlap: Prefetch N+1 completes before GPU finishes N
Key Insight: Tprefetch < Tcompute → zero stall
Layer Execution Time
Compute budget available for overlap
Layer Size
Transfer requirement per layer
Aggregate Bandwidth
endpoints × per-endpoint BW
The Math: Can Prefetch Keep Up?
Llama-70B Per-Layer Transfer
2.22 GB
attn 302 MB + FFN 1.41 GB + KV slice 512 MB
Layer Compute Budget
5–10 ms
B200, BF16 dense (2,250 TFLOPS)
Required BW
222–445 GB/s
2.22 GB ÷ 10 ms … ÷ 5 ms
1 Endpoint (64 GB/s)
2.22 GB ÷ 64 GB/s = 34.8 ms 3.5–7× the layer budget → GPU stalls
2 Endpoints (128 GB/s)
2.22 GB ÷ 128 GB/s = 17.4 ms Still above a 10 ms budget → stalls
4 Endpoints (256 GB/s)
2.22 GB ÷ 256 GB/s = 8.7 ms Fits inside a 10 ms layer budget; marginal at 5 ms
The Counterpoint: Demand, Not Supply, Sets the Endpoint Count
Steady-state CXL demand
11.4 GB/s
16 users @10 tok/s, 3% miss tail (0.072 GB/user/step)
Theoretical aggregate supply
256 GB/s
4 × 64 GB/s link budget
Fabric utilization
4.5%
11.4 ÷ 256 — the link is nearly idle in decode
The fabric is capacity-driven, not bandwidth-driven. Full-layer streaming above is the prefill/admission burst case (43 GB written at 256 GB/s = 168 ms). In steady-state decode only the miss tail crosses CXL — 11.4 GB/s, 4.5% of the theoretical aggregate. Four endpoints are chosen to reach 1 TB of DRAM and 16 TB of flash, not to reach 256 GB/s.
Llama-70B Example
140 GB FP16 weights total
The 140 GB of model weights are HBM-resident and never cross CXL. Only KV traffic uses the fabric: the miss tail in steady state, and whole-context admission on prefill. That is why four endpoints are sized for capacity (1 TB DRAM + 16 TB flash) rather than for the 256 GB/s theoretical aggregate.

Analytical model — all bandwidths, capacities and times are derived from link budgets and vendor specifications, not measured on hardware. Aggregate figures are theoretical ceilings. Canonical numbers v4.0.