Direct measurement on DGX Spark (GB10 Grace Blackwell), 19 August 2026: streaming bandwidth, two-term decode latency fits, and two bugs caught before they became published numbers.
All numbers in this chapter come from a single machine: an NVIDIA DGX Spark with a GB10 Grace Blackwell superchip and 128 GiB of coherent unified LPDDR5X memory, spec-rated at 273 GB/s. The model under test was Qwen2.5-7B-Instruct in fp16 — 28 layers, 4 KV heads, head dimension 128, giving 56 KiB of KV-cache per token. Weights load at 15.2 GB (14.16 GiB). Measurements were taken 19 August 2026 using two scripts, bw_wall.py for raw streaming bandwidth and vllm_slope.py for the batch × context decode-latency sweep. Nothing here is simulated or extrapolated from vendor datasheets; every number in the tables below is a measured fit or a direct read+write streaming copy.
The streaming bandwidth test ran a read+write copy over 4 GiB buffers, median of 20 runs, and landed at 236.5 GB/s, 87% of the 273 GB/s spec figure. This result retires an assumption carried in earlier internal drafts of this document, which had discounted spec bandwidth by roughly 50% as an "effective bandwidth" haircut. That haircut was never measured — it was a guess, and the guess was wrong in the pessimistic direction by a wide margin. Unified memory on this platform delivers far closer to its rated bandwidth than the earlier planning assumption credited it for, and every downstream estimate in this document that depends on achievable bandwidth uses 236.5 GB/s, not spec, and not the old 50% haircut.
Decode latency was modeled as a two-term linear fit, t = a + b·KV_GiB, across a sweep of batch sizes and context lengths, for two separate inference stacks: HuggingFace with SDPA attention (unpaged), and vLLM 0.20.1 with FlashAttention-2 (paged). The intercept a is interpreted as the fixed per-step cost dominated by weight traffic; the slope b is the marginal cost per GiB of KV-cache read.
| Stack | Fit (ms) | n | R² | Inferred fixed-path BW | Inferred KV BW |
|---|---|---|---|---|---|
| HuggingFace + SDPA (unpaged) | t = 100.3 + 16.34·KV_GiB | 7 | 0.991 | 151.5 GB/s | 65.7 GB/s |
| vLLM 0.20.1 + FlashAttention-2 (paged) | t = 73.9 + 17.58·KV_GiB | 9 | 0.945 | 205.7 GB/s | 61.1 GB/s |
Expressed against the 236.5 GB/s actually achieved on this hardware, the fixed (weight) path uses 64% of achievable bandwidth on HuggingFace and 87% on vLLM; the KV path uses 28% on HuggingFace and 26% on vLLM. The inferred effective KV path is roughly 3.4× more costly per byte than the fixed path (205.7 ÷ 61.1 = 3.37, using the vLLM fit). This ratio must be read for what it is: a quantity derived from two intercept/slope regressions, not a direct per-byte hardware measurement of two separate memory paths. No experiment in this chapter isolated KV traffic from weight traffic at the memory-controller level; the ratio is a model artifact of the latency fit, and should be cited as such.
Switching from HuggingFace/SDPA to vLLM/FlashAttention-2 changed the fixed-path bandwidth substantially — a 34% improvement, from 151.5 to 205.7 GB/s — but left the KV slope almost untouched: 16.34 ms/GiB versus 17.58 ms/GiB, within 8% of each other. Whatever is limiting KV-cache read throughput on this hardware, it was not eliminated by moving to a faster attention kernel with paged memory. This finding is scoped strictly to the two stacks actually tested here. It is not evidence that "the kernel never matters" in general — only that these two kernels, on this hardware, with this model, produced materially different fixed-path performance and materially similar KV-path performance.
Across every batch × context configuration tested, the 128 GiB unified memory pool had headroom to spare — nothing was evicted, and capacity was not the binding constraint in any run. Throughput fell from 150 tok/s at batch 16 × 4K context to 46 tok/s at batch 16 × 16K context purely as a function of the growing KV-cache read cost per decode step, with memory utilization well under the ceiling the whole time. The bottleneck in these measurements is bandwidth, not capacity.
Both bugs below were caught before any number derived from them was published, and both are included here because catching them is a methodology strength worth showing, not a lapse worth hiding.
The loading script originally passed torch_dtype= to the model constructor. That argument is deprecated in current transformers releases and was silently ignored rather than erroring, which caused the 7B model to load in fp32 — 28.4 GiB of resident weights instead of the expected fp16 footprint. On unified memory, a CUDA-side allocation failure of this kind does not surface as a clean CUDA OOM; it surfaces as a Linux OOM kill, which obscured the actual cause for longer than it should have. The fix was to switch to the current dtype= argument, and the script now prints resident weight bytes and dtype on load so a silent fallback of this kind is visible immediately rather than discovered after a crash.
An early version of the batch × context sweep used identical token ids across every sequence in a batch for simplicity. vLLM's prefix cache correctly recognized the batch as one shared prefix and stored a single copy of the KV-cache instead of one copy per sequence — which meant the actual KV bytes read at each measured point were a factor of B (the batch size) smaller than the script believed, inflating the effective x-axis and producing an apparent KV bandwidth of 342.9 GB/s. That number is above the 273 GB/s physical memory bus on this hardware, which is the tell: no configuration on this machine can exceed its physical bus. The rule this bug produced is worth stating plainly and applying everywhere in this document: any measured bandwidth above the physical bus is a bug, always. Also worth stating plainly: this bug produced a better-looking, more exciting wrong number — a bandwidth figure that would have made the KV path look cheap rather than expensive — and it was not caught by code review or inspection. It was caught by a physical impossibility check against the memory bus spec. That is the check that matters, and it is now run against every bandwidth figure in this document before it is written down.
Stated plainly, so nothing here is oversold: this is one model, one size, one machine. Batches were fixed-length and uniform, with no continuous batching. The vLLM fit is noisier than the HuggingFace fit (R² = 0.945 vs. 0.991), and one point — batch 16 at 16,384-token context — sits 26 ms above the fit line and needs a repeat run before it should be trusted at face value. No thermal log was captured during the sweep, so thermal throttling as a confound cannot be ruled out or ruled in. The attention kernel tested was FlashAttention-2, not FlashAttention-3. And fp8 KV-cache was not measured at all in this pass — it is named here as the next experiment, since it is the direct test of whether cache compression changes the byte-cost ratio described in Section 4.3, rather than just shrinking the KV-cache footprint at a fixed cost per byte.