Figure: a logarithmic time axis running from 10 microseconds to 100 milliseconds, comparing movement against regeneration for a single 5.00 mebibyte KV block, which is 16 tokens of Llama-3 70B under GQA at 320 kibibytes per token. The movement bar spans 43 microseconds at 121 gigabytes per second, through 82 microseconds at 64 gigabytes per second, to 164 microseconds at 32 gigabytes per second. The regeneration bar spans 9 milliseconds at 50 per cent MFU, through 11 milliseconds at 40 per cent MFU, to 18 milliseconds at 25 per cent MFU, for 16 tokens of forward compute. A labelled span between the two reads two to three orders of magnitude. A third, open-ended hatched bar shows that regenerating an arbitrary interior block costs at least 18 milliseconds and potentially far higher, because interior KV is not independently reproducible: hidden states depend recursively on all preceding context. No precise multiple is claimed, because the two quantities do not measure the same operation; 218 times, from 17.9 milliseconds divided by 82 microseconds, is one point inside the range and must always be quoted with those inputs. This is an analytical model. No CXL hardware was measured.

Figure 5.1 · Analytical model

Moving a KV block against regenerating it — per 5.00 MiB block, log scale

Both sides are ranges, because both rest on assumptions rather than measurements. The gap between them is the structural argument for tiering KV state rather than discarding and recomputing it.

Analytical model — no CXL hardware was measured. The three tier bandwidths are assumptions (121 GB/s CXL 3.0 ×16 theoretical one direction; 64 GB/s mid-range; 32 GB/s pessimistic), and the three compute costs rest on assumed MFU. Neither side is a measurement.
10 µs100 µs1 ms10 ms100 mstime per 5.00 MiB block — log scaletwo to three orders of magnitudeMOVE the block across the tier boundary121 / 64 / 32 GB/s — no CXL hardware was measured43 µs164 µs82 µsREGENERATE it — forward compute, 16 tokens50% / 40% / 25% MFU9 ms18 ms11 msREGENERATE an arbitrary interior blockhidden states depend recursively on all preceding context≥ 18 ms
Movement — fetch a 5.00 MiB block across the tier boundaryRegeneration — re-run the forward pass for those 16 tokensOpen-ended — interior block, floor only
Table view
OperationAssumptionCost
Tier fetch, 5.00 MiB121 GB/s (CXL 3.0 ×16 theoretical, one direction)43 µs
Tier fetch, 5.00 MiB64 GB/s (mid-range — unmeasured)82 µs
Tier fetch, 5.00 MiB32 GB/s (pessimistic)164 µs
Forward compute, 16 tokens50% MFU9 ms
Forward compute, 16 tokens40% MFU11 ms
Forward compute, 16 tokens25% MFU18 ms
Regeneration, arbitrary interior blockrecursive dependency on all preceding context≥ the above, potentially far higher
2–3 orders
of magnitude cheaper to move a block than to regenerate it. This is the claim.
no precise multiple
is claimed. The two quantities do not measure the same operation.
218×
is one point in that range — 17.9 ms ÷ 82 µs — and must always be quoted with those two inputs.

Claim the order of magnitude, not the digit.Pick the optimistic ends of both ranges and the ratio is nearer two orders of magnitude; pick the pessimistic ends and it is nearer three. Any single multiple is an artifact of which two assumptions were chosen. And the regeneration side is a floor: an arbitrary interior block may cost far more, because interior KV is not independently reproducible — a hidden state at position i depends recursively on every token from 0 through i, through every attention layer.

Analytical model — CANONICAL-NUMBERS.md §4. No CXL hardware was measured.