Figure: HBM hit rate against HBM budget for four eviction policies, from experiment e1 of a trace-driven KV tiering simulation, two seeds each, no persistent metadata and no transfer-compute overlap. Mean hit rates in per cent, with the two seed values in brackets. At 8 GiB: LRU 8.52 (8.04, 9.01), LFU 8.76 (7.57, 9.95), EMA alpha 0.15 12.35 (12.70, 12.01), EMA alpha 0.01 12.12 (11.08, 13.17). At 16 GiB: LRU 17.87 (17.16, 18.57), LFU 12.70 (11.17, 14.22), EMA alpha 0.15 22.58 (23.28, 21.88), EMA alpha 0.01 18.70 (16.54, 20.86). At 32 GiB: LRU 38.34 (36.85, 39.83), LFU 16.09 (15.56, 16.62), EMA alpha 0.15 44.18 (44.33, 44.04), EMA alpha 0.01 35.91 (31.54, 40.29). At 64 GiB: LRU 67.12 (64.92, 69.31), LFU 23.00 (24.49, 21.50), EMA alpha 0.15 69.65 (67.99, 71.31), EMA alpha 0.01 66.83 (64.27, 69.40). At 128 GiB: LRU 88.01 (87.74, 88.28), LFU 32.70 (32.49, 32.91), EMA alpha 0.15 88.14 (87.86, 88.41), EMA alpha 0.01 91.59 (91.16, 92.02). The honest reading: the policies are close, LFU collapses, and the EMA lead over LRU is +3.8, +4.7, +5.8, +2.5 and +0.1 points across the five budgets — a few points, not tens of points. The seed spread is often as large as the gap between policies. These are simulated provisional results, not measurements.

Figure 11.1 · Simulated (provisional)

Hit rate against HBM budget — the policies are closer than the story wanted

Experiment e1: Llama-3 70B GQA, 16-token blocks, 200 sessions, Zipf(1.1) return frequency, 2500 turns, 421 GiB total footprint, HBM tier over a 512 GiB CXL tier. No persistent metadata, no transfer/compute overlap, two seeds — both plotted, because at this scale the seed spread is often as wide as the gap between two policies.

Simulated (provisional). Revision 2 of this simulation is provisional, not settled: no Belady bound, no bandwidth or queueing model, whole-context residency assumed, synthetic workload, two seeds, and no hardware validation. A single data-structure defect in Revision 1 inverted four of five findings. Nothing on this chart is a measurement. See Appendix N for the standing caveat.
LRULFUEMA α=0.15EMA α=0.01whisker = the two seeds; dot = their mean

EMA α=0.15 leads at four of the five budgets and EMA α=0.01 leads at 128 GiB. The margin over LRU is +2.5 to +5.8 points, not the +25 points version 4.0 claimed — that figure is retracted. LFU is the one clear separation on the chart, and it separates downwards: it reaches 32.70% at 128 GiB where every other policy is near 90%. That is classical LFU aging once decode growth is modelled. At 128 GiB the LRU and EMA α=0.15 curves are 0.13 points apart, which is well inside the seed spread, so the chart does not distinguish them.

Show the data table (mean, and both seeds)
Mean hit rate per cent, with the two individual seed values in brackets.
HBM budgetLRULFUEMA α=0.15EMA α=0.01
8 GiB8.52 (8.04 / 9.01)8.76 (7.57 / 9.95)12.35 (12.70 / 12.01)12.12 (11.08 / 13.17)
16 GiB17.87 (17.16 / 18.57)12.70 (11.17 / 14.22)22.58 (23.28 / 21.88)18.70 (16.54 / 20.86)
32 GiB38.34 (36.85 / 39.83)16.09 (15.56 / 16.62)44.18 (44.33 / 44.04)35.91 (31.54 / 40.29)
64 GiB67.12 (64.92 / 69.31)23.00 (24.49 / 21.50)69.65 (67.99 / 71.31)66.83 (64.27 / 69.40)
128 GiB88.01 (87.74 / 88.28)32.70 (32.49 / 32.91)88.14 (87.86 / 88.41)91.59 (91.16 / 92.02)

Simulated (provisional) — results_v2.json experiment e1; see Appendix N for the standing caveat