Figure: the _pick_victim defect in Revision 1 of the KV tiering simulator. Before the fix, _pick_victim popped a heap entry and re-inserted it only when its recomputed score exceeded the stored key. That is correct for LFU, where counts rise so stored keys are stale-low, and wrong for a decaying EMA, where scores fall so stored keys are stale-high and the popped entry is returned without checking whether another block decayed further. Audited against a brute-force argmin: EMA alpha 0.15 chose 114,114 wrong victims out of 114,528 evictions, a rate of 99.6 per cent; EMA alpha 0.01 chose 73,650 wrong out of 74,987, or 98.2 per cent; LFU chose 0 wrong out of 74,987, or 0 per cent. After the fix, which uses the exact order-preserving key K equals the natural log of S touch plus t touch times minus the natural log of 1 minus alpha, the re-audit found 0 wrong out of 78,860 evictions. The Revision 1 to Revision 2 status table: R1, EMA alpha 0.15 worst at every budget, was reversed; R2, monotone convergence onto LFU, was withdrawn; R3, LRU wins the scan workload, was withdrawn; R4, EMA wins the cyclic sweep 2.2 times, was withdrawn; R5, 97 per cent of stall is recompute, was withdrawn; C1, the 2.50 gibibyte KV correction, stands. The lesson is that Revision 2 is therefore provisional too: it has had no equivalent audit of its remaining assumptions.

Figure 7.3 · Simulated (provisional)

The defect that inverted the study — _pick_victim()

A lazy heap re-evaluation that is correct when scores rise and wrong when they fall. It chose the wrong victim on 99.6% of evictions under EMA α=0.15, and nothing in the output looked broken.

Revision 2 is provisional too. A single data-structure defect inverted four of five findings once already. Nothing about the fix makes the remaining gaps go away: no Belady bound, no bandwidth/queueing model, whole-context residency assumed, synthetic workload, 2–3 seeds, no hardware validation. Revision 1 (kv_tiering_sim_rev1_retained_for_audit.py) is retained unmodified so the defect and its effect stay independently checkable.

Before — the defect

def _pick_victim(self): key, blk = heappop(self.heap) score = self.recompute(blk) if score > key: # ⚠ re-insert ONLY when the heappush(self.heap, # recomputed score EXCEEDS (score, blk)) # the stored key return self._pick_victim() return blk

Correct for LFU. Counts only rise, so a stored key is stale-low; a popped entry whose recomputed score exceeds its key must go back and the search continues.

Wrong for a decaying EMA. Scores only fall, so a stored key is stale-high. The popped entry is returned without ever checking whether another block has decayed further — so the victim is almost never the argmin.

After — the fix

K = ln S_touch + t_touch · ( −ln(1−α) )

An exact, order-preserving key: it orders identically to the current decayed score at any time, so no lazy re-evaluation is needed and the heap min is always the true argmin.

Audited against brute-force argmin

PolicyWrong victimsEvictionsRate
EMA α=0.15114,114114,52899.6%
EMA α=0.0173,65074,98798.2%
LFU074,9870.0%
After the fix (re-audit)078,8600.0%

bug_check.py produced the first three rows; bug_check2.py the fourth. Source: data/ERRATUM.md.

Why it went wrong in opposite directions

LFU — counts risestored keys are stale-LOW: true score ≥ keyb1b2b3b4✓ re-insert when recomputed > key is CORRECTDecaying EMA — scores fallstored keys are stale-HIGH: true score ≤ keyb1b2b3b4✗ the same test returns a victim that is not the argminSchematic — relative heights only, no values are claimed. The audit counts below are the evidence.
Stored heap key — the value written at push timeTrue score now, LFU — higher than the keyTrue score now, decaying EMA — lower than the key

Revision 1 → Revision 2 — how each finding fared

Revision 1 findingStatusRevision 2
R1 — EMA α=0.15 worst at every budgetReversedR1′ — EMA α=0.15 best at 4 of 5 budgets (44.18% vs LRU 38.34% @32 GiB)
R2 — hit rate monotone as α→0, converges exactly onto LFUWithdrawnR3′ — non-monotonic, interior optimum α≈0.05; no convergence onto LFU
R3 — LRU wins the scan workloadWithdrawnEMA α=0.01 leads scan (36.26% vs 31.51%)
R4 — EMA wins the cyclic sweep 2.2×WithdrawnOnly LFU survives loop (22.06%); EMA tracks LRU at 9.83%
R5 — 97% of stall is recompute; policy nearly irrelevantWithdrawnR5′ — warm recompute is 50–67%; policy spread reaches 86% at 128 GiB
C1 — 8K Llama-3 70B KV is 2.50 GiB, not 20 GBStandsUnchanged; analytic, simulator-independent

Quoted verbatim from data/ERRATUM.md. The package’s standing caveat states the same fact in one line: “a single data-structure defect in Revision 1 inverted four of five findings” (CANONICAL-NUMBERS.md §3). Where the erratum and the canonical spec differ — the erratum quotes 55.93% for the α optimum against the spec’s 55.96%, and 50–67% for warm recompute against the spec’s 44–89% — the spec is authoritative: it is computed directly from results_v2.json.

The lesson, and it applies to Revision 2 as well.One data-structure defect, invisible to code review, inverted four of five findings and produced a coherent, publishable, entirely wrong story. It was caught by an independent brute-force audit, not by inspection — the same way the 342.9 GB/s bandwidth bug was caught by a physical bound rather than by reading the code. Revision 2 has had no such audit of its remaining assumptions. Treat it as provisional, not settled.

Simulated (provisional) — data/ERRATUM.md (bug_check.py, bug_check2.py); CANONICAL-NUMBERS.md §3 for the standing caveat