Figure: the competitive landscape for KV-cache offloading, in four bands plus an industry timeline. Commercial hardware available today: Samsung CMM-D up to 512 gigabytes per module and CMM-B up to 16 terabytes per box at about 60 gigabytes per second and roughly 600 nanoseconds loaded; the XConn Apollo 256-lane CXL switch with MemVerge GISMO and about 100 terabyte pool demos; the Astera Labs Leo smart memory controller; and Marvell Structera near-memory compute. Emerging software frameworks and academic papers place cache policy in the serving framework, the GPU model runtime, or the host CPU. Where cache policy lives today, in four cards: the serving framework, meaning vLLM, SGLang with RadixAttention, NVIDIA Dynamo and LMCache; the model runtime, meaning H2O, SnapKV, Scissorhands, PyramidKV, Quest, Ada-KV and HeadKV; the host CPU, meaning FlexGen, InfiniGen and Mooncake; and the CXL device, meaning Astera Leo, Marvell Structera and Samsung CMM. None of them does attention-aware retention in controller firmware. The combination proposed here is four parts: 640 GQA-matched queues, EMA score tracking, a RoPE prefetch window of P minus W to P plus W, and policy execution in CXL endpoint firmware below the serving framework. Two things about that combination have been corrected. First, the scoring function is not novel: a decayed-frequency score is LRFU, published by Lee and colleagues at SIGMETRICS in 1999, and alpha is its documented knob between LRU and LFU behaviour, so the claim has to rest on placement and on the protocol, not on the score. Second, the alpha of 0.2 previously quoted here was never validated; the simulated optimum is alpha of about 0.05, giving 55.96 percent at a 32 gibibyte budget against LRU at 36.85 percent, which is plus 19.1 percentage points, while at the specified alpha the margin over LRU is only plus 2.5 to plus 5.8 points rather than the plus 25 previously claimed. The modelled 97 percent HBM hit rate and the 16 concurrent users at 128K context on one B200 that this figure used to promise are both retracted. There is no single hit rate: the corrected simulation gives 8.5 to 91.6 percent depending on HBM budget, policy and workload, and at a 32 gibibyte budget LRU gives 38.34, LFU 16.09, EMA alpha 0.15 gives 44.18 and EMA alpha 0.01 gives 35.91. Those values are provisional rather than settled, because a single data-structure defect in Revision 1 of the simulator inverted four of five findings. The 16 times user capacity was downstream of the 97 percent and is withdrawn outright with no replacement claim, and the 36 percent CapEx reduction that priced a configuration justified by it is withdrawn too, with no new cost figure offered. What survives is the capacity arithmetic, which never depended on a hit rate: 320 kibibytes per token, 2.50 gibibytes at 8K, 40.0 gibibytes at 128K, and a single-sequence crossover against the weights at about 427,000 tokens. The timeline runs 2022 to 2026, colour-coded green for hardware, amber for software, purple for research and blue for standards.

KV-Cache Offloading: Competitive Landscape

Where KV-cache policy lives today, and the narrow slice this work adds

Evidence labels: third-party capabilities are vendor specifications or published claims; prior-art mechanisms are external literature; capacity sizing is an analytical model over stated inputs; hit rates and policy margins are simulated (provisional) — Revision 2 of kv_tiering_sim_v2.py, 2–3 seeds, no Belady bound, no bandwidth or queueing model, no hardware validation, and provisional rather than settled because a single data-structure defect in Revision 1 inverted four of five findings. The only measured data in this package is on a DGX Spark (GB10, 19 Aug 2026): achieved streaming bandwidth 236.5 GB/s = 87% of the 273 GB/s spec, which retires the ~50% effective-bandwidth haircut earlier drafts applied. No CXL hardware was measured. The 97% hit rate, the 16× user capacity, the 65× access-latency headline and all cost figures including the 36% CapEx reduction are retracted — see RETRACTIONS.