A narrow, evidence-tagged claim: identity-keyed eviction-policy metadata should persist across tier eviction rather than reset. Backed by a hardware-measured bandwidth decomposition on real silicon and a simulated policy comparison — not by a new eviction algorithm, a validated CXL system, or a cost model.
Every number below carries its evidence class on the tile. None of these are cost, TCO, or end-to-end throughput claims.
When eviction-policy metadata (recency/frequency counters keyed to KV-block identity) is allowed to persist across a tier eviction and reload — instead of resetting to cold state — hit rate improves substantially more than switching between eviction policies does. On DGX Spark GB10, hardware measurement shows KV-cache traffic costs roughly 3.4× the bytes of the fixed weight path per decode step at the tested configuration, and streaming reads sustained 87% of the platform's advertised bandwidth. Simulation (not hardware) shows that persisting LFU metadata across eviction, under a 32 GiB HBM budget, lifts hit rate by 34.15 percentage points versus resetting it — a larger effect than swapping LFU for another policy. The scoring approach itself is not new: it follows LRFU (Lee et al., SIGMETRICS 1999). What's being argued is where the metadata should live relative to the eviction boundary.
The field has specialized, not stalled. Each system below optimizes a different decision; they are complementary, not competing claims to the same prize. Vendor-published figures are flagged explicitly.
| System | Primary decision/problem it optimizes |
|---|---|
| vLLM | GPU KV allocation / paging — the baseline serving-framework capability every offload layer sits on top of. |
| LMCache | KV persistence, reuse and movement across tiers and engines. Now a first-class multi-tier KV layer — GPU/CPU/storage/network, pinning, lookup, cleanup, movement, compression, cross-engine transfer, and prefill/decode disaggregation. "Block-level, offloads to CPU/SSD" understates what it does today. |
| TraCT | Rack-scale KV sharing and transfer over CXL. Implemented on NVIDIA Dynamo with direct GPU↔CXL operations behind a rack-wide, prefix-aware cache — already a CXL-native KV-cache system, not a gap waiting to be filled. |
| PNM-KV (arXiv:2511.00321) | Selects which KV/token pages are needed near CXL memory before paying the movement cost — selection before movement. Reports up to 21.9× throughput their reported figure. |
| Tutti (arXiv:2605.03375) | GPU-centric KV movement: removes the CPU from the critical I/O path via GPU-direct object I/O (GPU io_uring) and slack-aware scheduling. Vs. GDS-enabled SSD-backed LMCache: 78.3% TTFT reduction, 2× request rate, 27% lower cost. |
| CacheWise | Predicts future KV reuse from tool-call metadata for agent workloads. Reports 2–2.6× fewer evictions and up to 3.5× better agent-session completion time. |
| This package's direction | An open design question, not a shipped system: whether relevance estimation, reuse prediction, and residency/movement decisions should be unified by one control plane. The evidence here — persistent identity-keyed metadata surviving tier eviction, +34.15 pts, simulated/provisional — speaks to one piece of that picture (residency: does policy state survive tier movement), not to the whole. It does not beat or subsume the six systems above. |
The honest read: this field specialized into distinct, non-trivial pieces rather than leaving an obvious gap. The open question is whether a higher-level KV-state control plane should unify relevance estimation, reuse prediction, and residency/movement decisions across those pieces. This package's evidence currently speaks only to the residency piece.
Full methodology, hardware setup, simulation parameters, and caveats are in the package itself.
Read the full technical package →