The field has specialized, not stalled. This chapter maps what each system actually optimizes and where this package's own contribution fits — and does not fit — inside that map.
An earlier draft of this chapter argued that the KV-cache-over-CXL landscape was uniformly unsophisticated — generic LRU or random eviction everywhere, passive memory hardware, and a wide-open gap for a "complete" solution. That framing does not survive contact with what has actually shipped and published in 2026. It should be retired outright, not softened: the systems below are not doing the same coarse thing this package's early framing implied. They have specialized into distinct pieces of a larger problem, each solving a different sub-decision well. The honest question is no longer "does anyone do anything sophisticated with KV cache?" It is whether these specialized pieces should eventually be unified by a higher-level control architecture that jointly decides relevance, reuse, residency, and movement — and if so, what that architecture looks like. Nobody, including this package, has built or tested that unified architecture yet.
Table 10.1 organizes the landscape by the specific decision each system is built to make, rather than by how advanced its eviction policy sounds. Read down the right column first: these are complementary decisions, not competing claims to the same prize.
| System | Primary decision/problem it optimizes |
|---|---|
| vLLM | GPU KV allocation / paging — the baseline serving-framework capability every offload layer sits on top of. |
| LMCache | KV persistence, reuse and movement across tiers and engines. It has grown well past "block-level KV management with CPU/SSD offload" into a first-class KV layer: pinning, lookup, cleanup, movement, compression, batched movement, compute/I·O pipelining, cross-engine cache transfer, and prefill/decode disaggregation. Describing it as a simple offload cache understates it. |
| TraCT | Rack-scale KV sharing and transfer over CXL. TraCT is implemented on NVIDIA Dynamo and performs direct GPU↔CXL operations behind a rack-wide, prefix-aware cache — it is already a CXL-native KV-cache system, not a gap waiting to be filled. |
| PNM-KV | Select which KV/token pages are needed before paying the movement cost, using a processing-near-memory accelerator sitting in or near CXL memory. This is conceptually adjacent to this package's own emerging thinking about a control plane, not a cruder version of it — PNM-KV reports up to 21.9× throughput improvement in its own benchmarks (arXiv:2511.00321), a figure cited here as their reported result, not independently verified. |
| Tutti | Efficient GPU↔NVMe KV object movement. Tutti (arXiv:2605.03375) removes the CPU from the KV data/I·O critical path — the CPU only loads I·O kernels to the GPU once per layer — and uses GPU io_uring plus slack-aware I·O scheduling for asynchronous, GPU-direct object transfer. Against GDS-enabled SSD-backed LMCache, Tutti reports 78.3% TTFT reduction under strict SLO constraints, 2× higher achievable request rate, 27% lower serving cost, and near-DRAM performance at much greater capacity. |
| CacheWise | Predict future KV reuse from agent and tool-call behavior. CacheWise combines prefix-aware scheduling with reuse-aware eviction guided by lightweight predictions from tool-call metadata, rather than a generic recency or frequency policy. In a vLLM implementation over its own collected coding-agent traces, it reports 2–2.6× fewer KV-cache evictions and up to 3.5× faster agent session completion. |
| This package's direction | An open design question, not a shipped system: whether relevance estimation, reuse prediction, residency, and movement can be decided jointly by one control plane instead of separately by each layer above. |
Read this way, the claim that this field handles every KV block with one uniform policy does not hold up. CacheWise treats reuse as a prediction problem informed by agent state. PNM-KV treats selection as a near-memory decision made before movement, not after. LMCache treats the KV object itself as something to be pinned, moved, compressed, and shared across engines and tiers. TraCT treats CXL as rack-scale shared transport with its own prefix cache. Tutti treats the GPU-NVMe path as an I·O-scheduling problem, not a cache-policy problem. None of these reduce to a single generic recency, randomized, or fixed-rule policy — that older characterization no longer describes the field, if it ever fully did.
The pieces above are specialized, but they are not unified. CacheWise's tool-call-driven reuse prediction lives inside vLLM's eviction path; it does not decide where a block should physically reside once predicted useful. PNM-KV's near-memory selection avoids wasted recalls, but selection and cross-tier movement policy are still separate concerns in its design. LMCache spans tiers and engines but does not itself predict future reuse from agent or tool signals the way CacheWise does. TraCT solves rack-wide CXL transport but is not, on its own, a relevance-prediction or selection system. Tutti's contribution is squarely at the GPU↔NVMe movement layer.
The open question this package is now oriented around is whether a higher-level KV-state control plane should sit above these pieces — jointly deciding what is relevant, what will be reused, where it should live, and when it should move — rather than leaving each decision to a separate subsystem tuned in isolation. That is a design question. It has not been built, and this package has not tested it.
The sketch below is offered as exactly that: a design direction. It describes a plausible shape for a unified control plane, built by generalizing the roles the systems in §10.2 already play separately. None of it beyond the persistence sub-result in Chapter 7 has been implemented or measured by this package. It should not be read as a roadmap with committed timelines or as evidence of anything beyond the one narrow result it is built around.
The claim this chapter makes is narrower than earlier drafts implied, and it is the honest one available right now. An earlier draft's positioning of this package as the sole complete CXL-native KV-cache system does not hold — TraCT already is a CXL-native, rack-scale KV-cache system running on Dynamo, and describing this package that way is false on its face. This package also does not claim its per-head eviction-queue structure or its recency/frequency scoring function are novel contributions to a field that lacked sophistication; CacheWise, PNM-KV, LMCache, TraCT, and Tutti each already solve a specific, real, and non-trivial sub-problem in this space, several of them with hardware or trace-driven results this package has not attempted to reproduce. What this package does have is one measured-in-simulation result about metadata persistence across a tier boundary, and a design question — worth taking seriously precisely because the field has specialized enough that the pieces now look like they could compose — about whether a joint control plane over relevance, reuse, residency, and movement is worth building. RoPE-position-window prefetching, in particular, remains exactly that: an open idea, not a validated technique. A [P−W, P+W] prefetch window around a predicted rotary position has not been shown on this package's target workloads to provide a meaningful predictive advantage over the reuse-prediction approaches already shipping elsewhere (CacheWise's tool-call-driven prediction, for instance); it belongs in future-work discussion, not in a list of this package's differentiators.