This chapter is the architectural center of this package. It replaces the v5 framing — a single eviction-policy mechanism, headlined by "host owns policy; device owns mechanism" — with a control plane that separates two questions v5 had collapsed into one: what is worth keeping in consideration, and where should it physically live. Everything downstream in this package, including Chapter 7's simulation and Chapter 10's landscape survey, refers back to the architecture defined here.
v5's limitation was structural, not just a matter of degree: it treated "what to keep" and "where to put it" as a single eviction-policy decision, scored once per block and acted on once per block. That conflation is the thing this chapter fixes. A KV block's position in two separate hierarchies determines what happens to it, and the two hierarchies answer different questions and can move independently of each other.
The information hierarchy answers "what is worth considering?" — a question about relevance and likely reuse, independent of physical location:
The physical hierarchy answers "where should it live?" — a question about tier placement and transfer, independent of whether the content is currently useful:
These two hierarchies are orthogonal. A block's rank in the information hierarchy — how relevant it is, how likely it is to be reused soon — is a separate question from its rank in the physical hierarchy — which tier currently holds it. A highly relevant block can sit cold in NVMe waiting to be fetched; a block of marginal relevance can sit resident in HBM simply because it was placed there recently and nothing has evicted it yet. v5's single eviction score conflated these: a block's one "keep or evict" number was implicitly answering both "is this worth keeping around" and "should this occupy the expensive tier," as though a positive answer to one settled the other. It does not. Separating relevance/selection from placement/movement is the change this chapter makes to the architecture, and it is what makes a joint control plane — rather than one flat scoring function — the right shape for the problem.
The control plane sits between the two hierarchies. It consumes relevance and reuse signals from model/agent state and resolves them into three joint but distinct concerns, each answering a different sub-question, before the model actually attends to anything resident:
Unpacking the three concerns inside the control plane box:
| Stage | What it decides |
|---|---|
| SELECT | Which candidate regions or blocks are relevant enough to keep considering at all. This is new relative to v5 — v5 had no explicit selection stage; it went straight to an eviction decision that bundled selection and placement together. |
| PLACE | Which physical tier a block that survives SELECT should occupy — HBM, host/CXL DRAM, or NVMe/remote. |
| MOVE | How and when to actually transfer a block between tiers, and whether that transfer can be hidden behind ongoing compute rather than stalling the turn. |
| ATTEND | The model consumes whatever is currently resident in HBM. This stage does not query the control plane; it acts on the control plane's prior decisions. |
| OBSERVE | What actually got attended to, and what got reused, is recorded and fed back as a relevance/reuse signal for the next round of SELECT. |
SELECT is the genuinely new stage. Where v5 folded relevance and residency into one eviction score, this architecture asks first whether a block is worth considering at all — a question that can be answered from agent/model state before any tier-placement decision is made. PLACE and MOVE then operate only on what SELECT has already judged worth considering, and MOVE in particular carries the question of whether a transfer's latency can be overlapped with compute rather than incurred on the critical path.
This package's strongest empirical result — identity-keyed policy state surviving eviction rather than being reset, +34.15 points, simulated and provisional, measured in Chapter 7 — is evidence for exactly one mechanism inside this larger architecture: the residency sub-question inside PLACE, specifically whether persisting policy state across a tier crossing beats resetting it. It shows that residency state can matter more than which eviction-scoring policy is chosen. It does not validate SELECT, it does not validate MOVE, and it does not validate the joint architecture — SELECT, PLACE, and MOVE acting together — as a system. Treating it as evidence for more than that would be an overclaim this chapter deliberately avoids.
| Piece of the architecture | Status | Basis |
|---|---|---|
| PLACE — residency persistence (metadata survives eviction vs. reset) | Has evidence | Simulated / provisional — Chapter 7, §7.5, +34.15 pts, Revision 2 of a trace-driven simulator |
| SELECT — relevance/reuse-signal-driven candidate filtering | Unvalidated proposed design | No simulation or hardware result in this package |
| PLACE — deadline-aware tier assignment beyond the persistence mechanism | Unvalidated proposed design | No simulation or hardware result in this package |
| MOVE — transfer scheduling, hiding movement behind compute | Unvalidated proposed design | Analytical only (Chapter 7, §7.8, idealized-overlap model) |
| SELECT + PLACE + MOVE acting jointly, as one architecture | Unvalidated proposed design | Not built or tested end to end anywhere in this package |
The relevance and reuse signals that feed SELECT are not all the same kind of evidence, and this package's own evidence base covers only one kind of them. It is worth naming the distinction explicitly here, because Chapter 7 applies it directly and Chapter 10 uses it to place this package relative to the field.
This is exactly what v5's simulation tested (Chapter 7). All of it looks backward at what has already happened to a block; none of it uses information about what is about to happen.
This is the category CacheWise-class systems use, per Chapter 10 — reuse prediction informed by what the agent is doing, not just what a block's history looks like.
This package's evidence base — the +34.15-point persistence result — covers past-only signals exclusively: the metadata being persisted is a recency/frequency-style score, and the workload driving it is trace-replay, not live agent state. Predictive signals for SELECT are an open, unvalidated direction. Nothing in this package has tested whether tool-call state, prefix identity, or attention signals make SELECT more accurate than a past-only score would; that remains proposed design, not measured result.
v5 made this split — decision logic stays with host/controller software, while a narrower device-side mechanism preserves metadata bytes and moves them atomically across tier crossings — its headline framing for the whole architecture. It is retained here, but demoted to its correct scope: it is a design note about who executes the residency-persistence operation inside PLACE and MOVE, not a description of the architecture as a whole. The host still decides what belongs where; the device-side mechanism still just carries metadata across a crossing without losing it. That division is real and specific, and it is one implementation detail of PLACE — not, as v5 had it, the chapter's thesis.
This chapter replaces a single conflated eviction-policy decision with a control plane that separates two orthogonal hierarchies — information (what is worth considering) and physical (where it should live) — into three explicit, jointly-reasoned stages: SELECT, PLACE, and MOVE, feeding ATTEND and closing the loop through OBSERVE. It is a proposed architecture, not a validated one; §3.3's table states exactly what has evidence (residency persistence inside PLACE, simulated and provisional) and what remains proposed design (SELECT, deadline-aware PLACE beyond persistence, MOVE, and the joint system). The signal taxonomy in §3.4 marks a second boundary of the same kind: this package's evidence covers past-only signals only, leaving predictive signals as future work. The chapters that follow — Chapter 7's simulation and Chapter 10's landscape survey — operate inside this frame and refer back to it rather than restating it.