Figure: two hierarchies drawn side by side with the KV-state control plane between them. On the left, the information hierarchy answers the question what is worth considering, and runs downward through five levels: agent and model state, carrying task, conversation and tool-call context; context-region summaries, coarse summaries over spans; candidate regions, spans judged plausibly relevant; KV blocks, where 16 tokens is 5.00 mebibytes at 320 kibibytes per token; and token-level KV, the individual key and value pairs. On the right, the physical hierarchy answers the question where should it live, and runs downward through three tiers: HBM, the fastest, smallest and most expensive; host or CXL DRAM, larger and higher latency than HBM; and NVMe or remote storage, the largest, cheapest and highest latency. Between them sits the KV-state control plane containing SELECT, which reads the information hierarchy, PLACE, which writes the physical hierarchy, and MOVE, which executes the crossing. An arrow labelled relevance runs from the information hierarchy into the control plane, and an arrow labelled placement runs from the control plane into the physical hierarchy. The two hierarchies are orthogonal: rank in one does not determine rank in the other. A highly relevant block can sit cold in NVMe; a block of marginal relevance can sit resident in HBM simply because nothing has evicted it yet.
Figure 3.1 · Illustrative — proposed architecture
Two orthogonal hierarchies, and the control plane that connects them
“What is worth considering?” and “where should it live?” are different questions with different answers. v5 scored them as one number. v6.0 does not.
Why they do not collapse into one
A block’s rank in the information hierarchy — how relevant it is, how likely it is to be reused — is a separate question from its rank in the physical hierarchy, which is simply which tier currently holds it.
High relevance, cold tier. A highly relevant block can sit in NVMe waiting to be fetched.
Low relevance, hot tier. A block of marginal relevance can sit resident in HBM simply because it was placed there recently and nothing has evicted it yet.
v5 collapsed both into one eviction score: a single “keep or evict” number implicitly answered both is this worth keeping around and should this occupy the expensive tier, as though a positive answer to one settled the other. It does not.
What the control plane does with each
| Stage | Reads | Writes |
|---|---|---|
| SELECT | information hierarchy | the candidate set |
| PLACE | the candidate set | physical hierarchy |
| MOVE | placement intent | the transfer itself |
The information hierarchy decides what matters; the physical hierarchy decides where it can live. Separating relevance and selection from placement and movement is the change v6.0 makes, and it is what makes a joint control plane — rather than one flat scoring function — the right shape for the problem.
Block granularity is the one rung with a hard number attached: 16 tokens = 5.00 MiB at 320 KiB/token for Llama-3 70B under GQA.
Illustrative (proposed architecture) — ARCHITECTURE-v6.md §1; CANONICAL-NUMBERS.md §2 for the 5.00 MiB block