Figure: per-head attention patterns as scoring signals, shown as synthetic 5 by 5 attention matrices over the tokens "The model processes each token" for four assumed head types. Head 0.3 is a recency head, assumed about 40 percent of heads, attending to the last 50 to 200 positions and showing a strong diagonal. Head 1.7 is a syntactic head, about 20 percent, tracking grammatical dependencies so that "processes" attends strongly to its subject "model". Head 2.4 is an anchor head, about 15 percent, attending to the opening positions holding the system prompt regardless of how far generation has advanced. Head 3.11 is a retrieval head, about 25 percent, content-based rather than positional and spiking on whichever earlier tokens are relevant. These differences make heads useful as scoring signals, and useless as eviction units. Every head of every layer is required for every token, so evicting a head produces wrong outputs, not a cache miss. The unit that is placed, moved and evicted is a (sequence, token block) of 16 tokens, 5.00 mebibytes at 320 kibibytes per token; the four head signals are combined into one score per block, and every head of a resident block is kept. H2O and SnapKV evict (head, token) pairs — the token axis is not optional, and token dropping is a lossy relaxation carrying an accuracy-evaluation burden, distinct from lossless residency. The four-way taxonomy and the 40, 15, 25, 20 percent shares are a modeling assumption, not a head census, and the matrices are synthetic illustrations. The earlier claim that per-head tracking contributes plus 6 percentage points of hit rate, 87 percent to 93 percent cumulative, is retracted with the whole 72 to 80 to 87 to 93 to 97 ladder, which was never run as an ablation. No single hit rate exists: the simulated range is 8.5 to 91.6 percent across budgets, policies and workloads, and it is provisional because a single data-structure defect in Revision 1 inverted four of five findings. Work built on, cited not competed with: H2O, SnapKV, Scissorhands, PyramidKV, Quest, Ada-KV, HeadKV, InfiniGen, LayerKV, KVPR, PagedAttention.
Per-Head Attention as Scoring Signals
What each head is assumed to track — Recency · Anchor · Retrieval · Syntactic. A scoring granularity, never an eviction one.
Every head of every layer is required for every token. Evicting a head produces wrong outputs, not a cache miss. The placed, moved and evicted unit is a (sequence, token block) — 16 tokens, 5.00 MiB at 320 KiB/token — and every head of a resident block is kept. H2O and SnapKV evict (head, token) pairs: the token axis is not optional, and token dropping is a lossy relaxation with an accuracy-evaluation burden, not the same claim as lossless residency. External literature
ASSUMPTION, not a measured result. The four-way taxonomy (Recency / Anchor / Retrieval / Syntactic) and the 40 / 15 / 25 / 20 share used throughout this package are a modeling assumption adopted to make per-head cache budgets tractable. They are motivated by the induction-head and retrieval-head literature (and by per-head budget work such as Ada-KV and HeadKV), but no head census was run on Llama-2-70B for this work. The matrices below are synthetic illustrations, not model outputs.