Figure: per-head attention tracking, showing that one cache policy does not fit all attention heads. Four head types at layer 40 are contrasted: the recency head, head 0, looks at the last 50 to 100 tokens; the anchor head, head 1, always attends to the first 50 tokens holding the system instructions; the retrieval head, head 2, attends by semantic similarity anywhere in the context; the syntactic head, head 3, attends to grammatically related tokens. In the worked 128K-context document QA example — system prompt at positions 0 to 100, a 64K-token document, the word "France" at position 64K, and 64K tokens of conversation — the four heads need the last 100 tokens, positions 0 to 100, position 64K, and the recent question words respectively, so the combined decision keeps positions 0 to 100, position about 64K and the last 1000 tokens in HBM and sends everything else to CXL DRAM. A token stays cached if any head still needs it. Per-head tracking contributes plus 6 percentage points of hit rate, 87 percent to 93 percent cumulative. All values are an analytical model, not measured.

Per-Head Attention Tracking

Different attention heads serve different purposes — one cache policy doesn't fit all

Analytical model — the four-way head taxonomy (Recency / Anchor / Retrieval / Syntactic) and the head shares used elsewhere in this package (40 / 15 / 25 / 20) are a modeling assumption motivated by the induction-head and retrieval-head literature, not a measured head census. Per-head tracking contributes +6 percentage points of hit rate (87% → 93% cumulative). Canonical numbers v4.0.