Figure: GQA per-head attention as scoring streams over Llama-3 70B, where 64 query heads grouped 8 to 1 give 8 KV heads, and 8 KV heads across 80 layers give 640 per-(layer, KV-head) score streams. Query heads Q0 to Q7 map to KV-head 0 and score stream [layer, 0]; Q8 to Q15 map to KV-head 1 and stream [layer, 1]; and so on to Q56 to Q63 mapping to KV-head 7 and stream [layer, 7]. These 640 streams are a scoring granularity, not an eviction granularity. Every head of every layer is required for every token, so evicting a head produces wrong outputs, not a cache miss, and no head has its own eviction queue. All 640 streams are aggregated into a single score per (sequence, token block) — 16 tokens, 5.00 mebibytes at 320 kibibytes per token — and that block is the unit that is placed, moved and evicted, with all 8 KV heads across all 80 layers travelling together. H2O and SnapKV evict (head, token) pairs; the token axis is not optional, and token dropping is a lossy relaxation with an accuracy-evaluation burden, distinct from lossless residency. As an example, the aggregated block scores order hot to cold: block 7993 covering positions 127,888 to 127,903 with 847 touches and score 0.92; block 0 covering positions 0 to 15 with 512 touches and 0.88; block 64 covering 1,024 to 1,039 with 234 touches and 0.45; block 5563 covering 89,008 to 89,023 with 89 touches and 0.31; and block 2854 covering 45,664 to 45,679 with 12 touches and 0.08, which is the eviction candidate. When that block is evicted its identity key and score persist and are restored on re-admission rather than reset — the largest effect in the study, worth plus 34.15 percentage points to LFU, from 15.56 to 49.71 percent, and plus 24.85 to EMA alpha 0.01, from 31.54 to 56.39 percent, at a 32 gibibyte budget, while EMA alpha 0.15 is unchanged at minus 0.21 percentage points. That is larger than the spread between policies, and it is a protocol result testable with no CXL hardware. Per-block state is 64 bytes: a 16-byte identity key, an 8-byte score, an 8-byte touch timestamp, a 4-byte touch count and reserved space. At 128K context there are 131,072 divided by 16 equals 8,192 blocks, so 8,192 times 64 bytes equals 524,288 bytes, or 512 kibibytes, against 40.0 gibibytes of KV — a payload overhead of 0.0012 percent, the same ratio as 64 bytes against one 5.00 mebibyte block. Metadata processing overhead is not established and must be measured. Simulated figures are provisional, because a single data-structure defect in Revision 1 inverted four of five findings.

GQA Per-Head Scoring Streams

Llama-3 70B: 64 query heads → 8 KV heads × 80 layers = 640 score streams, aggregated into one score per (sequence, token block)

640 scoring streams — not 640 eviction queues

Every head of every layer is required for every token. Evicting a head produces wrong outputs, not a cache miss, so no head has its own eviction queue. The 640 streams are a scoring granularity; they are aggregated into one score per (sequence, token block) — 16 tokens, 5.00 MiB — and the block is what is placed, moved and evicted, with all 8 KV heads × 80 layers travelling together.

H2O and SnapKV evict (head, token) pairs — the token axis is not optional. Token dropping is a lossy relaxation carrying an accuracy-evaluation burden; it is a different claim from lossless residency. External literature

1. GQA Grouping Defines the Scoring Unit
KV-Head 0
Score stream [layer, 0]
KV-Head 1
Score stream [layer, 1]
KV-Head 7
Score stream [layer, 7]
2. 640 Score Streams — all feeding one block score
Layer 0
Layer 1
Layer 2 ... 79
3. What the Aggregated Score Orders — blocks, not heads
Aggregated block score — all 640 streams combined
One ordering per sequence, over blocks
EVICT
CANDIDATE
state persists
Per-block state (64 bytes)
identity key — 16 B (survives eviction)
score — 8 B
touch timestamp — 8 B
touch count + reserved — 32 B
8,192 blocks × 64 bytes = 524,288 B = 512 KiB against 40.0 GiB of KV  →  0.0012% payload overhead (the same ratio as 64 B against one 5.00 MiB block)
Why 640 streams but no 640 queues? Llama-3 70B has 80 layers × 8 KV heads (64 Q heads grouped 8:1 by GQA), so there are 640 places to read attention from. There are not 640 places to evict from: every head of every layer is needed for every token, so all 640 streams collapse into one score per block.
Why 8,192 blocks? A 128 K context is 217 = 131,072 positions; at 16 tokens per block that is 131,072 ÷ 16 = 8,192 blocks, each 5.00 MiB at 320 KiB/token, totalling 40.0 GiB. Block granularity is what makes the state small.
Why it is affordable — and what is not yet known. 512 KiB against 40.0 GiB is 0.0012%, so the payload overhead is negligible. Metadata processing overhead is not established and must be measured.
What the state buys. Keeping it identity-keyed and persistent across an eviction — rather than re-admitting a block as if newly created — is the largest effect in the study: LFU 15.56 → 49.71 (+34.15 pts), EMA α=0.01 31.54 → 56.39 (+24.85 pts) at 32 GiB, EMA α=0.15 unchanged (−0.21 pts). A protocol result — no CXL hardware required. Simulated (provisional)

Analytical model — stream counts, block counts and state sizes are derived from the model configuration (80 layers, 8 KV heads, head_dim 128, fp16 → 320 KiB/token, 5.00 MiB blocks, 40.0 GiB at 128K). The score ordering shown is Illustrative, never a result. The persistence deltas are Simulated (provisional): one data-structure defect in Revision 1 inverted four of five findings, so Revision 2 is provisional, not settled. Retracted from this figure: per-head eviction queues and the 640 MiB / 1.5% metadata footprint that followed from tracking every position in every head. Work built on, cited not competed with: H2O, SnapKV, Scissorhands, PyramidKV, Quest, Ada-KV, HeadKV, InfiniGen, LayerKV, KVPR, PagedAttention. Canonical numbers v6.0.