Figure: a RoPE geometry explainer. RoPE rotates Q and K in d/2 two-dimensional subspaces, each with its own frequency, so the inner product is a sum over those pairs and does not fall off monotonically. The guarantee is an upper-bound envelope: the magnitude of the inner product of q_m and k_n is at most B of the absolute distance m minus n, with B non-increasing. It is never proportional to cosine of (m minus n) times theta — writing it that way converts a bound into a decay law, which is the error this figure previously rested on. Critically, RoPE is a rotation applied to Q and K at compute time. The rotation is a function of position alone, known in advance for every position, and identical whether a block is resident or evicted, so it carries no information about which blocks will be needed next and cannot drive prefetch. The claim that RoPE-aware prefetch contributes plus 4 percentage points of hit rate, taking a cumulative 93 percent to 97 percent, is retracted, along with the entire 72 to 80 to 87 to 93 to 97 ladder, which was never run as an ablation and whose plus 8, plus 7, plus 6, plus 4 increments were assumed additive and independent. No single hit rate exists: across budgets, policies and workloads the simulated range is 8.5 to 91.6 percent. Section 2 shows a distance band around a query position P, from P minus W to P plus W, with W = 5 in the worked strip at P = 55, so positions 50 to 54 and 56 to 60 are inside the band and 48, 49, 61 and 62 outside. That band is where the bound is loosest, not a set of blocks predicted to be read: whether a block is resident is a separate question, decided by the PLACE stage, and a distant position may still attend strongly. Section 3 shows band widths against an assumed attention-mass distribution: W = 32 covers about 50 percent, W = 128 about 75 percent, W = 512 about 90 percent. All three widths and all three coverage figures are uncalibrated assumptions, not measurements — the distribution is illustrative, not a measured histogram, and the coverage is model-, layer-, head- and workload-dependent. A position-window prefetch remains an open idea, not a validated technique; it has not been shown to give a predictive advantage over the reuse-prediction approaches already shipping elsewhere, and it belongs in future work rather than in a list of contributions. Nothing on this page is a cache result. Related published work on which the reframing rests includes H2O, SnapKV, Scissorhands, PyramidKV, Quest, Ada-KV, HeadKV, InfiniGen, LayerKV, KVPR and PagedAttention.

RoPE — A Geometry Explainer

What rotary position embedding bounds, and what it cannot tell you. It is not a prefetch signal.

RoPE cannot drive prefetch — the hit-rate claim on this figure is retracted

RoPE is a rotation applied to Q/K at compute time. The rotation is a function of position alone: known in advance for every position, and identical whether a block sits in HBM or on NVMe. It therefore carries no information about which blocks are needed next. The former claim — RoPE-aware prefetch worth +4 pts of hit rate, 93% → 97% cumulative — is withdrawn with the whole 72→80→87→93→97 ladder, which was never run as an ablation.

Distance decay is an upper bound, |⟨q_m, k_n⟩| ≤ B(|m − n|) — never ∝ cos((m − n)·θ). A position-window prefetch remains an open idea, not a validated technique: it has not been shown to beat the reuse-prediction approaches already shipping (CacheWise's tool-call-driven prediction, for instance), and it belongs in future work.

1
RoPE Bounds Attention Magnitude by Distance
Attention Relative Position (distance from query)
Query Position
Rotary Encoding — what RoPE actually gives
RoPE encodes position by rotating Q/K in d/2 two-dimensional subspaces, each with its own frequency θ_i. The inner product is a sum over those d/2 frequency pairs, so it is not a single cosine of the distance and it does not fall off monotonically. The guarantee is an upper-bound envelope: |⟨q_m, k_n⟩| ≤ B(|m − n|) with B non-increasing. The curve at left sketches that envelope, not a measured attention trace.
Locality Bias — an ASSUMPTION of this model
This design assumes roughly 70% of attention mass falls within ±W positions of the query. That figure is not a universal property of transformers: it is model-, layer-, head- and workload-dependent, and it is stated here as an input assumption, not as an established fact or a measurement.
What it is NOT — ∝ cos((m − n)·θ)
The inner product is never proportional to a single cosine of the distance. Writing it that way turns an upper bound into a decay law, and a decay law reads like a prediction of which positions will be read. It is not one. A distant position may still attend strongly — the bound permits it — and nothing in the rotation says which blocks the next step will touch.
2
The [P − W, P + W] Band — a Bound, Not a Prediction
What the band means
|⟨q_m, k_n⟩| ≤ B(|m − n|),  B non-increasing
|m − n| ≤ W  →  the bound is loosest here
The band marks where a large inner product is permitted, not where one is predicted. Residency is decided separately, by the PLACE stage, from past-access signals — not from position geometry. W is an assumed width, not a fitted one.
Loosest-bound band (2W)
P
Query position
P−W P+W
P−2W
P−W
P
P+W
P+2W
Query position (P=55)
|m − n| ≤ W, W=5 — bound loosest
Outside the band — bound tighter
3
Band Width vs. Assumed Coverage — Illustrative, not empirical
Illustrative — an assumed attention-mass distribution, not a measured histogram · Green = within the band
W = 32
~50% attention captured (assumed)
Low bandwidth, more misses
W = 128
~75% attention captured (assumed)
A judgement call, not a fitted result
W = 512
~90% attention captured (assumed)
High bandwidth overhead
W = 32 / 128 / 512 are uncalibrated design parameters

These three window sizes are not derived and were not selected by experiment. They are round powers of two spanning two orders of magnitude, chosen to show the shape of the trade-off; the coverage percentages beside them (~50% / ~75% / ~90%) are assumptions of the model, not a measured histogram, and W = 128 is labelled "balanced for Llama-70B" as a judgement call, not a fitted result.

What W controls. W sets the radius of the band [P − W, P + W] around a query position, and so the fraction of attention mass that falls inside it under the assumed distribution. It is a property of the bound, not a speculative-fetch budget.

No hit rate is attached to W. The claim that widening W lifts the hit rate — "the rung worth +4 pts, 93% → 97%" — is retracted, and no replacement figure is offered, because the ablation that would have produced one was never run. The tier latency this paragraph once priced the miss tail against is retracted too: no CXL hardware was measured. Tier latency is an assumption across 32 / 64 / 121 GB/s, under which a 5.00 MiB block fetch is 164 / 82 / 43 µs. What the tier buys is avoiding regeneration — forward compute for 16 tokens is 9 / 11 / 18 ms at 50 / 40 / 25% MFU, two to three orders of magnitude above the transfer — and no precise multiple is claimed.

Why the per-head framing does not save it. A head is not an eviction unit. Every head of every layer is required for every token, so evicting a head produces wrong outputs, not a cache miss. Per-head attention statistics are a scoring signal; the placed and moved unit is a (sequence, token block) of 16 tokens = 5.00 MiB. H2O and SnapKV evict (head, token) pairs — the token axis is not optional.

Where the leverage actually is. Not in the position window. At a 32 GiB budget, letting identity-keyed policy state survive HBM eviction is worth +34.15 pts to LFU (15.56 → 49.71) and +24.85 pts to EMA α=0.01 (31.54 → 56.39), while EMA α=0.15 is unchanged (−0.21 pts) — larger than the whole spread between policies, and a protocol result, testable with no CXL hardware. Simulated (provisional) See Appendix K, section K.7 — Uncalibrated parameters.

Analytical model — the bound |⟨q_m, k_n⟩| ≤ B(|m − n|) follows from the RoPE construction. The locality percentages and band-coverage figures are Illustrative assumptions, not measurements. Retracted from this figure: the +4 pts / 93% → 97% prefetch contribution, the 97% hit rate, the whole 72→80→87→93→97 ladder, and the 200 ns CXL access — no CXL hardware was measured. No single hit rate exists; the simulated range is 8.5–91.6% Simulated (provisional), and a single data-structure defect in Revision 1 inverted four of five findings, so Revision 2 is provisional, not settled. Work this reframing builds on, cited not competed with: H2O, SnapKV, Scissorhands, PyramidKV, Quest, Ada-KV, HeadKV, InfiniGen, LayerKV, KVPR, PagedAttention. Canonical numbers v6.0.