Figure: the EMA update and eviction-priority calculation worked end to end. With alpha = 0.2, the implementation default giving a half-life of ln0.5 over ln0.8 = 3.1 decode steps, score[t] = 0.2 x attention[t] + 0.8 x score[t-1] starting from an initial EMA of 0.0 at token position 1024. The five decode steps run: new score 0.45 gives 0.090; 0.12 gives 0.096; 0.30 gives 0.137; 0.08 gives 0.126; 0.35 gives 0.171. Eviction priority is (1 - score_ema) x recency_decay, where recency_decay = 1 - exp(-beta x steps_since_access) with beta = 0.001. The coefficient beta = 0.001 is an uncalibrated design parameter, not a derived or measured value: it sets the timescale over which idleness becomes a reason to evict, a characteristic time of 1 over beta = 1,000 decode steps, about 100 seconds at the canonical 10 tokens per second. Raising beta makes eviction more aggressively recency-driven and puts long-lived but rarely touched tokens such as system prompts at risk; lowering it lets stale entries linger and consume capacity. Beta interacts with alpha and must be swept jointly against real attention traces. Appendix K section K.7 lists it with the package's other uncalibrated parameters. Token A at position 1024, score_ema 0.171 and accessed 50 steps ago, has recency_decay 0.049 and priority 0.041, so it is kept; token B at position 45678, score_ema 0.08 and accessed 2000 steps ago, has recency_decay 0.865 and priority 0.796, so it is evicted. Higher priority means evict sooner. All values are an analytical model, not measured.
EMA + Eviction Priority
Step-by-step calculation with concrete numbers
# α = 0.2 (implementation default) → half-life ln0.5/ln0.8 = 3.1 decode steps
| Step | new_score | Calculation | score_ema |
|---|---|---|---|
| t₁ | 0.45 | 0.2 × 0.45 + 0.8 × 0.000 | 0.090 |
| t₂ | 0.12 | 0.2 × 0.12 + 0.8 × 0.090 | 0.096 |
| t₃ | 0.30 | 0.2 × 0.30 + 0.8 × 0.096 | 0.137 |
| t₄ | 0.08 | 0.2 × 0.08 + 0.8 × 0.137 | 0.126 |
| t₅ | 0.35 | 0.2 × 0.35 + 0.8 × 0.126 | 0.171 |
# Higher priority = evict sooner
# β = 0.001 (uncalibrated design parameter), decays over ~1000 steps
The recency-decay coefficient is not derived from anything and was not fitted to data. It is stated here as an assumption.
What it controls. β sets the timescale over which "not accessed recently" becomes a reason to evict. In 1 − e−β·steps the characteristic time is 1/β = 1,000 decode steps, so a token reaches ~63% decay at 1,000 steps and ~86% at 2,000. At the canonical 10 tok/s per user that is about 100 seconds of wall-clock idleness before a token is treated as fully stale.
Which way it pushes. Raising β shortens that horizon and makes the policy more aggressively recency-driven: idle tokens are surrendered sooner, HBM and CXL DRAM are reclaimed faster, and long-lived-but-rarely-touched entries — system prompts, few-shot exemplars, anchor tokens — start losing to whatever was touched last, which is exactly the failure the +8-point anchor-pinning rung exists to prevent. Lowering β lengthens the horizon toward pure score-based eviction: the hot set holds its shape longer and stale entries linger, consuming capacity that admission of a new user would need. β → 0 removes the recency term entirely and leaves (1 − score_ema) alone.
How it would be calibrated. β and α interact and must be swept together: α = 0.2 already gives a 3.1-step half-life on the score itself, so a β chosen without reference to α double-counts recency. The calibration is a two-dimensional sweep over real per-head attention traces at the canonical 128 K context, scored on hit rate at fixed hot-set size. See Appendix K, section K.7 — Uncalibrated parameters.
Analytical model — illustrative worked example, not measured data. α = 0.2, half-life 3.1 decode steps. Canonical numbers v4.0.