EMA + Eviction Priority

Step-by-step calculation with concrete numbers

1. EMA Update (per decode step)
α (smoothing factor)
0.2
Token Position
1024
Initial EMA
0.0
Step new_score Calculation score_ema
t₁ 0.45 0.3 × 0.45 + 0.7 × 0.00 0.135
t₂ 0.12 0.3 × 0.12 + 0.7 × 0.135 0.131
t₃ 0.30 0.3 × 0.30 + 0.7 × 0.131 0.182
t₄ 0.08 0.3 × 0.08 + 0.7 × 0.182 0.151
t₅ 0.35 0.3 × 0.35 + 0.7 × 0.151 0.211
2. Eviction Priority Function
priority = (1 - score_ema) × recency_decay
# Higher priority = evict sooner
recency_decay = 1 - exp(-β × steps_since_access)
# β = 0.001, decays over ~1000 steps
Token A pos=1024, accessed 50 steps ago
score_ema = 0.211
steps_since_access = 50
recency_decay = 1 - exp(-0.001 × 50) = 0.049
priority = (1 - 0.211) × 0.049 = 0.039
Token B pos=45678, accessed 2000 steps ago
score_ema = 0.08
steps_since_access = 2000
recency_decay = 1 - exp(-0.001 × 2000) = 0.865
priority = (1 - 0.08) × 0.865 = 0.796
3. Eviction Decision
Token A (pos 1024)
score_ema 0.211
steps_since_access 50
recency_decay 0.049
eviction_priority 0.039
✓ KEEP (low priority)
Token B (pos 45678)
score_ema 0.08
steps_since_access 2000
recency_decay 0.865
eviction_priority 0.796
🗑 EVICT (high priority)