Appendix H

Memory Hierarchy and Caching Theory

Three-tier design, effective latency formulas, and hit rate analysis.

H.1 Three-Tier Hierarchy

Tier	Media	Capacity	Latency
Tier 0: HBM Pinned	GPU HBM	~5 GB	100 ns
Tier 1: HBM Evictable	GPU HBM	~37 GB	100 ns
Tier 2: CXL DRAM	Endpoint DDR5	1 TB	250 ns
Tier 3: Flash	NVMe SSD	16 TB	25 μs

H.2 Effective Latency

L_eff = Σ (hit_rate_i × latency_i)

The weighted-latency equation below is illustrative only. v5.0 does not assume a canonical HBM/CXL/flash hit-rate mix; corrected simulation spans a wide range and no CXL hardware was measured.

L_eff = h_HBML_HBM + h_warmL_warm + h_coldL_cold + h_missL_recompute

H.3 Hit Rate Breakdown

Technique	Contribution	Cumulative
LRU baseline	—	70%
+ Anchor pinning	+8%	78%
+ EMA scoring	+7%	85%
+ Per-head tracking	+6%	91%
+ RoPE prefetch	+4%	95%

← PreviousAppendix G Next →Appendix I