Chapter 1

Introduction: The Memory Wall Problem

Why KV-cache is the critical bottleneck in LLM inference and what it costs the industry.

1.1 The Memory Wall

Modern LLMs face a fundamental constraint: the KV-cache grows linearly with context length, rapidly exhausting GPU memory.

1.2 Distributed Endpoint Solution

1.3 KV-Cache Mathematics