The integration design space and current vendor landscape for CXL-attached KV-cache tiers — not a measured or built integration.
A CXL-attached memory tier is only useful to an inference server if bytes actually move between GPU high-bandwidth memory and that tier, and if something decides which bytes move and when. Both halves of that statement are easy to state and non-trivial to build. This chapter lays out the design space for the "move bytes" and "decide what to move" problems, and grounds it in the CXL controller and switch silicon that exists in the market as of this writing. Nothing in this chapter was built, simulated, or benchmarked as part of this package — the performance chapters elsewhere in this document do not depend on any of the integration paths described here actually existing in a tested form.
At a high level there are three places to put the logic that orchestrates a GPU-to-CXL data movement, and each trades off compatibility against hops and host involvement.
| Path | Who orchestrates | Tradeoffs |
|---|---|---|
| (a) Host-CPU-mediated | Host CPU issues and tracks DMA between GPU and CXL memory | Broadest platform compatibility today; adds CPU involvement and an extra hop on the data path, with latency cost that depends on the specific DMA engine and driver stack |
| (b) Direct GPU-to-CXL access | GPU initiates transfers directly against CXL-mapped memory, where the platform allows it | Fewer hops in principle, but availability depends on host platform, root complex, and driver/firmware support that varies by vendor and generation — not quantified here and not assumed to be uniformly available |
| (c) Fabric-resident controller | A controller or accelerator sitting on the CXL fabric participates in eviction and metadata decisions, not just raw transport | Closest to a "device owns mechanism" split (Ch. 3); requires controller logic beyond a passive memory expander and is not demonstrated on real silicon in this package |
Path (a) is the safe default: it works on essentially any platform that exposes CXL memory as host-attached and lets the existing GPU driver stack handle transfers the way it already handles host-memory staging. Path (b) is attractive on paper but its practicality is platform-specific — some host CPU and GPU combinations support more direct paths between accelerator and CXL-attached memory than others, and that support surface is still evolving generation to generation. This package does not claim a specific hop-count or latency reduction for path (b) because it was not measured on any platform here. Path (c) is the most architecturally interesting and the least proven; it is discussed further in Section 9.4.
For grounding, it is worth naming the silicon that actually exists in the CXL controller and switch space as of 2026, without attributing any performance figures in this package to that hardware — none of it was tested as part of this work.
None of these products were benchmarked, integrated, or otherwise tested in the preparation of this package. They are named here as evidence that CXL memory-expansion and switching silicon is a real, shipping category — which is a feasibility signal for the general approach — not as a source of the latency or bandwidth numbers used elsewhere in this document. Those numbers, where cited, come from the sources identified in their respective chapters, not from any hardware named here.
Chapter 3 introduced a policy split: the host decides what belongs in the KV-cache working set and when to evict or re-admit an entry, while the mechanism that actually carries out storage, retrieval, and identity tracking can live closer to the device. Path (c) above — a fabric-resident controller — is the concrete shape that split could take in a CXL deployment.
One illustrative example, not built here: a controller that preserves identity-keyed metadata for a KV-cache block across an eviction and later re-admission cycle, without requiring the host to resend the identity key or recompute it from scratch on every access. If a controller could hold that association itself, the host would issue a coarser-grained instruction — "this identity is wanted again" — and the controller would resolve it to a physical location using state it already maintained, rather than the host reconstructing that mapping through its own bookkeeping. That is one way "mechanism" could move onto the device side of the split.
This remains a design proposal. Nothing in the package instantiates such a controller, and no real CXL silicon — including the products named in Section 9.3 — was configured, programmed, or tested to perform identity-keyed metadata retention of this kind. Whether a given vendor's controller firmware could be extended to support it is an open question for actual hardware evaluation, not something this document answers.
This chapter described the integration design space for connecting GPU memory to a CXL-attached KV-cache tier — host-mediated DMA, direct GPU access where platforms support it, and a fabric-resident controller that could participate in policy mechanics — and named the CXL controller and switch vendors shipping real product as of 2026 as a maturity signal for the category. No GPU-to-CXL-controller integration was measured, simulated, or built as part of this package. Readers should treat this chapter as a survey of what is architecturally possible and who is building the underlying silicon, not as evidence supporting any specific latency, bandwidth, or throughput figure claimed elsewhere in this document.