Figure: GPU integration with CXL.mem — memory mapping, hint interface, driver-to-firmware translation and fault-path latency. Four endpoints contribute 1 TB of tier-2 DRAM and 16 TB of tier-3 flash; the GPU virtual address space holds 192 GB of local HBM at 0x0000 to 0x4FFF plus CXL.mem regions of 256 GB, 256 GB and 512 GB, while the model's 140 GB of weights stay HBM-resident and never cross CXL. Allocation hints — data type, access pattern, consistency, prefetch window, priority, expected size, plus prefetch parameters including EMA alpha of 0.2 with a half-life of 3.1 decode steps and a GQA head count of 8 — travel from the application through the driver into endpoint firmware in five stages over a CXL.io mailbox. The fault path costs 15 nanoseconds for request issue, 20 for the PCIe Gen5 by 16 PHY, 25 for CXL switch traversal, 40 for endpoint controller decode, 80 for the DDR5-5600 array access and 20 for the return path, totalling 200 nanoseconds. That compares with about 100 nanoseconds for local HBM at 8 TB/s, 13.0 microseconds for traditional PCIe DMA and 25 microseconds for an NVMe read from tier-3 flash, so the CXL.mem path is 13,000 divided by 200 equals 65 times faster than driver-mediated DMA at the same boundary. All values are an analytical model, not measured.

Section 9

GPU Integration

Memory mapping, hint interfaces, and fault handling for CXL.mem

9.1 Memory Mapping

CXL.mem regions appear in the GPU's unified virtual address space via PCIe BAR mapping with CXL.mem bridging. The GPU accesses remote memory using standard load/store semantics. Four endpoints contribute 1 TB of Tier-2 DRAM and 16 TB of Tier-3 flash. The model's 140 GB of weights stay HBM-resident and never cross CXL.

Target-platform capability, not a generic CXL guarantee. Whether a GPU can issue ordinary load/store directly against a CXL.mem region — rather than going through a driver-mediated copy — depends on the specific GPU, its MMU and BAR/HDM decoder support, the platform firmware and the CXL topology. It is assumed here as a property of the target platform. It is not a demonstrated capability of CXL in general, nor a published capability of any particular B200 system.

GPU Virtual Address Space
CXL.mem
Bridge
load/store
semantics
Endpoint Physical Memory
GPU Kernel Access Pattern — CONCEPTUAL PSEUDOCODE

Illustrative only. This fragment is not compilable CUDA and does not demonstrate a capability of CXL, of CUDA, or of any shipping B200 platform. It sketches what a kernel would look like if the target platform exposes a CXL.mem region as GPU-addressable mapped memory with load/store semantics. Real systems require explicit region registration, a platform-specific mapping path, and driver and firmware support; a plain pointer dereference into a remote CXL region is not portable and is not guaranteed by the CXL specification.

// GPU kernel accessing CXL.mem KV-cache
__global__ void attention_kernel(float* kv_cache) {
  // Direct load from CXL.mem address
  float4 kv = *(float4*)(kv_cache + offset);
  
  // GPU MMU translates VA → CXL.mem address
  // CXL bridge handles coherent access
  // No per-access DMA setup and no per-access driver
  // involvement in steady state; the control plane, region
  // mapping, fault handling and firmware still run on the host.
}
9.2 Hint Interface

Extended allocation API communicates memory characteristics to the endpoint, enabling intelligent caching and prefetch decisions.

struct cxl_alloc_hints
enum data_type KV_CACHE, WEIGHTS, ACTIVATIONS, SCRATCH
enum access_pattern SEQUENTIAL, RANDOM, STRIDED, BROADCAST
enum consistency RELAXED, ACQUIRE_RELEASE, SEQ_CST
u32 prefetch_window Lookahead distance in cache lines
u32 priority Eviction priority (0 = evict first)
u64 expected_size Allocation size hint for placement
struct prefetch_params
u32 window_size Number of entries to prefetch ahead
u32 stride Access stride for strided patterns
bool rope_aware Enable RoPE position locality
u32 rope_window Position window [P-W, P+W]
float ema_alpha EMA smoothing α for attention scoring — default 0.2, half-life ln0.5/ln0.8 = 3.1 decode steps
u32 head_count GQA head count for per-head tracking
DATA_TYPE_KV_CACHE
Per-head eviction, EMA scoring, RoPE prefetch enabled
DATA_TYPE_WEIGHTS
Read-only, layer prefetch, broadcast-optimized
DATA_TYPE_ACTIVATIONS
High-bandwidth, sequential, short-lived
DATA_TYPE_SCRATCH
Temporary workspace, lowest eviction priority
9.3 Driver → Firmware Translation

The GPU driver translates allocation hints into endpoint firmware configuration via CXL.io mailbox commands.

1
Application Allocation Request
Runtime calls extended allocation API with hints
cxl_malloc(size, &hints)
2
Driver Hint Processing
Driver validates hints, selects target endpoint based on capacity and locality
select_endpoint(hints) → EP0
3
Mailbox Command Construction
Pack hints into CXL.io mailbox payload with vendor-specific extensions
build_mailbox_cmd(VENDOR_ALLOC, hints)
4
CXL.io Mailbox Transfer
Command sent over PCIe to endpoint controller
cxl_mailbox_send(ep, cmd)
5
Firmware Configuration
Endpoint firmware configures caching policy, prefetcher, and eviction strategy
configure_region(addr, policy)
CXL.io Mailbox Protocol
GPU Driver
Host CPU
PCIe / CXL.io
Transport
Endpoint FW
ARM Cores
Mailbox Command Payload
opcode: 0xC0 (VENDOR_ALLOC) size: 4096 data_type: KV_CACHE pattern: RANDOM prefetch_window: 64 ema_alpha: 0.2 (half-life 3.1 steps) head_count: 8 rope_aware: true
9.4 Fault Handling & Latency Breakdown

When GPU accesses an uncached CXL.mem address, a fault triggers the full access path. Total latency is the sum of each stage.

Latency
GPU/host request issue
Attention kernel issues the access; GPU MMU resolves the VA and identifies the CXL.mem region from the PTE flags.
Latency
PCIe Gen5 ×16 PHY
Serialization and PHY processing onto the Gen5 ×16 link.
Latency
CXL switch traversal
CXL 3.0 switch decodes the address and routes the request to the owning endpoint port.
Latency
Endpoint CXL controller decode
Endpoint controller decodes the CXL.mem request and issues the local memory transaction.
Latency
DDR5-5600 array access
Endpoint DRAM row activate, column access and data return.
Latency
Return path
Response travels back through the switch and PHY to the requesting GPU.
Total Fault Latency
200 ns
Access Type Latency Bandwidth CPU Involved
HBM (Local) ~100 ns 8 TB/s No
CXL.mem (Direct) 200 ns 64 GB/s per EP No
PCIe DMA (Traditional) 13.0 μs 64 GB/s Yes (driver)
NVMe Read (Tier 3 flash) 25 μs 14 GB/s Yes (filesystem)

Analytical model — every latency and capacity on this page is derived from vendor specifications and an analytical model; nothing was measured on hardware. The 200 ns CXL.mem path versus the 13.0 µs PCIe DMA path gives 13,000 ÷ 200 = 65×, compared at the same boundary (kernel issue → data available) for the same 4 KiB payload: a load/store path against a driver-mediated DMA path. Canonical numbers v4.0.