Jonguk Cheong, Myungbin Hwang, Eulgyu Kim and Byoungyoung Lee submitted KVGuard to arXiv on 4 November 2026, describing a checker for key-value (KV) cache memory safety in large language model serving. The authors report a prototype on vLLM and SGLang that covers all 29 historical KV cache safety bugs they collected from those systems’ patch histories.[1]
The abstract locates the problem in PagedAttention, which divides a CUDA-allocated GPU memory region into physical blocks and maps requests to them through a CPU-built block table. The authors say errors including double-free, use-after-free, out-of-bounds access, uninitialized reads and permission violations can silently corrupt outputs or break request isolation without crashing. They argue that existing memory sanitizers track CUDA addresses rather than the cache’s blocks, requests and tables.[1]
KVGuard maintains compact shadow metadata for each physical KV block, recording permissions for each request and initialization for each entry. Its five checks span CPU allocation, block-table construction and translation, and GPU table lookup and cache access. The authors report eight previously unknown violations and no false positives in their evaluation. On Qwen 2.5 14B, they report geometric-mean TPOT overhead of 3.0% for vLLM and 7.4% for SGLang, with memory overhead at most 0.11% of the KV cache.[1]
[1]: KVGuard: A KV Cache Safety Checker for LLM Serving Systems. arXiv, submitted 4 November 2026. https://arxiv.org/abs/2611.06138
Sources
Check or correct this page
Verification record
The digest below shows whether the text you are reading matches the published source. It does not show that the reporting is correct. The build computes it from this page's Markdown source and lists it in the site manifest.
5191f3e3f7a20e34bf473b99664b3969642ad7f72d9dbcd9a82b16ee95a2b435