Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2609.38706

Preserving Provenance in Shared KV Caches for LLM Serving

Wei Song·Yuxin Cao·Xi Zheng·Leo Zhang·Xiao Cheng
View original source ↗Read source document ↗Export metadata ↓

Abstract

Production LLM serving stacks combine an inference engine's local prefix cache with a shared KV-cache tier for fleet-wide reuse. The local cache distinguishes requests by adapter, weight configuration and sharing domain, but the shared tier may key entries only by token content and coarse model metadata. This boundary erases provenance and lets identical tokens under incompatible computational or sharing contexts collide. We call this composition gap provenance-blind reuse and present its first systematic study. A source audit of three vLLM connectors confirms the structural omission, while runtime experiments reproduce it across vLLM and two SGLang releases, 12 models from 7 families (0.5 B-32 B), and over 160 configurations. Cross-adapter collisions reduce accuracy from 0.94 to 0.64, incompatible KV representations reduce reasoning accuracy to zero, and salt omission enables 93% prompt identification from timing. We formalize the missing guarantee as the KV provenance contract: for a declared dimension registry, shared keys must be injective over computational and sharing provenance, with identities stable across workers. Any dimension with a stable identity can therefore be added without connector-specific key logic. A canonical descriptor binds per-request and per-worker provenance into lookup and store keys, while a differential checker detects dimensions that change KV state without changing the key. Implemented in vLLM and SGLang 0.5.20 across three cache paths, provenance binding eliminates unsafe reuse while preserving legitimate sharing. Hit-path latency changes remain within 0.34 ms and below run-to-run variation; retention grows with provenance diversity.

Explore related subjects

cs.DC

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Wei Song, Yuxin Cao, Xi Zheng, Leo Zhang, Xiao Cheng. 2026-09-30. Preserving Provenance in Shared KV Caches for LLM Serving. https://arxiv.org/abs/2609.38706

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Rank-Aware CPU Reservation and In-Place Adjustment for MPI CFD on Kubernetes

Non-uniform meshes in computational fluid dynamics (CFD) can leave some Message Passing Interface (MPI) ranks waiting while others determine execution time. On Kubernetes, reserving a full CPU for every rank can then restrict the admission of further simulations. We investigate rank-aware CPU reservation and in-place request adjustment using OpenFOAM on three Kubernetes testbeds. The framework lowers reservations without hard CPU caps and uses online MPI profiles to adjust each rank's CPU request while the solver runs. We reduced the per-job CPU reservation from 16.0 to 5.9 vCPU, with single-job wall-clock time increasing by at most 6%. With these reservations on twelve nodes, six co-located simulations reached 4.09 times the throughput of sequential execution. Matched controls showed no consistent benefit from proportional request shaping. Beyond the nominal CPU capacity, throughput collapsed with default busy-waiting; yielding waits softened the collapse. In one six-simulation run, the online controller performed 120 in-place resizes of running solvers with zero pod restarts; the pipeline reached 2.5 times the throughput of sequential execution. In a separate comparison on 16 physical cores, reallocating ranks from the far field to the near-wall zone reduced fixed-iteration wall-clock time by factors of 2.0 to 4.2 relative to the original concentric allocation, depending on mesh and iteration horizon. The reallocation also outperformed balanced Scotch partitions. The implementation requires no changes to OpenFOAM, the MPI library, or the Kubernetes scheduler.

cs.DC↗
arXiv2026

The Illusion of Power Capping in LLM Decode: A Phase-Aware Energy Characterisation Across Attention Architectures

Power capping is a common GPU energy lever in LLM serving, and it appears to work: the cap is set, the GPU reports drawing less than it, and the energy budget is met. We show the cap does little for the phase that dominates LLM serving: autoregressive decode. Across four attention paradigms (GQA, MLA, Gated DeltaNet, and Mamba2) on an NVIDIA H200 under vLLM eager execution, decode kernels wait on HBM and the GPU idles between kernel launches, so decode draws less than half of the GPU's 700W rating and throughput stays within a few percent under every cap we set. The cap engages only when a large batch runs a long generation. Locking the SM clock instead lowers power without slowing memory-bound decode and recovers up to 32% of decode energy at minimal throughput loss. We identify three architecture-dependent frequency-scaling classes and find that, once prefill is measured without prefix caching, the recurrent architectures pay no prefill penalty: their linear-time mixing undercuts quadratic attention on long prompts, so their efficient decode adds to the prefill saving and lowers total request energy from the first output token at large batch.

cs.DC↗
arXiv2026

Simple-IT: Practical Low-Latency Signature-Free BFT Consensus

Recent advances in quantum computing pose a looming threat to most current Byzantine fault-tolerant (BFT) consensus protocols, which rely on quantum-vulnerable public-key signature schemes such as Ed25519 and BLS12-381. Instead of switching to much more expensive post-quantum secure signature schemes, an alternative is to use signature-free protocols, which rely only on cheap, post-quantum secure authenticated channels. In this paper, we ask whether signature-free BFT consensus protocols can match the performance of current state-of-the-art, quantum-vulnerable BFT consensus protocols. While previous work on the Sailfish++ protocol showed that state-of-the-art throughput is attainable signature-free, the question of latency is still open. Several recent signature-free protocols have low latency in theory, but they are all very intricate, and no practical implementation has so far been presented. In this work, we propose Simple-IT, a new leader-based, signature-free BFT consensus protocol that achieves a theoretical latency of 4 message delays (one more than the optimum), and only 3 on its optimistic path. Crucially, Simple-IT is simple enough to be amenable to implementation and to practical optimizations such as speculative pipelining, and, as we show experimentally in a geo-distributed testbed, it achieves both throughput and latency competitive with state-of-the-art quantum-vulnerable protocols.

cs.DC↗