Cross-IP Request Coalescing: Relocating the Fan-out Point in Virtualized I/O

Abstract

Cloud data centers rely on virtualization technologies to serve AI workloads in multi-tenant environments. With the growing scale of data-intensive AI workloads, the performance of storage I/O paths at the virtualization layer has become a critical factor. A single user request often crosses multiple IP blocks, where functional units such as storage, GPU, and accelerator devices under virtualization fan out into separate stack traversals between the guest and the backend. As a result, round-trip and context-switching overheads accumulate with the number of devices. In this letter, we identify that a dominant factor in this overhead lies not in the kernel-mediated I/O path alone, but in the per-device submission structure itself, which persists even in user-space, kernel-bypass storage frameworks such as SPDK. To address this, we propose cross-IP request coalescing, which relocates the fan-out point from the guest to the SPDK vhost-user backend. The guest submits multi-device I/O as a single compound request, and the accelerated bdev at the backend decomposes and dispatches it to each target device, replacing multiple per-device guest-backend round trips with a single submission. Evaluation in an SPDK-based virtualized environment shows that the proposed approach achieves up to 1.78x lower latency than the per-device baseline, with the benefit growing as concurrency increases.