arXiv subjects

Ziyu Zhong

Publications and source records attributed to Ziyu Zhong.

3 recordsLinked to original sources

arXiv2026

SSV: Sparse Speculative Verification for Efficient LLM Inference

Speculative decoding and dynamic sparse attention are two complementary approaches for accelerating long-context LLM inference: the former amortizes target-model execution across multiple verifier queries, while the latter reduces each query's KV-cache working set. Directly combining them, however, exposes a structural mismatch: speculative verification relies on cross-query commonality, whereas dynamic sparse attention assigns query-specific sparse layouts. This mismatch limits KV-block reuse, amplifies NSA's branch-wise overheads, and makes verification strategy selection input- and regime-dependent. We present SSV, a sparse speculative-verification framework that turns dynamic sparse attention into a verification-oriented workload. SSV combines overlap-aware grouped-query execution, refresh/reuse-based NSA kernel fusion, and profile-guided prompt-adaptive orchestration to improve cross-query reuse, reduce selected-index and branch-fusion overheads, and select effective draft-verification strategies under user-specified precision classes. Experiments on NVIDIA H100 GPUs show that SSV achieves up to 3.49x end-to-end throughput over autoregressive NSA decoding and up to 6.86x kernel speedups for sparse speculative verification.

cs.OS↗

arXiv2025

Predictability-Aware Motion Prediction for Edge XR via High-Order Error-State Kalman Filtering

As 6G networks are developed and defined, offloading of XR applications is emerging as one of the strong new use cases. The reduced 6G latency coupled with edge processing infrastructure will for the first time provide a realistic offloading scenario in cellular networks where several computationally intensive functions, including rendering, can migrate from the user device and into the network. A key advantage of doing so is the lowering of the battery needs in the user devices and the possibility to design new devices with smaller form factors. However, offloading introduces increased delays compared to local execution, primarily due to network transmission latency and queuing delays at edge servers, especially under multi-user concurrency. Despite the computational power of edge platforms, the resulting motion-to-photon (MTP) latency negatively impacts user experience. To mitigate this, motion prediction has been proposed to offset delays. Existing approaches build on either deep learning or Kalman filtering. Deep learning techniques face scalability limitations at the resource-constrained edge, as their computational expense intensifies with increasing user concurrency, while Kalman filtering suffers from poor handling of complex movements and fragility to packet loss inherent in 6G's high-frequency radio interfaces. In this work, we introduce a context-aware error-state Kalman filter (ESKF) prediction framework, which forecasts the user's head motion trajectory to compensate for MTP latency in remote XR. By integrating a motion classifier that categorizes head motions based on their predictability, our algorithm demonstrates reduced prediction error across different motion classes. Our findings demonstrate that the optimized ESKF not only surpasses traditional Kalman filters in positional and orientational accuracy but also exhibits enhanced robustness and resilience to packet loss.

cs.NI↗

arXiv2025

Video Streaming with Kairos: An MPC-Based ABR with Streaming-Aware Throughput Prediction

In this paper, we present Kairos, a model predictive control (MPC)-based adaptive bitrate (ABR) scheme that integrates streaming-aware throughput predictions to enhance video streaming quality. Kairos features an attention-based throughput predictor with buffer-aware uncertainty control, improving prediction accuracy and adaptability to network conditions. Specifically, we introduce a multi-time attention network to handle the irregularly sampled sequences in streaming data, creating uniformly spaced latent representations. Additionally, we design a separate prediction network that estimates future throughput at multiple percentiles and incorporates a buffer-aware uncertainty adjustment module. This module dynamically selects the appropriate throughput percentile based on the buffer size, enhancing robustness to varying network conditions. Lastly, to mitigate QoE smoothness penalties caused by predictors focused solely on accuracy, we introduce a smoothness regularizer. By embedding streaming-aware characteristics, such as sampling irregularity, buffer occupancy, and smoothness, into the throughput predictor design, Kairos significantly improves bitrate decision-making within the MPC framework. Extensive trace-driven and real-world experiments demonstrate that Kairos outperforms state-of-the-art ABR schemes, achieving an average QoE improvement of 1.52% to 7.28% under various network conditions.

cs.NI↗