arXiv subjects

Keqiu Li

Publications and source records attributed to Keqiu Li.

11 recordsLinked to original sources

arXiv2026

AsymSpec: Efficient Cloud-Edge Speculative Decoding over Asymmetric Networks

Cloud-edge speculative decoding places a lightweight draft model at an edge gateway and a higher-quality target model in the cloud, but inserts communication into every speculative block. Under a constrained uplink, candidate messages may queue while the verifier is idle. Stop-and-wait scheduling leaves edge compute underutilized; optimistic same-request runahead can waste work when a rejection or an unexpected bonus token invalidates dependent drafts. We present AsymSpec, which addresses uplink-gated verification and invalid dependent work with two corresponding mechanisms. Its asymmetric verification protocol keeps the common-path acceptance upload compact and moves richer, rejection-only correction information to the downlink. A total-variation (TV) certificate for the residual distribution determines whether a small target top-K response suffices; if not, the protocol progressively escalates through proposal-based exact recovery before falling back to the full distribution. Its confirmed-prefix pipeline exposes only independent, valid requests to the edge scheduler and lets the cloud re-batch arrived blocks, hiding verification waits when another confirmed-prefix request is ready without using same-request runahead. Across three draft-target pairs, two workloads, and three asymmetric network profiles, our end-to-end evaluation shows that AsymSpec delivers 2.82-28.03$\times$ the output-token throughput of the strongest baseline.

cs.DC↗

arXiv2026

RAC: Reference-Aware Activation Compression for Communication-Efficient Split LLM Inference

Large language model (LLM) agents repeatedly process long, privacy-sensitive contexts, while cloud-only deployment exposes user data beyond the trusted endpoint and fully local deployment often requires costly hardware. Split inference offers a middle ground by executing the model head, tail, and tools locally and the middle layers in the cloud, but its local-cloud-local path transfers boundary hidden states at every invocation and creates a critical communication bottleneck. We present \system, a reference-aware codec that retrieves exact-token historical spans for prefill uplinks, reuses the reconstructed uplink state for same-round prefill downlinks, and generates boundary-specific decode references with lightweight causal predictors. RAC applies grouped affine alignment and calibrated residual quantization with optional prefill outliers, while sender-side wire-format reconstruction synchronizes subsequent references and offline calibration accounts for quality and packed representation costs. Across three models and nine evaluated model-link pairs, Raw-to-RAC mean time to first token (TTFT) and time per output token (TPOT) ratios are 1.24-2.72$\times$ and 1.01-2.79$\times$, while the 12 non-perplexity task-score changes range from $-0.40$ to $+2.50$ points.

cs.DC↗

arXiv2026

Mosaic: Unlocking Long-Context Inference for Diffusion LLMs via Global Memory Planning and Dynamic Peak Taming

Diffusion-based large language models (dLLMs) have emerged as a promising paradigm, utilizing simultaneous denoising to enable global planning and iterative refinement. While these capabilities are particularly advantageous for long-context generation, deploying such models faces a prohibitive memory capacity barrier stemming from severe system inefficiencies. We identify that existing inference systems are ill-suited for this paradigm: unlike autoregressive models constrained by the cumulative KV-cache, dLLMs are bottlenecked by transient activations recomputed at every step. Furthermore, general-purpose memory reuse mechanisms lack the global visibility to adapt to dLLMs' dynamic memory peaks, which toggle between logits and FFNs. To address these mismatches, we propose Mosaic, a memory-efficient inference system that shifts from local, static management to a global, dynamic paradigm. Mosaic integrates a mask-only logits kernel to eliminate redundancy, a lazy chunking optimizer driven by an online heuristic search to adaptively mitigate dynamic peaks, and a global memory manager to resolve fragmentation via virtual addressing. Extensive evaluations demonstrate that Mosaic achieves an average 2.71$\times$ reduction in the memory peak-to-average ratio and increases the maximum inference sequence length supportable on identical hardware by 15.89-32.98$\times$. This scalability is achieved without compromising accuracy and speed, and in fact reducing latency by 4.12%-23.26%.

cs.LG↗

arXiv2025

PAT: Accelerating LLM Decoding via Prefix-Aware Attention with Resource Efficient Multi-Tile Kernel

LLM serving is increasingly dominated by decode attention, which is a memory-bound operation due to massive KV cache loading from global memory. Meanwhile, real-world workloads exhibit substantial, hierarchical shared prefixes across requests (e.g., system prompts, tools/templates, RAG). Existing attention implementations fail to fully exploit prefix sharing: one-query-per-CTA execution repeatedly loads shared prefix KV cache, while one-size-fits-all tiling leaves on-chip resources idle and exacerbates bubbles for uneven KV lengths. These choices amplify memory bandwidth pressure and stall memory-bound decode attention. This paper introduces PAT, a prefix-aware attention kernel implementation for LLM decoding that organizes execution with a pack-forward-merge paradigm. PAT packs queries by shared prefix to reduce repeated memory accesses, runs a customized multi-tile kernel to achieve high resource efficiency. It further applies practical multi-stream forwarding and KV splitting to reduce resource bubbles. The final merge performs online softmax with negligible overhead. We implement PAT as an off-the-shelf plugin for vLLM. Evaluation on both real-world and synthetic workloads shows that PAT reduces attention latency by 53.5% on average and TPOT by 17.0-93.1% under the same configurations against state-of-the-art attention kernels. PAT's source code is publicly available at https://github.com/flashserve/PAT.

cs.DC↗

arXiv2025

RAGPulse: An Open-Source RAG Workload Trace to Optimize RAG Serving Systems

Retrieval-Augmented Generation (RAG) is a critical paradigm for building reliable, knowledge-intensive Large Language Model (LLM) applications. However, the multi-stage pipeline (retrieve, generate) and unique workload characteristics (e.g., knowledge dependency) of RAG systems pose significant challenges for serving performance optimization. Existing generic LLM inference traces fail to capture these RAG-specific dynamics, creating a significant performance gap between academic research and real-world deployment. To bridge this gap, this paper introduces RAGPulse, an open-source RAG workload trace dataset. This dataset was collected from an university-wide Q&A system serving that has served more than 40,000 students and faculties since April 2024. We detail RAGPulse's system architecture, its privacy-preserving hash-based data format, and provide an in-depth statistical analysis. Our analysis reveals that real-world RAG workloads exhibit significant temporal locality and a highly skewed hot document access pattern. RAGPulse provides a high-fidelity foundation for researchers to develop and validate novel optimization strategies for RAG systems, such as content-aware batching and retrieval caching, ultimately enhancing the efficiency and reliability of RAG services. The code is available at https://github.com/flashserve/RAGPulse.

cs.LG↗

arXiv2024

Harpagon: Minimizing DNN Serving Cost via Efficient Dispatching, Scheduling and Splitting

Advances in deep neural networks (DNNs) have significantly contributed to the development of real-time video processing applications. Efficient scheduling of DNN workloads in cloud-hosted inference systems is crucial to minimizing serving costs while meeting application latency constraints. However, existing systems suffer from excessive module latency during request dispatching, low execution throughput during module scheduling, and wasted latency budget during latency splitting for multi-DNN application, which undermines their capability to minimize the serving cost. In this paper, we design a DNN inference system called Harpagon, which minimizes the serving cost under latency constraints with a three-level design. It first maximizes the batch collection rate with a batch-aware request dispatch policy to minimize the module latency. It then maximizes the module throughput with multi-tuple configurations and proper amount of dummy requests. It also carefully splits the end-to-end latency into per-module latency budget to minimize the total serving cost for multi-DNN applications. Evaluation shows that Harpagon outperforms the state of the art by 1.49 to 2.37 times in serving cost while satisfying the latency objectives. Additionally, compared to the optimal solution using brute force search, Harpagon derives the lower bound of serving cost for 91.5% workloads with millisecond level runtime.

cs.DC↗

arXiv2024

A Comprehensive Survey of Blockchain Scalability: Shaping Inner-Chain and Inter-Chain Perspectives

Blockchain is widely applied in logistics, finance, and agriculture. As single blockchain users grow, scalability becomes crucial. However, existing works lack a comprehensive summary of blockchain scalability. They focus on single chains or cross-chain technologies. This survey summarizes scalability across the physical and logical layers, as well as inner-chain, inter-chain, and technology dimensions. The physical layer covers data and protocols, while the logical layer represents blockchain architecture. Each component is analyzed from inner-chain and inter-chain perspectives, considering technological factors. The aim is to enhance researchers' understanding of blockchain's architecture, data, and protocols to advance scalability research.

cs.DB↗

arXiv2022

PKDGA: A Partial Knowledge-based Domain Generation Algorithm for Botnets

Domain generation algorithms (DGAs) can be categorized into three types: zero-knowledge, partial-knowledge, and full-knowledge. While prior research merely focused on zero-knowledge and full-knowledge types, we characterize their anti-detection ability and practicality and find that zero-knowledge DGAs present low anti-detection ability against detectors, and full-knowledge DGAs suffer from low practicality due to the strong assumption that they are fully detector-aware. Given these observations, we propose PKDGA, a partial knowledge-based domain generation algorithm with high anti-detection ability and high practicality. PKDGA employs the reinforcement learning architecture, which makes it evolve automatically based only on the easily-observable feedback from detectors. We evaluate PKDGA using a comprehensive set of real-world datasets, and the results demonstrate that it reduces the detection performance of existing detectors from 91.7% to 52.5%. We further apply PKDGA to the Mirai malware, and the evaluations show that the proposed method is quite lightweight and time-efficient.

cs.CR↗

arXiv2021

A Comprehensive Study on Learning-Based PE Malware Family Classification Methods

Driven by the high profit, Portable Executable (PE) malware has been consistently evolving in terms of both volume and sophistication. PE malware family classification has gained great attention and a large number of approaches have been proposed. With the rapid development of machine learning techniques and the exciting results they achieved on various tasks, machine learning algorithms have also gained popularity in the PE malware family classification task. Three mainstream approaches that use learning based algorithms, as categorized by the input format the methods take, are image-based, binary-based and disassembly-based approaches. Although a large number of approaches are published, there is no consistent comparisons on those approaches, especially from the practical industry adoption perspective. Moreover, there is no comparison in the scenario of concept drift, which is a fact for the malware classification task due to the fast evolving nature of malware. In this work, we conduct a thorough empirical study on learning-based PE malware classification approaches on 4 different datasets and consistent experiment settings. Based on the experiment results and an interview with our industry partners, we find that (1) there is no individual class of methods that significantly outperforms the others; (2) All classes of methods show performance degradation on concept drift (by an average F1-score of 32.23%); and (3) the prediction time and high memory consumption hinder existing approaches from being adopted for industry usage.

cs.CR↗

arXiv2021

Applications of Federated Learning in Smart Cities: Recent Advances, Taxonomy, and Open Challenges

Federated learning plays an important role in the process of smart cities. With the development of big data and artificial intelligence, there is a problem of data privacy protection in this process. Federated learning is capable of solving this problem. This paper starts with the current developments of federated learning and its applications in various fields. We conduct a comprehensive investigation. This paper summarize the latest research on the application of federated learning in various fields of smart cities. In-depth understanding of the current development of federated learning from the Internet of Things, transportation, communications, finance, medical and other fields. Before that, we introduce the background, definition and key technologies of federated learning. Further more, we review the key technologies and the latest results. Finally, we discuss the future applications and research directions of federated learning in smart cities.

cs.LG↗

arXiv2016

Technique Report: Near-Optimal Routing Protection for SDN Networks Using Distributed Markov Approximation

Software Defined Networking (SDN) brings numbers of advantages along with many challenges. One particular concern is on the control-plane resilience, while the existing protection approaches proposed for SDN networks mainly focus on data-plane. In order to achieve the carrier-grade recovery from link failures, we adopt the dedicated protection scheme towards finding optimal protection routing for control-plane traffic. To this end, we study a weighted cost minimization problem, in which the traffic load balancing and flow table rule placement are jointly considered when selecting protection paths for controller-switch sessions. Because this problem is known as NP-hard, we propose a Markov approximation based combinatorial optimization approach for routing protection in SDN control-plane, which produces near-optimal solution in a distributed fashion. We then extend our solution to an on-line case that can handle the single-link failure one at a time. The induced performance fluctuation is also analyzed with theoretical derivation. Extensive experimental results show that our proposed algorithm has fast convergence and high efficiency in resource utilization.

cs.DC↗