arXiv subjects

Sa Wang

Publications and source records attributed to Sa Wang.

At least 19 recordsLinked to original sources

arXiv2026

Energy-Efficient LLM Serving via Disaggregated Attention--FFN and Flexible Frequency Scaling

Large language model (LLM) serving spans diverse applications with stringent service-level objectives (SLOs), often requiring GPUs to run at maximum frequencies and increasing energy consumption. Existing energy-management approaches adapt GPU frequencies only at the request or inference-phase level, overlooking operator-level differences in frequency sensitivity between Attention and feed-forward networks (FFNs). We find that the energy-optimal frequencies of Attention and FFN (A/F) differ and vary with the inference phase, workload, and system configurations. However, runtime variability and independent A/F frequency control create a large search space and high communication overhead. To address these challenges, we present AFlex, a framework that jointly optimizes resource provisioning and GPU frequency scaling for disaggregated A/F serving. AFlex introduces a global scheduler and a local operator-level dynamic voltage and frequency scaling (DVFS) controller to determine A/F resource allocations and frequencies. It further introduces an interleaved A/F pipeline with dynamic microbatch depth and adaptive request batching to reduce pipeline bubbles. We implement AFlex in SGLang and evaluate it on NVIDIA A800 GPUs using Qwen3-32B and Mixtral-8$\times$7B under production Conversation and Coding traces. \AFlex reduces energy per token by up to 49\% over state-of-the-art disaggregated serving and 48\% over frequency-scaling systems while satisfying TTFT and TPOT SLOs.

cs.DC↗

arXiv2026

SwiftCache: Efficient LLM Serving for Multi-turn Conversations with Heterogeneous KV Cache Sharing

Multi-turn conversation is a fundamental scenario in LLM applications, widely used in chatbots and AI agents. As the conversation evolves, historical tokens accumulate continuously. Existing systems cache their key-value (KV) pairs to avoid redundant computation. However, limited GPU memory (HBM) capacity often forces these KV caches to be offloaded to CPU memory or SSD, making KV cache reloads increasingly costly in terms of latency as the context grows. Meanwhile, the constrained HBM capacity also limits the maximum inference length, thereby restricting the number of turns that can be supported in a conversation. To address these two challenges, we propose SwiftCache, a collaborative inference system that enables heterogeneous models to share underutilized GPU memory and NVLink bandwidth within a server. Specifically, models with low KV cache demand donate idle GPU memory to store the prefix cache of high-demand models, allowing cross-model KV cache sharing over NVLink and avoiding slow PCIe transfers. SwiftCache further reduces memory pressure by keeping only the KV cache of the currently active layer in local GPU memory, thereby enabling longer-context inference. Our experiments on real-world workloads show that SwiftCache reduces P99 time-to-first-token (TTFT) by up to 69% and extends maximum context length by up to 3.98x compared to vLLM and SGLang, with minimal interference to co-located models.

cs.DC↗

arXiv2026

Cloud-native and Distributed Systems for Efficient and Scalable Large Language Models -- A Research Agenda

The rapid rise of Large Language Models (LLMs) has revolutionized various artificial intelligence (AI) applications, from natural language processing to code generation. However, the computational demands of these models, particularly in training and inference, present significant challenges. Traditional systems are often unable to meet these requirements, necessitating the integration of cloud-native and distributed architectures. This paper explores the role of cloud platforms and distributed systems in supporting the scalability, efficiency, and optimization of LLMs. We discuss the complexities of LLM deployment, including data management, resource optimization, and the need for microservices, autoscaling, and hybrid cloud-edge solutions. Additionally, we examine emerging research trends, such as serverless inference, quantum computing, and federated learning, and their potential to drive the next phase of LLM innovation. The paper concludes with a roadmap for future developments, emphasizing the need for continued research, standardization, and cross-sector collaboration to sustain the growth of LLMs in both research and enterprise applications.

cs.DC↗

arXiv2026

Heavy-quark transport across the QCD crossover driven by a lattice-constrained in-medium potential

We present a self-consistent framework for heavy-quark transport in the quark-gluon plasma across the QCD crossover region. By synthesizing perturbative and nonperturbative interactions into a unified interaction kernel, we circumvent the traditional reliance on arbitrary soft-hard momentum separation scales. The interaction is governed by an in-medium effective potential, incorporating short-range Yukawa screening and long-range confining string contributions, both rigorously constrained by the latest lattice QCD data. Our results reveal that the nonperturbative string tension is indispensable for capturing the extreme opacity of the medium near the critical temperature $T_c$. Specifically, our model predicts a spatial diffusion coefficient of $2\pi T D_s \approx 0.5 \sim 1.7$, demonstrating a striking quantitative agreement with the recent lattice QCD extractions. Ultimately, our results provide a robust dynamical interpretation of the strong heavy-quark coupling near the QCD crossover and offer a unified framework for describing heavy-flavor transport in hot and dense QCD matter.

hep-ph↗

arXiv2026

UCAgent: An End-to-End Agent for Block-Level Functional Verification

Functional verification remains a critical bottleneck in modern IC development cycles, accounting for approximately 70% of total development time in many projects. However, traditional methods, including constrained-random and formal verification, struggle to keep pace with the growing complexity of modern semiconductor designs. While recent advances in Large Language Models (LLMs) have shown promise in code generation and task automation, significant challenges hinder the realization of end-to-end functional verification automation. These challenges include (i) limited accuracy in generating Verilog/SystemVerilog verification code, (ii) the fragility of LLMs when executing complex, multi-step verification workflows, and (iii) the difficulty of maintaining verification consistency across specifications, coverage models, and test cases throughout the workflow. To address these challenges, we propose UCAgent, an end-to-end agent that automates hardware block-level functional verification based on three core mechanisms. First, we establish a pure Python verification environment using Picker and Toffee to avoid relying on LLM-generated SystemVerilog verification code. Second, we introduce a configurable 31-stage fine-grained verification workflow to guide the LLM, where each stage is verified by an automated checker. Furthermore, we propose a Verification Consistency Labeling Mechanism (VCLM) that assigns hierarchical labels to LLM-generated artifacts, improving the reliability and traceability of verification. Experimental results show that UCAgent can complete end-to-end automated verification on multiple modules, including the UART, FPU, and integer divider modules, achieving up to 98.5% code coverage and up to 100% functional coverage. UCAgent also discovers previously unidentified design defects in realistic designs, demonstrating its practical potential.

cs.SE↗

arXiv2026

RRAttention: Dynamic Block Sparse Attention via Per-Head Round-Robin Shifts for Long-Context Inference

The quadratic complexity of attention mechanisms poses a critical bottleneck for large language models processing long contexts. While dynamic sparse attention methods offer input-adaptive efficiency, they face fundamental trade-offs: requiring preprocessing, lacking global evaluation, violating query independence, or incurring high computational overhead. We present RRAttention, a novel dynamic sparse attention method that simultaneously achieves all desirable properties through a head \underline{r}ound-\underline{r}obin (RR) sampling strategy. By rotating query sampling positions across attention heads within each stride, RRAttention maintains query independence while enabling efficient global pattern discovery with stride-level aggregation. Our method reduces complexity from $O(L^2)$ to $O(L^2/S^2)$ and employs adaptive Top-$\tau$ selection for optimal sparsity. Extensive experiments on natural language understanding (HELMET) and multimodal video comprehension (Video-MME) demonstrate that RRAttention recovers over 99\% of full attention performance while computing only half of the attention blocks, achieving 2.4$\times$ speedup at 128K context length and outperforming existing dynamic sparse attention methods.

cs.CL↗

arXiv2025

Substructure grooming of inclusive and photon-tagged jets in heavy-ion collisions

Jet substructure provides a powerful probe of partonic interactions within the quark-gluon plasma (QGP) in heavy-ion collisions. In this paper, we present a systematic theoretical study of the groomed substructures for both inclusive jets and photon-tagged jets ($\gamma+$jets) utilizing the Dynamical and Soft-Drop Grooming algorithms in PbPb collisions by employing the SHELL transport model. Our theoretical calculations exhibit a suppression at high $k_{\rm T,g}$, the relative transverse momentum between the two subjets in the groomed substructure, consistent with the recent ALICE measurements. We show that the suppression of high $k_{\rm T,g}$ arises from the combined effects of the reduction of the subleading subjet transverse momentum due to partonic energy loss and the narrowing of the groomed jet radius $R_g$ induced by selection bias. Our findings demonstrate that no enhancement is observed at high $k_{\rm T,g}$, even in the complete absence of selection bias. Furthermore, we propose that the broadening of $R_g$ in photon-tagged jets, which are less susceptible to selection bias compared to inclusive jets, provides relatively direct evidence of the jet substructure broadening. Our analysis reveals that the $R_g$ broadening becomes more pronounced as the jet radius increases, where the medium-induced gluon radiation plays a dominant role in driving such broadening. In particular, we find that as the jet radius increases, the Soft Drop grooming algorithm exhibits a better resolving power for the contribution of the medium response to the jet substructure broadening.

hep-ph↗

arXiv2025

Lyra: A Hardware-Accelerated RISC-V Verification Framework with Generative Model-Based Processor Fuzzing

As processor designs grow more complex, verification remains bottlenecked by slow software simulation and low-quality random test stimuli. Recent research has applied software fuzzers to hardware verification, but these rely on semantically blind random mutations that may generate shallow, low-quality stimuli unable to explore complex behaviors. These limitations result in slow coverage convergence and prohibitively high verification costs. In this paper, we present Lyra, a heterogeneous RISC-V verification framework that addresses both challenges by pairing hardware-accelerated verification with an ISA-aware generative model. Lyra executes the DUT and reference model concurrently on an FPGA SoC, enabling high-throughput differential checking and hardware-level coverage collection. Instead of creating verification stimuli randomly or through simple mutations, we train a domain-specialized generative model, LyraGen, with inherent semantic awareness to generate high-quality, semantically rich instruction sequences. Empirical results show Lyra achieves up to $1.27\times$ higher coverage and accelerates end-to-end verification by up to $107\times$ to $3343\times$ compared to state-of-the-art software fuzzers, while consistently demonstrating lower convergence difficulty.

cs.AR↗

arXiv2025

Perturbative and nonperturbative properties of heavy quark transport in a thermal SU(3) gluon plasma

We investigate the perturbative and nonperturbative aspects of heavy quark transport in a thermal SU(3) gluon plasma. Based on the soft-hard factorized model, we extend the original perturbative framework to the near-critical temperature region, where nonperturbative effects become significant. The transition behavior of the semi-quark-gluon-plasma (semi-QGP) is described via a temperature-dependent background field incorporated in the background field effective theory. By implementing this approach, we quantitatively evaluate the collisional energy loss and momentum diffusion coefficients of charm and bottom quarks as functions of the incoming energy and medium temperature. Our results show a distinct suppression of both the energy loss and the diffusion coefficients relative to conventional perturbative estimates, especially near the critical temperature. This suppression originates from the emergence of a temperature-dependent color background field, which effectively reduces the color charge screening of the medium. These findings provide important theoretical insight into the phenomenology of heavy-flavor probes, offering a unified theoretical framework applicable across both high- and low-momentum regimes.

hep-ph↗

arXiv2025

Soft-hard factorization of heavy-quark transport in QCD matter at finite chemical potential

We calculate the collisional energy loss and momentum diffusion coefficients of heavy quarks traversing a hot and dense QCD medium at finite quark chemical potential, $\mu\neq0$. The analysis is performed within an extended soft-hard factorization model (SHFM) that consistently incorporates the $\mu$-dependence of the Debye screening mass $M_D(\mu)$ and of the fermionic thermal distribution functions. Both the energy loss and the diffusion coefficients are found to increase with $\mu$, with the enhancement being most pronounced at low temperatures where the chemical potential effects dominate the medium response. To elucidate the origin of this dependence, we derive analytic high-energy approximations in which the leading $\mu$-corrections appear as logarithmic terms: a soft logarithm $\sim\mu^{2}\ln(|t^{*}|/M_{D}^{2})$ from $t$-channel scattering off thermal gluonic excitations, and a hard logarithm $\sim\mu^{2}\ln(E_{1}T/|t^{*}|)$ from scattering off thermal quarks. In the complete result the dependence on the intermediate separation scale $t^{\ast}$ cancels, as required. We also confirm the expected mass hierarchy $-dE/dz(charm)<-dE/dz(bottom)$ at fixed velocity. Our findings demonstrate that finite chemical potential plays a significant role in heavy-quark transport and must be included in theoretical descriptions of heavy-flavor dynamics in baryon-rich environments, such as those probed in the RHIC Beam Energy Scan, and at FAIR and NICA.

hep-ph↗

arXiv2025

TurboFuzz: FPGA Accelerated Hardware Fuzzing for Processor Agile Verification

Verification is a critical process for ensuring the correctness of modern processors. The increasing complexity of processor designs and the emergence of new instruction set architectures (ISAs) like RISC-V have created demands for more agile and efficient verification methodologies, particularly regarding verification efficiency and faster coverage convergence. While simulation-based approaches now attempt to incorporate advanced software testing techniques such as fuzzing to improve coverage, they face significant limitations when applied to processor verification, notably poor performance and inadequate test case quality. Hardware-accelerated solutions using FPGA or ASIC platforms have tried to address these issues, yet they struggle with challenges including host-FPGA communication overhead, inefficient test pattern generation, and suboptimal implementation of the entire multi-step verification process. In this paper, we present TurboFuzz, an end-to-end hardware-accelerated verification framework that implements the entire Test Generation-Simulation-Coverage Feedback loop on a single FPGA for modern processor verification. TurboFuzz enhances test quality through optimized test case (seed) control flow, efficient inter-seed scheduling, and hybrid fuzzer integration, thereby improving coverage and execution efficiency. Additionally, it employs a feedback-driven generation mechanism to accelerate coverage convergence. Experimental results show that TurboFuzz achieves up to 2.23x more coverage collection than software-based fuzzers within the same time budget, and up to 571x performance speedup when detecting real-world issues, while maintaining full visibility and debugging capabilities with moderate area overhead.

cs.AR↗

arXiv2025

DDiT: Dynamic Resource Allocation for Diffusion Transformer Model Serving

The Text-to-Video (T2V) model aims to generate dynamic and expressive videos from textual prompts. The generation pipeline typically involves multiple modules, such as language encoder, Diffusion Transformer (DiT), and Variational Autoencoders (VAE). Existing serving systems often rely on monolithic model deployment, while overlooking the distinct characteristics of each module, leading to inefficient GPU utilization. In addition, DiT exhibits varying performance gains across different resolutions and degrees of parallelism, and significant optimization potential remains unexplored. To address these problems, we present DDiT, a flexible system that integrates both inter-phase and intra-phase optimizations. DDiT focuses on two key metrics: optimal degree of parallelism, which prevents excessive parallelism for specific resolutions, and starvation time, which quantifies the sacrifice of each request. To this end, DDiT introduces a decoupled control mechanism to minimize the computational inefficiency caused by imbalances in the degree of parallelism between the DiT and VAE phases. It also designs a greedy resource allocation algorithm with a novel scheduling mechanism that operates at the single-step granularity, enabling dynamic and timely resource scaling. Our evaluation on the T5 encoder, OpenSora SDDiT, and OpenSora VAE models across diverse datasets reveals that DDiT significantly outperforms state-of-the-art baselines by up to 1.44x in p99 latency and 1.43x in average latency.

cs.DC↗

arXiv2024

Probing the mass effect of heavy quark jets in high-energy nuclear collisions

The production of heavy quark (HQ) jets provides a new arena to address the mass effect of jet quenching in heavy-ion physics. This paper presents a theoretical study of HQ jet yield suppression in Pb+Pb collisions at the LHC and focuses on the energy loss of HQ jets produced by different mechanisms. The p+p baseline is carried out by the SHERPA generator, and the jet-medium interactions are described by the SHELL transport model, which considers the elastic and inelastic partonic energy loss in the quark-gluon plasma (QGP). In p+p collisions, our numerical results indicate that the HQ jets from gluon splitting ($g \rightarrow Q$-jet) give the dominant contribution at high $p_T$, and it shows more dispersive structures than the HQ-initiated one ($Q \rightarrow Q$-jet). In nucleus-nucleus collisions, our calculations are consistent with the inclusive and b-jet $R_{AA}$ recently measured by the ATLAS collaboration, which suggests a remarkable manifestation of the mass effect of jet energy loss. As a result of the dispersive substructure, the $g \rightarrow Q$-jet will lose more energy than the $Q \rightarrow Q$-jet in the QGP. Due to the significant contribution of $g \rightarrow c$-jet, the $R_{AA}$ of c-jet will be comparable or even smaller than that of inclusive jet. To experimentally distinguish the $g \rightarrow Q$-jet and $Q \rightarrow Q$-jet, we propose the event selection strategies based on their topological features and test the performances. By isolating the $c \rightarrow c$-jet and $b \rightarrow b$-jet, the jets initiated by heavy quarks, we predict that the order of their $R_{AA}$ are in line with the mass hierarchy of energy loss. Future measurements on the $R_{AA}$ of $Q \rightarrow Q$-jet and $g \rightarrow Q$-jet will provide a unique chance to test the flavor/mass dependence of energy loss at the jet level.

hep-ph↗

arXiv2024

Medium modifications of heavy-flavor jet angularities in high-energy nuclear collisions

We present the first theoretical study of heavy-flavor jet angularities ($\lambda^{\kappa}_{\alpha}$) in Pb+Pb collisions at $\sqrt{s_{\rm NN}}=$ 5.02 TeV. The initial production of heavy-flavor jets is carried out using the POWHEG+PYTHIA8 prescription, while the jet evolution in the quark-gluon plasma (QGP) is described by the SHELL transport model. In p+p collisions, we observe narrower angularity distributions for D$^0$-tagged jets compared to inclusive jets, consistent with the ALICE preliminary results. We then demonstrate that jet quenching in the QGP slightly widens the angularity distribution of D$^0$-tagged jets in Pb+Pb collisions relative to that in p+p collisions for jet transverse momentum of $10 < p_{\rm T,jet} < 20$ GeV/c, while the angularity distributions of inclusive and D$^0$-tagged jets become narrower in Pb+Pb collisions relative to p+p at $p_{\rm T,jet} > 20$ GeV/c due to the strong influence of the selection bias. Additionally, by comparing the average angularities $\langle \lambda^{\kappa}_{\alpha} \rangle$ of inclusive, D$^0$-tagged and B$^0$-tagged jets with varying $\alpha$ and $\kappa$, we show that the larger the quark mass is, the lower the jet's $\langle \lambda^{\kappa}_{\alpha} \rangle$ values are. As a result of the slenderer initial distribution, we predict that as compared to inclusive jets, the heavy-flavor jets, especially the B$^0$-tagged ones, will suffer stronger modifications of $\langle \lambda^{\kappa}_{\alpha} \rangle$ in Pb+Pb relative to p+p at $10 < p_{\rm T,jet} < 20$ GeV/c. For a larger jet radius, a more significant broadening of jet angularities is predicted because of the enhanced contributions of the wide-angle particles.

hep-ph↗

arXiv2024

Unveiling the jet angular broadening with photon-tagged jets in high-energy nuclear collisions

The medium modification of jet substructure in hot and dense nuclear matter has garnered significant interest from the heavy-ion physics community in recent years. Measurements of inclusive jets show an angular narrowing in nucleus-nucleus collisions, while recent CMS results for photon-tagged jets ($\gamma$+jets) suggest evidence of broadening. In this study, we conduct a theoretical analysis of the angular structure of inclusive jets and $\gamma$+jets using a transport approach that accounts for jet energy loss and the medium response in the quark-gluon plasma. We examine the girth modification of $\gamma$+jets in $0-30\%$ PbPb collisions at $\sqrt{s_{NN}} = 5.02$ TeV, achieving satisfactory agreement with recent CMS measurements. We explore the relationship between selection bias and jet kinematics by varying the threshold for $x_{j\gamma} = p_T^{\rm jet}/p_T^{\gamma}$. Notably, we quantitatively demonstrate that $\gamma$+jets significantly reduce selection bias and can effectively select jets that have been sufficiently quenched in PbPb collisions, which is crucial for capture the jet angular broadening. Additionally, we estimate the contributions of medium-induced gluon radiation and the medium response to the broadening of the jet angular substructure. Lastly, we analyze the modification patterns of jet $R_g$ and $\Delta R_{\rm axis}$ in PbPb collisions, which indicate slight broadening for $\gamma$+jets and noticeable narrowing for inclusive jets compared to pp collisions.

hep-ph↗

arXiv2024

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool

Large language model (LLM) serving has transformed from stateless to stateful systems, utilizing techniques like context caching and disaggregated inference. These optimizations extend the lifespan and domain of the KV cache, necessitating a new architectural approach. We present MemServe, a unified system that integrates both inter-request and intra-request optimizations. MemServe introduces MemPool, an elastic memory pool managing distributed memory and KV caches across serving instances. Using MemPool APIs, MemServe combines context caching with disaggregated inference for the first time, supported by a global scheduler that enhances cache reuse through a global prompt tree-based locality-aware policy. Tests show that MemServe significantly improves job completion time and time-to-first-time.

cs.DC↗

arXiv2024

Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloads

Transformer-based large language model (LLM) inference serving is now the backbone of many cloud services. LLM inference consists of a prefill phase and a decode phase. However, existing LLM deployment practices often overlook the distinct characteristics of these phases, leading to significant interference. To mitigate interference, our insight is to carefully schedule and group inference requests based on their characteristics. We realize this idea in TetriInfer through three pillars. First, it partitions prompts into fixed-size chunks so that the accelerator always runs close to its computationsaturated limit. Second, it disaggregates prefill and decode instances so each can run independently. Finally, it uses a smart two-level scheduling algorithm augmented with predicted resource usage to avoid decode scheduling hotspots. Results show that TetriInfer improves time-to-first-token (TTFT), job completion time (JCT), and inference efficiency in turns of performance per dollar by a large margin, e.g., it uses 38% less resources all the while lowering average TTFT and average JCT by 97% and 47%, respectively.

cs.DC↗

arXiv2024

Transverse momentum balance of dijets in Xe+Xe collisions at the LHC

We present a theoretical study of the medium modifications of the $p_T$ balance ($x_J$) of dijets in Xe+Xe collisions at $\sqrt{s_{NN}}=5.44$ TeV. The initial production of dijets was carried out using the POWHEG+PYTHIA8 prescription, which matches the next-to-leading-order (NLO) QCD matrix elements with the parton shower (PS) effect. The SHELL model described the in-medium evolution of nucleus-nucleus collisions using a transport approach. The theoretical results of the dijet $x_J$ in the Xe+Xe collisions exhibit more imbalanced distributions than those in the p+p collisions, consistent with recently reported ATLAS data. By utilizing the Interleaved Flavor Neutralisation, an infrared-and-collinear-safe jet flavor algorithm, to identify the flavor of the reconstructed jets, we classify dijets processes into three categories: gluon-gluon ($gg$), quark-gluon ($qg$), and quark-quark ($qq$), and investigated the respective medium modification patterns and fraction changes of the $gg$, $qg$, and $qq$ components of the dijet sample in Xe+Xe collisions. It is shown that the increased fraction of $qg$ component at a small $x_J$ contributes to the imbalance of the dijet; in particular, the $q_1g_2$ (quark-jet-leading) dijets experience more significant asymmetric energy loss than the $g_1q_2$ (gluon-jet-leading) dijets traversing the QGP. By comparing the $\Delta \langle x_J\rangle = \langle x_J \rangle_{pp} - \langle x_J \rangle_{AA}$ of inclusive, $c\bar{c}$ and $b\bar{b}$ dijets in Xe+Xe collisions, we observe $\Delta\langle x_J \rangle_{incl.}>\Delta \langle x_J \rangle_{c\bar{c}}>\Delta \langle x_J \rangle_{b\bar{b}}$. Moreover, $\rho_{Xe, Pb}$, the ratios of the nuclear modification factors of dijets in Xe+Xe to those in Pb+Pb, were calculated, which indicates that the yield suppression of dijets in Pb+Pb is more pronounced than that in Xe+Xe owing to the larger radius of the lead nucleus.

hep-ph↗