arXiv subjects

Kai Zhou

Publications and source records attributed to Kai Zhou.

At least 55 records · Page 3Linked to original sources

arXiv2025

Fast LLM Post-training via Decoupled and Fastest-of-N Speculation

Rollout dominates the training time in large language model (LLM) post-training, where the trained model is used to generate tokens given a batch of prompts. This work, SpecActor, achieves fast rollout with speculative decoding that deploys a fast draft path to accelerate the unparallelizable generation, while the correctness is guaranteed by fast parallel verification of the outputs with the original model. SpecActor addresses two foundational challenges that hinder speculation efficiency: (1) a Decoupled speculation method that overcomes the computation inefficiency issue when executing speculative decoding with relative large per-worker batch size -- a common configuration in training but unfriendly to speculation, and (2) a Fastest-of-N speculation method that selects and combines different draft methods according to the rollout progress to approximate the optimal draft method even when the best one is unknown a priori. Extensive evaluations on production traces show that SpecActor accelerates mean rollout speed by 2.0--2.4x, with up to 2.7x speedup, over common post-training baselines. The results are consistent across both dense and MoE models and across different RL algorithms. Notably, SpecActor is 1.1--2.6x faster compared to vanilla speculative rollout in different traces. The accelerated rollout achieves 1.4--2.3x faster end-to-end training time.

cs.DC↗

arXiv2025

Toward a foundation model for heavy-ion collision experiments based on point-cloud diffusion

A novel point cloud diffusion model for relativistic heavy-ion collisions, capable of ultra-fast generation of complete, event-by-event collision output, is introduced. When trained on UrQMD cascade simulations, the model generates realistic collision event output containing 26 distinct hadron species, as a list of particle momentum vectors along with their particle ID. From solving inverse problems to accelerating model calculations or detector simulations, the model can be a promising general purpose tool for heavy-ion collisions beneficial to both theoretical studies and experimental applications.

hep-ph↗

arXiv2025

Ultra fast, event-by-event heavy-ion simulations for next generation experiments

We present a novel deep generative framework that uses probabilistic diffusion models for ultra fast, event-by-event simulations of heavy-ion collision output. This new framework is trained on UrQMD cascade data to generate a full collision event output containing 26 distinct hadron species. The output is represented as a point cloud, where each point is defined by a particle's momentum vector and its corresponding species information (ID). Our architecture integrates a normalizing flow-based condition generator that encodes global event features into a latent vector, and a diffusion model that synthesizes a point cloud of particles based on this condition. A detailed description of the model and an in-depth analysis of its performance is provided. The conditional point cloud diffusion model learns to generate realistic output particles of collision events which successfully reproduce the UrQMD distributions for multiplicity, momentum and rapidity of each hadron type. The flexible point cloud representation of the event output preserves full event-level granularity, enabling direct application to inverse problems and parameter estimation tasks while also making it easily adaptable for accelerating any event-by-event model calculation or detector simulation.

hep-ph↗

arXiv2025

A Review of Machine Learning for Cavitation Intensity Recognition in Complex Industrial Systems

Cavitation intensity recognition (CIR) is a critical technology for detecting and evaluating cavitation phenomena in hydraulic machinery, with significant implications for operational safety, performance optimization, and maintenance cost reduction in complex industrial systems. Despite substantial research progress, a comprehensive review that systematically traces the development trajectory and provides explicit guidance for future research is still lacking. To bridge this gap, this paper presents a thorough review and analysis of hundreds of publications on intelligent CIR across various types of mechanical equipment from 2002 to 2025, summarizing its technological evolution and offering insights for future development. The early stages are dominated by traditional machine learning approaches that relied on manually engineered features under the guidance of domain expert knowledge. The advent of deep learning has driven the development of end-to-end models capable of automatically extracting features from multi-source signals, thereby significantly improving recognition performance and robustness. Recently, physical informed diagnostic models have been proposed to embed domain knowledge into deep learning models, which can enhance interpretability and cross-condition generalization. In the future, transfer learning, multi-modal fusion, lightweight network architectures, and the deployment of industrial agents are expected to propel CIR technology into a new stage, addressing challenges in multi-source data acquisition, standardized evaluation, and industrial implementation. The paper aims to systematically outline the evolution of CIR technology and highlight the emerging trend of integrating deep learning with physical knowledge. This provides a significant reference for researchers and practitioners in the field of intelligent cavitation diagnosis in complex industrial systems.

eess.SP↗

arXiv2025

Data-Driven Einstein-Dilaton Model for Pure Yang-Mills Thermodynamics and Glueball Spectrum

We develop a machine learning assisted holographic model that consistently describes both the equation of state and glueball spectrum of pure Yang-Mills theory, achieved through neural network reconstruction of Einstein-dilaton gravity. Our framework incorporates key non-perturbative constraints of lattice QCD data: the ground ($0^{++}$) and first-excited ($0^{++*}$) scalar glueball masses pins down the infrared (IR) geometry, while entropy density data anchors the ultraviolet (UV) behavior of the metric. A multi-stage neural network optimization then yields the full gravitational dual -- warp factor $A(z)$ and dilaton field $Φ(z)$ -- that satisfies both spectroscopic and thermodynamic constraints. The resulting model accurately reproduces the deconfinement phase transition thermodynamics (pressure, energy density, trace anomaly) and predicts higher glueball excitations ($0^{++**}$, $0^{++***}$) consistent with available lattice calculations. This work establishes a new paradigm for data-driven holographic reconstruction, solving the long-standing challenge of unified description of confinement thermodynamics and spectroscopy.

hep-ph↗

arXiv2025

Neural Unfolding of the Chiral Magnetic Effect in Heavy-Ion Collisions

The search for the chiral magnetic effect (CME) in relativistic heavy-ion collisions (HICs) is challenged by significant background contamination. We present a novel deep learning approach based on a U-Net architecture to time-reversely unfold the dynamics of CME-related charge separation, enabling the reconstruction of the physics signal across the entire evolution of HICs. Trained on the events simulated by a multi-phase transport model with different cases of CME settings, our model learns to recover the charge separation based on final-state transverse momentum distributions at either the quark-gloun plasma freeze-out or hadronic freeze-out. This devises a methodological tool for the study of CME and underscores the promise of deep learning approaches in retrieving physics signals in HICs.

nucl-th↗

arXiv2025

Stealthy Dual-Trigger Backdoors: Attacking Prompt Tuning in LM-Empowered Graph Foundation Models

The emergence of graph foundation models (GFMs), particularly those incorporating language models (LMs), has revolutionized graph learning and demonstrated remarkable performance on text-attributed graphs (TAGs). However, compared to traditional GNNs, these LM-empowered GFMs introduce unique security vulnerabilities during the unsecured prompt tuning phase that remain understudied in current research. Through empirical investigation, we reveal a significant performance degradation in traditional graph backdoor attacks when operating in attribute-inaccessible constrained TAG systems without explicit trigger node attribute optimization. To address this, we propose a novel dual-trigger backdoor attack framework that operates at both text-level and struct-level, enabling effective attacks without explicit optimization of trigger node text attributes through the strategic utilization of a pre-established text pool. Extensive experimental evaluations demonstrate that our attack maintains superior clean accuracy while achieving outstanding attack success rates, including scenarios with highly concealed single-trigger nodes. Our work highlights critical backdoor risks in web-deployed LM-empowered GFMs and contributes to the development of more robust supervision mechanisms for open-source platforms in the era of foundation models.

cs.CR↗

arXiv2025

On the Potential of Quantum Computing in Classical Program Analysis

Classical program analysis techniques, such as abstract interpretation and symbolic execution, are essential for ensuring software correctness, optimizing performance, and enabling compiler optimizations. However, these techniques face computational limitations when analyzing programs with large or exponential state spaces, limiting their effectiveness in ensuring system reliability. Quantum computing, with its parallelism and ability to process superposed states, offers a promising solution to these challenges. In this work, we present QEX, a design that uses quantum computing to analyze classical programs. By synthesizing quantum circuits that encode program states in superposition and trace data dependency between program variables through entanglement, QEX enables the simultaneous exploration of program behaviors, significantly improving scalability and precision. This advancement has broad applications, from debugging and security verification to optimizing compilers for next-generation hardware. As a proof-of-concept, we evaluated QEX on 22 benchmark programs, demonstrating its effectiveness in analyzing program states. To support more language features and make QEX realized sooner in Fault-Tolerant Quantum Computing (FTQC), we propose QEX-H which hybridizes QEX with classical analysis techniques. To our knowledge, this work is the first proposal to use quantum computing for classical program analysis.

quant-ph↗

arXiv2025

Provably Robust Adaptation for Language-Empowered Foundation Models

Language-empowered foundation models (LeFMs), such as CLIP and GraphCLIP, have transformed multimodal learning by aligning visual (or graph) features with textual representations, enabling powerful downstream capabilities like few-shot learning. However, the reliance on small, task-specific support datasets collected in open environments exposes these models to poisoning attacks, where adversaries manipulate the support samples to degrade performance. Existing defenses rely on empirical strategies, which lack formal guarantees and remain vulnerable to unseen and adaptive attacks. Certified robustness offers provable guarantees but has been largely unexplored for few-shot classifiers based on LeFMs. This study seeks to fill these critical gaps by proposing the first provably robust few-shot classifier that is tailored for LeFMs. We term our model Language-empowered Few-shot Certification (\textbf{LeFCert}). It integrates both textual and feature embeddings with an adaptive blending mechanism. To achieve provable robustness, we propose a twofold trimmed mean prototype and derive provable upper and lower bounds for classification scores, enabling certification under worst-case poisoning scenarios. To further enhance the performance, we extend LeFCert with two variants by considering a more realistic and tighter attack budget: LeFCert-L incorporates randomized smoothing to provide Lipschitz continuity and derive robustness under dual budget constraints, and LeFCert-C provides collective certification for scenarios where attackers distribute a shared poisoning budget across multiple samples. Experiments demonstrate that LeFCert achieves state-of-the-art performance, significantly improving both clean and certified accuracy compared to existing baselines. Despite its advanced robustness mechanisms, LeFCert is computationally efficient, making it practical for real-world applications.

cs.LG↗

arXiv2025

Combining complex Langevin dynamics with score-based and energy-based diffusion models

Theories with a sign problem due to a complex action or Boltzmann weight can sometimes be numerically solved using a stochastic process in the complexified configuration space. However, the probability distribution effectively sampled by this complex Langevin process is not known a priori and notoriously hard to understand. In generative AI, diffusion models can learn distributions, or their log derivatives, from data. We explore the ability of diffusion models to learn the distributions sampled by a complex Langevin process, comparing score-based and energy-based diffusion models, and speculate about possible applications.

hep-lat↗

arXiv2025

Efficient Dynamic Ensembling for Multiple LLM Experts

LLMs have demonstrated impressive performance across various language tasks. However, the strengths of LLMs can vary due to different architectures, model sizes, areas of training data, etc. Therefore, ensemble reasoning for the strengths of different LLM experts is critical to achieving consistent and satisfactory performance on diverse inputs across a wide range of tasks. However, existing LLM ensemble methods are either computationally intensive or incapable of leveraging complementary knowledge among LLM experts for various inputs. In this paper, we propose an efficient Dynamic Ensemble Reasoning paradigm, called DER to integrate the strengths of multiple LLM experts conditioned on dynamic inputs. Specifically, we model the LLM ensemble reasoning problem as a Markov Decision Process, wherein an agent sequentially takes inputs to request knowledge from an LLM candidate and passes the output to a subsequent LLM candidate. Moreover, we devise a reward function to train a DER-Agent to dynamically select an optimal answering route given the input questions, aiming to achieve the highest performance with as few computational resources as possible. Last, to fully transfer the expert knowledge from the prior LLMs, we develop a Knowledge Transfer Prompt that enables the subsequent LLM candidates to transfer complementary knowledge effectively. Experiments demonstrate that our method uses fewer computational resources to achieve better performance compared to state-of-the-art baselines. Code and appendix are available at https://github.com/Fhujinwu/DER

cs.AI↗

arXiv2025

The Incompressible Navier-Stokes-Fourier Limits from Boltzmann-Fermi-Dirac Equation for Low Regularity Data

We consider the hydrodynamic limits of the quantum Boltzmann equation with Fermi-Dirac statistics for hard sphere and hard potentials in the whole space. By analyzing the spectrum of the linearized collision operator combined with the transport operator and its associated semigroup, the incompressible Navier-Stokes-Fourier limits from the BFD equation is verified rigorously. Compared to the results in [Jiang-Xiong-Zhou,J. Differ. Equ.,2022], this paper works with a lower regularity for the initial data. In addition, the fixed-point arguments together with a time iteration ensure us to obtain the lifespan of kinetic solution coincides with those of limiting fluid solution.

math.AP↗

arXiv2025

SparseDoctor: Towards Efficient Chat Doctor with Mixture of Experts Enhanced Large Language Models

Large language models (LLMs) have achieved great success in medical question answering and clinical decision-making, promoting the efficiency and popularization of the personalized virtual doctor in society. However, the traditional fine-tuning strategies on LLM require the updates of billions of parameters, substantially increasing the training cost, including the training time and utility cost. To enhance the efficiency and effectiveness of the current medical LLMs and explore the boundary of the representation capability of the LLMs on the medical domain, apart from the traditional fine-tuning strategies from the data perspective (i.e., supervised fine-tuning or reinforcement learning from human feedback), we instead craft a novel sparse medical LLM named SparseDoctor armed with contrastive learning enhanced LoRA-MoE (low rank adaptation-mixture of experts) architecture. To this end, the crafted automatic routing mechanism can scientifically allocate the computational resources among different LoRA experts supervised by the contrastive learning. Additionally, we also introduce a novel expert memory queue mechanism to further boost the efficiency of the overall framework and prevent the memory overflow during training. We conduct comprehensive evaluations on three typical medical benchmarks: CMB, CMExam, and CMMLU-Med. Experimental results demonstrate that the proposed LLM can consistently outperform the strong baselines such as the HuatuoGPT series.

cs.CL↗

arXiv2025

Neural Earthquake Forecasting with Minimal Information: Limits, Interpretability, and the Role of Markov Structure

Forecasting earthquake sequences remains a central challenge in seismology, particularly under non-stationary conditions. While deep learning models have shown promise, their ability to generalize across time remains poorly understood. We evaluate neural and hybrid (NN plus Markov) models for short-term earthquake forecasting on a regional catalog using temporally stratified cross-validation. Models are trained on earlier portions of the catalog and evaluated on future unseen events, enabling realistic assessment of temporal generalization. We find that while these models outperform a purely Markovian model on validation data, their test performance degrades substantially in the most recent quintile. A detailed attribution analysis reveals a shift in feature relevance over time, with later data exhibiting simpler, more Markov-consistent behavior. To support interpretability, we apply Integrated Gradients, a type of explainable AI (XAI) to analyze how models rely on different input features. These results highlight the risks of overfitting to early patterns in seismicity and underscore the importance of temporally realistic benchmarks. We conclude that forecasting skill is inherently time-dependent and benefits from combining physical priors with data-driven methods.

physics.geo-ph↗

arXiv2025

Adversarial Robustness of Link Sign Prediction in Signed Graphs

Signed graphs serve as fundamental data structures for representing positive and negative relationships in social networks, with signed graph neural networks (SGNNs) emerging as the primary tool for their analysis. Our investigation reveals that balance theory, while essential for modeling signed relationships in SGNNs, inadvertently introduces exploitable vulnerabilities to black-box attacks. To showcase this, we propose balance-attack, a novel adversarial strategy specifically designed to compromise graph balance degree, and develop an efficient heuristic algorithm to solve the associated NP-hard optimization problem. While existing approaches attempt to restore attacked graphs through balance learning techniques, they face a critical challenge we term "Irreversibility of Balance-related Information," as restored edges fail to align with original attack targets. To address this limitation, we introduce Balance Augmented-Signed Graph Contrastive Learning (BA-SGCL), an innovative framework that combines contrastive learning with balance augmentation techniques to achieve robust graph representations. By maintaining high balance degree in the latent space, BA-SGCL not only effectively circumvents the irreversibility challenge but also significantly enhances model resilience. Extensive experiments across multiple SGNN architectures and real-world datasets demonstrate both the effectiveness of our proposed balance-attack and the superior robustness of BA-SGCL, advancing the security and reliability of signed graph analysis in social networks. Datasets and codes of the proposed framework are at the github repository https://anonymous.4open.science/r/BA-SGCL-submit-DF41/.

cs.LG↗

arXiv2025

Dense Matter in Neutron Stars with eXTP

In this White Paper, we present the potential of the enhanced X-ray Timing and Polarimetry (eXTP) mission to constrain the equation of state of dense matter in neutron stars, exploring regimes not directly accessible to terrestrial experiments. By observing a diverse population of neutron stars - including isolated objects, X-ray bursters, and accreting systems - eXTP's unique combination of timing, spectroscopy, and polarimetry enables high-precision measurements of compactness, spin, surface temperature, polarimetric signals, and timing irregularity. These multifaceted observations, combined with advances in theoretical modeling, pave the way toward a comprehensive description of the properties and phases of dense matter from the crust to the core of neutron stars. Under development by an international Consortium led by the Institute of High Energy Physics of the Chinese Academy of Sciences, the eXTP mission is planned to be launched in early 2030.

astro-ph.HE↗

arXiv2025

SEASONED: Semantic-Enhanced Self-Counterfactual Explainable Detection of Adversarial Exploiter Contracts

Decentralized Finance (DeFi) attacks have resulted in significant losses, often orchestrated through Adversarial Exploiter Contracts (AECs) that exploit vulnerabilities in victim smart contracts. To proactively identify such threats, this paper targets the explainable detection of AECs. Existing detection methods struggle to capture semantic dependencies and lack interpretability, limiting their effectiveness and leaving critical knowledge gaps in AEC analysis. To address these challenges, we introduce SEASONED, an effective, self-explanatory, and robust framework for AEC detection. SEASONED extracts semantic information from contract bytecode to construct a semantic relation graph (SRG), and employs a self-counterfactual explainable detector (SCFED) to classify SRGs and generate explanations that highlight the core attack logic. SCFED further enhances robustness, generalizability, and data efficiency by extracting representative information from these explanations. Both theoretical analysis and experimental results demonstrate the effectiveness of SEASONED, which showcases outstanding detection performance, robustness, generalizability, and data efficiency learning ability. To support further research, we also release a new dataset of 359 AECs.

cs.CR↗

arXiv2025

Towards Robust Graph Structural Learning Beyond Homophily via Preserving Neighbor Similarity

Despite the tremendous success of graph-based learning systems in handling structural data, it has been widely investigated that they are fragile to adversarial attacks on homophilic graph data, where adversaries maliciously modify the semantic and topology information of the raw graph data to degrade the predictive performances. Motivated by this, a series of robust models are crafted to enhance the adversarial robustness of graph-based learning systems on homophilic graphs. However, the security of graph-based learning systems on heterophilic graphs remains a mystery to us. To bridge this gap, in this paper, we start to explore the vulnerability of graph-based learning systems regardless of the homophily degree, and theoretically prove that the update of the negative classification loss is negatively correlated with the pairwise similarities based on the powered aggregated neighbor features. The theoretical finding inspires us to craft a novel robust graph structural learning strategy that serves as a useful graph mining module in a robust model that incorporates a dual-kNN graph constructions pipeline to supervise the neighbor-similarity-preserved propagation, where the graph convolutional layer adaptively smooths or discriminates the features of node pairs according to their affluent local structures. In this way, the proposed methods can mine the ``better" topology of the raw graph data under diverse graph homophily and achieve more reliable data management on homophilic and heterophilic graphs.

cs.LG↗