arXiv subjects

Ang Li

Publications and source records attributed to Ang Li.

At least 163 records · Page 9Linked to original sources

arXiv2025

Local Linear Attention: An Optimal Interpolation of Linear and Softmax Attention For Test-Time Regression

Transformer architectures have achieved remarkable success in various domains. While efficient alternatives to Softmax Attention have been widely studied, the search for more expressive mechanisms grounded in theoretical insight-even at greater computational cost-has been relatively underexplored. In this work, we bridge this gap by proposing Local Linear Attention (LLA), a novel attention mechanism derived from nonparametric statistics through the lens of test-time regression. First, we show that LLA offers theoretical advantages over Linear and Softmax Attention for associative memory via a bias-variance trade-off analysis. Next, we address its computational challenges and propose two memory-efficient primitives to tackle the $Θ(n^2 d)$ and $Θ(n d^2)$ complexity. We then introduce FlashLLA, a hardware-efficient, blockwise algorithm that enables scalable and parallel computation on modern accelerators. In addition, we implement and profile a customized inference kernel that significantly reduces memory overheads. Finally, we empirically validate the advantages and limitations of LLA on test-time regression, in-context regression, associative recall and state tracking tasks. Experiment results demonstrate that LLA effectively adapts to non-stationarity, outperforming strong baselines in test-time training and in-context learning, and exhibiting promising evidence for its scalability and applicability in large-scale models. Code is available at https://github.com/Yifei-Zuo/Flash-LLA.

cs.LG↗

arXiv2025

A Perspective on Quantum Computing Applications in Quantum Chemistry using 25--100 Logical Qubits

The intersection of quantum computing and quantum chemistry represents a promising frontier for achieving quantum utility in domains of both scientific and societal relevance. Owing to the exponential growth of classical resource requirements for simulating quantum systems, quantum chemistry has long been recognized as a natural candidate for quantum computation. This perspective focuses on identifying scientifically meaningful use cases where early fault-tolerant quantum computers, which are considered to be equipped with approximately 25--100 logical qubits, could deliver tangible impact. While recent advances in classical computing have pushed the boundaries of tractable simulations to unprecedented scales, this logical-qubit regime represents the first window where quantum devices can pursue qualitatively distinct strategies, such as polynomial-scaling phase estimation, direct simulation of quantum dynamics, and active-space embedding, that remain challenging for classical solvers, for instance, multireference charge-transfer and conical-intersection states central to photochemistry and materials design. We highlight near-term opportunities in algorithm and software design, discuss representative chemical problems suited for quantum acceleration, and propose strategic roadmaps and collaborative pathways for advancing practical quantum utility in quantum chemistry.

quant-ph↗

arXiv2025

DermINO: Hybrid Pretraining for a Versatile Dermatology Foundation Model

Skin diseases impose a substantial burden on global healthcare systems, driven by their high prevalence (affecting up to 70% of the population), complex diagnostic processes, and a critical shortage of dermatologists in resource-limited areas. While artificial intelligence(AI) tools have demonstrated promise in dermatological image analysis, current models face limitations-they often rely on large, manually labeled datasets and are built for narrow, specific tasks, making them less effective in real-world settings. To tackle these limitations, we present DermNIO, a versatile foundation model for dermatology. Trained on a curated dataset of 432,776 images from three sources (public repositories, web-sourced images, and proprietary collections), DermNIO incorporates a novel hybrid pretraining framework that augments the self-supervised learning paradigm through semi-supervised learning and knowledge-guided prototype initialization. This integrated method not only deepens the understanding of complex dermatological conditions, but also substantially enhances the generalization capability across various clinical tasks. Evaluated across 20 datasets, DermNIO consistently outperforms state-of-the-art models across a wide range of tasks. It excels in high-level clinical applications including malignancy classification, disease severity grading, multi-category diagnosis, and dermatological image caption, while also achieving state-of-the-art performance in low-level tasks such as skin lesion segmentation. Furthermore, DermNIO demonstrates strong robustness in privacy-preserving federated learning scenarios and across diverse skin types and sexes. In a blinded reader study with 23 dermatologists, DermNIO achieved 95.79% diagnostic accuracy (versus clinicians' 73.66%), and AI assistance improved clinician performance by 17.21%.

eess.IV↗

arXiv2025

FAIR Universe HiggsML Uncertainty Dataset and Competition

The FAIR Universe HiggsML Uncertainty Challenge focused on measuring the physical properties of elementary particles with imperfect simulators. Participants were required to compute and report confidence intervals for a parameter of interest regarding the Higgs boson while accounting for various systematic (epistemic) uncertainties. The dataset is a tabular dataset of 28 features and 280 million instances. Each instance represents a simulated proton-proton collision as observed at CERN's Large Hadron Collider in Geneva, Switzerland. The features of these simulations were chosen to capture key characteristics of different types of particles. These include primary attributes, such as the energy and three-dimensional momentum of the particles, as well as derived attributes, which are calculated from the primary ones using domain-specific knowledge. Additionally, a label feature designates each instance's type of proton-proton collision, distinguishing the Higgs boson events of interest from three background sources. As outlined in this paper, the permanent release of the dataset allows long-term benchmarking of new techniques. The leading submissions, including Contrastive Normalising Flows and Density Ratios estimation through classification, are described. Our challenge has brought together the physics and machine learning communities to advance our understanding and methodologies in handling systematic uncertainties within AI techniques.

hep-ph↗

arXiv2025

Breaking the News: Taking the Roles of Influencer vs. Journalist in a LLM-Based Game for Raising Misinformation Awareness

Effectively mitigating online misinformation requires understanding of their mechanisms and learning of practical skills for identification and counteraction. Serious games may serve as tools for combating misinformation, teaching players to recognize common misinformation tactics, and improving their skills of discernment. However, current interventions are designed as single-player, choice-based games, which present players with limited predefined choices. Such restrictions reduce replayability and may lead to an overly simplistic understanding of misinformation and how to debunk them. This study seeks to empower people to understand opinion-influencing and misinformation-debunking processes. We created a Player vs. Player (PvP) game in which participants attempt to generate or debunk misinformation to convince the public opinion represented by LLM. Using a within-subjects mixed-methods study design (N=47), we found that this game significantly raised participants' media literacy and improved their ability to identify misinformation. Qualitative analyses revealed how participants' use of debunking and content creation strategies deepened their understanding of misinformation. This work shows the potential for illuminating contrasting viewpoints of social issues by LLM-based mechanics in PvP games.

cs.HC↗

arXiv2025

SymRTLO: Enhancing RTL Code Optimization with LLMs and Neuron-Inspired Symbolic Reasoning

Optimizing Register Transfer Level (RTL) code is crucial for improving the power, performance, and area (PPA) of digital circuits in the early stages of synthesis. Manual rewriting, guided by synthesis feedback, can yield high-quality results but is time-consuming and error-prone. Most existing compiler-based approaches have difficulty handling complex design constraints. Large Language Model (LLM)-based methods have emerged as a promising alternative to address these challenges. However, LLM-based approaches often face difficulties in ensuring alignment between the generated code and the provided prompts. This paper presents SymRTLO, a novel neuron-symbolic RTL optimization framework that seamlessly integrates LLM-based code rewriting with symbolic reasoning techniques. Our method incorporates a retrieval-augmented generation (RAG) system of optimization rules and Abstract Syntax Tree (AST)-based templates, enabling LLM-based rewriting that maintains syntactic correctness while minimizing undesired circuit behaviors. A symbolic module is proposed for analyzing and optimizing finite state machine (FSM) logic, allowing fine-grained state merging and partial specification handling beyond the scope of pattern-based compilers. Furthermore, a fast verification pipeline, combining formal equivalence checks with test-driven validation, further reduces the complexity of verification. Experiments on the RTL-Rewriter benchmark with Synopsys Design Compiler and Yosys show that SymRTLO improves power, performance, and area (PPA) by up to 43.9%, 62.5%, and 51.1%, respectively, compared to the state-of-the-art methods.

cs.AR↗

arXiv2025

Dense Video Understanding with Gated Residual Tokenization

High temporal resolution is essential for capturing fine-grained details in video understanding. However, current video large language models (VLLMs) and benchmarks mostly rely on low-frame-rate sampling, such as uniform sampling or keyframe selection, discarding dense temporal information. This compromise avoids the high cost of tokenizing every frame, which otherwise leads to redundant computation and linear token growth as video length increases. While this trade-off works for slowly changing content, it fails for tasks like lecture comprehension, where information appears in nearly every frame and requires precise temporal alignment. To address this gap, we introduce Dense Video Understanding (DVU), which enables high-FPS video comprehension by reducing both tokenization time and token overhead. Existing benchmarks are also limited, as their QA pairs focus on coarse content changes. We therefore propose DIVE (Dense Information Video Evaluation), the first benchmark designed for dense temporal reasoning. To make DVU practical, we present Gated Residual Tokenization (GRT), a two-stage framework: (1) Motion-Compensated Inter-Gated Tokenization uses pixel-level motion estimation to skip static regions during tokenization, achieving sub-linear growth in token count and compute. (2) Semantic-Scene Intra-Tokenization Merging fuses tokens across static regions within a scene, further reducing redundancy while preserving dynamic semantics. Experiments on DIVE show that GRT outperforms larger VLLM baselines and scales positively with FPS. These results highlight the importance of dense temporal information and demonstrate that GRT enables efficient, scalable high-FPS video understanding.

cs.CV↗

arXiv2025

FCC feasibility studies: Impact of tracker- and calorimeter-detector performance on jet flavor identification and Higgs physics analyses

The extensive and ambitious physics program planned at the Future Circular Collider for electrons and positrons (FCC-ee) imposes strict constraints on detector performance. This work investigates how different detector properties impact jet flavor identification and their subsequent effects on high-profile physics analyses. Using Higgs boson coupling measurements and searches for invisible Higgs decays as benchmarks, we systematically evaluate the sensitivity of these analyses to tracker and calorimeter detector configurations. We examine variations in single-point resolution, material budget, silicon layer placement, and particle identification capabilities, quantifying their effects on flavor-tagging performance. Additionally, we present the first comprehensive study of Higgs-to-invisible decay detection using full detector simulation, providing important insights for optimizing future detector designs at lepton colliders.

hep-ex↗

arXiv2025

Dense Matter in Neutron Stars with eXTP

In this White Paper, we present the potential of the enhanced X-ray Timing and Polarimetry (eXTP) mission to constrain the equation of state of dense matter in neutron stars, exploring regimes not directly accessible to terrestrial experiments. By observing a diverse population of neutron stars - including isolated objects, X-ray bursters, and accreting systems - eXTP's unique combination of timing, spectroscopy, and polarimetry enables high-precision measurements of compactness, spin, surface temperature, polarimetric signals, and timing irregularity. These multifaceted observations, combined with advances in theoretical modeling, pave the way toward a comprehensive description of the properties and phases of dense matter from the crust to the core of neutron stars. Under development by an international Consortium led by the Institute of High Energy Physics of the Chinese Academy of Sciences, the eXTP mission is planned to be launched in early 2030.

astro-ph.HE↗

arXiv2025

Prospects for Time-Domain and Multi-Messenger Science with eXTP

In this new era of time-domain and multi-messenger astronomy, various new transients and new phenomena are constantly being discovered thanks to the rapid advances in observations, which provide the excellent opportunity to study the physics in the extreme environments. The enhanced X-ray Timing and Polarimetry mission (eXTP), planned to be launched in 2030, has several key advantages, including advanced polarimetry, high sensitivity & large effective area, and wide energy range coverage, which make it a groundbreaking project in high-energy astrophysics. In this article, we briefly introduce the potential time-domain and multi-messenger targets for eXTP, including gravitational-wave (GW) counterparts, gamma-ray bursts (GRBs), magnetars and fast radio bursts (FRBs), tidal disruption events (TDEs), supernovae, high energy neutrinos and TeV active galactic nucleus (AGNs), and so on. We discuss the advantages of future eXTP observations for detecting these sources, their detection capabilities, the abilities to distinguish theoretical models, and their applications in gravity and cosmology.

astro-ph.HE↗

arXiv2025

Tableau-Based Framework for Efficient Logical Quantum Compilation

Quantum computing holds the promise of solving problems intractable for classical computers, but practical large-scale quantum computation requires error correction to protect against errors. Fault-tolerant quantum computing (FTQC) enables reliable execution of quantum algorithms, yet they often demand substantial physical qubit overhead. Resource-efficient FTQC architectures minimize the number of physical qubits required, saving more than half compared to other architectures, but impose constraints that introduce up to 4.7$\times$ higher runtime overhead. In this paper, we present TQC, a \underline{T}ableau-based \underline{Q}uantum \underline{C}ompiler framework that minimizes FTQC runtime overhead without requiring additional physical qubits. By leveraging operation reorderability and latency hiding through parallel execution, TQC reduces FTQC runtime overhead by \textbf{2.57$\times$} on average. Furthermore, FTQC circuits often contain millions of gates, leading to substantial compilation overhead. To address this, we optimize the core data structure, the tableau, used in stabilizer formalism. We provide two tailored versions of the Tableau data type, each designed for different usage scenarios. These optimizations yield an overall performance improvement of more than \textbf{1000$\times$} compared to state-of-the-art FTQC optimization tools.

quant-ph↗

arXiv2025

Discovery of the anti-glitch in PSR J1835$-$1106

We report the detection of an anti-glitch with a fractional frequency change of $Δν/ν=-3.46(6)\times10^{-9}$ in the rotation-powered pulsar PSR J1835$-$1106 at MJD 55813, based on timing observations collected with the Nanshan 26-m and Parkes 64-m radio telescopes from January 2000 to July 2022. A comparison of the average pulse profiles within $\pm300$ d of the event reveals no significant morphological changes. We also estimate the angular velocity lag between the normal and superfluid components at the time of the glitch, showing that one of the superfluid glitch models is incompatible with PSR J1835$-$1106 due to its insufficient spin-down rate and angular velocity lag. The wind braking scenario offers a viable alternative, consistent with the observed spin-down behavior, glitch amplitude, and post-glitch recovery. High-cadence, high-sensitivity monitoring of similar events is essential to distinguish between internal (superfluid) and external (wind-related) glitch mechanisms.

astro-ph.HE↗

arXiv2025

CoMET: A Contrastive-Masked Brain Foundation Model for Universal EEG Representation

Electroencephalography (EEG) is a non-invasive technique for recording brain activity, widely used in brain-computer interfaces, clinic, and healthcare. Traditional EEG deep models typically focus on specific dataset and task, limiting model size and generalization. Recently, self-supervised brain foundation models have emerged and been applied to various downstream tasks. Nevertheless, these models still have limitations: current SOTA models typically rely on masked reconstruction strategy; however, EEG features of adjacent channels are highly correlated, which causes the pre-training to overly focus on low-dimensional signal-similarity features in local regions and neglect the global discriminative patterns vital for downstream tasks. To address these limitations, we propose a brain foundation model called CoMET. Specifically, we employ the masked autoencoder with redesigned patching and embedding for EEG as backbone and devise a novel contrastive learning framework with mirror-scale augmentation to strengthen the global discrimination ability. CoMET is pre-trained on mixed EEG datasets over 3000 subjects with over one million samples. It is evaluated on ten different downstream datasets, and the SOTA results demonstrate CoMET's superior ability in extracting universal EEG representations and strong clinical potential.

eess.SP↗

arXiv2025

Know When to Explore: Difficulty-Aware Certainty as a Guide for LLM Reinforcement Learning

Reinforcement Learning with Verifiable Feedback (RLVF) has become a key technique for enhancing the reasoning abilities of Large Language Models (LLMs). However, its reliance on sparse, outcome based rewards, which only indicate if a final answer is correct or not, fails to provide granular guidance on the reasoning process itself. This limitation hinders efficient learning, as the model cannot distinguish between high quality and inefficient solutions, nor can it learn effectively from different types of failures. To address this, we observe that an LLMs self-certainty often correlates with task difficulty and solution quality. We introduce Difficulty Aware Certainty guided Exploration (DACE), a novel RL algorithm that leverages this insight to dynamically balance the exploration exploitation trade-off. DACE assesses task difficulty online based on the policys success rate. It then uses this signal to modulate an intrinsic reward: for difficult tasks where the model is struggling, DACE encourages exploration by penalizing high certainty; for easier tasks, it encourages learning efficiency by rewarding high certainty. Experiments on challenging mathematical reasoning benchmarks (AIME, MATH) show that DACE significantly outperforms strong baselines. The DACE-trained models not only achieve higher accuracy but also demonstrate more robust performance when scaling test-time compute, validating that our adaptive approach fosters effective exploration without sacrificing precision.

cs.AI↗

arXiv2025

DualSparse-MoE: Coordinating Tensor/Neuron-Level Sparsity with Expert Partition and Reconstruction

Mixture of Experts (MoE) has become a mainstream architecture for building Large Language Models (LLMs) by reducing per-token computation while enabling model scaling. It can be viewed as partitioning a large Feed-Forward Network (FFN) at the tensor level into fine-grained sub-FFNs, or experts, and activating only a sparse subset for each input. While this sparsity improves efficiency, MoE still faces substantial challenges due to their massive computational scale and unpredictable activation patterns. To enable efficient MoE deployment, we identify dual sparsity at the tensor and neuron levels in pre-trained MoE modules as a key factor for both accuracy and efficiency. Unlike prior work that increases tensor-level sparsity through finer-grained expert design during pre-training, we introduce post-training expert partitioning to induce such sparsity without retraining. This preserves the mathematical consistency of model transformations and enhances both efficiency and accuracy in subsequent fine-tuning and inference. Building upon this, we propose DualSparse-MoE, an inference system that integrates dynamic tensor-level computation dropping with static neuron-level reconstruction to deliver significant efficiency gains with minimal accuracy loss. Experimental results show that enforcing an approximate 25% drop rate with our approach reduces average accuracy by only 0.08%-0.28% across three prevailing MoE models, while nearly all degrees of computation dropping consistently yield proportional computational speedups. Furthermore, incorporating load-imbalance awareness into expert parallelism achieves a 1.41x MoE module speedup with just 0.5% average accuracy degradation.

cs.LG↗

arXiv2025

Unbinned inclusive cross-section measurements with machine-learned systematic uncertainties

We introduce a novel methodology for addressing systematic uncertainties in unbinned inclusive cross-section measurements and related collider-based inference problems. Our approach incorporates known analytic dependencies on parameters of interest, including signal strengths and nuisance parameters. When these dependencies are unknown, as is frequently the case for systematic uncertainties, dedicated neural network parametrizations provide an approximation that is trained on simulated data. The resulting machine-learned surrogate captures the complete parameter dependence of the likelihood ratio, providing a near-optimal test statistic. As a case study, we perform a first-principles inclusive cross-section measurement of $\textrm{H}\rightarrowττ$ in the single-lepton channel, utilizing simulated data from the FAIR Universe Higgs Uncertainty Challenge. Results in Asimov data, from large-scale toy studies, and using the Fisher information demonstrate significant improvements over traditional binned methods. Our computer code ``Guaranteed Optimal Log-Likelihood-based Unbinned Method'' (GOLLUM) for machine-learning and inference is publicly available.

hep-ph↗

arXiv2025

STABSim: A Parallelized Clifford Simulator with Features Beyond Direct Simulation

The quantum stabilizer formalism became foundational for understanding error correction soon after the realization of the first useful quantum error correction codes. Stabilizers provide a way to describe sets of quantum states which are valid codewords within a quantum error correction (QEC) scheme. Existing stabilizer simulators are single threaded applications used to sample larger codes than is possible with other methods. However, there is an outstanding gap in the scaling and accuracy of current simulators for QEC as quantum computing exceeds hundreds of qubits, along with an under-utilization of the capabilities of highly-efficient stabilizer simulation across other quantum domains. In this work, we present the first GPU-accelerated tableau stabilizer simulator to scale better than CPU methods in QEC workloads, by trivializing Clifford gates and exploiting the large parallelism of dedicated GPUs with CUDA warp-level primitives to quickly overcome costly measurement gates. We then implement a new error model that captures non-unitarity in T1/T2 error channels much faster and with exact accuracy for most physical qubits, demonstrate a chemistry use case, and present a new Clifford+T to Pauli-Based Computing (PBC) transpilation optimization through our simulator.

quant-ph↗

arXiv2025

G1: Teaching LLMs to Reason on Graphs with Reinforcement Learning

Although Large Language Models (LLMs) have demonstrated remarkable progress, their proficiency in graph-related tasks remains notably limited, hindering the development of truly general-purpose models. Previous attempts, including pretraining graph foundation models or employing supervised fine-tuning, often face challenges such as the scarcity of large-scale, universally represented graph data. We introduce G1, a simple yet effective approach demonstrating that Reinforcement Learning (RL) on synthetic graph-theoretic tasks can significantly scale LLMs' graph reasoning abilities. To enable RL training, we curate Erdõs, the largest graph reasoning dataset to date comprising 50 diverse graph-theoretic tasks of varying difficulty levels, 100k training data and 5k test data, all drived from real-world graphs. With RL on Erdõs, G1 obtains substantial improvements in graph reasoning, where our finetuned 3B model even outperforms Qwen2.5-72B-Instruct (24x size). RL-trained models also show strong zero-shot generalization to unseen tasks, domains, and graph encoding schemes, including other graph-theoretic benchmarks as well as real-world node classification and link prediction tasks, without compromising general reasoning abilities. Our findings offer an efficient, scalable path for building strong graph reasoners by finetuning LLMs with RL on graph-theoretic tasks, which combines the strengths of pretrained LLM capabilities with abundant, automatically generated synthetic data, suggesting that LLMs possess graph understanding abilities that RL can elicit successfully. Our implementation is open-sourced at https://github.com/PKU-ML/G1, with models and datasets hosted on Hugging Face collections https://huggingface.co/collections/PKU-ML/g1-683d659e992794fc99618cf2 for broader accessibility.

cs.LG↗