arXiv subjects

Yang Gao

Publications and source records attributed to Yang Gao.

At least 271 records · Page 15Linked to original sources

arXiv2025

MIXRTs: Toward Interpretable Multi-Agent Reinforcement Learning via Mixing Recurrent Soft Decision Trees

While achieving tremendous success in various fields, existing multi-agent reinforcement learning (MARL) with a black-box neural network makes decisions in an opaque manner that hinders humans from understanding the learned knowledge and how input observations influence decisions. In contrast, existing interpretable approaches usually suffer from weak expressivity and low performance. To bridge this gap, we propose MIXing Recurrent soft decision Trees (MIXRTs), a novel interpretable architecture that can represent explicit decision processes via the root-to-leaf path and reflect each agent's contribution to the team. Specifically, we construct a novel soft decision tree using a recurrent structure and demonstrate which features influence the decision-making process. Then, based on the value decomposition framework, we linearly assign credit to each agent by explicitly mixing individual action values to estimate the joint action value using only local observations, providing new insights into interpreting the cooperation mechanism. Theoretical analysis confirms that MIXRTs guarantee additivity and monotonicity in the factorization of joint action values. Evaluations on complex tasks like Spread and StarCraft II demonstrate that MIXRTs compete with existing methods while providing clear explanations, paving the way for interpretable and high-performing MARL systems.

cs.LG↗

arXiv2025

Search for $Λ$-$\barΛ $ oscillation in $J/ψ\rightarrowΛ\barΛ$ decay

Using $(10087\pm44)\times 10^{6}$ $J/ψ$ decays collected by the BESIII detector at the BEPCII collider, we search for baryon number violation via $Λ-\barΛ$ oscillation in the decay $J/ψ\to Λ\barΛ$. No evidence for $Λ-\barΛ$ oscillation is observed. The upper limit on the time-integrated probability of $Λ-\barΛ$ oscillation is estimated to be $1.4\times 10^{-6}$, corresponding to an oscillation parameter less than $2.1\times 10^{-18}~\mathrm{GeV}$ at $90\%$ confidence level.

hep-ex↗

arXiv2025

RRM: Robust Reward Model Training Mitigates Reward Hacking

Reward models (RMs) play a pivotal role in aligning large language models (LLMs) with human preferences. However, traditional RM training, which relies on response pairs tied to specific prompts, struggles to disentangle prompt-driven preferences from prompt-independent artifacts, such as response length and format. In this work, we expose a fundamental limitation of current RM training methods, where RMs fail to effectively distinguish between contextual signals and irrelevant artifacts when determining preferences. To address this, we introduce a causal framework that learns preferences independent of these artifacts and propose a novel data augmentation technique designed to eliminate them. Extensive experiments show that our approach successfully filters out undesirable artifacts, yielding a more robust reward model (RRM). Our RRM improves the performance of a pairwise reward model trained on Gemma-2-9b-it, on RewardBench, increasing accuracy from 80.61% to 84.15%. Additionally, we train two DPO policies using both the RM and RRM, demonstrating that the RRM significantly enhances DPO-aligned policies, improving MT-Bench scores from 7.27 to 8.31 and length-controlled win-rates in AlpacaEval-2 from 33.46% to 52.49%.

cs.CL↗

arXiv2025

LongSafety: Enhance Safety for Long-Context LLMs

Recent advancements in model architectures and length extrapolation techniques have significantly extended the context length of large language models (LLMs), paving the way for their application in increasingly complex tasks. However, despite the growing capabilities of long-context LLMs, the safety issues in long-context scenarios remain underexplored. While safety alignment in short context has been widely studied, the safety concerns of long-context LLMs have not been adequately addressed. In this work, we introduce \textbf{LongSafety}, a comprehensive safety alignment dataset for long-context LLMs, containing 10 tasks and 17k samples, with an average length of 40.9k tokens. Our experiments demonstrate that training with LongSafety can enhance long-context safety performance while enhancing short-context safety and preserving general capabilities. Furthermore, we demonstrate that long-context safety does not equal long-context alignment with short-context safety data and LongSafety has generalizing capabilities in context length and long-context safety scenarios.

cs.CL↗

arXiv2025

Data-free Weight Compress and Denoise for Large Language Models

Large Language Models (LLMs) are reshaping the research landscape in artificial intelligence, particularly as model parameters scale up significantly, unlocking remarkable capabilities across various domains. Nevertheless, the scalability of model parameters faces constraints due to limitations in GPU memory and computational speed. To address these constraints, various weight compression methods have emerged, such as Pruning and Quantization. Given the low-rank nature of weight matrices in language models, the reduction of weights through matrix decomposition undoubtedly holds significant potential and promise. In this paper, drawing upon the intrinsic structure of LLMs, we propose a novel approach termed Data-free Joint Rank-k Approximation for compressing the parameter matrices. Significantly, our method is characterized by without necessitating additional involvement of any corpus, while simultaneously preserving orthogonality in conjunction with pruning and quantization methods. We achieve a model pruning of 80% parameters while retaining 93.43% of the original performance without any calibration data. Additionally, we explore the fundamental properties of the weight matrix of LLMs undergone Rank-k Approximation and conduct comprehensive experiments to elucidate our hypothesis.

cs.CL↗

arXiv2025

Measurement of the Decay $Ξ^{0}\toΛγ$ with Entangled $Ξ^{0}\barΞ^{0}$ Pairs

In this Letter, a systematic study of the weak radiative hyperon decay $Ξ^{0}\toΛγ$ at an electron-positron collider using entangled $Ξ^{0}\barΞ^{0}$ pair events is presented. The absolute branching fraction for this decay has been measured for the first time, and is $\left(1.347 \pm 0.066_{\mathrm stat.}\pm0.054_{\mathrm syst.}\right)\times 10^{-3}$. The decay asymmetry parameter, which characterizes the effect of parity violation in the decay, is determined to be $-0.741 \pm 0.062_{\mathrm stat.}\pm 0.019_{\mathrm syst.}$. The obtained results are consistent with the world average values within the uncertainties, offering valuable insights into the underlying mechanism governing the weak radiative hyperon decays. The charge conjugation parity ($CP$) symmetries of branching fraction and decay asymmetry parameter in the decay are also studied. No statistically significant violation of charge conjugation parity symmetry is observed.

hep-ex↗

arXiv2025

Partial wave analyses of $ψ(3686)\to p\bar{p}π^0$ and $ψ(3686)\to p\bar{p}η$

Using a sample of $(2712\pm14)\times10^6$ $ψ(3686)$ events collected with the BESIII detector, we perform partial wave analyses of the decays $ψ(3686)\to p\bar{p}π^0$ and $ψ(3686)\to p\bar{p}η$. The branching fractions of $ψ(3686)\to p\bar{p}π^0$ and $ψ(3686)\to p\bar{p}η$ are determined to be $(133.9\pm11.2\pm2.3)\times10^{-6}$ or $(183.7\pm13.7\pm3.2)\times10^{-6}$ and $(61.5\pm6.5\pm1.1)\times10^{-6}$ or $(84.4\pm6.9\pm1.4)\times10^{-6}$, respectively, where the two solutions are caused by an ambiguous phase angle between resonant and continuum processes. Several well-established $N^*$ states are observed in the $pπ^0$ and $pη$ systems, and the corresponding branching fractions are measured. The ratio of decay widths $Γ_{N(1535)\to Nη}/Γ_{N(1535)\to Nπ}$ is determined to be $0.99\pm0.05\pm0.19$.

hep-ex↗

arXiv2025

Measurement of the Inclusive Cross Sections of Prompt $J/ψ$ and $ψ(3686)$ Production in $e^{+}e^{-}$ Annihilation from $\sqrt{s}=3.808$ to $4.951$ GeV

The inclusive cross sections of prompt $J/ψ$ and $ψ(3686)$ production are measured at center-of-mass energies from 3.808 to 4.951 GeV. The dataset used is 22 fb$^{-1}$ of $e^{+}e^{-}$ annihilation data collected with the BESIII detector operating at the BEPCII storage ring. The results obtained are in agreement with the previous BESIII measurements of exclusive $J/ψ$ and $ψ(3686)$ production. The average values obtained for the cross sections measured in the center-of-mass energy ranges from 4.527 to 4.951 GeV for $J/ψ$ and from 4.843 to 4.951 GeV for $ψ(3686)$, where the impact of known resonances is negligible, are $14.0\pm1.7\pm3.1$ pb and $15.3\pm3.0$ pb, respectively. For $J/ψ$, the first and the second uncertainties are statistical and systematic, respectively. For $ψ(3686)$, the uncertainty is total. These values are useful for testing charmonium production models.

hep-ex↗

arXiv2025

Fine-Tuning Hard-to-Simulate Objectives for Quadruped Locomotion: A Case Study on Total Power Saving

Legged locomotion is not just about mobility; it also encompasses crucial objectives such as energy efficiency, safety, and user experience, which are vital for real-world applications. However, key factors such as battery power consumption and stepping noise are often inaccurately modeled or missing in common simulators, leaving these aspects poorly optimized or unaddressed by current sim-to-real methods. Hand-designed proxies, such as mechanical power and foot contact forces, have been used to address these challenges but are often problem-specific and inaccurate. In this paper, we propose a data-driven framework for fine-tuning locomotion policies, targeting these hard-to-simulate objectives. Our framework leverages real-world data to model these objectives and incorporates the learned model into simulation for policy improvement. We demonstrate the effectiveness of our framework on power saving for quadruped locomotion, achieving a significant 24-28\% net reduction in total power consumption from the battery pack at various speeds. In essence, our approach offers a versatile solution for optimizing hard-to-simulate objectives in quadruped locomotion, providing an easy-to-adapt paradigm for continual improving with real-world knowledge. Project page https://hard-to-sim.github.io/.

cs.RO↗

arXiv2025

Video2Policy: Scaling up Manipulation Tasks in Simulation through Internet Videos

Simulation offers a promising approach for cheaply scaling training data for generalist policies. To scalably generate data from diverse and realistic tasks, existing algorithms either rely on large language models (LLMs) that may hallucinate tasks not interesting for robotics; or digital twins, which require careful real-to-sim alignment and are hard to scale. To address these challenges, we introduce Video2Policy, a novel framework that leverages internet RGB videos to reconstruct tasks based on everyday human behavior. Our approach comprises two phases: (1) task generation in simulation from videos; and (2) reinforcement learning utilizing in-context LLM-generated reward functions iteratively. We demonstrate the efficacy of Video2Policy by reconstructing over 100 videos from the Something-Something-v2 (SSv2) dataset, which depicts diverse and complex human behaviors on 9 different tasks. Our method can successfully train RL policies on such tasks, including complex and challenging tasks such as throwing. Finally, we show that the generated simulation data can be scaled up for training a general policy, and it can be transferred back to the real robot in a Real2Sim2Real way.

cs.RO↗

arXiv2025

Towards Empowerment Gain through Causal Structure Learning in Model-Based RL

In Model-Based Reinforcement Learning (MBRL), incorporating causal structures into dynamics models provides agents with a structured understanding of the environments, enabling efficient decision. Empowerment as an intrinsic motivation enhances the ability of agents to actively control their environments by maximizing the mutual information between future states and actions. We posit that empowerment coupled with causal understanding can improve controllability, while enhanced empowerment gain can further facilitate causal reasoning in MBRL. To improve learning efficiency and controllability, we propose a novel framework, Empowerment through Causal Learning (ECL), where an agent with the awareness of causal dynamics models achieves empowerment-driven exploration and optimizes its causal structure for task learning. Specifically, ECL operates by first training a causal dynamics model of the environment based on collected data. We then maximize empowerment under the causal structure for exploration, simultaneously using data gathered through exploration to update causal dynamics model to be more controllable than dense dynamics model without causal structure. In downstream task learning, an intrinsic curiosity reward is included to balance the causality, mitigating overfitting. Importantly, ECL is method-agnostic and is capable of integrating various causal discovery methods. We evaluate ECL combined with 3 causal discovery methods across 6 environments including pixel-based tasks, demonstrating its superior performance compared to other causal MBRL methods, in terms of causal discovery, sample efficiency, and asymptotic performance.

cs.AI↗

arXiv2025

Causal Information Prioritization for Efficient Reinforcement Learning

Current Reinforcement Learning (RL) methods often suffer from sample-inefficiency, resulting from blind exploration strategies that neglect causal relationships among states, actions, and rewards. Although recent causal approaches aim to address this problem, they lack grounded modeling of reward-guided causal understanding of states and actions for goal-orientation, thus impairing learning efficiency. To tackle this issue, we propose a novel method named Causal Information Prioritization (CIP) that improves sample efficiency by leveraging factored MDPs to infer causal relationships between different dimensions of states and actions with respect to rewards, enabling the prioritization of causal information. Specifically, CIP identifies and leverages causal relationships between states and rewards to execute counterfactual data augmentation to prioritize high-impact state features under the causal understanding of the environments. Moreover, CIP integrates a causality-aware empowerment learning objective, which significantly enhances the agent's execution of reward-guided actions for more efficient exploration in complex environments. To fully assess the effectiveness of CIP, we conduct extensive experiments across 39 tasks in 5 diverse continuous control environments, encompassing both locomotion and manipulation skills learning with pixel-based and sparse reward settings. Experimental results demonstrate that CIP consistently outperforms existing RL methods across a wide range of scenarios.

cs.AI↗

arXiv2025

Search for the decay $χ_{c1}(3872)\toπ^{+}π^{-}χ_{c1}$

Using a data sample corresponding to an integrated luminosity of 10.9 fb$^{-1}$ collected at center-of-mass energies from 4.16 to 4.34 GeV with the BESIII detector, we search for the decay $χ_{c1}(3872) \to π^{+}π^{-}χ_{c1}$ in the radiative production $e^{+}e^{-} \to γχ_{c1}(3872)$. No significant signal is observed, and the ratio for the branching fraction of $χ_{c1}(3872) \to π^{+}π^{-}χ_{c1}$ to $χ_{c1}(3872) \to π^{+}π^{-}J/ψ$ is measured as $\mathcal{R}\equiv\frac{\mathcal{B}[χ_{c1}(3872) \to π^{+}π^{-}χ_{c1}]}{\mathcal{B}[χ_{c1}(3872) \to π^{+}π^{-} J/ψ]}<0.18$ at 90$\%$ confidence level. The upper limit on the product of the cross section $σ[e^{+}e^{-}\toγχ_{c1}(3872)]$ and the branching fraction $\mathcal{B}[χ_{c1}(3872)\toπ^{+}π^{-}χ_{c1}]$ at each center-of-mass energy is also given. These measurements favor the non-conventional charmonium nature of the $χ_{c1}(3872)$ state.

hep-ex↗

arXiv2025

Search for the $e^+e^- \to ϕχ_{c1}(3872)$ process at BESIII

Based on 368.5 pb$^{-1}$ of $e^+e^-$ collision data collected at center-of-mass energies 4.914 and 4.946 GeV by the BESIII detector, the $e^+e^- \to ϕχ_{c1}(3872)$ process is searched for the first time. No significant signal is observed and the upper limits at the 90\% confidence level on the product of the Born cross section $σ(e^+e^- \to ϕχ_{c1}(3872))$ and the branching fraction $\mathcal{B}[χ_{c1}(3872)\toπ^+π^- J/ψ]$ at 4.914 and 4.946 GeV are set to be 0.85 and 0.96 pb, respectively. These measurements provide useful information for the production of the $χ_{c1}(3872)$ at $e^+e^-$ collider and deepen our understanding about the nature of this particle.

hep-ex↗

arXiv2025

Learning Manipulation Skills through Robot Chain-of-Thought with Sparse Failure Guidance

Defining reward functions for skill learning has been a long-standing challenge in robotics. Recently, vision-language models (VLMs) have shown promise in defining reward signals for teaching robots manipulation skills. However, existing work often provides reward guidance that is too coarse, leading to insufficient learning processes. In this paper, we address this issue by implementing more fine-grained reward guidance. We decompose tasks into simpler sub-tasks, using this decomposition to offer more informative reward guidance with VLMs. We also propose a VLM-based self imitation learning process to speed up learning. Empirical evidence demonstrates that our algorithm consistently outperforms baselines such as CLIP, LIV, and RoboCLIP. Specifically, our algorithm achieves a $5.4 \times$ higher average success rates compared to the best baseline, RoboCLIP, across a series of manipulation tasks.

cs.RO↗

arXiv2025

LLM4GNAS: A Large Language Model Based Toolkit for Graph Neural Architecture Search

Graph Neural Architecture Search (GNAS) facilitates the automatic design of Graph Neural Networks (GNNs) tailored to specific downstream graph learning tasks. However, existing GNAS approaches often require manual adaptation to new graph search spaces, necessitating substantial code optimization and domain-specific knowledge. To address this challenge, we present LLM4GNAS, a toolkit for GNAS that leverages the generative capabilities of Large Language Models (LLMs). LLM4GNAS includes an algorithm library for graph neural architecture search algorithms based on LLMs, enabling the adaptation of GNAS methods to new search spaces through the modification of LLM prompts. This approach reduces the need for manual intervention in algorithm adaptation and code modification. The LLM4GNAS toolkit is extensible and robust, incorporating LLM-enhanced graph feature engineering, LLM-enhanced graph neural architecture search, and LLM-enhanced hyperparameter optimization. Experimental results indicate that LLM4GNAS outperforms existing GNAS methods on tasks involving both homogeneous and heterogeneous graphs.

cs.LG↗

arXiv2025

Partial Wave Analysis of $J/ψ\rightarrow γγϕ$

Using a sample of $(10087\pm44)\times10^{6}$ $J/ψ$ events collected by the BESIII detector at the BEPCII collider, a partial-wave analysis of the decay $γγϕ$ is performed to investigate the intermediate resonances in $J/ψ\rightarrowγR, R\arγϕ$. Resonances including $f_{1}(1285)$, $η(1405)$, $f_{1}(1420)$, $f_{1}(1510)$, $f_{2}(1525)$, $X(1835)$, $f_{2}(1950)$, $f_{2}(2010)$, $f_{0}(2200)$ and $η_{c}$ have been detected with statistical significance greater than 5$σ$. The product branching fractions $\mathcal{B}(J/ψ\rightarrowγR, R\ar γϕ)$ are reported, and the resonance parameters of $η(1405)$ and $X(1835)$ have been precisely determined. It is more probable that $η(1405)$ and $X(1835)$ act as the first - and second - excited states of $η^{\prime}$, respectively. The observation of $η_{c}\rightarrowγϕ$ holds great significance for the in depth study of charmonium properties. No evidence of $η(1295)$, $η_{1}(1855)$ or X(2370) is found. The measured upper limits of relevant BF products do not conflict with theoretical expectations for X(2370) as a pseudoscalar glueball and $η_{1}(1855)$ as a hybrid state.

hep-ex↗

arXiv2025

Efficient Implementation of Arbitrary Two-Qubit Gates via Unified Control

The native gate set is fundamental to the performance of quantum devices, as it governs the accuracy of basic quantum operations and dictates the complexity of implementing quantum algorithms. Traditional approaches to extending gate sets often require accessing multiple transitions within an extended Hilbert space, leading to increased control complexity while offering only a limited set of gates. Here, we experimentally demonstrate a unified and highly versatile gate scheme capable of natively generating arbitrary two-qubit gates using only exchange interaction and qubit driving on a superconducting quantum processor, achieving maximum expressivity. Using a state-of-the-art transmon-coupler-transmon architecture, we achieve high fidelities averaging $99.37 \pm 0.07\%$ across a wide range of commonly used two-qubit unitaries. This outstanding performance, combined with reduced complexity, enables precise multipartite entangled state preparation, as demonstrated. To further enhance its applicability, we also show the high-fidelity realization of the unique B gate, which efficiently synthesizes the entire family of two-qubit gates. Our results highlight that fully exploiting the capabilities of a single interaction can yield a comprehensive and highly accurate gate set. With maximum expressivity, gate-time optimality, demonstrated high fidelity, and easy adaptability to other quantum platforms, our unified control scheme paves the way for optimal performance in quantum devices, offering exciting prospects for advancing quantum hardware and algorithm development.

quant-ph↗