arXiv subjects

Yichen Sun

Publications and source records attributed to Yichen Sun.

At least 19 recordsLinked to original sources

arXiv2026

SCOPE: Selective Critical-Step On-Policy Self-Distillation for Agentic Reinforcement Learning

Outcome-based reinforcement learning provides coarse supervision for long-horizon LLM agents: failed trajectories may contain correct steps, while successful trajectories can contain errors. Trajectory-level skills can also shift teacher preferences at unrelated steps, introducing off-target distillation signals. We introduce SCOPE, a framework for Selective Critical-Step On-Policy Self-Distillation. After a supervised warm-up, the policy analyzes its own trajectories, generates step-specific hindsight skills, and distills their guidance only at selected error steps. A bounded signed teacher-student coefficient stabilizes auxiliary updates while preserving positive reinforcement and negative correction. A success-relative pairwise length reward further encourages efficient task completion. Experiments on ALFWorld and WebShop demonstrate consistent improvements across multiple model backbones. With Qwen2.5-3B, SCOPE achieves 97.7% success on ALFWorld and 85.2% on WebShop, outperforming SEED by 5.9 and 6.3 percentage points, respectively. On unseen ALFWorld tasks, SCOPE reaches 89.6% success, improving over GRPO by 23.2 percentage points with 46.7% fewer interactions. Scaling to Qwen3-32B, SCOPE achieves 99.2% success on seen ALFWorld tasks and 94.0% on unseen tasks.

cs.AI↗

arXiv2026

EOPSA: Efficient On-Policy Self-Distilled Safety Alignment

On-Policy Self-Distillation (OPSD) has emerged as a promising paradigm for safety alignment, delivering dense, token-level supervision by distilling from a teacher conditioned on refusal-oriented privileged prompts. However, we reveal that this paradigm suffers from critical inefficiencies that degrade both training efficiency and general reasoning capabilities. Specifically, we diagnose two fundamental bottlenecks: (1) supervisory collapse over extended rollouts, where the teacher's corrective efficacy degrades precipitously as the student's generation prefix lengthens, injecting noisy gradients into late-stage tokens; and (2) gradient dilution from stylistic shifts, where the distillation objective is dominated by safety-irrelevant stylistic discrepancies induced by privileged prompting, washing out genuine safety signals and impairing base reasoning. To resolve these issues, we propose Efficient On-Policy Self-Distilled Safety Alignment (EOPSA), which concentrates computational and gradient budgets exclusively on reliably supervised, safety-critical tokens. EOPSA incorporates two coordinated mechanisms: (i) Adaptive Rollout Scheduling, which dynamically bounds the generation horizon guided by a novel Teacher Rescue Rate (TRR) metric to operate strictly within reliable supervision regimes; and (ii) Selective Distillation, which filters out safety-neutral tokens to restrict gradient updates exclusively to safety-pivotal transitions. Extensive evaluations across reasoning models up to 32B parameters demonstrate that EOPSA slashes rollout computation by $\sim$50% and backpropagates through merely $\sim$2% of tokens, consistently outperforming full-token distillation baselines in both safety compliance and reasoning retention.

cs.AI↗

arXiv2026

CleanRM: Reward Modeling from Noisy Feedback with a Few Clean Feedback for RLHF

Feedback noise is pervasive in preference data, arising from subjective human judgments, annotation mistakes, and imperfect LLM judges. Standard reward modeling trained on such feedback yields a biased estimate of the clean risk. Although classical denoising methods are directly applicable to noisy feedback learning, they generally assume a uniform noise ratio across samples. However, this assumption fails for feedback data, where noise is intrinsically sample-wise. To address this challenge, we propose CleanRM, which uses a few clean feedback to calibrate sample-wise noise ratios via optimal transport. Specifically, CleanRM aligns the joint distribution of preferences reverted from noisy feedback with that of clean feedback and then trains the reward model using a noise-corrected surrogate loss. Theoretically, we prove that the resulting objective is unbiased with respect to the clean risk. Extensive experiments demonstrate that CleanRM outperforms state-of-the-art denoising baselines and yields more effective downstream RLHF alignment.

cs.AI↗

arXiv2026

Rubric Judgments Lack Uncertainty: Conformal Weighting for LLM Evaluation

Rubric-based evaluation is widely used to assess large language model (LLM) outputs by decomposing quality into multiple criteria and assigning each a binary judgment from an LLM verifier. However, judgment certainty varies substantially across criteria. Existing methods uniformly aggregate these judgments into a final score, implicitly treating every criterion as equally certain and overlooking this heterogeneity. As a result, uncertain judgments remain incorporated into reward estimation without distinction, potentially yielding misleading training signals for downstream reinforcement learning (RL). To address this limitation, we propose Uncertainty-aware Rubric Weighting (URW), a training-free framework that quantifies criterion-level judgment uncertainty via conformal calibration. Specifically, URW elicits a soft uncertainty signal from a fixed pool of paraphrased judgment templates and converts it into a per-criterion weight using split-conformal $p$-values with provable coverage guarantees. Extensive experiments demonstrate that URW consistently improves pairwise preference accuracy and delivers stronger downstream RL alignment.

cs.AI↗

arXiv2026

Executable but Unfaithful: Mock Object, Stub, and Assertion Generation by LLMs and Coding Agents

Mocking is a foundational practice in unit testing, enabling developers to isolate the code under test by replacing real dependencies with simulated alternatives. Automating the generation of mock objects, stubs, and mock assertions has been a long-standing research goal. However, capture-and-replay approaches rely on recording the runtime behavior of actual dependencies. Evolutionary approaches such as StubCoder overcome this requirement but constrain generation to predefined symbol pools. Large language models (LLMs) and coding agents offer an alternative by generating mocking artifacts from source code without requiring recorded dependency executions or fixed symbol pools. Yet, no systematic study has evaluated their capabilities across all three artifact types. To address this gap, we construct MockBench, a benchmark comprising 165 mock artifact generation tasks from real-world Java and Python projects, with validated reference mock objects, stubs, and mock assertions. We evaluate six LLMs and three coding agents in terms of executability, behavior fidelity, and interaction-fault detection capability, complemented by comparisons with existing tools and sensitivity analysis. Experimental results show that LLMs and coding agents indeed have been the state-of-the-art techniques in mock-based testing. More specifically, compared with LLMs, coding agents substantially improve executability, while these gains do not consistently translate into higher behavior fidelity or better interaction-fault detection. Furthermore, we analyze 660 failures from LLMs and coding agents, and identify ten error types across the three artifact categories, revealing behavioral deviations that passing tests can conceal. Our findings highlight the challenges of mock artifact generation and provide practical guidance for both developers and researchers.

cs.SE↗

arXiv2026

DeShortcut-Align: Decoupling Spurious Shortcuts for Robust Safety Alignment in Large Reasoning Models

Safety alignment of large reasoning models (LRMs) via supervised fine-tuning (SFT) and reinforcement learning (RL) often yields near-perfect safety scores, yet this apparent success comes at the cost of severe over-refusal and degraded general capabilities. Through systematic empirical analysis, we find that these failures are closely associated with the learning of spurious shortcuts rather than robust intent-sensitive safety evaluation. Specifically, we identify two dominant shortcuts: formatting shortcuts, where refusal behaviors are overly bound to structural prompt templates that frequently appear in safety alignment corpora; and lexical shortcuts, where sensitive keywords reflexively trigger refusals on benign queries. To mitigate reliance on these shortcuts, we propose DeShortcut-Align, a shortcut-decoupling alignment framework that reduces dependence on superficial cues. DeShortcut-Align operates across three coordinated stages: (1) Refusal Sensitivity Attribution, which masks input tokens to quantify their impact on the final refusal response distribution; (2) Attribution-Guided Contrastive Augmentation, which constructs benign contrastive samples using high-sensitivity tokens to mitigate lexical shortcuts; and (3) Counterfactual Consistency Regularization, which constructs template-ablated states via attention blinding to enforce decision consistency across SFT and RL, mitigating formatting shortcut dependence. Experiments on 7B and 14B models demonstrate that DeShortcut-Align significantly improves robustness against template-stripping bypass attacks (reducing performance drops by up to 72%), substantially reduces over-refusal by over 58%, and better preserves general-purpose reasoning capabilities, thereby mitigating the alignment tax commonly observed in safety training.

cs.AI↗

arXiv2026

Low-energy Muon-Nucleon scattering experiment: LUNE (White Paper)

The HIAF will provide high-intensity, high-quality muon beams with momenta from 0.5 to 7.5 GeV/c. This energy range is uniquely suited for precision muon scattering, bridging the gap between low-energy electron facilities and future high-energy lepton-ion colliders. In particular, HIAF will enable precision measurements with both positive and negative muon beams over a broad kinematic range, complementing existing electron-scattering facilities such as JLab, EicC and EIC. Based on HIAF muon source, the LUNE Collaboration has been established to address several fundamental questions in nuclear and particle physics, including the proton charge radius puzzle, nucleon electromagnetic structure, and the dynamics of quantum electrodynamics and hadronic interactions. The program proceeds in two phases, from elastic scattering to nucleon structure and beyond-Standard-Model searches. The experiment is expected to determine the proton charge radius with a precision of approximately 1.0\% using elastic muon-proton scattering. It will also perform systematic measurements of the proton electromagnetic form factors with both $μ^+$ and $μ^-$ beams, enabling precise studies of two-photon exchange effects and stringent tests of quantum electrodynamics. Beyond elastic scattering, LUNE will investigate TMD, gravitational form factors, and nuclear charge radii, providing new insights into the 3D structure of nucleons and nuclei. The experiment will further address important topics including Coulomb-distortion corrections, nuclear medium effects, and possible signatures of physics beyond the Standard Model. This white paper presents the scientific motivation, detector concept, expected performance, and long-term strategy of LUNE.

hep-ex↗

arXiv2026

Uncertainty-Aware Reward Modeling for Stable RLHF

Reinforcement learning from human feedback (RLHF) aligns large language models by training reward models on preference data and optimizing policies to maximize predicted rewards. However, this pipeline faces two fundamental challenges: (1) reward models cannot signal when their predictions are unreliable, since they usually act as deterministic point estimators; and (2) modern group-based policy optimization can amplify unreliable reward signals, as exemplified by GRPO's uniform treatment of rewards during advantage computation. As policies explore increasingly diverse responses, these two limitations create a critical vulnerability: unreliable reward estimates may be granted disproportionate influence, triggering severe reward hacking. We propose Uncertainty-Aware Reward Modeling (UARM), which equips reward models with calibrated uncertainty via quantile-based conformal prediction and reweights GRPO advantages through heteroscedastic variance decomposition. Experiments across HelpSteer, UltraFeedback, and PKU-SafeRLHF demonstrate that UARM significantly improves reward model calibration, reduces reward hacking, and enhances downstream alignment quality compared to standard GRPO and uncertainty-agnostic baselines.

cs.LG↗

arXiv2026

SWAP: Towards Copyright Auditing of Soft Prompts via Sequential Watermarking

Large-scale vision-language models, especially CLIP, have demonstrated remarkable performance across diverse downstream tasks. Soft prompts, as carefully crafted modules that efficiently adapt vision-language models to specific tasks, necessitate effective copyright protection. In this paper, we investigate model copyright protection by auditing whether suspicious third-party models incorporate protected soft prompts. While this can be viewed as a special case of model ownership auditing, our analysis shows that existing techniques are ineffective due to prompt learning's unique characteristics. Non-intrusive auditing is inherently prone to false positives when independent models share similar data distributions with victim models. Intrusive approaches also fail: backdoor methods designed for CLIP cannot embed functional triggers, while extending traditional DNN backdoor techniques to prompt learning suffers from harmfulness and ambiguity challenges. We find that these failures in intrusive auditing stem from the same fundamental reason: watermarking operates within the same decision space as the primary task yet pursues opposing objectives. Motivated by these findings, we propose sequential watermarking for soft prompts (SWAP), which implants watermarks into a different and more complex space. SWAP encodes watermarks through a specific order of defender-specified out-of-distribution classes, inspired by the zero-shot prediction capability of CLIP. This watermark, which is embedded in a more complex space, keeps the original prediction label unchanged, making it less opposed to the primary task. We further design a hypothesis-test-guided verification protocol for SWAP and provide a theoretical analysis of when verification works. Extensive experiments on 11 datasets demonstrate SWAP's effectiveness, harmlessness, and robustness against potential attacks.

cs.CR↗

arXiv2026

Fixturize: Bridging the Fixture Gap in Test Generation

Current Large Language Models (LLMs) have advanced automated unit test generation but face a critical limitation: they often neglect to construct the necessary test fixtures, which are the environmental setups required for a test to run. To bridge this gap, this paper proposes Fixturize, a diagnostic framework that proactively identifies fixture-dependent functions and synthesizes test fixtures accordingly through an iterative, feedback-driven process, thereby improving the quality of auto-generated test suites of existing approaches. For rigorous evaluation, the authors introduce FixtureEval, a dedicated benchmark comprising 600 curated functions across two Programming Languages (PLs), i.e., Python and Java, with explicit fixture dependency labels, enabling both the corresponding classification and generation tasks. Empirical results demonstrate that Fixturize is highly effective, achieving 88.38%-97.00% accuracy across benchmarks in identifying the dependence of test fixtures and significantly enhancing the Suite Pass rate (SuitePS) by 18.03%-42.86% on average across both PLs with the auto-generated fixtures. Owing to the maintenance of test fixtures, Fixturize further improves line/branch coverage when integrated with existing testing tools of both LLM-based and Search-based by 16.85%/24.08% and 31.54%/119.66% on average, respectively. The findings establish fixture awareness as an essential, missing component in modern auto-testing pipelines.

cs.SE↗

arXiv2025

Scientific Objectives of the Xue-shan-mu-chang 15-meter Submillimeter Telescope

Submillimeter astronomy is poised to revolutionize our understanding of the Universe by revealing cosmic phenomena hidden from optical and near-infrared observations, particularly those associated with interstellar dust, molecular gas, and star formation. The Xue-shan-mu-chang 15-meter submillimeter telescope (XSMT-15m), to be constructed at a premier high-altitude site (4813 m) in Qinghai, China, marks a major milestone for Chinese astronomy, establishing the China mainland's first independently developed, world-class submillimeter facility. Equipped with state-of-the-art instruments, XSMT-15m will address a diverse range of frontier scientific questions spanning extragalactic astronomy, Galactic structure, time-domain astrophysics, and astrochemistry. In synergy with current and forthcoming observatories, XSMT-15m will illuminate the formation and evolution of galaxies, unravel the physical and chemical processes shaping the interstellar medium, and explore transient phenomena in the submillimeter regime. These capabilities will advance our understanding across extragalactic astronomy, Galactic ecology, astrochemistry, and time-domain astrophysics, inaugurating a new era for submillimeter research in China and the northern hemisphere.

astro-ph.GA↗

arXiv2025

Dynamic Incentive Strategies for Smart EV Charging Stations: An LLM-Driven User Digital Twin Approach

This paper presents an enhanced electric vehicle demand response system based on large language models, aimed at optimizing the application of vehicle-to-grid technology. By leveraging an large language models-driven multi-agent framework to construct user digital twins integrated with multidimensional user profile features, it enables deep simulation and precise prediction of users' charging and discharging decision-making patterns. Additionally, a data- and knowledge-driven dynamic incentive mechanism is proposed, combining a distributed optimization model under network constraints to optimize the grid-user interaction while ensuring both economic viability and security. Simulation results demonstrate that the approach significantly improves load peak-valley regulation and charging/discharging strategies. Experimental validation highlights the system's substantial advantages in load balancing, user satisfaction and grid stability, providing decision-makers with a scalable V2G management tool that promotes the sustainable, synergistic development of vehicle-grid integration.

eess.SY↗

arXiv2025

Inadequate turbulent support in low-metallicity molecular clouds

The dynamic properties of molecular clouds are set by the interplay of their self-gravity, turbulence, external pressure and magnetic fields. Extended surveys of Galactic molecular clouds typically find that their kinetic energy ($E_{\rm k}$) counterbalances their self-gravitational energy ($E_{\rm g}$), setting their virial parameter $α_{\rm vir}=2E_{\rm k}/|E_{\rm g}|\approx1$. However, past studies either have been biased by the use of optically-thick lines or have been limited within the solar neighborhood and the inner Galaxy (Galactocentric radius $R_{\rm gc}<R_{\rm gc,\odot} \approx 8$ kpc). Here we present sensitive mapping observations of optically thin $^{13}$CO lines towards molecular clouds in the low-metallicity Galactic outer disk ($R_{\rm gc}\sim9-24$ kpc). By combining archival data from the inner Galaxy and four nearby metal-poor dwarf galaxies, we reveal a systematic trend of $α_{\rm vir}$, which declines from supervirial dynamic states in metal-rich clouds to extremely subvirial dynamic states in metal-poor clouds. In these metal-poor environments, turbulence alone is insufficient to counterbalance the self-gravity of a cloud. A cloud-volumetric magnetic field may replace turbulence as the dominant cloud-supporting mechanism in low-metallicity conditions, for example, the outermost galactic disks, dwarf galaxies and galaxies in the early Universe, which would then inevitably impact the initial conditions for star formation in such environments.

astro-ph.GA↗

arXiv2024

A new measurement of the Galactic $^{12}$C/$^{13}$C gradient from sensitive HCO$^+$ absorption observations

We present a new constraint on the Galactic $^{12}$C/$^{13}$C gradient with sensitive HCO$^+$ absorption observations against strong continuum sources. The new measurements suffer less from beam dilution, optical depths, and chemical fractionation, allowing us to derive the isotopic ratios precisely. The measured $^{12}$C/$^{13}$C ratio in the Solar neighborhood (66$\pm$5) is consistent with those obtained from CH$^+$. Two measurements toward the Galactic Center are 42.2$\pm$1.7 and 37.5$\pm$6.5. Though the values are a factor of 2$\sim$3 higher than those derived from dense gas tracers (e.g., H$_2$CO, complex organic molecules) toward Sagittarius (Sgr) B2 regions, our results are consistent with the absorption measurements from c-C$_3$H$_2$ toward Sgr B2 ($\sim$40), and those from CH$^+$ toward Sgr A$^*$ and Sgr B2(N) ($>$30). We calculate a new Galactic $^{12}$C/$^{13}$C gradient of (6.4$\pm$1.9)$R_{\rm GC}$/kpc+(25.9$\pm$10.5), and find an increasing trend of $^{12}$C/$^{13}$C gradient obtained from high-density to low-density gas tracers, suggesting opacity effects and chemical fractionation may have a strong impact on the isotopic ratios observed at high-density regions.

astro-ph.GA↗

arXiv2024

Prompt-Consistency Image Generation (PCIG): A Unified Framework Integrating LLMs, Knowledge Graphs, and Controllable Diffusion Models

The rapid advancement of Text-to-Image(T2I) generative models has enabled the synthesis of high-quality images guided by textual descriptions. Despite this significant progress, these models are often susceptible in generating contents that contradict the input text, which poses a challenge to their reliability and practical deployment. To address this problem, we introduce a novel diffusion-based framework to significantly enhance the alignment of generated images with their corresponding descriptions, addressing the inconsistency between visual output and textual input. Our framework is built upon a comprehensive analysis of inconsistency phenomena, categorizing them based on their manifestation in the image. Leveraging a state-of-the-art large language module, we first extract objects and construct a knowledge graph to predict the locations of these objects in potentially generated images. We then integrate a state-of-the-art controllable image generation model with a visual text generation module to generate an image that is consistent with the original prompt, guided by the predicted object locations. Through extensive experiments on an advanced multimodal hallucination benchmark, we demonstrate the efficacy of our approach in accurately generating the images without the inconsistency with the original prompt. The code can be accessed via https://github.com/TruthAI-Lab/PCIG.

cs.CV↗

arXiv2024

Sora Detector: A Unified Hallucination Detection for Large Text-to-Video Models

The rapid advancement in text-to-video (T2V) generative models has enabled the synthesis of high-fidelity video content guided by textual descriptions. Despite this significant progress, these models are often susceptible to hallucination, generating contents that contradict the input text, which poses a challenge to their reliability and practical deployment. To address this critical issue, we introduce the SoraDetector, a novel unified framework designed to detect hallucinations across diverse large T2V models, including the cutting-edge Sora model. Our framework is built upon a comprehensive analysis of hallucination phenomena, categorizing them based on their manifestation in the video content. Leveraging the state-of-the-art keyframe extraction techniques and multimodal large language models, SoraDetector first evaluates the consistency between extracted video content summary and textual prompts, then constructs static and dynamic knowledge graphs (KGs) from frames to detect hallucination both in single frames and across frames. Sora Detector provides a robust and quantifiable measure of consistency, static and dynamic hallucination. In addition, we have developed the Sora Detector Agent to automate the hallucination detection process and generate a complete video quality report for each input video. Lastly, we present a novel meta-evaluation benchmark, T2VHaluBench, meticulously crafted to facilitate the evaluation of advancements in T2V hallucination detection. Through extensive experiments on videos generated by Sora and other large T2V models, we demonstrate the efficacy of our approach in accurately detecting hallucinations. The code and dataset can be accessed via GitHub.

cs.LG↗

arXiv2024

The CO-dark molecular gas in the cold HI arc

The CO-dark molecular gas (DMG), which refers to the molecular gas not traced by CO emission, is crucial for the evolution of the interstellar medium (ISM). While the gas properties of DMG have been widely explored in the Solar neighborhood, whether or not they are similar in the outer disk regions of the Milky Way is still not well understood. In this Letter, we confirm the existence of DMG toward a cold HI arc structure at 13 kpc away from the Galactic center with both OH emission and HI narrow self-absorption (HINSA). This is the first detection of HINSA in the outer disk region, in which the HINSA fraction ($N_{\rm HINSA}$/$N_{\rm H_2}$ = 0.022$\pm$0.011) is an order of magnitude higher than the average value observed in nearby evolved dark clouds, but is consistent with that of the early evolutionary stage of dark clouds. The inferred H$_2$ column density from both extinction and OH emission ($N_{\rm H_2} \approx 10^{20}$ cm$^{-2}$) is an order of magnitude higher than previously estimated. Although the ISM environmental parameters are expected to be different between the outer Galactic disk regions and the Solar neighborhood, we find that the visual extinction ($A_{\rm V}$ = 0.19$\pm$0.03 mag), H$_2$-gas density ($n_{\rm H_2} = 91\pm46$ cm$^{-3}$), and molecular fraction (58\%$\pm$28\%) of the DMG are rather similar to those of nearby diffuse molecular clouds. The existence of DMG associated with the expanding HI supershell supports a scenario where the expansion of supershells may trigger the formation of molecular clouds within a crossing timescale of the shock wave ($\sim$10$^6$ yr).

astro-ph.GA↗

arXiv2023

$α$-enhanced Astrochemistry: the Carbon cycle in extreme galactic conditions

Astrochemistry has been widely developed as a power tool to probe physical properties of the interstellar medium (ISM) in various conditions of the Milky Way (MW) Galaxy, and in near and distant galaxies. Most current studies conventionally apply linear scaling to all elemental abundances based on the gas-phase metallicity. However, these elements, including carbon and oxygen, are enriched differentially by stellar nucleosynthesis and the overall galactic chemical evolution, evident from $α$-enhancement in multiple galactic observations such as starbursts, high-redshift star-forming galaxies, and low-metallicity dwarfs. We perform astrochemical modeling to simulate the impact of an $α$-enhanced ISM gas cloud on the abundances of the three phases of carbon (C$^+$, C, CO) dubbed as `the carbon cycle'. The ISM environmental parameters considered include two cosmic-ray ionization rates ($ζ_{\rm CR}=10^{-17}$ and $10^{-15}\,{\rm s}^{-1}$), two isotropic FUV radiation field strengths ($χ/χ_0=1$ and $10^2$), and (sub-)linear dust-to-gas relations against metallicity, mimicking the ISM conditions of different galaxy types. In galaxies with [C/O] $<$ 0, CO, C and C$^+$ all decrease in both abundances and emission, though with differential biases. The low-$J$ CO emission is found to be the most stable tracer for the molecular gas, while C and C$^+$ trace H$_2$ gas only under limited conditions, in line with recent discoveries of [CI]-dark galaxies. We call for caution when using [CII]~$158μ$m and [CI](1-0) as alternative H$_2$-gas tracers for both diffuse and dense gas with non-zero [C/O] ratios.

astro-ph.GA↗