arXiv subjects

Liu Kang

Publications and source records attributed to Liu Kang.

3 recordsLinked to original sources

arXiv2026

Outcome-Grounded Advantage Reshaping for Fine-Grained Credit Assignment in Mathematical Reasoning

Group Relative Policy Optimization (GRPO) has emerged as a promising critic-free reinforcement learning paradigm for reasoning tasks. However, standard GRPO employs a coarse-grained credit assignment mechanism that propagates group-level rewards uniformly to to every token in a sequence, neglecting the varying contribution of individual reasoning steps. We address this limitation by introducing Outcome-grounded Advantage Reshaping (OAR), a fine-grained credit assignment mechanism that redistributes advantages based on how much each token influences the model's final answer. We instantiate OAR via two complementary strategies: (1) OAR-P, which estimates outcome sensitivity through counterfactual token perturbations, serving as a high-fidelity attribution signal; (2) OAR-G, which uses an input-gradient sensitivity proxy to approximate the influence signal with a single backward pass. These importance signals are integrated with a conservative Bi-Level advantage reshaping scheme that suppresses low-impact tokens and boosts pivotal ones while preserving the overall advantage mass. Empirical results on extensive mathematical reasoning benchmarks demonstrate that while OAR-P sets the performance upper bound, OAR-G achieves comparable gains with negligible computational overhead, both significantly outperforming a strong GRPO baseline, pushing the boundaries of critic-free LLM reasoning.

cs.CL↗

arXiv2026

IRPM: Intergroup Relative Preference Modeling for Pointwise Generative Reward Models

Generative Reward Models (GRMs) have demonstrated strong performance in reward modeling, due to their interpretability and potential for refinement through reinforcement learning (RL). However, widely used pairwise GRMs create a computational bottleneck in reinforcement learning from human feedback (RLHF), when calibrating or aggregating preference signals over n candidates, often incurring O(n^2) pairwise judgments. To address this issue, we propose Intergroup Relative Preference Modeling (IRPM), an RL-based method that extends the Bradley--Terry preference-learning paradigm via intergroup comparisons to train pointwise GRMs from pairwise preference data. IRPM derives pointwise reward for each response by contrasting groups of chosen vs. rejected samples, enabling pointwise scores comparable across candidate sets and O(n) reward evaluation for a variable number of candidates during RL training, while preserving interpretability and scalability. Experiments show that IRPM achieves state-of-the-art performance among pointwise GRMs on RM-Bench, JudgeBench and RewardBench, and approaches the performance of leading pairwise GRMs. In addition, IRPM achieves substantial gains in post-training evaluations, demonstrating its effectiveness.

cs.LG↗

arXiv2020

Detection and characterisation of oscillating red giants: first results from the TESS satellite

Since the onset of the `space revolution' of high-precision high-cadence photometry, asteroseismology has been demonstrated as a powerful tool for informing Galactic archaeology investigations. The launch of the NASA TESS mission has enabled seismic-based inferences to go full sky -- providing a clear advantage for large ensemble studies of the different Milky Way components. Here we demonstrate its potential for investigating the Galaxy by carrying out the first asteroseismic ensemble study of red giant stars observed by TESS. We use a sample of 25 stars for which we measure their global asteroseimic observables and estimate their fundamental stellar properties, such as radius, mass, and age. Significant improvements are seen in the uncertainties of our estimates when combining seismic observables from TESS with astrometric measurements from the Gaia mission compared to when the seismology and astrometry are applied separately. Specifically, when combined we show that stellar radii can be determined to a precision of a few percent, masses to 5-10% and ages to the 20% level. This is comparable to the precision typically obtained using end-of-mission Kepler data

astro-ph.SR↗