arXiv subjects

Xintong Li

Publications and source records attributed to Xintong Li.

At least 19 recordsLinked to original sources

arXiv2026

Aggregate-then-Calibrate for Human-centered Assessment with Theoretical Guarantees

Human-centered assessment tasks, which are essential for systematic decision-making, rely heavily on human judgment and typically lack verifiable ground truth. Existing approaches face a dilemma: methods using only human judgments suffer from heterogeneous expertise and inconsistent rating scales, while methods using only model-generated scores must learn from imperfect proxies or incomplete features. We propose Aggregate-then-Calibrate (AtC), a two-stage framework that combines these complementary sources. Stage-1 aggregates heterogeneous comparative judgments into a consensus ranking using a rank-aggregation model that accounts for annotator reliability. Stage-2 calibrates any predictive model's scores by an isotonic projection onto the order, enforcing ordinal consistency while preserving as much of the model's quantitative information as possible. Theoretically, we show: (1) modeling annotator heterogeneity yields strictly more efficient consensus estimation than homogeneity; (2) isotonic calibration enjoys risk bounds even when the consensus ranking is misspecified; and (3) AtC asymptotically outperforms model-only assessment. Across semi-synthetic and real-world datasets, AtC consistently improves accuracy and robustness over human-only or model-only assessments. Our results bridge judgment aggregation with model-free calibration, providing a principled recipe for human-centered assessment when ground truth is costly, scarce, or unverifiable.

cs.LG↗

arXiv2026

UniPolymer: A Unified Framework for Property Prediction, Structure Recommendation, and Evaluation in Polyimide Design

Designing polyimide structures with specific glass transition temperatures (Tg) is highly challenging. Existing methods primarily focus on target-conditioned generation, lacking an assessment of the consistency between the generated structure and the target properties. This leads to low-quality candidates deviating from the design objective entering subsequent processes, increasing invalid experiments and prolonging the development cycle. To address this issue, we propose UniPolymer, a unified framework for property prediction, target-conditioned generation, candidate evaluation, and structure recommendation in polyimide design and a dataset containing 10066 deduplicated polyimide repeating units with Tg tags (PITg-Curated) was constructed. To improve the consistency between generated candidate structures and the target Tg, UniPolymer first establishes a reliable structure-property relationship mapping through self-supervised chemical semantic learning, structural consistency enhancement, and multi-scale information fusion. Subsequently, the model employs a continuous-discrete joint Tg representation to guide the autoregressive generation of SELFIES. The generated candidate structures are further evaluated using a frozen property predictor and polyimide-specific structural constraints, and ranked according to their deviation from the target Tg, thereby preventing structures deviating from the target from entering the subsequent validation stage. Experimental results show that UniPolymer achieved a property prediction accuracy of R^2=0.93 and a candidate structure evaluation pass rate of 73.79%, which are 2% and 1.21% higher than the best baseline, respectively. Meanwhile, the predicted Tg values of the recommended candidates are in high agreement with the results of molecular dynamics simulations, thereby reducing the number of candidates that enter the high-cost experimental stage.

cs.LG↗

arXiv2026

RRPO: Reference-Relative Policy Optimization with Stratified Conditional Rollouts

Group Relative Policy Optimization (GRPO) has shown strong effectiveness in reinforcement learning from verifiable feedback, where sampled rollouts can be compared within a group using task-provided correctness signals. However, extending group-relative optimization beyond verifiable settings is challenging because success in many tasks is not captured by a single correctness criterion. We propose \textbf{Reference-Relative Policy Optimization (RRPO)}, which generalizes GRPO by replacing direct correctness-based advantage construction with reference-relative contrastive comparisons. RRPO first uses \emph{stratified conditional rollouts} to construct positive and negative anchor sets, and then trains a metric projection head with a set-contrastive objective to compare candidate rollouts against these anchors. The resulting alignment scores directly define contrastive advantages: during policy optimization, the projection head is frozen, and the scores are centered within each rollout group in a standard group-relative objective. We evaluate RRPO using anchor-based contrastive advantages throughout policy optimization, without relying on task ground-truth verifiers. Across verifiable reasoning, open-ended generation, and post-SFT settings, RRPO remains competitive with verifier-based optimization, improves over weakly supervised baselines, and provides additional gains after supervised fine-tuning.

cs.LG↗

arXiv2026

Process Reward Informed Tree Rollout for Effective Multi-Turn RL

Reinforcement learning (RL) has become a key approach for training LLM agents, yet popular methods such as GRPO/RLOO rely on multiple independently sampled complete trajectories for advantage estimation. In long-horizon agentic tasks, such a uniform rollout strategy can waste budget on uninformative dead-end attempts, while promising intermediate states do not receive sufficient exploration. The multi-turn structure of agentic trajectories, with interleaved actions and observations, naturally supports organizing a trajectory group as a tree, where each turn serves as a decision point for exploration. This perspective reframes effective exploration as the problem of deciding where to branch. We propose Process-Scorer Guided Adaptive Tree Rollout (PATR), a quality-aware rollout framework for multi-turn agent RL. PATR uses task-appropriate process feedback to score partial trajectories, selectively branches from promising states, reuses shared prefixes, and conservatively stops degenerate paths to reduce wasted sampling. The resulting rollout groups remain compatible with standard policy optimization while providing more efficient exploration under the same training budget. We evaluate PATR on FrozenLake and the challenging SWE-Bench, which is largely unexplored by prior tree-rollout agent RL methods. Experiments show that PATR improves performance by up to +5.0 points on SWE-Bench and +9.3 points on FrozenLake, highlighting process-guided tree rollouts as an effective strategy for scalable multi-turn RL.

cs.CL↗

arXiv2026

Coexistence of High Temperature Superconductivity and Antiferromagnetic Order in a Cuprate with Multiple Hole Fermi Pockets

The intricate relationship between high temperature superconductivity and antiferromagnetic order in cuprates, and the fundamental origin of electron pairing remain open questions. By utilizing high-resolution laser-based spatially-resolved angle-resolved photoemission spectroscopy, we investigate the seven-layer $Bi_{2}Sr_{2}Ca_{6}Cu_{7}O_{18+\delta}$ (Bi2267) and identify a cuprate system that consists of multiple hole Fermi pockets. The observed Fermi pockets exhibit pronounced momentum-, temperature- and Fermi surface-dependent energy gaps. Crucially, high temperature superconductivity with a critical temperature ($T_{\mathrm{c}}$) of $\sim$75 K emerges in a system with multiple Fermi pockets and the presence of strong antiferromagnetic order and correlations. In particular, substantial electron pairing is observed along the Fermi pocket with an energy gap up to $\sim$42 meV in lightly-doped CuO$_{2}$ planes ($p\sim$0.05). These findings challenge the conventional understanding of the roles of the nodal and antinodal electronic states in driving high-temperature superconductivity. They show that superconductivity and antiferromagnetism can coexist in a cuprate with multiple Fermi pockets, offering further insights into the pairing mechanism in cuprate superconductors.

cond-mat.supr-con↗

arXiv2026

Suppression of p-Wave Altermagnetism by Localized 4f Electrons in CeNiAsO

Altermagnetism, characterized by momentum-dependent spin splitting and zero net magnetization, has so far been explored mainly in weakly or moderately correlated d-electron systems. How symmetry-allowed altermagnetic band splitting manifests in heavy-fermion materials, where magnetic exchange competes with Kondo correlations, remains unclear. Here we use high-resolution angle-resolved photoemission spectroscopy (ARPES) to investigate CeNiAsO, a Kondo-lattice system that was predicted to be a candidate for p-wave altermagnetism. Fermi surface mapping and polarization-dependent ARPES show that the experimentally observed itinerant bands are mainly derived from Ni 3d orbitals, while resonant photoemission reveals that the Ce 4f states remain predominantly localized with residual c-f hybridization. Ultra-low-temperature measurements reveal no resolvable near-Fermi-level p-wave-like exchange splitting on the Ni 3d-derived conduction bands across the successive antiferromagnetic transitions. These experimental observations cannot be captured by an itinerant-4f band-structure description, which predicts a sizable p-wave splitting in the itinerant bands. When the localized Ce 4f character is incorporated, our band structure calculations indicate that the itinerant Ce 4f band weight is shifted away from the Fermi level and the p-wave-like splitting on the Ni 3d-derived bands is reduced to the few-meV scale. These results establish CeNiAsO as a strongly correlated f-electron setting in which the magnetic symmetry allows p-wave-like band splitting, but localized 4f electrons strongly suppress its observable itinerant single-particle signature.

cond-mat.str-el↗

arXiv2026

F-GRPO: Factorized Group-Relative Policy Optimization for Unified Candidate Generation and Ranking

Traditional retrieval pipelines optimize utility through stages of candidate retrieval and reranking, where ranking operates over a predefined candidate set. Large Language Models (LLMs) broaden this into a generative process: given a candidate pool, an LLM can generate a subset and order it within a single autoregressive pass. However, this flexibility introduces a new optimization challenge: the model must search a combinatorial output space while receiving utility feedback only after the full ranked list is generated. Because this feedback is defined over the completed sequence, it cannot distinguish whether a poor result arises from failing to generate a relevant subset or from failing to rank that subset correctly. This credit assignment gap makes end-to-end optimization unstable and sample-inefficient. Existing systems often address this by separating candidate generation from ranking. However, such decoupling remains misaligned with downstream utility because ranking is limited by the candidate set it receives. To bridge this gap, we propose a unified framework that performs both within a single autoregressive rollout and optimizes them end-to-end via factorized group-relative policy optimization (F-GRPO). Our framework factorizes the policy into candidate generation and ranking while sharing a single LLM backbone, and jointly trains them with an order-invariant coverage reward and a position-aware utility reward. To address the resulting phase-specific credit assignment problem, we use separate group-relative advantages for generation and ranking within a two-phase sequence-level objective. Across sequential recommendation and multi-hop question answering benchmarks, F-GRPO improves top-ranked performance over GRPO and decoupled baselines, outperforms supervised alternatives, and remains competitive with strong zero-shot rerankers, with no architectural changes at inference time.

cs.LG↗

arXiv2026

Learning with Rare Success but Rich Feedback via Reflection-Enhanced Self-Distillation

Enabling Large Language Models (LLMs) to continuously improve from environmental interactions is a central challenge in post-training. While on-policy self-distillation offers a promising paradigm, existing methods predominantly treat environmental feedback as a passive conditioning signal. Consequently, they heavily rely on successful demonstrations and struggle to learn in rare-success regimes. To bridge this gap, we introduce Reflection-Enhanced Self-Distillation (RESD), a framework that transforms raw failure feedback into an active source of corrective supervision. Instead of passively appending feedback, RESD interprets failed trajectories by generating retrospective reflections to diagnose local errors, and curates a persistent global playbook to preserve reusable lessons across training steps. The enriched context enables the self-teacher to provide actionable token-level supervision even in the absence of successful rollouts. Empirical evaluations on multiple continual learning tasks demonstrate that RESD substantially outperforms standard self-distillation baselines. Furthermore, RESD achieves significantly faster early-stage improvement than GRPO with $8\times$ samples using only a single rollout per prompt, highlighting its superior interaction efficiency.

cs.LG↗

arXiv2026

MASS-DPO: Multi-negative Active Sample Selection for Direct Policy Optimization

Multi-negative preference optimization under the Plackett--Luce (PL) model extends Direct Preference Optimization (DPO) by leveraging comparative signals across one preferred and multiple rejected responses. However, optimizing over large negative pools is costly, and many candidates contribute redundant gradients due to their similar effects on policy updates. We introduce MASS-DPO, a multi-negative active sample selection method that derives a PL-specific Fisher-information objective for selecting compact, informative negative subsets within each prompt. The resulting log-determinant objective selects negatives that contribute complementary information for policy updates, yielding compact subsets that retain the full pool's information while reducing redundancy. In practice, this favors negatives whose gradients cover different update directions, reducing redundant signal from near-duplicate candidates while preserving the most useful training information. Across four benchmarks spanning recommendation and multiple-choice QA and three model families, MASS-DPO consistently exceeds or matches existing methods in accuracy, improves Recall/NDCG and margin-based optimization dynamics, and delivers stronger alignment with substantially fewer negatives.

cs.LG↗

arXiv2026

OLIVIA: Online Learning via Inference-time Action Adaptation for Decision Making in LLM ReAct Agents

Large language model agents interleave reasoning, action selection, and observation to solve sequential decision-making tasks. In deployed settings where agents repeatedly handle related multi-step tasks, small action-selection errors can accumulate into wasted tool calls, latency, and reduced reliability. Despite this need for deployment-time improvement, existing inference-time adaptation methods for LLM agents mainly rely on prompting or retrieval, which influence behavior indirectly through context manipulation. For ReAct-style agents, such approaches do not expose an explicit decision layer that can score candidate actions, represent uncertainty, or be updated online from action-level feedback. As a result, they provide limited support for trackable, fine-grained, and uncertainty-aware adaptation during deployment. We propose OLIVIA, an inference-time action adaptation framework for ReAct-style agents. OLIVIA models the LLM's final action-selection layer as a contextual linear bandit over candidate actions, with frozen hidden states as decision contexts. This choice is particularly suitable for deployment because it adapts behavior directly at the action-selection interface, preserves the underlying reasoning process, and provides explicit uncertainty estimates and lightweight online updates from action-level feedback. With upper-confidence-bound exploration, OLIVIA improves the policy sample-efficiently with minimal computational overhead. We instantiate OLIVIA on four benchmarks and show that it consistently improves task performance over static ReAct and prompt-based inference-time baselines. Our results suggest that explicit online decision layers provide an effective alternative to purely prompt- or retrieval-based adaptation for LLM agents during deployment.

cs.AI↗

arXiv2026

Skill-R1: Agent Skill Evolution via Reinforcement Learning

Agentic large language models often rely on skills, reusable natural language procedures that guide planning, action, and tool use. In practice, skills are typically improved through prompt engineering or by aligning the task LLM itself, which is costly, model-specific, and often infeasible for closed-source models. Skill optimization is not a one-step problem but a recurrent process with two coupled levels of credit assignment: a useful skill must improve rollout quality under current conditioning, while a useful revision must turn observed outcomes into a better skill for the next round. We propose Skill-R1, a reinforcement learning framework for instance-level recurrent skill optimization from verifiable rewards. Rather than updating the task LLM, Skill-R1 trains a lightweight skill generator that conditions on the task context, prior rollouts, and their verified outcomes to produce skills that steer a frozen task LLM. This preserves black-box compatibility with both open- and closed-source models while making adaptation substantially cheaper than model-level updates. Skill-R1 proceeds over multiple generations: at each step, the current skill induces rollouts whose verified outcomes are fed back to produce the next revision. To optimize this recurrent process, we introduce a bi-level group-relative policy optimization objective combining intra-generation and inter-generation advantages. The intra-generation term compares rollouts under shared skill conditioning, while the inter-generation term rewards revisions that improve behavior across successive generations. Together, these provide a principled objective for directional skill evolution rather than one-shot self-refinement. Empirically, Skill-R1 achieves consistent gains over no-skill baselines and standard GRPO across benchmarks with verifiable rewards, with particularly strong improvements on complex, multi-step tasks.

cs.LG↗

arXiv2026

Persistent Fermi Pockets and Robust Electron Pairing in Lightly Doped CuO$_2$ Planes of Cuprate Superconductors

High temperature superconductivity in cuprate superconductors is generally considered to be generated from doping the Mott insulators. The fundamental nature of the doped parent compounds as well as the microscopic origin of electron pairing remain critical issues in understanding the emergence of superconductivity. Here, using high-resolution spatially-resolved laser angle-resolved photoemission spectroscopy, we investigate the intrinsic electronic structures of the CuO$_2$ planes in multilayer cuprates Bi$_2$Sr$_2$Ca$_{n-1}$Cu$_n$O$_{2n+4+\delta}$ (n=5$\sim$8). The inner CuO$_2$ planes are well shielded from the disorders and provide a rare and ideal platform to probe the intrinsic electronic phase diagram. We observe well-defined Fermi pockets with hole doping levels as low as 0.007, demonstrating an abrupt transition from the parent Mott insulator to a metallic state upon the introduction of an infinitesimal amount of doping. The innermost CuO$_2$ planes (IP$_0$) display gapless Fermi pockets, while the second innermost planes (IP$_1$) exhibit anisotropic superconducting gaps up to $\sim$33$\,$meV, indicative of robust electron pairing coexisting with strong antiferromagnetic order. Our findings provide a revised framework for understanding the doping-driven transitions and pairing mechanisms in cuprate superconductors.

cond-mat.supr-con↗

arXiv2026

Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learning

Reinforcement learning (RL) has become a central post-training tool for improving the reasoning abilities of large language models (LLMs). In these systems, the rollout, the trajectory sampled from a prompt to termination, including intermediate reasoning steps and optional tool or environment interactions, determines the data the optimizer learns from, yet rollout design is often underreported. This survey provides an optimizer-agnostic view of rollout strategies for RL-based post-training of reasoning LLMs. We formalize rollout pipelines with unified notation and introduce Generate-Filter-Control-Replay (GFCR), a lifecycle taxonomy that decomposes rollout pipelines into four modular stages: Generate proposes candidate trajectories and topologies; Filter constructs intermediate signals via verifiers, judges, critics; Control allocates compute and makes continuation/branching/stopping decisions under budgets; and Replay retains and reuses artifacts across rollouts without weight updates, including self-evolving curricula that autonomously generate new training tasks. We complement GFCR with a criterion taxonomy of reliability, coverage, and cost sensitivity that characterizes rollout trade-offs. Using this framework, we synthesize methods spanning RL with verifiable rewards, process supervision, judge-based gating, guided and tree/segment rollouts, adaptive compute allocation, early-exit and partial rollouts, throughput optimization, and replay/recomposition for self-improvement. We ground the framework with case studies in math, code/SQL, multimodal reasoning, tool-using agents, and agentic skill benchmarks that evaluate skill induction, reuse, and cross-task transfer. Finally, we provide a diagnostic index that maps common rollout pathologies to GFCR modules and mitigation levers, alongside open challenges for building reproducible, compute-efficient, and trustworthy rollout pipelines.

cs.LG↗

arXiv2026

Mott-Derived Local Moments and Kondo Hybridization in a d-electron Kagome lattice

Unlike canonical Kondo lattices in f-electron systems, where localized f orbitalsnaturally provide local moments, d-electron Kondo lattices require a distinct mechanism for local-moment formation. However, the study of d-electron Kondo lattices in bulk materials remains far from settled, particularly with regard to the microscopic origin of the local moments. Here, we report a microscopic mechanism for this process in the bilayer kagome metal CsCr6Sb6, where strong correlations drive a Mott splitting of the kagome flat band to supply the requisite local moments. By combining STM/STS and ARPES, we resolve a spectroscopic hierarchy between high-energy correlation effects and low temperature hybridization. Low-temperature STS reveals a robust asymmetric suppression of the density of states near EF that is well captured phenomenologically by a Fano-type lineshape, while ARPES detects a sharp quasiparticlepeak near EF. These low-energy signatures evolveon the same temperature scale and disappear upon warming, consistent with the onset of Kondo hybridization. At the same time, STS resolves symmetric humps at approximately +-50 mV and ARPES identifies a weakly dispersive feature around 50 meV below EF; unlike the near-EF hybridization signatures, these features persist to substantially higher temperatures. This separation of energy and temperature scales supports a two-stage picture in which a kagome flat band first undergoes correlation-driven splitting into lower and upper Hubbard bands, and the occupied lower Hubbard band supplies the local moments that later hybridize with itinerant electrons at lower temperature. Our results therefore move beyond the phenomenology of a kagome Kondo lattice candidate and instead provide a microscopic spectroscopic picture linking Mottness to Kondo hybridization in a frustrated d-electron system.

cond-mat.str-el↗

arXiv2026

Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoning

Large reasoning models improve with more test-time computation, but often overthink, producing unnecessarily long chains-of-thought that raise cost without improving accuracy. Prior reinforcement learning approaches typically rely on a single outcome reward with trajectory-level length penalties, which cannot distinguish essential from redundant reasoning steps and therefore yield blunt compression. Although recent work incorporates step-level signals, such as offline pruning, supervised data construction, or verifier-based intermediate rewards, reasoning length is rarely treated as an explicit step-level optimization objective during RL. We propose Step-wise Adaptive Penalization (SWAP), a fine-grained framework that allocates length reduction across steps based on intrinsic contribution. We estimate step importance from the model's on-policy log-probability improvement toward the correct answer, then treat excess length as a penalty mass redistributed to penalize low-importance steps more heavily while preserving high-importance reasoning. We optimize with a unified outcome-process advantage within group-relative policy optimization. Extensive experiments demonstrate that SWAP reduces reasoning length by 64.3% on average while improving accuracy by 5.7% relative to the base model.

cs.CL↗

arXiv2026

WS-GRPO: Weakly-Supervised Group-Relative Policy Optimization for Rollout-Efficient Reasoning

Group Relative Policy Optimization (GRPO) is effective for training language models on complex reasoning. However, since the objective is defined relative to a group of sampled trajectories, extended deliberation can create more chances to realize relative gains, leading to inefficient reasoning and overthinking, and complicating the trade-off between correctness and rollout efficiency. Controlling this behavior is difficult in practice, considering (i) Length penalties are hard to calibrate because longer rollouts may reflect harder problems that require longer reasoning, penalizing tokens risks truncating useful reasoning along with redundant continuation; and (ii) supervision that directly indicates when to continue or stop is typically unavailable beyond final answer correctness. We propose Weakly Supervised GRPO (WS-GRPO), which improves rollout efficiency by converting terminal rewards into correctness-aware guidance over partial trajectories. Unlike global length penalties that are hard to calibrate, WS-GRPO trains a preference model from outcome-only correctness to produce prefix-level signals that indicate when additional continuation is beneficial. Thus, WS-GRPO supplies outcome-derived continue/stop guidance, reducing redundant deliberation while maintaining accuracy. We provide theoretical results and empirically show on reasoning benchmarks that WS-GRPO substantially reduces rollout length while remaining competitive with GRPO baselines.

cs.LG↗

arXiv2026

AMPS: Adaptive Modality Preference Steering via Functional Entropy

Multimodal Large Language Models (MLLMs) often exhibit significant modality preference, which is a tendency to favor one modality over another. Depending on the input, they may over-rely on linguistic priors relative to visual evidence, or conversely over-attend to visually salient but facts in textual contexts. Prior work has applied a uniform steering intensity to adjust the modality preference of MLLMs. However, strong steering can impair standard inference and increase error rates, whereas weak steering is often ineffective. In addition, because steering sensitivity varies substantially across multimodal instances, a single global strength is difficult to calibrate. To address this limitation with minimal disruption to inference, we introduce an instance-aware diagnostic metric that quantifies each modality's information contribution and reveals sample-specific susceptibility to steering. Building on these insights, we propose a scaling strategy that reduces steering for sensitive samples and a learnable module that infers scaling patterns, enabling instance-aware control of modality preference. Experimental results show that our instance-aware steering outperforms conventional steering in modulating modality preference, achieving effective adjustment while keeping generation error rates low.

cs.LG↗

arXiv2026

Electronic Structure and Superconducting Gap of HgBa$_2$Ca$_2$Cu$_3$O$_{8+\delta}$ Revealed by Laser-Based Angle-Resolved Photoemission Spectroscopy

The spatially-resolved laser-based high resolution angle resolved photoemission spectroscopy (ARPES) measurements have been performed on the optimally-doped HgBa$_2$Ca$_2$Cu$_3$O$_{8+\delta}$ (Hg1223) superconductor with a $T_c$ at 133 K. Two distinct regions are identified on the cleaved surface: the single Fermi surface region where only one Fermi surface is observed, and the double Fermi surface region where two Fermi surface sheets are resolved coming from both the inner (IP) and outer (OP) CuO$_2$ planes. The electronic structure and superconducting gap are measured on both of these two regions. In both cases, the observed electronic states are mainly concentrated near the nodal region. The momentum dependence of superconducting gap deviates from the standard d-wave form. These results indicate that the surface electronic structure of Hg1223 behaves more like that of underdoped cuprates.

cond-mat.supr-con↗