arXiv subjects

Haocheng Yang

Publications and source records attributed to Haocheng Yang.

16 recordsLinked to original sources

arXiv2026

Rubrics on Trial: Evolving Rubrics from a Single Query via Synthetic Pairwise Evidence

Rubrics provide structured, fine-grained signals for training and evaluating large language models (LLMs). Yet reliable query-specific rubrics are difficult to construct. Existing approaches often derive supervision from human-written rubrics, preference data, or sampled responses. Direct query-to-rubric generation avoids these resources, but provides no explicit check that a plausible rubric is useful. Such a rubric may fail to distinguish answer quality, reward an optional style, or penalize a valid alternative strategy. We introduce Rubrics on Trial, a query-only framework that evolves a rubric set from an empty set without external annotations or model training. It derives supervision solely from synthetic rubric-conditioned response pairs and validates each proposed rubric before adding it, screening out non-discriminative, over-specific, and style-only candidate rubrics. Experiments across five preference benchmark suites demonstrate the effectiveness of Rubrics on Trial, which achieves the best average accuracy and leads on six of seven evaluation sets.

cs.CL↗

arXiv2026

Uncertainty-Aware Reward Modeling for Stable RLHF

Reinforcement learning from human feedback (RLHF) aligns large language models by training reward models on preference data and optimizing policies to maximize predicted rewards. However, this pipeline faces two fundamental challenges: (1) reward models cannot signal when their predictions are unreliable, since they usually act as deterministic point estimators; and (2) modern group-based policy optimization can amplify unreliable reward signals, as exemplified by GRPO's uniform treatment of rewards during advantage computation. As policies explore increasingly diverse responses, these two limitations create a critical vulnerability: unreliable reward estimates may be granted disproportionate influence, triggering severe reward hacking. We propose Uncertainty-Aware Reward Modeling (UARM), which equips reward models with calibrated uncertainty via quantile-based conformal prediction and reweights GRPO advantages through heteroscedastic variance decomposition. Experiments across HelpSteer, UltraFeedback, and PKU-SafeRLHF demonstrate that UARM significantly improves reward model calibration, reduces reward hacking, and enhances downstream alignment quality compared to standard GRPO and uncertainty-agnostic baselines.

cs.LG↗

arXiv2026

Distributed TD Tracking with Linear Function Approximation over Directed Communication Networks

We study the policy evaluation problem in multi-agent reinforcement learning (MARL) over directed communication networks, where agents cooperate with each other to explore an unknown environment and accomplish a specific task. We propose a Push-Pull-type distributed algorithm, named PP-DTD, for policy evaluation in MARL within the framework of temporal difference (TD) learning with linear function approximation. PP-DTD integrates TD learning with the Push-Pull mechanism to accommodate directed communication networks, and further utilizes variance reduction techniques to enhance both algorithmic stability and convergence rate. We show that PP-DTD achieves linear convergence to a neighborhood of the optimum under constant step-sizes and a convergence rate of $\mathcal{O}({T^{-1}})$ under decaying step-sizes when the sample is independent and identically distributed or Markovian. To the best of our knowledge, PP-DTD is the first distributed algorithm for policy evaluation in MARL over directed graphs that achieves a comparable convergence rate to single-agent TD. The numerical experiments on cooperative navigation tasks demonstrate the robustness and effectiveness of PP-DTD.

math.OC↗

arXiv2026

ImplicitRM: Unbiased Reward Modeling from Implicit Preference Data for LLM alignment

Reward modeling represents a long-standing challenge in reinforcement learning from human feedback (RLHF) for aligning language models. Current reward modeling is heavily contingent upon experimental feedback data with high collection costs. In this work, we study \textit{implicit reward modeling} -- learning reward models from implicit human feedback (e.g., clicks and copies) -- as a cost-effective alternative. We identify two fundamental challenges in implicit reward modeling: (1) Implicit preference data lacks definitive negative samples, which makes standard positive-negative classification methods inapplicable; (2) Implicit preference data suffers from user preference bias, where different responses have different propensities to elicit user feedback actions, which exacerbates the difficulty of distinguishing definitive negative samples. To address these challenges, we propose ImplicitRM, which aims to learn unbiased reward models from implicit preference data. ImplicitRM stratifies training samples into four latent groups via a stratification model. Building on this, it derives a learning objective through likelihood maximization, which we prove is theoretically unbiased, effectively resolving both challenges. Experiments demonstrate that ImplicitRM learns accurate reward models across implicit preference datasets. Code is available on our project website.

cs.CL↗

arXiv2026

Local Invariant Structures in the Dynamics of Capillary Water Jet

Physical experiments show that a capillary water jet is exponentially unstable under long wave perturbations, while remaining stable under short wave perturbations. Measurements indicate that the exponential growth rate in the long wave regime agrees quantitatively with the classical predictions of Rayleigh and Plateau, known as \emph{Rayleigh-Plateau instability}. In this paper, we provide a mathematical justification of these experimental observations. The motion of the water jet is modeled by an axisymmetric irrotational Eulerian free-boundary system governed by surface tension. We prove that the (un)stable directions in the linearized system, corresponding to long wave perturbations, are indeed tangent to a (un)stable invariant manifold of the full nonlinear system. On the other hand, the elliptic directions, corresponding to short wave perturbations, are indeed tangent to a center invariant set in a generalized sense. These results give a positive answer to the question raised by Lin-Zeng concerning the existence of invariant manifolds for Eulerian free-boundary systems. A notable finding is the existence of infinite dimensional hyperbolic invariant manifolds in the dynamics of infinitely long capillary water jet, in absence of spectral gap. The major methodological contribution is the construction of ``paradifferential propagator" corresponding to linear paradifferential hyperbolic systems, effectively balancing the loss of regularity due to quasilinearity. The method can be generalized to a broad class of quasilinear problems.

math.AP↗

arXiv2026

An Efficient Energy Stable Structure Preserving Method for The Landau-Lifshitz Equation

One of the main difficulties in micromagnetics simulation is the norm preserving constraints $\|\mathbf{m}\|=1$ at the continuous or the discrete level. Another difficulty is the stability with the time step constraint. Using standard explicit integrators leads to a physical time step of sub-pico seconds, which is often two orders of magnitude smaller than the fastest physical time scales. Direct implicit integrators require solving complicated, coupled systems. Another major difficulty with the projection method in this field is the lack of rigorous theoretical guarantees regarding its stability of the projection step. In this paper, we introduce a first order method. Such a method is structure preserving based on a combination of a Gauss-Seidel iteration, a double diffusion iteration and a Crank-Nicolson iteration to preserve the norm constraints.

math.NA↗

arXiv2026

Deep Time-series Forecasting Needs Kernelized Moment Balancing

Deep time-series forecasting can be formulated as a distribution balancing problem aimed at aligning the distribution of the forecasts and ground truths. According to Imbens' criterion, true distribution balance requires matching the first moments with respect to any balancing function. We demonstrate that existing objectives fail to meet this criterion, as they enforce moment matching only for one or two predefined balancing functions, thus failing to achieve full distribution balance. To address this limitation, we propose direct forecasting with kernelized moment balancing (KMB-DF). Unlike existing objectives, KMB-DF adaptively selects the most informative balancing functions from a reproducing kernel hilbert space (RKHS) to enforce sufficient distribution balancing. We derive a tractable and differentiable objective that enables efficient estimation from empirical samples and seamless integration into gradient-based training pipelines. Extensive experiments across multiple models and datasets show that KMB-DF consistently improves forecasting accuracy and achieves state-of-the-art performance. Code is available at https://anonymous.4open.science/r/KMB-DF-403C.

cs.LG↗

arXiv2025

ToolForge: A Data Synthesis Pipeline for Multi-Hop Search without Real-World APIs

Training LLMs to invoke tools and leverage retrieved information necessitates high-quality, diverse data. However, existing pipelines for synthetic data generation often rely on tens of thousands of real API calls to enhance generalization, incurring prohibitive costs while lacking multi-hop reasoning and self-reflection. To address these limitations, we introduce ToolForge, an automated synthesis framework that achieves strong real-world tool-calling performance by constructing only a small number of virtual tools, eliminating the need for real API calls. ToolForge leverages a (question, golden context, answer) triple to synthesize large-scale tool-learning data specifically designed for multi-hop search scenarios, further enriching the generated data through multi-hop reasoning and self-reflection mechanisms. To ensure data fidelity, we employ a Multi-Layer Validation Framework that integrates both rule-based and model-based assessments. Empirical results show that a model with only 8B parameters, when trained on our synthesized data, outperforms GPT-4o on multiple benchmarks. Our code and dataset are publicly available at https://github.com/Buycar-arb/ToolForge .

cs.AI↗

arXiv2025

Adaptive Selection of Symbolic Languages for Improving LLM Logical Reasoning

Large Language Models (LLMs) still struggle with complex logical reasoning. While previous works achieve remarkable improvements, their performance is highly dependent on the correctness of translating natural language (NL) problems into a symbolic language (SL). Though numerous works focusing on improving this translation accuracy, they only consider the similarity between the meaning of SL and NL, overlooking another crucial influencing factor, the selection of the target SL type itself. For example, first-order logic language specializes in logical reasoning with categorical syllogisms and complex quantifiers, while Boolean satisfiability formalism excels at representing constraint satisfaction like partial problems. To our knowledge, this is the first paper to claim and verify that different NL logical reasoning problem corresponds to different optimal SL formalization for translation. Based on this, we propose a methods to improve the logical reasoning performance of LLMs by adaptively selecting the most suitable SL for each problem prior to translation. Specifically, we leverage LLMs to select the target SL among first-order logic, logic programming and Boolean satisfiability and then translate the problem in NL to target SL expressions as well as employ the corresponding logical solver to derive the final answer. Experimental results on benchmarks show that our adaptive selection method significantly outperforms translating all into single SL and randomly selecting the SL. On a mixed dataset of these benchmarks, our approach achieves 96% accuracy, which improving performance by 25% compared to the second highest accuracy from the first-order logic translation.

cs.AI↗

arXiv2025

Global well-posedness of a 2D fluid-structure interaction problem with free surface

This paper is devoted to the analysis of the incompressible Euler equation in a time-dependent fluid domain, whose interface evolution is governed by the law of linear elasticity. Our main result asserts that the Cauchy problem is globally well-posed in time for any irrotational initial data in the energy space, without any smallness assumption. We also prove continuity with respect to the initial data and the propagation of regularity. The main novelty is that no dissipative effect is assumed in the system. In the absence of parabolic regularization, the key observation is that the system can be transformed into a nonlinear Schr\"odinger-type equation, to which dispersive estimates are applied. This allows us to construct solutions that are very rough from the point of view of fluid dynamics-the initial fluid velocity has merely one-half derivative in $L^2$. The main difficulty is that the problem is critical in the energy space with respect to several key inequalities from harmonic analysis. The proof incorporates new estimates for the Dirichlet-to-Neumann operator in the low-regularity regime, including refinements of paralinearization formulas and shape derivative formulas, which played a key role in the analysis of water waves.

math.AP↗

arXiv2024

Cauchy Problem for Cylinder-like Capillary Jets

The motion of liquid jets plays an important role in physics and engineering, and needs rigorous mathematical investigations. Recently, Huang-Karakhanyan proved the first local well-posedness in Sobolev spaces for axisymmetric jets. In this paper, we will extend this result to general jets, namely without any axisymmetry condition.

math.AP↗

arXiv2024

Virial Theorem and Its Applications in Instability of Two-Phase Water-Wave

In this paper, we analyze the dynamics of two layers of immiscible, inviscid, incompressible, and irrotational fluids through a full nonlinear system. Our goal is to establish a virial theorem and prove the polynomial growth of slope and curvature of the interface over time when the fluid below is no denser than the one above. These phenomena, known as Rayleigh-Taylor instability, will be proved for a broad class of regular initial data, including the case of 2D overlapping interface.

math.AP↗

arXiv2023

Microlocal Partition of Energy for Fractional-Type Dispersive Equations

This paper is devoted to the proof of microlocal partition of energy for fractional-type dispersive equations including Schr\"odinger equation, linearized gravity or capillary water-wave equation and half-Klein-Gordon equation. Roughly speaking, a quarter of the $L^2$ energy lies inside or outside the "light cone" $|x| = |tP'(\xi)|$ for large time. In addition, based on the study of half-Klein-Gordon equation, the microlocal partition of energy will also be proved for Klein-Gordon equation.

math.AP↗

arXiv2023

Dynamically Mitigating Data Discrepancy with Balanced Focal Loss for Replay Attack Detection

It becomes urgent to design effective anti-spoofing algorithms for vulnerable automatic speaker verification systems due to the advancement of high-quality playback devices. Current studies mainly treat anti-spoofing as a binary classification problem between bonafide and spoofed utterances, while lack of indistinguishable samples makes it difficult to train a robust spoofing detector. In this paper, we argue that for anti-spoofing, it needs more attention for indistinguishable samples over easily-classified ones in the modeling process, to make correct discrimination a top priority. Therefore, to mitigate the data discrepancy between training and inference, we propose D3M, to leverage a balanced focal loss function as the training objective to dynamically scale the loss based on the traits of the sample itself. Besides, in the experiments, we select three kinds of features that contain both magnitude-based and phase-based information to form complementary and informative features. Experimental results on the ASVspoof2019 dataset demonstrate the superiority of the proposed methods by comparison between our systems and top-performing ones. Systems trained with the balanced focal loss perform significantly better than conventional cross-entropy loss. With complementary features, our fusion system with only three kinds of features outperforms other systems containing five or more complex single models by 22.5% for min-tDCF and 7% for EER, achieving a min-tDCF and an EER of 0.0124 and 0.55% respectively. Furthermore, we present and discuss the evaluation results on real replay data apart from the simulated ASVspoof2019 data, indicating that research for anti-spoofing still has a long way to go. Source code, analysis data, and other details are publicly available at https://github.com/asvspoof/D3M.

cs.CV↗

arXiv2021

Data-driven Nonlinear Model Reduction to Spectral Submanifolds in Mechanical Systems

While data-driven model reduction techniques are well-established for linearizable mechanical systems, general approaches to reducing non-linearizable systems with multiple coexisting steady states have been unavailable. In this paper, we review such a data-driven nonlinear model reduction methodology based on spectral submanifolds (SSMs). As input, this approach takes observations of unforced nonlinear oscillations to construct normal forms of the dynamics reduced to very low dimensional invariant manifolds. These normal forms capture amplitude-dependent properties and are accurate enough to provide predictions for non-linearizable system response under the additions of external forcing. We illustrate these results on examples from structural vibrations, featuring both synthetic and experimental data.

math.DS↗

arXiv2020

Optical image decomposition and noise filtering based on Laguerre-Gaussian modes

We propose and experimentally demonstrate an efficient image decomposition in the Laguerre-Gaussian (LG) domain. By developing an advanced computing method, the sampling points are much fewer than those in the existing methods, which can significantly improve the calculation efficiency. The beam waist, azimuthal and radial truncation orders of the LG modes are optimized depending on the image information to be restored. In the experiment, we decompose an image by using about 30000 LG modes and realize a high-fidelity reconstruction. Furthermore, we show image noise reduction through LG domain filtering. Our results open a door for LG-mode based image processing.

eess.IV↗