arXiv subjects

Shuxing Fang

Publications and source records attributed to Shuxing Fang.

4 recordsLinked to original sources

arXiv2026

Variance-aware Reward Modeling with Anchor Guidance

Standard Bradley--Terry (BT) reward models are limited when human preferences are pluralistic. Although soft preference labels preserve disagreement information, BT can only express it by shrinking reward margins. Gaussian reward models provide an alternative by jointly predicting a reward mean and a reward variance, but suffer from a fundamental non-identifiability from pairwise preferences alone. We propose Anchor-guided Variance-aware Reward Modeling, a framework that resolves this non-identifiability by augmenting preference data with two coarse response-level anchor labels. Building on this, we prove that two anchors are sufficient for identification, develop a joint training objective and establish a non-asymptotic convergence rate for both the estimated reward mean and variance functions. Across simulation studies and four real-world diverging-preference datasets, our method consistently improves reward modeling performance and downstream RLHF, including PPO training and best-of-$N$ selection.

stat.ML↗

arXiv2026

Deep Ranking with Heterogeneous Effects

Classical latent-score ranking models often fail to distinguish objects' intrinsic scores from contextual effects, which are typically nonlinear and can dominate the observed outcomes. To address this, we introduce a semiparametric ranking framework in which the log-score of each object is modeled as the sum of a utility parameter and a nonparametric covariate effects. Within this framework, we establish model identifiability under mild regularity and connectivity conditions. For estimation, we approximate the covariate effects using a neural network and estimate the parameters via maximum likelihood. Under random design assumptions, we prove that the resulting estimator exists with high probability and derive non-asymptotic error bounds that achieve minimax optimality for both the parametric and nonparametric components. Numerical experiments on both synthetic data and an ATP tennis dataset are conducted to support our findings.

stat.ME↗

arXiv2026

Recent advances in the Bradley--Terry Model: theory, algorithms, and applications

This article surveys recent progress in the Bradley-Terry (BT) model and its extensions. We focus on the statistical and computational aspects, with emphasis on the regime in which both the number of objects and the volume of comparisons tend to infinity, a setting relevant to large-scale applications. The main topics include asymptotic theory for statistical estimation and inference, along with the associated algorithms. We also discuss applications of these models, including recent work on preference alignment in machine learning. Finally, we discuss several key challenges and outline directions for future research.

stat.ME↗

arXiv2024

Two-way Deconfounder for Off-policy Evaluation in Causal Reinforcement Learning

This paper studies off-policy evaluation (OPE) in the presence of unmeasured confounders. Inspired by the two-way fixed effects regression model widely used in the panel data literature, we propose a two-way unmeasured confounding assumption to model the system dynamics in causal reinforcement learning and develop a two-way deconfounder algorithm that devises a neural tensor network to simultaneously learn both the unmeasured confounders and the system dynamics, based on which a model-based estimator can be constructed for consistent policy value estimation. We illustrate the effectiveness of the proposed estimator through theoretical results and numerical experiments.

cs.LG↗