arXiv subjects

Xiaotian Hou

Publications and source records attributed to Xiaotian Hou.

6 recordsLinked to original sources

arXiv2026

Finite-Sample and Distribution-Free Fair Classification: Optimal Trade-off Between Excess Risk and Fairness, and the Cost of Group-Blindness

Algorithmic fairness has become a central concern in modern machine learning and AI applications. However, two pressing challenges remain: (1) The fairness guarantees of existing methods often rely on specific data distributional assumptions and large sample sizes, which can lead to fairness violations in practice. (2) Due to legal and societal considerations, using sensitive group attributes during decision-making (referred to as the group-blind setting) may not always be feasible. In this work, we quantify the impact of enforcing algorithmic fairness and group-blindness/awareness in binary classification under group fairness constraints. Specifically, we propose a unified framework for fair classification that provides distribution-free and finite-sample fairness guarantees with controlled excess risk. This framework is applicable to various group fairness notions in both group-aware and group-blind scenarios. Our approach is based on a post-processing procedure that can be applied to arbitrary black-box models, making it directly compatible with modern machine learning pipelines. Furthermore, we establish a minimax lower bound showing the minimax rate-optimality of our proposed algorithm up to logarithmic factors. Through extensive synthetic and real data studies, we further demonstrate the competitive or superior performance of our algorithm compared to existing methods, and provide empirical support for our theoretical findings.

stat.ME↗

arXiv2026

Labels or Preferences? Budget-Constrained Learning with Human Judgments over AI-Generated Outputs

The increasing reliance on human preference feedback to judge AI-generated pseudo labels has created a pressing need for principled, budget-conscious data acquisition strategies. We address the crucial question of how to optimally allocate a fixed annotation budget between ground-truth labels and pairwise preferences in AI. Our solution, grounded in semi-parametric inference, casts the budget allocation problem as a monotone missing data framework. Building on this formulation, we introduce Preference-Calibrated Active Learning (PCAL), a novel method that learns the optimal data acquisition strategy and develops a statistically efficient estimator for functionals of the data distribution. Theoretically, we prove the asymptotic optimality of our PCAL estimator and establish a key robustness guarantee that ensures robust performance even with poorly estimated nuisance models. Our flexible framework applies to a general class of problems, by directly optimizing the estimator's variance instead of requiring a closed-form solution. This work provides a principled and statistically efficient approach for budget-constrained learning in modern AI. Simulations and real-data analysis demonstrate the practical benefits and superior performance of our proposed method.

stat.ML↗

arXiv2026

Unified Inference Framework for Single and Multi-Player Performative Prediction: Method and Asymptotic Optimality

Performative prediction characterizes environments where predictive models alter the very data distributions they aim to forecast, triggering complex feedback loops. While prior research treats single-agent and multi-agent performativity as distinct phenomena, this paper introduces a unified statistical inference framework that bridges these contexts, treating the former as a special case of the latter. Our contribution is two-fold. First, we put forward the Repeated Risk Minimization (RRM) procedure for estimating the performative stability, and establish a rigorous inferential theory for admitting its asymptotic normality and confirming its asymptotic efficiency. Second, for the performative optimality, we introduce a novel two-step plug-in estimator that integrates the idea of Recalibrated Prediction Powered Inference (RePPI) with Importance Sampling, and further provide formal derivations for the Central Limit Theorems of both the underlying distributional parameters and the plug-in results. The theoretical analysis demonstrates that our estimator achieves the semiparametric efficiency bound and maintains robustness under mild distributional misspecification. This work provides a principled toolkit for reliable estimation and decision-making in dynamic, performative environments.

stat.ML↗

arXiv2025

Repro Samples Method for Model-Free Inference in High-Dimensional Binary Classification

This paper presents a novel method for statistical inference in high-dimensional binary models with unspecified structure, where we leverage a (potentially misspecified) sparsity-constrained working generalized linear model (GLM) to facilitate the inference process. Our method is based on the repro samples framework, which generates artificial samples that mimic the actual data-generating process. Our inference targets include the model support, case probabilities, and the oracle regression coefficients defined in the working GLM. The proposed method has three major advantages. First, this approach is model-free, that is, it does not rely on specific model assumptions such as logistic or probit regression, nor does it require sparsity assumptions on the underlying model. Second, for model support, we construct a model candidate set for the most influential covariates that achieves guaranteed coverage under a weak signal strength assumption. Third, for oracle regression coefficients, we establish confidence sets for any group of linear combinations of regression coefficients. Simulation results demonstrate that the proposed method produces valid and small model candidate sets. It also achieves better coverage for regression coefficients than the state-of-the-art debiasing methods when the working model is the actual model that generates the sample data. Additionally, we analyze single-cell RNA-seq data on the immune response. Besides identifying genes previously proven as relevant in the literature, our method also discovers a significant gene that has not been studied before, revealing a potential new direction in understanding cellular immune response mechanisms.

stat.ME↗

arXiv2024

FactTest: Factuality Testing in Large Language Models with Finite-Sample and Distribution-Free Guarantees

The propensity of Large Language Models (LLMs) to generate hallucinations and non-factual content undermines their reliability in high-stakes domains, where rigorous control over Type I errors (the conditional probability of incorrectly classifying hallucinations as truthful content) is essential. Despite its importance, formal verification of LLM factuality with such guarantees remains largely unexplored. In this paper, we introduce FactTest, a novel framework that statistically assesses whether a LLM can confidently provide correct answers to given questions with high-probability correctness guarantees. We formulate factuality testing as hypothesis testing problem to enforce an upper bound of Type I errors at user-specified significance levels. Notably, we prove that our framework also ensures strong Type II error control under mild conditions and can be extended to maintain its effectiveness when covariate shifts exist. Our approach is distribution-free and works for any number of human-annotated samples. It is model-agnostic and applies to any black-box or white-box LM. Extensive experiments on question-answering (QA) and multiple-choice benchmarks demonstrate that FactTest effectively detects hallucinations and improves the model's ability to abstain from answering unknown questions, leading to an over 40% accuracy improvement.

cs.CL↗

arXiv2024

Repro Samples Method for High-dimensional Logistic Model

This paper presents a novel method to make statistical inferences for both the model support and regression coefficients in a high-dimensional logistic regression model. Our method is based on the repro samples framework, in which we conduct statistical inference by generating artificial samples mimicking the actual data-generating process. The proposed method has two major advantages. Firstly, for model support, we introduce the first method for constructing model confidence set in a high-dimensional setting and the proposed method only requires a weak signal strength assumption. Secondly, in terms of regression coefficients, we establish confidence sets for any group of linear combinations of regression coefficients. Our simulation results demonstrate that the proposed method produces valid and small model confidence sets and achieves better coverage for regression coefficients than the state-of-the-art debiasing methods. Additionally, we analyze single-cell RNA-seq data on the immune response. Besides identifying genes previously proved as relevant in the literature, our method also discovers a significant gene that has not been studied before, revealing a potential new direction in understanding cellular immune response mechanisms.

stat.ME↗