arXiv subjects

Fangzheng Xie

Publications and source records attributed to Fangzheng Xie.

At least 19 recordsLinked to original sources

arXiv2025

Meta-Router: Bridging Gold-standard and Preference-based Evaluations in Large Language Model Routing

In language tasks that require extensive human--model interaction, deploying a single "best" model for every query can be expensive. To reduce inference cost while preserving the quality of the responses, a large language model (LLM) router selects the most appropriate model from a pool of candidates for each query. A central challenge to training a high-quality router is the scarcity of reliable supervision. Gold-standard data (e.g., expert-verified labels or rubric-based scores) provide accurate quality evaluations of LLM responses but are costly and difficult to scale. In contrast, preference-based data, collected via crowdsourcing or LLM-as-a-judge systems, are cheaper and more scalable, yet often biased in reflecting the true quality of responses. We cast the problem of LLM router training with combined gold-standard and preference-based data into a causal inference framework by viewing the response evaluation mechanism as the treatment assignment. This perspective further reveals that the bias in preference-based data corresponds to the well-known causal estimand: the conditional average treatment effect. Based on this new perspective, we develop an integrative causal router training framework that corrects preference-data bias, address imbalances between two data sources, and improve routing robustness and efficiency. Numerical experiments demonstrate that our approach delivers more accurate routing and improves the trade-off between cost and quality.

cs.LG↗

arXiv2025

Uncertainty quantification for mixed membership in multilayer networks with degree heterogeneity using Gaussian variational inference

Analyzing multilayer networks is central to understanding complex relational measurements collected across multiple conditions or over time. A pivotal task in this setting is to quantify uncertainty in community structure while appropriately pooling information across layers and accommodating layer-specific heterogeneity. Building on the multilayer degree-corrected mixed-membership (ML-DCMM) model, which captures both stable community membership profiles and layer-specific vertex activity levels, we propose a Bayesian inference framework based on a spectral-assisted likelihood. We then develop a computationally efficient Gaussian variational inference algorithm implemented via stochastic gradient descent. Our theoretical analysis establishes a variational Bernstein--von Mises theorem, which provides a frequentist guarantee for using the variational posterior to construct confidence sets for mixed memberships. We demonstrate the utility of the method on a U.S. airport longitudinal network, where the procedure yields robust estimates, natural uncertainty quantification, and competitive performance relative to state-of-the-art methods.

stat.ME↗

arXiv2025

SANVI: A Fast Spectral-Assisted Network Variational Inference Method with an Extended Surrogate Likelihood Function

Bayesian inference has been broadly applied to statistical network analysis, but suffers from the expensive computational costs due to the nature of Markov chain Monte Carlo sampling algorithms. This paper proposes a novel and computationally efficient Spectral-Assisted Network Variational Inference (SANVI) method within the framework of the generalized random dot product graph. The key idea is a cleverly designed extended surrogate likelihood function that enjoys two convenient features. Firstly, it decouples the generalized inner product of latent positions in the random graph model. Secondly, it relaxes the complicated domain of the original likelihood function to the entire Euclidean space. Leveraging these features, we design a computationally efficient Gaussian variational inference algorithm via stochastic gradient descent. Furthermore, we show the asymptotic efficiency of the maximum extended surrogate likelihood estimator and the Bernstein-von Mises limit of the variational posterior distribution. Through extensive numerical studies, we demonstrate the usefulness of the proposed SANVI algorithm compared to the classical Markov chain Monte Carlo algorithm, including comparable estimation accuracy for the latent positions and less computational costs.

stat.CO↗

arXiv2025

Statistical inference of random graphs with a surrogate likelihood function

Spectral estimators have been broadly applied to statistical network analysis, but they do not incorporate the likelihood information of the network sampling model. This paper proposes a novel surrogate likelihood function for statistical inference of a class of popular network models referred to as random dot product graphs. In contrast to the structurally complicated exact likelihood function, the surrogate likelihood function has a separable structure and is log-concave yet approximates the exact likelihood function well. From the frequentist perspective, we study the maximum surrogate likelihood estimator and establish the accompanying theory. We show its existence, uniqueness, large sample properties, and that it improves upon the baseline spectral estimator with a smaller sum of squared errors. Furthermore, we derive the second-order bias of the proposed estimator and gain insight into why it outperforms some of the existing estimators. A computationally convenient stochastic gradient descent algorithm is designed to find the maximum surrogate likelihood estimator in practice. From the Bayesian perspective, we establish the Bernstein--von Mises theorem of the posterior distribution with the surrogate likelihood function and show that the resulting credible sets have the correct frequentist coverage. The empirical performance of the proposed surrogate-likelihood-based methods is validated through the analyses of simulation examples and a real-world Wikipedia graph dataset.

stat.ME↗

arXiv2024

Bias-Corrected Joint Spectral Embedding for Multilayer Networks with Invariant Subspace: Entrywise Eigenvector Perturbation and Inference

In this paper, we propose to estimate the invariant subspace across heterogeneous multiple networks using a novel bias-corrected joint spectral embedding algorithm. The proposed algorithm recursively calibrates the diagonal bias of the sum of squared network adjacency matrices by leveraging the closed-form bias formula and iteratively updates the subspace estimator using the most recent estimated bias. Correspondingly, we establish a complete recipe for the entrywise subspace estimation theory for the proposed algorithm, including a sharp entrywise subspace perturbation bound and the entrywise eigenvector central limit theorem. Leveraging these results, we settle two multiple network inference problems: the exact community detection in multilayer stochastic block models and the hypothesis testing of the equality of membership profiles in multilayer mixed membership models. Our proof relies on delicate leave-one-out and leave-two-out analyses that are specifically tailored to block-wise symmetric random matrices and a martingale argument that is of fundamental interest for the entrywise eigenvector central limit theorem.

math.ST↗

arXiv2024

Higher-Order Entrywise Eigenvectors Analysis of Low-Rank Random Matrices: Bias Correction, Edgeworth Expansion, and Bootstrap

Understanding the distributions of spectral estimators in low-rank random matrix models, also known as signal-plus-noise matrix models, is fundamentally important in various statistical learning problems, including network analysis, matrix denoising, and matrix completion. This paper investigates the entrywise eigenvector distributions in a broad range of low-rank signal-plus-noise matrix models by establishing their higher-order accurate stochastic expansions. At a high level, the stochastic expansion states that the eigenvector perturbation approximately decomposes into the sum of a first-order term and a second-order term, where the first-order term in the expansion is a linear function of the noise matrix, and the second-order term is a linear function of the squared noise matrix. Our theoretical finding is used to derive the bias correction procedure for the eigenvectors. We further establish the Edgeworth expansion formula for the studentized entrywise eigenvector statistics. In particular, under mild conditions, we show that Cramér's condition on the smoothness of noise distribution is not required, thanks to the self-smoothing effect of the second-order term in the eigenvector stochastic expansion. The Edgeworth expansion result is then applied to justify the higher-order correctness of the residual bootstrap procedure for approximating the distributions of the studentized entrywise eigenvector statistics.

math.ST↗

arXiv2024

Bayesian Sparse Gaussian Mixture Model in High Dimensions

We study the sparse high-dimensional Gaussian mixture model when the number of clusters is allowed to grow with the sample size. A minimax lower bound for parameter estimation is established, and we show that a constrained maximum likelihood estimator achieves the minimax lower bound. However, this optimization-based estimator is computationally intractable because the objective function is highly nonconvex and the feasible set involves discrete structures. To address the computational challenge, we propose a Bayesian approach to estimate high-dimensional Gaussian mixtures whose cluster centers exhibit sparsity using a continuous spike-and-slab prior. Posterior inference can be efficiently computed using an easy-to-implement Gibbs sampler. We further prove that the posterior contraction rate of the proposed Bayesian method is minimax optimal. The mis-clustering rate is obtained as a by-product using tools from matrix perturbation theory. The proposed Bayesian sparse Gaussian mixture model does not require pre-specifying the number of clusters, which can be adaptively estimated via the Gibbs sampler. The validity and usefulness of the proposed method is demonstrated through simulation studies and the analysis of a real-world single-cell RNA sequencing dataset.

math.ST↗

arXiv2022

An approximate Bayes factor based high dimensional MANOVA using Random Projections

High-dimensional mean vector testing problem for two or more groups remain a very active research area. In these setting, traditional tests are not applicable because they involve the inversion of rank deficient group covariance matrix. In current approaches, this problem is addressed by simply looking at a test assuming a sparse or diagonal covariance matrix potentially ignoring complex dependency between features. In this paper, we develop a Bayes factor (BF) based testing procedure for comparing two or more population means in (very) high dimensional settings. Two versions of the Bayes factor based test statistics are considered which are based on a Random projection (RP) approach. RPs are appealing since they make not assumption about the form of the dependency across features in the data. The final test statistic is based on an ensemble of Bayes factors corresponding to multiple replications of randomly projected data. Both proposed test statistics are compared through a battery of simulation settings. Finally they are applied to the analysis of a publicly available genomic single cell RNA-seq (scRNA-seq) dataset.

stat.ME↗

arXiv2022

Eigenvector-Assisted Statistical Inference for Signal-Plus-Noise Matrix Models

In this paper, we develop a generalized Bayesian inference framework for a collection of signal-plus-noise matrix models arising in high-dimensional statistics and many applications. The framework is built upon an asymptotically unbiased estimating equation with the assistance of the leading eigenvectors of the data matrix. The solution to the estimating equation coincides with the maximizer of an appropriate statistical criterion function. The generalized posterior distribution is constructed by replacing the usual log-likelihood function in the Bayes formula with the criterion function. The proposed framework does not require the complete specification of the sampling distribution and is convenient for uncertainty quantification via a Markov Chain Monte Carlo sampler, circumventing the inconvenience of resampling the data matrix. Under mild regularity conditions, we establish the large sample properties of the estimating equation estimator and the generalized posterior distributions. In particular, the generalized posterior credible sets have the correct frequentist nominal coverage probability provided that the so-called generalized information equality holds. The validity and usefulness of the proposed framework are demonstrated through the analysis of synthetic datasets and the real-world ENZYMES network datasets.

math.ST↗

arXiv2022

Entrywise limit theorems of eigenvectors for signal-plus-noise matrix models with weak signals

We establish a finite-sample Berry-Esseen theorem for the entrywise limits of the eigenvectors for a broad collection of signal-plus-noise random matrix models under challenging weak signal regimes. The signal strength is characterized by a scaling factor $ρ_n$ through $nρ_n$, where $n$ is the dimension of the random matrix, and we allow $nρ_n$ to grow at the rate of $\log n$. The key technical contribution is a sharp finite-sample entrywise eigenvector perturbation bound. The existing error bounds on the two-to-infinity norms of the higher-order remainders are not sufficient when $nρ_n$ is proportional to $\log n$. We apply the general entrywise eigenvector analysis results to the symmetric noisy matrix completion problem, random dot product graphs, and two subsequent inference tasks for random graphs: the estimation of pure nodes in mixed membership stochastic block models and the hypothesis testing of the equality of latent positions in random graphs.

math.ST↗

arXiv2021

Euclidean Representation of Low-Rank Matrices and Its Statistical Applications

Low-rank matrices are pervasive throughout statistics, machine learning, signal processing, optimization, and applied mathematics. In this paper, we propose a novel and user-friendly Euclidean representation framework for low-rank matrices. Correspondingly, we establish a collection of technical and theoretical tools for analyzing the intrinsic perturbation of low-rank matrices in which the underlying referential matrix and the perturbed matrix both live on the same low-rank matrix manifold. Our analyses show that, locally around the referential matrix, the sine-theta distance between subspaces is equivalent to the Euclidean distance between two appropriately selected orthonormal basis, circumventing the orthogonal Procrustes analysis. We also establish the regularity of the proposed Euclidean representation function, which has a profound statistical impact and a meaningful geometric interpretation. These technical devices are applicable to a broad range of statistical problems. Specific applications considered in detail include Bayesian sparse spiked covariance model with non-intrinsic loss, efficient estimation in stochastic block models where the block probability matrix may be degenerate, and least-squares estimation in biclustering problems. Both the intrinsic perturbation analysis of low-rank matrices and the regularity theorem may be of independent interest.

math.ST↗

arXiv2020

Efficient Estimation for Random Dot Product Graphs via a One-step Procedure

We propose a one-step procedure to estimate the latent positions in random dot product graphs efficiently. Unlike the classical spectral-based methods such as the adjacency and Laplacian spectral embedding, the proposed one-step procedure takes advantage of both the low-rank structure of the expected adjacency matrix and the Bernoulli likelihood information of the sampling model simultaneously. We show that for each vertex, the corresponding row of the one-step estimator converges to a multivariate normal distribution after proper scaling and centering up to an orthogonal transformation, with an efficient covariance matrix. The initial estimator for the one-step procedure needs to satisfy the so-called approximate linearization property. The one-step estimator improves the commonly-adopted spectral embedding methods in the following sense: Globally for all vertices, it yields an asymptotic sum of squares error no greater than those of the spectral methods, and locally for each vertex, the asymptotic covariance matrix of the corresponding row of the one-step estimator dominates those of the spectral embeddings in spectra. The usefulness of the proposed one-step procedure is demonstrated via numerical examples and the analysis of a real-world Wikipedia graph dataset.

math.ST↗

arXiv2020

A theoretical framework of the scaled Gaussian stochastic process in prediction and calibration

Model calibration or data inversion is one of fundamental tasks in uncertainty quantification. In this work, we study the theoretical properties of the scaled Gaussian stochastic process (S-GaSP), to model the discrepancy between reality and imperfect mathematical models. We establish the explicit connection between Gaussian stochastic process (GaSP) and S-GaSP through the orthogonal series representation. The predictive mean estimator in the S-GaSP calibration model converges to the reality at the same rate as the GaSP with a suitable choice of the regularization and scaling parameters. We also show the calibrated mathematical model in the S-GaSP calibration converges to the one that minimizes the $L_2$ loss between the reality and mathematical model, whereas the GaSP model with other widely used covariance functions does not have this property. Numerical examples confirm the excellent finite sample performance of our approaches compared to a few recent approaches.

math.ST↗

arXiv2019

BAREB: A Bayesian repulsive biclustering model for periodontal data

Preventing periodontal diseases (PD) and maintaining the structure and function of teeth are important goals for personal oral care. To understand the heterogeneity in patients with diverse PD patterns, we develop BAREB, a Bayesian repulsive biclustering method that can simultaneously cluster the PD patients and their tooth sites after taking the patient- and site- level covariates into consideration. BAREB uses the determinantal point process (DPP) prior to induce diversity among different biclusters to facilitate parsimony and interpretability. Since PD progression is hypothesized to be spatially-referenced, BAREB factors in the spatial dependence among tooth sites. In addition, since PD is the leading cause for tooth loss, the missing data mechanism is non-ignorable. Such nonrandom missingness is incorporated into BAREB. For the posterior inference, we design an efficient reversible jump Markov chain Monte Carlo sampler. Simulation studies show that BAREB is able to accurately estimate the biclusters, and compares favorably to alternatives. For real world application, we apply BAREB to a dataset from a clinical PD study, and obtain desirable and interpretable results. A major contribution of this paper is the Rcpp implementation of BAREB, available at https://github.com/YanxunXu/ BAREB.

stat.AP↗

arXiv2019

A Theoretical Framework for Bayesian Nonparametric Regression

We develop a unifying framework for Bayesian nonparametric regression to study the rates of contraction with respect to the integrated $L_2$-distance without assuming the regression function space to be uniformly bounded. The framework is very flexible and can be applied to a wide class of nonparametric prior models. Three non-trivial applications of the proposed framework are provided: The finite random series regression of an $α$-Hölder function, with adaptive rates of contraction up to a logarithmic factor; The un-modified block prior regression of an $α$-Sobolev function, with adaptive-and-exact rates of contraction; The Gaussian spline regression of an $α$-Hölder function, with the near-optimal posterior contraction. These applications serve as generalization or complement of their respective results in the literature. Extensions to the fixed-design regression problem and sparse additive models in high dimensions are discussed as well.

math.ST↗

arXiv2019

Optimal Bayesian Estimation for Random Dot Product Graphs

We propose a Bayesian approach, called the posterior spectral embedding, for estimating the latent positions in random dot product graphs, and prove its optimality. Unlike the classical spectral-based adjacency/Laplacian spectral embedding, the posterior spectral embedding is a fully-likelihood based graph estimation method taking advantage of the Bernoulli likelihood information of the observed adjacency matrix. We develop a minimax-lower bound for estimating the latent positions, and show that the posterior spectral embedding achieves this lower bound since it both results in a minimax-optimal posterior contraction rate, and yields a point estimator achieving the minimax risk asymptotically. The convergence results are subsequently applied to clustering in stochastic block models, the result of which strengthens an existing result concerning the number of mis-clustered vertices. We also study a spectral-based Gaussian spectral embedding as a natural Bayesian analogy of the adjacency spectral embedding, but the resulting posterior contraction rate is sub-optimal with an extra logarithmic factor. The practical performance of the proposed methodology is illustrated through extensive synthetic examples and the analysis of a Wikipedia graph data.

math.ST↗

arXiv2019

Bayesian Projected Calibration of Computer Models

We develop a Bayesian approach called Bayesian projected calibration to address the problem of calibrating an imperfect computer model using observational data from a complex physical system. The calibration parameter and the physical system are parametrized in an identifiable fashion via $L_2$-projection. The physical process is assigned a Gaussian process prior, which naturally induces a prior distribution on the calibration parameter through the $L_2$-projection constraint. The calibration parameter is estimated through its posterior distribution, which provides a natural and non-asymptotic way for the uncertainty quantification. We provide a rigorous large sample justification for the proposed approach by establishing the asymptotic normality of the posterior of the calibration parameter with the efficient covariance matrix. In addition, two efficient computational algorithms based on stochastic approximation are designed with theoretical guarantees. Through extensive simulation studies and two real-world datasets analyses, we show that the Bayesian projected calibration can accurately estimate the calibration parameters, appropriately calibrate the computer models, and compare favorably to alternative approaches.

stat.ME↗

arXiv2019

Bayesian Estimation of Sparse Spiked Covariance Matrices in High Dimensions

We propose a Bayesian methodology for estimating spiked covariance matrices with jointly sparse structure in high dimensions. The spiked covariance matrix is reparametrized in terms of the latent factor model, where the loading matrix is equipped with a novel matrix spike-and-slab LASSO prior, which is a continuous shrinkage prior for modeling jointly sparse matrices. We establish the rate-optimal posterior contraction for the covariance matrix with respect to the operator norm as well as that for the principal subspace with respect to the projection operator norm loss. We also study the posterior contraction rate of the principal subspace with respect to the two-to-infinity norm loss, a novel loss function measuring the distance between subspaces that is able to capture element-wise eigenvector perturbations. We show that the posterior contraction rate with respect to the two-to-infinity norm loss is tighter than that with respect to the routinely used projection operator norm loss under certain low-rank and bounded coherence conditions. In addition, a point estimator for the principal subspace is proposed with the rate-optimal risk bound with respect to the projection operator norm loss. These results are based on a collection of concentration and large deviation inequalities for the matrix spike-and-slab LASSO prior. The numerical performance of the proposed methodology is assessed through synthetic examples and the analysis of a real-world face data example.

stat.ME↗