arXiv subjects

Xuehu Zhu

Publications and source records attributed to Xuehu Zhu.

15 recordsLinked to original sources

arXiv2025

Integral-Operator-Based Spectral Algorithms for Goodness-of-Fit Tests

The widespread adoption of the \emph{maximum mean discrepancy} (MMD) in goodness-of-fit testing has spurred extensive research on its statistical performance. However, recent studies indicate that the inherent structure of MMD may constrain its ability to distinguish between distributions, leaving room for improvement. Regularization techniques have the potential to overcome this limitation by refining the discrepancy measure. In this paper, we introduce a family of regularized kernel-based discrepancy measures constructed via spectral filtering. Our framework can be regarded as a natural generalization of prior studies, removing restrictive assumptions on both kernel functions and filter functions, thereby broadening the methodological scope and the theoretical inclusiveness. We establish non-asymptotic guarantees showing that the resulting tests achieve valid Type~I error control and enhanced power performance. Numerical experiments are conducted to demonstrate the broader generality and competitive performance of the proposed tests compared with existing methods.

stat.ME↗

arXiv2025

Statistical Theory of Multi-stage Newton Iteration Algorithm for Online Continual Learning

We focus on the critical challenge of handling non-stationary data streams in online continual learning environments, where constrained storage capacity prevents complete retention of historical data, leading to catastrophic forgetting during sequential task training. To more effectively analyze and address the problem of catastrophic forgetting in continual learning, we propose a novel continual learning framework from a statistical perspective. Our approach incorporates random effects across all model parameters and allows the dimension of parameters to diverge to infinity, offering a general formulation for continual learning problems. To efficiently process streaming data, we develop a Multi-step Newton Iteration algorithm that significantly reduces computational costs in certain scenarios by alleviating the burden of matrix inversion. Theoretically, we derive the asymptotic normality of the estimator, enabling subsequent statistical inference. Comprehensive validation through synthetic data experiments and two real datasets analyses demonstrates the effectiveness of our proposed method.

stat.ME↗

arXiv2025

Functional Change Point Detection via Adjacent Deviation Subspace

This paper develops the concept of the Adjacent Deviation Subspace (ADS), a novel framework for reducing infinite-dimensional functional data into finite-dimensional vector or scalar representations while preserving critical information of functional change points. To identify this functional subspace, we propose an efficient dimension reduction operator that overcomes the critical limitation of information loss inherent in traditional functional principal component analysis (FPCA). Building upon this foundation, we first construct a test statistic based on the dimension-reducing target operator to test the existence of change points. Second, we present the MPULSE criterion to estimate change point locations in lower-dimensional representations. This approach not only reduces computational complexity and mitigates false positives but also provides intuitive graphical visualization of change point locations. Lastly, we establish a unified analytical framework that seamlessly integrates dimension reduction with precise change point detection. Extensive simulation studies and real-world data applications validate the robustness and efficacy of our method, consistently demonstrating superior performance over existing approaches.

stat.ME↗

arXiv2025

Testing for large-dimensional covariance matrix under differential privacy

The increasing prevalence of high-dimensional data across various applications has raised significant privacy concerns in statistical inference. In this paper, we propose a differentially private integrated statistic for testing large-dimensional covariance structures, enabling accurate statistical insights while safeguarding privacy. First, we analyze the global sensitivity of sample eigenvalues for sub-Gaussian populations, where our method bypasses the commonly assumed boundedness of data covariates. For sufficiently large sample size, the privatized statistic guarantees privacy with high probability. Furthermore, when the ratio of dimension to sample size, $d/n \to y \in (0, \infty)$, the privatized test is asymptotically distribution-free with well-known critical values, and detects the local alternative hypotheses distinct from the null at the fastest rate of $1/\sqrt{n}$. Extensive numerical studies on synthetic and real data showcase the validity and powerfulness of our proposed method.

stat.ME↗

arXiv2024

Distributed quasi-Newton robust estimation under differential privacy

For distributed computing with Byzantine machines under Privacy Protection (PP) constraints, this paper develops a robust PP distributed quasi-Newton estimation, which only requires the node machines to transmit five vectors to the central processor with high asymptotic relative efficiency. Compared with the gradient descent strategy which requires more rounds of transmission and the Newton iteration strategy which requires the entire Hessian matrix to be transmitted, the novel quasi-Newton iteration has advantages in reducing privacy budgeting and transmission cost. Moreover, our PP algorithm does not depend on the boundedness of gradients and second-order derivatives. When gradients and second-order derivatives follow sub-exponential distributions, we offer a mechanism that can ensure PP with a sufficiently high probability. Furthermore, this novel estimator can achieve the optimal convergence rate and the asymptotic normality. The numerical studies on synthetic and real data sets evaluate the performance of the proposed algorithm.

stat.ML↗

arXiv2023

Byzantine-robust distributed one-step estimation

This paper proposes a Robust One-Step Estimator(ROSE) to solve the Byzantine failure problem in distributed M-estimation when a moderate fraction of node machines experience Byzantine failures. To define ROSE, the algorithms use the robust Variance Reduced Median Of the Local(VRMOL) estimator to determine the initial parameter value for iteration, and communicate between the node machines and the central processor in the Newton-Raphson iteration procedure to derive the robust VRMOL estimator of the gradient, and the Hessian matrix so as to obtain the final estimator. ROSE has higher asymptotic relative efficiency than general median estimators without increasing the order of computational complexity. Moreover, this estimator can also cope with the problems involving anomalous or missing samples on the central processor. We prove the asymptotic normality when the parameter dimension p diverges as the sample size goes to infinity, and under weaker assumptions, derive the convergence rate. Numerical simulations and a real data application are conducted to evidence the effectiveness and robustness of ROSE.

math.ST↗

arXiv2023

Corrected kernel principal component analysis for model structural change detection

This paper develops a method to detect model structural changes by applying a Corrected Kernel Principal Component Analysis (CKPCA) to construct the so-called central distribution deviation subspaces. This approach can efficiently identify the mean and distribution changes in these dimension reduction subspaces. We derive that the locations and number changes in the dimension reduction data subspaces are identical to those in the original data spaces. Meanwhile, we also explain the necessity of using CKPCA as the classical KPCA fails to identify the central distribution deviation subspaces in these problems. Additionally, we extend this approach to clustering by embedding the original data with nonlinear lower dimensional spaces, providing enhanced capabilities for clustering analysis. The numerical studies on synthetic and real data sets suggest that the dimension reduction versions of existing methods for change point detection and clustering significantly improve the performances of existing approaches in finite sample scenarios.

stat.ME↗

arXiv2023

Multiple change point detection in tensors

This paper proposes a criterion for detecting change structures in tensor data. To accommodate tensor structure with structural mode that is not suitable to be equally treated and summarized in a distance to measure the difference between any two adjacent tensors, we define a mode-based signal-screening Frobenius distance for the moving sums of slices of tensor data to handle both dense and sparse model structures of the tensors. As a general distance, it can also deal with the case without structural mode. Based on the distance, we then construct signal statistics using the ratios with adaptive-to-change ridge functions. The number of changes and their locations can then be consistently estimated in certain senses, and the confidence intervals of the locations of change points are constructed. The results hold when the size of the tensor and the number of change points diverge at certain rates, respectively. Numerical studies are conducted to examine the finite sample performances of the proposed method. We also analyze two real data examples for illustration.

stat.ME↗

arXiv2020

Detecting multiple change points: a PULSE criterion

The research described herewith investigates detecting change points of means and of variances in a sequence of observations. The number of change points can be divergent at certain rate as the sample size goes to infinity. We define a MOSUM-based objective function for this purpose. Unlike all existing MOSUM-based methods, the novel objective function exhibits an useful ``PULSE" pattern near change points in the sense: at the population level, the value at any change point plus 2 times of the segment length of the moving average attains a local minimum tending to zero following by a local maximum going to infinity. This feature provides an efficient way to simultaneously identify all change points at the sample level. In theory, the number of change points can be consistently estimated and the locations can also be consistently estimated in a certain sense. Further, because of its visualization nature, in practice, the locations can be relatively more easily identified by plots than existing methods in the literature. The method can also handle the case in which the signals of some change points are very weak in the sense that those changes go to zero. Further, the computational cost is very inexpensive. The numerical studies we conduct validate its good performance.

stat.ME↗

arXiv2019

Adaptive-to-model hybrid of tests for regressions

In model checking for regressions, nonparametric estimation-based tests usually have tractable limiting null distributions and are sensitive to oscillating alternative models, but suffer from the curse of dimensionality. In contrast, empirical process-based tests can, at the fastest possible rate, detect local alternatives distinct from the null model, but is less sensitive to oscillating alternative models and with intractable limiting null distributions. It has long been an issue on how to construct a test that can fully inherit the merits of these two types of tests and avoid the shortcomings. We in this paper propose a generic adaptive-to-model hybrid of moment and conditional moment-based test to achieve this goal. Further, a significant feature of the method is to make nonparametric estimation-based tests, under the alternatives, also share the merits of existing empirical process-based tests. This methodology can be readily applied to other kinds of data and constructing other hybrids. As a by-product in sufficient dimension reduction field, the estimation of residual-related central subspace is used to indicate the underlying models for model adaptation. A systematic study is devoted to showing when alternative models can be indicated and when cannot. This estimation is of its own interest and can be applied to the problems with other kinds of data. Numerical studies are conducted to verify the powerfulness of the proposed test.

stat.ME↗

arXiv2016

Dimensionality determination: a thresholding double ridge ratio criterion

Popularly used eigendecomposition-based criteria such as BIC type, ratio estimation and principal component-based criterion often underdetermine model dimensionality for regressions or the number of factors for factor models. This longstanding problem is caused by the existence of one or two dominating eigenvalues compared to other nonzero eigenvalues. To alleviate this difficulty, we propose a thresholding double ridge ratio criterion such that the true dimension can be better identified and is less underdetermined. Unlike all existing eigendecomposition-based criteria, this criterion can define consistent estimate without requiring the uniqueness of minimum and can then handle possible multiple local minima scenarios. This generic strategy would be readily applied to other dimensionality or order determination problems. In this paper, we systematically investigate, for general sufficient dimension reduction theory, the dimensionality determination with fixed and divergent dimensions; for local alternative models that converge to its limiting model with fewer projected covariates, discuss when the number of projected covariates can be consistently estimated, when cannot; and for ultra-high dimensional factor models, study the estimation consistency for the number of common factors. Numerical studies are conducted to examine the finite sample performance of the method.

math.ST↗

arXiv2016

A projection-based adaptive-to-model test for regressions

A longstanding problem of existing empirical process-based tests for regressions is that when the number of covariates is greater than one, they either have no tractable limiting null distributions or are not omnibus. To attack this problem, we in this paper propose a projection-based adaptive-to-model approach. When the hypothetical model is parametric single-index, the method can fully utilize the dimension reduction model structure under the null hypothesis as if the covariate were one-dimensional such that the martingale transformation-based test can be asymptotically distribution-free. Further, the test can automatically adapt to the underlying model structure such that the test can be omnibus and thus detect alternative models distinct from the hypothetical model at the fastest possible rate in hypothesis testing. The method is examined through simulation studied and is illustrated by a real data analysis.

stat.ME↗

arXiv2015

An adaptive-to-model test for partially parametric single-index models

Residual marked empirical process-based tests are commonly used in regression models. However, they suffer from data sparseness in high-dimensional space when there are many covariates. This paper has three purposes. First, we suggest a partial dimension reduction adaptive-to-model testing procedure that can be omnibus against general global alternative models although it fully use the dimension reduction structure under the null hypothesis. This feature is because that the procedure can automatically adapt to the null and alternative models, and thus greatly overcomes the dimensionality problem. Second, to achieve the above goal, we propose a ridge-type eigenvalue ratio estimate to automatically determine the number of linear combinations of the covariates under the null and alternatives. Third, a Monte-Carlo approximation to the sampling null distribution is suggested. Unlike existing bootstrap approximation methods, this gives an approximation as close to the sampling null distribution as possible by fully utilising the dimension reduction model structure under the null. Simulation studies and real data analysis are then conducted to illustrate the performance of the new test and compare it with existing tests.

stat.ME↗

arXiv2015

Dimension reduction-based significance testing in nonparametric regression

A dimension reduction-based adaptive-to-model test is proposed for significance of a subset of covariates in the context of a nonparametric regression model. Unlike existing local smoothing significance tests, the new test behaves like a local smoothing test as if the number of covariates were just that under the null hypothesis and it can detect local alternatives distinct from the null at the rate that is only related to the number of covariates under the null hypothesis. Thus, the curse of dimensionality is largely alleviated when nonparametric estimation is inevitably required. In the cases where there are many insignificant covariates, the improvement of the new test is very significant over existing local smoothing tests on the significance level maintenance and power enhancement. Simulation studies and a real data analysis are conducted to examine the finite sample performance of the proposed test.

stat.ME↗

arXiv2015

Heteroscedasticity Testing for Regression Models: A Dimension Reduction-based Model Adaptive

Heteroscedasticity testing is of importance in regression analysis. Existing local smoothing tests suffer severely from curse of dimensionality even when the number of covariates is moderate because of use of nonparametric estimation. In this paper, a dimension reduction-based model adaptive test is proposed which behaves like a local smoothing test as if the number of covariates were equal to the number of their linear combinations in the mean regression function, in particular, equal to 1 when the mean function contains a single index. The test statistic is asymptotically normal under the null hypothesis such that critical values are easily determined. The finite sample performances of the test are examined by simulations and a real data analysis.

stat.ME↗