arXiv subjects

Hengtao Zhang

Publications and source records attributed to Hengtao Zhang.

5 recordsLinked to original sources

arXiv2026

Uniform-Design Subsampling for Compute-Budgeted Double Machine Learning

Double machine learning (DML) combines orthogonal scores with flexible nuisance estimation, but repeated cross-fitting and repeated analysis can make full-data workflows expensive. When a fixed computational budget requires a working sample of size $r\ll n$, simple random subsampling may cover the covariate space poorly and produce unstable treated--control composition. We propose Uniform Design Double Machine Learning (UD-DML), which maps a common low-discrepancy skeleton through empirical marginal quantiles and matches each anchor to one treated and one control observation, without replacement within each treatment arm. We establish finite-sample integration and balance bounds, an exact selected-target decomposition, and a selection-aware leading-score central limit theorem on the root-$r$ scale. Under explicit moment, weighted-calibration, variance-stabilisation, and target-transport conditions, a residual-based variance estimator consistently studentises the original UD-DML estimator. Full-scale simulations show that the common-skeleton construction provides its clearest improvements over uniform subsampling under non-uniform covariate geometry and limited treated--control overlap, while maintaining empirical coverage near the nominal level. These results provide a principled working-sample design for repeated causal learning under an explicit computational budget.

stat.ME↗

arXiv2021

PCA Rerandomization

Mahalanobis distance between treatment group and control group covariate means is often adopted as a balance criterion when implementing a rerandomization strategy. However, this criterion may not work well for high-dimensional cases because it balances all orthogonalized covariates equally. Here, we propose leveraging principal component analysis (PCA) to identify proper subspaces in which Mahalanobis distance should be calculated. Not only can PCA effectively reduce the dimensionality for high-dimensional cases while capturing most of the information in the covariates, but it also provides computational simplicity by focusing on the top orthogonal components. We show that our PCA rerandomization scheme has desirable theoretical properties on balancing covariates and thereby on improving the estimation of average treatment effects. We also show that this conclusion is supported by numerical studies using both simulated and real examples.

stat.ME↗

arXiv2020

An Effective and Efficient Initialization Scheme for Training Multi-layer Feedforward Neural Networks

Network initialization is the first and critical step for training neural networks. In this paper, we propose a novel network initialization scheme based on the celebrated Stein's identity. By viewing multi-layer feedforward neural networks as cascades of multi-index models, the projection weights to the first hidden layer are initialized using eigenvectors of the cross-moment matrix between the input's second-order score function and the response. The input data is then forward propagated to the next layer and such a procedure can be repeated until all the hidden layers are initialized. Finally, the weights for the output layer are initialized by generalized linear modeling. Such a proposed SteinGLM method is shown through extensive numerical results to be much faster and more accurate than other popular methods commonly used for training neural networks.

cs.LG↗

arXiv2020

Balance-Subsampled Stable Prediction

In machine learning, it is commonly assumed that training and test data share the same population distribution. However, this assumption is often violated in practice because the sample selection bias may induce the distribution shift from training data to test data. Such a model-agnostic distribution shift usually leads to prediction instability across unknown test data. In this paper, we propose a novel balance-subsampled stable prediction (BSSP) algorithm based on the theory of fractional factorial design. It isolates the clear effect of each predictor from the confounding variables. A design-theoretic analysis shows that the proposed method can reduce the confounding effects among predictors induced by the distribution shift, hence improve both the accuracy of parameter estimation and prediction stability. Numerical experiments on both synthetic and real-world data sets demonstrate that our BSSP algorithm significantly outperforms the baseline methods for stable prediction across unknown test data.

cs.LG↗

arXiv2020

Adaptive Iterative Hessian Sketch via A-Optimal Subsampling

Iterative Hessian sketch (IHS) is an effective sketching method for modeling large-scale data. It was originally proposed by Pilanci and Wainwright (2016; JMLR) based on randomized sketching matrices. However, it is computationally intensive due to the iterative sketch process. In this paper, we analyze the IHS algorithm under the unconstrained least squares problem setting, then propose a deterministic approach for improving IHS via A-optimal subsampling. Our contributions are three-fold: (1) a good initial estimator based on the A-optimal design is suggested; (2) a novel ridged preconditioner is developed for repeated sketching; and (3) an exact line search method is proposed for determining the optimal step length adaptively. Extensive experimental results demonstrate that our proposed A-optimal IHS algorithm outperforms the existing accelerated IHS methods.

stat.ML↗