Regularization Using Synthetic Data for High-Dimensional Inference

Abstract

To address the challenges of obtaining reliable inference in high-dimensional models, we introduce the Synthetic-data Regularized Estimator (SRE). Unlike traditional regularization methods, the SRE regularizes the complex target model via a weighted likelihood based on synthetic data generated from a simpler, more stable model. This method provides a theoretically sound and practically effective alternative to parameter penalization. We establish key theoretical properties of the SRE in generalized linear models, including existence, stability, consistency, and minimax rate optimality. We leverage the Convex Gaussian Min-max Theorem to derive precise asymptotic characterizations in high-dimensional linear regimes where $n/p \to \delta > 0$, both for noninformative synthetic data and for informative auxiliary data in a transfer learning setting. Our asymptotic results characterize how performance depends on the signal strength and the similarity between target and auxiliary data sources. Building upon these results, we develop practical methodologies for high-dimensional inference, including tuning parameter selection, confidence interval construction, and calibrated variable selection. The effectiveness of the SRE is demonstrated through simulation studies and real-data applications.