Bayesian Transfer Learning for Enhanced Estimation and Inference

Abstract

Transfer learning enhances model performance in a target population with limited samples by leveraging knowledge from related studies. While many works focus on improving predictive performance, challenges of statistical inference persist. Bayesian approaches naturally offer uncertainty quantification for parameter estimates, yet existing Bayesian transfer learning methods are typically limited to single-source scenarios or require individual-level data. We introduce TRansfer leArning via guideD horseshoE prioR (TRADER), a novel approach enabling multi-source transfer through pre-trained models in high-dimensional linear regression. TRADER shrinks target parameters towards a weighted average of source estimates, accommodating sources with different scales. Theoretical investigation shows that TRADER achieves faster posterior contraction rates than standard continuous shrinkage priors when sources align well with the target while preventing negative transfer from heterogeneous sources. The analysis of finite-sample marginal posterior behavior reveals that TRADER achieves desired frequentist coverage probabilities, even for coefficients with moderate signal strength--a scenario where standard continuous shrinkage priors struggle. Extensive numerical studies and a real-data application estimating the association between blood glucose and insulin use in the Hispanic diabetic population demonstrate that TRADER improves estimation and inference accuracy over continuous shrinkage priors using target data alone, while outperforming a state-of-the-art transfer learning method that requires individual-level data.