arXiv · 2611.10943
Forecasting Replication Study Results from Meta-Analyses
Abstract
Multisite replication projects often rely on published meta-analyses to plan their sample sizes, but meta-analyses tend to overestimate population effects because of publication bias and differences in study characteristics. We developed methods to forecast site-level estimates of a not-yet-conducted replication from a meta-analysis. Extending calibrated estimates from meta-regression, we constructed nonparametric prediction intervals that condition on each site's moderator levels and correct for publication bias through inverse-probability-of-publication weighting under an assumed severity. These intervals achieve nominal coverage asymptotically when population effects are normally distributed, with a bounded coverage error otherwise. We compared four forecasting models (intercept-only meta-analysis or meta-regression, each with and without publication bias correction) in simulation study (81 scenarios and blinded retrospective forecasts of two multisite replications in infant cognition: ManyBabies 1 (104 sites) and ManyBabies 4 (33 sites). Correcting for publication bias improved interval calibration, with gains that increased with bias severity. Meta-regression improved forecasts only when measured moderators explained effect variation and replication sites differed meaningfully on them. In practice, publication bias correction should be the default when forecasting replications from meta-analyses, and replication projects that deliberately vary measured moderators across sites would enable more informative forecasts.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Yi Li, Alvin Wei Ming Tan, Francesco Margoni, Michael C. Frank, Maya B. Mathur. 2026-11-07. Forecasting Replication Study Results from Meta-Analyses. https://searcharxiv.com/abs/2611.10943
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.