arXiv · 2611.14526
Partial pooling beats synthetic respondents as estimators, not as covariates: a preregistered small-sample tournament
Abstract
Survey estimates for population segments, such as media use by sex and age, rest on few respondents per cell, and a growing literature offers synthetic respondents generated by large language models to fill the gap. That literature has not tested them against the small-area estimators a methodologist would otherwise use. We ran ten estimators, preregistered, on identical household-clustered splits of a Korean media panel across five calibration budgets and three holdout regimes. A multilevel partial-pooling model of the MRP family led every calibration-only estimator at every budget, matching at one hundred respondents what direct estimation reaches with roughly four times as many. On unobserved segments, information fixed as a per-segment prior was inert for synthetic panel and previous wave alike; attached as a bias correction it transferred, at the price of estimation noise. No registered synthetic advantage over MRP survived the paired decision rule. Entered instead as a covariate inside the multilevel model, an exploratory hybrid, the same panel improved on MRP on unobserved segments: the information survives where the registered constructions did not. Interval coverage stayed below nominal throughout and worsened with budget for partial pooling, contrary to prediction.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Howard Kim. 2026-09-30. Partial pooling beats synthetic respondents as estimators, not as covariates: a preregistered small-sample tournament. https://searcharxiv.com/abs/2611.14526
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.