Evidence aggregation with ignorance in mind: learning what we do (not) know for archetypes discovery

Abstract

When evaluating policy interventions, researchers often pursue two related goals: identifying which individuals or contexts benefit most, and determining whether patterns of treatment effect heterogeneity can be used to aggregate evidence across environments. We develop a framework that aggregates treatment effect heterogeneity, defined over individual and environmental characteristics, into interpretable summaries while setting aside contexts in which extrapolation is unreliable and further evidence is needed. The procedure therefore learns both how to summarize heterogeneous effects and when researchers should admit ignorance. We derive finite-sample regret guarantees, provide data-driven guarantees for selecting the complexity of the summary class, and inference procedures that quantify the value of follow-up data collection. We illustrate the approach by reanalyzing a multifaceted anti-poverty program implemented in six countries.