Zikai Li | Searcharxiv

arXiv2026

Classification testing: A new framework for drawing qualitative conclusions from quantitative estimates

Social scientists rely on hypothesis testing to support their research conclusions, but the standard tests are designed for testing one hypothesis rather than adjudicating between rival possibilities. We develop a new framework, "classification testing", as an alternative. Instead of selecting one hypothesis to test, a researcher conducting a classification test decides what qualitative distinctions ("classes") are most substantively relevant; the test either assigns the estimand to a class with error control similar to that of a conventional hypothesis test, or declares the result inconclusive. We argue that classification testing is superior to current practice not just when the objective is to adjudicate between rival possibilities but also when there is one research hypothesis to be tested, because classification testing exposes that hypothesis to refutation. We illustrate the framework by applying it to a well-known media experiment and offer an R package to aid in implementation.

stat.ME↗

arXiv2025

Bridging Stratification and Regression Adjustment: Batch-Adaptive Stratification with Post-Design Adjustment in Randomized Experiments

To increase statistical efficiency in a randomized experiment, researchers often use stratification (i.e., blocking) in the design stage. However, conventional practices of stratification fail to exploit valuable information about the predictive relationship between covariates and potential outcomes. In this paper, I introduce an adaptive stratification procedure for increasing statistical efficiency when some information is available about the relationship between covariates and potential outcomes. I show that, in a paired design, researchers can rematch observations across different batches. For inference, I propose a stratified estimator that allows for nonparametric covariate adjustment. I then discuss the conditions under which researchers should expect gains in efficiency from stratification. I show that stratification complements rather than substitutes for regression adjustment, insuring against adjustment error even when researchers plan to use covariate adjustment. To evaluate the performance of the method relative to common alternatives, I conduct simulations using both synthetic data and more realistic data derived from a political science experiment. Results demonstrate that the gains in precision and efficiency can be substantial.

stat.ME↗