Publication Bias in Asset Pricing Research

Abstract

Researchers are more likely to share notable findings. As a result, published findings tend to overstate the magnitude of real-world phenomena. This bias is a natural concern for asset pricing research, which has found hundreds of return predictors and little consensus on their origins. Empirical evidence on publication bias comes from large scale meta-studies. Meta-studies of cross-sectional return predictability have settled on four stylized facts that demonstrate publication bias is not a dominant factor: (1) almost all findings can be replicated, (2) predictability persists out-of-sample, (3) empirical $t$-statistics are much larger than 2.0, and (4) predictors are weakly correlated. Each of these facts has been demonstrated in at least three meta-studies. Empirical Bayes statistics turn these facts into publication bias corrections. Estimates from three meta-studies find that the average correction (shrinkage) accounts for only 10 to 15 percent of in-sample mean returns and that the risk of inference going in the wrong direction (the false discovery rate) is less than 10%. Meta-studies also find that $t$-statistic hurdles exceed 3.0 in multiple testing algorithms and that returns are 30 to 50 percent weaker in alternative portfolio tests. These facts are easily misinterpreted as evidence of publication bias effects. We clarify these misinterpretations and others, including the conflating of ``mostly false findings'' with ``many insignificant findings,'' ``data snooping'' with ``liquidity effects,'' and ``failed replications'' with ``insignificant ad-hoc trading strategies.'' Meta-studies outside of the cross-sectional literature are rare. The four facts from cross-sectional meta-studies provide a framework for future research. We illustrate with a preliminary re-examination of equity premium predictability.