How much is too much? Measuring divergence from Benford's Law with the Equivalent Contamination Proportion (ECP)

Abstract

Conformity with Benford's Law is widely used to detect irregularities in numerical datasets, particularly in accounting, finance, and economics. However, the statistical tools commonly used for this purpose (such as Chi-squared, MAD, or KS) suffer from three key limitations: sensitivity to sample size, lack of interpretability of their scale, and the absence of a common metric that allows for comparison across different statistics. This paper introduces the Equivalent Contamination Proportion (ECP) to address these issues. Defined as the proportion of contamination in a hypothetical Benford-conforming sample such that the expected value of the divergence statistic matches the one observed in the actual data, the ECP provides a continuous and interpretable measure of deviation (ranging from 0 to 1), is robust to sample size, and offers consistent results across different divergence statistics under mild conditions. Closed-form and simulation-based methods are developed for estimating the ECP, and, through a retrospective analysis of three influential studies, it is shown how the ECP can complement the information provided by traditional divergence statistics and enhance the interpretation of results.