Singularity of Data Analytic Operations

Abstract

Statistical data by their very nature are indeterminate in the sense that if one repeats the process of collecting the data the new data set will be different from the original. But two data sets generated in the same way should ``tell the same story''. Therefore, a statistical method, a map $\Phi$ taking a data set $x$ to a point in some space $\mathsf{F}$, should be stable at $x$: Small perturbations in $x$ should result in a small change in $\Phi(x)$. Otherwise, $\Phi$ is useless at $x$ or -- and this is important -- near $x$. So one doesn't want $\Phi$ to have "singularities," data sets $x$ such that the the limit of $\Phi(y)$ as $y$ approaches $x$ doesn't exist. (The same issue arises elsewhere in applied math.) We prove that broad classes of statistical methods have topological obstructions to continuity: They must have singularities. We derive broadly applicable lower bounds on the Hausdorff dimension, even Hausdorff measure, of the set of singularities of data maps. General results concerning severity of singularities are proved. For illustration, we show our results apply to plane fitting, measuring location of data on spheres, and to linear classification. This is not a "final" version, merely another attempt.