On the Euclidean Distance Degree of Shallow Polynomial Networks

Abstract

We study the projective geometry of shallow polynomial neural networks, namely families of homogeneous polynomial maps realised by a single hidden layer with monomial activation. The closure of the set of maps realised by such a network is a projective variety, its neurovariety. A natural measure of the algebraic complexity of the corresponding best-approximation problem is the generic Euclidean distance degree $\mathrm{gED}$, which counts the complex critical points of the squared distance from a general point. For fixed input dimension, width and activation degree, we prove that $\mathrm{gED}$ is eventually polynomial in the output dimension. Its degree and leading coefficient are governed by an auxiliary variety parametrising the spaces spanned by powers of linear forms. When the activation degree is at least the width, this extends: on the same stable range, $\mathrm{gED}$ is a polynomial in the output dimension and the activation degree jointly. Our proof constructs a proper birational model of the conormal variety over a base that does not depend on the output dimension, and expresses $\mathrm{gED}$ through Chern classes on it. We give closed formulas in two basic cases and determine the leading term for width two.