Polynomial Learning of Positive ReLU Networks under Gaussian Inputs

Abstract

We study positive, bias-free, one-hidden-layer ReLU networks under standard Gaussian inputs. We obtain algorithms with polynomial sample and real-arithmetic complexity, without separation or conditioning assumptions on the hidden directions. When the conditional label mean is a width-$k$ network with total output weight at most one, the learner achieves squared prediction error $\varepsilon$ with output width at most $k+\min\{k,d\}$ and complexity linear in one plus the residual second moment. For arbitrary labels with bounded second moment, we obtain a positive ReLU network of width $O(k\log(2/\varepsilon))$. Its risk is bounded by the irreducible noise variance plus $Ck^3\log^{9/2}(2k/\varepsilon)$ times the squared distance of the conditional mean to the target class, plus $\varepsilon$. The proofs combine implicit Hermite moments, orthogonal tensor-train compression, and a direction decoder whose probability of returning a direction far from every atom is bounded linearly in the compression and estimation errors. Exact Gaussian localization refines the resulting direction covers. In the agnostic setting, a cone-constrained convex program with a scale-dependent penalty produces a positive network; in the conditional-mean setting, exact sparsification yields the smaller width. We also establish projective direction recovery and confidence amplification with sample complexity proportional to the logarithm of the reciprocal failure probability.