Bases of Lebesgue spaces formed by neural networks

Abstract

The seminal work of Daubechies, DeVore, Foucart, Hanin, and Petrova introduced in 2022 a sequence of univariate piece-wise linear functions, which resemble the classical Fourier basis and which, at the same time, can be easily reproduced by artificial neural networks with ReLU activation function. We give an alternative way how to calculate the inner products of functions from this system and discuss the spectral properties of the Gram matrix generated by this system. The univariate system was later generalized to the multivariate setting by two of the authors of this work. Instead of the usual tensor product construction, this generalization relied on the inner products inside of the argument of the univariate sequence. It turned out that such a system forms a Riesz basis of $L_2(0,1)^n$ for every $n\ge 1$ with Riesz constants independent of $n$. In this work, we investigate the properties of these new sequences of functions in $L_q(0,1)^n$ for $q\not =2.$ First, we show that the univariate system is a Schauder basis in $L_q(0,1)$ for every $1<q<\infty$. By a general argument, it follows that the tensor products of this system also form a Schauder basis in $L_q(0,1)^n$ for every $n\ge 2$ and $1<q<\infty.$ The same fact can also be shown by measuring the distance of the tensor product system to the classical multivariate Fourier basis, but - surprisingly - this argument only works for $n\le 3$. If, on the other hand, we replace the outer tensor products by inner products directly in the argument of the univariate system, the same approach is applicable for an arbitrary dimension $n\in{\mathbb N}.$