On Generalized Barron Spaces for Shallow Neural Networks

Abstract

Classical Barron spaces are function spaces specifically designed for shallow neural networks mostly with ReLU, $\mathrm{ReLU}^k$ (RePU) or Lipschitz continuous activation functions. In the present work, we introduce a generalized Barron space \( B_{\sigma}^{\varphi} \) for shallow neural networks with a generic activation function possessing certain smoothness properties. The subscript \( \sigma \) denotes the activation function, while the superscript \( \varphi \) controls the smoothness of the generalized Barron spaces, defined via a \( \varphi \)-weighted integral norm imposed on neural network parameter measures. Under certain assumptions on \( \varphi \) and \( \sigma \), we show that \( B_{\sigma}^{\varphi} \) can be continuously embedded into Sobolev spaces. We also explore the relationships among various function spaces for shallow neural networks, demonstrating that our definition encompasses most conventional ones. As applications of the proposed generalized Barron spaces, we derive approximation rates within these spaces and establish error bounds for numerical differentiation with regularization penalized by the newly introduced generalized Barron norm. Numerical examples confirm that the proposed spaces allow the construction of neural networks with varying degrees of smoothness while using the same activation function.