Approximation of BV functions by neural networks: A regularity theory approach

Benny Avelin; Vesa Julin

Home/Archive/arXiv

arXiv · 2012.08291

Approximation of BV functions by neural networks: A regularity theory approach

Benny Avelin·Vesa Julin

View original source ↗Read source document ↗Export metadata ↓

Abstract

In this paper we are concerned with the approximation of functions by single hidden layer neural networks with ReLU activation functions on the unit circle. In particular, we are interested in the case when the number of data-points exceeds the number of nodes. We first study the convergence to equilibrium of the stochastic gradient flow associated with the cost function with a quadratic penalization. Specifically, we prove a Poincar\'e inequality for a penalized version of the cost function with explicit constants that are independent of the data and of the number of nodes. As our penalization biases the weights to be bounded, this leads us to study how well a network with bounded weights can approximate a given function of bounded variation (BV). Our main contribution concerning approximation of BV functions, is a result which we call the localization theorem. Specifically, it states that the expected error of the constrained problem, where the length of the weights are less than $R$, is of order $R^{-1/9}$ with respect to the unconstrained problem (the global optimum). The proof is novel in this topic and is inspired by techniques from regularity theory of elliptic partial differential equations. Finally we quantify the expected value of the global optimum by proving a quantitative version of the universal approximation theorem.

Explore related subjects

math.AP stat.ML

Keep this discovery

BibTeX RIS

Benny Avelin, Vesa Julin. 2020-12-15. Approximation of BV functions by neural networks: A regularity theory approach. https://arxiv.org/abs/2012.08291

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Well-posedness of the two-dimensional unsteady Prandtl system in Sobolev space with degenerate critical points

This paper is devoted to the well-posedness of classical Prandtl equations in a finite order Sobolev space. For a initial data with degenerate critical points and general outflow, we obtain the local-in-time existence and uniqueness of the solution to the Prandtl equations in a Sobolev space, by introducing a new iteration scheme and linear cancelation. This result shows that Oleinik's monotonicity condition is not a necessary condition for the Prandtl equations to be well-posed in Sobolev spaces and provides evidence to demonstrate that zero shear stress does not necessarily lead to boundary layer separation in two-dimensional unsteady boundary layers.

math.AP↗

arXiv2026

Global existence and time decay for a bipolar Euler-Poisson system with one pressureless and undamped fluid

We study the Cauchy problem for a three-dimensional bipolar Euler--Poisson system in which one fluid is pressureless and undamped, while the other is subject to momentum relaxation. For sufficiently small smooth perturbations of a constant equilibrium, we prove the global existence and uniqueness of smooth solutions under an irrotationality assumption on the initial velocity of the pressureless fluid, together with algebraic time-decay estimates. The main difficulty is that the velocity of the pressureless fluid is dissipated only indirectly through the Poisson coupling, and this mechanism degenerates strongly at high frequencies, leading to a regularity-loss structure. We overcome this difficulty by combining refined Green-function estimates, a low--middle--high frequency decomposition, and high-order nonlinear energy estimates adapted to the asymmetric regularity hierarchy. The result establishes a global small-data theory for this asymmetric regime, in which pressure and damping are simultaneously absent from the same fluid.

math.AP↗

arXiv2026

Boundary layer of 2D Chemotaxis Navier-Stokes equations with logarithmic Sensitivity. II. viscous vanishing limit

This is the second part of a two-part work concerning boundary layer solutions to the coupled Chemotaxis-Navier-Stokes system in the two-dimensional half-space. In the present work, we address the convergence of boundary layer solutions to singular chemotaxis-fluid equations under slip boundary conditions with respect to the chemical diffusion-viscosity parameter $\varepsilon$ in the two-dimensional half-plane. More precisely, we show that the boundary layer for $\varepsilon>0$ (viscous convection coefficient) converges to the superposition of the outer layer (solution with $\varepsilon=0$) and the inner layer as $\varepsilon\rightarrow0$. The outer and inner profiles are explicitly derived as in the first part\cite{WWZ}. Furthermore, the well-posedness results of the coupled Chemotaxis-Navier-Stokes system in conormal Sobolev spaces will be presented in Appendix. They answer the question mentioned in the first part of the two-part work. This study could help the understanding of the chemotactic movement of aerobic bacteria to the water-air surface observed experimentally in fluids, and enrich the theoretical results of boundary layer in chemotactic fluid models.

math.AP↗