Approximation Rates for Neural Networks with Encodable Weights in Smoothness Spaces

Ingo Gühring; Mones Raslan

Home/Archive/arXiv

arXiv · 2006.16822

Approximation Rates for Neural Networks with Encodable Weights in Smoothness Spaces

Ingo Gühring·Mones Raslan

View original source ↗Read source document ↗Export metadata ↓

Abstract

We examine the necessary and sufficient complexity of neural networks to approximate functions from different smoothness spaces under the restriction of encodable network weights. Based on an entropy argument, we start by proving lower bounds for the number of nonzero encodable weights for neural network approximation in Besov spaces, Sobolev spaces and more. These results are valid for all sufficiently smooth activation functions. Afterwards, we provide a unifying framework for the construction of approximate partitions of unity by neural networks with fairly general activation functions. This allows us to approximate localized Taylor polynomials by neural networks and make use of the Bramble-Hilbert Lemma. Based on our framework, we derive almost optimal upper bounds in higher-order Sobolev norms. This work advances the theory of approximating solutions of partial differential equations by neural networks.

Explore related subjects

math.FA cs.NA math.NA

Keep this discovery

BibTeX RIS

Ingo Gühring, Mones Raslan. 2020-06-30. Approximation Rates for Neural Networks with Encodable Weights in Smoothness Spaces. https://arxiv.org/abs/2006.16822

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Typical dynamical properties of operators on $\ell_p$

We investigate the typical dynamical properties of hypercyclic operators in $\mathcal{L}_M(X)$, the set of all bounded linear operators on $X$ whose norms are at most $M$, when $X=\ell_p$, $1< p<\infty$. We show that, with respect to SOT$^*$, a typical operator $T\in \mathcal{L}_M(X)$ is weakly mixing, is weakly disjoint from a given hypercyclic operator $S$, is not topologically ergodic, and satisfies $(T,T^2,\dotsc,T^k)$ is disjoint hypercyclic for any $k\geq 2$. We also study the typical dynamical properties for the concrete family $\mathcal{M}=\{I+B_w\in \mathcal{L}(X)\colon w\in c_0(\mathbb{Z})\}$, endowed with the norm topology, where $B_w$ is a bilateral weighted backward shift.

math.FA↗

arXiv2026

A bi-Lipschitz characterization of strong minimum-attainment for Lipschitz maps

We completely characterize the denseness of strongly minimum-attaining Lipschitz functions, a minimum analogue for strongly norm-attaining Lipschitz functions, in terms of bi-Lipschitz embeddings. More precisely, our main result shows that the set of strongly minimum-attaining Lipschitz functions defined on a complete metric space $M$ fails the denseness if and only if $M$ is bi-Lipschitz equivalent to a subset of $\mathbb{R}$ with positive Lebesgue measure, or equivalently, if $M$ admits a bi-Lipschitz embedding into $\mathbb{R}$ and $M$ has positive 1-dimensional Hausdorff measure. As a consequence, we provide an isometric characterization of the pure 1-unrectifiability of $M$ in terms of strongly minimum-attaining Lipschitz maps defined on bi-Lipschitz copies of closed subsets of $M$. Several counterexamples showing that the main result cannot be naturally extended to the vector-valued setting are also presented.

math.FA↗

arXiv2026

On weak dominance of t-conorms over t-norms

The weak dominance of aggregation operators, particularly between triangular norms (t-norms) and triangular conorms (t-conorms), has attracted considerable attention in aggregation operator theory. While several characterizations have been obtained for Archimedean and continuous cases, a general criterion for continuous t-conorms over continuous t-norms remains to be fully clarified. In this paper, we provide a complete characterization of a continuous t-conorm weakly dominating a continuous t-norm. We first reduce the problem for ordinal sum operators to that for their single Archimedean components, and then express the weak dominance condition entirely in terms of the additive generators of these components. Our approach covers both strict and nilpotent cases uniformly, and recovers the known results for Archimedean operators as a special case.

math.FA↗