Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2005.10807

Kolmogorov Width Decay and Poor Approximators in Machine Learning: Shallow Neural Networks, Random Feature Models and Neural Tangent Kernels

Weinan E·Stephan Wojtowytsch
View original source ↗Read source document ↗Export metadata ↓

Abstract

We establish a scale separation of Kolmogorov width type between subspaces of a given Banach space under the condition that a sequence of linear maps converges much faster on one of the subspaces. The general technique is then applied to show that reproducing kernel Hilbert spaces are poor $L^2$-approximators for the class of two-layer neural networks in high dimension, and that multi-layer networks with small path norm are poor approximators for certain Lipschitz functions, also in the $L^2$-topology.

Explore related subjects

math.FAcs.LGstat.ML

Keep this discovery

BibTeXRIS

Weinan E, Stephan Wojtowytsch. 2020-05-21. Kolmogorov Width Decay and Poor Approximators in Machine Learning: Shallow Neural Networks, Random Feature Models and Neural Tangent Kernels. https://arxiv.org/abs/2005.10807

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Typical dynamical properties of operators on $\ell_p$

We investigate the typical dynamical properties of hypercyclic operators in $\mathcal{L}_M(X)$, the set of all bounded linear operators on $X$ whose norms are at most $M$, when $X=\ell_p$, $1< p<\infty$. We show that, with respect to SOT$^*$, a typical operator $T\in \mathcal{L}_M(X)$ is weakly mixing, is weakly disjoint from a given hypercyclic operator $S$, is not topologically ergodic, and satisfies $(T,T^2,\dotsc,T^k)$ is disjoint hypercyclic for any $k\geq 2$. We also study the typical dynamical properties for the concrete family $\mathcal{M}=\{I+B_w\in \mathcal{L}(X)\colon w\in c_0(\mathbb{Z})\}$, endowed with the norm topology, where $B_w$ is a bilateral weighted backward shift.

math.FA↗
arXiv2026

A bi-Lipschitz characterization of strong minimum-attainment for Lipschitz maps

We completely characterize the denseness of strongly minimum-attaining Lipschitz functions, a minimum analogue for strongly norm-attaining Lipschitz functions, in terms of bi-Lipschitz embeddings. More precisely, our main result shows that the set of strongly minimum-attaining Lipschitz functions defined on a complete metric space $M$ fails the denseness if and only if $M$ is bi-Lipschitz equivalent to a subset of $\mathbb{R}$ with positive Lebesgue measure, or equivalently, if $M$ admits a bi-Lipschitz embedding into $\mathbb{R}$ and $M$ has positive 1-dimensional Hausdorff measure. As a consequence, we provide an isometric characterization of the pure 1-unrectifiability of $M$ in terms of strongly minimum-attaining Lipschitz maps defined on bi-Lipschitz copies of closed subsets of $M$. Several counterexamples showing that the main result cannot be naturally extended to the vector-valued setting are also presented.

math.FA↗
arXiv2026

On weak dominance of t-conorms over t-norms

The weak dominance of aggregation operators, particularly between triangular norms (t-norms) and triangular conorms (t-conorms), has attracted considerable attention in aggregation operator theory. While several characterizations have been obtained for Archimedean and continuous cases, a general criterion for continuous t-conorms over continuous t-norms remains to be fully clarified. In this paper, we provide a complete characterization of a continuous t-conorm weakly dominating a continuous t-norm. We first reduce the problem for ordinal sum operators to that for their single Archimedean components, and then express the weak dominance condition entirely in terms of the additive generators of these components. Our approach covers both strict and nilpotent cases uniformly, and recovers the known results for Archimedean operators as a special case.

math.FA↗