Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 1904.02311

Approximation Rates for Neural Networks with General Activation Functions

Jonathan W. Siegel·Jinchao Xu
View original source ↗Read source document ↗Export metadata ↓

Abstract

We prove some new results concerning the approximation rate of neural networks with general activation functions. Our first result concerns the rate of approximation of a two layer neural network with a polynomially-decaying non-sigmoidal activation function. We extend the dimension independent approximation rates previously obtained to this new class of activation functions. Our second result gives a weaker, but still dimension independent, approximation rate for a larger class of activation functions, removing the polynomial decay assumption. This result applies to any bounded, integrable activation function. Finally, we show that a stratified sampling approach can be used to improve the approximation rate for polynomially decaying activation functions under mild additional assumptions.

Explore related subjects

math.CAcs.LG

Keep this discovery

BibTeXRIS

Jonathan W. Siegel, Jinchao Xu. 2019-04-04. Approximation Rates for Neural Networks with General Activation Functions. https://arxiv.org/abs/1904.02311

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

On the log-concavity of the composite Bessel function $x^{\alpha}J_{\nu }\left( \beta x^{\gamma}\right) $

For a twice differentiable function $f:\left( a,b\right) \rightarrow \mathbb{R}$ define $v\left( f\right) =f^{\prime}f^{\prime}-f^{\prime\prime }f.$ It is well known that the positivity of $v\left( f\right) $ implies that the function $\left\vert f\right\vert $ is strictly log-concave on each subinterval which does not contain zeros of $f.$ In this paper we provide criteria for the positivity of $v\left( F\right) $ for the composite Bessel function $F\left( x\right) =J_{\alpha,\beta,\gamma,\nu}\left( x\right) :=x^{\alpha}J_{\nu}\left( \beta x^{\gamma}\right) $ for positive numbers $\beta$ and $\gamma$ and real numbers $\alpha$ and $\nu.$

math.CA↗
arXiv2026

Riesz capacity ratios with negative exponents

We investigate sharp inequalities for ratios of Riesz capacities with negative exponents by combining computational experiments with rigorous analysis. For finite subsets of the line, we prove positivity of equilibrium masses when $-1<p<0$, enabling numerical tests of conjectured extremal ratios. In the plane, comparisons of the disk with regular polygon vertex sets reveal a cascade of transitions among the tested competitors and suggest a precise conjecture for the equilibrium measure of odd polygons, for which we give a partial proof. Numerical intersections of equality curves show that the regions where these sets outperform the disk are not simply nested. Similar numerical intersections occur in three dimensions between the regular-simplex equality curve and those of explicit five-point and six-point configurations. Motivated by the dimensional dependence of these comparisons, we prove that for each fixed $p<-2<q<0$, the regular simplex has a larger capacity ratio than the ball in all sufficiently large dimensions. Accompanying Python and Mathematica code supports reproduction and further testing of the conjectures.

math.CA↗
arXiv2026

Shorter proof of dimension-free $L^p$ estimates for maximal Riesz transforms

We provide a shorter and more direct proof of $L^p$ estimates for maximal Riesz transforms (of an arbitrary order) in terms of the corresponding Riesz transforms, with a constant independent of the dimension of the Euclidean space $\mathbb R^d$. This result was originally proved by Mateu, Orobitg, P\'erez and Verdera with a constant depending on the dimension, and improved to a dimension-free inequality by Kucharski, Wr\'obel and Zienkiewicz.

math.CA↗