arXiv subjects

Thomas Hamm

Publications and source records attributed to Thomas Hamm.

3 recordsLinked to original sources

arXiv2022

A Derivation of Feedforward Neural Network Gradients Using Fréchet Calculus

We present a derivation of the gradients of feedforward neural networks using Fréchet calculus which is arguably more compact than the ones usually presented in the literature. We first derive the gradients for ordinary neural networks working on vectorial data and show how these derived formulas can be used to derive a simple and efficient algorithm for calculating a neural networks gradients. Subsequently we show how our analysis generalizes to more general neural network architectures including, but not limited to, convolutional networks.

cs.LG↗

arXiv2021

Intrinsic Dimension Adaptive Partitioning for Kernel Methods

We prove minimax optimal learning rates for kernel ridge regression, resp.~support vector machines based on a data dependent partition of the input space, where the dependence of the dimension of the input space is replaced by the fractal dimension of the support of the data generating distribution. We further show that these optimal rates can be achieved by a training validation procedure without any prior knowledge on this intrinsic dimension of the data. Finally, we conduct extensive experiments which demonstrate that our considered learning methods are actually able to generalize from a dataset that is non-trivially embedded in a much higher dimensional space just as well as from the original dataset.

math.ST↗

arXiv2021

Adaptive Learning Rates for Support Vector Machines Working on Data with Low Intrinsic Dimension

We derive improved regression and classification rates for support vector machines using Gaussian kernels under the assumption that the data has some low-dimensional intrinsic structure that is described by the box-counting dimension. Under some standard regularity assumptions for regression and classification we prove learning rates, in which the dimension of the ambient space is replaced by the box-counting dimension of the support of the data generating distribution. In the regression case our rates are in some cases minimax optimal up to logarithmic factors, whereas in the classification case our rates are minimax optimal up to logarithmic factors in a certain range of our assumptions and otherwise of the form of the best known rates. Furthermore, we show that a training validation approach for choosing the hyperparameters of an SVM in a data dependent way achieves the same rates adaptively, that is without any knowledge on the data generating distribution.

math.ST↗