Searcharxiv⌕ Search

arXiv subjects

Hans-Peter Beise

Publications and source records attributed to Hans-Peter Beise.

10 recordsLinked to original sources
arXiv2026

On Unstable Fixed Points in Modern Continuous Hopfield Networks

The recently introduced continuous Hopfield network (see Ramsauer et al.) exhibits large memorization capabilities, which manifest as attractive fixed points of its update rule -- a differentiable function consisting of two linear mappings composed with the scaled softmax function. The authors of the aforementioned work provide proofs for the existence and approximate position of such attractive fixed points. For the softmax function alone, the fixed point structure has been fully characterized in earlier work by P. Tiňo, from which it turns out that for sufficiently large scaling factors there are exponentially more unstable fixed points than attractive ones. In this work, we complement the findings of Ramsauer et al. by showing that, under natural geometric conditions on the vectors defining the continuous Hopfield network, unstable fixed points must occur, analogous to the findings of Tiňo. Our results show that, under these geometric conditions, continuous Hopfield networks necessarily admit additional unstable fixed points associated with higher-dimensional faces of the pattern polytope.

math.DS↗
arXiv2024

Hyperplane Arrangements and Fixed Points in Iterated PWL Neural Networks

We leverage the framework of hyperplane arrangements to analyze potential regions of (stable) fixed points. We provide an upper bound on the number of fixed points for multi-layer neural networks equipped with piecewise linear (PWL) activation functions with arbitrary many linear pieces. The theoretical optimality of the exponential growth in the number of layers of the latter bound is shown. Specifically, we also derive a sharper upper bound on the number of stable fixed points for one-hidden-layer networks with hard tanh activation.

cs.LG↗
arXiv2023

Topological properties of basins of attraction and expressiveness of width bounded neural networks

In Radhakrishnan et al. [2020], the authors empirically show that autoencoders trained with usual SGD methods shape out basins of attraction around their training data. We consider network functions of width not exceeding the input dimension and prove that in this situation basins of attraction are bounded and their complement cannot have bounded components. Our conditions in these results are met in several experiments of the latter work and we thus address a question posed therein. We also show that under some more restrictive conditions the basins of attraction are path-connected. The tightness of the conditions in our results is demonstrated by means of several examples. Finally, the arguments used to prove the above results allow us to derive a root cause why scalar-valued neural network functions that fulfill our bounded width condition are not dense in spaces of continuous functions.

cs.LG↗
arXiv2022

Mixing operators with prescribed unimodular eigenvectors

For arbitrary closed countable subsets $Z$ of the unit circle examples of topologically mixing operators on Hilbert spaces are given which have a densely spanning set of unimodular eigenvectors with eigenvalues restricted to $Z$. In particular, these operators cannot be ergodic in the Gaussian sense.

math.DS↗
arXiv2021

On decision regions of narrow deep neural networks

We show that for neural network functions that have width less or equal to the input dimension all connected components of decision regions are unbounded. The result holds for continuous and strictly monotonic activation functions as well as for the ReLU activation function. This complements recent results on approximation capabilities by [Hanin 2017 Approximating] and connectivity of decision regions by [Nguyen 2018 Neural] for such narrow neural networks. Our results are illustrated by means of numerical experiments.

cs.LG↗
arXiv2020

A Hybrid Objective Function for Robustness of Artificial Neural Networks -- Estimation of Parameters in a Mechanical System

In several studies, hybrid neural networks have proven to be more robust against noisy input data compared to plain data driven neural networks. We consider the task of estimating parameters of a mechanical vehicle model based on acceleration profiles. We introduce a convolutional neural network architecture that is capable to predict the parameters for a family of vehicle models that differ in the unknown parameters. We introduce a convolutional neural network architecture that given sequential data predicts the parameters of the underlying data's dynamics. This network is trained with two objective functions. The first one constitutes a more naive approach that assumes that the true parameters are known. The second objective incorporates the knowledge of the underlying dynamics and is therefore considered as hybrid approach. We show that in terms of robustness, the latter outperforms the first objective on noisy input data.

cs.LG↗
arXiv2020

SVIRO: Synthetic Vehicle Interior Rear Seat Occupancy Dataset and Benchmark

We release SVIRO, a synthetic dataset for sceneries in the passenger compartment of ten different vehicles, in order to analyze machine learning-based approaches for their generalization capacities and reliability when trained on a limited number of variations (e.g. identical backgrounds and textures, few instances per class). This is in contrast to the intrinsically high variability of common benchmark datasets, which focus on improving the state-of-the-art of general tasks. Our dataset contains bounding boxes for object detection, instance segmentation masks, keypoints for pose estimation and depth images for each synthetic scenery as well as images for each individual seat for classification. The advantage of our use-case is twofold: The proximity to a realistic application to benchmark new approaches under novel circumstances while reducing the complexity to a more tractable environment, such that applications and theoretical questions can be tested on a more challenging dataset as toy problems. The data and evaluation server are available under https://sviro.kl.dfki.de.

cs.CV↗
arXiv2015

Generic boundary behaviour of Taylor series in Hardy and Bergman spaces

It is known that, generically, Taylor series of functions holomorphic in the unit disc turn out to be universal series outside of the unit disc and in particular on the unit circle. Due to classical and recent results on the boundary behaviour of Taylor series, for functions in Hardy spaces and Bergman spaces the situation is essentially different. In this paper it is shown that in many respects these results are sharp in the sense that universality generically appears on maximal exceptional sets. As a main tool it is proved that the Taylor (backward) shift on certain Bergman spaces is mixing.

math.CV↗
arXiv2012

Growth of frequently Birkhoff-universal functions of exponential type on rays

We consider growth conditions for (frequently) Birkhoff-universal functions of exponential type with respect to the different rays emanating from the origin. For that purpose, we investigate their (conjugate) indicator diagram or, equivalently, their indicator function. Some known results, where growth is measured with respect to the maximum modulus, are extended.

math.CV↗