Searcharxiv⌕ Search

arXiv subjects

S. Dereich

Publications and source records attributed to S. Dereich.

3 recordsLinked to original sources
arXiv2022

Cooling down stochastic differential equations: almost sure convergence

We consider almost sure convergence of the SDE $dX_t=α_t d t + β_t d W_t$ under the existence of a $C^2$-Lyapunov function $F:\mathbb R^d \to \mathbb R$. More explicitly, we show that on the event that the process stays local we have almost sure convergence in the Lyapunov function $(F(X_t))$ as well as $\nabla F(X_t)\to 0$, if $|β_t|=\mathcal O( t^{-β})$ for a $β>1/2$. If, additionally, one assumes that $F$ is a Lojasiewicz function, we get almost sure convergence of the process itself, given that $|β_t|=\mathcal O(t^{-β})$ for a $β>1$. The assumptions are shown to be optimal in the sense that there is a divergent counterexample where $|β_t|$ is of order $t^{-1}$.

math.PR↗
arXiv2021

On minimal representations of shallow ReLU networks

The realization function of a shallow ReLU network is a continuous and piecewise affine function $f:\mathbb R^d\to \mathbb R$, where the domain $\mathbb R^{d}$ is partitioned by a set of $n$ hyperplanes into cells on which $f$ is affine. We show that the minimal representation for $f$ uses either $n$, $n+1$ or $n+2$ neurons and we characterize each of the three cases. In the particular case, where the input layer is one-dimensional, minimal representations always use at most $n+1$ neurons but in all higher dimensional settings there are functions for which $n+2$ neurons are needed. Then we show that the set of minimal networks representing $f$ forms a $C^\infty$-submanifold $M$ and we derive the dimension and the number of connected components of $M$. Additionally, we give a criterion for the hyperplanes that guarantees that all continuous, piecewise affine functions are realization functions of appropriate ReLU networks.

cs.LG↗