SearcharxivSearch

arXiv subjects

Anton Tarasenko

Publications and source records attributed to Anton Tarasenko.

3 recordsLinked to original sources

Only Segmented Heavy Tails Can Produce a Light-Tailed Minimum

A random variable $\xi$ has a {\it light-tailed} distribution (for short: is light-tailed) if it possesses a finite exponential moment, $\E \exp (\lambda \xi) <\infty$ for some $\lambda >0$, and has a {\it heavy-tailed} distribution (is heavy-tailed) if $\E \exp (\lambda\xi) = \infty$, for all $\lambda>0$. In \cite{LSK1}, the authors presented a particular example of a light-tailed random variable that is the minimum of two independent heavy-tailed random variables. In \cite{FKT}, it was shown that any light-tailed random variable with right-unbounded support may be represented as the minimum of two independent heavy-tailed random variables, with further generalisations of the result in a number of directions. We analyse an ``inverse'' question. Namely, we obtain necessary and sufficient conditions on the distribution of a heavy-tailed random variable, say $\xi_1$, that allow to find another independent heavy-tailed random variable, say $\xi_2$, such that their minimum $\min (\xi_1,\xi_2)$ is light-tailed. We also provide a number of extensions of this result

math.PR

Convergence, Sticking and Escape: Stochastic Dynamics Near Critical Points in SGD

We study the convergence properties and escape dynamics of Stochastic Gradient Descent (SGD) in one-dimensional landscapes, separately considering infinite- and finite-variance noise. Our main focus is to identify the time scales on which SGD reliably moves from an initial point to the local minimum in the same ''basin''. Under suitable conditions on the noise distribution, we prove that SGD converges to the basin's minimum unless the initial point lies too close to a local maximum. In that near-maximum scenario, we show that SGD can linger for a long time in its neighborhood. For initial points near a ''sharp'' maximum, we show that SGD does not remain stuck there, and we provide results to estimate the probability that it will reach each of the two neighboring minima. Overall, our findings present a nuanced view of SGD's transitions between local maxima and minima, influenced by both noise characteristics and the underlying function geometry.

cs.LG

Any random variable with right-unbounded distributional support is the minimum of independent and very heavy-tailed random variables

A random variable $\xi$ has a {\it light-tailed} distribution (for short: is light-tailed) if it possesses a finite exponential moment, $\E \exp (\lambda \xi) <\infty$ for some $\lambda >0$, and has a {\it heavy-tailed} distribution (is heavy-tailed) if $\E \exp (\lambda\xi) = \infty$, for all $\lambda>0$. In (Leipus et al., AIMS Mathematics, 2023), the authors presented a particular example of a light-tailed random variable that is the minimum of two independent heavy-tailed random variables. We will show that this phenomenon is universal: {\it any} light-tailed random variable with right-unbounded support may be represented as the minimum of two independent heavy-tailed random variables. Moreover, a more general fact holds: these two independent random variables may have as heavy-tailed distributions as one wishes. Further, we will extend the latter result onto the minimum of any finite number of independent random variables. We will also comment on possible generalizations of our result to the case of dependent random variables.

math.PR