Accelerated Stochastic Method under $(H_0,H_1)$-Smoothness and Heavy-Tailed Noise

Abstract

We develop an accelerated stochastic method for convex $(H_0,H_1)$-smooth optimization under heavy-tailed noise. The unbiased oracle has a finite $p$-th noise moment, $1<p\le2$, with constant, gradient-dependent, and gap-dependent terms. Our method combines accelerated updates with clipping, projection, and phase restarts. Each iteration uses a single stochastic-gradient sample, without minibatching. We prove high-probability convergence to any desired accuracy despite clipping bias. The deterministic terms retain square-root dependence on both $H_0$ and $H_1$. In our three-component noise model, strong convexity leaves a polynomial accuracy cost only for the constant component; without it, the dependence is logarithmic even for $p<2$. More generally, for noise moments scaling as $(f(x)-f^\star)^α$, the stochastic cost becomes logarithmic at $α=p$ for convex objectives and $α=p/2$ under strong convexity.