arXiv · 2611.14811
Improved last-iterate bounds for SGD with and without replacement
Abstract
In this work, we study last-iterate convergence bounds for stochastic gradient descent (SGD) on convex and $β$-smooth loss functions and improve the bound from $\cO((\log T)^3/T)$ to $\cO((\log T)^3/(T\log\log T))$ in the interpolation regime. The improvement comes from applying a horizon-dependent multiscale estimate of gradients to the comparison of trajectories. The bound applies to both sampling with replacement and sampling without replacement within one pass.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Yimin Zhong, Jiaxin Zhou. 2026-11-10. Improved last-iterate bounds for SGD with and without replacement. https://searcharxiv.com/abs/2611.14811
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.