Heavy-Tailed First-Order Optimization for Polyak-\L{}ojasiewicz Condition: High-Dimensional Minimax Bounds, High-Probability Guarantee, and Fixed-Dimensional Improvements

Abstract

We study smooth Polyak--\L{}ojasiewicz (PL) optimization with conditionally unbiased stochastic gradients satisfying \[ \mathbb E\!\left[ \|G_t-\nabla f(x_t)\|^\alpha \mid\mathcal F_{t-1} \right]\le \sigma^\alpha, \qquad 1<\alpha\le2. \] When the dimension may depend on the oracle budget, we prove the noise-adaptive lower bound \[ T_\epsilon = \Omega_\alpha\!\left[ \kappa\log\frac{\Delta_0}{\epsilon} + \kappa \left( \frac{\sigma^2}{\mu\epsilon} \right)^{\frac{\alpha}{2(\alpha-1)}} \right], \] which recovers the noiseless PL lower bound when $\sigma=0$. Under the appropriate mirror-PL condition, we give a centered-clipped mirror-descent method attaining the matching high-probability upper bound up to logarithmic factors, without bounded-domain, bounded-gradient, or sub-Gaussian assumptions. We further characterize the stochastic complexity in prescribed fixed dimensions. For $d=1,2,3$, the optimal stochastic term is \[ \widetilde\Theta_\alpha\!\left[ \left( \frac{\sigma^2}{\mu\epsilon} \right)^{\frac{\alpha}{2(\alpha-1)}} \right]. \] For every fixed $d>3$, the same characterization holds whenever \[ \frac{\alpha}{\alpha-1}\ge d-1. \] In the complementary regime, we provide an upper bound with an additional surface-entropy factor and explicitly identify the remaining gap.