Finding Stationary Points under Higher-Order Smoothness: Trapping Algorithms and First-Order Lower Bounds

Abstract

We study the query complexity of finding approximate stationary points under higher-order smoothness using only function values or only gradients. Our results cover different dimensions, oracle models, randomization, and constraints. For constant dimension $d$ and $C^{p,β_p}$ objectives, with $ν:=p+β_p$, we give deterministic function-value gradient trapping algorithms with query complexities $O(\varepsilon^{-(d-1)})$ on $\mathbb R^d$ and $O(\varepsilon^{-(d-1)/ν})$ on $[0,1]^d$. High-order line integration enables gradient-only trapping. For two-dimensional $C^{1,1}$ objectives, randomized integration yields $O(\varepsilon^{-4/3})$ gradient queries on $\mathbb R^2$, breaking the deterministic gradient-only $Θ(\varepsilon^{-2})$ barrier. For general dimension, we introduce a trapping-or-descent framework that lazily reuses boundary models. For $C^{p,β_p}$ objectives with $p\ge2$, it achieves deterministic query complexities $O(d^p+d^{p-1+β_p/(ν-1)} \varepsilon^{-ν/(ν-1)})$ using function values and $O(d^{p-1}+d^{p-2+β_p/(ν-1)} \varepsilon^{-ν/(ν-1)})$ using gradients. These bounds apply to approximate stationary points on $\mathbb R^d$ and approximate KKT points on $[0,1]^d$. Their dimension-independent accuracy exponent matches the classical AR$p$ exponent $(p+1)/p$ when $β_p=1$. For sufficiently high dimension, we prove randomized lower bounds for gradient-only methods. For $C^{1,1}\cap C^{p,1}$ objectives, our lower bound is $Ω(\varepsilon^{-(3/2+1/(2p))})$, matching known upper bounds for $p=2$ and, up to logarithmic factors, for $p=3$. Without the $C^{1,1}$ assumption, we obtain an $Ω(\varepsilon^{-5/3})$ lower bound for every fixed $p\ge3$.