An $Ω(κ_y^8ε^{-6})$ Lower Bound for Stochastic NC-SC Bilevel Optimization with First-order Oracles

Abstract

We study the oracle complexity of finding $ε$-stationary points of smooth bilevel optimization problems with a nonconvex upper-level objective and a strongly convex lower-level problem. We consider a stochastic first-order oracle that returns unbiased stochastic gradients of both the upper- and lower-level objectives, with variance bounded by $σ^2$. We prove that, for any initial optimality gap $Δ>0$ and all sufficiently small $ε>0$, every adaptive randomized first-order algorithm requires $Ω\!\left(Δκ_y^2 ε^{-2}\max\{1,σ^2κ_y^6ε^{-4}\}\right)$ oracle queries to find an $ε$-stationary point of its hyper-objective function, where $κ_y$ denotes the condition number of the lower-level problem. In particular, in the noise-dominated regime, the lower bound is $Ω\!\left(Δσ^2κ_y^8ε^{-6}\right)$. This establishes the optimality of the $ε^{-6}$ dependence achieved by the best-known first-order stochastic methods.