Accelerated Stochastic Zeroth-Order Quasar-Convex Optimization

Abstract

We consider unconstrained minimization of smooth quasar-convex functions when only noisy function evaluations are accessible through a stochastic zeroth-order oracle. For these non-convex functions, the standard acceleration method relies on subspace-search mechanisms that require first-order information, being therefore unavailable in zeroth-order regimes. In contrast, the alternative and less conventional continuized method enable to avoid such mechanisms. In this work, we design a zeroth-order continuized algorithm, leading to accelerated convergence guarantees that parallel those of smooth convex optimization up to a quasar-convexity parameter. Our method incorporates a mirror step, improving the dimension dependence when there exists sparse solution.