A Stochastic Riemannian Alternating Descent Ascent Method for Nonsmooth Composite Expectation Optimization on Riemannian Manifolds
In this paper, we consider a class of Riemannian nonsmooth composite expectation optimization problems, which arises in various machine learning, signal processing, and statistics applications. Noting that these problems admit structured minimax reformulations, we propose an efficient algorithm, named stochastic Riemannian alternating descent ascent method with recursive momentum (StoRADA-RM), to tackle them. StoRADA-RM performs one or multiple Riemannian stochastic gradient descent steps and then a proximal gradient ascent step at each iteration. To compute the Riemannian stochastic gradient, we propose a vector transport-free recursive momentum estimator that requires only $O(1)$ stochastic gradient evaluations per iteration. We prove that StoRADA-RM returns an $\epsilon$-Riemannian-stochastic-stationary point of a given problem in the said class in $O(\epsilon^{-3})$ iterations while making $O(\epsilon^{-3})$ calls to a stochastic first-order oracle (SFO). Both the iteration complexity and SFO complexity bounds are the best known in the literature for the said class of problems. The latter even matches the optimal lower bound for smooth nonconvex optimization with stochastic first-order algorithms. We then present numerical results on sparse principal component analysis and coordinate-independent sparse estimation to demonstrate the superior performance of our proposed method.