Adaptive Schauder Stochastic Mirror Descent in Banach Spaces

Abstract

In this paper, we extend stochastic mirror descent (SMD) to infinite-dimensional Banach spaces for solving a class of risk functional minimization problems, where stochastic gradient information is only available through sampling. We first choose the Bregman distance according to the uniform convexity properties of the Banach space. For the non-uniformly convex $\mathcal{L}^1$ space, we instead construct a Bregman distance induced by the entropy function. Based on a Schauder basis of the Banach space, we introduce a family of finite-dimensional subspaces that adapt to the sample size $n$. At each SMD iteration, we restrict the subproblem to the corresponding finite-dimensional subspace and project the stochastic gradient onto the associated finite-dimensional dual space, thereby introducing an adaptive regularization in Banach spaces. This regularization strategy allows the SMD subproblem to be solved efficiently. By developing a new analytical framework, we prove that the proposed algorithm achieves a convergence rate of $\mathcal{O}\left(n^{-1/p_1}\right)$ up to logarithmic factors, where $p_1\geq 2$ is determined by the convexity properties of the underlying space. In the misspecified setting, where the minimizer satisfies only weaker regularity conditions, we further show that the risk functional still converges to its minimum value. Moreover, processing $n$ samples requires only $\mathcal{O}(n^{1+\theta})$ computational time and $\mathcal{O}(n^\theta)$ memory, where $\theta>0$ can be chosen arbitrarily small when the minimizer has sufficient regularity. We further apply the algorithm to solve statistical inverse problems and validate its effectiveness in numerical experiments.