arXiv subjects

Uday V. Shanbhag

Publications and source records attributed to Uday V. Shanbhag.

At least 19 recordsLinked to original sources

arXiv2026

A Stochastic LineseArch Method (SLAM) for Minimizing Expectation Residuals

Most existing rate and complexity guarantees for stochastic gradient methods in $L$-smooth settings mandates that such sequences be non-adaptive, non-increasing, and upper bounded by $\tfrac{a}{L}$ for $a > 0$. This requires knowledge of $L$ and may preclude larger steps. Motivated by these shortcomings, we present an Armijo-enabled stochastic linesearch framework with common random number (CRN)-enabled stochastic zeroth- and first-order oracles. The resulting steplength sequence is non-monotone and requires neither knowledge of $L$ nor any other problem parameters. We then prove that the expected stationarity residual diminishes at a rate of $\mathcal{O}(1/\sqrt{K})$, where $K$ denotes the iteration budget. Furthermore, the resulting iteration and sample complexities for computing an $ε$-stationary point are $\mathcal{O}(ε^{-2})$ and $\mathcal{O}\left(ε^{-4}\right)$. The proposed method allows for a simple nonsmooth convex component in the objective, addressed through proximal gradient updates. Analogous guarantees are provided in the Polyak-Lojasiewicz (PL) setting and convex regimes. Preliminary numerical experiments are seen to be promising.

math.OC↗

arXiv2026

Computing Equilibria in Stochastic Nonconvex and Non-monotone Games via Gradient-Response Schemes

We consider a class of smooth $N$-player noncooperative games, where players' objectives are expectation-valued and potentially nonconvex. In such a setting, we consider the largely open question of efficiently computing a quasi-Nash equilibrium (QNE) via a single-step gradient-response framework. First, under a suitably defined quadratic growth property, we prove that both the stochastic synchronous gradient-response (\textbf{SSGR}) scheme and its asynchronous counterpart (\textbf{SAGR}) are characterized by almost sure convergence to a QNE and a sublinear rate guarantee. Second, under a quasi sharpness property, we show that the deterministic synchronous variant displays a linear rate of convergence to a QNE by leveraging a geometric decay in steplengths. This paves the way for developing a practically implementable two-stage scheme that combines sublinearly convergent schemes with a locally linearly convergent second phase. Notably, when the game admits a pseudoconvex potential function, the above convergence claims can be strengthened to a Nash equilibrium (NE), rather than merely to a QNE. Third, when player problems are convex but the associated concatenated gradient map is potentially non-monotone, we propose a stochastic asynchronous modified gradient-response (\textbf{SAMGR}) scheme which can efficiently obtain an NE under the strict copositivity condition. Collectively, our findings represent some of the first inroads into the tractable computation of QNE/NE in nonconvex settings, leading to a set of single-step schemes that are characterized by broader reach while providing rate guarantees. We present applications satisfying the prescribed requirements where preliminary empirical studies appear promising.

math.OC↗

arXiv2026

On the Resolution of Stochastic MPECs over Networks: Distributed Implicit Zeroth-Order Gradient Tracking Methods

The mathematical program with equilibrium constraints (MPEC) is a powerful yet challenging class of constrained optimization problems, where the constraints are characterized by a parametrized variational inequality (VI) problem. While efficient algorithms for addressing MPECs and their stochastic variants (SMPECs) have been recently presented, distributed SMPECs over networks pose significant challenges. This work aims to develop fully iterative methods with complexity guarantees for resolving distributed SMPECs in two problem settings: (1) distributed single-stage SMPECs and (2) distributed two-stage SMPECs. In both cases, the global objective function is distributed among a network of agents that communicate cooperatively. Under the assumption that the parametrized VI is uniquely solvable, the resulting implicit problem in upper-level decisions is generally neither convex nor smooth. Under some standard assumptions, including the uniqueness of the solution to the VI problems and the Lipschitz continuity of the implicit global objective function, we propose single-stage and two-stage zeroth-order distributed gradient tracking optimization methods where the gradient of a smoothed implicit objective function is approximated using two (possibly inexact) evaluations of the lower-level VI solutions. In the exact setting of both the single-stage and two-stage problems, we achieve the best-known complexity bound for centralized nonsmooth nonconvex stochastic optimization. This complexity bound is also achieved (for the first time) for our method in addressing the inexact setting of the distributed two-stage SMPEC. In addressing the inexact setting of the single-stage problem, we derive an overall complexity bound, improving the dependence on the dimension compared to the existing results for the centralized SMPECs.

math.OC↗

arXiv2026

Complexity Guarantees for Zeroth-order Methods via Exponentially-shifted Gaussian Smoothing: Mitigating Dimension-dependence and Incorporating Decision-dependence

In this paper, we consider two distinct challenges in the resolution of nonsmooth stochastic optimization. Of these, the first pertains to the pronounced dependence of dimension in Gaussian smoothing-enabled zeroth-order schemes, impeding applications to large-scale settings. Second, no unified analysis {exists} for smoothing-enabled stochastic zeroth-order methods, allowing for capturing standard and decision-dependent stochastic optimization. To contend with the first challenge, we introduce a new exponentially-shifted Gaussian smoothing {\bf esGs} estimator whose moment bounds enjoy a linear dependence on dimension (rather than a quadratic dependence as in standard Gaussian smoothing estimators). Second, we show that such an estimator can be extended in two distinct ways to address decision-dependent regimes where the underlying densities are either available in closed form or not. Notably, the resulting gradient estimators continue to display a linear dependence in dimension. We then develop an {\bf esGs}-enabled stochastic zeroth-order method applicable to nonsmooth strongly convex, convex, and {non-}convex regimes. Importantly, our guarantees provide improved dimension-dependence in iteration complexities (by a factor of problem dimension $n$) while maintaing similar oracle complexities. In addition, we also provide novel high probability guarantees and almost sure sublinear rate guarantees in convex settings, while a new subsequential almost sure convergence guarantee is provided in nonconvex regimes. Preliminary numerics support our theoretical findings show that the proposed schemes display improved computational times and more refined empirical accuracies.

math.OC↗

arXiv2026

Equilibrium Invariance, Proximality, and Surrogation: Moreau-Smoothed Best-Response Pathways in Stochastic Nonsmooth Games

Best-response (BR) schemes represent an important avenue for learning equilibria in noncooperative games. However, extant rate guarantees for BR schemes generally necessitate stringent smoothness requirements on player objectives and the availability of suitably defined eigenvalue bounds, significantly limiting the reach of such schemes, and few schemes if any exist for the efficient resolution of a broad class of nonsmooth and nonconvex games with expectation-valued objectives. This motivates our study of Moreau-smoothed BR schemes that allow for nonsmooth objectives. First, we consider a class of nonsmooth and strongly convex games (but potentially non-monotone) under uncertainty. By presenting an equilibrium invariance claim, we present synchronous and asynchronous schemes, equipped with linear and sublinear rate guarantees and associated complexity statements. Second, faced by weakly convex player objectives, we incorporate surrogation into the Moreau-smoothed best-response and show that the resulting smoothed quasi-Nash equilibrium (QNE) constitutes an $\mathcal{O}(η)$-QNE of the original weakly convex game, where $η> 0$ denotes the Moreau smoothing parameter. In this setting, we again present synchronous and asynchronous BR schemes, equipped with linear and sublinear rates and analogous complexity bounds. Preliminary numerics on a range of such games appear promising.

math.OC↗

arXiv2025

On the Sampling-based Computation of Nash Equilibria under Uncertainty via the Nikaido-Isoda Function

We consider the computation of an equilibrium of a stochastic Nash equilibrium problem, where the player objectives are assumed to be $L_0$-Lipschitz continuous and convex given rival decisions with convex and closed player-specific feasibility sets. To address this problem, we consider minimizing a suitably defined value function associated with the Nikaido-Isoda function. Such an avenue does not necessitate either monotonicity properties of the concatenated gradient map or potentiality requirements on the game but does require a suitable regularity requirement under which a stationary point is a Nash equilibrium. We design and analyze a sampling-enabled projected gradient descent-type method, reliant on inexact resolution of a player-level best-response subproblem. By deriving suitable Lipschitzian guarantees on the value function, we derive both asymptotic guarantees for the sequence of iterates as well as rate and complexity guarantees for computing a stationary point by appropriate choices of the sampling rate and inexactness sequence.

math.OC↗

arXiv2025

Zeroth-order Gradient and Quasi-Newton Methods for Nonsmooth Nonconvex Stochastic Optimization

We consider the minimization of a Lipschitz continuous and expectation-valued function, denoted by $f$ and defined as $f(\mathbf{x}) \triangleq \mathbb{E}[\tilde{f}(\mathbf{x}, \mathbfξ)]$, over a closed and convex set $\mathcal{X}$. We obtain asymptotics as well as rate and complexity guarantees for computing approximate Clarke-stationary points via zeroth-order schemes. We adopt an approach reliant on minimizing $f_η$ where $f_η(\mathbf{x}) \triangleq \mathbb{E}_{\mathbf{u}}\left[\mathbf{x}, f(\mathbf{x}+η\mathbf{u})\, \right]$, $\mathbf{u}$ is a random variable defined on a unit sphere, and $η> 0$. In fact, it is known that a stationary point of the $η$-smoothed problem is an $η$-stationary point for the original problem in the Clarke sense. In such a setting, we develop two schemes with promising empirical behavior. (I) We develop a variance-reduced zeroth-order gradient framework (VRG-ZO) for minimizing $f_η$ over $\mathcal{X}$. In this setting, we make two sets of contributions for the sequence generated by the proposed zeroth-order gradient scheme. (a) The residual function of the smoothed problem tends to zero almost surely along the generated sequence, guaranteeing $η$-Clarke stationary solutions of the original problem; (b) To compute an $\mathbf{x}$ such that the expected norm of the residual of the $η$-smoothed problem is within $ε$ requires no greater than $\mathcal{O}({n^{1/2}}{(L_0η^{-1} +L_0^2)} ε^{-2})$ projection steps and $\mathcal{O}({n^{3/2}(L_0^3η^{-2}+L_0^5)} ε^{-4})$ function evaluations. (II) Our second scheme is a zeroth-order stochastic quasi-Newton scheme (VRSQN-ZO) reliant on randomized and Moreau smoothing; the iteration and sample complexities are $\mathcal{O}({L_0^{4}}{n^{2}}{η^{-4}}ε^{-2})$ and $\mathcal{O}(L_0^{9} n^{5}η^{-5}ε^{-5})$, respectively.

math.OC↗

arXiv2025

Zeroth-Order Federated Methods for Stochastic MPECs and Nondifferentiable Nonconvex Hierarchical Optimization

Motivated by the emergence of federated learning (FL), we design and analyze federated methods for addressing: (i) Nondifferentiable nonconvex optimization; (ii) Bilevel optimization; (iii) Minimax problems; and (iv) Two-stage stochastic mathematical programs with equilibrium constraints (2s-SMPEC). Research on these problems has been limited and afflicted by reliance on strong assumptions, including the need for differentiability of the implicit function and the absence of constraints in the lower-level problem, among others. We make the following contributions. In (i), by leveraging convolution-based smoothing and Clarke's subdifferential calculus, we devise a randomized smoothing-enabled zeroth-order FL method and derive communication and iteration complexity guarantees for computing an approximate Clarke stationary point. To contend with (ii) and (iii), we devise a unifying randomized implicit zeroth-order FL framework, equipped with explicit communication and iteration complexities. Importantly, our method utilizes delays during local steps to skip calls to the inexact lower-level FL oracle. This results in significant reduction in communication overhead. In (iv), we devise an inexact implicit variant of the method in (i). Remarkably, this method achieves a total communication complexity matching that of single-level nonsmooth nonconvex optimization in FL. We empirically validate the theoretical findings on instances of federated nonsmooth and hierarchical problems.

math.OC↗

arXiv2025

Complexity guarantees for risk-neutral generalized Nash equilibrium problems

In this paper, we address \ac{SGNEP} seeking with risk-neutral agents. Our main contribution lies the development of a stochastic variance-reduced gradient (SVRG) technique, modified to contend with general sample spaces, within a stochastic forward-backward-forward splitting scheme for resolving structured monotone inclusion problems. This stochastic scheme is a double-loop method, in which the mini-batch gradient estimator is computed periodically in the outer loop, while only cheap sampling is required in a frequently activated inner loop, thus achieving significant speed-ups when sampling costs cannot be overlooked. The algorithm is fully distributed and it guarantees almost sure convergence under appropriate batch size and strong monotonicity assumptions. Moreover, it exhibits a linear rate with possible biased estimators, which is rather mild and imposed in many simulation-based optimization schemes. Under monotone regimes, the expectation of the gap function of an averaged iterate diminishes at a suitable sublinear rate while the sample-complexity of computing an $ε$-solution is provably $\mathcal{O}(ε^{-3})$. A numerical study on a class of networked Cournot games reflects the performance of our proposed algorithm.

math.OC↗

arXiv2024

A Distributed Iterative Tikhonov Method for Networked Monotone Aggregative Hierarchical Stochastic Games

We consider a class of nonsmooth aggregative games over networks in stochastic regimes, where each player is characterized by a composite cost function $f_i+r_i$, $f_i$ is a smooth expectation-valued function dependent on its own strategy and an aggregate function of rival strategies, and $r_i$ is a nonsmooth convex function of its strategy with an efficient prox-evaluation. We design a fully distributed iterative proximal stochastic gradient method overlaid by a Tikhonov regularization, where each player may independently choose its steplengths and regularization parameters while meeting some coordination requirements. Under a monotonicity assumption on the pseudo-gradient mapping, we prove the almost sure convergence to the least-norm Nash equilibrium. In addition, when each $r_i$ is an indicator function of a compact convex set, we establish the convergence rate associated with the expected gap function at the time-averaged sequence. We further establish high probability bounds for the gap function via both Markov's inequality as well as a more refined argument that leverages Azuma's inequality. Furthermore, we consider the extension to the private hierarchical regime where each player is a leader with respect to a collection of private followers competing in a strongly monotone game, parametrized by leader decisions. By leveraging a convolution-smoothing framework, we present amongst the first fully distributed schemes for computing a Nash equilibrium of a game complicated by such a hierarchical structure. Based on this framework, we extend the rate statements to accommodate the computation of a hierarchical stochastic Nash equilibrium by using a Fitzpatrick gap function. Finally, we validate the proposed methods on a networked Nash-Cournot equilibrium problem and a hierarchical generalization, observing that regularization has a beneficial impact on empirical behavior.

math.OC↗

arXiv2024

Variance-Reduced Accelerated First-order Methods: Central Limit Theorems and Confidence Statements

In this paper, we study a stochastic strongly convex optimization problem and propose three classes of variable sample-size stochastic first-order methods including the standard stochastic gradient descent method, its accelerated variant, and the stochastic heavy ball method. In the iterates of each scheme, the unavailable exact gradients are approximated by averaging across an increasing batch size of sampled gradients. We prove that when the sample-size increases geometrically, the generated estimates converge in mean to the optimal solution at a geometric rate. Based on this result, we provide a unified framework to show that the rescaled estimation errors converge in distribution to a normal distribution, in which the covariance matrix depends on the Hessian matrix, covariance of the gradient noise, and the steplength. If the sample-size increases at a polynomial rate, we show that the estimation errors decay at the corresponding polynomial rate and establish the corresponding central limit theorems (CLTs). Finally, we provide an avenue to construct confidence regions for the optimal solution based on the established CLTs, and test the theoretic findings on a stochastic parameter estimation problem.

math.OC↗

arXiv2024

Distributed Gradient Tracking Methods with Guarantees for Computing a Solution to Stochastic MPECs

We consider a class of hierarchical multi-agent optimization problems over networks where agents seek to compute an approximate solution to a single-stage stochastic mathematical program with equilibrium constraints (MPEC). MPECs subsume several important problem classes including Stackelberg games, bilevel programs, and traffic equilibrium problems, to name a few. Our goal in this work is to provably resolve stochastic MPECs in distributed regimes where the agents only have access to their local objectives and an inexact best-response to the lower-level equilibrium problem. To this end, we devise a new method called randomized smoothed distributed zeroth-order gradient tracking (rs-DZGT). This is a novel gradient tracking scheme where agents employ a zeroth-order implicit scheme to approximate their (unavailable) local gradients. Leveraging the properties of a randomized smoothing technique, we establish the convergence of the method and derive complexity guarantees for computing a stationary point of an optimization problem with a smoothed implicit global objective. We also provide preliminary numerical experiments where we compare the performance of rs-DZGT on networks under different settings with that of its centralized counterpart.

math.OC↗

arXiv2024

A regularized variance-reduced modified extragradient method for stochastic hierarchical games

We consider an N-player hierarchical game in which the i-th player's objective comprises of an expectation-valued term, parametrized by rival decisions, and a hierarchical term. Such a framework allows for capturing a broad range of stochastic hierarchical optimization problems, Stackelberg equilibrium problems, and leader-follower games. We develop an iteratively regularized and smoothed variance-reduced modified extragradient framework for iteratively approaching hierarchical equilibria in a stochastic setting. We equip our analysis with rate statements, complexity guarantees, and almost-sure convergence results. We then extend these statements to settings where the lower-level problem is solved inexactly and provide the corresponding rate and complexity statements. Our model framework encompasses many game theoretic equilibrium problems studied in the context of power markets. We present a realistic application to the virtual power plants, emphasizing the role of hierarchical decision making and regularization.

math.OC↗

arXiv2022

Smoothed Variable Sample-size Accelerated Proximal Methods for Nonsmooth Stochastic Convex Programs

We consider minimizing $f(x) = \mathbb{E}[f(x,ω)]$ when $f(x,ω)$ is possibly nonsmooth and either strongly convex or convex in $x$. (I) Strongly convex. When $f(x,ω)$ is $μ-$strongly convex in $x$, we propose a variable sample-size accelerated proximal scheme (VS-APM) and apply it on $f_η(x)$, the ($η$-)Moreau smoothed variant of $\mathbb{E}[f(x,ω)]$; we term such a scheme as (m-VS-APM). We consider three settings. (a) Bounded domains. In this setting, VS-APM displays linear convergence in inexact gradient steps, each of which requires utilizing an inner (SSG) scheme. Specifically, mVS-APM achieves an optimal oracle complexity in SSG steps; (b) Unbounded domains. In this regime, under a weaker assumption of suitable state-dependent bounds on subgradients, an unaccelerated variant mVS-PM is linearly convergent; (c) Smooth ill-conditioned $f$. When $f$ is $L$-smooth and $κ= L/μ\ggg 1$, we employ mVS-APM where increasingly accurate gradients $\nabla_x f_η(x)$ are obtained by VS-APM. Notably, mVS-APM displays linear convergence and near-optimal complexity in inner proximal evaluations (upto a log factor) compared to VS-APM. But, unlike a direct application of VS-APM, this scheme is characterized by larger steplengths and better empirical behavior; (II) Convex. When $f(x,ω)$ is merely convex but smoothable, by suitable choices of the smoothing, steplength, and batch-size sequences, smoothed VS-APM (or sVS-APM) produces sequences for which expected sub-optimality diminishes at the rate of $\mathcal{O}(1/k)$ with an optimal oracle complexity of $\mathcal{O}(1/ε^2)$. Finally, sVS-APM and VS-APM produce sequences that converge almost surely to a solution of the original problem.

math.OC↗

arXiv2022

Complexity guarantees for an implicit smoothing-enabled method for stochastic MPECs

Stochastic MPECs have found increasing relevance for modeling a broad range of settings in engineering and statistics. Yet, there seem to be no efficient first/zeroth-order schemes equipped with non-asymptotic rate guarantees for resolving even deterministic variants of such problems. We consider SMPECs where the parametrized lower-level equilibrium problem is given by a deterministic/stochastic VI problem whose mapping is strongly monotone. We develop a zeroth-order implicit algorithmic framework by leveraging a locally randomized spherical smoothing scheme. We present schemes for single-stage and two-stage stochastic MPECs when the upper-level problem is either convex or nonconvex. (I). Single-stage SMPECs: In convex regimes, our proposed inexact schemes are characterized by a complexity in upper-level projections, upper-level samples, and lower-level projections of $\mathcal{O}(\tfrac{1}{ε^2})$, $\mathcal{O}(\tfrac{1}{ε^2})$, and $\mathcal{O}(\tfrac{1}{ε^2}\ln(\tfrac{1}ε))$ , respectively. Analogous bounds for the nonconvex regime are $\mathcal{O}(\tfrac{1}ε)$, $\mathcal{O}(\tfrac{1}{ε^2})$, and $\mathcal{O}(\tfrac{1}{ε^3})$, respectively . (II). Two-stage SMPECs: In convex regimes, our proposed inexact schemes have a complexity in upper-level projections, upper-level samples, and lower-level projections of $\mathcal{O}(\tfrac{1}{ε^2}),\mathcal{O}(\tfrac{1}{ε^2})$, and $\mathcal{O}(\tfrac{1}{ε^2}\ln(\tfrac{1}ε))$ while the corresponding bounds in the nonconvex regime are $\mathcal{O}(\tfrac{1}ε)$, $\mathcal{O}(\tfrac{1}{ε^2})$, and $\mathcal{O}(\tfrac{1}{ε^2}\ln(\tfrac{1}ε))$ , respectively . In addition, we derive statements for exact as well as accelerated counterparts. We also provide a comprehensive set of numerical results for validating the theoretical findings.

math.OC↗

arXiv2022

Probability Maximization via Minkowski Functionals: Convex Representations and Tractable Resolution

In this paper, we consider the maximization of a probability $\mathbb{P}\{ ζ\mid ζ\in \mathbf{K}(\mathbf x)\}$ over a closed and convex set $\mathcal X$, a special case of the chance-constrained optimization problem. We define $\mathbf{K}(\mathbf x)$ as $\mathbf{K}(\mathbf x) \triangleq \{ ζ\in \mathcal{K} \mid c(\mathbf{x},ζ) \geq 0 \}$ where $ζ$ is uniformly distributed on a convex and compact set $\mathcal{K}$ and $c(\mathbf{x},ζ)$ is defined as either {$c(\mathbf{x},ζ) \triangleq 1-|ζ^T\mathbf{x}|^m$, $m\geq 0$} (Setting A) or $c(\mathbf{x},ζ) \triangleq T\mathbf{x} -ζ$ (Setting B). We show that in either setting, $\mathbb{P}\{ ζ\mid ζ\in \mathbf{K(x)}\}$ can be expressed as the expectation of a suitably defined function $F(\mathbf{x},ξ)$ with respect to an appropriately defined Gaussian density (or its variant), i.e. $\mathbb{E}_{\tilde p} [F(\mathbf x,ξ)]$. We then develop a convex representation of the original problem requiring the minimization of ${g(\mathbb{E}[F(\mathbf{x},ξ)])}$ over $\mathcal X$ where $g$ is an appropriately defined smooth convex function. Traditional stochastic approximation schemes cannot contend with the minimization of ${g(\mathbb{E}[F(\cdot,ξ)])}$ over $\mathcal X$, since conditionally unbiased sampled gradients are unavailable. We then develop a regularized variance-reduced stochastic approximation (r-VRSA) scheme that obviates the need for such unbiasedness by combining iterative regularization with variance-reduction. Notably, (r-VRSA) is characterized by both almost-sure convergence guarantees, a convergence rate of $\mathcal{O}(1/k^{1/2-a})$ in expected sub-optimality where $a > 0$, and a sample complexity of $\mathcal{O}(1/ε^{6+δ})$ where $δ> 0$.

math.OC↗

arXiv2022

On the computation of equilibria in monotone and potential stochastic hierarchical games

We consider a class of hierarchical noncooperative $N$-player games where the $i$th player solves a parametrized stochastic mathematical program with equilibrium constraints (MPEC) with the caveat that the implicit form of the $i$th player's in MPEC is convex in player strategy, given rival decisions. We develop computational schemes in two distinct regimes: (a) {\em Monotone regimes.} When player-specific implicit problems are convex, then the necessary and sufficient equilibrium conditions are given by a stochastic inclusion. Under a monotonicity assumption on the operator, we develop a variance-reduced stochastic proximal-point scheme that achieves deterministic rates of convergence in terms of solving proximal-point problems in monotone/strongly monotone regimes and the schemes are characterized by optimal or near-optimal sample-complexity guarantees. (b) {\em Potentiality.} When the implicit form of the game admits a potential function, we develop an asynchronous relaxed inexact smoothed proximal best-response framework. We consider the smoothed counterpart of this game where each player's problem is smoothed via randomized smoothing. Notably, under suitable assumptions, we show that an $η$-smoothed game admits an $η$-approximate Nash equilibrium of the original game. Our proposed scheme produces a sequence that converges almost surely to an $η$-approximate Nash equilibrium. The smoothing framework allows for developing a variance-reduced zeroth-order scheme for such problems that admits a fast rate of convergence. Numerical studies on a class of multi-leader multi-follower games suggest that variance-reduced proximal schemes provide significantly better accuracy with far lower run-times. The relaxed best-response scheme scales well will problem size and generally displays more stability than its unrelaxed counterpart.

math.OC↗

arXiv2021

Variance-Reduced Splitting Schemes for Monotone Stochastic Generalized Equations

We consider monotone inclusion problems where the operators may be expectation-valued, a class of problems that subsumes convex stochastic optimization problems as well as subclasses of stochastic variational inequality and equilibrium problems. A direct application of splitting schemes is complicated by the need to resolve problems with expectation-valued maps at each step, a concern that is addressed by using sampling. Accordingly, we propose an avenue for addressing uncertainty in the mapping: Variance-reduced stochastic modified forward-backward splitting scheme (vr-SMFBS). In constrained settings, we consider structured settings when the map can be decomposed into an expectation-valued map A and a maximal monotone map B with a tractable resolvent. We show that the proposed schemes are equipped with a.s. convergence guarantees, linear (strongly monotone A) and O(1/k) (monotone A) rates of convergence while achieving optimal oracle complexity bounds. The rate statements in monotone regimes appear to be amongst the first and rely on leveraging the Fitzpatrick gap function for monotone inclusions. Furthermore, the schemes rely on weaker moment requirements on noise and allow for weakening unbiasedness requirements on oracles in strongly monotone regimes. Preliminary numerics on a class of two-stage stochastic variational inequality problems reflect these findings and show that the variance-reduced schemes outperform stochastic approximation schemes and sample-average approximation approaches. The benefits of attaining deterministic rates of convergence become even more salient when resolvent computation is expensive.

math.OC↗