arXiv subjects

Afrooz Jalilzadeh

Publications and source records attributed to Afrooz Jalilzadeh.

At least 19 recordsLinked to original sources

arXiv2026

Regularized Projection Algorithms for Monotone Inverse Variational Inequalities

Stochastic inverse variational inequalities (SIVIs) arise in applications in which equilibrium responses are observed under uncertainty, such as inverse road pricing and network equilibrium control. Existing methods typically rely on co-coercivity or strong monotonicity, while general monotone SIVIs remain less understood. We propose a regularized projection algorithm that combines Tikhonov regularization with increasing batch sizes. Under monotonicity and Lipschitz continuity, we prove almost sure boundedness of the iterates and almost sure convergence of their distance to the SIVI solution set. We further establish, to the best of our knowledge, the first explicit nonasymptotic rate of $O(T^{-1/2})$ for the expected squared residual under general monotonicity. This yields $O(ε^{-2})$ iterations and $O(ε^{-4-2δ})$ stochastic oracle calls, for any $δ>0$, to obtain an $ε$-solution in expected squared residual. A deterministic variant attains the same iteration complexity using $O(ε^{-2})$ exact operator evaluations. Numerical experiments illustrate the proposed methods on monotone SIVI problems.

math.OC↗

arXiv2026

Riemannian Inexact Gradient Descent for Quadratic Discrimination

We propose an inexact optimization algorithm on Riemannian manifolds, motivated by quadratic discrimination tasks in high-dimensional, low-sample-size (HDLSS) imaging settings. In such applications, gradient evaluations are often biased due to limited sample sizes. To address this, we introduce a novel Riemannian optimization algorithm that is robust to inexact gradient information and prove an $\mathcal O(1/K)$ convergence rate under standard assumptions. We also present a line search variant that requires access to function values but not exact gradients, maintaining the same convergence rate and ensuring sufficient descent. The algorithm is tailored to the Grassmann manifold by leveraging its geometric structure, and its convergence rate is validated numerically. A simulation of heteroscedastic images shows that when bias is introduced into the problem, both intentionally and through estimation of the covariance matrix, the detection performance of the algorithm solution is comparable to when true gradients are used in the optimization. The optimal subspace learned via the algorithm encodes interpretable patterns and shows qualitative similarity to known optimal solutions. By ensuring robust convergence and interpretability, our algorithm offers a tool for manifold-based dimensionality reduction in the presence of inexact gradients due to sample statistics.

math.OC↗

arXiv2026

On the Analysis of Misspecified Variational Inequalities with Nonlinear Constraints

In this paper, we study a class of misspecified variational inequalities (VIs) where both the monotone operator and nonlinear convex constraints depend on an unknown parameter learned via a secondary VI. Existing data-driven VI methods typically follow a decoupled learn-then-optimize scheme, causing the approximation error from the learning to propagate the main decision-making problem and hinder convergence. We instead consider a simultaneous approach that jointly solves the main and secondary VIs. To efficiently handle nonlinear constraints with parameter misspecification, we propose a single-loop inexact Augmented Lagrangian method that simultaneously updates the primal decision variables, dual multipliers, and the misspecified parameter. The method combines a forward-reflected-backward step with an Augmented Lagrangian penalty, and explicitly handles misspecification on both the operator and constraint functions. Moreover, we introduce a relaxed performance metric based on the Minty VI gap combined with an aggregated infeasibility metric. By proving boundedness of the dual iterates, we establish $\mathcal{O}(1/K)$ ergodic convergence rates for these metrics. Numerical Experiments are provided to showcase the superior performance of our algorithm compared to state-of-the-art methods.

math.OC↗

arXiv2025

Convergence Analysis of Non-Strongly-Monotone Stochastic Quasi-Variational Inequalities

While Variational Inequality (VI) is a well-established mathematical framework that subsumes Nash equilibrium and saddle-point problems, less is known about its extension, Quasi-Variational Inequalities (QVI). QVI allows for cases where the constraint set changes as the decision variable varies allowing for a more versatile setting. In this paper, we propose extra-gradient and gradient-based methods for solving a class of monotone Stochastic Quasi-Variational Inequalities (SQVI) and establish a rigorous convergence rate analysis for these methods. Our approach not only advances the theoretical understanding of SQVI but also demonstrates its practical applicability. Specifically, we highlight its effectiveness in reformulating and solving problems such as generalized Nash Equilibrium, bilevel optimization, and saddle-point problems with coupling constraints.

math.OC↗

arXiv2025

Distributionally Robust Nash Equilibria via Variational Inequalities

Nash Equilibrium and its robust counterpart, Distributionally Robust Nash Equilibrium (DRNE), are fundamental problems in game theory with applications in economics, engineering, and machine learning. This paper addresses the problem of DRNE, where multiple players engage in a noncooperative game under uncertainty. Each player aims to minimize their objective against the worst-case distribution within an ambiguity set, resulting in a minimax structure. We reformulate the DRNE problem as a Variational Inequality (VI) problem, providing a unified framework for analysis and algorithm development. We propose a gradient descent-ascent type algorithm with convergence guarantee that effectively addresses the computational challenges of high-dimensional and nonsmooth objectives.

math.OC↗

arXiv2025

Linear Convergence of a Unified Primal--Dual Algorithm for Convex--Concave Saddle Point Problems with Quadratic Growth

In this paper, we study saddle point (SP) problems, focusing on convex-concave optimization involving functions that satisfy either two-sided quadratic functional growth (QFG) or two-sided quadratic gradient growth (QGG)--novel conditions tailored specifically for SP problems as extensions of quadratic growth conditions in minimization. These conditions relax the traditional requirement of strong convexity-strong concavity, thereby encompassing a broader class of problems. We propose a generalized accelerated primal-dual (GAPD) algorithm to solve SP problems with non-bilinear objective functions, unifying and extending existing methods. We prove that our method achieves a linear convergence rate under these relaxed conditions. Additionally, we provide examples of structured SP problems that satisfy either two-sided QFG or QGG, demonstrating the practical applicability and relevance of our approach.

math.OC↗

arXiv2025

Semi-infinite Nonconvex Constrained Min-Max Optimization

Semi-Infinite Programming (SIP) has emerged as a powerful framework for modeling problems with infinite constraints, however, its theoretical development in the context of nonconvex and large-scale optimization remains limited. In this paper, we investigate a class of nonconvex min-max optimization problems with nonconvex infinite constraints, motivated by applications such as adversarial robustness and safety-constrained learning. We propose a novel inexact dynamic barrier primal-dual algorithm and establish its convergence properties. Specifically, under the assumption that the squared infeasibility residual function satisfies the Lojasiewicz inequality with exponent $θ\in (0,1)$, we prove that the proposed method achieves $\mathcal{O}(ε^{-3})$, $\mathcal{O}(ε^{-6θ})$, and $\mathcal{O}(ε^{-3θ/(1-θ)})$ iteration complexities to achieve an $ε$-approximate stationarity, infeasibility, and complementarity slackness, respectively. Numerical experiments on robust multitask learning with task priority further illustrate the practical effectiveness of the algorithm.

math.OC↗

arXiv2025

A Randomized Block-Coordinate Primal-Dual Method for Large-scale Stochastic Saddle Point Problems

We consider (stochastic) convex-concave saddle point (SP) problems with high-dimensional decision variables, arising in various applications including machine learning problems. To contend with the challenges in computing full gradients, we employ a randomized block-coordinate primal-dual scheme in which randomly selected primal and dual blocks of variables are updated. We consider both deterministic and stochastic settings, where deterministic partial gradients and their randomly sampled estimates are used, respectively, at each iteration. We investigate the convergence of the proposed method under different blocking strategies and provide the corresponding complexity results. While the best-known computational complexity result for computing a saddle point with $\varepsilon$ primal-dual gap for deterministic primal-dual methods using full gradients is $\mathcal O(\max\{m,n\}^2/\varepsilon)$, where $m$ and $n$ denote the dimensions of primal and dual variables, respectively, we show that our proposed randomized block-coordinate method achieves an improved complexity of $\mathcal O(mn/\varepsilon)$ assuming a coordinate-friendly structure on the problem. Moreover, for the stochastic setting where a mini-batch sample gradient is utilized, we show a computational complexity of $\tilde{\mathcal{O}}(m^2n^2/\varepsilon^2)$ through acceleration. Finally, almost sure convergence of the iterate sequence to a saddle point is established.

math.OC↗

arXiv2025

Variance-reduction for Variational Inequality Problems with Bregman Distance Function

In this paper, we address variational inequalities (VI) with a finite-sum structure. We introduce a novel single-loop stochastic variance-reduced algorithm, incorporating the Bregman distance function, and establish an optimal convergence guarantee under a monotone setting. Additionally, we explore a structured class of non-monotone problems that exhibit weak Minty solutions, and analyze the complexity of our proposed method, highlighting a significant improvement over existing approaches. Numerical experiments are presented to demonstrate the performance of our algorithm compared to state-of-the-art methods

math.OC↗

arXiv2023

A Projection-Based Algorithm for Solving Stochastic Inverse Variational Inequality Problems

We consider a stochastic Inverse Variational Inequality (IVI) problem defined by a continuous and co-coercive map over a closed and convex set. Motivated by the absence of performance guarantees for stochastic IVI, we present a variance-reduced projection-based gradient method. Our proposed method ensures an almost sure convergence of the generated iterates to the solution, and we establish a convergence rate guarantee. To verify our results, we apply the proposed algorithm to a network equilibrium control problem.

math.OC↗

arXiv2023

Projection-Free Methods for Solving Nonconvex-Concave Saddle Point Problems

In this paper, we investigate a class of constrained saddle point (SP) problems where the objective function is nonconvex-concave and smooth. This class of problems has wide applicability in machine learning, including robust multi-class classification and dictionary learning. Several projection-based primal-dual methods have been developed for tackling this problem; however, the availability of methods with projection-free oracles remains limited. To address this gap, we propose efficient single-loop projection-free methods reliant on first-order information. In particular, using regularization and nested approximation techniques, we propose a primal-dual conditional gradient method that solely employs linear minimization oracles to handle constraints. Assuming that the constraint set in the maximization is strongly convex, our method achieves an $ε$-stationary solution within $\mathcal{O}(ε^{-6})$ iterations. When the projection onto the constraint set of maximization is easy to compute, we propose a one-sided projection-free method that achieves an $ε$-stationary solution within $\mathcal{O}(ε^{-4})$ iterations. Moreover, we present improved iteration complexities of our methods under a strong concavity assumption. To the best of our knowledge, our proposed algorithms are among the first projection-free methods with convergence guarantees for solving nonconvex-concave SP problems.

math.OC↗

arXiv2023

Stochastic Approximation for Estimating the Price of Stability in Stochastic Nash Games

The goal in this paper is to approximate the Price of Stability (PoS) in stochastic Nash games using stochastic approximation (SA) schemes. PoS is amongst the most popular metrics in game theory and provides an avenue for estimating the efficiency of Nash games. In particular, knowing the value of PoS can help with designing efficient networked systems, including transportation networks and power market mechanisms. Motivated by the lack of efficient methods for computing the PoS, first we consider stochastic optimization problems with a nonsmooth and merely convex objective function and a merely monotone stochastic variational inequality (SVI) constraint. This problem appears in the numerator of the PoS ratio. We develop a randomized block-coordinate stochastic extra-(sub)gradient method where we employ a novel iterative penalization scheme to account for the mapping of the SVI in each of the two gradient updates of the algorithm. We obtain an iteration complexity of the order $ε^{-4}$ that appears to be best known result for this class of constrained stochastic optimization problems, where $ε$ denotes an arbitrary bound on suitably defined infeasibility and suboptimality metrics. Second, we develop an SA-based scheme for approximating the PoS and derive lower and upper bounds on the approximation error. To validate the theoretical findings, we provide preliminary simulation results on a networked stochastic Nash Cournot competition.

math.OC↗

arXiv2023

Accelerated Primal-dual Scheme for a Class of Stochastic Nonconvex-concave Saddle Point Problems

Stochastic nonconvex-concave min-max saddle point problems appear in many machine learning and control problems including distributionally robust optimization, generative adversarial networks, and adversarial learning. In this paper, we consider a class of nonconvex saddle point problems where the objective function satisfies the Polyak-Łojasiewicz condition with respect to the minimization variable and it is concave with respect to the maximization variable. The existing methods for solving nonconvex-concave saddle point problems often suffer from slow convergence and/or contain multiple loops. Our main contribution lies in proposing a novel single-loop accelerated primal-dual algorithm with new convergence rate results appearing for the first time in the literature, to the best of our knowledge. In particular, in the stochastic regime, we demonstrate a convergence rate of $\mathcal O(ε^{-4})$ to find an $ε$-gap solution which can be improved to $\mathcal O(ε^{-2})$ in deterministic setting.

math.OC↗

arXiv2023

Randomized Lagrangian Stochastic Approximation for Large-Scale Constrained Stochastic Nash Games

In this paper, we consider stochastic monotone Nash games where each player's strategy set is characterized by possibly a large number of explicit convex constraint inequalities. Notably, the functional constraints of each player may depend on the strategies of other players, allowing for capturing a subclass of generalized Nash equilibrium problems (GNEP). While there is limited work that provide guarantees for this class of stochastic GNEPs, even when the functional constraints of the players are independent of each other, the majority of the existing methods rely on employing projected stochastic approximation (SA) methods. However, the projected SA methods perform poorly when the constraint set is afflicted by the presence of a large number of possibly nonlinear functional inequalities. Motivated by the absence of performance guarantees for computing the Nash equilibrium in constrained stochastic monotone Nash games, we develop a single timescale randomized Lagrangian multiplier stochastic approximation method where in the primal space, we employ an SA scheme, and in the dual space, we employ a randomized block-coordinate scheme where only a randomly selected Lagrangian multiplier is updated. We show that our method achieves a convergence rate of $\mathcal{O}\left(\frac{\log(k)}{\sqrt{k}}\right)$ for suitably defined suboptimality and infeasibility metrics in a mean sense.

math.OC↗

arXiv2023

An Accelerated Asynchronous Distributed Method for Convex Constrained Optimization Problems

We consider a class of multi-agent cooperative consensus optimization problems with local nonlinear convex constraints where only those agents connected by an edge can directly communicate, hence, the optimal consensus decision lies in the intersection of these private sets. We develop an asynchronous distributed accelerated primal-dual algorithm to solve the considered problem. The proposed scheme is the first asynchronous method with an optimal convergence guarantee for this class of problems, to the best of our knowledge. In particular, we provide an optimal convergence rate of $\mathcal{O(1/K)}$ for suboptimality, infeasibility, and consensus violation.

math.OC↗

arXiv2022

Smoothed Variable Sample-size Accelerated Proximal Methods for Nonsmooth Stochastic Convex Programs

We consider minimizing $f(x) = \mathbb{E}[f(x,ω)]$ when $f(x,ω)$ is possibly nonsmooth and either strongly convex or convex in $x$. (I) Strongly convex. When $f(x,ω)$ is $μ-$strongly convex in $x$, we propose a variable sample-size accelerated proximal scheme (VS-APM) and apply it on $f_η(x)$, the ($η$-)Moreau smoothed variant of $\mathbb{E}[f(x,ω)]$; we term such a scheme as (m-VS-APM). We consider three settings. (a) Bounded domains. In this setting, VS-APM displays linear convergence in inexact gradient steps, each of which requires utilizing an inner (SSG) scheme. Specifically, mVS-APM achieves an optimal oracle complexity in SSG steps; (b) Unbounded domains. In this regime, under a weaker assumption of suitable state-dependent bounds on subgradients, an unaccelerated variant mVS-PM is linearly convergent; (c) Smooth ill-conditioned $f$. When $f$ is $L$-smooth and $κ= L/μ\ggg 1$, we employ mVS-APM where increasingly accurate gradients $\nabla_x f_η(x)$ are obtained by VS-APM. Notably, mVS-APM displays linear convergence and near-optimal complexity in inner proximal evaluations (upto a log factor) compared to VS-APM. But, unlike a direct application of VS-APM, this scheme is characterized by larger steplengths and better empirical behavior; (II) Convex. When $f(x,ω)$ is merely convex but smoothable, by suitable choices of the smoothing, steplength, and batch-size sequences, smoothed VS-APM (or sVS-APM) produces sequences for which expected sub-optimality diminishes at the rate of $\mathcal{O}(1/k)$ with an optimal oracle complexity of $\mathcal{O}(1/ε^2)$. Finally, sVS-APM and VS-APM produce sequences that converge almost surely to a solution of the original problem.

math.OC↗

arXiv2022

An Inexact Variance-Reduced Method For Stochastic Quasi-Variational Inequality Problems With An Application In Healthcare

This paper is focused on a stochastic quasi-variational inequality (SQVI) problem with a continuous and strongly-monotone mapping over a closed and convex set where the projection onto the constraint set may not be easy to compute. We present an inexact variance reduced stochastic scheme to solve SQVI problems and analyzed its convergence rate and oracle complexity. A linear rate of convergence is obtained by progressively increasing sample-size and approximating the projection operator. Moreover, we show how a competition among blood donation organizations can be modeled as an SQVI and we provide some preliminary simulation results to validate our findings.

math.OC↗

arXiv2022

Probability Maximization via Minkowski Functionals: Convex Representations and Tractable Resolution

In this paper, we consider the maximization of a probability $\mathbb{P}\{ ζ\mid ζ\in \mathbf{K}(\mathbf x)\}$ over a closed and convex set $\mathcal X$, a special case of the chance-constrained optimization problem. We define $\mathbf{K}(\mathbf x)$ as $\mathbf{K}(\mathbf x) \triangleq \{ ζ\in \mathcal{K} \mid c(\mathbf{x},ζ) \geq 0 \}$ where $ζ$ is uniformly distributed on a convex and compact set $\mathcal{K}$ and $c(\mathbf{x},ζ)$ is defined as either {$c(\mathbf{x},ζ) \triangleq 1-|ζ^T\mathbf{x}|^m$, $m\geq 0$} (Setting A) or $c(\mathbf{x},ζ) \triangleq T\mathbf{x} -ζ$ (Setting B). We show that in either setting, $\mathbb{P}\{ ζ\mid ζ\in \mathbf{K(x)}\}$ can be expressed as the expectation of a suitably defined function $F(\mathbf{x},ξ)$ with respect to an appropriately defined Gaussian density (or its variant), i.e. $\mathbb{E}_{\tilde p} [F(\mathbf x,ξ)]$. We then develop a convex representation of the original problem requiring the minimization of ${g(\mathbb{E}[F(\mathbf{x},ξ)])}$ over $\mathcal X$ where $g$ is an appropriately defined smooth convex function. Traditional stochastic approximation schemes cannot contend with the minimization of ${g(\mathbb{E}[F(\cdot,ξ)])}$ over $\mathcal X$, since conditionally unbiased sampled gradients are unavailable. We then develop a regularized variance-reduced stochastic approximation (r-VRSA) scheme that obviates the need for such unbiasedness by combining iterative regularization with variance-reduction. Notably, (r-VRSA) is characterized by both almost-sure convergence guarantees, a convergence rate of $\mathcal{O}(1/k^{1/2-a})$ in expected sub-optimality where $a > 0$, and a sample complexity of $\mathcal{O}(1/ε^{6+δ})$ where $δ> 0$.

math.OC↗