arXiv subjects

Sepideh Samadi

Publications and source records attributed to Sepideh Samadi.

4 recordsLinked to original sources

arXiv2025

On iteratively regularized first-order methods for simple bilevel optimization

We consider simple bilevel optimization (SBO) problems where the goal is to compute among the optimal solutions of a composite convex optimization problem, one that minimizes a secondary objective function. Our main contribution is threefold. (i) When the upper-level objective is composite and strongly convex, we propose IR-ISTAs, an iteratively regularized proximal gradient method with a prescribed update rule for the regularization parameter. We establish asymptotic convergence of the iterates to the unique optimal solution and simultaneous sublinear rates for suitably defined infeasibility and suboptimality error metrics. (ii) For the same setting, we propose IR-VFISTAs, an iteratively regularized accelerated proximal gradient method, establish its asymptotic convergence, and, under weak sharp minimality, derive faster simultaneous rates than IR-ISTAs. These appear to be the best-known convergence rate guarantees for SBO problems with a strongly convex upper-level objective and improve upon the rates previously established for methods requiring convexity of both levels. (iii) When the upper-level objective is smooth and nonconvex, we propose IPR-VFISTAnc, an inexactly projected iteratively regularized accelerated gradient method, and establish both asymptotic and nonasymptotic convergence guarantees. To the best of our knowledge, this is the first asymptotic stationarity result for this class of SBO problems that does not rely on the weak sharp minimality of the lower-level problem. Moreover, the total iteration complexity of IPR-VFISTAnc matches that of existing methods, while providing a sharper lower-level infeasibility guarantee. We present preliminary numerical experiments on three ill-posed linear inverse problems and an optimal classifier-selection problem.

math.OC↗

arXiv2023

Achieving optimal complexity guarantees for a class of bilevel convex optimization problems

We design and analyze a novel accelerated gradient-based algorithm for a class of bilevel optimization problems. These problems have various applications arising from machine learning and image processing, where optimal solutions of the two levels are interdependent. That is, achieving the optimal solution of an upper-level problem depends on the solution set of a lower-level optimization problem. We significantly improve existing iteration complexity to $\mathcal{O}(\epsilon^{-0.5})$ for both suboptimality and infeasibility error metrics, where $\epsilon>0$ denotes an arbitrary scalar. In addition, contrary to existing methods that require solving the optimization problem sequentially (initially solving an optimization problem to approximate the solution of the lower-level problem followed by a second algorithm), our algorithm concurrently solves the optimization problem. To the best of our knowledge, the proposed algorithm has the fastest known iteration complexity, which matches the optimal complexity for single-level optimization. We conduct numerical experiments on sparse linear regression problems to demonstrate the efficacy of our approach.

math.OC↗

arXiv2023

Improved guarantees for optimal Nash equilibrium seeking and bilevel variational inequalities

We consider a class of hierarchical variational inequality (VI) problems that subsumes VI-constrained optimization and several other problem classes including the optimal solution selection problem and the optimal Nash equilibrium (NE) seeking problem. Our main contributions are threefold. (i) We consider bilevel VIs with monotone and Lipschitz continuous mappings and devise a single-timescale iteratively regularized extragradient method, named IR-EG$_{{\texttt{m,m}}}$. We improve the existing iteration complexity results for addressing both bilevel VI and VI-constrained convex optimization problems. (ii) Under the strong monotonicity of the outer level mapping, we develop a method named IR-EG$_{{\texttt{s,m}}}$ and derive faster guarantees than those in (i). We also study the iteration complexity of this method under a constant regularization parameter. These results appear to be new for both bilevel VIs and VI-constrained optimization. (iii) To our knowledge, complexity guarantees for computing the optimal NE in nonconvex settings do not exist. Motivated by this lacuna, we consider VI-constrained nonconvex optimization problems and devise an inexactly-projected gradient method, named IPR-EG, where the projection onto the unknown set of equilibria is performed using IR-EG$_{{\texttt{s,m}}}$ with a prescribed termination criterion and an adaptive regularization parameter. We obtain new complexity guarantees in terms of a residual map and an infeasibility metric for computing a stationary point. We validate the theoretical findings using preliminary numerical experiments for computing the best and the worst Nash equilibria.

math.OC↗

arXiv2021

An Incremental Gradient Method for Optimization Problems with Variational Inequality Constraints

We consider minimizing a sum of agent-specific nondifferentiable merely convex functions over the solution set of a variational inequality (VI) problem in that each agent is associated with a local monotone mapping. This problem finds an application in computation of the best equilibrium in nonlinear complementarity problems arising in transportation networks. We develop an iteratively regularized incremental gradient method where at each iteration, agents communicate over a cycle graph to update their solution iterates using their local information about the objective and the mapping. The proposed method is single-timescale in the sense that it does not involve any excessive hard-to-project computation per iteration. We derive non-asymptotic agent-wise convergence rates for the suboptimality of the global objective function and infeasibility of the VI constraints measured by a suitably defined dual gap function. The proposed method appears to be the first fully iterative scheme equipped with iteration complexity that can address distributed optimization problems with VI constraints over cycle graphs. Preliminary numerical experiments for a transportation network problem and a support vector machine model are presented.

math.OC↗