arXiv subjects

Erfan Yazdandoost Hamedani

Publications and source records attributed to Erfan Yazdandoost Hamedani.

At least 19 recordsLinked to original sources

arXiv2026

A Bilevel Market Equilibrium Model for Natural Gas Pipeline Network Operations: Convex Relaxations and Single-level Reformulations

We propose an integrated market equilibrium model for determining the spot market prices and flow rates in a natural gas pipeline network. The traditional approach solves the market equilibrium model separately from the natural gas network flow allocation problem in a sequential manner. Our proposed framework synthesizes marketing and operations by using a bilevel optimization formulation where the market equilibrium model in the upper level is connected and simultaneously solved with the natural gas network flow model in the lower level. We introduce convex relaxations for the nonconvex Weymouth pipe pressure drop equation and a surrogate for the compressor operating cost. Furthermore, we compare multiple single level reformulations of the bilevel problem based on Karush-Kuhn-Tucker (KKT) optimality conditions, Wolfe duality, and Lagrange duality conditions. Numerical studies on a small 6-node network demonstrates the efficacy and accuracy of our model and the comparison between different duality and optimality based reformulations.

math.OC↗

arXiv2026

Stochastic Dynamic Barrier Perturbed Gradient Methods for Nonconvex Simple Bilevel Optimization

We study stochastic simple bilevel optimization with smooth, possibly nonconvex upper- and lower-level objectives accessed only through stochastic gradient oracles. A key challenge is that the dual multiplier induced by the lower-level constraint may become unbounded near lower-level stationary points, invalidating bounded-dual analyses and destabilizing stochastic gradient estimates. To address this, we propose \emph{Stochastic Dynamic Barrier Perturbed Gradient} (SDBPG), a single-loop method that adaptively perturbs the dual formulation to regularize this degeneracy. The perturbation stabilizes the multiplier and yields controlled bias and variance even near the lower-level stationarity region. Under a rare-visit assumption governed by a parameter $δ\in (0, \tfrac{1}{2}]$, SDBPG finds an $(ε, ε)$-stationary point in $\mathcal{O}(ε^{-1/δ})$ iterations, with sample gradient complexities $\mathcal{O}(ε^{-2/δ})$ and $\mathcal{O}(ε^{-3/δ})$ for the upper- and lower-level objectives, where larger $δ$ corresponds to rarer visits to the bad region describing the negative alignment between the two objectives when the lower-level gradient is small. We further develop PR-SDBPG, a penalty-regularized variant that eliminates the rare-visit assumption, and VR-PR-SDBPG, which improves the resulting sample complexities entirely through variance reduction. To our knowledge, these are the first explicit $(ε_f,ε_g)$-stationarity guarantees for stochastic nonconvex-nonconvex simple bilevel optimization.

math.OC↗

arXiv2026

On the Analysis of Misspecified Variational Inequalities with Nonlinear Constraints

In this paper, we study a class of misspecified variational inequalities (VIs) where both the monotone operator and nonlinear convex constraints depend on an unknown parameter learned via a secondary VI. Existing data-driven VI methods typically follow a decoupled learn-then-optimize scheme, causing the approximation error from the learning to propagate the main decision-making problem and hinder convergence. We instead consider a simultaneous approach that jointly solves the main and secondary VIs. To efficiently handle nonlinear constraints with parameter misspecification, we propose a single-loop inexact Augmented Lagrangian method that simultaneously updates the primal decision variables, dual multipliers, and the misspecified parameter. The method combines a forward-reflected-backward step with an Augmented Lagrangian penalty, and explicitly handles misspecification on both the operator and constraint functions. Moreover, we introduce a relaxed performance metric based on the Minty VI gap combined with an aggregated infeasibility metric. By proving boundedness of the dual iterates, we establish $\mathcal{O}(1/K)$ ergodic convergence rates for these metrics. Numerical Experiments are provided to showcase the superior performance of our algorithm compared to state-of-the-art methods.

math.OC↗

arXiv2025

Linear Convergence of a Unified Primal--Dual Algorithm for Convex--Concave Saddle Point Problems with Quadratic Growth

In this paper, we study saddle point (SP) problems, focusing on convex-concave optimization involving functions that satisfy either two-sided quadratic functional growth (QFG) or two-sided quadratic gradient growth (QGG)--novel conditions tailored specifically for SP problems as extensions of quadratic growth conditions in minimization. These conditions relax the traditional requirement of strong convexity-strong concavity, thereby encompassing a broader class of problems. We propose a generalized accelerated primal-dual (GAPD) algorithm to solve SP problems with non-bilinear objective functions, unifying and extending existing methods. We prove that our method achieves a linear convergence rate under these relaxed conditions. Additionally, we provide examples of structured SP problems that satisfy either two-sided QFG or QGG, demonstrating the practical applicability and relevance of our approach.

math.OC↗

arXiv2025

Semi-infinite Nonconvex Constrained Min-Max Optimization

Semi-Infinite Programming (SIP) has emerged as a powerful framework for modeling problems with infinite constraints, however, its theoretical development in the context of nonconvex and large-scale optimization remains limited. In this paper, we investigate a class of nonconvex min-max optimization problems with nonconvex infinite constraints, motivated by applications such as adversarial robustness and safety-constrained learning. We propose a novel inexact dynamic barrier primal-dual algorithm and establish its convergence properties. Specifically, under the assumption that the squared infeasibility residual function satisfies the Lojasiewicz inequality with exponent $θ\in (0,1)$, we prove that the proposed method achieves $\mathcal{O}(ε^{-3})$, $\mathcal{O}(ε^{-6θ})$, and $\mathcal{O}(ε^{-3θ/(1-θ)})$ iteration complexities to achieve an $ε$-approximate stationarity, infeasibility, and complementarity slackness, respectively. Numerical experiments on robust multitask learning with task priority further illustrate the practical effectiveness of the algorithm.

math.OC↗

arXiv2025

Simultaneous Learning and Optimization via Misspecified Saddle Point Problems

We study a class of misspecified saddle point (SP) problems, where the optimization objective depends on an unknown parameter that must be learned concurrently from data. Unlike existing studies that assume parameters are fully known or pre-estimated, our framework integrates optimization and learning into a unified formulation, enabling a more flexible problem class. To address this setting, we propose two algorithms based on the accelerated primal-dual (APD) by Hamedani & Aybat 2021. In particular, we first analyze the naive extension of the APD method by directly substituting the evolving parameter estimates into the primal-dual updates; then, we design a new learning-aware variant of the APD method that explicitly accounts for parameter dynamics by adjusting the momentum updates. Both methods achieve a provable convergence rate of $\mathcal{O}(\log K / K)$, while the learning-aware approach attains a tighter $\mathcal{O}(1)$ constant and further benefits from an adaptive step-size selection enabled by a backtracking strategy. Furthermore, we extend the framework to problems where the learning problem admits multiple optimal solutions, showing that our modified algorithm for a structured setting achieves an $\mathcal{O}(1/\sqrt{K})$ rate. To demonstrate practical impact, we evaluate our methods on a misspecified portfolio optimization problem and show superior empirical performance compared to state-of-the-art algorithms.

cs.LG↗

arXiv2025

A Randomized Block-Coordinate Primal-Dual Method for Large-scale Stochastic Saddle Point Problems

We consider (stochastic) convex-concave saddle point (SP) problems with high-dimensional decision variables, arising in various applications including machine learning problems. To contend with the challenges in computing full gradients, we employ a randomized block-coordinate primal-dual scheme in which randomly selected primal and dual blocks of variables are updated. We consider both deterministic and stochastic settings, where deterministic partial gradients and their randomly sampled estimates are used, respectively, at each iteration. We investigate the convergence of the proposed method under different blocking strategies and provide the corresponding complexity results. While the best-known computational complexity result for computing a saddle point with $\varepsilon$ primal-dual gap for deterministic primal-dual methods using full gradients is $\mathcal O(\max\{m,n\}^2/\varepsilon)$, where $m$ and $n$ denote the dimensions of primal and dual variables, respectively, we show that our proposed randomized block-coordinate method achieves an improved complexity of $\mathcal O(mn/\varepsilon)$ assuming a coordinate-friendly structure on the problem. Moreover, for the stochastic setting where a mini-batch sample gradient is utilized, we show a computational complexity of $\tilde{\mathcal{O}}(m^2n^2/\varepsilon^2)$ through acceleration. Finally, almost sure convergence of the iterate sequence to a saddle point is established.

math.OC↗

arXiv2025

Sequential QCQP for Bilevel Optimization with Line Search

Bilevel optimization involves a hierarchical structure where one problem is nested within another, leading to complex interdependencies between levels. We propose a single-loop, tuning-free algorithm that guarantees anytime feasibility, i.e., approximate satisfaction of the lower-level optimality condition, while ensuring descent of the upper-level objective. At each iteration, a convex quadratically-constrained quadratic program (QCQP) with a closed-form solution yields the search direction, followed by a backtracking line search inspired by control barrier functions to ensure safe, uniformly positive step sizes. The resulting method is scalable, requires no hyperparameter tuning, and converges under mild local regularity assumptions. We establish an O(1/k) ergodic convergence rate in terms of a first-order stationary metric and demonstrate the algorithm's effectiveness on representative bilevel tasks.

math.OC↗

arXiv2025

On the Complexity of Finding Stationary Points in Nonconvex Simple Bilevel Optimization

In this paper, we study the problem of solving a simple bilevel optimization problem, where the upper-level objective is minimized over the solution set of the lower-level problem. We focus on the general setting in which both the upper- and lower-level objectives are smooth but potentially nonconvex. Due to the absence of additional structural assumptions for the lower-level objective-such as convexity or the Polyak-Łojasiewicz (PL) condition-guaranteeing global optimality is generally intractable. Instead, we introduce a suitable notion of stationarity for this class of problems and aim to design a first-order algorithm that finds such stationary points in polynomial time. Intuitively, stationarity in this setting means the upper-level objective cannot be substantially improved locally without causing a larger deterioration in the lower-level objective. To this end, we show that a simple and implementable variant of the dynamic barrier gradient descent (DBGD) framework can effectively solve the considered nonconvex simple bilevel problems up to stationarity. Specifically, to reach an $(ε_f, ε_g)$-stationary point-where $ε_f$ and $ε_g$ denote the target stationarity accuracies for the upper- and lower-level objectives, respectively-the considered method achieves a complexity of $\mathcal{O}\left(\max\left(ε_f^{-\frac{3+p}{1+p}}, ε_g^{-\frac{3+p}{2}}\right)\right)$, where $p \geq 0$ is an arbitrary constant balancing the terms. To the best of our knowledge, this is the first complexity result for a discrete-time algorithm that guarantees joint stationarity for both levels in general nonconvex simple bilevel problems.

math.OC↗

arXiv2025

Variance-reduction for Variational Inequality Problems with Bregman Distance Function

In this paper, we address variational inequalities (VI) with a finite-sum structure. We introduce a novel single-loop stochastic variance-reduced algorithm, incorporating the Bregman distance function, and establish an optimal convergence guarantee under a monotone setting. Additionally, we explore a structured class of non-monotone problems that exhibit weak Minty solutions, and analyze the complexity of our proposed method, highlighting a significant improvement over existing approaches. Numerical experiments are presented to demonstrate the performance of our algorithm compared to state-of-the-art methods

math.OC↗

arXiv2025

Perturbed Gradient Descent via Convex Quadratic Approximation for Nonconvex Bilevel Optimization

Bilevel optimization is a fundamental tool in hierarchical decision-making and has been widely applied to machine learning tasks such as hyperparameter tuning, meta-learning, and continual learning. While significant progress has been made in bilevel optimization, existing methods predominantly focus on the {nonconvex-strongly convex, or the} nonconvex-PL settings, leaving the more general nonconvex-nonconvex framework underexplored. In this paper, we address this gap by developing an efficient gradient-based method inspired by the recently proposed Relaxed Gradient Flow (RXGF) framework with a continuous-time dynamic. In particular, we introduce a discretized variant of RXGF and formulate convex quadratic program subproblems with closed-form solutions. We provide a rigorous convergence analysis, demonstrating that under the existence of a KKT point and a regularity assumption {(lower-level gradient PL assumption)}, our method achieves an iteration complexity of $\mathcal{O}(1/ε^{1.5})$ in terms of the squared norm of the KKT residual for the reformulated problem. Moreover, even in the absence of the regularity assumption, we establish an iteration complexity of $\mathcal{O}(1/ε^{3})$ for the same metric. Through extensive numerical experiments on convex and nonconvex synthetic benchmarks and a hyper-data cleaning task, we illustrate the efficiency and scalability of our approach.

math.OC↗

arXiv2025

Single-loop Projection-free and Projected Gradient-based Algorithms for Nonconvex-concave Saddle Point Problems with Bilevel Structure

In this paper, we explore a broad class of constrained saddle point problems with a bilevel structure, wherein the upper-level objective function is nonconvex-concave and smooth over compact and convex constraint sets, subject to a strongly convex lower-level objective function. This class of problems finds wide applicability in machine learning, encompassing robust multi-task learning, adversarial learning, and robust meta-learning. Our study extends the current literature in two main directions: (i) We consider a more general setting where the upper-level function is not necessarily strongly concave or linear in the maximization variable. (ii) While existing methods for solving saddle point problems with a bilevel structure are projection-based algorithms, we propose a one-sided projection-free method employing a linear minimization oracle. Specifically, by utilizing regularization and nested approximation techniques, we introduce a novel single-loop one-sided projection-free algorithm, requiring $\cO(ε^{-4})$ iterations to attain an $ε$-stationary solution, moreover, when the objective function in the upper-level is linear in the maximization component, our result improve to $\cO(ε^{-3})$. Subsequently, we develop an efficient single-loop fully projected gradient-based algorithm capable of achieving an $ε$-stationary solution within $\cO(ε^{-5})$ iterations. This result improves to $\cO(ε^{-4})$ when the upper-level objective function is strongly concave in the maximization component. Finally, we tested our proposed methods against the state-of-the-art algorithms for solving a robust multi-task regression problem to showcase the superiority of our algorithms.

math.OC↗

arXiv2025

Safe Gradient Flow for Bilevel Optimization

Bilevel optimization is a key framework in hierarchical decision-making, where one problem is embedded within the constraints of another. In this work, we propose a control-theoretic approach to solving bilevel optimization problems. Our method consists of two components: a gradient flow mechanism to minimize the upper-level objective and a safety filter to enforce the constraints imposed by the lower-level problem. Together, these components form a safe gradient flow that solves the bilevel problem in a single loop. To improve scalability with respect to the lower-level problem's dimensions, we introduce a relaxed formulation and design a compact variant of the safe gradient flow. This variant minimizes the upper-level objective while ensuring the lower-level decision variable remains within a user-defined suboptimality. Using Lyapunov analysis, we establish convergence guarantees for the dynamics, proving that they converge to a neighborhood of the optimal solution. Numerical experiments further validate the effectiveness of the proposed approaches. Our contributions provide both theoretical insights and practical tools for efficiently solving bilevel optimization problems.

math.OC↗

arXiv2024

An Accelerated Gradient Method for Convex Smooth Simple Bilevel Optimization

In this paper, we focus on simple bilevel optimization problems, where we minimize a convex smooth objective function over the optimal solution set of another convex smooth constrained optimization problem. We present a novel bilevel optimization method that locally approximates the solution set of the lower-level problem using a cutting plane approach and employs an accelerated gradient-based update to reduce the upper-level objective function over the approximated solution set. We measure the performance of our method in terms of suboptimality and infeasibility errors and provide non-asymptotic convergence guarantees for both error criteria. Specifically, when the feasible set is compact, we show that our method requires at most $\mathcal{O}(\max\{1/\sqrt{ε_{f}}, 1/ε_g\})$ iterations to find a solution that is $ε_f$-suboptimal and $ε_g$-infeasible. Moreover, under the additional assumption that the lower-level objective satisfies the $r$-th Hölderian error bound, we show that our method achieves an iteration complexity of $\mathcal{O}(\max\{ε_{f}^{-\frac{2r-1}{2r}},ε_{g}^{-\frac{2r-1}{2r}}\})$, which matches the optimal complexity of single-level convex constrained optimization when $r=1$.

math.OC↗

arXiv2024

An Inexact Conditional Gradient Method for Constrained Bilevel Optimization

Bilevel optimization is an important class of optimization problems where one optimization problem is nested within another. While various methods have emerged to address unconstrained general bilevel optimization problems, there has been a noticeable gap in research when it comes to methods tailored for the constrained scenario. The few methods that do accommodate constrained problems, often exhibit slow convergence rates or demand a high computational cost per iteration. To tackle this issue, our paper introduces a novel single-loop projection-free method employing a nested approximation technique. This innovative approach not only boasts an improved per-iteration complexity compared to existing methods but also achieves optimal convergence rate guarantees that match the best-known complexity of projection-free algorithms for solving convex constrained single-level optimization problems. In particular, when the hyper-objective function corresponding to the bilevel problem is convex, our method requires $\tilde{\mathcal{O}}(ε^{-1})$ iterations to find an $ε$-optimal solution. Moreover, when the hyper-objective function is non-convex, our method's complexity for finding an $ε$-stationary point is $\mathcal{O}(ε^{-2})$. To showcase the effectiveness of our approach, we present a series of numerical experiments that highlight its superior performance relative to state-of-the-art methods.

math.OC↗

arXiv2023

Projection-Free Methods for Solving Nonconvex-Concave Saddle Point Problems

In this paper, we investigate a class of constrained saddle point (SP) problems where the objective function is nonconvex-concave and smooth. This class of problems has wide applicability in machine learning, including robust multi-class classification and dictionary learning. Several projection-based primal-dual methods have been developed for tackling this problem; however, the availability of methods with projection-free oracles remains limited. To address this gap, we propose efficient single-loop projection-free methods reliant on first-order information. In particular, using regularization and nested approximation techniques, we propose a primal-dual conditional gradient method that solely employs linear minimization oracles to handle constraints. Assuming that the constraint set in the maximization is strongly convex, our method achieves an $ε$-stationary solution within $\mathcal{O}(ε^{-6})$ iterations. When the projection onto the constraint set of maximization is easy to compute, we propose a one-sided projection-free method that achieves an $ε$-stationary solution within $\mathcal{O}(ε^{-4})$ iterations. Moreover, we present improved iteration complexities of our methods under a strong concavity assumption. To the best of our knowledge, our proposed algorithms are among the first projection-free methods with convergence guarantees for solving nonconvex-concave SP problems.

math.OC↗

arXiv2023

Graph-Theoretic Approach for Manufacturing Cybersecurity Risk Modeling and Assessment

Identifying, analyzing, and evaluating cybersecurity risks are essential to assess the vulnerabilities of modern manufacturing infrastructures and to devise effective decision-making strategies to secure critical manufacturing against potential cyberattacks. In response, this work proposes a graph-theoretic approach for risk modeling and assessment to address the lack of quantitative cybersecurity risk assessment frameworks for smart manufacturing systems. In doing so, first, threat attributes are represented using an attack graphical model derived from manufacturing cyberattack taxonomies. Attack taxonomies offer consistent structures to categorize threat attributes, and the graphical approach helps model their interdependence. Second, the graphs are analyzed to explore how threat events can propagate through the manufacturing value chain and identify the manufacturing assets that threat actors can access and compromise during a threat event. Third, the proposed method identifies the attack path that maximizes the likelihood of success and minimizes the attack detection probability, and then computes the associated cybersecurity risk. Finally, the proposed risk modeling and assessment framework is demonstrated via an interconnected smart manufacturing system illustrative example. Using the proposed approach, practitioners can identify critical connections and manufacturing assets requiring prioritized security controls and develop and deploy appropriate defense measures accordingly.

cs.CR↗

arXiv2023

Projection-Free Methods for Stochastic Simple Bilevel Optimization with Convex Lower-level Problem

In this paper, we study a class of stochastic bilevel optimization problems, also known as stochastic simple bilevel optimization, where we minimize a smooth stochastic objective function over the optimal solution set of another stochastic convex optimization problem. We introduce novel stochastic bilevel optimization methods that locally approximate the solution set of the lower-level problem via a stochastic cutting plane, and then run a conditional gradient update with variance reduction techniques to control the error induced by using stochastic gradients. For the case that the upper-level function is convex, our method requires $\tilde{\mathcal{O}}(\max\{1/ε_f^{2},1/ε_g^{2}\}) $ stochastic oracle queries to obtain a solution that is $ε_f$-optimal for the upper-level and $ε_g$-optimal for the lower-level. This guarantee improves the previous best-known complexity of $\mathcal{O}(\max\{1/ε_f^{4},1/ε_g^{4}\})$. Moreover, for the case that the upper-level function is non-convex, our method requires at most $\tilde{\mathcal{O}}(\max\{1/ε_f^{3},1/ε_g^{3}\}) $ stochastic oracle queries to find an $(ε_f, ε_g)$-stationary point. In the finite-sum setting, we show that the number of stochastic oracle calls required by our method are $\tilde{\mathcal{O}}(\sqrt{n}/ε)$ and $\tilde{\mathcal{O}}(\sqrt{n}/ε^{2})$ for the convex and non-convex settings, respectively, where $ε=\min \{ε_f,ε_g\}$.

math.OC↗