arXiv subjects

Mohammadjavad Ebrahimi

Publications and source records attributed to Mohammadjavad Ebrahimi.

5 recordsLinked to original sources

arXiv2026

Self-tuned Regularized Federated Methods with Guarantees for Optimal Solution Selection

We study a hierarchical federated learning (FL) problem, where clients cooperatively seek to select among multiple optimal solutions of a primary distributed learning problem, a solution that minimizes a secondary loss function. This problem arises from over-parameterized learning and ill-posed optimization problems. First, we consider the setting where the inner-level objective is convex and the outer-level objective is either convex or strongly convex. We propose a self-tuned regularized federated averaging (StR-FedAvg) method where the stepsize and regularization parameter are characterized by the number of communication rounds and problem parameters. We derive new complexity guarantees for addressing the optimal solution selection problem in FL. Second, when the outer-level objective is nonconvex, we propose a two-loop FL scheme in which the outer loop employs an inexact projected first-order method and the inner loop applies StR-FedAvg with an iteratively updated regularization parameter. We derive new communication complexity guarantees for computing a stationary point of the nonconvex solution-selection problem. To our knowledge, this is the first work to establish complexity guarantees for this class of problems in FL. Preliminary experiments validate our theoretical findings.

math.OC↗

arXiv2026

On the Resolution of Stochastic MPECs over Networks: Distributed Implicit Zeroth-Order Gradient Tracking Methods

The mathematical program with equilibrium constraints (MPEC) is a powerful yet challenging class of constrained optimization problems, where the constraints are characterized by a parametrized variational inequality (VI) problem. While efficient algorithms for addressing MPECs and their stochastic variants (SMPECs) have been recently presented, distributed SMPECs over networks pose significant challenges. This work aims to develop fully iterative methods with complexity guarantees for resolving distributed SMPECs in two problem settings: (1) distributed single-stage SMPECs and (2) distributed two-stage SMPECs. In both cases, the global objective function is distributed among a network of agents that communicate cooperatively. Under the assumption that the parametrized VI is uniquely solvable, the resulting implicit problem in upper-level decisions is generally neither convex nor smooth. Under some standard assumptions, including the uniqueness of the solution to the VI problems and the Lipschitz continuity of the implicit global objective function, we propose single-stage and two-stage zeroth-order distributed gradient tracking optimization methods where the gradient of a smoothed implicit objective function is approximated using two (possibly inexact) evaluations of the lower-level VI solutions. In the exact setting of both the single-stage and two-stage problems, we achieve the best-known complexity bound for centralized nonsmooth nonconvex stochastic optimization. This complexity bound is also achieved (for the first time) for our method in addressing the inexact setting of the distributed two-stage SMPEC. In addressing the inexact setting of the single-stage problem, we derive an overall complexity bound, improving the dependence on the dimension compared to the existing results for the centralized SMPECs.

math.OC↗

arXiv2026

On Performance Guarantees for Federated Learning with Personalized Constraints

Federated learning (FL) has emerged as a communication-efficient algorithmic framework for distributed learning across multiple agents. While standard FL formulations capture unconstrained or globally constrained problems, many practical settings involve heterogeneous resource or model constraints, leading to optimization problems with agent-specific feasible sets. Here, we study a personalized constrained federated optimization problem in which each agent is associated with a convex local objective and a private constraint set. We propose PC-FedAvg, a method in which each agent maintains cross-estimates of the other agents' variables through a multi-block local decision vector. Each agent updates all blocks locally, penalizing infeasibility only in its own block. Moreover, the cross-estimate mechanism enables personalization without requiring consensus or sharing constraint information among agents. We establish communication-complexity rates of $\mathcal{O}(ε^{-2})$ for suboptimality and $\mathcal{O}(ε^{-1})$ for agent-wise infeasibility. Preliminary experiments on the MNIST and CIFAR-10 datasets validate our theoretical findings.

cs.LG↗

arXiv2024

Distributed Gradient Tracking Methods with Guarantees for Computing a Solution to Stochastic MPECs

We consider a class of hierarchical multi-agent optimization problems over networks where agents seek to compute an approximate solution to a single-stage stochastic mathematical program with equilibrium constraints (MPEC). MPECs subsume several important problem classes including Stackelberg games, bilevel programs, and traffic equilibrium problems, to name a few. Our goal in this work is to provably resolve stochastic MPECs in distributed regimes where the agents only have access to their local objectives and an inexact best-response to the lower-level equilibrium problem. To this end, we devise a new method called randomized smoothed distributed zeroth-order gradient tracking (rs-DZGT). This is a novel gradient tracking scheme where agents employ a zeroth-order implicit scheme to approximate their (unavailable) local gradients. Leveraging the properties of a randomized smoothing technique, we establish the convergence of the method and derive complexity guarantees for computing a stationary point of an optimization problem with a smoothed implicit global objective. We also provide preliminary numerical experiments where we compare the performance of rs-DZGT on networks under different settings with that of its centralized counterpart.

math.OC↗

arXiv2023

Stochastic Approximation for Estimating the Price of Stability in Stochastic Nash Games

The goal in this paper is to approximate the Price of Stability (PoS) in stochastic Nash games using stochastic approximation (SA) schemes. PoS is amongst the most popular metrics in game theory and provides an avenue for estimating the efficiency of Nash games. In particular, knowing the value of PoS can help with designing efficient networked systems, including transportation networks and power market mechanisms. Motivated by the lack of efficient methods for computing the PoS, first we consider stochastic optimization problems with a nonsmooth and merely convex objective function and a merely monotone stochastic variational inequality (SVI) constraint. This problem appears in the numerator of the PoS ratio. We develop a randomized block-coordinate stochastic extra-(sub)gradient method where we employ a novel iterative penalization scheme to account for the mapping of the SVI in each of the two gradient updates of the algorithm. We obtain an iteration complexity of the order $ε^{-4}$ that appears to be best known result for this class of constrained stochastic optimization problems, where $ε$ denotes an arbitrary bound on suitably defined infeasibility and suboptimality metrics. Second, we develop an SA-based scheme for approximating the PoS and derive lower and upper bounds on the approximation error. To validate the theoretical findings, we provide preliminary simulation results on a networked stochastic Nash Cournot competition.

math.OC↗