Searcharxiv⌕ Search

arXiv subjects

Zhuoyu Xiao

Publications and source records attributed to Zhuoyu Xiao.

5 recordsLinked to original sources

Potentiality-Based Smoothed Randomized Stochastic Gradient Schemes for Solving Nonconvex Nonsmooth Games under Uncertainty

We develop a potentiality-based gradient-response framework for computing Clarke-Nash equilibria (CNE) in stochastic $N$-player nonconvex nonsmooth potential games. The key observation is that under smoothness and potentiality, the concatenated vector of player-specific gradients coincides with the gradient of the potential function, thereby providing a synchronous gradient-response framework whose dynamics are subsequently analyzed. In the smooth regime, we introduce a randomized stochastic gradient (RSG) scheme and establish the optimal sample complexity $\mathcal{O}(N^{2}ε^{-4})$ for computing a point with expected residual norm at most $ε$. When the potential function is pseudoconvex, this guarantee can be strengthened to Nash equilibrium (NE) convergence. We then incorporate player-wise randomized smoothing and propose a randomized smoothed RSG (RS-RSG) scheme for Lipschitz continuous objectives, deriving complexity guarantees for the smoothed game and an approximation guarantee for CNE of the original nonsmooth game. Finally, biased variants of these schemes are developed to accommodate hierarchical settings with inexact lower-level solutions.

math.OC↗

Computing Equilibria in Stochastic Nonconvex and Non-monotone Games via Gradient-Response Schemes

We consider a class of smooth $N$-player noncooperative games, where players' objectives are expectation-valued and potentially nonconvex. In such a setting, we consider the largely open question of efficiently computing a quasi-Nash equilibrium (QNE) via a single-step gradient-response framework. First, under a suitably defined quadratic growth property, we prove that both the stochastic synchronous gradient-response (\textbf{SSGR}) scheme and its asynchronous counterpart (\textbf{SAGR}) are characterized by almost sure convergence to a QNE and a sublinear rate guarantee. Second, under a quasi sharpness property, we show that the deterministic synchronous variant displays a linear rate of convergence to a QNE by leveraging a geometric decay in steplengths. This paves the way for developing a practically implementable two-stage scheme that combines sublinearly convergent schemes with a locally linearly convergent second phase. Notably, when the game admits a pseudoconvex potential function, the above convergence claims can be strengthened to a Nash equilibrium (NE), rather than merely to a QNE. Third, when player problems are convex but the associated concatenated gradient map is potentially non-monotone, we propose a stochastic asynchronous modified gradient-response (\textbf{SAMGR}) scheme which can efficiently obtain an NE under the strict copositivity condition. Collectively, our findings represent some of the first inroads into the tractable computation of QNE/NE in nonconvex settings, leading to a set of single-step schemes that are characterized by broader reach while providing rate guarantees. We present applications satisfying the prescribed requirements where preliminary empirical studies appear promising.

math.OC↗

Equilibrium Invariance, Proximality, and Surrogation: Moreau-Smoothed Best-Response Pathways in Stochastic Nonsmooth Games

Best-response (BR) schemes represent an important avenue for learning equilibria in noncooperative games. However, extant rate guarantees for BR schemes generally necessitate stringent smoothness requirements on player objectives and the availability of suitably defined eigenvalue bounds, significantly limiting the reach of such schemes, and few schemes if any exist for the efficient resolution of a broad class of nonsmooth and nonconvex games with expectation-valued objectives. This motivates our study of Moreau-smoothed BR schemes that allow for nonsmooth objectives. First, we consider a class of nonsmooth and strongly convex games (but potentially non-monotone) under uncertainty. By presenting an equilibrium invariance claim, we present synchronous and asynchronous schemes, equipped with linear and sublinear rate guarantees and associated complexity statements. Second, faced by weakly convex player objectives, we incorporate surrogation into the Moreau-smoothed best-response and show that the resulting smoothed quasi-Nash equilibrium (QNE) constitutes an $\mathcal{O}(η)$-QNE of the original weakly convex game, where $η> 0$ denotes the Moreau smoothing parameter. In this setting, we again present synchronous and asynchronous BR schemes, equipped with linear and sublinear rates and analogous complexity bounds. Preliminary numerics on a range of such games appear promising.

math.OC↗

Optimality conditions and constraint qualifications for cardinality constrained optimization problems

The cardinality constrained optimization problem (CCOP) is an optimization problem where the maximum number of nonzero components of any feasible point is bounded. In this paper, we consider CCOP as a mathematical program with disjunctive subspaces constraints (MPDSC). Since a subspace is a special case of a convex polyhedral set, MPDSC is a special case of the mathematical program with disjunctive constraints (MPDC). Using the special structure of subspaces, we are able to obtain more precise formulas for the tangent and (directional) normal cones for the disjunctive set of subspaces. We then obtain first and second order optimality conditions by using the corresponding results from MPDC. Thanks to the special structure of the subspace, we are able to obtain some results for MPDSC that do not hold in general for MPDC. In particular we show that the relaxed constant positive linear dependence (RCPLD) is a sufficient condition for the metric subregularity/error bound property for MPDSC which is not true for MPDC in general. Finally we show that under all constraint qualifications presented in this paper, certain exact penalization holds for CCOP.

math.OC↗

Generalized Karush-Kuhn-Tucker Conditions in Variational and Set-Valued Analysis

This expository paper contains a concise introduction to some significant works concerning the Karush-Kuhn-Tucker condition, a necessary condition for a solution in local optimality in problems with equality and inequality constraints. The study of this optimality condition has a long history and culminated in the appearance of subdifferentials. The 1970s and early 1980s were important periods for new developments and various generalizations of subdifferentials were introduced, including the Clarke subdifferential and Demyanov-Rubinov quasidifferential. In this paper, we mainly present four generalized Karush-Kuhn-Tucker conditions or Fritz John conditions in variational analysis and set-valued analysis via Lagrange multiplier methods besides Fr$\Acute{e}$chet differentiable situation, namely subdifferentials of convex functions, generalized gradients of locally Lipschitz functions, quasidifferentials of quasidifferentiable functions and contingent epiderivatives of set-valued maps and discuss the limits of Lagrangian methods slightly in the last chapter. These results represent remarkable developments in the theory of generalized differentiation. The purpose of this paper is to use Karush-Kuhn-Tucker condition as a guide to provide our readers with some advanced topics in modern nonlinear analysis.

math.OC↗