arXiv subjects

Jong-Shi Pang

Publications and source records attributed to Jong-Shi Pang.

34 records · Page 2Linked to original sources

arXiv2018

Exact Penalization of Generalized Nash Equilibrium Problems

This paper presents an exact penalization theory of the generalized Nash equilibrium problem (GNEP) that has its origin from the renowned Arrow-Debreu general economic equilibrium model. While the latter model is the foundation of much of mathematical economics, the GNEP provides a mathematical model of multi-agent non-cooperative competition that has found many contemporary applications in diverse engineering domains. The most salient feature of the GNEP that distinguishes it from a standard non-cooperative (Nash) game is that each player's optimization problem contains constraints that couple all players' decision variables. Extending results for stand-alone optimization problems, the penalization theory aims to convert the GNEP into a game of the standard kind without the coupled constraints, which is known to be more readily amenable to solution methods and analysis. Starting with an illustrative example to motivate the development, the paper focuses on two kinds of coupled constraints, shared (i.e., common) and finitely representable. Constraint residual functions and the associated error bound theory play an important role throughout the development.

cs.GT↗

arXiv2018

Composite Difference-Max Programs for Modern Statistical Estimation Problems

Many modern statistical estimation problems are defined by three major components: a statistical model that postulates the dependence of an output variable on the input features; a loss function measuring the error between the observed output and the model predicted output; and a regularizer that controls the overfitting and/or variable selection in the model. We study the sampling version of this generic statistical estimation problem where the model parameters are estimated by empirical risk minimization, which involves the minimization of the empirical average of the loss function at the data points weighted by the model regularizer. In our setup we allow all three component functions discussed above to be of the difference-of-convex (dc) type and illustrate them with a host of commonly used examples, including those in continuous piecewise affine regression and in deep learning (where the activation functions are piecewise affine). We describe a nonmonotone majorization-minimization (MM) algorithm for solving the unified nonconvex, nondifferentiable optimization problem which is formulated as a specially structured composite dc program of the pointwise max type, and present convergence results to a directional stationary solution. An efficient semismooth Newton method is proposed to solve the dual of the MM subproblems. Numerical results are presented to demonstrate the effectiveness of the proposed algorithm and the superiority of continuous piecewise affine regression over the standard linear model.

math.OC↗

arXiv2018

A Study of Piecewise Linear-Quadratic Programs

Motivated by a growing list of nontraditional statistical estimation problems of the piecewise kind, this paper provides a survey of known results supplemented with new results for the class of piecewise linear-quadratic programs. These are linearly constrained optimization problems with piecewise linear-quadratic (PLQ) objective functions. Starting from a study of the representation of such a function in terms of a family of elementary functions consisting of squared affine functions, squared plus-composite-affine functions, and affine functions themselves, we summarize some local properties of a PLQ function in terms of their first and second-order directional derivatives. We extend some well-known necessary and sufficient second-order conditions for local optimality of a quadratic program to a PLQ program and provide a dozen such equivalent conditions for strong, strict, and isolated local optimality, showing in particular that a PLQ program has the same characterizations for local minimality as a standard quadratic program. As a consequence of one such condition, we show that the number of strong, strict, or isolated local minima of a PLQ program is finite; this result supplements a recent result about the finite number of directional stationary objective values. Interestingly, these finiteness results can be uncovered by invoking a very powerful property of subanalytic functions; our proof is fairly elementary, however. We discuss applications of PLQ programs in some modern statistical estimation problems. These problems lead to a special class of unconstrained composite programs involving the non-differentiable $\ell_1$-function, for which we show that the task of verifying the second-order stationary condition can be converted to the problem of checking the copositivity of certain Schur complement on the nonnegative orthant.

math.OC↗

arXiv2018

On the Finite Number of Directional Stationary Values of Piecewise Programs

Extending a fundamental result for (indefinite) quadratic programs, this paper shows that certain non-convex piecewise programs have only a finite number of directional stationary values, and thus, possess only finitely many locally minimum values. We present various special cases of our main results, in particular, an application to a least-squares piecewise affine regression problem for which every directional stationary point is locally minimizing.

math.OC↗

arXiv2018

Solving Linear Programs with Complementarity Constraints using Branch-and-Cut

A linear program with linear complementarity constraints (LPCC) requires the minimization of a linear objective over a set of linear constraints together with additional linear complementarity constraints. This class has emerged as a modeling paradigm for a broad collection of problems, including bilevel programs, Stackelberg games, inverse quadratic programs, and problems involving equilibrium constraints. The presence of the complementarity constraints results in a nonconvex optimization problem. We develop a branch-and-cut algorithm to find a global optimum for this class of optimization problems, where we branch directly on complementarities. We develop branching rules and feasibility recovery procedures and demonstrate their computational effectiveness in a comparison with CPLEX. The implementation builds on CPLEX through the use of callback routines. The computational results show that our approach is a strong alternative to constructing an integer programming formulation using big-$M$ terms to represent bounds for variables, with testing conducted on general LPCCs as well as on instances generated from bilevel programs with convex quadratic lower level problems.

math.OC↗

arXiv2018

On Synchronous, Asynchronous, and Randomized Best-Response schemes for computing equilibria in Stochastic Nash games

This work considers a stochastic Nash game in which each player solves a parameterized stochastic optimization problem. In deterministic regimes, best-response schemes have been shown to be convergent under a suitable spectral property associated with the proximal best-response map. However, a direct application of this scheme to stochastic settings requires obtaining exact solutions to stochastic optimization at each iteration. Instead, we propose an inexact generalization in which an inexact solution is computed via an increasing number of projected stochastic gradient steps. Based on this framework, we present three inexact best-response schemes: (i) First, we propose a synchronous scheme where all players simultaneously update their strategies; (ii) Subsequently, we extend this to a randomized setting where a subset of players is randomly chosen to their update strategies while the others keep their strategies invariant; (iii) Finally, we propose an asynchronous scheme, where each player determines its own update frequency and may use outdated rival-specific data in updating its strategy. Under a suitable contractive property of the proximal best-response map, we derive a.s. convergence of the iterates for (i) and (ii) and mean-convergence for (i) -- (iii). In addition, we show that for (i) -- (iii), the iterates converge to the unique equilibrium in mean at a prescribed linear rate. Finally, we establish the overall iteration complexity in terms of projected stochastic gradient steps for computing an $ε-$Nash equilibrium and in all settings, the iteration complexity is ${\cal O}(1/ε^{2(1+c) + δ})$ where $c = 0$ in the context of (i) and represents the positive cost of randomization (in (ii)) and asynchronicity and delay (in (iii)). The schemes are further extended to linear and quadratic recourse-based stochastic Nash games.

math.OC↗

arXiv2015

A Unified Algorithmic Framework for Block-Structured Optimization Involving Big Data

This article presents a powerful algorithmic framework for big data optimization, called the Block Successive Upper bound Minimization (BSUM). The BSUM includes as special cases many well-known methods for analyzing massive data sets, such as the Block Coordinate Descent (BCD), the Convex-Concave Procedure (CCCP), the Block Coordinate Proximal Gradient (BCPG) method, the Nonnegative Matrix Factorization (NMF), the Expectation Maximization (EM) method and so on. In this article, various features and properties of the BSUM are discussed from the viewpoint of design flexibility, computational efficiency, parallel/distributed implementation and the required communication overhead. Illustrative examples from networking, signal processing and machine learning are presented to demonstrate the practical performance of the BSUM framework

math.OC↗

arXiv2015

Computing B-Stationary Points of Nonsmooth DC Programs

Motivated by a class of applied problems arising from physical layer based security in a digital communication system, in particular, by a secrecy sum-rate maximization problem, this paper studies a nonsmooth, difference-of-convex (dc) minimization problem. The contributions of this paper are: (i) clarify several kinds of stationary solutions and their relations; (ii) develop and establish the convergence of a novel algorithm for computing a d-stationary solution of a problem with a convex feasible set that is arguably the sharpest kind among the various stationary solutions; (iii) extend the algorithm in several directions including: a randomized choice of the subproblems that could help the practical convergence of the algorithm, a distributed penalty approach for problems whose objective functions are sums of dc functions, and problems with a specially structured (nonconvex) dc constraint. For the latter class of problems, a pointwise Slater constraint qualification is introduced that facilitates the verification and computation of a B(ouligand)-stationary point.

math.OC↗

arXiv2014

Parallel Successive Convex Approximation for Nonsmooth Nonconvex Optimization

Consider the problem of minimizing the sum of a smooth (possibly non-convex) and a convex (possibly nonsmooth) function involving a large number of variables. A popular approach to solve this problem is the block coordinate descent (BCD) method whereby at each iteration only one variable block is updated while the remaining variables are held fixed. With the recent advances in the developments of the multi-core parallel processing technology, it is desirable to parallelize the BCD method by allowing multiple blocks to be updated simultaneously at each iteration of the algorithm. In this work, we propose an inexact parallel BCD approach where at each iteration, a subset of the variables is updated in parallel by minimizing convex approximations of the original objective function. We investigate the convergence of this parallel BCD method for both randomized and cyclic variable selection rules. We analyze the asymptotic and non-asymptotic convergence behavior of the algorithm for both convex and non-convex objective functions. The numerical experiments suggest that for a special case of Lasso minimization problem, the cyclic block selection rule can outperform the randomized rule.

math.OC↗

arXiv2013

Real and Complex Monotone Communication Games

Noncooperative game-theoretic tools have been increasingly used to study many important resource allocation problems in communications, networking, smart grids, and portfolio optimization. In this paper, we consider a general class of convex Nash Equilibrium Problems (NEPs), where each player aims to solve an arbitrary smooth convex optimization problem. Differently from most of current works, we do not assume any specific structure for the players' problems, and we allow the optimization variables of the players to be matrices in the complex domain. Our main contribution is the design of a novel class of distributed (asynchronous) best-response- algorithms suitable for solving the proposed NEPs, even in the presence of multiple solutions. The new methods, whose convergence analysis is based on Variational Inequality (VI) techniques, can select, among all the equilibria of a game, those that optimize a given performance criterion, at the cost of limited signaling among the players. This is a major departure from existing best-response algorithms, whose convergence conditions imply the uniqueness of the NE. Some of our results hinge on the use of VI problems directly in the complex domain; the study of these new kind of VIs also represents a noteworthy innovative contribution. We then apply the developed methods to solve some new generalizations of SISO and MIMO games in cognitive radios and femtocell systems, showing a considerable performance improvement over classical pure noncooperative schemes.

cs.GT↗

arXiv2013

A New Distributed DC-Programming Method and its Applications

We propose a novel decomposition framework for the distributed optimization of Difference Convex (DC)-type nonseparable sum-utility functions subject to coupling convex constraints. A major contribution of the paper is to develop for the first time a class of (inexact) best-response-like algorithms with provable convergence, where a suitably convexified version of the original DC program is iteratively solved. The main feature of the proposed successive convex approximation method is its decomposability structure across the users, which leads naturally to distributed algorithms in the primal and/or dual domain. The proposed framework is applicable to a variety of multiuser DC problems in different areas, ranging from signal processing, to communications and networking. As a case study, in the second part of the paper we focus on two examples, namely: i) a novel resource allocation problem in the emerging area of cooperative physical layer security; ii) and the renowned sum-rate maximization of MIMO Cognitive Radio networks. Our contribution in this context is to devise a class of easy-to-implement distributed algorithms with provable convergence to stationary solution of such problems. Numerical results show that the proposed distributed schemes reach performance close to (and sometimes better than) that of centralized methods.

cs.IT↗

arXiv2013

Decomposition by Partial Linearization: Parallel Optimization of Multi-Agent Systems

We propose a novel decomposition framework for the distributed optimization of general nonconvex sum-utility functions arising naturally in the system design of wireless multiuser interfering systems. Our main contributions are: i) the development of the first class of (inexact) Jacobi best-response algorithms with provable convergence, where all the users simultaneously and iteratively solve a suitably convexified version of the original sum-utility optimization problem; ii) the derivation of a general dynamic pricing mechanism that provides a unified view of existing pricing schemes that are based, instead, on heuristics; and iii) a framework that can be easily particularized to well-known applications, giving rise to very efficient practical (Jacobi or Gauss-Seidel) algorithms that outperform existing adhoc methods proposed for very specific problems. Interestingly, our framework contains as special cases well-known gradient algorithms for nonconvex sum-utility problems, and many blockcoordinate descent schemes for convex functions.

cs.IT↗

arXiv2012

Joint Sensing and Power Allocation in Nonconvex Cognitive Radio Games: Nash Equilibria and Distributed Algorithms

In this paper, we propose a novel class of Nash problems for Cognitive Radio (CR) networks, modeled as Gaussian frequency-selective interference channels, wherein each secondary user (SU) competes against the others to maximize his own opportunistic throughput by choosing jointly the sensing duration, the detection thresholds, and the vector power allocation. The proposed general formulation allows to accommodate several (transmit) power and (deterministic/probabilistic) interference constraints, such as constraints on the maximum individual and/or aggregate (probabilistic) interference tolerable at the primary receivers. To keep the optimization as decentralized as possible, global (coupling) interference constraints are imposed by penalizing each SU with a set of time-varying prices based upon his contribution to the total interference; the prices are thus additional variable to optimize. The resulting players' optimization problems are nonconvex; moreover, there are possibly price clearing conditions associated with the global constraints to be satisfied by the solution. All this makes the analysis of the proposed games a challenging task; none of classical results in the game theory literature can be successfully applied. The main contribution of this paper is to develop a novel optimization-based theory for studying the proposed nonconvex games; we provide a comprehensive analysis of the existence and uniqueness of a standard Nash equilibrium, devise alternative best-response based algorithms, and establish their convergence.

cs.IT↗

arXiv2012

Joint Sensing and Power Allocation in Nonconvex Cognitive Radio Games: Quasi-Nash Equilibria

In this paper, we propose a novel class of Nash problems for Cognitive Radio (CR) networks composed of multiple primary users (PUs) and secondary users (SUs) wherein each SU (player) competes against the others to maximize his own opportunistic throughput by choosing jointly the sensing duration, the detection thresholds, and the vector power allocation over a multichannel link. In addition to power budget constraints, several (deterministic or probabilistic) interference constraints can be accommodated in the proposed general formulation, such as constraints on the maximum individual/aggregate (probabilistic) interference tolerable from the PUs. To keep the optimization as decentralized as possible, global interference constraints, when present, are imposed via pricing; the prices are thus additional variables to be optimized. The resulting players' optimization problems are nonconvex and there are price clearance conditions associated with the nonconvex global interference constraints to be satisfied by the equilibria of the game, which make the analysis of the proposed game a challenging task; none of classical results in the game theory literature can be successfully applied. To deal with the nonconvexity of the game, we introduce a relaxed equilibrium concept, the Quasi-Nash Equilibrium (QNE), and study its main properties, performance, and connection with local Nash equilibria. Quite interestingly, the proposed game theoretical formulations yield a considerable performance improvement with respect to current centralized and decentralized designs of CR systems, which validates the concept of QNE.

cs.IT↗

arXiv2008

Distributed Power Allocation with Rate Constraints in Gaussian Parallel Interference Channels

This paper considers the minimization of transmit power in Gaussian parallel interference channels, subject to a rate constraint for each user. To derive decentralized solutions that do not require any cooperation among the users, we formulate this power control problem as a (generalized) Nash equilibrium game. We obtain sufficient conditions that guarantee the existence and nonemptiness of the solution set to our problem. Then, to compute the solutions of the game, we propose two distributed algorithms based on the single user waterfilling solution: The \emph{sequential} and the \emph{simultaneous} iterative waterfilling algorithms, wherein the users update their own strategies sequentially and simultaneously, respectively. We derive a unified set of sufficient conditions that guarantee the uniqueness of the solution and global convergence of both algorithms. Our results are applicable to all practical distributed multipoint-to-multipoint interference systems, either wired or wireless, where a quality of service in terms of information rate must be guaranteed for each link.

cs.IT↗

arXiv2004

On the Global Minimization of the Value-at-Risk

In this paper, we consider the nonconvex minimization problem of the value-at-risk (VaR) that arises from financial risk analysis. By considering this problem as a special linear program with linear complementarity constraints (a bilevel linear program to be more precise), we develop upper and lower bounds for the minimum VaR and show how the combined bounding procedures can be used to compute the latter value to global optimality. A numerical example is provided to illustrate the methodology.

math.OC↗