arXiv subjects

Lacra Pavel

Publications and source records attributed to Lacra Pavel.

34 records · Page 2Linked to original sources

arXiv2018

On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning

In this paper, we utilize results from convex analysis and monotone operator theory to derive additional properties of the softmax function that have not yet been covered in the existing literature. In particular, we show that the softmax function is the monotone gradient map of the log-sum-exp function. By exploiting this connection, we show that the inverse temperature parameter determines the Lipschitz and co-coercivity properties of the softmax function. We then demonstrate the usefulness of these properties through an application in game-theoretic reinforcement learning.

math.OC↗

arXiv2018

Distributed GNE seeking under partial-decision information over networks via a doubly-augmented operator splitting approach

We consider distributed computation of generalized Nash equilibrium (GNE) over networks, in games with shared coupling constraints. Existing methods require that each player has full access to opponents' decisions. In this paper, we assume that players have only partial-decision information, and can communicate with their neighbours over an arbitrary undirected graph. We recast the problem as that of finding a zero of a sum of monotone operators through primal-dual analysis. To distribute the problem, we doubly augment variables, so that each player has local decision estimates and local copies of Lagrangian multipliers. We introduce a single-layer algorithm, fully distributed with respect to both primal and dual variables. We show its convergence to a variational GNE with fixed step-sizes, by reformulating it as a forward-backward iteration for a pair of doubly-augmented monotone operators.

math.OC↗

arXiv2018

On Passivity, Reinforcement Learning and Higher-Order Learning in Multi-Agent Finite Games

In this paper, we propose a passivity-based methodology for analysis and design of reinforcement learning in multi-agent finite games. Starting from a known exponentially-discounted reinforcement learning scheme, we show that convergence to a Nash distribution can be shown in the class of games characterized by the monotonicity property of their (negative) payoff. We further exploit passivity to propose a class of higher-order schemes that preserve convergence properties, can improve the speed of convergence and can even converge in cases whereby their first-order counterpart fail to converge. We demonstrate these properties through numerical simulations for several representative games.

math.OC↗

arXiv2018

Asynchronous distributed algorithms for seeking generalized Nash equilibria under full and partial decision information

We investigate asynchronous distributed algorithms with delayed information for seeking generalized Nash equilibrium over multi-agent networks. The considered game model has all players' local decisions coupled with a shared affine constraint. We assume each player can only access its local objective function, local constraint, and a local block matrix of the affine constraint. We first give the algorithm for the case when each agent is able to fully access all other players' decisions. With the help of auxiliary edge variables and edge Laplacian matrix, each player can carry on its local iteration in an asynchronous manner, using only local data and possibly delayed neighbour information. And then we investigate the case when the agents cannot know all other players' decisions, which is called a partial decision case. We introduce a local estimation of the overall decisions for each agent in the partial decision case, and develop another asynchronous algorithm by incorporating consensus dynamics on the local estimations of the overall decisions. Since both algorithms do not need any centralized clock coordination, the algorithms fully exploit the local computation resource of each player, and remove the idle time due to waiting for the "slowest" agent. Both algorithms are developed by a preconditioned forward-backward operator splitting method, while convergence is shown with the help of asynchronous fixed-point iterations under proper assumptions and fixed step-size choices. Numerical studies verify both algorithms' convergence and efficiency.

math.OC↗

arXiv2018

A Two-Step Linear Programming Model for Energy-Efficient Timetables in Metro Railway Networks

In this paper we propose a novel two-step linear optimization model to calculate energy-efficient timetables in metro railway networks. The resultant timetable minimizes the total energy consumed by all trains and maximizes the utilization of regenerative energy produced by braking trains, subject to the constraints in the railway network. In contrast to other existing models, which are NP-hard, our model is computationally the most tractable one being a linear program. We apply our optimization model to different instances of service PES2-SFM2 of line 8 of Shanghai Metro network spanning a full service period of one day (18 hours) with thousands of active trains. For every instance, our model finds an optimal timetable very quickly (largest runtime being less than 13s) with significant reduction in effective energy consumption (the worst case being 19.27%). Code based on the model has been integrated with Thales Timetable Compiler - the industrial timetable compiler of Thales Inc that has the largest installed base of communication-based train control systems worldwide.

math.OC↗

arXiv2017

Distributed generalized Nash equilibria computation of monotone games via a preconditioned proximal point algorithm

In this paper, we investigate distributed generalized Nash equilibrium (GNE) computation of monotone games with affine coupling constraints. Each player can only utilize its local objective function, local feasible set and a local block of the coupling constraint, and can only communicate with its neighbours. We assume the game has monotone pseudo-subdifferential without Lipschitz continuity restrictions. We design novel center-free distributed GNE seeking algorithms for equality and inequality affine coupling constraints, respectively. A proximal alternating direction method of multipliers(ADMM) is proposed for the equality case, while for the inequality case, a parallel splitting type algorithm is proposed. In both algorithms, the GNE seeking task is decomposed into a sequential NE computation of regularized subgames and distributed update of multipliers and auxiliary variables, based on local data and local communication. Our two double-layer GNE algorithms need not specify the inner-loop NE seeking algorithm and moreover, only require that the strongly monotone subgames are inexactly solved. We prove their convergence by showing that the two algorithms can be seen as specific instances of preconditioned proximal point algorithms} (PPPA) for finding zeros of monotone operators. Applications and numerical simulations are given for illustration.

math.OC↗

arXiv2017

A Passivity-Based Approach to Nash Equilibrium Seeking over Networks

In this paper we consider the problem of distributed Nash equilibrium (NE) seeking over networks, a setting in which players have limited local information. We start from a continuous-time gradient-play dynamics that converges to an NE under strict monotonicity of the pseudo-gradient and assumes perfect information, i.e., instantaneous all-to-all player communication. We consider how to modify this gradient-play dynamics in the case of partial, or networked information between players. We propose an augmented gradient-play dynamics with correction in which players communicate locally only with their neighbours to compute an estimate of the other players' actions. We derive the new dynamics based on the reformulation as a multi-agent coordination problem over an undirected graph. We exploit incremental passivity properties and show that a synchronizing, distributed Laplacian feedback can be designed using relative estimates of the neighbours. Under a strict monotonicity property of the pseudo-gradient, we show that the augmented gradient-play dynamics converges to consensus on the NE of the game. We further discuss two cases that highlight the tradeoff between properties of the game and the communication graph.

math.OC↗

arXiv2017

Nash Equilibrium Seeking with Non-doubly Stochastic Communication Weight Matrix

A distributed Nash equilibrium seeking algorithm is presented for networked games. We assume an incomplete information available to each player about the other players' actions. The players communicate over a strongly connected digraph to send/receive the estimates of the other players' actions to/from the other local players according to a gossip communication protocol. Due to asymmetric information exchange between the players, a non-doubly (row) stochastic weight matrix is defined. We show that, due to the non-doubly stochastic property, the total average of all players' estimates is not preserved for the next iteration which results in having no exact convergence. We present an almost sure convergence proof of the algorithm to a Nash equilibrium of the game. Then, we extend the algorithm for graphical games in which all players' cost functions are only dependent on the local neighboring players over an interference digraph. We design an assumption on the communication digraph such that the players are able to update all the estimates of the players who interfere with their cost functions. It is shown that the communication digraph needs to be a superset of a transitive reduction of the interference digraph. Finally, we verify the efficacy of the algorithm via a simulation on a social media behavioral case.

eess.SY↗

arXiv2017

A Distributed Nash Equilibrium Seeking in Networked Graphical Games

This paper considers a distributed gossip approach for finding a Nash equilibrium in networked games on graphs. In such games a player's cost function may be affected by the actions of any subset of players. An interference graph is employed to illustrate the partially-coupled cost functions and the asymmetric information requirements. For a given interference graph, network communication between players is considered to be limited. A generalized communication graph is designed so that players exchange only their required information. An algorithm is designed whereby players, with possibly partially-coupled cost functions, make decisions based on the estimates of other players' actions obtained from local neighbors. It is shown that this choice of communication graph guarantees that all players' information is exchanged after sufficiently many iterations. Using a set of standard assumptions on the cost functions, the interference and the communication graphs, almost sure convergence to a Nash equilibrium is proved for diminishing step sizes. Moreover, the case when the cost functions are not known by the players is investigated and a convergence proof is presented for diminishing step sizes. The effect of the second largest eigenvalue of the expected communication matrix on the convergence rate is quantified. The trade-off between parameters associated with the communication graph and the ones associated with the interference graph is illustrated. Numerical results are presented for a large-scale networked game.

cs.GT↗

arXiv2017

Generalized Nash Equilibrium Problem by the Alternating Direction Method of Multipliers

In this paper, the problem of finding a generalized Nash equilibrium (GNE) of a networked game is studied. Players are only able to choose their decisions from a feasible action set. The feasible set is considered to be a private linear equality constraint that is coupled through decisions of the other players. We consider that each player has his own private constraint and it has not to be shared with the other players. This general case also embodies the one with shared constraints between players and it can be also simply extended to the case with inequality constraints. Since the players don't have access to other players' actions, they need to exchange estimates of others' actions and a local copy of the Lagrangian multiplier with their neighbors over a connected communication graph. We develop a relatively fast algorithm by reformulating the conservative GNE problem within the framework of inexact-ADMM. The convergence of the algorithm is guaranteed under a few mild assumptions on cost functions. Finally, the algorithm is simulated for a wireless ad-hoc network.

cs.GT↗

arXiv2017

A distributed primal-dual algorithm for computation of generalized Nash equilibria with shared affine coupling constraints via operator splitting methods

In this paper, we propose a distributed primal-dual algorithm for computation of a generalized Nash equilibrium (GNE) in noncooperative games over network systems. In the considered game, not only each player's local objective function depends on other players' decisions, but also the feasible decision sets of all the players are coupled together with a globally shared affine inequality constraint. Adopting the variational GNE, that is the solution of a variational inequality, as a refinement of GNE, we introduce a primal-dual algorithm that players can use to seek it in a distributed manner. Each player only needs to know its local objective function, local feasible set, and a local block of the affine constraint. Meanwhile, each player only needs to observe the decisions on which its local objective function explicitly depends through the interference graph and share information related to multipliers with its neighbors through a multiplier graph. Through a primal-dual analysis and an augmentation of variables, we reformulate the problem as finding the zeros of a sum of monotone operators. Our distributed primal-dual algorithm is based on forward-backward operator splitting methods. We prove its convergence to the variational GNE for fixed step-sizes under some mild assumptions. Then a distributed algorithm with inertia is also introduced and analyzed for variational GNE seeking. Finally, numerical simulations for network Cournot competition are given to illustrate the algorithm efficiency and performance.

math.OC↗

arXiv2016

Distributed Nash Equilibrium Seeking via the Alternating Direction Method of Multipliers

In this paper, the problem of finding a Nash equilibrium of a multi-player game is considered. The players are only aware of their own cost functions as well as the action space of all players. We develop a relatively fast algorithm within the framework of inexact-ADMM. It requires a communication graph for the information exchange between the players as well as a few mild assumptions on cost functions. The convergence proof of the algorithm to a Nash equilibrium of the game is then provided. Moreover, the convergence rate is investigated via simulations.

eess.SY↗

arXiv2016

Distributed Nash Equilibrium Seeking By Gossip in Games on Graphs

We consider a gossip approach for finding a Nash equilibrium in a distributed multi-player network game. We extend previous results on Nash equilibrium seeking to the case when the players' cost functions may be affected by the actions of any subset of players. An interference graph is employed to illustrate the partially-coupled cost functions and the asymmetric information requirements. For a given interference graph, we design a generalized communication graph so that players with possibly partially-coupled cost functions exchange only their required information and make decisions based on them. Using a set of standard assumptions on the cost functions, interference and communication graphs, we prove almost sure convergence to a Nash equilibrium for diminishing step sizes. We then quantify the effect of the second largest eigenvalue of the expected communication matrix on the convergence rate, and illustrate the trade-off between the parameters associated with the communication and the interference graphs. Finally, the efficacy of the proposed algorithm on a large-scale networked game is demonstrated via simulation.

eess.SY↗

arXiv2015

A Robust Mixed Integer Optimization Model to Utilize Regenerative Energy of Trains in a Railway Network

In this paper we present a robust mixed integer optimization model to utilize regenerative braking energy produced by trains in a railway network. An electric train produces regenerative energy during braking, which is often lost in present technology. To utilize this energy we calculate a timetable which maximizes the total overlapping time between the braking and accelerating phases of suitable train pairs, so that the regenerative energy of a braking train can be transferred to a suitable accelerating one. We apply our optimization model to different instances of a railway network for a time horizon spanning six hours. For each instance, our model finds an optimal timetable very quickly (largest running time being 86.64 seconds). Compared to the existing timetables, we observe significant increase in utilization of regenerative energy for every instance.

math.OC↗

arXiv2011

Enabling Differentiated Services Using Generalized Power Control Model in Optical Networks

This paper considers a generalized framework to study OSNR optimization-based end-to-end link level power control problems in optical networks. We combine favorable features of game-theoretical approach and central cost approach to allow different service groups within the network. We develop solutions concepts for both cases of empty and nonempty feasible sets. In addition, we derive and prove the convergence of a distributed iterative algorithm for different classes of users. In the end, we use numerical examples to illustrate the novel framework.

cs.GT↗

arXiv2010

An Optimization and Control Theoretic Approach to Noncooperative Game Design

This paper investigates design of noncooperative games from an optimization and control theoretic perspective. Pricing mechanisms are used as a design tool to ensure that the Nash equilibrium of a fairly general class of noncooperative games satisfies certain global objectives such as welfare maximization or achieving a certain level of quality-of-service (QoS). The class of games considered provide a theoretical basis for decentralized resource allocation and control problems including network congestion control, wireless uplink power control, and optical power control. The game design problem is analyzed under different knowledge assumptions (full versus limited information) and design objectives (QoS versus utility maximization) for separable and non-separable utility functions. The ``price of anarchy'' is shown not to be an inherent feature of full-information games that incorporate pricing mechanisms. Moreover, a simple linear pricing is shown to be sufficient for design of Nash equilibrium according to a chosen global objective for a fairly general class of games. Stability properties of the game and pricing dynamics are studied under the assumption of time-scale separation and in two separate time-scales. Thus, sufficient conditions are derived, which allow the designer to place the Nash equilibrium solution or to guide the system trajectory to a desired region or point. The obtained results are illustrated with a number of examples.

cs.GT↗