arXiv subjects

Daniel Calderone

Publications and source records attributed to Daniel Calderone.

5 recordsLinked to original sources

arXiv2020

Stability of Gradient Learning Dynamics in Continuous Games: Scalar Action Spaces

Learning processes in games explain how players grapple with one another in seeking an equilibrium. We study a natural model of learning based on individual gradients in two-player continuous games. In such games, the arguably natural notion of a local equilibrium is a differential Nash equilibrium. However, the set of locally exponentially stable equilibria of the learning dynamics do not necessarily coincide with the set of differential Nash equilibria of the corresponding game. To characterize this gap, we provide formal guarantees for the stability or instability of such fixed points by leveraging the spectrum of the linearized game dynamics. We provide a comprehensive understanding of scalar games and find that equilibria that are both stable and Nash are robust to variations in learning rates.

cs.GT↗

arXiv2020

Stability of Gradient Learning Dynamics in Continuous Games: Vector Action Spaces

Towards characterizing the optimization landscape of games, this paper analyzes the stability of gradient-based dynamics near fixed points of two-player continuous games. We introduce the quadratic numerical range as a method to characterize the spectrum of game dynamics and prove the robustness of equilibria to variations in learning rates. By decomposing the game Jacobian into symmetric and skew-symmetric components, we assess the contribution of a vector field's potential and rotational components to the stability of differential Nash equilibria. Our results show that in zero-sum games, all Nash are stable and robust; in potential games, all stable points are Nash. For general-sum games, we provide a sufficient condition for instability. We conclude with a numerical example in which learning with timescale separation results in faster convergence.

cs.GT↗

arXiv2020

Graph-theoretic optimization for edge consensus

We consider network structures that optimize the $\mathcal{H}_2$ norm of weighted, time scaled consensus networks, under a minimal representation of such consensus networks described by the edge Laplacian. We show that a greedy algorithm can be used to find the minimum-$\mathcal{H}_2$ norm spanning tree, as well as how to choose edges to optimize the $\mathcal{H}_2$ norm when edges are added back to a spanning tree. In the case of edge consensus with a measurement model considering all edges in the graph, we show that adding edges between slow nodes in the graph provides the smallest increase in the $\mathcal{H}_2$ norm.

math.OC↗

arXiv2019

Sensitivity Analysis for Markov Decision Process Congestion Games

We consider a non-atomic congestion game where each decision maker performs selfish optimization over states of a common MDP. The decision makers optimize for their own expected costs, and influence each other through congestion effects on the state-action costs. We analyze on the sensitivity of MDP congestion game equilibria to uncertainty and perturbations in the state-action costs by applying an implicit function type analysis. The occurrence of a stochastic Braess paradox is defined, analyzed based on sensitivity of game equilibria and demonstrated in simulation. We further analyze how the introduction of stochastic dynamics affects the magnitude of Braess paradox in comparison to deterministic dynamics.

cs.GT↗

arXiv2019

Tolling for Constraint Satisfaction in Markov Decision Process Congestion Games

Markov decision process (MDP) congestion game is an extension of classic congestion games, where a continuous population of selfish agents solves Markov decision processes with congestion: the payoff of a strategy decreases as more population uses it. We draw parallels between key concepts from capacitated congestion games and MDP. In particular, we show that population mass constraints in MDP congestion games are equivalent to imposing tolls/incentives on the reward function, which can be utilized by social planners to achieve auxiliary objectives. We demonstrate such methods in a simulated Seattle ride-share model, where tolls and incentives are enforced for two separate objectives: to guarantee minimum driver density in downtown Seattle, and to shift the game equilibrium towards a maximum social output.

cs.GT↗