arXiv subjects

Behçet Açıkmeşe

Publications and source records attributed to Behçet Açıkmeşe.

47 records · Page 3Linked to original sources

arXiv2020

Stability of Gradient Learning Dynamics in Continuous Games: Scalar Action Spaces

Learning processes in games explain how players grapple with one another in seeking an equilibrium. We study a natural model of learning based on individual gradients in two-player continuous games. In such games, the arguably natural notion of a local equilibrium is a differential Nash equilibrium. However, the set of locally exponentially stable equilibria of the learning dynamics do not necessarily coincide with the set of differential Nash equilibria of the corresponding game. To characterize this gap, we provide formal guarantees for the stability or instability of such fixed points by leveraging the spectrum of the linearized game dynamics. We provide a comprehensive understanding of scalar games and find that equilibria that are both stable and Nash are robust to variations in learning rates.

cs.GT↗

arXiv2020

Bounding Fixed Points of Set-Based Bellman Operator and Nash Equilibria of Stochastic Games

Motivated by uncertain parameters encountered in Markov decision processes (MDPs) and stochastic games, we study the effect of parameter uncertainty on Bellman operator-based algorithms under a set-based framework. Specifically, we first consider a family of MDPs where the cost parameters are in a given compact set; we then define a Bellman operator acting on a set of value functions to produce a new set of value functions as the output under all possible variations in the cost parameter. We prove the existence of a fixed point of this set-based Bellman operator by showing that it is contractive on a complete metric space, and explore its relationship with the corresponding family of MDPs and stochastic games. Additionally, we show that given interval set bounded cost parameters, we can form exact bounds on the set of optimal value functions. Finally, we utilize our results to bound the value function trajectory of a player in a stochastic game.

cs.GT↗

arXiv2020

Disturbance Decoupling for Gradient-based Multi-Agent Learning with Quadratic Costs

Motivated by applications of multi-agent learning in noisy environments, this paper studies the robustness of gradient-based learning dynamics with respect to disturbances. While disturbances injected along a coordinate corresponding to any individual player's actions can always affect the overall learning dynamics, a subset of players can be disturbance decoupled---i.e., such players' actions are completely unaffected by the injected disturbance. We provide necessary and sufficient conditions to guarantee this property for games with quadratic cost functions, which encompass quadratic one-shot continuous games, finite-horizon linear quadratic (LQ) dynamic games, and bilinear games. Specifically, disturbance decoupling is characterized by both algebraic and graph-theoretic conditions on the learning dynamics, the latter is obtained by constructing a game graph based on gradients of players' costs. For LQ games, we show that disturbance decoupling imposes constraints on the controllable and unobservable subspaces of players. For two player bilinear games, we show that disturbance decoupling within a player's action coordinates imposes constraints on the payoff matrices. Illustrative numerical examples are provided.

cs.GT↗

arXiv2020

RC Circuits based Distributed Conditional Gradient Method

We consider distributed optimization on undirected connected graphs. We propose a novel distributed conditional gradient method with (O(1/\sqrt{k})) convergence. Compared with existing methods, each iteration of our method uses both communication and linear minimization step only once rather than multiple times. We further extend our results to cases with composite local constraints. We demonstrate our results via examples on distributed matrix completion problem.

math.OC↗

arXiv2020

Fixed Points of the Set-Based Bellman Operator

Motivated by uncertain parameters encountered in Markov decision processes (MDPs), we study the effect of parameter uncertainty on Bellman operator-based methods. Specifically, we consider a family of MDPs where the cost parameters are from a given compact set. We then define a Bellman operator acting on an input set of value functions to produce a new set of value functions as the output under all possible variations in the cost parameters. Finally we prove the existence of a fixed point of this set-based Bellman operator by showing that it is a contractive operator on a complete metric space.

math.OC↗

arXiv2020

RLC Circuits based Distributed Mirror Descent Method

We consider distributed optimization with smooth convex objective functions defined on an undirected connected graph. Inspired by mirror descent mehod and RLC circuits, we propose a novel distributed mirror descent method. Compared with mirror-prox method, our algorithm achieves the same $\mathcal{O}(1/k)$ iteration complexity with only half the computation cost per iteration. We further extend our results to cases where a) gradients are corrupted by stochastic noise, and b) objective function is composed of both smooth and non-smooth terms. We demonstrate our theoretical results via numerical experiments.

math.OC↗

arXiv2019

Mass-spring-damper Networks for Distributed Optimization in Non-Euclidean Spaces

We consider the problem of minimizing the sum of non-smooth convex functions in non-Euclidean spaces, e.g., probability simplex, via only local computation and communication on an undirected graph. We propose two algorithms motivated by mass-spring-damper dynamics. The first algorithm uses an explicit update that computes subgradients and Bregman projections only, and matches the convergence behavior of centralized mirror descent. The second algorithm uses an implicit update that solves a penalized subproblem at each iteration, and achieves the iteration complexity of $\mathcal{O}(1/K)$. The results are also demonstrated via numerical examples.

math.OC↗

arXiv2019

Maximal Invariant Set Computation and Design for Markov Chains

We describe an algorithm for computing the maximal invariant set for a Markov chain with linear safety constraints on the distribution over states. We then propose a Markov chain synthesis method that guarantees finite determination of the maximal invariant set. Although this problem is bilinear in the general case, we are able to optimize the convergence rate to a desirable steady-state distribution over reversible Markov chains by solving a Semidefinite Program (SDP), which promotes efficient computation of the maximal invariant set. We then demonstrate this approach with a decentralized swarm guidance application subject to density upper bounds.

math.OC↗

arXiv2019

Stochastic Bregman Parallel Direction Method of Multipliers for Distributed Optimization

Bregman parallel direction method of multipliers (BPDMM) efficiently solves distributed optimization over a network, which arises in a wide spectrum of collaborative multi-agent learning applications. In this paper, we generalize BPDMM to stochastic BPDMM, where each iteration only solves local optimization on a randomly selected subset of nodes rather than all the nodes in the network. Such generalization reduce the need for computational resources and allows applications to larger scale networks. We establish both the global convergence and the $O(1/T)$ iteration complexity of stochastic BPDMM. We demonstrate our results via numerical examples.

math.OC↗

arXiv2018

Bregman Parallel Direction Method of Multipliers for Distributed Optimization via Mirror Averaging

Distributed optimization aims to optimize a global objective formed by a sum of coupled local convex functions over a graph via only local computation and communication. In this paper, we propose the Bregman parallel direction method of multipliers (PDMM) based on a generalized averaging step named mirror averaging. We establish the global convergence and $O(1/T)$ convergence rate of the Bregman PDMM, along with its $O(n/\ln n)$ improvement over existing PDMM, where $T$ denotes the number of iterations and $n$ the dimension of solution variable. In addition, we can enhance its performance by optimizing the spectral gap of the averaging matrix. We demonstrate our results via a numerical example.

math.OC↗

arXiv2017

Successive Convexification of Non-Convex Optimal Control Problems with State Constraints

This paper presents a Successive Convexification ($ \texttt{SCvx} $) algorithm to solve a class of non-convex optimal control problems with certain types of state constraints. Sources of non-convexity may include nonlinear dynamics and non-convex state/control constraints. To tackle the challenge posed by non-convexity, first we utilize exact penalty function to handle the nonlinear dynamics. Then the proposed algorithm successively convexifies the problem via a $ \textit{project-and-linearize} $ procedure. Thus a finite dimensional convex programming subproblem is solved at each succession, which can be done efficiently with fast Interior Point Method (IPM) solvers. Global convergence to a local optimum is demonstrated with certain convexity assumptions, which are satisfied in a broad range of optimal control problems. The proposed algorithm is particularly suitable for solving trajectory planning problems with collision avoidance constraints. Through numerical simulations, we demonstrate that the algorithm converges reliably after only a few successions. Thus with powerful IPM based custom solvers, the algorithm can be implemented onboard for real-time autonomous control applications.

math.OC↗