arXiv subjects

Tamer Basar

Publications and source records attributed to Tamer Basar.

At least 37 records · Page 2Linked to original sources

arXiv2021

Infinite-horizon Risk-constrained Linear Quadratic Regulator with Average Cost

The behaviour of a stochastic dynamical system may be largely influenced by those low-probability, yet extreme events. To address such occurrences, this paper proposes an infinite-horizon risk-constrained Linear Quadratic Regulator (LQR) framework with time-average cost. In addition to the standard LQR objective, the average one-stage predictive variance of the state penalty is constrained to lie within a user-specified level. By leveraging the duality, its optimal solution is first shown to be stationary and affine in the state, i.e., $u(x,λ^*) = -K(λ^*)x + l(λ^*)$, where $λ^*$ is an optimal multiplier, used to address the risk constraint. Then, we establish the stability of the resulting closed-loop system. Furthermore, we propose a primal-dual method with sublinear convergence rate to find an optimal policy $u(x,λ^*)$. Finally, a numerical example is provided to demonstrate the effectiveness of the proposed framework and the primal-dual method.

math.OC↗

arXiv2020

Fixed-Time Nash Equilibrium Seeking in Non-Cooperative Games

We introduce a novel class of Nash equilibrium seeking dynamics for non-cooperative games with a finite number of players, where the convergence to the Nash equilibrium is bounded by a KL function with a settling time that can be upper bounded by a positive constant that is independent of the initial conditions of the players, and which can be prescribed a priori by the system designer. The dynamics are model-free, in the sense that the mathematical forms of the cost functions of the players are unknown. Instead, in order to update its own action, each player needs to have access only to real-time evaluations of its own cost, as well as to auxiliary states of neighboring players characterized by a communication graph. Stability and convergence properties are established for both potential games and strongly monotone games. Numerical examples are presented to illustrate our theoretical results.

math.OC↗

arXiv2020

Persuasion-based Robust Sensor Design Against Attackers with Unknown Control Objectives

In this paper, we introduce a robust sensor design framework to provide "persuasion-based" defense in stochastic control systems against an unknown type attacker with a control objective exclusive to its type. For effective control, such an attacker's actions depend on its belief on the underlying state of the system. We design a robust "linear-plus-noise" signaling strategy to encode sensor outputs in order to shape the attacker's belief in a strategic way and correspondingly to persuade the attacker to take actions that lead to minimum damage with respect to the system's objective. The specific model we adopt is a Gauss-Markov process driven by a controller with a (partially) "unknown" malicious/benign control objective. We seek to defend against the worst possible distribution over control objectives in a robust way under the solution concept of Stackelberg equilibrium, where the sensor is the leader. We show that a necessary and sufficient condition on the covariance matrix of the posterior belief is a certain linear matrix inequality and we provide a closed-form solution for the associated signaling strategy. This enables us to formulate an equivalent tractable problem, indeed a semi-definite program, to compute the robust sensor design strategies "globally" even though the original optimization problem is non-convex and highly nonlinear. We also extend this result to scenarios where the sensor makes noisy or partial measurements. Finally, we analyze the ensuing performance numerically for various scenarios.

eess.SY↗

arXiv2020

Bayesian Persuasion with State-Dependent Quadratic Cost Measures

We address Bayesian persuasion between a sender and a receiver with state-dependent quadratic cost measures for general classes of distributions. The receiver seeks to make mean-square-error estimate of a state based on a signal sent by the sender while the sender signals strategically in order to control the receiver's estimate in a certain way. Such a scheme could model, e.g., deception and privacy, problems in multi-agent systems. Existing solution concepts are not viable since here the receiver has continuous action space. We show that for finite state spaces, optimal signaling strategies can be computed through an equivalent linear optimization problem over the cone of completely positive matrices. We then establish its strong duality to a copositive program. To exemplify the effectiveness of this equivalence result, we adopt sequential polyhedral approximation of completely-positive cones and analyze its performance numerically. We also quantify the approximation error for a quantized version of a continuous distribution and show that a semi-definite program relaxation of the equivalent problem could be a benchmark lower bound for the sender's cost for large state spaces.

cs.GT↗

arXiv2020

Zero-Sum Differential Games on the Wasserstein Space

We consider two-player zero-sum differential games (ZSDGs), where the state process (dynamical system) depends on the random initial condition and the state process's distribution, and the objective functional includes the state process's distribution and the random target variable. Unlike ZSDGs studied in the existing literature, the ZSDG of this paper introduces a new technical challenge, since the corresponding (lower and upper) value functions are defined on $\mathcal{P}_2$ (the set of probability measures with finite second moments) or $\mathcal{L}_2$ (the set of random variables with finite second moments), both of which are infinite-dimensional spaces. We show that the (lower and upper) value functions on $\mathcal{P}_2$ and $\mathcal{L}_2$ are equivalent (law invariant) and continuous, satisfying dynamic programming principles. We use the notion of derivative of a function of probability measures in $\mathcal{P}_2$ and its lifted version in $\mathcal{L}_2$ to show that the (lower and upper) value functions are unique viscosity solutions to the associated (lower and upper) Hamilton-Jacobi-Isaacs equations that are (infinite-dimensional) first-order PDEs on $\mathcal{P}_2$ and $\mathcal{L}_2$, where the uniqueness is obtained via the comparison principle. Under the Isaacs condition, we show that the ZSDG has a value.

math.OC↗

arXiv2020

Analysis, Identification, and Validation of Discrete-Time Epidemic Processes

Models of spread processes over non-trivial networks are commonly motivated by modeling and analysis of biological networks, computer networks, and human contact networks. However, identification of such models has not yet been explored in detail, and the models have not been validated by real data. In this paper, we present several different spread models from the literature and explore their relationships to each other; for one of these processes, we present a sufficient condition for asymptotic stability of the healthy equilibrium, show that the condition is necessary and sufficient for uniqueness of the healthy equilibrium, and present necessary and sufficient conditions for learning the spread parameters. Finally, we employ two real datasets, one from John Snow's seminal work on cholera epidemics in London in the 1850's and the other one from the United States Department of Agriculture, to validate an approximation of a well-studied network-dependent susceptible-infected-susceptible (SIS) model.

math.OC↗

arXiv2019

A Multi-Agent Off-Policy Actor-Critic Algorithm for Distributed Reinforcement Learning

This paper extends off-policy reinforcement learning to the multi-agent case in which a set of networked agents communicating with their neighbors according to a time-varying graph collaboratively evaluates and improves a target policy while following a distinct behavior policy. To this end, the paper develops a multi-agent version of emphatic temporal difference learning for off-policy policy evaluation, and proves convergence under linear function approximation. The paper then leverages this result, in conjunction with a novel multi-agent off-policy policy gradient theorem and recent work in both multi-agent on-policy and single-agent off-policy actor-critic methods, to develop and give convergence guarantees for a new multi-agent off-policy actor-critic algorithm.

cs.LG↗

arXiv2019

Hierarchical Multistage Gaussian Signaling Games in Noncooperative Communication and Control Systems

We analyze in this paper finite horizon hierarchical signaling games between (information provider) senders and (decision maker) receivers in a dynamic environment. The underlying information evolves in time while sender and receiver interact repeatedly. Different from the classical communication (control) models, however, the sender (sensor) and the receiver (controller) have different objectives and there is a hierarchy between the players such that the sender leads the game by announcing his policies beforehand. He needs to anticipate the reaction of the receiver and the impact of the actions on the horizon while controlling the transparency of the disclosed information at each interaction. With quadratic cost functions and multivariate Gaussian processes, evolving according to first order auto-regressive models, we show that memoryless "linear" sender signaling rules are optimal (in the sense of game-theoretic hierarchical equilibrium) within the general class of measurable policies in the noncooperative communication context. In the noncooperative control context, we also analyze the hierarchical equilibrium for linear signaling rules and provide an algorithm to compute the optimal linear signaling rules numerically with global optimality guarantees.

cs.GT↗

arXiv2019

Reliable Smart Road Signs

In this paper, we propose a game theoretical adversarial intervention detection mechanism for reliable smart road signs. A future trend in intelligent transportation systems is ``smart road signs" that incorporate smart codes (e.g., visible at infrared) on their surface to provide more detailed information to smart vehicles. Such smart codes make road sign classification problem aligned with communication settings more than conventional classification. This enables us to integrate well-established results in communication theory, e.g., error-correction methods, into road sign classification problem. Recently, vision-based road sign classification algorithms have been shown to be vulnerable against (even) small scale adversarial interventions that are imperceptible for humans. On the other hand, smart codes constructed via error-correction methods can lead to robustness against small scale intelligent or random perturbations on them. In the recognition of smart road signs, however, humans are out of the loop since they cannot see or interpret them. Therefore, there is no equivalent concept of imperceptible perturbations in order to achieve a comparable performance with humans. Robustness against small scale perturbations would not be sufficient since the attacker can attack more aggressively without such a constraint. Under a game theoretical solution concept, we seek to ensure certain measure of guarantees against even the worst case (intelligent) attackers that can perturb the signal even at large scale. We provide a randomized detection strategy based on the distance between the decoder output and the received input, i.e., error rate. Finally, we examine the performance of the proposed scheme over various scenarios.

cs.LG↗

arXiv2019

Deception-As-Defense Framework for Cyber-Physical Systems

We introduce deceptive signaling framework as a new defense measure against advanced adversaries in cyber-physical systems. In general, adversaries look for system-related information, e.g., the underlying state of the system, in order to learn the system dynamics and to receive useful feedback regarding the success/failure of their actions so as to carry out their malicious task. To this end, we craft the information that is accessible to adversaries strategically in order to control their actions in a way that will benefit the system, indirectly and without any explicit enforcement. Under the solution concept of game-theoretic hierarchical equilibrium, we arrive at a semi-definite programming problem equivalent to the infinite-dimensional optimization problem faced by the defender while selecting the best strategy when the information of interest is Gaussian and both sides have quadratic cost functions. The equivalence result holds also for the scenarios where the defender can have partial or noisy measurements or the objective of the adversary is not known. We show the optimality of linear signaling rule within the general class of measurable policies in communication scenarios and also compute the optimal linear signaling rule in control scenarios.

cs.CR↗

arXiv2019

Analysis and Control of a Continuous-Time Bi-Virus Model

This paper studies a distributed continuous-time bi-virus model in which two competing viruses spread over a network consisting of multiple groups of individuals. Limiting behaviors of the network are characterized by analyzing the equilibria of the system and their stability. Specifically, when the two viruses spread over possibly different directed infection graphs, the system may have (1) a unique equilibrium, the healthy state, which is globally stable, implying that both viruses will eventually be eradicated, (2) two equilibria including the healthy state and a dominant virus state, which is almost globally stable, implying that one virus will pervade the entire network causing a single-virus epidemic while the other virus will be eradicated, or (3) at least three equilibria including the healthy state and two dominant virus states, depending on certain conditions on the healing and infection rates. When the two viruses spread over the same directed infection graph, the system may have zero or infinitely many coexisting epidemic equilibria, which represents the pervasion of the two viruses. Sensitivity properties of some nontrivial equilibria are investigated in the context of a decentralized control technique, and an impossibility result is given for a certain type of distributed feedback controller.

math.OC↗

arXiv2018

Discrete-time Risk-sensitive Mean-field Games

In this paper, we study a class of discrete-time mean-field games under the infinite-horizon risk-sensitive discounted-cost optimality criterion. Risk-sensitivity is introduced for each agent (player) via an exponential utility function. In this game model, each agent is coupled with the rest of the population through the empirical distribution of the states, which affects both the agent's individual cost and its state dynamics. Under mild assumptions, we establish the existence of a mean-field equilibrium in the infinite-population limit as the number of agents ($N$) goes to infinity, and then show that the policy obtained from the mean-field equilibrium constitutes an approximate Nash equilibrium when $N$ is sufficiently large.

math.OC↗

arXiv2018

Revisiting Client Puzzles for State Exhaustion Attacks Resilience

In this paper, we address the challenges facing the adoption of client puzzles as means to protect the TCP connection establishment channel from state exhaustion DDoS attacks. We model the problem of selecting the puzzle difficulties as a Stackelberg game with the server as the leader and the clients as the followers and obtain the equilibrium solution for the puzzle difficulty. We then present an implementation of client puzzles inside the TCP stack of the Linux 4.13.0 kernel. We evaluate the performance of our implementation and the obtained solution against a range of attacks through experiments on the DETER testbed. Our results show that client puzzles are effective at boosting the tolerance of the TCP handshake channel to state exhaustion DDoS attacks by rate limiting the flood rate of malicious attackers while allocating resources for legitimate clients. Our results illustrate the benefits that the servers and clients amass from the deployment of TCP client puzzles and incentivize their adoption as means to enhance tolerance to multi-vectored DDoS attacks

cs.CR↗

arXiv2018

Approximate Nash Equilibria in Partially Observed Stochastic Games with Mean-Field Interactions

Establishing the existence of Nash equilibria for partially observed stochastic dynamic games is known to be quite challenging, with the difficulties stemming from the noisy nature of the measurements available to individual players (agents) and the decentralized nature of this information. When the number of players is sufficiently large and the interactions among agents is of the mean-field type, one way to overcome this challenge is to investigate the infinite-population limit of the problem, which leads to a mean-field game. In this paper, we consider discrete-time partially observed mean-field games with infinite-horizon discounted cost criteria. Using the technique of converting the original partially observed stochastic control problem to a fully observed one on the belief space and the dynamic programming principle, we establish the existence of Nash equilibria for these game models under very mild technical conditions. Then, we show that the mean-field equilibrium policy, when adopted by each agent, forms an approximate Nash equilibrium for games with sufficiently many agents.

eess.SY↗

arXiv2018

On Remote Estimation with Multiple Communication Channels

This paper considers a sequential sensor scheduling and remote estimation problem with multiple communication channels. Departing from the classical remote estimation paradigm, which involves one communication channel (noiseless or noisy), we consider here the more realistic setting of two channels with different characteristics (one is cheap but noisy, the other one is costly but noiseless). We first show, via a counter-example, that the common folklore of applying symmetric threshold-based policy, which is well known to be optimal (for unimodal state densities) in the classical remote estimation problem, can no longer be optimal in our setting. In view of that, and in order to make the problem tractable, we introduce a side channel which signals to the receiver the sign of the underlying state. We show, under some technical assumptions, that a threshold-in-threshold based communication scheduling is optimal. The impact of the results is analyzed numerically based on dynamic programming. This numerical analysis reveals some rather surprising results inheriting known properties from the single channel settings, such as not exhausting all the opportunities available for the noisy channel.

eess.SY↗

arXiv2017

Controlling and Stabilizing a Rigid Formation Using a Few Agents

We show in this paper that a small subset of agents of a formation of n agents in Euclidean space can control the position and orientation of the entire formation. We consider here formations tasked with maintaining inter-agent distances at prescribed values. It is known that when the inter-agent distances specified can be realized as the edges of a rigid graph, there is a finite number of possible configurations of the agents that satisfy the distance constraints, up to rotations and translations of the entire formation. We show here that under mild conditions on the type of control used by the agents, a subset of them forming a clique can work together to control the position and orientation of the formation as a whole. Mathematically, we investigate the effect of certain allowable perturbations of a nominal dynamics of the formation system. In particular, we show that any such perturbation leads to a rigid motion of the entire formation. Furthermore, we show that the map which assigns to a perturbation the infinitesimal generator of the corresponding rigid motion is locally surjective.

math.DS↗

arXiv2017

On Convergence Rate of a Continuous-Time Distributed Self-Appraisal Model with Time-Varying Relative Interaction Matrices

This paper studies a recently proposed continuous-time distributed self-appraisal model with time-varying interactions among a network of $n$ individuals which are characterized by a sequence of time-varying relative interaction matrices. The model describes the evolution of the social-confidence levels of the individuals via a reflected appraisal mechanism in real time. We first show by example that when the relative interaction matrices are stochastic (not doubly stochastic), the social-confidence levels of the individuals may not converge to a steady state. We then show that when the relative interaction matrices are doubly stochastic, the $n$ individuals' self-confidence levels will all converge to $1/n$, which indicates a democratic state, exponentially fast under appropriate assumptions, and provide an explicit expression of the convergence rate.

math.OC↗

arXiv2017

Optimal Communication Strategies in Networked Cyber-Physical Systems with Adversarial Elements

This paper studies optimal communication and coordination strategies in cyber-physical systems for both defender and attacker within a game-theoretic framework. We model the communication network of a cyber-physical system as a sensor network which involves one single Gaussian source observed by many sensors, subject to additive independent Gaussian observation noises. The sensors communicate with the estimator over a coherent Gaussian multiple access channel. The aim of the receiver is to reconstruct the underlying source with minimum mean squared error. The scenario of interest here is one where some of the sensors are captured by the attacker and they act as the adversary (jammer): they strive to maximize distortion. The receiver (estimator) knows the captured sensors but still cannot simply ignore them due to the multiple access channel, i.e., the outputs of all sensors are summed to generate the estimator input. We show that the ability of transmitter sensors to secretly agree on a random event, that is "coordination", plays a key role in the analysis...

cs.GT↗