Searcharxiv⌕ Search

arXiv subjects

Serdar Yüksel

Publications and source records attributed to Serdar Yüksel.

At least 55 records · Page 3Linked to original sources
arXiv2022

Nash Equilibria for Exchangeable Team against Team Games, their Mean Field Limit, and Role of Common Randomness

We study stochastic mean-field games among finite number of teams with large finite as well as infinite number of decision makers. For this class of games within static and dynamic settings, we establish the existence of a Nash equilibrium, and show that a Nash equilibrium exhibits exchangeability in the finite decision maker regime and symmetry in the infinite one. To arrive at these existence and structural theorems, we endow the set of randomized policies with a suitable topology under various decentralized information structures, which leads to the desired convexity and compactness of the set of randomized policies. Then, we establish the existence of a randomized Nash equilibrium that is exchangeable (not necessarily symmetric) among decision makers within each team for a general class of exchangeable stochastic games. As the number of decision makers within each team goes to infinity (that is for the mean-field game among teams), using a de Finetti representation theorem, we show existence of a randomized Nash equilibrium that is symmetric (i.e., identical) among decision makers within each team and also independently randomized. Finally, we establish that a Nash equilibrium for a class of mean-field games among teams (which is symmetric) constitutes an approximate Nash equilibrium for the corresponding pre-limit (exchangeable) game among teams with large but finite number of decision makers. We thus show that common randomness is not necessary for large team-against-team games, unlike the case with small sized teams.

math.OC↗
arXiv2022

Continuity Properties of Value Functions in Information Structures for Zero-Sum and General Games and Stochastic Teams

We study continuity properties of stochastic game problems with respect to various topologies on information structures, defined as probability measures characterizing a game. We will establish continuity properties of the value function under total variation, setwise, and weak convergence of information structures. Our analysis reveals that the value function for a bounded game is continuous under total variation convergence of information structures in both zero-sum games and team problems. Continuity may fail to hold under setwise or weak convergence of information structures, however, the value function exhibits upper semicontinuity properties under weak and setwise convergence of information structures for team problems, and upper or lower semicontinuity properties hold for zero-sum games when such convergence is through a Blackwell-garbled sequence of information structures. If the individual channels are independent, fixed, and satisfy a total variation continuity condition, then the value functions are continuous under weak convergence of priors. We finally show that value functions for players may not be continuous even under total variation convergence of information structures in general non-zero-sum games.

math.OC↗
arXiv2022

Convex Analytic Method Revisited: Further Optimality Results and Performance of Deterministic Policies in Average Cost Stochastic Control

The convex analytic method has proved to be a very versatile method for the study of infinite horizon average cost optimal stochastic control problems. In this paper, we revisit the convex analytic method and make three primary contributions: (i) We present an existence result for controlled Markov models that lack weak continuity of the transition kernel but are strongly continuous in the action variable for every fixed state variable. (ii) For average cost stochastic control problems in standard Borel spaces, while existing results establish the optimality of stationary (possibly randomized) policies, few results are available on the optimality of deterministic policies. We review existing results and present further conditions under which an average cost optimal stochastic control problem admits optimal solutions that are deterministic stationary. (iii) We establish conditions under which the performance under stationary deterministic (and also quantized) policies is dense in the set of performance values under randomized stationary policies.

math.OC↗
arXiv2022

Zero-Sum Games involving Teams against Teams: Existence of Equilibria, and Comparison and Regularity in Information

Many emerging problems involve teams of agents taking part in a game. Such problems require a stochastic analysis with regard to the correlation structures among the agents belonging to a given team. In the context of Standard Borel spaces, this paper makes the following contributions for two teams of finitely many agents taking part in a zero-sum game: (i) An existence result will be presented for saddle-point equilibria in zero-sum games involving teams against teams when common randomness is assumed to be available in each team with an analysis on conditions for compactness of strategic team measures to be presented. (ii) Blackwell's ordering of information structures is generalized to $n$-player teams with standard Borel spaces, where correlated garbling of information structures is introduced as a key attribute; (iii) building on this result Blackwell's ordering of information structures is established for team-against-team zero-sum game problems. (iv) Finally, continuity of the equilibrium value of team-against-team zero-sum game problems in the space of information structures under total variation is established.

math.OC↗
arXiv2022

Quadratic Privacy-Signaling Games and the MMSE Information Bottleneck Problem for Gaussian Sources

We investigate a privacy-signaling game problem in which a sender with privacy concerns observes a pair of correlated random vectors which are modeled as jointly Gaussian. The sender aims to hide one of these random vectors and convey the other one whereas the objective of the receiver is to accurately estimate both of the random vectors. We analyze these conflicting objectives in a game theoretic framework with quadratic costs where depending on the commitment conditions (of the sender), we consider Nash or Stackelberg (Bayesian persuasion) equilibria. We show that a payoff dominant Nash equilibrium among all admissible policies is attained by a set of explicitly characterized linear policies. We also show that a payoff dominant Nash equilibrium coincides with a Stackelberg equilibrium. We formulate the information bottleneck problem within our Stackelberg framework under the mean squared error distortion criterion where the information bottleneck setup has a further restriction that only one of the random variables is observed at the sender. We show that this MMSE Gaussian Information Bottleneck Problem admits a linear solution which is explicitly characterized in the paper. We provide explicit conditions on when the optimal solutions, or equilibrium solutions in the Nash setup, are informative or noninformative.

cs.IT↗
arXiv2022

Sequential Stochastic Control (Single or Multi-Agent) Problems Nearly Admit Change of Measures with Independent Measurements

Change of measures has been an effective method in stochastic control and analysis; in continuous-time control this follows Girsanov's theorem applied to both fully observed and partially observed models, in decentralized stochastic control (or stochastic dynamic team theory) this is known as Witsenhausen's static reduction, and in discrete-time classical stochastic control Borkar has considered this method for partially observed Markov Decision processes (POMDPs) generalizing Fleming and Pardoux's approach in continuous-time. This method allows for equivalent optimal stochastic control or filtering in a new probability space where the measurements form an independent exogenous process in both discrete-time and continuous-time and the Radon-Nikodym derivative (between the true measure and the reference measure formed via the independent measurement process) is pushed to the cost or dynamics. However, for this to be applicable, an absolute continuity condition is necessary. This raises the following question: can we perturb any discrete-time sequential stochastic control problem by adding some arbitrarily small additive (e.g. Gaussian or otherwise) noise to the measurements to make the system measurements absolutely continuous, so that a change-of-measure (or static reduction) can be applicable with arbitrarily small error in the optimal cost? That is, are all sequential stochastic (single-agent or decentralized multi-agent) problems $ε$-away from being static reducible as far as optimal cost is concerned, for any $ε> 0$? We show that this is possible when the cost function is bounded and continuous in controllers' actions and the action spaces are convex. We also note that the solution and the cost obtained for the perturbed system is realizable (under a randomized policy) for the original model.

math.OC↗
arXiv2021

Isomorphism Properties of Optimality and Equilibrium Solutions under Equivalent Information Structure Transformations I: Stochastic Dynamic Teams

In stochastic optimal control, change of measure arguments have been crucial for stochastic analysis. Such an approach is often called static reduction in dynamic team theory (or decentralized stochastic control) and has been an effective method for establishing existence and approximation results for optimal policies. In this paper, we place such static reductions into three categories: (i) those that are policy-independent (as those introduced by Witsenhausen), (ii) those that are policy-dependent (as those introduced by Ho and Chu for partially nested dynamic teams), and (iii) those that we will refer to as static measurements with control-sharing reduction (where the measurements are static although control actions are shared according to the partially nested information structure). For the first type, we show that there is a bijection between person-by-person optimal (globally optimal) policies of dynamic teams and their policy-independent static reductions. For the second type, although there is a bijection between globally optimal policies of dynamic teams with partially nested information structures and their static reductions, in general there is no bijection between person-by-person optimal policies of dynamic teams and their policy-dependent static reductions. We also establish a stronger negative result concerning stationary solutions. We present sufficient conditions under which bijection relationships hold. Under static measurements with control-sharing reduction, connections between optimality concepts can be established under relaxed conditions. An implication is a convexity characterization of dynamic team problems under static measurements with control-sharing reduction. Finally, we introduce multi-stage refinements of such reductions. Part II of the paper addresses similar issues in the context of stochastic dynamic games, where further subtleties arise.

math.OC↗
arXiv2021

Quadratic Signaling with Prior Mismatch at an Encoder and Decoder: Equilibria, Continuity and Robustness Properties

We consider communications through a Gaussian noise channel between an encoder and a decoder which have subjective probabilistic models on the source distribution. Although they consider the same cost function, the induced expected costs are misaligned due to their prior mismatch, which requires a game theoretic approach. We consider two approaches: a Nash setup, with no prior commitment, and a Stackelberg solution concept, where the encoder is committed to a given announced policy apriori. We show that the Stackelberg equilibrium cost of the encoder is upper semi continuous, under the Wasserstein metric, as encoder's prior approaches the decoder's prior, and it is also lower semi continuous with Gaussian priors. For the Stackelberg setup, the optimality of affine policies for Gaussian signaling no longer holds under prior mismatch, and thus team-theoretic optimality of linear/affine policies are not robust to perturbations. We provide conditions under which there exist informative Nash and Stackelberg equilibria with affine policies. Finally, we show existence of fully informative Nash and Stackelberg equilibria for the cheap talk problem under an absolute continuity condition.

cs.IT↗
arXiv2021

Signaling Games for Log-Concave Distributions: Number of Bins and Properties of Equilibria

We investigate the equilibrium behavior for the decentralized cheap talk problem for real random variables and quadratic cost criteria in which an encoder and a decoder have misaligned objective functions. In prior work, it has been shown that the number of bins in any equilibrium has to be countable, generalizing a classical result due to Crawford and Sobel who considered sources with density supported on $[0,1]$. In this paper, we first refine this result in the context of log-concave sources. For sources with two-sided unbounded support, we prove that, for any finite number of bins, there exists a unique equilibrium. In contrast, for sources with semi-unbounded support, there may be a finite upper bound on the number of bins in equilibrium depending on certain conditions stated explicitly. Moreover, we prove that for log-concave sources, the expected costs of the encoder and the decoder in equilibrium decrease as the number of bins increases. Furthermore, for strictly log-concave sources with two-sided unbounded support, we prove convergence to the unique equilibrium under best response dynamics which starts with a given number of bins, making a connection with the classical theory of optimal quantization and convergence results of Lloyd's method. In addition, we consider more general sources which satisfy certain assumptions on the tail(s) of the distribution and we show that there exist equilibria with infinitely many bins for sources with two-sided unbounded support. Further explicit characterizations are provided for sources with exponential, Gaussian, and compactly-supported probability distributions.

cs.IT↗
arXiv2021

Robustness to Incorrect Priors and Controlled Filter Stability in Partially Observed Stochastic Control

We study controlled filter stability and its effects on the robustness properties of optimal control policies designed for systems with incorrect priors applied to a true system. Filter stability refers to the correction of an incorrectly initialized filter for a partially observed stochastic dynamical system (controlled or control-free) with increasing measurements. This problem has been studied extensively in the control-free context, and except for the standard machinery for linear Gaussian systems involving the Kalman Filter, few studies exist for the controlled setup. One of the main differences between control-free and controlled partially observed Markov chains is that the filter is always Markovian under the former, whereas under a controlled model the filter process may not be Markovian since the control policy may depend on past measurements in an arbitrary (measurable) fashion. This complicates the dependency structure and therefore results from the control-free literature do not directly apply to the controlled setup. In this paper, we study the filter stability problem for controlled stochastic dynamical systems, and provide sufficient conditions for when a falsely initialized filter merges with the correctly initialized filter over time. These stability results are applied to robust stochastic control problems: under filter stability, we bound the difference in the expected cost incurred for implementing an incorrectly designed control policy compared to an optimal policy. A conclusion is that filter stability leads to stronger robustness results to incorrect priors (compared with setups without controlled filter stability). Furthermore, if the optimum cost is that same for each prior, the cost of mismatch between the true prior and the assumed prior is zero.

math.OC↗
arXiv2021

Ergodicity of Controlled Stochastic Nonlinear Systems under Information Constraints: Refined Bounds via Splitting

This paper considers the problem of stabilizing a discrete-time non-linear stochastic system over a finite capacity noiseless channel. Our focus is on systems which decompose into a stable and unstable component, and the stability notion considered is asymptotic ergodicity of the $\mathbb{R}^N$-valued state process. We establish a necessary lower bound on channel capacity for the existence of a coding and control policy which renders the closed-loop system stochastically stable. In the literature, it has been established that under technical assumptions, the channel capacity must not be smaller than the logarithm of the determinant of the system linearization, averaged over the noise and ergodic state measures. In this paper, we establish that for systems with a stable component, it suffices to consider only the unstable dimensions, providing a refinement on the general channel capacity bound for a large class of systems. The result is established using the notion of stabilization entropy, a notion adapted from invariance entropy, used in the study of noise-free systems under information constraints.

math.OC↗
arXiv2021

Optimality of Independently Randomized Symmetric Policies for Exchangeable Stochastic Teams with Infinitely Many Decision Makers

We study stochastic team (known also as decentralized stochastic control or identical interest stochastic dynamic game) problems with large or countably infinite number of decision makers, and characterize existence and structural properties for (globally) optimal policies. We consider both static and dynamic non-convex team problems where the cost function and dynamics satisfy an exchangeability condition. To arrive at existence and structural results on optimal policies, we first introduce a topology on control policies, which involves various relaxations given the decentralized information structure. This is then utilized to arrive at a de Finetti type representation theorem for exchangeable policies. This leads to a representation theorem for policies which admit an infinite exchangeability condition. For a general setup of stochastic team problems with $N$ decision makers, under exchangeability of observations of decision makers and the cost function, we show that without loss of global optimality, the search for optimal policies can be restricted to those that are $N$-exchangeable. Then, by extending $N$-exchangeable policies to infinitely-exchangeable ones, establishing a convergence argument for the induced costs, and using the presented de Finetti type theorem, we establish the existence of an optimal decentralized policy for static and dynamic teams with countably infinite number of decision makers, which turns out to be symmetric (i.e., identical) and randomized. In particular, unlike prior work, convexity of the cost in policies is not assumed. Finally, we show near optimality of symmetric independently randomized policies for finite $N$-decision maker team problems and thus establish approximation results for $N$-decision maker weakly coupled stochastic teams.

math.OC↗
arXiv2021

Decentralized Learning for Optimality in Stochastic Dynamic Teams and Games with Local Control and Global State Information

Stochastic dynamic teams and games are rich models for decentralized systems and challenging testing grounds for multi-agent learning. Previous work that guaranteed team optimality assumed stateless dynamics, or an explicit coordination mechanism, or joint-control sharing. In this paper, we present an algorithm with guarantees of convergence to team optimal policies in teams and common interest games. The algorithm is a two-timescale method that uses a variant of Q-learning on the finer timescale to perform policy evaluation while exploring the policy space on the coarser timescale. Agents following this algorithm are "independent learners": they use only local controls, local cost realizations, and global state information, without access to controls of other agents. The results presented here are the first, to our knowledge, to give formal guarantees of convergence to team optimality using independent learners in stochastic dynamic teams and common interest games.

math.OC↗
arXiv2021

Comparison of Information Structures for Zero-Sum Games and a Partial Converse to Blackwell Ordering in Standard Borel Spaces

In statistical decision theory involving a single decision-maker, an information structure is said to be better than another one if for any cost function involving a hidden state variable and an action variable which is restricted to be conditionally independent from the state given some measurement, the solution value under the former is not worse than that under the latter. For finite spaces, a theorem due to Blackwell leads to a complete characterization on when one information structure is better than another. For stochastic games, in general, such an ordering is not possible since additional information can lead to equilibria perturbations with positive or negative values to a player. However, for zero-sum games in a finite probability space, Pęski introduced a complete characterization of ordering of information structures. In this paper, we obtain an infinite dimensional (standard Borel) generalization of Pęski's result. A corollary is that more information cannot hurt a decision maker taking part in a zero-sum game. We establish two supporting results which are essential and explicit though modest improvements on prior literature: (i) a partial converse to Blackwell's ordering in the standard Borel setup and (ii) an existence result for equilibria in zero-sum games with incomplete information.

math.OC↗
arXiv2020

Optimal Policies for Convex Symmetric Stochastic Dynamic Teams and their Mean-field Limit

This paper studies convex stochastic dynamic team problems with finite and infinite time horizons under decentralized information structures. First, we introduce two notions called exchangeable teams and symmetric information structures. We show that in convex exchangeable team problems an optimal policy exhibits a symmetry structure. We give a characterization for such symmetrically optimal teams for a general class of convex dynamic team problems under a mild conditional independence condition. In addition, through concentration of measure arguments, we establish the convergence of optimal policies for teams with $N$ decision makers to the corresponding optimal policies for symmetric mean-field teams with infinitely many decision makers. As a by-product, we present an existence result for convex mean-field teams, where the main contribution of our paper is with respect to the information structure in the system when compared with the related results in the literature that have either assumed a classical information structure or a static information structure. We also apply these results to the important special case of Linear Quadratic Gaussian (LQG) team problems, where while for partially nested LQG team problems with finite time horizons it is known that the optimal policies are linear, for infinite horizon problems the linearity of optimal policies has not been established in full generality. We also study average cost finite and infinite horizon dynamic team problems with a symmetric partially nested information structure and obtain globally optimal solutions where we establish linearity of optimal policies.

math.OC↗
arXiv2020

A Universal Dynamic Program and Refined Existence Results for Decentralized Stochastic Control

For sequential stochastic control problems with standard Borel measurement and control action spaces, we introduce a general (universally applicable) dynamic programming formulation, establish its well-posedness, and provide new existence results for optimal policies. Our dynamic program builds in part on Witsenhausen's standard form, but with a different formulation for the state, action, and transition dynamics. Using recent results on measurability properties of strategic measures in decentralized control, we obtain a standard Borel controlled Markov model. This allows for a well-defined dynamic programming recursion through universal measurability properties of the value functions for each time stage. In addition, new existence results are obtained for optimal policies in decentralized stochastic control. These state that for a static team with independent measurements, it suffices for the cost function to be continuous (only) in the actions for the existence of an optimal policy under mild compactness (or tightness) conditions. These also apply to dynamic teams which admit static reductions with independent measurements through a change of measure transformation. We show through a counterexample that weaker conditions may not lead to existence of an optimal team policy. The paper's existence results generalize those previously reported in the literature. A summary of and comparison with previously reported results and some applications are presented.

math.OC↗
arXiv2020

Optimal Solutions to Infinite-Player Stochastic Teams and Mean-Field Teams

We study stochastic static teams with countably infinite number of decision makers, with the goal of obtaining (globally) optimal policies under a decentralized information structure. We present sufficient conditions to connect the concepts of team optimality and person by person optimality for static teams with countably infinite number of decision makers. We show that under uniform integrability and uniform convergence conditions, an optimal policy for static teams with countably infinite number of decision makers can be established as the limit of sequences of optimal policies for static teams with $N$ decision makers as $N \to \infty$. Under the presence of a symmetry condition, we relax the conditions and this leads to optimality results for a large class of mean-field optimal team problems where the existing results have been limited to person-by-person-optimality and not global optimality (under strict decentralization). In particular, we establish the optimality of symmetric (i.e., identical) policies for such problems. As a further condition, this optimality result leads to an existence result for mean-field teams. We consider a number of illustrative examples where the theory is applied to setups with either infinitely many decision makers or an infinite-horizon stochastic control problem reduced to a static team.

math.OC↗
arXiv2020

Invariance Properties of Controlled Stochastic Nonlinear Systems under Information Constraints

Given a stochastic nonlinear system controlled over a possibly noisy communication channel, the paper studies the largest class of channels for which there exist coding and control policies so that the closed-loop system is stochastically stable. The stability criterion considered is asymptotic mean stationarity (AMS). We develop a general method based on ergodic theory and probability to derive fundamental bounds on information transmission requirements leading to stabilization. Through this method we develop a new notion of entropy which is tailored to derive lower bounds for asymptotic mean stationarity for both noise-free and noisy channels. The bounds obtained through probabilistic and ergodic-theoretic analysis are more refined in comparison with the bounds obtained earlier via information-theoretic methods. Moreover, our approach is more versatile in view of the models considered and allows for finer lower bounds when the AMS measure is known to admit further properties such as moment bounds.

math.OC↗