Non-Sequential Decentralized Stochastic Control Revisited: Causality and Static Reducibility

Abstract

In decentralized stochastic control (or stochastic team theory) and game theory, if there is a pre-defined order in a system in which agents act, the system is called \textit{sequential}, otherwise it is non-sequential. Much of the literature on stochastic control theory, such as studies on the existence analysis, approximation methods, and on dynamic programming or other analytical or learning theoretic methods, have focused on sequential systems. Many complex practical systems, however, are non-sequential where the order of agents acting is random, and dependent on the realization of solution paths and prior actions taken. The study of such systems is particularly challenging as tools applicable for sequential models are not directly applicable. In this paper, we will first revisit the notion of Causality (a definition due to Witsenhausen and which has been refined by Andersland and Tekenetzis), and provide an alternative representation using imaginary agents. We show that Causality is equivalent to Causal Implementability (and Dead-Lock Freeness), thus, generalizing previous results. We show that Causality, under an absolute continuity condition, allows for an equivalent static model whose reduction is policy-independent. Since the static reduction method for sequential control problems (via change of measures or other techniques), has been shown to be very effective in arriving at existence, structural, approximation and learning theoretic results, our analysis facilitates much of the stochastic analysis available for sequential systems to also be applicable for a class of non-sequential systems.