arXiv subjects

Charalambos D. Charalambous

Publications and source records attributed to Charalambos D. Charalambous.

At least 19 recordsLinked to original sources

arXiv2026

Information-Theoretic Meta Dynamic Programming for Signalling and Control of POMDPs

In this paper, we study the information-theoretic characterization of simultaneous signalling and control over channels modeled by partially observable Markov decision processes (POMDPs). The problem is formulated as an optimization over randomized control strategies that maximize the directed information from actions to observations, subject to an average-cost constraint. We derive a novel dynamic programming framework in which the state is defined on the space of conditional probability distributions, leading to a high-level ``meta'' dynamic program. Specifically, we show that two coupled information states, namely, the posterior distribution of the system state and a distribution over such posteriors, satisfy Markov recursions and provide sufficient statistics for optimal control. This structure enables the decomposition of optimal strategies into separated randomized policies that depend only on these information states. Our results establish necessary and sufficient conditions for optimality and unify classical stochastic control and information-theoretic formulations. In particular, we show that in the absence of signalling, the proposed framework reduces to the standard dynamic programming equations for POMDPs. The developed approach provides a principled foundation for analyzing and designing control systems with intrinsic information constraints.

cs.IT↗

arXiv2026

Private & Common Information States in Decentralized Team Equilibrium via Dynamic Programming for POMDPs with Delayed Sharing

Witsenhausen, in his seminal 1971 paper [1], introduced decentralized partially observable Markov decision problems (POMDPs), with multiple agents or controls operating under T-step delayed sharing information patterns. A fundamental problem in [1] is the identification of structural properties of optimal strategies that compress the information patterns into multiple information states. In this paper, we develop such structural properties of optimal strategies and associated dynamic programming (DP) equations, using the concept of decentralized sequential team equilibrium (a generalization of person-by-person optimality from static team theory). Within this framework, each strategy is assigned an individual value function conditioned on its delayed sharing information pattern, while the strategies of all other agents are held fixed. The resulting DP framework yields several new DP equations and characterizations of decentralized team equilibrium. Moreover, these DP equations exhibit fundamental properties analogous to those of centralized DP of POMDPs: the optimization in each agent's DP equations is performed over the agent's action space rather than over strategy spaces; each agent's multiple information states satisfy Markov recursions; and a separation principle holds. The DP equations reveal a structural compression property of optimal strategies: each agent compresses its delayed sharing information pattern into three components: 1) a private posterior distribution conditioned on the agent's delayed sharing information pattern, 2) a centralized posterior distribution conditioned on the common information shared by all agents, and 3) the agent's private information component. This structural result substantially extends Witsenhausen's Assertion 8 in [1].

eess.SY↗

arXiv2026

Comments and Corrections on the DP Equations of Paper "On Team Decision Problems With Nonclassical Information Structures"

The 2023 paper ``On Team Decision Problems With Nonclassical Information Structures'' [1] presented information states and dynamic programming (DP) equations for delayed sharing information patterns, based on the concept of person-by-person (PbP) optimality of static team theory. In particular, [Section IV, 1], Theorem 5 presents recursions of the information states and Theorem 7, eqn(57), eqn(58), presents DP equations, of each team member. In this note we show that the proof of Theorem 5 and the DP eqn(57), eqn(58) of Theorem 7 of [1] are incorrect. Consequently, Lemma 8, Theorem 6, and the full statement of Theorem 7 in [1] are invalid, because their proofs rely on erroneous information states and the optimality of separated strategies (i.e., functions of the information states). We further provide the correct DP equations for PbP optimality, thereby highlighting the subtleties and challenges inherent in the analysis of delayed sharing information patterns.

math.OC↗

arXiv2026

Private and Common Information States in Decentralized Parallel Dynamic Programming for Delayed Sharing Patterns

This paper develops a dynamic programming (DP) approach for decentralized stochastic optimal control problems with delayed sharing information patterns, which exhibits the fundamental Properties of classical DP of centralized partially observable Markov decision problems (POMDPs): the value functions and information states depend on the actions of the minimizing controls and not their strategies. This is achieved by invoking the concept of Person-by-Person (PbP) optimality, in which each control strategy is associated with a value function conditioned on its assigned delayed sharing information pattern, when all other strategies are fixed to their optimal responses. The value functions satisfy generalized and simplified DP equations. These are used to derive necessary and sufficient conditions for PbP optimality. The simplified DP equations are obtained by invoking the structural property that optimal strategies are separated and functionals of two information states: 1) a private a posteriori probability distribution based on the information pattern of the strategy, and 2) a centralized a posteriori probability distribution based on the shared or common information to all strategies, each satisfying a Markov recursion. The DP approach of this paper, settles a long standing open problem since the appearance of T-step delayed sharing patterns in [1, Section IV.G], in terms of generalizing the fundamental properties of classical DP approach.

eess.SY↗

arXiv2025

General Decentralized Stochastic Optimal Control via Change of Measure: Applications to the Witsenhausen Counterexample

In this paper we present global and person-by-person (PbP) optimality conditions for general decentralized stochastic dynamic optimal control problems, using a discrete-time version of Girsanov's change of measure. The PbP optimality conditions are applied to the Witsenhausen counterexample to show that the two strategies satisfy two coupled nonlinear integral equations. Further, we prove a fixed point theorem in a function space, establishing existence and uniqueness of solutions to the integral equations. We also provide numerical solutions of the two integral equations using the Gauss Hermite Quadrature scheme, and include a detail comparison to other numerical methods of the literature. The numerical solutions confirm Witsehausen's observation that, for certain choices of parameters, linear or affine strategies are optimal, while for other choices of parameters nonlinear strategies outperformed affine strategies.

eess.SY↗

arXiv2025

Remote Estimation for Markov Jump Linear Systems: A Distributionally Robust Approach

This paper considers the problem of remote state estimation for Markov jump linear systems in the presence of uncertainty in the posterior mode probabilities. Such uncertainty may arise when the estimator receives noisy or incomplete measurements over an unreliable communication network. To address this challenge, the estimation problem is formulated within a distributionally robust framework, where the true posterior is assumed to lie within a total variation distance ball centered at the nominal posterior. The resulting minimax formulation yields an estimator that extends the classical MMSE solution with additional terms that account for mode uncertainty. A tractable implementation is developed using a distributionally robust variant of the first-order generalized pseudo-Bayesian algorithm. A numerical example is provided to illustrate the applicability and effectiveness of the approach.

eess.SY↗

arXiv2025

Rollout-Based Approximate Dynamic Programming for MDPs with Information-Theoretic Constraints

This paper studies a finite-horizon Markov decision problem with information-theoretic constraints, where the goal is to minimize directed information from the controlled source process to the control process, subject to stage-wise cost constraints, aiming for an optimal control policy. We propose a new way of approximating a solution for this problem, which is known to be formulated as an unconstrained MDP with a continuous information-state using Q-factors. To avoid the computational complexity of discretizing the continuous information-state space, we propose a truncated rollout-based backward-forward approximate dynamic programming (ADP) framework. Our approach consists of two phases: an offline base policy approximation over a shorter time horizon, followed by an online rollout lookahead minimization, both supported by provable convergence guarantees. We supplement our theoretical results with a numerical example where we demonstrate the cost improvement of the rollout method compared to a previously proposed policy approximation method, and the computational complexity observed in executing the offline and online phases for the two methods.

eess.SY↗

arXiv2025

Implicit and Explicit Formulas of the Joint RDF for a Tuple of Multivariate Gaussian Sources with Individual Square-Error Distortions

This paper analyzes the joint Rate Distortion Function (RDF) of correlated multivariate Gaussian sources with individual square-error distortions. Leveraging Hotelling's canonical variable form, presented is a closed-form characterization of the joint RDF, that involves {a system of nonlinear equations. Furthermore, for the special case of symmetric distortions (i.e., equal distortions), the joint RDF is explicitly expressed in terms of} two water-filling variables. The results greatly improve our understanding and advance the development of closed-form solutions of the joint RDF for multivariate Gaussian sources with individual square-error distortions.

cs.IT↗

arXiv2025

Remote Estimation over Packet-Dropping Wireless Channels with Partial State Information

In this paper, we study the design of an optimal transmission policy for remote state estimation over packet-dropping wireless channels with imperfect channel state information. A smart sensor uses a Kalman filter to estimate the system state and transmits its information to a remote estimator. Our objective is to minimize the state estimation error and energy consumption by deciding whether to transmit new information or retransmit previously failed packets. To balance the trade-off between information freshness and reliability, the sensor applies a hybrid automatic repeat request protocol. We formulate this problem as a finite horizon partially observable Markov decision process with an augmented state-space that incorporates both the age of information and the unknown channel state. By defining an information state, we derive the dynamic programming equations for evaluating the optimal policy. This transmission policy is computed numerically using the point-based value iteration algorithm.

eess.SY↗

arXiv2024

A New Finite-Horizon Dynamic Programming Analysis of Nonanticipative Rate-Distortion Function for Markov Sources

This paper deals with the computation of a non-asymptotic lower bound by means of the nonanticipative rate-distortion function (NRDF) on the discrete-time zero-delay variable-rate lossy compression problem for discrete Markov sources with per-stage, single-letter distortion. First, we derive a new information structure of the NRDF for Markov sources and single-letter distortions. Second, we derive new convexity results on the NRDF, which facilitate the use of Lagrange duality theorem to cast the problem as an unconstrained partially observable finite-time horizon stochastic dynamic programming (DP) algorithm subject to a probabilistic state (belief state) that summarizes the past information about the reproduction symbols and takes values in a continuous state space. Instead of approximating the DP algorithm directly, we use Karush-Kuhn-Tucker (KKT) conditions to find an implicit closed-form expression of the optimal control policy of the stochastic DP (i.e., the minimizing distribution of the NRDF) and approximate the control policy and the cost-to-go function (a function of the rate) stage-wise, via a novel dynamic alternating minimization (AM) approach, that is realized by an offline algorithm operating using backward recursions, with provable convergence guarantees. We obtain the clean values of the aforementioned quantities using an online (forward) algorithm operating for any finite-time horizon. Our methodology provides an approximate solution to the exact NRDF solution, which becomes near-optimal as the search space of the belief state becomes sufficiently large at each time stage. We corroborate our theoretical findings with simulation studies where we apply our algorithms assuming time-varying and time-invariant binary Markov processes.

cs.IT↗

arXiv2024

Indirect Rate Distortion Functions with Side Information: Structural Properties and Multivariate Gaussian Sources

In this paper, we analyze the indirect source coding problem with side information at both the encoder and decoder, as well as only at the decoder. We first derive structural properties of the two rate distortion functions (RDFs) for general abstract spaces and identify conditions under which the RDFs coincide. For multivariate jointly Gaussian random variables with square-error fidelity, we establish structural properties of the optimal test channels, show that side information at both the encoder and decoder does not reduce compression, and provide water-filling solutions using parallel Gaussian channel realizations. This paper uses a novel realization theory approach to establish achievability of the converse coding theorem lower bounds of the two RDFs.

cs.IT↗

arXiv2024

Signalling and Control in Nonlinear Stochastic Systems: An Information State Approach with Applications

We consider optimal signalling and control of discrete-time nonlinear partially observable stochastic systems in state space form. In the first part of the paper, we characterize the operational {\it control-coding capacity}, $C_{FB}$ in bits/second, by an information theoretic optimization problem of encoding signals or messages into randomized controller-encoder strategies, and reproducing the messages at the output of the system using a decoder or estimator with arbitrary small asymptotic error probability. Our analysis of $C_{FB}$ is based on realizations of randomized strategies (controller-encoders), in terms of information states of nonlinear filtering theory, and either uniform or arbitrary distributed random variables (RVs). In the second part of the paper, we analyze the linear-quadratic Gaussian partially observable stochastic system (LQG-POSS). We show that simultaneous signalling and control leads to randomized strategies described by finite-dimensional sufficient statistics, that involve two Kalman-filters, and consist of control, estimation and signalling strategies. We apply decentralized optimization techniques to prove a separation principle, and to derive the optimal control part of randomized strategies explicitly in terms of a control matrix difference Riccati equation (DRE).

cs.IT↗

arXiv2022

Characterization of the Gray-Wyner Rate Region for Multivariate Gaussian Sources: Optimality of Gaussian Auxiliary RV

Examined in this paper, is the Gray and Wyner achievable lossy rate region for a tuple of correlated multivariate Gaussian random variables (RVs) $X_1 : Ω\rightarrow {\mathbb R}^{p_1}$ and $X_2 : Ω\rightarrow {\mathbb R}^{p_2}$ with respect to square-error distortions at the two decoders. It is shown that among all joint distributions induced by a triple of RVs $(X_1,X_2, W)$, such that $W : Ω\rightarrow {\mathbb W} $ is the auxiliary RV taking continuous, countable, or finite values, the Gray and Wyner achievable rate region is characterized by jointly Gaussian RVs $(X_1,X_2, W)$ such that $W $ is an $n$-dimensional Gaussian RV. It then follows that the achievable rate region is parametrized by the three conditional covariances $Q_{X_1,X_2|W}, Q_{X_1|W}, Q_{X_2|W}$ of the jointly Gaussian RVs. Furthermore, if the RV $W$ makes $X_1$ and $X_2$ conditionally independent, then the corresponding subset of the achievable rate region, is simpler, and parametrized by only the two conditional covariances $Q_{X_1|W}, Q_{X_2|W}$. The paper also includes the characterization of the Pangloss plane of the Gray-Wyner rate region along with the characterizations of the corresponding rate distortion functions, their test-channel distributions, and structural properties of the realizations which induce these distributions.

cs.IT↗

arXiv2022

On the Capacity of Gaussian MIMO Channels with Memory

The operational capacity of Gaussian MIMO channels with memory was obtained by Brandenburg and Wyner in [9] under certain mild assumptions on the channel impulse response and its noise covariance matrix, which essentuially require channel memory to be not too strong. This channel was also considered by Tsybakov in [10] and its information capacity was obtained in some cases. It was further conjectured, based on numerical evidence, that these capacities are the same in all cases. This conjecture is proved here. An explicit closed-form expression for the optimal input power spectral density matrix is also given. The obtained result is further extended to the case of joint constraints, including per-antenna and interference power constraints as well as energy harvesting constraints. These results imply the information-theoretic optimality of OFDM-type transmission systems for such channels with memory.

cs.IT↗

arXiv2022

A Riccati-Lyapunov Approach to Nonfeedback Capacity of MIMO Gaussian Channels Driven by Stable and Unstable Noise

In this paper it is shown that the nonfeedback capacity of multiple-input multiple-output (MIMO) additive Gaussian noise (AGN) channels, when the noise is nonstationary and unstable, is characterized by an asymptotic optimization problem that involves, a generalized matrix algebraic Riccati equation (ARE) of filtering theory, and a matrix Lyapunov equation of stability theory of Gaussian systems. Furthermore, conditions are identified such that, the characterization of nonfeedback capacity corresponds to the uniform asymptotic per unit time limit, over all initial distributions, of the characterization of a finite block or transmission without feedback information (FTwFI) capacity, which involves, two generalized matrix difference Riccati equations (DREs) and a matrix difference Lyapunov equation.

cs.IT↗

arXiv2021

Time-Invariant Feedback Strategies Do Not Increase Capacity of AGN Channels Driven by Stable and Certain Unstable Autoregressive Noise

The capacity of additive Gaussian noise (AGN) channels, $Y_t=X_t+V_t, t=1, \ldots, n$, $\frac{1}{n} {\bf E}\big\{\sum_{t=1}^n |X_t|^2 \big\}\leq κ, κ\in [0,\infty)$, with time-invariant channel input feedback strategies, is characterized and conditions are identified for entropy rates, and limit of average power to exist, when the noise is described by {\it stable and unstable} autoregressive models, AR$(c)$, $V_t=cV_{t-1}+ W_t, V_0=v_0, t=1, \ldots, n$, where $c\in (-\infty,\infty)$, $W_t, t=1,\ldots, n$, is a zero mean, variance $K_W$, independent Gaussian sequence, independent of $V_0$. For stable AR$(c), c\in (-1,1)$ the conditions are necessary and sufficient for asymptotic stationarity of the processes $(X_t, Y_t), t=1, 2, \ldots$. New closed form capacity formulas and lower bounds are derived, for the AR$(c), c\in (-\infty,\infty)$ noise, which are fundamentally different from existing formulas in the literature, and illustrate multiple regimes of capacity, as a function of the parameters $(c,K_W,κ)$, as follows.\\ 1) feedback increases capacity for the regime, $c^2 \in (1, \infty),$ for $κ> \frac{K_W\big(1+\sqrt{4c^2-3}\big)}{2\big(c^2-1\big)^2}$, \\ 2) feedback does not increase capacity for the regime $c^2 \in (1, \infty)$, for $κ\leq \frac{K_W\big(1+\sqrt{4c^2-3}\big)}{2\big(c^2-1\big)^2}$, and \\ 3) feedback does not increase capacity for the regime $c \in [-1,1]$, for $ κ\in [0,\infty)$.

cs.IT↗

arXiv2021

On the Fragile Rates of Linear Feedback Coding Schemes of Gaussian Channels with Memory

In \cite{butman1976} the linear coding scheme is applied, $X_t =g_t\Big(Θ- {\bf E}\Big\{Θ\Big|Y^{t-1}, V_0=v_0\Big\}\Big)$, $t=2,\ldots,n$, $X_1=g_1Θ$, with $Θ: Ω\to {\mathbb R}$, a Gaussian random variable, to derive a lower bound on the feedback rate, for additive Gaussian noise (AGN) channels, $Y_t=X_t+V_t, t=1, \ldots, n$, where $V_t$ is a Gaussian autoregressive (AR) noise, and $κ\in [0,\infty)$ is the total transmitter power. For the unit memory AR noise, with parameters $(c, K_W)$, where $c\in [-1,1]$ is the pole and $K_W$ is the variance of the Gaussian noise, the lower bound is $C^{L,B} =\frac{1}{2} \log χ^2$, where $χ=\lim_{n\longrightarrow \infty} χ_n$ is the positive root of $χ^2=1+\Big(1+ \frac{|c|}χ\Big)^2 \fracκ{K_W}$, and the sequence $χ_n \triangleq \Big|\frac{g_n}{g_{n-1}}\Big|, n=2, 3, \ldots,$ satisfies a certain recursion, and conjectured that $C^{L,B}$ is the feedback capacity. In this correspondence, it is observed that the nontrivial lower bound $C^{L,B}=\frac{1}{2} \log χ^2$ such that $χ>1$, necessarily implies the scaling coefficients of the feedback code, $g_n$, $n=1,2, \ldots$, grow unbounded, in the sense that, $\lim_{n\longrightarrow\infty}|g_n| =+\infty$. The unbounded behaviour of $g_n$ follows from the ratio limit theorem of a sequence of real numbers, and it is verified by simulations. It is then concluded that such linear codes are not practical, and fragile with respect to a mismatch between the statistics of the mathematical model of the channel and the real statistics of the channel. In particular, if the error is perturbed by $ε_n>0$ no matter how small, then $X_n =g_t\Big(Θ- {\bf E}\Big\{Θ\Big|Y^{t-1}, V_0=v_0\Big\}\Big)+g_n ε_n$, and $|g_n|ε_n \longrightarrow \infty$, as $n \longrightarrow \infty$.

cs.IT↗

arXiv2021

Feedback Capacity Formulas of AGN Channels Driven by Nonstationary Autoregressive Moving Average Noise

In this paper we derive closed-form formulas of feedback capacity and nonfeedback achievable rates, for Additive Gaussian Noise (AGN) channels driven by nonstationary autoregressive moving average (ARMA) noise (with unstable one poles and zeros), based on time-invariant feedback codes and channel input distributions. From the analysis and simulations follows the surprising observations, (i) the use of time-invariant channel input distributions gives rise to multiple regimes of capacity that depend on the parameters of the ARMA noise, which may or may not use feedback, (ii) the more unstable the pole (resp. zero) of the ARMA noise the higher (resp. lower) the feedback capacity, (iii) certain conditions, known as detectability and stabilizability are necessary and sufficient to ensure the feedback capacity formulas and nonfeedback achievable rates {\it are independent of the initial state of the ARMA noise}. Another surprizing observation is that Kim's \cite{kim2010} characterization of feedback capacity which is developed for stable ARMA noise, if applied to the unstable ARMA noise, gives a lower value of feedback capacity compared to our feedback capacity formula.

cs.IT↗