Searcharxiv⌕ Search

arXiv subjects

Charalambos D. Charalambous

Publications and source records attributed to Charalambos D. Charalambous.

71 records · Page 4Linked to original sources
arXiv2013

Nonanticipative Rate Distortion Function for General Source-Channel Matching

In this paper we invoke a nonanticipative information Rate Distortion Function (RDF) for sources with memory, and we analyze its importance in probabilistic matching of the source to the channel so that transmission of a symbol-by-symbol code with memory without anticipation is optimal, with respect to an average distortion and excess distortion probability. We show achievability of the symbol-by-symbol code with memory without anticipation, and we evaluate the probabilistic performance of the code for a Markov source.

cs.IT↗
arXiv2013

Team Games Optimality Conditions of Distributed Stochastic Differential Decision Systems with Decentralized Noisy Information Structures

We consider a team game reward, and we derive a stochastic Pontryagin's maximum principle for distributed stochastic differential systems with decentralized noisy information structures. Our methodology utilizes the semi martingale representation theorem, variational methods, and backward stochastic differential equations. Furthermore, we derive necessary and sufficient optimality conditions that characterize team and person-by-person optimality of decentralized strategies. Finally, we apply the stochastic maximum principle to several examples from the application areas of communications, filtering and control.

math.OC↗
arXiv2013

Centralized Versus Decentralized Team Games of Distributed Stochastic Differential Decision Systems with Noiseless Information Structures-Part II: Applications

In this second part of our two-part paper, we invoke the stochastic maximum principle, conditional Hamiltonian and the coupled backward-forward stochastic differential equations of the first part [1] to derive team optimal decentralized strategies for distributed stochastic differential systems with noiseless information structures. We present examples of such team games of nonlinear as well as linear quadratic forms. In some cases we obtain closed form expressions of the optimal decentralized strategies. Through the examples, we illustrate the effect of information signaling among the decision makers in reducing the computational complexity of optimal decentralized decision strategies.

math.OC↗
arXiv2013

Centralized Versus Decentralized Team Games of Distributed Stochastic Differential Decision Systems with Noiseless Information Structures-Part I: General Theory

Decentralized optimization of distributed stochastic differential systems has been an active area of research for over half a century. Its formulation utilizing static team and person-by-person optimality criteria is well investigated. However, the results have not been generalized to nonlinear distributed stochastic differential systems possibly due to technical difficulties inherent with decentralized decision strategies. In this first part of the two-part paper, we derive team optimality and person-by-person optimality conditions for distributed stochastic differential systems with different information structures. The optimality conditions are given in terms of a Hamiltonian system of equations described by a system of coupled backward and forward stochastic differential equations and a conditional Hamiltonian, under both regular and relaxed strategies. Our methodology is based on the semi martingale representation theorem and variational methods. Throughout the presentation we discuss similarities to optimality conditions of centralized decision making.

math.OC↗
arXiv2013

Optimal Nonstationary Reproduction Distribution for Nonanticipative RDF on Abstract Alphabets

In this paper we introduce a definition for nonanticipative Rate Distortion Function (RDF) on abstract alphabets, and we invoke weak convergence of probability measures to show various of its properties, such as, existence of the optimal reproduction conditional distribution, compactness of the fidelity set, lower semicontinuity of the RDF functional, etc. Further, we derive the closed form expression of the optimal nonstationary reproduction distribution. This expression is computed recursively backward in time. Throughout the paper we point out an operational meaning of the nonanticipative RDF by recalling the coding theorem derive in \cite{tatikonda2000}, and we state relations to Gorbunov-Pinsker's nonanticipatory $ε-$entropy \cite{gorbunov-pinsker}.

cs.IT↗
arXiv2013

Variable Length Lossless Coding for Variational Distance Class: An Optimal Merging Algorithm

In this paper we consider lossless source coding for a class of sources specified by the total variational distance ball centred at a fixed nominal probability distribution. The objective is to find a minimax average length source code, where the minimizers are the codeword lengths -- real numbers for arithmetic or Shannon codes -- while the maximizers are the source distributions from the total variational distance ball. Firstly, we examine the maximization of the average codeword length by converting it into an equivalent optimization problem, and we give the optimal codeword lenghts via a waterfilling solution. Secondly, we show that the equivalent optimization problem can be solved via an optimal partition of the source alphabet, and re-normalization and merging of the fixed nominal probabilities. For the computation of the optimal codeword lengths we also develop a fast algorithm with a computational complexity of order ${\cal O}(n)$.

cs.IT↗
arXiv2013

Extremum Problems with Total Variation Distance and their Applications

The aim of this paper is to investigate extremum problems with pay-off being the total variational distance metric defined on the space of probability measures, subject to linear functional constraints on the space of probability measures, and vice-versa; that is, with the roles of total variational metric and linear functional interchanged. Utilizing concepts from signed measures, the extremum probability measures of such problems are obtained in closed form, by identifying the partition of the support set and the mass of these extremum measures on the partition. The results are derived for abstract spaces; specifically, complete separable metric spaces known as Polish spaces, while the high level ideas are also discussed for denumerable spaces endowed with the discrete topology. These extremum problems often arise in many areas, such as, approximating a family of probability distributions by a given probability distribution, maximizing or minimizing entropy subject to total variational distance metric constraints, quantifying uncertainty of probability distributions by total variational distance metric, stochastic minimax control, and in many problems of information, decision theory, and minimax theory.

math.OC↗
arXiv2012

Optimal Merging Algorithms for Lossless Codes with Generalized Criteria

This paper presents lossless prefix codes optimized with respect to a pay-off criterion consisting of a convex combination of maximum codeword length and average codeword length. The optimal codeword lengths obtained are based on a new coding algorithm which transforms the initial source probability vector into a new probability vector according to a merging rule. The coding algorithm is equivalent to a partition of the source alphabet into disjoint sets on which a new transformed probability vector is defined as a function of the initial source probability vector and a scalar parameter. The pay-off criterion considered encompasses a trade-off between maximum and average codeword length; it is related to a pay-off criterion consisting of a convex combination of average codeword length and average of an exponential function of the codeword length, and to an average codeword length pay-off criterion subject to a limited length constraint. A special case of the first related pay-off is connected to coding problems involving source probability uncertainty and codeword overflow probability, while the second related pay-off compliments limited length Huffman coding algorithms.

cs.IT↗
arXiv2012

Directed Information on Abstract spaces: Properties and Extremum Problems

This paper describes a framework in which directed information is defined on abstract spaces. The framework is employed to derive properties of directed information such as convexity, concavity, lower semicontinuity, by using the topology of weak convergence of probability measures on Polish spaces. Two extremum problems of directed information related to capacity of channels with memory and feedback, and non-anticipative and sequential rate distortion are analyzed showing existence of maximizing and minimizing distributions, respectively.

cs.IT↗
arXiv2012

Realizable Rate Distortion Function and Bayesian FIltering Theory

The relation between rate distortion function (RDF) and Bayesian filtering theory is discussed. The relation is established by imposing a causal or realizability constraint on the reconstruction conditional distribution of the RDF, leading to the definition of a causal RDF. Existence of the optimal reconstruction distribution of the causal RDF is shown using the topology of weak convergence of probability measures. The optimal non-stationary causal reproduction conditional distribution of the causal RDF is derived in closed form; it is given by a set of recursive equations which are computed backward in time. The realization of causal RDF is described via the source-channel matching approach, while an example is briefly discussed to illustrate the concepts.

cs.IT↗
arXiv2012

Causal Rate Distortion Function on Abstract Alphabets: Optimal Reconstruction and Properties

A causal rate distortion function with a general fidelity criterion is formulated on abstract alphabets and a coding theorem is derived. Existence of the minimizing kernel is shown using the topology of weak convergence of probability measures. The optimal reconstruction kernel is derived, which is causal, and certain properties of the causal rate distortion function are presented.

cs.IT↗
arXiv2011

Minimum Redundancy Coding for Uncertain Sources

Consider the set of source distributions within a fixed maximum relative entropy with respect to a given nominal distribution. Lossless source coding over this relative entropy ball can be approached in more than one way. A problem previously considered is finding a minimax average length source code. The minimizing players are the codeword lengths --- real numbers for arithmetic codes, integers for prefix codes --- while the maximizing players are the uncertain source distributions. Another traditional minimizing objective is the first one considered here, maximum (average) redundancy. This problem reduces to an extension of an exponential Huffman objective treated in the literature but heretofore without direct practical application. In addition to these, this paper examines the related problem of maximal minimax pointwise redundancy and the problem considered by Gawrychowski and Gagie, which, for a sufficiently small relative entropy ball, is equivalent to minimax redundancy. One can consider both Shannon-like coding based on optimal real number ("ideal") codeword lengths and a Huffman-like optimal prefix coding.

cs.IT↗
arXiv2011

Causal Rate Distortion Function on Abstract Alphabets and Optimal Reconstruction Kernel

A Causal rate distortion function with a general fidelity criterion is formulated on abstract alphabets and the optimal reconstruction kernel is derived, which consists of a product of causal kernels. In the process, general abstract spaces are introduced to show existence of the minimizing kernel using weak*-convergence. Certain properties of the causal rate distortion function are presented.

cs.IT↗
arXiv2011

Lossless Coding with Generalised Criteria

This paper presents prefix codes which minimize various criteria constructed as a convex combination of maximum codeword length and average codeword length or maximum redundancy and average redundancy, including a convex combination of the average of an exponential function of the codeword length and the average redundancy. This framework encompasses as a special case several criteria previously investigated in the literature, while relations to universal coding is discussed. The coding algorithm derived is parametric resulting in re-adjusting the initial source probabilities via a weighted probability vector according to a merging rule. The level of desirable merging has implication in applications where the maximum codeword length is bounded.

cs.IT↗
arXiv2011

Compound Outage Probability and Capacity of a Class of Fading MIMO Channels with Channel Distribution Uncertainty

Outage probability and capacity of a class of block-fading MIMO channels are considered with partial channel distribution information. Specifically, the channel or its distribution are not known but the latter is known to belong to a class of distributions where each member is within a certain distance (uncertainty) from a nominal distribution. Relative entropy is used as a measure of distance between distributions. Compound outage probability defined as min (over the transmit signal distribution) -max (over the channel distribution class) outage probability is introduced and investigated. This generalizes the standard outage probability to the case of partial channel distribution information. Compound outage probability characterization (via one-dimensional convex optimization), its properties and approximations are given. It is shown to have two-regime behavior: when the nominal outage probability decreases (e.g. by increasing the SNR), the compound outage first decreases linearly down to a certain threshold (related to relative entropy distance) and then only logarithmically (i.e. very slowly), so that no significant further decrease is possible. The compound outage depends on the relative entropy distance and the nominal outage only, all other details (nominal fading and noise distributions) being irrelevant. The transmit signal distribution optimized for the nominal channel distribution is shown to be also optimal for the whole class of distributions. The effect of swapping the distributions in relative entropy is investigated and an error floor effect is established. The compound outage probability under Lp distance constraint is also investigated. The obtained results hold for a generic channel model (arbitrary nominal fading and noise distributions).

cs.IT↗