arXiv subjects

Photios A. Stavrou

Publications and source records attributed to Photios A. Stavrou.

At least 37 records · Page 2Linked to original sources

arXiv2024

Alternating Minimization Schemes for Computing Rate-Distortion-Perception Functions with $f$-Divergence Perception Constraints

We study the computation of the rate-distortion-perception function (RDPF) for discrete memoryless sources subject to a single-letter average distortion constraint and a perception constraint belonging to the family of $f$-divergences. In this setting, the RDPF forms a convex programming problem for which we characterize optimal parametric solutions. We employ the developed solutions in an alternating minimization scheme, namely Optimal Alternating Minimization (OAM), for which we provide convergence guarantees. Nevertheless, the OAM scheme does not lead to a direct implementation of a generalized Blahut-Arimoto (BA) type of algorithm due to implicit equations in the iteration's structure. To overcome this difficulty, we propose two alternative minimization approaches whose applicability depends on the smoothness of the used perception metric: a Newton-based Alternating Minimization (NAM) scheme, relying on Newton's root-finding method for the approximation of the optimal solution of the iteration, and a Relaxed Alternating Minimization (RAM) scheme, based on relaxing the OAM iterates. We show, by deriving necessary and sufficient conditions, that both schemes guarantee convergence to a globally optimal solution. We also provide sufficient conditions on the distortion and perception constraints, which guarantee that the proposed algorithms converge exponentially fast in the number of iteration steps. We corroborate our theoretical results with numerical simulations and establish connections with existing results.

cs.IT↗

arXiv2024

Goal-Oriented and Semantic Communication in 6G AI-Native Networks: The 6G-GOALS Approach

Recent advances in AI technologies have notably expanded device intelligence, fostering federation and cooperation among distributed AI agents. These advancements impose new requirements on future 6G mobile network architectures. To meet these demands, it is essential to transcend classical boundaries and integrate communication, computation, control, and intelligence. This paper presents the 6G-GOALS approach to goal-oriented and semantic communications for AI-Native 6G Networks. The proposed approach incorporates semantic, pragmatic, and goal-oriented communication into AI-native technologies, aiming to facilitate information exchange between intelligent agents in a more relevant, effective, and timely manner, thereby optimizing bandwidth, latency, energy, and electromagnetic field (EMF) radiation. The focus is on distilling data to its most relevant form and terse representation, aligning with the source's intent or the destination's objectives and context, or serving a specific goal. 6G-GOALS builds on three fundamental pillars: i) AI-enhanced semantic data representation, sensing, compression, and communication, ii) foundational AI reasoning and causal semantic data representation, contextual relevance, and value for goal-oriented effectiveness, and iii) sustainability enabled by more efficient wireless services. Finally, we illustrate two proof-of-concepts implementing semantic, goal-oriented, and pragmatic communication principles in near-future use cases. Our study covers the project's vision, methodologies, and potential impact.

eess.SP↗

arXiv2024

Copula-based Estimation of Continuous Sources for a Class of Constrained Rate-Distortion-Functions

We present a new method to estimate the rate-distortion-perception function in the perfect realism regime (PR-RDPF), for multivariate continuous sources subject to a single-letter average distortion constraint. The proposed approach is not only able to solve the specific problem but also two related problems: the entropic optimal transport (EOT) and the output-constrained rate-distortion function (OC-RDF), of which the PR-RDPF represents a special case. Using copula distributions, we show that the OC-RDF can be cast as an I-projection problem on a convex set, based on which we develop a parametric solution of the optimal projection proving that its parameters can be estimated, up to an arbitrary precision, via the solution of a convex program. Subsequently, we propose an iterative scheme via gradient methods to estimate the convex program. Lastly, we characterize a Shannon lower bound (SLB) for the PR-RDPF under a mean squared error (MSE) distortion constraint. We support our theoretical findings with numerical examples by assessing the estimation performance of our iterative scheme using the PR-RDPF with the obtained SLB for various sources.

cs.IT↗

arXiv2023

On the Computation of the Gaussian Rate-Distortion-Perception Function

In this paper, we study the computation of the rate-distortion-perception function (RDPF) for a multivariate Gaussian source under mean squared error (MSE) distortion and, respectively, Kullback-Leibler divergence, geometric Jensen-Shannon divergence, squared Hellinger distance, and squared Wasserstein-2 distance perception metrics. To this end, we first characterize the analytical bounds of the scalar Gaussian RDPF for the aforementioned divergence functions, also providing the RDPF-achieving forward "test-channel" realization. Focusing on the multivariate case, we establish that, for tensorizable distortion and perception metrics, the optimal solution resides on the vector space spanned by the eigenvector of the source covariance matrix. Consequently, the multivariate optimization problem can be expressed as a function of the scalar Gaussian RDPFs of the source marginals, constrained by global distortion and perception levels. Leveraging this characterization, we design an alternating minimization scheme based on the block nonlinear Gauss-Seidel method, which optimally solves the problem while identifying the Gaussian RDPF-achieving realization. Furthermore, the associated algorithmic embodiment is provided, as well as the convergence and the rate of convergence characterization. Lastly, for the "perfect realism" regime, the analytical solution for the multivariate Gaussian RDPF is obtained. We corroborate our results with numerical simulations and draw connections to existing results.

cs.IT↗

arXiv2023

Indirect Rate Distortion Functions with $f$-Separable Distortion Criterion

We consider a remote source coding problem subject to a {distortion function}. Contrary to the use of the classical separable distortion criterion, herein we consider the more general, $f$-separable distortion measure and study its implications on the characterization of the minimum achievable rates (also called $f$-separable indirect rate distortion function (iRDF)) under both excess and average distortion constraints. First, we provide a single-letter characterization of the optimal rates subject to an excess distortion using properties of the $f$-separable distortion. Our main result is a single-letter characterization of the $f$-separable iRDF subject to an average distortion constraint. As a consequence of the previous results, we also show a series of equalities that hold using either indirect or classical RDF under $f$-separable excess or average distortions. We corroborate our results with two application examples in which new closed-form solutions are derived, and based on these, we also recover known special cases.

cs.IT↗

arXiv2023

Computation of Rate-Distortion-Perception Function under f-Divergence Perception Constraints

In this paper, we study the computation of the rate-distortion-perception function (RDPF) for discrete memoryless sources subject to a single-letter average distortion constraint and a perception constraint that belongs to the family of f-divergences. For that, we leverage the fact that RDPF, assuming mild regularity conditions on the perception constraint, forms a convex programming problem. We first develop parametric characterizations of the optimal solution and utilize them in an alternating minimization approach for which we prove convergence guarantees. The resulting structure of the iterations of the alternating minimization approach renders the implementation of a generalized Blahut-Arimoto (BA) type of algorithm infeasible. To overcome this difficulty, we propose a relaxed formulation of the structure of the iterations in the alternating minimization approach, which allows for the implementation of an approximate iterative scheme. This approximation is shown, via the derivation of necessary and sufficient conditions, to guarantee convergence to a globally optimal solution. We also provide sufficient conditions on the distortion and the perception constraints which guarantee that our algorithm converges exponentially fast. We corroborate our theoretical results with numerical simulations, and we draw connections with existing results.

cs.IT↗

arXiv2021

Privacy signaling games with binary alphabets

In this paper, we consider a privacy signaling game problem for binary alphabets and single-bit transmission where a transmitter has a pair of messages, one of which is a casual message that needs to be conveyed, whereas the other message contains sensitive data and needs to be protected. The receiver wishes to estimate both messages to acquire as much information as possible. For this setup, we study the interactions between the transmitter and the receiver with non-aligned information-theoretic objectives (modeled by mutual information and hamming distance) due to the privacy concerns of the transmitter. We derive conditions under which Nash and/or Stackelberg equilibria exist and identify the optimal responses of the encoder and decoders strategies for each type of game. One particularly surprising result is that when both types of equilibria exist, they admit the same encoding and decoding strategies. We corroborate our analysis with simulation studies.

cs.IT↗

arXiv2021

Generalized Talagrand Inequality for Sinkhorn Distance using Entropy Power Inequality

In this paper, we study the connection between entropic optimal transport and entropy power inequality (EPI). First, we prove an HWI-type inequality making use of the infinitesimal displacement convexity of optimal transport map. Second, we derive two Talagrand-type inequalities using the saturation of EPI that corresponds to a numerical term in our expression. We evaluate for a wide variety of distributions this term whereas for Gaussian and i.i.d. Cauchy distributions this term is found in explicit form. We show that our results extend previous results of Gaussian Talagrand inequality for Sinkhorn distance to the strongly log-concave case.

cs.IT↗

arXiv2021

Secure Source Coding with Side-information at Decoder and Shared Key at Encoder and Decoder

We study the problem of rate-distortion-equivocation with side-information only available at the decoder when an independent private random key is shared between the sender and the receiver. The sender compresses the sequence, and the receiver reconstructs it such that the average distortion between the source and the output is limited. The equivocation is measured at an eavesdropper that intercepts the source encoded message, utilizing side-information correlated with the source and the side-information at the decoder. We have derived the entire achievable rate-distortion-equivocation region for this problem.

cs.IT↗

arXiv2021

Secure Block Source Coding with Sequential Encoding

We introduce fundamental bounds on achievable cumulative rate distribution functions (CRDF) to characterize a sequential encoding process that ensures lossless or lossy reconstruction subject to an average distortion criterion using a non-causal decoder. The CRDF describes the rate resources spent sequentially to compress the sequence. We also include a security constraint that affects the set of achievable CRDF. The information leakage is defined sequentially based on the mutual information between the source and its compressed representation, as it evolves. To characterize the security constraints, we introduce the concept of cumulative leakage distribution functions (CLF), which determines the allowed information leakage as distributed over encoded sub-blocks. Utilizing tools from majorization theory, we derive necessary and sufficient conditions on the achievable CRDF for a given independent and identically distributed (IID) source and CLF. One primary result of this paper is that the concave-hull of the CRDF characterizes the optimal achievable rate distribution.

cs.IT↗

arXiv2021

Adaptive Interference Coordination over Channels with Unknown State at the Encoder and the Decoder

We generalize the problem of controlling the interference created to an external observer while communicating over a discrete memoryless channel (DMC) which was studied in \cite{serrano:2014}. In particular, we consider the scenario where the transmission is established over a compound DMC channel with unknown state at both the encoder and the decoder. Depending on the exact state $s$ of the channel, we ask for a different level of average precision $\Delta_s$ on the establishment of the interference coordination with the external observer. For this set-up, we fully characterize the capacity region.

cs.IT↗

arXiv2021

Quadratic Signaling Games with Channel Combining Ratio

In this study, Nash and Stackelberg equilibria of single-stage and multi-stage quadratic signaling games between an encoder and a decoder are investigated. In the considered setup, the objective functions of the encoder and the decoder are misaligned, there is a noisy channel between the encoder and the decoder, the encoder has a soft power constraint, and the decoder has also noisy observation of the source to be estimated. We show that there exist only linear encoding and decoding strategies at the Stackelberg equilibrium, and derive the equilibrium strategies and costs. Regarding the Nash equilibrium, we explicitly characterize affine equilibria for the single-stage setup and show that the optimal encoder (resp. decoder) is affine for an affine decoder (resp. encoder) for the multi-stage setup. For the decoder side, between the information coming from the encoder and noisy observation of the source, our results describe what should be the combining ratio of these two channels. Regarding the encoder, we derive the conditions under which it is meaningful to transmit a message.

math.OC↗

arXiv2020

Remote Empirical Coordination

We apply the framework of imperfect empirical coordination to a two-node setup where the action $X$ of the first node is not observed directly but via $L$ agents who observe independently impaired measurements $\hat X$ of the action. These $L$ agents, using a rate-limited communication that is available to all of them, help the second node to generate the action $Y$ in order to establish the desired coordinated behaviour. When $L<\infty$, we prove that it suffices $R_i\geq I\left(\hat X;\hat{Y}\right)$ for at least one agent whereas for $L\longrightarrow\infty$, we show that it suffices $R_i\geq I\left(\hat X;\hat Y|X\right)$ for all agents where $\hat Y$ is a random variable such that $X-\hat X-\hat Y$ and $\|p_{X,\hat Y}\left(x,y\right)-p_{X,Y}\left(x,y\right)\|_{TV}\leq \Delta$ ( $\Delta$ is the pre-specified fidelity).

cs.IT↗

arXiv2019

Indirect NRDF for Partially Observable Gauss-Markov Processes with MSE Distortion: Complete Characterizations and Optimal Solutions

In this paper we study the problem of characterizing and computing the nonanticipative rate distortion function (NRDF) for partially observable multivariate Gauss-Markov processes with hard mean squared error (MSE) distortion constraints. For the finite time horizon case, we first derive the complete characterization of this problem and its corresponding optimal realization which is shown to be a linear functional of the current time sufficient statistic of the past and current observations signals. We show that when the problem is strictly feasible, it can be computed via semidefinite programming (SDP) algorithm. For time-varying scalar processes with average total MSE distortion we derive an optimal closed form expression by means of a dynamic reverse-waterfilling solution that we also implement via an iterative scheme that convergences linearly in finite time, and a closed-form solution under pointwise MSE distortion constraint. For the infinite time horizon, we give necessary and sufficient conditions to ensure that asymptotically the sufficient statistic process of the observation signals achieves a steady-state solution for the corresponding covariance matrices and impose conditions that allow existence of a time-invariant solution. Then, we show that when a finite solution exists in the asymptotic limit, it can be computed via SDP algorithm. We also give strong structural properties on the characterization of the problem in the asymptotic limit that allow for an optimal solution via a reverse-waterfilling algorithm that we implement via an iterative scheme that converges linearly under a finite number of spatial components. Our results are corroborated with various simulation studies and are also compared with existing results in the literature.

cs.IT↗

arXiv2019

Empirical Coordination with Multiple Descriptions

We extend the framework of empirical coordination to a distributed setup where for a given action by nature, multiple descriptions of the action of the decoder are available. We adopt the coding strategy applied by El Gamal and Cover in \cite{gamal:1982} to get a lower bound of the coordination region. Then, we improve this region by applying the coding scheme applied by Zhang and Berger in \cite{zhang:1987}.

cs.IT↗

arXiv2019

Empirical Coordination Subject to a Fidelity Criterion

We study the problem of empirical coordination subject to a fidelity criterion for a general set-up. We prove a result which indicates a strong connection between our framework and the framework of empirical coordination developed in [1]. It turns out that when we design codes that achieve empirical coordination according to a given distribution and subject to the fidelity criterion, it is sufficient to consider codes that produce actions of the same joint type for a class of types which is close enough to our desired distribution is some sense.

cs.IT↗

arXiv2019

Sequential Source Coding for Stochastic Systems Subject to Finite Rate Constraints

In this paper, we revisit the sequential source coding framework to analyze fundamental performance limitations of discrete-time stochastic control systems subject to feedback data-rate constraints in finite-time horizon. The basis of our results is a new characterization of the lower bound on the minimum total-rate achieved by sequential codes subject to a total (across time) distortion constraint and a computational algorithm that allocates optimally the rate-distortion for any fixed finite-time horizon. This characterization facilitates the derivation of analytical, non-asymptotic, and finite-dimensional lower and upper bounds in two control-related scenarios. (a) A parallel time-varying Gauss-Markov process with identically distributed spatial components that is quantized and transmitted through a noiseless channel to a minimum mean-squared error (MMSE) decoder. (b) A time-varying quantized LQG closed-loop control system, with identically distributed spatial components and with a random data-rate allocation. Our non-asymptotic lower bound on the quantized LQG control problem, reveals the absolute minimum data-rates for (mean square) stability of our time-varying plant for any fixed finite time horizon. We supplement our framework with illustrative simulation experiments.

eess.SY↗

arXiv2018

Zero-Delay Rate Distortion via Filtering for Vector-Valued Gaussian Sources

We deal with zero-delay source coding of a vector-valued Gauss-Markov source subject to a mean-squared error (MSE) fidelity criterion characterized by the operational zero-delay vector-valued Gaussian rate distortion function (RDF). We address this problem by considering the nonanticipative RDF (NRDF) which is a lower bound to the causal optimal performance theoretically attainable (OPTA) function and operational zero-delay RDF. We recall the realization that corresponds to the optimal "test-channel" of the Gaussian NRDF, when considering a vector Gauss-Markov source subject to a MSE distortion in the finite time horizon. Then, we introduce sufficient conditions to show existence of solution for this problem in the infinite time horizon. For the asymptotic regime, we use the asymptotic characterization of the Gaussian NRDF to provide a new equivalent realization scheme with feedback which is characterized by a resource allocation (reverse-waterfilling) problem across the dimension of the vector source. We leverage the new realization to derive a predictive coding scheme via lattice quantization with subtractive dither and joint memoryless entropy coding. This coding scheme offers an upper bound to the operational zero-delay vector-valued Gaussian RDF. When we use scalar quantization, then for "r" active dimensions of the vector Gauss-Markov source the gap between the obtained lower and theoretical upper bounds is less than or equal to 0.254r + 1 bits/vector. We further show that it is possible when we use vector quantization, and assume infinite dimensional Gauss-Markov sources to make the previous gap to be negligible, i.e., Gaussian NRDF approximates the operational zero-delay Gaussian RDF. We also extend our results to vector-valued Gaussian sources of any finite memory under mild conditions. Our theoretical framework is demonstrated with illustrative numerical experiments.

cs.IT↗