Searcharxiv⌕ Search

arXiv subjects

Fady Alajaji

Publications and source records attributed to Fady Alajaji.

At least 55 records · Page 3Linked to original sources

Lossy Transmission of Correlated Sources over Two-Way Channels

Achievability and converse results for the lossy transmission of correlated sources over Shannon's two-way channels (TWCs) are presented. A joint source-channel coding theorem for independent sources and TWCs for which adaptation cannot enlarge the capacity region is also established. We further investigate the optimality of scalar coding for TWCs with discrete modulo additive noise as well as additive white Gaussian noise. Comparing the distortion of scalar coding with the derived bounds, we observe that scalar coding achieves the minimum distortion over both families of TWCs for independent and uniformly distributed sources and independent Gaussian sources.

cs.IT↗

On the Noise-Information Separation of a Private Principal Component Analysis Scheme

In a survey disclosure model, we consider an additive noise privacy mechanism and study the trade-off between privacy guarantees and statistical utility. Privacy is approached from two different but complementary viewpoints: information and estimation theoretic. Motivated by the performance of principal component analysis, statistical utility is measured via the spectral gap of a certain covariance matrix. This formulation and its motivation rely on classical results from random matrix theory. We prove some properties of this statistical utility function and discuss a simple numerical method to evaluate it.

cs.IT↗

A Polya Contagion Model for Networks

A network epidemics model based on the classical Polya urn scheme is investigated. Temporal contagion processes are generated on the network nodes using a modified Polya sampling scheme that accounts for spatial infection among neighbouring nodes. The stochastic properties and the asymptotic behaviour of the resulting network contagion process are analyzed. Unlike the classical Polya process, the network process is noted to be non-stationary in general, although it is shown to be time-invariant in its first and some of its second-order statistics and to satisfy martingale convergence properties under certain conditions. Three classical Polya processes, one computational and two analytical, are proposed to statistically approximate the contagion process of each node, showing a good fit for a range of system parameters. Finally, empirical results compare and contrast our model with the well-known discrete time SIS model.

cs.SI↗

Curing Epidemics on Networks using a Polya Contagion Model

We study the curing of epidemics of a network contagion, which is modelled using a variation of the classical Polya urn process that takes into account spatial infection among neighbouring nodes. We introduce several quantities for measuring the overall infection in the network and use them to formulate an optimal control problem for minimizing the average infection rate using limited curing resources. We prove the feasibility of this problem under high curing budgets by deriving conservative lower bounds on the amount of curing per node that turns our measures of network infection into supermartingales. We also provide a provably convergent gradient descent algorithm to find the allocation of curing under limited budgets. Motivated by the fact that this strategy is computationally expensive, we design a suit of heuristic methods that are locally implementable and nearly as effective. Extensive simulations run on largescale networks demonstrate the effectiveness of our proposed strategies.

math.OC↗

A Short Survey on Bounding the Union Probability using Partial Information

This is a short survey on existing upper and lower bounds on the probability of the union of a finite number of events using partial information given in terms of the individual or pairwise event probabilities (or their sums). New proofs for some of the existing bounds are provided and new observations regarding the existing Gallot--Kounias bound are given.

math.PR↗

Optimized Signaling of Binary Correlated Sources over GMACs

This work focuses on the construction of optimized binary signaling schemes for two-sender uncoded transmission of correlated sources over non-orthogonal Gaussian multiple access channels. Specifically, signal constellations with binary pulse-amplitude-modulation are designed for two senders to optimize the overall system performance. Although the two senders transmit their own messages independently, it is observed that the correlation between message sources can be exploited to mitigate the interference present in the non-orthogonal multiple access channel. Based on a performance analysis under joint maximum-a-posteriori decoding, optimized constellations for various basic waveform correlations between the senders are derived. Numerical results further confirm the effectiveness of the proposed design.

cs.IT↗

Capacity of Burst Noise-Erasure Channels With and Without Feedback and Input Cost

A class of burst noise-erasure channels which incorporate both errors and erasures during transmission is studied. The channel, whose output is explicitly expressed in terms of its input and a stationary ergodic noise-erasure process, is shown to have a so-called "quasi-symmetry" property under certain invertibility conditions. As a result, it is proved that a uniformly distributed input process maximizes the channel's block mutual information, resulting in a closed-form formula for its non-feedback capacity in terms of the noise-erasure entropy rate and the entropy rate of an auxiliary erasure process. The feedback channel capacity is also characterized, showing that feedback does not increase capacity and generalizing prior related results. The capacity-cost function of the channel with and without feedback is also investigated. A sequence of finite-letter upper bounds for the capacity-cost function without feedback is derived. Finite-letter lower bonds for the capacity-cost function with feedback are obtained using a specific encoding rule. Based on these bounds, it is demonstrated both numerically and analytically that feedback can increase the capacity-cost function for a class of channels with Markov noise-erasure processes.

cs.IT↗

Privacy-Aware Guessing Efficiency

We investigate the problem of guessing a discrete random variable $Y$ under a privacy constraint dictated by another correlated discrete random variable $X$, where both guessing efficiency and privacy are assessed in terms of the probability of correct guessing. We define $h(P_{XY}, ε)$ as the maximum probability of correctly guessing $Y$ given an auxiliary random variable $Z$, where the maximization is taken over all $P_{Z|Y}$ ensuring that the probability of correctly guessing $X$ given $Z$ does not exceed $ε$. We show that the map $ε\mapsto h(P_{XY}, ε)$ is strictly increasing, concave, and piecewise linear, which allows us to derive a closed form expression for $h(P_{XY}, ε)$ when $X$ and $Y$ are connected via a binary-input binary-output channel. For $(X^n, Y^n)$ being pairs of independent and identically distributed binary random vectors, we similarly define $\underline{h}_n(P_{X^nY^n}, ε)$ under the assumption that $Z^n$ is also a binary vector. Then we obtain a closed form expression for $\underline{h}_n(P_{X^nY^n}, ε)$ for sufficiently large, but nontrivial values of $ε$.

cs.IT↗

Almost Perfect Privacy for Additive Gaussian Privacy Filters

We study the maximal mutual information about a random variable $Y$ (representing non-private information) displayed through an additive Gaussian channel when guaranteeing that only $ε$ bits of information is leaked about a random variable $X$ (representing private information) that is correlated with $Y$. Denoting this quantity by $g_ε(X,Y)$, we show that for perfect privacy, i.e., $ε=0$, one has $g_0(X,Y)=0$ for any pair of absolutely continuous random variables $(X,Y)$ and then derive a second-order approximation for $g_ε(X,Y)$ for small $ε$. This approximation is shown to be related to the strong data processing inequality for mutual information under suitable conditions on the joint distribution $P_{XY}$. Next, motivated by an operational interpretation of data privacy, we formulate the privacy-utility tradeoff in the same setup using estimation-theoretic quantities and obtain explicit bounds for this tradeoff when $ε$ is sufficiently small using the approximation formula derived for $g_ε(X,Y)$.

cs.IT↗

Adaptation is Useless for Two Discrete Additive-Noise Two-Way Channels

In two-way channels, each user transmits and receives at the same time. This allows each encoder to interactively adapt the current input to its own message and all previously received signals. Such coding approach can introduce correlation between inputs of different users, since all the users' outputs are correlated by the nature of the channel. However, for some channels, such adaptation in the coding scheme and its induced correlation among users are useless in the sense that they do not help enlarge the capacity region with respect to the standard coding method (where each user encodes only based on its own message). In this paper, it is shown that adaptation is not helpful for enlarging the capacity region of two classes of two-way discrete channels: the modulo additive-noise channel with memory and the multiple access/degraded broadcast channel.

cs.IT↗

On Bounding the Union Probability Using Partial Weighted Information

Effective bounds on the union probability are well known to be beneficial in the analysis of stochastic problems in many areas, including probability theory, information theory, statistical communications, computing and operations research. In this work we present new results on bounding the probability of a finite union of events, ${P\left(\bigcup_{i=1}^N A_i\right)}$, for a fixed positive integer ${N}$, using partial information on the events in terms of ${\{P(A_i)\}}$ and ${\{\sum_j c_j P(A_i\cap A_j)\}}$ where ${c_1}$, ${\dots}$, ${c_N}$ are given weights. We derive two new classes of lower bounds of at most pseudo-polynomial computational complexity. These classes of lower bounds generalize the existing bound in \cite{Kuai2000} and recent bounds in \cite{Yang2014,Yang2014ISIT} and are numerically shown to be tighter in some cases than the Gallot-Kounias bound \cite{Gallot1966,Kounias1968} and the Pr{é}kopa-Gao bound \cite{Prekopa2005} which require more information on the events probabilities.

math.PR↗

Privacy-Aware MMSE Estimation

We investigate the problem of the predictability of random variable $Y$ under a privacy constraint dictated by random variable $X$, correlated with $Y$, where both predictability and privacy are assessed in terms of the minimum mean-squared error (MMSE). Given that $X$ and $Y$ are connected via a binary-input symmetric-output (BISO) channel, we derive the \emph{optimal} random mapping $P_{Z|Y}$ such that the MMSE of $Y$ given $Z$ is minimized while the MMSE of $X$ given $Z$ is greater than $(1-ε)\mathsf{var}(X)$ for a given $ε\geq 0$. We also consider the case where $(X,Y)$ are continuous and $P_{Z|Y}$ is restricted to be an additive noise channel.

cs.IT↗

Information Extraction Under Privacy Constraints

A privacy-constrained information extraction problem is considered where for a pair of correlated discrete random variables $(X,Y)$ governed by a given joint distribution, an agent observes $Y$ and wants to convey to a potentially public user as much information about $Y$ as possible without compromising the amount of information revealed about $X$. To this end, the so-called {\em rate-privacy function} is introduced to quantify the maximal amount of information (measured in terms of mutual information) that can be extracted from $Y$ under a privacy constraint between $X$ and the extracted information, where privacy is measured using either mutual information or maximal correlation. Properties of the rate-privacy function are analyzed and information-theoretic and estimation-theoretic interpretations of it are presented for both the mutual information and maximal correlation privacy measures. It is also shown that the rate-privacy function admits a closed-form expression for a large family of joint distributions of $(X,Y)$. Finally, the rate-privacy function under the mutual information privacy measure is considered for the case where $(X,Y)$ has a joint probability density function by studying the problem where the extracted information is a uniform quantization of $Y$ corrupted by additive Gaussian noise. The asymptotic behavior of the rate-privacy function is studied as the quantization resolution grows without bound and it is observed that not all of the properties of the rate-privacy function carry over from the discrete to the continuous case.

cs.IT↗

Notes on Information-Theoretic Privacy

We investigate the tradeoff between privacy and utility in a situation where both privacy and utility are measured in terms of mutual information. For the binary case, we fully characterize this tradeoff in case of perfect privacy and also give an upper-bound for the case where some privacy leakage is allowed. We then introduce a new quantity which quantifies the amount of private information contained in the observable data and then connect it to the optimal tradeoff between privacy and utility.

cs.IT↗

On Maximal Correlation, Mutual Information and Data Privacy

The rate-privacy function is defined in \cite{Asoodeh} as a tradeoff between privacy and utility in a distributed private data system in which both privacy and utility are measured using mutual information. Here, we use maximal correlation in lieu of mutual information in the privacy constraint. We first obtain some general properties and bounds for maximal correlation and then modify the rate-privacy function to account for the privacy-constrained estimation problem. We find a bound for the utility in this problem when the maximal correlation privacy is set to some threshold $ε>0$ and construct an explicit privacy scheme which achieves this bound.

cs.IT↗

Lossless Secure Source Coding: Yamamoto's Setting

Given a private source of information, $X^n$ and a public correlated source, $Y^n$, we study the problem of encoding the two-dimensional source $(X^n, Y^n)$ into an index $J$ such that a remote party, knowing $J$ and some external side information $Z^n$, can losslessly recover $Y^n$ while any eavesdropper knowing $J$ and possibly a correlated side information $E^n$ can retrieve very little information about $X^n$. We give general converse results for the amount of information about $X^n$ that might be leaked in such systems and and also achievability results that are optimal in some special cases.

cs.IT↗

On the Multiple Access Channel with Asymmetric Noisy State Information at the Encoders

We consider the problem of reliable communication over multiple-access channels (MAC) where the channel is driven by an independent and identically distributed state process and the encoders and the decoder are provided with various degrees of asymmetric noisy channel state information (CSI). For the case where the encoders observe causal, asymmetric noisy CSI and the decoder observes complete CSI, we provide inner and outer bounds to the capacity region, which are tight for the sum-rate capacity. We then observe that, under a Markov assumption, similar capacity results also hold in the case where the receiver observes noisy CSI. Furthermore, we provide a single letter characterization for the capacity region when the CSI at the encoders are asymmetric deterministic functions of the CSI at the decoder and the encoders have non-causal noisy CSI (its causal version is recently solved in \cite{como-yuksel}). When the encoders observe asymmetric noisy CSI with asymmetric delays and the decoder observes complete CSI, we provide a single letter characterization for the capacity region. Finally, we consider a cooperative scenario with common and private messages, with asymmetric noisy CSI at the encoders and complete CSI at the decoder. We provide a single letter expression for the capacity region for such channels. For the cooperative scenario, we also note that as soon as the common message encoder does not have access to CSI, then in any noisy setup, covering the cases where no CSI or noisy CSI at the decoder, it is possible to obtain a single letter characterization for the capacity region. The main component in these results is a generalization of a converse coding approach, recently introduced in [1] for the MAC with asymmetric quantized CSI at the encoders and herein considerably extended and adapted for the noisy CSI setup.

cs.IT↗

A Generalized Poor-Verdu Error Bound for Multihypothesis Testing and the Channel Reliability Function

A lower bound on the minimum error probability for multihypothesis testing is established. The bound, which is expressed in terms of the cumulative distribution function of the tilted posterior hypothesis distribution given the observation with tilting parameter theta larger than or equal to 1, generalizes an earlier bound due the Poor and Verdu (1995). A sufficient condition is established under which the new bound (minus a multiplicative factor) provides the exact error probability in the limit of theta going to infinity. Examples illustrating the new bound are also provided. The application of this generalized Poor-Verdu bound to the channel reliability function is next carried out, resulting in two information-spectrum upper bounds. It is observed that, for a class of channels including the finite-input memoryless Gaussian channel, one of the bounds is tight and gives a multi-letter asymptotic expression for the reliability function, albeit its determination or calculation in single-letter form remains an open challenging problem. Numerical examples regarding the other bound are finally presented.

cs.IT↗