arXiv subjects
Publications and source records attributed to Alex Dytso.
The paper establishes the equality condition in the I-MMSE proof of the entropy power inequality (EPI). This is done by establishing an exact expression for the deficit between the two sides of the EPI. Interestingly, a necessary condition for the equality is established by making a connection to the famous Cauchy functional equation.
The problem of estimating an arbitrary random vector from its observation corrupted by additive white Gaussian noise, where the cost function is taken to be the Minimum Mean $p$-th Error (MMPE), is considered. The classical Minimum Mean Square Error (MMSE) is a special case of the MMPE. Several bounds, properties and applications of the MMPE are derived and discussed. The optimal MMPE estimator is found for Gaussian and binary input distributions. Properties of the MMPE as a function of the input distribution, SNR and order $p$ are derived. In particular, it is shown that the MMPE is a continuous function of $p$ and SNR. These results are possible in view of interpolation and change of measure bounds on the MMPE. The `Single-Crossing-Point Property' (SCPP) that bounds the MMSE for all SNR values {\it above} a certain value, at which the MMSE is known, together with the I-MMSE relationship is a powerful tool in deriving converse proofs in information theory. By studying the notion of conditional MMPE, a unifying proof (i.e., for any $p$) of the SCPP is shown. A complementary bound to the SCPP is then shown, which bounds the MMPE for all SNR values {\it below} a certain value, at which the MMPE is known. As a first application of the MMPE, a bound on the conditional differential entropy in terms of the MMPE is provided, which then yields a generalization of the Ozarow-Wyner lower bound on the mutual information achieved by a discrete input on a Gaussian noise channel. As a second application, the MMPE is shown to improve on previous characterizations of the phase transition phenomenon that manifests, in the limit as the length of the capacity achieving code goes to infinity, as a discontinuity of the MMSE as a function of SNR. As a final application, the MMPE is used to show bounds on the second derivative of mutual information, that tighten previously known bounds.
This paper considers a Gaussian channel with one transmitter and two receivers. The goal is to maximize the communication rate at the intended/primary receiver subject to a disturbance constraint at the unintended/secondary receiver. The disturbance is measured in terms of minimum mean square error (MMSE) of the interference that the transmission to the primary receiver inflicts on the secondary receiver. The paper presents a new upper bound for the problem of maximizing the mutual information subject to an MMSE constraint. The new bound holds for vector inputs of any length and recovers a previously known limiting (when the length of vector input tends to infinity) expression from the work of Bustin $\textit{et al.}$ The key technical novelty is a new upper bound on the MMSE. This bound allows one to bound the MMSE for all signal-to-noise ratio (SNR) values $\textit{below}$ a certain SNR at which the MMSE is known (which corresponds to the disturbance constraint). This bound complements the `single-crossing point property' of the MMSE that upper bounds the MMSE for all SNR values $\textit{above}$ a certain value at which the MMSE value is known. The MMSE upper bound provides a refined characterization of the phase-transition phenomenon which manifests, in the limit as the length of the vector input goes to infinity, as a discontinuity of the MMSE for the problem at hand. For vector inputs of size $n=1$, a matching lower bound, to within an additive gap of order $O \left( \log \log \frac{1}{\sf MMSE} \right)$ (where ${\sf MMSE}$ is the disturbance constraint), is shown by means of the mixed inputs technique recently introduced by Dytso $\textit{et al.}$
This paper shows that for the two-user Gaussian Interference Channel (G-IC) Treating Interference as Noise without Time Sharing (TINnoTS) achieves the closure of the capacity region to within either a constant gap, or to within a gap of the order O(logln(min(S,I))) where S is the largest Signal to Noise Ratio (SNR) on the direct links and I is the largest Interference to Noise Ratio (INR) on the cross links. As a consequence, TINnoTS is optimal from a generalized Degrees of Freedom (gDoF) perspective for all channel gains except for a subset of zero measure. TINnoTS with Gaussian inputs is known to be optimal to within 1/2 bit for a subset of the weak interference regime. Surprisingly, this paper shows that TINnoTS is gDoG optimal in all parameter regimes, even in the strong and very strong interference regimes where joint decoding of Gaussian inputs is optimal. For approximate optimality of TINnoTS in all parameter regimes it is critical to use non-Gaussian inputs. This work thus proposes to use mixed inputs as channel inputs where a mixed input is the sum of a discrete and a Gaussian random variable. Interestingly, compared to the Han-Kobayashi inner bound, the discrete part of a mixed input is shown to effectively act as a common message in the sense that, although treated as noise, its effect on the achievable rate region is as if it were jointly decoded together with the desired messages at a non-intended receiver. The practical implication is that a discrete interfering input is a 'friend', while a Gaussian interfering input is in general a 'foe'. Since TINnoTS requires neither joint decoding nor time sharing, the results of this paper are applicable to a variety of oblivions or asynchronous channels, such as the block asynchronous G-IC (which is not an information stable) and the G-IC with partial codebook knowledge at one or more receivers.
In multi-user information theory it is often assumed that every node in the network possesses all codebooks used in the network. This assumption may be impractical in distributed ad-hoc, cognitive or heterogeneous networks. This work considers the two-user Interference Channel with one Oblivious Receiver (IC-OR), i.e., one receiver lacks knowledge of the interfering cookbook while the other receiver knows both codebooks. The paper asks whether, and if so how much, the channel capacity of the IC-OR is reduced compared to that of the classical IC where both receivers know all codebooks. A novel outer bound is derived and shown to be achievable to within a gap for the class of injective semi-deterministic IC-ORs; the gap is shown to be zero for injective fully deterministic IC-ORs. For the linear deterministic IC-OR that models the Gaussian noise channel at high SNR, non i.i.d. Bernoulli(1/2) input bits are shown to achieve points not achievable by i.i.d. Bernoulli(1/2) input bits used in the same achievability scheme. For the real-valued Gaussian IC-OR the gap is shown to be at most 1/2 bit per channel use, even though the set of optimal input distributions for the derived outer bound could not be determined. Towards understanding the Gaussian IC-OR, an achievability strategy is evaluated in which the input alphabets at the non-oblivious transmitter are a mixture of discrete and Gaussian random variables, where the cardinality of the discrete part is appropriately chosen as a function of the channel parameters. Surprisingly, as the oblivious receiver intuitively should not be able to 'jointly decode' the intended and interfering messages (whose codebook is unavailable), it is shown that with this choice of input, the capacity region of the symmetric Gaussian IC-OR is to within 3.34 bits (per channel use per user) of an outer bound for the classical Gaussian IC with full codebook knowledge.
This paper considers a variation of the classical two-user interference channel where the communication of two interfering source-destination pairs is aided by an additional node that has a priori knowledge of the messages to be transmitted, which is referred to as the it cognitive relay. For this Interference Channel with a Cognitive Relay (ICCR) In particular, for the class of injective semi-deterministic ICCRs, a sum-rate upper bound is derived for the general memoryless ICCR and further tightened for the Linear Deterministic Approximation (LDA) of the Gaussian noise channel at high SNR, which disregards the noise and focuses on the interaction among the users' signals. The capacity region of the symmetric LDA is completely characterized except for the regime of moderately weak interference and weak links from the CR to the destinations. The insights gained from the analysis of the LDA are then translated back to the symmetric Gaussian noise channel (GICCR). For the symmetric GICCR, an approximate characterization (to within a constant gap) of the capacity region is provided for a parameter regime where capacity was previously unknown. The approximately optimal scheme suggests that message cognition at a relay is beneficial for interference management as it enables simultaneous over the air neutralization of the interference at both destinations.
In multi-user information theory it is often assumed that every node in the network possesses all codebooks used in the network. This assumption is however impractical in distributed ad-hoc and cognitive networks. This work considers the two- user Gaussian Interference Channel with one Oblivious Receiver (G-IC-OR), i.e., one receiver lacks knowledge of the interfering cookbook while the other receiver knows both codebooks. We ask whether, and if so how much, the channel capacity of the G-IC- OR is reduced compared to that of the classical G-IC where both receivers know all codebooks. Intuitively, the oblivious receiver should not be able to jointly decode its intended message along with the unintended interfering message whose codebook is unavailable. We demonstrate that in strong and very strong interference, where joint decoding is capacity achieving for the classical G-IC, lack of codebook knowledge does not reduce performance in terms of generalized degrees of freedom (gDoF). Moreover, we show that the sum-capacity of the symmetric G-IC- OR is to within O(log(log(SNR))) of that of the classical G-IC. The key novelty of the proposed achievable scheme is the use of a discrete input alphabet for the non-oblivious transmitter, whose cardinality is appropriately chosen as a function of SNR.