arXiv subjects

Pablo Piantanida

Publications and source records attributed to Pablo Piantanida.

At least 109 records · Page 6Linked to original sources

arXiv2018

Caching in Combination Networks: A Novel Delivery by Leveraging the Network Topology

Maddah-Ali and Niesen (MAN) in 2014 surprisingly showed that it is possible to serve an arbitrarily large number of cache-equipped users with a constant number of transmissions by using coded caching in shared-link broadcast networks. This paper studies the tradeoff between the user's cache size and the file download time for combination networks, where users with caches communicate with the servers through intermediate relays. Motivated by the so-called separation approach, it is assumed that placement and multicast message generation are done according to the MAN original scheme and regardless of the network topology. The main contribution of this paper is the design of a novel two-phase delivery scheme that, accounting to the network topology, outperforms schemes available in the literature. The key idea is to create additional (compared to MAN) multicasting opportunities: in the first phase coded messages are sent with the goal of increasing the amount of `side information' at the users, which is then leveraged during the second phase. The download time with the novel scheme is shown to be proportional to 1=H (with H being the number or relays) and to be order optimal under the constraint of uncoded placement for some parameter regimes.

cs.IT↗

arXiv2018

A Novel Asymmetric Coded Placement in Combination Networks with end-user Caches

The tradeoff between the user's memory size and the worst-case download time in the $(H,r,M,N)$ combination network is studied, where a central server communicates with $K$ users through $H$ immediate relays, and each user has local cache of size $M$ files and is connected to a different subset of $r$ relays. The main contribution of this paper is the design of a coded caching scheme with asymmetric coded placement by leveraging coordination among the relays, which was not exploited in past work. Mathematical analysis and numerical results show that the proposed schemes outperform existing schemes.

cs.IT↗

arXiv2018

Learning Anonymized Representations with Adversarial Neural Networks

Statistical methods protecting sensitive information or the identity of the data owner have become critical to ensure privacy of individuals as well as of organizations. This paper investigates anonymization methods based on representation learning and deep neural networks, and motivated by novel information theoretical bounds. We introduce a novel training objective for simultaneously training a predictor over target variables of interest (the regular labels) while preventing an intermediate representation to be predictive of the private labels. The architecture is based on three sub-networks: one going from input to representation, one from representation to predicted regular labels, and one from representation to predicted private labels. The training procedure aims at learning representations that preserve the relevant part of the information (about regular labels) while dismissing information about the private labels which correspond to the identity of a person. We demonstrate the success of this approach for two distinct classification versus anonymization tasks (handwritten digits and sentiment analysis).

stat.ML↗

arXiv2018

The Role of Information Complexity and Randomization in Representation Learning

A grand challenge in representation learning is to learn the different explanatory factors of variation behind the high dimen- sional data. Encoder models are often determined to optimize performance on training data when the real objective is to generalize well to unseen data. Although there is enough numerical evidence suggesting that noise injection (during training) at the representation level might improve the generalization ability of encoders, an information-theoretic understanding of this principle remains elusive. This paper presents a sample-dependent bound on the generalization gap of the cross-entropy loss that scales with the information complexity (IC) of the representations, meaning the mutual information between inputs and their representations. The IC is empirically investigated for standard multi-layer neural networks with SGD on MNIST and CIFAR-10 datasets; the behaviour of the gap and the IC appear to be in direct correlation, suggesting that SGD selects encoders to implicitly minimize the IC. We specialize the IC to study the role of Dropout on the generalization capacity of deep encoders which is shown to be directly related to the encoder capacity, being a measure of the distinguishability among samples from their representations. Our results support some recent regularization methods.

stat.ML↗

arXiv2018

Dictator Functions Maximize Mutual Information

Let $(\mathbf X, \mathbf Y)$ denote $n$ independent, identically distributed copies of two arbitrarily correlated Rademacher random variables $(X, Y)$. We prove that the inequality $I(f(\mathbf X); g(\mathbf Y)) \le I(X; Y)$ holds for any two Boolean functions: $f,g \colon \{-1,1\}^n \to \{-1,1\}$ ($I(\cdot; \cdot)$ denotes mutual information). We further show that equality in general is achieved only by the dictator functions $f(\mathbf x)=\pm g(\mathbf x)=\pm x_i$, $i \in \{1,2,\dots,n\}$.

cs.IT↗

arXiv2018

Performance Evaluation of Advanced Relaying Protocols in Large Wireless Networks

This paper studies the performance of some state-of-the-art cooperative full-duplex relaying protocols in the context of a large wireless network modelled using stochastic geometry tools. We investigate the outage behaviour for different cooperative schemes, namely, decode-and-forward, noisy-network coding and mixed noisy-network coding, considering fading, path loss and interference from other sources and relays. Due to the high complexity of the network topology and the protocols considered, a closed-form analysis is not possible, so our study is performed through extensive but careful numerical simulations, sweeping a large number of relevant parameters. Several scenarios of particular interest are investigated. In this way, insightful conclusions are drawn regarding the network regimes in which relay-assisted cooperation is most beneficial and the potential gains that could be achieved through it.

cs.IT↗

arXiv2017

Compression-Based Regularization with an Application to Multi-Task Learning

This paper investigates, from information theoretic grounds, a learning problem based on the principle that any regularity in a given dataset can be exploited to extract compact features from data, i.e., using fewer bits than needed to fully describe the data itself, in order to build meaningful representations of a relevant content (multiple labels). We begin by introducing the noisy lossy source coding paradigm with the log-loss fidelity criterion which provides the fundamental tradeoffs between the \emph{cross-entropy loss} (average risk) and the information rate of the features (model complexity). Our approach allows an information theoretic formulation of the \emph{multi-task learning} (MTL) problem which is a supervised learning framework in which the prediction models for several related tasks are learned jointly from common representations to achieve better generalization performance. Then, we present an iterative algorithm for computing the optimal tradeoffs and its global convergence is proven provided that some conditions hold. An important property of this algorithm is that it provides a natural safeguard against overfitting, because it minimizes the average risk taking into account a penalization induced by the model complexity. Remarkably, empirical results illustrate that there exists an optimal information rate minimizing the \emph{excess risk} which depends on the nature and the amount of available training data. An application to hierarchical text categorization is also investigated, extending previous works.

stat.ML↗

arXiv2017

The Multi-layer Information Bottleneck Problem

The muti-layer information bottleneck (IB) problem, where information is propagated (or successively refined) from layer to layer, is considered. Based on information forwarded by the preceding layer, each stage of the network is required to preserve a certain level of relevance with regards to a specific hidden variable, quantified by the mutual information. The hidden variables and the source can be arbitrarily correlated. The optimal trade-off between rates of relevance and compression (or complexity) is obtained through a single-letter characterization, referred to as the rate-relevance region. Conditions of successive refinabilty are given. Binary source with BSC hidden variables and binary source with BSC/BEC mixed hidden variables are both proved to be successively refinable. We further extend our result to Guassian models. A counterexample of successive refinability is also provided.

stat.ML↗

arXiv2017

Novel Delivery Schemes for Decentralized Coded Caching in the Finite File Size Regime

This paper analyzes the achievable tradeoff between cache~size and download~rate in decentralized caching systems with the uncoded cache placement originally proposed by Maddah-Ali and Niesen. It proposes two novel delivery schemes that take advantage of the multicasting opportunities that arise when a file is demanded by multiple users. These delivery schemes are extensions of known ones to the regime where the file size is finite. Numerical evaluations for the case of file uniform popularity show that the proposed schemes outperform previous ones for all value of the cache size.

cs.IT↗

arXiv2017

A Novel Transmission Scheme for the $K$-user Broadcast Channel with Delayed CSIT

The state-dependent $K$-user memoryless Broadcast Channel~(BC) with state feedback is investigated. We propose a novel transmission scheme and derive its corresponding achievable rate region, which, compared to some general schemes that deal with feedback, has the advantage of being relatively simple and thus is easy to evaluate. In particular, it is shown that the capacity region of the symmetric erasure BC with an arbitrary input alphabet size is achievable with the proposed scheme. For the fading Gaussian BC, we derive a symmetric achievable rate as a function of the signal-to-noise ratio~(SNR) and a small set of parameters. Besides achieving the optimal degrees of freedom at high SNR, the proposed scheme is shown, through numerical results, to outperform existing schemes from the literature in the finite SNR regime.

cs.IT↗

arXiv2017

Distributed Binary Detection with Lossy Data Compression

Consider the problem where a statistician in a two-node system receives rate-limited information from a transmitter about marginal observations of a memoryless process generated from two possible distributions. Using its own observations, this receiver is required to first identify the legitimacy of its sender by declaring the joint distribution of the process, and then depending on such authentication it generates the adequate reconstruction of the observations satisfying an average per-letter distortion. The performance of this setup is investigated through the corresponding rate-error-distortion region describing the trade-off between: the communication rate, the error exponent induced by the detection and the distortion incurred by the source reconstruction. In the special case of testing against independence, where the alternative hypothesis implies that the sources are independent, the optimal rate-error-distortion region is characterized. An application example to binary symmetric sources is given subsequently and the explicit expression for the rate-error-distortion region is provided as well. The case of "general hypotheses" is also investigated. A new achievable rate-error-distortion region is derived based on the use of non-asymptotic binning, improving the quality of communicated descriptions. Further improvement of performance in the general case is shown to be possible when the requirement of source reconstruction is relaxed, which stands in contrast to the case of general hypotheses.

cs.IT↗

arXiv2017

A Novel Index Coding Scheme and its Application to Coded Caching

This paper proposes a novel achievable scheme for the index problem and applies it to the caching problem. Index coding and caching are noiseless broadcast channel problems where receivers have message side information.In the index coding problem the side information sets are fixed, while in the caching problem the side information sets correspond the cache contents, which are under the control of the system designer. The proposed index coding scheme, based on distributed source coding and non-unique decoding,is shown to strictly enlarge the rate region achievable by composite coding.The novel index coding scheme applied to the caching problem is then shown to match an outer bound (previously proposed by the authors and also based on known results for the index coding problem) under the assumption of uncoded cache placement/prefetching.

cs.IT↗

arXiv2016

Collaborative Distributed Hypothesis Testing

A collaborative distributed binary decision problem is considered. Two statisticians are required to declare the correct probability measure of two jointly distributed memoryless process, denoted by $X^n=(X_1,\dots,X_n)$ and $Y^n=(Y_1,\dots,Y_n)$, out of two possible probability measures on finite alphabets, namely $P_{XY}$ and $P_{\bar{X}\bar{Y}}$. The marginal samples given by $X^n$ and $Y^n$ are assumed to be available at different locations. The statisticians are allowed to exchange limited amount of data over multiple rounds of interactions, which differs from previous work that deals mainly with unidirectional communication. A single round of interaction is considered before the result is generalized to any finite number of communication rounds. A feasibility result is shown, guaranteeing the feasibility of an error exponent for general hypotheses, through information-theoretic methods. The special case of testing against independence is revisited as being an instance of this result for which also an unfeasibility result is proven. A second special case is studied where zero-rate communication is imposed (data exchanges grow sub-exponentially with $n$) for which it is shown that interaction does not improve asymptotic performance.

cs.IT↗

arXiv2016

Capacity Results for the Multicast Cognitive Interference Channel

The capacity region of the Multicast Cognitive Interference Channel (CIFC) is investigated. This channel consists of two independent transmitters that wish to multicast two different messages, each of them to a different set of users. In addition, one of the transmitters --commonly referred to as the cognitive transmitter-- has prior non-causal knowledge of both messages to be transmitted. This scenario combines difficulties and challenges arising in the Interference Channel, the Broadcast Channel and multicasting communications. Our aim concerns the derivation of optimal interference mitigation techniques in such a challenging communication setup. We investigate to this end the multi-primary CIFC and its dual multi-secondary CIFC under various interference regimes as an attempt to build a thorough understanding for the more general setting. It is shown that, for some regimes, well-known coding techniques for the conventional CIFC remain still optimal in the presence of multicasting. While in other regimes, evolved encoding and/or decoding strategies are crucial. A careful use of these coding schemes and new outer bounding techniques allows us to characterize the capacity region for several classes of discrete memoryless and Gaussian channels in different interference regimes.

cs.IT↗

arXiv2016

On Caching with More Users than Files

Caching appears to be an efficient way to reduce peak hour network traffic congestion by storing some content at the user's cache without knowledge of later demands. Recently, Maddah-Ali and Niesen proposed a two-phase, placement and delivery phase, coded caching strategy for centralized systems (where coordination among users is possible in the placement phase), and for decentralized systems. This paper investigates the same setup under the further assumption that the number of users is larger than the number of files. By using the same uncoded placement strategy of Maddah-Ali and Niesen, a novel coded delivery strategy is proposed to profit from the multicasting opportunities that arise because a file may be demanded by multiple users. The proposed delivery method is proved to be optimal under the constraint of uncoded placement for centralized systems with two files, moreover it is shown to outperform known caching strategies for both centralized and decentralized systems.

cs.IT↗

arXiv2016

The Three-Terminal Interactive Lossy Source Coding Problem

The three-node multiterminal lossy source coding problem is investigated. We derive an inner bound to the general rate-distortion region of this problem which is a natural extension of the seminal work by Kaspi'85 on the interactive two-terminal source coding problem. It is shown that this (rather involved) inner bound contains several rate-distortion regions of some relevant source coding settings. In this way, besides the non-trivial extension of the interactive two terminal problem, our results can be seen as a generalization and hence unification of several previous works in the field. Specializing to particular cases we obtain novel rate-distortion regions for several lossy source coding problems. We finish by describing some of the open problems and challenges. However, the general three-node multiterminal lossy source coding problem seems to offer a formidable mathematical complexity.

cs.IT↗

arXiv2015

On the Optimality of Uncoded Cache Placement

Caching is an efficient way to reduce peak-hour network traffic congestion by storing some contents at user's local cache without knowledge of later demands. Maddah-Ali and Niesen initiated a fundamental study of caching systems; they proposed a scheme (with uncoded cache placement and linear network coding delivery) that is provably optimal to within a factor 12. In this paper, by noticing that when the cache contents and the demands are fixed, the caching problem can be seen as an index coding problem, we show the optimality of Maddah-Ali and Niesen's scheme assuming that cache placement is restricted to be uncoded and the number of users is not less than the number of files. Furthermore, this result states that further improvement to the Maddah-Ali and Niesen's scheme in this regimes can be obtained only by coded cache placement.

cs.IT↗

arXiv2015

The Second-Order Coding Rate of the MIMO Rayleigh Block-Fading Channel

The second-order coding rate of the multiple-input multiple-output (MIMO) quasi-static Rayleigh fading channel is studied. We tackle this problem via an information-spectrum approach and statistical bounds based on recent random matrix theory techniques. We derive a central limit theorem (CLT) to analyze the information density in the regime where the block-length n and the number of transmit and receive antennas K and N, respectively, grow simultaneously large. This result leads to the characterization of closed-form upper and lower bounds on the optimal average error probability when the coding rate is within O((nK)^-1/2) of the asymptotic capacity.

cs.IT↗