arXiv subjects

Ravi Tandon

Publications and source records attributed to Ravi Tandon.

At least 73 records · Page 4Linked to original sources

arXiv2017

Online Edge Caching in Fog-Aided Wireless Network

In a Fog Radio Access Network (F-RAN) architecture, edge nodes (ENs), such as base stations, are equipped with limited-capacity caches, as well as with fronthaul links that can support given transmission rates from a cloud processor. Existing information-theoretic analyses of content delivery in F-RANs have focused on offline caching with separate content placement and delivery phases. In contrast, this work considers an online caching set-up, in which the set of popular files is time-varying and both cache replenishment and content delivery can take place in each time slot. The analysis is centered on the characterization of the long-term Normalized Delivery Time (NDT), which captures the temporal dependence of the coding latencies accrued across multiple time slots in the high signal-to- noise ratio regime. Online caching and delivery schemes based on reactive and proactive caching are investigated, and their performance is compared to optimal offline caching schemes both analytically and via numerical results.

cs.IT↗

arXiv2017

Combating Computational Heterogeneity in Large-Scale Distributed Computing via Work Exchange

Owing to data-intensive large-scale applications, distributed computation systems have gained significant recent interest, due to their ability of running such tasks over a large number of commodity nodes in a time efficient manner. One of the major bottlenecks that adversely impacts the time efficiency is the computational heterogeneity of distributed nodes, often limiting the task completion time due to the slowest worker. In this paper, we first present a lower bound on the expected computation time based on the work-conservation principle. We then present our approach of work exchange to combat the latency problem, in which faster workers can be reassigned additional leftover computations that were originally assigned to slower workers. We present two variations of the work exchange approach: a) when the computational heterogeneity knowledge is known a priori; and b) when heterogeneity is unknown and is estimated in an online manner to assign tasks to distributed workers. As a baseline, we also present and analyze the use of an optimized Maximum Distance Separable (MDS) coded distributed computation scheme over heterogeneous nodes. Simulation results also compare the proposed approach of work exchange, the baseline MDS coded scheme and the lower bound obtained via work-conservation principle. We show that the work exchange scheme achieves time for computation which is very close to the lower bound with limited coordination and communication overhead even when the knowledge about heterogeneity levels is not available.

cs.DC↗

arXiv2017

Private Information Retrieval from Storage Constrained Databases -- Coded Caching meets PIR

Private information retrieval (PIR) allows a user to retrieve a desired message out of $K$ possible messages from $N$ databases without revealing the identity of the desired message. Majority of existing works on PIR assume the presence of replicated databases, each storing all the $K$ messages. In this work, we consider the problem of PIR from storage constrained databases. Each database has a storage capacity of $μKL$ bits, where $K$ is the number of messages, $L$ is the size of each message in bits, and $μ\in [1/N, 1]$ is the normalized storage. In the storage constrained PIR problem, there are two key design questions: a) how to store content across each database under storage constraints; and b) construction of schemes that allow efficient PIR through storage constrained databases. The main contribution of this work is a general achievable scheme for PIR from storage constrained databases for any value of storage. In particular, for any $(N,K)$, with normalized storage $μ= t/N$, where the parameter $t$ can take integer values $t \in \{1, 2, \ldots, N\}$, we show that our proposed PIR scheme achieves a download cost of $\left(1+ \frac{1}{t}+ \frac{1}{t^{2}}+ \cdots + \frac{1}{t^{K-1}}\right)$. The extreme case when $μ=1$ (i.e., $t=N$) corresponds to the setting of replicated databases with full storage. For this extremal setting, our scheme recovers the information-theoretically optimal download cost characterized by Sun and Jafar as $\left(1+ \frac{1}{N}+ \cdots + \frac{1}{N^{K-1}}\right)$. For the other extreme, when $μ= 1/N$ (i.e., $t=1$), the proposed scheme achieves a download cost of $K$. The interesting aspect of the result is that for intermediate values of storage, i.e., $1/N < μ<1$, the proposed scheme can strictly outperform memory-sharing between extreme values of storage.

cs.IT↗

arXiv2017

The Capacity of Cache Aided Private Information Retrieval

The problem of cache enabled private information retrieval (PIR) is considered in which a user wishes to privately retrieve one out of $K$ messages, each of size $L$ bits from $N$ distributed databases. The user has a local cache of storage $SL$ bits which can be used to store any function of the $K$ messages. The main contribution of this work is the exact characterization of the capacity of cache aided PIR as a function of the storage parameter $S$. In particular, for a given cache storage parameter $S$, the information-theoretically optimal download cost $D^{*}(S)/L$ (or the inverse of capacity) is shown to be equal to $(1- \frac{S}{K})\left(1+ \frac{1}{N}+ \ldots + \frac{1}{N^{K-1}}\right)$. Special cases of this result correspond to the settings when $S=0$, for which the optimal download cost was shown by Sun and Jafar to be $\left(1+ \frac{1}{N}+ \ldots + \frac{1}{N^{K-1}}\right)$, and the case when $S=K$, i.e., cache size is large enough to store all messages locally, for which the optimal download cost is $0$. The intermediate points $S\in (0, K)$ can be readily achieved through a simple memory-sharing based PIR scheme. The key technical contribution of this work is the converse, i.e., a lower bound on the download cost as a function of storage $S$ which shows that memory sharing is information-theoretically optimal.

cs.IT↗

arXiv2017

Degrees of Freedom and Achievable Rate of Wide-Band Multi-cell Multiple Access Channels With No CSIT

This paper considers a $K$-cell multiple access channel with inter-symbol interference. The primary finding of this paper is that, without instantaneous channel state information at the transmitters (CSIT), the sum degrees-of-freedom (DoF) of the considered channel is $\frac{β-1}βK$ with $β\geq 2$ when the number of users per cell is sufficiently large, where $β$ is the ratio of the maximum channel-impulse-response (CIR) length of desired links to that of interfering links in each cell. Our finding implies that even without instantaneous CSIT, \textit{interference-free DoF per cell} is achievable as $β$ approaches infinity with a sufficiently large number of users per cell. This achievability is shown by a blind interference management method that exploits the relativity in delay spreads between desired and interfering links. In this method, all inter-cell-interference signals are aligned to the same direction by using a discrete-Fourier-transform-based precoding with cyclic prefix that only depends on the number of CIR taps. Using this method, we also characterize the achievable sum rate of the considered channel, in a closed-form expression.

cs.IT↗

arXiv2016

On the Worst-case Communication Overhead for Distributed Data Shuffling

Distributed learning platforms for processing large scale data-sets are becoming increasingly prevalent. In typical distributed implementations, a centralized master node breaks the data-set into smaller batches for parallel processing across distributed workers to achieve speed-up and efficiency. Several computational tasks are of sequential nature, and involve multiple passes over the data. At each iteration over the data, it is common practice to randomly re-shuffle the data at the master node, assigning different batches for each worker to process. This random re-shuffling operation comes at the cost of extra communication overhead, since at each shuffle, new data points need to be delivered to the distributed workers. In this paper, we focus on characterizing the information theoretically optimal communication overhead for the distributed data shuffling problem. We propose a novel coded data delivery scheme for the case of no excess storage, where every worker can only store the assigned data batches under processing. Our scheme exploits a new type of coding opportunity and is applicable to any arbitrary shuffle, and for any number of workers. We also present an information theoretic lower bound on the minimum communication overhead for data shuffling, and show that the proposed scheme matches this lower bound for the worst-case communication overhead.

cs.IT↗

arXiv2016

Information Theoretic Limits of Data Shuffling for Distributed Learning

Data shuffling is one of the fundamental building blocks for distributed learning algorithms, that increases the statistical gain for each step of the learning process. In each iteration, different shuffled data points are assigned by a central node to a distributed set of workers to perform local computations, which leads to communication bottlenecks. The focus of this paper is on formalizing and understanding the fundamental information-theoretic trade-off between storage (per worker) and the worst-case communication overhead for the data shuffling problem. We completely characterize the information theoretic trade-off for $K=2$, and $K=3$ workers, for any value of storage capacity, and show that increasing the storage across workers can reduce the communication overhead by leveraging coding. We propose a novel and systematic data delivery and storage update strategy for each data shuffle iteration, which preserves the structural properties of the storage across the workers, and aids in minimizing the communication overhead in subsequent data shuffling iterations.

cs.IT↗

arXiv2016

Improved Approximation of Storage-Rate Tradeoff for Caching with Multiple Demands

Caching at the network edge has emerged as a viable solution for alleviating the severe capacity crunch in modern content centric wireless networks by leveraging network load-balancing in the form of localized content storage and delivery. In this work, we consider a cache-aided network where the cache storage phase is assisted by a central server and users can demand multiple files at each transmission interval. To service these demands, we consider two delivery models - $(1)$ centralized content delivery where user demands at each transmission interval are serviced by the central server via multicast transmissions; and $(2)$ device-to-device (D2D) assisted distributed delivery where users multicast to each other in order to service file demands. For such cache-aided networks, we present new results on the fundamental cache storage vs. transmission rate tradeoff. Specifically, we develop a new technique for characterizing information theoretic lower bounds on the storage-rate tradeoff and show that the new lower bounds are strictly tighter than cut-set bounds from literature. Furthermore, using the new lower bounds, we establish the optimal storage-rate tradeoff to within a constant multiplicative gap. We show that, for multiple demands per user, achievable schemes based on repetition of schemes for single demands are order-optimal under both delivery models.

cs.IT↗

arXiv2016

Hierarchical Quickest Change Detection via Surrogates

Change detection (CD) in time series data is a critical problem as it reveal changes in the underlying generative processes driving the time series. Despite having received significant attention, one important unexplored aspect is how to efficiently utilize additional correlated information to improve the detection and the understanding of changepoints. We propose hierarchical quickest change detection (HQCD), a framework that formalizes the process of incorporating additional correlated sources for early changepoint detection. The core ideas behind HQCD are rooted in the theory of quickest detection and HQCD can be regarded as its novel generalization to a hierarchical setting. The sources are classified into targets and surrogates, and HQCD leverages this structure to systematically assimilate observed data to update changepoint statistics across layers. The decision on actual changepoints are provided by minimizing the delay while still maintaining reliability bounds. In addition, HQCD also uncovers interesting relations between changes at targets from changes across surrogates. We validate HQCD for reliability and performance against several state-of-the-art methods for both synthetic dataset (known changepoints) and several real-life examples (unknown changepoints). Our experiments indicate that we gain significant robustness without loss of detection delay through HQCD. Our real-life experiments also showcase the usefulness of the hierarchical setting by connecting the surrogate sources (such as Twitter chatter) to target sources (such as Employment related protests that ultimately lead to major uprisings).

cs.LG↗

arXiv2016

Flow of Information in Feed-Forward Deep Neural Networks

Feed-forward deep neural networks have been used extensively in various machine learning applications. Developing a precise understanding of the underling behavior of neural networks is crucial for their efficient deployment. In this paper, we use an information theoretic approach to study the flow of information in a neural network and to determine how entropy of information changes between consecutive layers. Moreover, using the Information Bottleneck principle, we develop a constrained optimization problem that can be used in the training process of a deep neural network. Furthermore, we determine a lower bound for the level of data representation that can be achieved in a deep neural network with an acceptable level of distortion.

cs.IT↗

arXiv2015

Cache Aided Wireless Networks: Tradeoffs between Storage and Latency

We investigate the fundamental information theoretic limits of cache-aided wireless networks, in which edge nodes (or transmitters) are endowed with caches that can store popular content, such as multimedia files. This architecture aims to localize popular multimedia content by proactively pushing it closer to the edge of the wireless network, thereby alleviating backhaul load. An information theoretic model of such networks is presented, that includes the introduction of a new metric, namely normalized delivery time (NDT), which captures the worst case time to deliver any requested content to the users. We present new results on the trade-off between latency, measured via the NDT, and the cache storage capacity of the edge nodes. In particular, a novel information theoretic lower bound on NDT is presented for cache aided networks. The optimality of this bound is shown for several system parameters.

cs.IT↗

arXiv2015

Perfect Output Feedback in the Two-User Decentralized Interference Channel

In this paper, the $η$-Nash equilibrium ($η$-NE) region of the two-user Gaussian interference channel (IC) with perfect output feedback is approximated to within $1$ bit/s/Hz and $η$ arbitrarily close to $1$ bit/s/Hz. The relevance of the $η$-NE region is that it provides the set of rate-pairs that are achievable and stable in the IC when both transmitter-receiver pairs autonomously tune their own transmit-receive configurations seeking an $η$-optimal individual transmission rate. Therefore, any rate tuple outside the $η$-NE region is not stable as there always exists one link able to increase by at least $η$ bits/s/Hz its own transmission rate by updating its own transmit-receive configuration. The main insights that arise from this work are: $(i)$ The $η$-NE region achieved with feedback is larger than or equal to the $η$-NE region without feedback. More importantly, for each rate pair achievable at an $η$-NE without feedback, there exists at least one rate pair achievable at an $η$-NE with feedback that is weakly Pareto superior. $(ii)$ There always exists an $η$-NE transmit-receive configuration that achieves a rate pair that is at most $1$ bit/s/Hz per user away from the outer bound of the capacity region.

cs.IT↗

arXiv2015

On the Latency and Energy Efficiency of Erasure-Coded Cloud Storage Systems

The increase in data storage and power consumption at data-centers has made it imperative to design energy efficient Distributed Storage Systems (DSS). The energy efficiency of DSS is strongly influenced not only by the volume of data, frequency of data access and redundancy in data storage, but also by the heterogeneity exhibited by the DSS in these dimensions. To this end, we propose and analyze the energy efficiency of a heterogeneous distributed storage system in which $n$ storage servers (disks) store the data of $R$ distinct classes. Data of class $i$ is encoded using a $(n,k_{i})$ erasure code and the (random) data retrieval requests can also vary across classes. We show that the energy efficiency of such systems is closely related to the average latency and hence motivates us to study the energy efficiency via the lens of average latency. Through this connection, we show that erasure coding serves the dual purpose of reducing latency and increasing energy efficiency. We present a queuing theoretic analysis of the proposed model and establish upper and lower bounds on the average latency for each data class under various scheduling policies. Through extensive simulations, we present qualitative insights which reveal the impact of coding rate, number of servers, service distribution and number of redundant requests on the average latency and energy efficiency of the DSS.

cs.DC↗

arXiv2015

MISO Broadcast Channel with Hybrid CSIT: Beyond Two Users

We study the impact of heterogeneity of channel-state-information available at the transmitters (CSIT) on the capacity of broadcast channels with a multiple-antenna transmitter and $k$ single-antenna receivers (MISO BC). In particular, we consider the $k$-user MISO BC, where the CSIT with respect to each receiver can be either instantaneous/perfect, delayed, or not available; and we study the impact of this heterogeneity of CSIT on the degrees-of-freedom (DoF) of such network. We first focus on the $3$-user MISO BC; and we completely characterize the DoF region for all possible heterogeneous CSIT configurations, assuming linear encoding strategies at the transmitters. The result shows that the state-of-the-art achievable schemes in the literature are indeed sum-DoF optimal, when restricted to linear encoding schemes. To prove the result, we develop a novel bound, called Interference Decomposition Bound, which provides a lower bound on the interference dimension at a receiver which supplies delayed CSIT based on the average dimension of constituents of that interference, thereby decomposing the interference into its individual components. Furthermore, we extend our outer bound on the DoF region to the general $k$-user MISO BC, and demonstrate that it leads to an approximate characterization of linear sum-DoF to within an additive gap of $0.5$ for a broad range of CSIT configurations. Moreover, for the special case where only one receiver supplies delayed CSIT, we completely characterize the linear sum-DoF.

cs.IT↗

arXiv2015

Secure Degrees of Freedom Region of the Two-User MISO Broadcast Channel with Alternating CSIT

The two user multiple-input single-output (MISO) broadcast channel with confidential messages (BCCM) is studied in which the nature of channel state information at the transmitter (CSIT) from each user can be of the form $I_{i}$, $i=1,2$ where $I_{1}, I_{2}\in \{\mathsf{P}, \mathsf{D}, \mathsf{N}\}$, and the forms $\mathsf{P}$, $\mathsf{D}$ and $\mathsf{N}$ correspond to perfect and instantaneous, completely delayed, and no CSIT, respectively. Thus, the overall CSIT can alternate between $9$ possible states corresponding to all possible values of $I_{1}I_{2}$, with each state occurring for $λ_{I_{1}I_{2}}$ fraction of the total duration. The main contribution of this paper is to establish the secure degrees of freedom (s.d.o.f.) region of the MISO BCCM with alternating CSIT with the symmetry assumption, where $λ_{I_{1} I_{2}}=λ_{I_{2}I_{1}}$. The main technical contributions include developing a) novel achievable schemes for MISO BCCM with alternating CSIT with security constraints which also highlight the synergistic benefits of inter-state coding for secrecy, b) new converse proofs via local statistical equivalence and channel enhancement; and c) showing the interplay between various aspects of channel knowledge and their impact on s.d.o.f.

cs.IT↗

arXiv2014

Fundamental Limits of Caching with Secure Delivery

Caching is emerging as a vital tool for alleviating the severe capacity crunch in modern content-centric wireless networks. The main idea behind caching is to store parts of popular content in end-users' memory and leverage the locally stored content to reduce peak data rates. By jointly designing content placement and delivery mechanisms, recent works have shown order-wise reduction in transmission rates in contrast to traditional methods. In this work, we consider the secure caching problem with the additional goal of minimizing information leakage to an external wiretapper. The fundamental cache memory vs. transmission rate trade-off for the secure caching problem is characterized. Rather surprisingly, these results show that security can be introduced at a negligible cost, particularly for large number of files and users. It is also shown that the rate achieved by the proposed caching scheme with secure delivery is within a constant multiplicative factor from the information-theoretic optimal rate for almost all parameter values of practical interest.

cs.IT↗

arXiv2014

Distributed Space-Time Interference Alignment with Moderately-Delayed CSIT

This paper proposes an interference alignment method with distributed and delayed channel state information at the transmitter (CSIT) for a class of interference networks. The core idea of the proposed method is to align interference signals over time at the unintended receivers in a distributed manner. With the proposed method, achievable trade-offs between the sum of degrees of freedom (sum-DoF) and feedback delay of CSI are characterized in both the X-channel and three-user interference channel to reveal the impact on how the CSI feedback delay affects the sum-DoF of the interference networks. A major implication of derived results is that distributed and moderately- delayed CSIT is useful to strictly improve the sum-DoF over the case of no CSI at the transmitter in a certain class of interference networks. For a class of X-channels, the results show how to optimally use distributed and moderately-delayed CSIT to yield the same sum-DoF as instantaneous and global CSIT. Further, leveraging the proposed transmission method and the known outer bound results, the sum-capacity of the two-user X-channel with a particular set of channel coefficients is characterized within a constant number of bits.

cs.IT↗

arXiv2014

On the Degrees-of-freedom of the 3-user MISO Broadcast Channel with Hybrid CSIT

The 3-user multiple-input single-output (MISO) broadcast channel (BC) with hybrid channel state information at the transmitter (CSIT) is considered. In this framework, there is perfect and instantaneous CSIT from a subset of users and delayed CSIT from the remaining users. We present new results on the degrees of freedom (DoF) of the 3-user MISO BC with hybrid CSIT. In particular, for the case of 2 transmit antennas, we show that with perfect CSIT from one user and delayed CSIT from the remaining two users, the optimal DoF is 5/3. For the case of 3 transmit antennas and the same hybrid CSIT setting, it is shown that a higher DoF of 9/5 is achievable and this result improves upon the best known bound. Furthermore, with 3 transmit antennas, and the hybrid CSIT setting in which there is perfect CSIT from two users and delayed CSIT from the third one, a novel scheme is presented which achieves 9/4 DoF. Our results also reveal new insights on how to utilize hybrid channel knowledge for multi-user scenarios.

cs.IT↗