arXiv subjects

Swanand Kadhe

Publications and source records attributed to Swanand Kadhe.

34 records · Page 2Linked to original sources

arXiv2020

Communication-Efficient Gradient Coding for Straggler Mitigation in Distributed Learning

Distributed implementations of gradient-based methods, wherein a server distributes gradient computations across worker machines, need to overcome two limitations: delays caused by slow running machines called 'stragglers', and communication overheads. Recently, Ye and Abbe [ICML 2018] proposed a coding-theoretic paradigm to characterize a fundamental trade-off between computation load per worker, communication overhead per worker, and straggler tolerance. However, their proposed coding schemes suffer from heavy decoding complexity and poor numerical stability. In this paper, we develop a communication-efficient gradient coding framework to overcome these drawbacks. Our proposed framework enables using any linear code to design the encoding and decoding functions. When a particular code is used in this framework, its block-length determines the computation load, dimension determines the communication overhead, and minimum distance determines the straggler tolerance. The flexibility of choosing a code allows us to gracefully trade-off the straggler threshold and communication overhead for smaller decoding complexity and higher numerical stability. Further, we show that using a maximum distance separable (MDS) code generated by a random Gaussian matrix in our framework yields a gradient code that is optimal with respect to the trade-off and, in addition, satisfies stronger guarantees on numerical stability as compared to the previously proposed schemes. Finally, we evaluate our proposed framework on Amazon EC2 and demonstrate that it reduces the average iteration time by 16% as compared to prior gradient coding schemes.

cs.IT↗

arXiv2019

On an Equivalence Between Single-Server PIR with Side Information and Locally Recoverable Codes

Private Information Retrieval (PIR) problem has recently attracted a significant interest in the information-theory community. In this problem, a user wants to privately download one or more messages belonging to a database with copies stored on a single or multiple remote servers. In the single server scenario, the user must have prior side information, i.e., a subset of messages unknown to the server, to be able to privately retrieve the required messages in an efficient way. In the last decade, there has also been a significant interest in Locally Recoverable Codes (LRC), a class of storage codes in which each symbol can be recovered from a limited number of other symbols. More recently, there is an interest in 'cooperative' locally recoverable codes, i.e., codes in which multiple symbols can be recovered from a small set of other code symbols. In this paper, we establish a relationship between coding schemes for the single-server PIR problem and LRCs. In particular, we show the following results: (i) PIR schemes designed for retrieving a single message are equivalent to classical LRCs; and (ii) PIR schemes for retrieving multiple messages are equivalent to cooperative LRCs. These equivalence results allow us to recover upper bounds on the download rate for PIR-SI schemes, and to obtain a novel rate upper bound on cooperative LRCs. We show results for both linear and non-linear codes.

cs.IT↗

arXiv2019

SeF: A Secure Fountain Architecture for Slashing Storage Costs in Blockchains

Full nodes, which synchronize the entire blockchain history and independently validate all the blocks, form the backbone of any blockchain network by playing a vital role in ensuring security properties. On the other hand, a user running a full node needs to pay a heavy price in terms of storage costs. E.g., the Bitcoin blockchain size has grown over 215GB, in spite of its low throughput. The ledger size for a high throughput blockchain Ripple has already reached 9TB, and it is growing at an astonishing rate of 12GB per day! In this paper, we propose an architecture based on 'fountain codes', a class of erasure codes, that enables any full node to 'encode' validated blocks into a small number of 'coded blocks', thereby reducing its storage costs by orders of magnitude. In particular, our proposed "Secure Fountain (SeF)" architecture can achieve a near-optimal trade-off between the storage savings per node and the 'bootstrap cost' in terms of the number of (honest) storage-constrained nodes a new node needs to contact to recover the blockchain. A key technical innovation in SeF codes is to make fountain codes secure against adversarial nodes that can provide maliciously formed coded blocks. Our idea is to use the header-chain as a 'side-information' to check whether a coded block is maliciously formed while it is getting decoded. Further, the 'rateless property' of fountain codes helps in achieving high decentralization and scalability. Our experiments demonstrate that SeF codes tuned to achieve 1000x storage savings enable full nodes to encode the 191GB Bitcoin blockchain into 195MB on average. A new node can recover the blockchain from an arbitrary set of storage-constrained nodes as long as the set contains ~1100 honest nodes on average. Note that for a 1000x storage savings, the fundamental bound on the number of honest nodes to contact is 1000: we need about 10% more in practice.

cs.CR↗

arXiv2019

Codes with Locality in the Rank and Subspace Metrics

We extend the notion of locality from the Hamming metric to the rank and subspace metrics. Our main contribution is to construct a class of array codes with locality constraints in the rank metric. Our motivation for constructing such codes stems from designing codes for efficient data recovery from correlated and/or mixed (i.e., complete and partial) failures in distributed storage systems. Specifically, the proposed local rank-metric codes can recover locally from 'crisscross errors and erasures', which affect a limited number of rows and/or columns of the storage system. We also derive a Singleton-like upper bound on the minimum rank distance of (linear) codes with 'rank-locality' constraints. Our proposed construction achieves this bound for a broad range of parameters. The construction builds upon Tamo and Barg's method for constructing locally repairable codes with optimal minimum Hamming distance. Finally, we construct a class of constant-dimension subspace codes (also known as Grassmannian codes) with locality constraints in the subspace metric. The key idea is to show that a Grassmannian code with locality can be easily constructed from a rank-metric code with locality by using the lifting method proposed by Silva et al. We present an application of such codes for distributed storage systems, wherein nodes are connected over a network that can introduce errors and erasures.

cs.IT↗

arXiv2019

Gradient Coding Based on Block Designs for Mitigating Adversarial Stragglers

Distributed implementations of gradient-based methods, wherein a server distributes gradient computations across worker machines, suffer from slow running machines, called 'stragglers'. Gradient coding is a coding-theoretic framework to mitigate stragglers by enabling the server to recover the gradient sum in the presence of stragglers. 'Approximate gradient codes' are variants of gradient codes that reduce computation and storage overhead per worker by allowing the server to approximately reconstruct the gradient sum. In this work, our goal is to construct approximate gradient codes that are resilient to stragglers selected by a computationally unbounded adversary. Our motivation for constructing codes to mitigate adversarial stragglers stems from the challenge of tackling stragglers in massive-scale elastic and serverless systems, wherein it is difficult to statistically model stragglers. Towards this end, we propose a class of approximate gradient codes based on balanced incomplete block designs (BIBDs). We show that the approximation error for these codes depends only on the number of stragglers, and thus, adversarial straggler selection has no advantage over random selection. In addition, the proposed codes admit computationally efficient decoding at the server. Next, to characterize fundamental limits of adversarial straggling, we consider the notion of 'adversarial threshold' -- the smallest number of workers that an adversary must straggle to inflict certain approximation error. We compute a lower bound on the adversarial threshold, and show that codes based on symmetric BIBDs maximize this lower bound among a wide class of codes, making them excellent candidates for mitigating adversarial stragglers.

cs.IT↗

arXiv2019

Single-Server Multi-Message Individually-Private Information Retrieval with Side Information

We consider a multi-user variant of the private information retrieval problem described as follows. Suppose there are $D$ users, each of which wants to privately retrieve a distinct message from a server with the help of a trusted agent. We assume that the agent has a random subset of $M$ messages that is not known to the server. The goal of the agent is to collectively retrieve the users' requests from the server. For protecting the privacy of users, we introduce the notion of individual-privacy -- the agent is required to protect the privacy only for each individual user (but may leak some correlations among user requests). We refer to this problem as Individually-Private Information Retrieval with Side Information (IPIR-SI). We first establish a lower bound on the capacity, which is defined as the maximum achievable download rate, of the IPIR-SI problem by presenting a novel achievability protocol. Next, we characterize the capacity of IPIR-SI problem for $M = 1$ and $D = 2$. In the process of characterizing the capacity for arbitrary $M$ and $D$ we present a novel combinatorial conjecture, that may be of independent interest.

cs.IT↗

arXiv2018

On the Capacity of Single-Server Multi-Message Private Information Retrieval with Side Information

We study Private Information Retrieval with Side Information (PIR-SI) in the single-server multi-message setting. In this setting, a user wants to download $D$ messages from a database of $K\geq D$ messages, stored on a single server, without revealing any information about the identities of the demanded messages to the server. The goal of the user is to achieve information-theoretic privacy by leveraging the side information about the database. The side information consists of a random subset of $M$ messages in the database which could have been obtained in advance from other users or from previous interactions with the server. The identities of the messages forming the side information are initially unknown to the server. Our goal is to characterize the capacity of this setting, i.e., the maximum achievable download rate. In our previous work, we have established the PIR-SI capacity for the special case in which the user wants a single message, i.e., $D=1$ and showed that the capacity can be achieved through the Partition and Code (PC) scheme. In this paper, we focus on the case when the user wants multiple messages, i.e., $D>1$. Our first result is that if the user wants more messages than what they have as side information, i.e., $D>M$, then the capacity is $\frac{D}{K-M}$, and it can be achieved using a scheme based on the Generalized Reed-Solomon (GRS) codes. In this case, the user must learn all the messages in the database in order to obtain the desired messages. Our second result shows that this may not be necessary when $D\leq M$, and the capacity in this case can be higher. We present a lower bound on the capacity based on an achievability scheme which we call Generalized Partition and Code (GPC).

cs.IT↗

arXiv2018

Synthesis of Logical Clifford Operators via Symplectic Geometry

Quantum error-correcting codes can be used to protect qubits involved in quantum computation. This requires that logical operators acting on protected qubits be translated to physical operators (circuits) acting on physical quantum states. We propose a mathematical framework for synthesizing physical circuits that implement logical Clifford operators for stabilizer codes. Circuit synthesis is enabled by representing the desired physical Clifford operator in $\mathbb{C}^{N \times N}$ as a partial $2m \times 2m$ binary symplectic matrix, where $N = 2^m$. We state and prove two theorems that use symplectic transvections to efficiently enumerate all symplectic matrices that satisfy a system of linear equations. As an important corollary of these results, we prove that for an $[\![ m,m-k ]\!]$ stabilizer code every logical Clifford operator has $2^{k(k+1)/2}$ symplectic solutions. The desired physical circuits are then obtained by decomposing each solution as a product of elementary symplectic matrices. Our assembly of the possible physical realizations enables optimization over them with respect to a suitable metric. Furthermore, we show that any circuit that normalizes the stabilizer of the code can be transformed into a circuit that centralizes the stabilizer, while realizing the same logical operation. Our method of circuit synthesis can be applied to any stabilizer code, and this paper provides a proof of concept synthesis of universal Clifford gates for the $[\![ 6,4,2 ]\!]$ CSS code. We conclude with a classical coding-theoretic perspective for constructing logical Pauli operators for CSS codes. Since our circuit synthesis algorithm builds on the logical Pauli operators for the code, this paper provides a complete framework for constructing all logical Clifford operators for CSS codes. Programs implementing our algorithms can be found at https://github.com/nrenga/symplectic-arxiv18a

cs.IT↗

arXiv2017

Universally Weakly Secure Coset Coding Schemes for Minimum Storage Regenerating (MSR) Codes

We consider the problem of designing codes for distributed storage that protect user data against eavesdroppers that can gain access to network links as well as individual nodes. Our goal is to achieve weak security (also known as block security) that requires that the eavesdroppers would not be able to decode individual files or combinations of a small number of files. The standard approach for achieving block security is to use a joint design scheme that consists of (inner) storage code and the (outer) coset code. However, jointly designing the codes requires that the user, who pre-processes and stores the files, should know the underlying storage code in order to design the (outer) linear transformation for achieving weak security. In many practical scenarios, such as storing the files on the third party cloud storage system, it may not be possible for the user to know the underlying storage code. In this work, we present universal schemes that separate the outer code design from the storage code design for minimum storage regenerating codes (MSR). Our schemes allow the independent design of the storage code and the outer code. Our schemes use small field size and can be used in a broad range of practical settings.

cs.IT↗

arXiv2017

On the Service Capacity Region of Accessing Erasure Coded Content

Cloud storage systems generally add redundancy in storing content files such that $K$ files are replicated or erasure coded and stored on $N > K$ nodes. In addition to providing reliability against failures, the redundant copies can be used to serve a larger volume of content access requests. A request for one of the files can be either be sent to a systematic node, or one of the repair groups. In this paper, we seek to maximize the service capacity region, that is, the set of request arrival rates for the $K$ files that can be supported by a coded storage system. We explore two aspects of this problem: 1) for a given erasure code, how to optimally split incoming requests between systematic nodes and repair groups, and 2) choosing an underlying erasure code that maximizes the achievable service capacity region. In particular, we consider MDS and Simplex codes. Our analysis demonstrates that erasure coding makes the system more robust to skews in file popularity than simply replicating a file at multiple servers, and that coding and replication together can make the capacity region larger than either alone.

cs.IT↗

arXiv2017

Rate Optimal Binary Linear Locally Repairable Codes with Small Availability

A locally repairable code with availability has the property that every code symbol can be recovered from multiple, disjoint subsets of other symbols of small size. In particular, a code symbol is said to have $(r,t)$-availability if it can be recovered from $t$ disjoint subsets, each of size at most $r$. A code with availability is said to be 'rate-optimal', if its rate is maximum among the class of codes with given locality, availability, and alphabet size. This paper focuses on rate-optimal binary, linear codes with small availability, and makes four contributions. First, it establishes tight upper bounds on the rate of binary linear codes with $(r,2)$ and $(2,3)$ availability. Second, it establishes a uniqueness result for binary rate-optimal codes, showing that for certain classes of binary linear codes with $(r,2)$ and $(2,3)$-availability, any rate optimal code must be a direct sum of shorter rate optimal codes. Third, it presents novel upper bounds on the rates of binary linear codes with $(2,t)$ and $(r,3)$-availability. In particular, the main contribution here is a new method for bounding the number of cosets of the dual of a code with availability, using its covering properties. Finally, it presents a class of locally repairable linear codes associated with convex polyhedra, focusing on the codes associated with the Platonic solids. It demonstrates that these codes are locally repairable with $t = 2$, and that the codes associated with (geometric) dual polyhedra are (coding theoretic) duals of each other.

cs.IT↗

arXiv2017

Private Information Retrieval with Side Information

We study the problem of Private Information Retrieval (PIR) in the presence of prior side information. The problem setup includes a database of $K$ independent messages possibly replicated on several servers, and a user that needs to retrieve one of these messages. In addition, the user has some prior side information in the form of a subset of $M$ messages, not containing the desired message and unknown to the servers. This problem is motivated by practical settings in which the user can obtain side information opportunistically from other users or has previously downloaded some messages using classical PIR schemes. The objective of the user is to retrieve the required message without revealing its identity while minimizing the amount of data downloaded from the servers. We focus on achieving information-theoretic privacy in two scenarios: (i) the user wants to protect jointly its demand and side information; (ii) the user wants to protect only the information about its demand, but not the side information. To highlight the role of side information, we focus first on the case of a single server (single database). In the first scenario, we prove that the minimum download cost is $K-M$ messages, and in the second scenario it is $\lceil \frac{K}{M+1}\rceil$ messages, which should be compared to $K$ messages, the minimum download cost in the case of no side information. Then, we extend some of our results to the case of the database replicated on multiple servers. Our proof techniques relate PIR with side information to the index coding problem. We leverage this connection to prove converse results, as well as to design achievability schemes.

cs.IT↗

arXiv2016

Codes with Unequal Locality

For a code $\code$, its $i$-th symbol is said to have locality $r$ if its value can be recovered by accessing some other $r$ symbols of $\code$. Locally repairable codes (LRCs) are the family of codes such that every symbol has locality $r$. In this paper, we focus on (linear) codes whose individual symbols can be partitioned into disjoint subsets such that the symbols in one subset have different locality than the symbols in other. We call such codes as "codes with unequal locality". For codes with "unequal information locality", we compute a tight upper bound on the minimum distance as a function of number of information symbols of each locality. We demonstrate that the construction of Pyramid codes can be adapted to design codes with unequal information locality that achieve the minimum distance bound. This result generalizes the classical result of Gopalan et al. for codes with unequal locality. Next, we consider codes with "unequal all symbol locality", and establish an upper bound on the minimum distance as a function of number of symbols of each locality. We show that the construction based on rank-metric codes by Silberstein et al. can be adapted to obtain codes with unequal all symbol locality that achieve the minimum distance bound. Finally, we introduce the concept of "locality requirement" on a code, which can be viewed as a recoverability requirement on symbols. Information locality requirement on a code essentially specifies the minimum number of information symbols of different localities that must be present in the code. We present a greedy algorithm that assigns localities to information symbols so as to maximize the minimum distance among all codes that satisfy a given locality requirement.

cs.IT↗

arXiv2015

Coding against a Limited-view Adversary: The Effect of Causality and Feedback

We consider the problem of communication over a multi-path network in the presence of a causal adversary. The limited-view causal adversary is able to eavesdrop on a subset of links and also jam on a potentially overlapping subset of links based on the current and past information. To ensure that the communication takes place reliably and secretly, resilient network codes with necessary redundancy are needed. We study two adversarial models - additive and overwrite jamming and we optionally assume passive feedback from decoder to encoder, i.e., the encoder sees everything that the decoder sees. The problem assumes transmissions are in the large alphabet regime. For both jamming models, we find the capacity under four scenarios - reliability without feedback, reliability and secrecy without feedback, reliability with passive feedback, reliability and secrecy with passive feedback. We observe that, in comparison to the non-causal setting, the capacity with a causal adversary is strictly increased for a wide variety of parameter settings and present our intuition through several examples.

cs.IT↗

arXiv2014

Weakly Secure Regenerating Codes for Distributed Storage

We consider the problem of secure distributed data storage under the paradigm of \emph{weak security}, in which no \emph{meaningful information} is leaked to the eavesdropper. More specifically, the eavesdropper cannot get any information about any individual message file or a small group of files. The key benefit of the weak security paradigm is that it incurs no loss in the storage capacity, which makes it practically appealing. In this paper, we present a coding scheme, using a coset coding based outer code and a Product-Matrix Minimum Bandwidth Regenerating code (proposed by Rashmi et al.) as an inner code, that achieves weak security when the eavesdropper can observe any single storage node. We show that the proposed construction has good security properties and requires small finite field size.

cs.IT↗

arXiv2014

Reliable, Deniable, and Hidable Communication over Multipath Networks

We consider the scenario wherein Alice wants to (potentially) communicate to the intended receiver Bob over a network consisting of multiple parallel links in the presence of a passive eavesdropper Willie, who observes an unknown subset of links. A primary goal of our communication protocol is to make the communication "deniable", {\it i.e.}, Willie should not be able to {\it reliably} estimate whether or not Alice is transmitting any {\it covert} information to Bob. Moreover, if Alice is indeed actively communicating, her covert messages should be information-theoretically "hidable" in the sense that Willie's observations should not {\it leak any information} about Alice's (potential) message to Bob -- our notion of hidability is slightly stronger than the notion of information-theoretic strong secrecy well-studied in the literature, and may be of independent interest. It can be shown that deniability does not imply either hidability or (weak or strong) information-theoretic secrecy; nor does any form of information-theoretic secrecy imply deniability. We present matching inner and outer bounds on the capacity for deniable and hidable communication over {\it multipath networks}.

cs.IT↗