arXiv subjects

Vipul Gupta

Publications and source records attributed to Vipul Gupta.

43 records · Page 3Linked to original sources

arXiv2020

Serverless Straggler Mitigation using Local Error-Correcting Codes

Inexpensive cloud services, such as serverless computing, are often vulnerable to straggling nodes that increase end-to-end latency for distributed computation. We propose and implement simple yet principled approaches for straggler mitigation in serverless systems for matrix multiplication and evaluate them on several common applications from machine learning and high-performance computing. The proposed schemes are inspired by error-correcting codes and employ parallel encoding and decoding over the data stored in the cloud using serverless workers. This creates a fully distributed computing framework without using a master node to conduct encoding or decoding, which removes the computation, communication and storage bottleneck at the master. On the theory side, we establish that our proposed scheme is asymptotically optimal in terms of decoding time and provide a lower bound on the number of stragglers it can tolerate with high probability. Through extensive experiments, we show that our scheme outperforms existing schemes such as speculative execution and other coding theoretic methods by at least 25%.

cs.DC↗

arXiv2020

Stochastic Weight Averaging in Parallel: Large-Batch Training that Generalizes Well

We propose Stochastic Weight Averaging in Parallel (SWAP), an algorithm to accelerate DNN training. Our algorithm uses large mini-batches to compute an approximate solution quickly and then refines it by averaging the weights of multiple models computed independently and in parallel. The resulting models generalize equally well as those trained with small mini-batches but are produced in a substantially shorter time. We demonstrate the reduction in training time and the good generalization performance of the resulting models on the computer vision datasets CIFAR10, CIFAR100, and ImageNet.

cs.LG↗

arXiv2019

OverSketch: Approximate Matrix Multiplication for the Cloud

We propose OverSketch, an approximate algorithm for distributed matrix multiplication in serverless computing. OverSketch leverages ideas from matrix sketching and high-performance computing to enable cost-efficient multiplication that is resilient to faults and straggling nodes pervasive in low-cost serverless architectures. We establish statistical guarantees on the accuracy of OverSketch and empirically validate our results by solving a large-scale linear program using interior-point methods and demonstrate a 34% reduction in compute time on AWS Lambda.

cs.DC↗

arXiv2017

A Sequential Approximation Framework for Coded Distributed Optimization

Building on the previous work of Lee et al. and Ferdinand et al. on coded computation, we propose a sequential approximation framework for solving optimization problems in a distributed manner. In a distributed computation system, latency caused by individual processors ("stragglers") usually causes a significant delay in the overall process. The proposed method is powered by a sequential computation scheme, which is designed specifically for systems with stragglers. This scheme has the desirable property that the user is guaranteed to receive useful (approximate) computation results whenever a processor finishes its subtask, even in the presence of uncertain latency. In this paper, we give a coding theorem for sequentially computing matrix-vector multiplications, and the optimality of this coding scheme is also established. As an application of the results, we demonstrate solving optimization problems using a sequential approximation approach, which accelerates the algorithm in a distributed system with stragglers.

cs.IT↗

arXiv2017

On Secure Communication using RF Energy Harvesting Two-Way Untrusted Relay

We focus on a scenario where two wireless source nodes wish to exchange confidential information via an RF energy harvesting untrusted two-way relay. Despite its cooperation in forwarding the information, the relay is considered untrusted out of the concern that it might attempt to decode the confidential information that is being relayed. To discourage the eavesdropping intention of the relay, we use a friendly jammer. Under the total power constraint, to maximize the sum-secrecy rate, we allocate the power among the sources and the jammer optimally and calculate the optimal power splitting ratio to balance between the energy harvesting and the information processing at the relay. We further examine the effect of imperfect channel state information at both sources on the sum-secrecy rate. Numerical results highlight the role of the jammer in achieving the secure communication under channel estimation errors. We have shown that, as the channel estimation error on any of the channels increases, the power allocated to the jammer decreases to abate the interference caused to the confidential information reception due to the imperfect cancellation of jammer's signal.

cs.IT↗

arXiv2015

Iterative Matrix Inversion Based Low Complexity Detection in Large/Massive MIMO Systems

Linear detectors such as zero forcing (ZF) or minimum mean square error (MMSE) are imperative for large/massive MIMO systems for both the downlink and uplink scenarios. However these linear detectors require matrix inversion which is computationally expensive for such huge systems. In this paper, we assert that calculating an exact inverse is not necessary to find the ZF/MMSE solution and an approximate inverse would yield a similar performance. This is possible if the quantized solution calculated using the approximate inverse is same as the one calculated using the exact inverse. We quantify the amount of approximation that can be tolerated for this to happen. Motivated by this, we propose to use the existing iterative methods for obtaining low complexity approximate inverses. We show that, after a sufficient number of iterations, the inverse using iterative methods can provide a similar error performance. In addition, we also show that the advantage of using an approximate inverse is not limited to linear detectors but can be extended to non linear detectors such as sphere decoders (SD). An approximate inverse can be used for any SD that requires matrix inversion. We prove that application of approximate inverse leads to a smaller radius, which in turn reduces the search space leading to reduction in complexity. Numerical results corroborate our claim that using approximate matrix inversion reduces decoding complexity in large/massive MIMO systems with no loss in error performance.

cs.IT↗

arXiv2015

Joint Sparsity Pattern Recovery with 1-bit Compressive Sensing in Sensor Networks

We study the problem of jointly sparse support recovery with 1-bit compressive measurements in a sensor network. Sensors are assumed to observe sparse signals having the same but unknown sparse support. Each sensor quantizes its measurement vector element-wise to 1-bit and transmits the quantized observations to a fusion center. We develop a computationally tractable support recovery algorithm which minimizes a cost function defined in terms of the likelihood function and the $l_{1,\infty}$ norm. We observe that even with noisy 1-bit measurements, jointly sparse support can be recovered accurately with multiple sensors each collecting only a small number of measurements.

cs.IT↗