arXiv subjects

Ashwin Pananjady

Publications and source records attributed to Ashwin Pananjady.

47 records · Page 3Linked to original sources

arXiv2018

Existence of Stein Kernels under a Spectral Gap, and Discrepancy Bound

We establish existence of Stein kernels for probability measures on $\mathbb{R}^d$ satisfying a Poincaré inequality, and obtain bounds on the Stein discrepancy of such measures. Applications to quantitative central limit theorems are discussed, including a new CLT in Wasserstein distance $W_2$ with optimal rate and dependence on the dimension. As a byproduct, we obtain a stability version of an estimate of the Poincaré constant of probability measures under a second moment constraint. The results extend more generally to the setting of converse weighted Poincaré inequalities. The proof is based on simple arguments of calculus of variations. Further, we establish two general properties enjoyed by the Stein discrepancy, holding whenever a Stein kernel exists: Stein discrepancy is strictly decreasing along the CLT, and it controls the skewness of a random vector.

math.PR↗

arXiv2018

Gradient Diversity: a Key Ingredient for Scalable Distributed Learning

It has been experimentally observed that distributed implementations of mini-batch stochastic gradient descent (SGD) algorithms exhibit speedup saturation and decaying generalization ability beyond a particular batch-size. In this work, we present an analysis hinting that high similarity between concurrently processed gradients may be a cause of this performance degradation. We introduce the notion of gradient diversity that measures the dissimilarity between concurrent gradient updates, and show its key role in the performance of mini-batch SGD. We prove that on problems with high gradient diversity, mini-batch SGD is amenable to better speedups, while maintaining the generalization performance of serial (one sample) SGD. We further establish lower bounds on convergence where mini-batch SGD slows down beyond a particular batch-size, solely due to the lack of gradient diversity. We provide experimental evidence indicating the key role of gradient diversity in distributed learning, and discuss how heuristics like dropout, Langevin dynamics, and quantization can improve it.

cs.LG↗

arXiv2017

Worst-case vs Average-case Design for Estimation from Fixed Pairwise Comparisons

Pairwise comparison data arises in many domains, including tournament rankings, web search, and preference elicitation. Given noisy comparisons of a fixed subset of pairs of items, we study the problem of estimating the underlying comparison probabilities under the assumption of strong stochastic transitivity (SST). We also consider the noisy sorting subclass of the SST model. We show that when the assignment of items to the topology is arbitrary, these permutation-based models, unlike their parametric counterparts, do not admit consistent estimation for most comparison topologies used in practice. We then demonstrate that consistent estimation is possible when the assignment of items to the topology is randomized, thus establishing a dichotomy between worst-case and average-case designs. We propose two estimators in the average-case setting and analyze their risk, showing that it depends on the comparison topology only through the degree sequence of the topology. The rates achieved by these estimators are shown to be optimal for a large class of graphs. Our results are corroborated by simulations on multiple comparison topologies.

cs.LG↗

arXiv2017

Denoising Linear Models with Permuted Data

The multivariate linear regression model with shuffled data and additive Gaussian noise arises in various correspondence estimation and matching problems. Focusing on the denoising aspect of this problem, we provide a characterization the minimax error rate that is sharp up to logarithmic factors. We also analyze the performance of two versions of a computationally efficient estimator, and establish their consistency for a large range of input parameters. Finally, we provide an exact algorithm for the noiseless problem and demonstrate its performance on an image point-cloud matching task. Our analysis also extends to datasets with outliers.

stat.ML↗

arXiv2016

Wasserstein Stability of the Entropy Power Inequality for Log-Concave Densities

We establish quantitative stability results for the entropy power inequality (EPI). Specifically, we show that if uniformly log-concave densities nearly saturate the EPI, then they must be close to Gaussian densities in the quadratic Wasserstein distance. Further, if one of the densities is log-concave and the other is Gaussian, then the deficit in the EPI can be controlled in terms of the $L^1$-Wasserstein distance. As a counterpoint, an example shows that the EPI can be unstable with respect to the quadratic Wasserstein distance when densities are uniformly log-concave on sets of measure arbitrarily close to one. Our stability results can be extended to non-log-concave densities, provided certain regularity conditions are met. The proofs are based on optimal transportation.

cs.IT↗

arXiv2016

Linear Regression with an Unknown Permutation: Statistical and Computational Limits

Consider a noisy linear observation model with an unknown permutation, based on observing $y = Π^* A x^* + w$, where $x^* \in \mathbb{R}^d$ is an unknown vector, $Π^*$ is an unknown $n \times n$ permutation matrix, and $w \in \mathbb{R}^n$ is additive Gaussian noise. We analyze the problem of permutation recovery in a random design setting in which the entries of the matrix $A$ are drawn i.i.d. from a standard Gaussian distribution, and establish sharp conditions on the SNR, sample size $n$, and dimension $d$ under which $Π^*$ is exactly and approximately recoverable. On the computational front, we show that the maximum likelihood estimate of $Π^*$ is NP-hard to compute, while also providing a polynomial time algorithm when $d =1$.

math.ST↗

arXiv2015

Compressing Sparse Sequences under Local Decodability Constraints

We consider a variable-length source coding problem subject to local decodability constraints. In particular, we investigate the blocklength scaling behavior attainable by encodings of $r$-sparse binary sequences, under the constraint that any source bit can be correctly decoded upon probing at most $d$ codeword bits. We consider both adaptive and non-adaptive access models, and derive upper and lower bounds that often coincide up to constant factors. Notably, such a characterization for the fixed-blocklength analog of our problem remains unknown, despite considerable research over the last three decades. Connections to communication complexity are also briefly discussed.

cs.IT↗

arXiv2014

The Online Disjoint Set Cover Problem and its Applications

Given a universe $U$ of $n$ elements and a collection of subsets $\mathcal{S}$ of $U$, the maximum disjoint set cover problem (DSCP) is to partition $\mathcal{S}$ into as many set covers as possible, where a set cover is defined as a collection of subsets whose union is $U$. We consider the online DSCP, in which the subsets arrive one by one (possibly in an order chosen by an adversary), and must be irrevocably assigned to some partition on arrival with the objective of minimizing the competitive ratio. The competitive ratio of an online DSCP algorithm $A$ is defined as the maximum ratio of the number of disjoint set covers obtained by the optimal offline algorithm to the number of disjoint set covers obtained by $A$ across all inputs. We propose an online algorithm for solving the DSCP with competitive ratio $\ln n$. We then show a lower bound of $Ω(\sqrt{\ln n})$ on the competitive ratio for any online DSCP algorithm. The online disjoint set cover problem has wide ranging applications in practice, including the online crowd-sourcing problem, the online coverage lifetime maximization problem in wireless sensor networks, and in online resource allocation problems.

cs.DS↗

arXiv2014

Optimally Approximating the Coverage Lifetime of Wireless Sensor Networks

We consider the problem of maximizing the lifetime of coverage (MLCP) of targets in a wireless sensor network with battery-limited sensors. We first show that the MLCP cannot be approximated within a factor less than $\ln n$ by any polynomial time algorithm, where $n$ is the number of targets. This provides closure to the long-standing open problem of showing optimality of previously known $\ln n$ approximation algorithms. We also derive a new $\ln n$ approximation to the MLCP by showing a $\ln n$ approximation to the maximum disjoint set cover problem (DSCP), which has many advantages over previous MLCP algorithms, including an easy extension to the $k$-coverage problem. We then present an improvement (in certain cases) to the $\ln n$ algorithm in terms of a newly defined quantity "expansiveness" of the network. For the special one-dimensional case, where each sensor can monitor a contiguous region of possibly different lengths, we show that the MLCP solution is equal to the DSCP solution, and can be found in polynomial time. Finally, for the special two-dimensional case, where each sensor can monitor a circular area with a given radius around itself, we combine existing results to derive a $1+ε$ approximation algorithm for solving MLCP for any $ε>0$.

cs.NI↗

arXiv2014

On the Complexity of Making a Distinguished Vertex Minimum or Maximum Degree by Vertex Deletion

In this paper, we investigate the approximability of two node deletion problems. Given a vertex weighted graph $G=(V,E)$ and a specified, or "distinguished" vertex $p \in V$, MDD(min) is the problem of finding a minimum weight vertex set $S \subseteq V\setminus \{p\}$ such that $p$ becomes the minimum degree vertex in $G[V \setminus S]$; and MDD(max) is the problem of finding a minimum weight vertex set $S \subseteq V\setminus \{p\}$ such that $p$ becomes the maximum degree vertex in $G[V \setminus S]$. These are known $NP$-complete problems and have been studied from the parameterized complexity point of view in previous work. Here, we prove that for any $ε> 0$, both the problems cannot be approximated within a factor $(1 - ε)\log n$, unless $NP \subseteq DTIME(n^{\log\log n})$. We also show that for any $ε> 0$, MDD(min) cannot be approximated within a factor $(1 -ε)\log n$ on bipartite graphs, unless $NP \subseteq DTIME(n^{\log\log n})$, and that for any $ε> 0$, MDD(max) cannot be approximated within a factor $(1/2 - ε)\log n$ on bipartite graphs, unless $NP \subseteq DTIME(n^{\log\log n})$. We give an $O(\log n)$ factor approximation algorithm for MDD(max) on general graphs, provided the degree of $p$ is $O(\log n)$. We then show that if the degree of $p$ is $n-O(\log n)$, a similar result holds for MDD(min). We prove that MDD(max) is $APX$-complete on 3-regular unweighted graphs and provide an approximation algorithm with ratio $1.583$ when $G$ is a 3-regular unweighted graph. In addition, we show that MDD(min) can be solved in polynomial time when $G$ is a regular graph of constant degree.

cs.DS↗

arXiv2013

Maximizing Utility Among Selfish Users in Social Groups

We consider the problem of a social group of users trying to obtain a "universe" of files, first from a server and then via exchange amongst themselves. We consider the selfish file-exchange paradigm of give-and-take, whereby two users can exchange files only if each has something unique to offer the other. We are interested in maximizing the number of users who can obtain the universe through a schedule of file-exchanges. We first present a practical paradigm of file acquisition. We then present an algorithm which ensures that at least half the users obtain the universe with high probability for $n$ files and $m=O(\log n)$ users when $n\rightarrow\infty$, thereby showing an approximation ratio of 2. Extending these ideas, we show a $1+ε_1$ - approximation algorithm for $m=O(n)$, $ε_1>0$ and a $(1+z)/2 +ε_2$ - approximation algorithm for $m=O(n^z)$, $z>1$, $ε_2>0$. Finally, we show that for any $m=O(e^{o(n)})$, there exists a schedule of file exchanges which ensures that at least half the users obtain the universe.

cs.NI↗