arXiv subjects

Shashank Singh

Publications and source records attributed to Shashank Singh.

35 records · Page 2Linked to original sources

arXiv2020

Nonparametric Density Estimation & Convergence Rates for GANs under Besov IPM Losses

We study the problem of estimating a nonparametric probability density under a large family of losses called Besov IPMs, which include, for example, $\mathcal{L}^p$ distances, total variation distance, and generalizations of both Wasserstein and Kolmogorov-Smirnov distances. For a wide variety of settings, we provide both lower and upper bounds, identifying precisely how the choice of loss function and assumptions on the data interact to determine the minimax optimal convergence rate. We also show that linear distribution estimates, such as the empirical distribution or kernel density estimator, often fail to converge at the optimal rate. Our bounds generalize, unify, or improve several recent and classical results. Moreover, IPMs can be used to formalize a statistical model of generative adversarial networks (GANs). Thus, we show how our results imply bounds on the statistical error of a GAN, showing, for example, that GANs can strictly outperform the best linear estimator.

math.ST↗

arXiv2019

Minimax Distribution Estimation in Wasserstein Distance

The Wasserstein metric is an important measure of distance between probability distributions, with applications in machine learning, statistics, probability theory, and data analysis. This paper provides upper and lower bounds on statistical minimax rates for the problem of estimating a probability distribution under Wasserstein loss, using only metric properties, such as covering and packing numbers, of the sample space, and weak moment assumptions on the probability distributions.

math.ST↗

arXiv2019

Simplified Algorithm for Dynamic Demand Response in Smart Homes Under Smart Grid Environment

Under Smart Grid environment, the consumers may respond to incentive--based smart energy tariffs for a particular consumption pattern. Demand Response (DR) is a portfolio of signaling schemes from the utility to the consumers for load shifting/shedding with a given deadline. The signaling schemes include Time--of--Use (ToU) pricing, Maximum Demand Limit (MDL) signals etc. This paper proposes a DR algorithm which schedules the operation of home appliances/loads through a minimization problem. The category of loads and their operational timings in a day have been considered as the operational parameters of the system. These operational parameters determine the dynamic priority of a load, which is an intermediate step of this algorithm. The ToU pricing, MDL signals, and the dynamic priority of loads are the constraints in this formulated minimization problem, which yields an optimal schedule of operation for each participating load within the consumer provided duration. The objective is to flatten the daily load curve of a smart home by distributing the operation of its appliances in possible low--price intervals without violating the MDL constraint. This proposed algorithm is simulated in MATLAB environment against various test cases. The obtained results are plotted to depict significant monetary savings and flattened load curves.

eess.SP↗

arXiv2019

Smart Load Node for Nonsmart Load Under Smart Grid Paradigm: A New Home Energy Management System

This article presents a novel approach for efficient operation of non-smart household appliances under smart grid environment using the proposed smart load node (SLN). In real world scenario, there are so many non-smart loads currently in use and embedding appliance specific intelligence into them to make them as smart loads will be more expensive compared to the proposed SLN, which is a common solution for all types of non-smart loads. This makes the proposed low-cost SLN, which neither requires any infrastructural change in the electrical wiring of a house nor any constructional change in home appliances at the manufacturing stage and at the consumer end, as a feasible solution for intelligent operation of non-smart home appliances under smart grid environment. The SLNs, which are placed in a home like distributed wireless sensor nodes, form a home area network (HAN). The HAN includes a load management unit (LMU) which acts as master for all distributed SLNs. Wi-Fi is chosen as a medium of communication in the HAN. The LMU incorporates load management algorithm which is written in Python script.

eess.SP↗

arXiv2019

DARC: Differentiable ARchitecture Compression

In many learning situations, resources at inference time are significantly more constrained than resources at training time. This paper studies a general paradigm, called Differentiable ARchitecture Compression (DARC), that combines model compression and architecture search to learn models that are resource-efficient at inference time. Given a resource-intensive base architecture, DARC utilizes the training data to learn which sub-components can be replaced by cheaper alternatives. The high-level technique can be applied to any neural architecture, and we report experiments on state-of-the-art convolutional neural networks for image classification. For a WideResNet with $97.2\%$ accuracy on CIFAR-10, we improve single-sample inference speed by $2.28\times$ and memory footprint by $5.64\times$, with no accuracy loss. For a ResNet with $79.15\%$ Top1 accuracy on ImageNet, we improve batch inference speed by $1.29\times$ and memory footprint by $3.57\times$ with $1\%$ accuracy loss. We also give theoretical Rademacher complexity bounds in simplified cases, showing how DARC avoids overfitting despite over-parameterization.

cs.LG↗

arXiv2018

Nonparametric Density Estimation under Adversarial Losses

We study minimax convergence rates of nonparametric density estimation under a large class of loss functions called "adversarial losses", which, besides classical $\mathcal{L}^p$ losses, includes maximum mean discrepancy (MMD), Wasserstein distance, and total variation distance. These losses are closely related to the losses encoded by discriminator networks in generative adversarial networks (GANs). In a general framework, we study how the choice of loss and the assumed smoothness of the underlying density together determine the minimax rate. We also discuss implications for training GANs based on deep ReLU networks, and more general connections to learning implicit generative models in a minimax statistical sense.

math.ST↗

arXiv2018

Regression Based Approach for Measurement of Current in Single-Phase Smart Energy Meter

In the present scenario rate of utilization of electrical energy is continuously increasing, to preserve the non-renewable energy resources, efficient consumption of electrical energy is one of the most important needs of this era. Smart Energy Meter (SEM) plays a vital role in efficient use of electrical energy in consumer premises. Accuracy in the measurement of current is one of the key factors to be considered while measuring energy consumption. This paper introduces Arduino based low cost single-phase SEM and proposes a linear regression based technique for accurate measurement of current. The proposed current measurement approach gives accurate measurement for linear as well as highly non-linear loads. The SEM built in the laboratory has data transfer capability and also embedded with Demand Response (DR) feature. Block Rate Tariff structure used by Tamil Nadu Generation and Distribution Company is considered to incorporate DR that warns the consumers whenever their energy consumption reaches the boundary of a block.

eess.SP↗

arXiv2018

Minimax Estimation of Quadratic Fourier Functionals

We study estimation of (semi-)inner products between two nonparametric probability distributions, given IID samples from each distribution. These products include relatively well-studied classical $\mathcal{L}^2$ and Sobolev inner products, as well as those induced by translation-invariant reproducing kernels, for which we believe our results are the first. We first propose estimators for these quantities, and the induced (semi)norms and (pseudo)metrics. We then prove non-asymptotic upper bounds on their mean squared error, in terms of weights both of the inner product and of the two distributions, in the Fourier basis. Finally, we prove minimax lower bounds that imply rate-optimality of the proposed estimators over Fourier ellipsoids.

math.ST↗

arXiv2018

On the Reconstruction Risk of Convolutional Sparse Dictionary Learning

Sparse dictionary learning (SDL) has become a popular method for adaptively identifying parsimonious representations of a dataset, a fundamental problem in machine learning and signal processing. While most work on SDL assumes a training dataset of independent and identically distributed samples, a variant known as convolutional sparse dictionary learning (CSDL) relaxes this assumption, allowing more general sequential data sources, such as time series or other dependent data. Although recent work has explored the statistical properties of classical SDL, the statistical properties of CSDL remain unstudied. This paper begins to study this by identifying the minimax convergence rate of CSDL in terms of reconstruction risk, by both upper bounding the risk of an established CSDL estimator and proving a matching information-theoretic lower bound. Our results indicate that consistency in reconstruction risk is possible precisely in the `ultra-sparse' setting, in which the sparsity (i.e., the number of feature occurrences) is in $o(N)$ in terms of the length N of the training sequence. Notably, our results make very weak assumptions, allowing arbitrary dictionaries and dependent measurement noise. Finally, we verify our theoretical results with numerical experiments on synthetic data.

math.ST↗

arXiv2017

Nonparanormal Information Estimation

We study the problem of using i.i.d. samples from an unknown multivariate probability distribution $p$ to estimate the mutual information of $p$. This problem has recently received attention in two settings: (1) where $p$ is assumed to be Gaussian and (2) where $p$ is assumed only to lie in a large nonparametric smoothness class. Estimators proposed for the Gaussian case converge in high dimensions when the Gaussian assumption holds, but are brittle, failing dramatically when $p$ is not Gaussian. Estimators proposed for the nonparametric case fail to converge with realistic sample sizes except in very low dimensions. As a result, there is a lack of robust mutual information estimators for many realistic data. To address this, we propose estimators for mutual information when $p$ is assumed to be a nonparanormal (a.k.a., Gaussian copula) model, a semiparametric compromise between Gaussian and nonparametric extremes. Using theoretical bounds and experiments, we show these estimators strike a practical balance between robustness and scaling with dimensionality.

math.ST↗

arXiv2016

Efficient Nonparametric Smoothness Estimation

Sobolev quantities (norms, inner products, and distances) of probability density functions are important in the theory of nonparametric statistics, but have rarely been used in practice, partly due to a lack of practical estimators. They also include, as special cases, $L^2$ quantities which are used in many applications. We propose and analyze a family of estimators for Sobolev quantities of unknown probability density functions. We bound the bias and variance of our estimators over finite samples, finding that they are generally minimax rate-optimal. Our estimators are significantly more computationally tractable than previous estimators, and exhibit a statistical/computational trade-off allowing them to adapt to computational constraints. We also draw theoretical connections to recent work on fast two-sample testing. Finally, we empirically validate our estimators on synthetic data.

math.ST↗

arXiv2016

Analysis of k-Nearest Neighbor Distances with Application to Entropy Estimation

Estimating entropy and mutual information consistently is important for many machine learning applications. The Kozachenko-Leonenko (KL) estimator (Kozachenko & Leonenko, 1987) is a widely used nonparametric estimator for the entropy of multivariate continuous random variables, as well as the basis of the mutual information estimator of Kraskov et al. (2004), perhaps the most widely used estimator of mutual information in this setting. Despite the practical importance of these estimators, major theoretical questions regarding their finite-sample behavior remain open. This paper proves finite-sample bounds on the bias and variance of the KL estimator, showing that it achieves the minimax convergence rate for certain classes of smooth functions. In proving these bounds, we analyze finite-sample behavior of k-nearest neighbors (k-NN) distance statistics (on which the KL estimator is based). We derive concentration inequalities for k-NN distances and a general expectation bound for statistics of k-NN distances, which may be useful for other analyses of k-NN methods.

math.ST↗

arXiv2016

Finite-Sample Analysis of Fixed-k Nearest Neighbor Density Functional Estimators

We provide finite-sample analysis of a general framework for using k-nearest neighbor statistics to estimate functionals of a nonparametric continuous probability density, including entropies and divergences. Rather than plugging a consistent density estimate (which requires $k \to \infty$ as the sample size $n \to \infty$) into the functional of interest, the estimators we consider fix k and perform a bias correction. This is more efficient computationally, and, as we show in certain cases, statistically, leading to faster convergence rates. Our framework unifies several previous estimators, for most of which ours are the first finite sample guarantees.

math.ST↗

arXiv2016

On Irreducible Polynomials of the Form $b(x^d)$

Let $p$ be a prime and $b(x)$ be an irreducible polynomial of degree $k$ over $\mathbb{F}_p$. Let $d\geq 1$ be an integer. Consider the following question: Is $b(x^d)$ irreducible? We derive necessary conditions for $b(x^d)$ to be irreducible. Further, when the necessary conditions are satisfied, we obtain the probability for $b(x^d)$ to be irreducible.

math.NT↗

arXiv2016

Distributed Gradient Descent in Bacterial Food Search

Communication and coordination play a major role in the ability of bacterial cells to adapt to ever changing environments and conditions. Recent work has shown that such coordination underlies several aspects of bacterial responses including their ability to develop antibiotic resistance. Here we develop a new distributed gradient descent method that helps explain how bacterial cells collectively search for food in harsh environments using extremely limited communication and computational complexity. This method can also be used for computational tasks when agents are facing similarly restricted conditions. We formalize the communication and computation assumptions required for successful coordination and prove that the method we propose leads to convergence even when using a dynamically changing interaction network. The proposed method improves upon prior models suggested for bacterial foraging despite making fewer assumptions. Simulation studies and analysis of experimental data illustrate the ability of the method to explain and further predict several aspects of bacterial swarm food search.

q-bio.QM↗

arXiv2016

Exponential Concentration of a Density Functional Estimator

We analyze a plug-in estimator for a large class of integral functionals of one or more continuous probability densities. This class includes important families of entropy, divergence, mutual information, and their conditional versions. For densities on the $d$-dimensional unit cube $[0,1]^d$ that lie in a $β$-Hölder smoothness class, we prove our estimator converges at the rate $O \left( n^{-\fracβ{β+ d}} \right)$. Furthermore, we prove the estimator is exponentially concentrated about its mean, whereas most previous related results have proven only expected error bounds on estimators.

math.ST↗

arXiv2016

Generalized Exponential Concentration Inequality for Rényi Divergence Estimation

Estimating divergences in a consistent way is of great importance in many machine learning tasks. Although this is a fundamental problem in nonparametric statistics, to the best of our knowledge there has been no finite sample exponential inequality convergence bound derived for any divergence estimators. The main contribution of our work is to provide such a bound for an estimator of Rényi-$α$ divergence for a smooth Hölder class of densities on the $d$-dimensional unit cube $[0, 1]^d$. We also illustrate our theoretical results with a numerical experiment.

cs.IT↗