arXiv subjects

Marianna Pensky

Publications and source records attributed to Marianna Pensky.

At least 37 records · Page 2Linked to original sources

arXiv2018

Density Deconvolution with Small Berkson Errors

The present paper studies density deconvolution in the presence of small Berkson errors, in particular, when the variances of the errors tend to zero as the sample size grows. It is known that when the Berkson errors are present, in some cases, the unknown density estimator can be obtain by simple averaging without using kernels. However, this may not be the case when Berkson errors are asymptotically small. By treating the former case as a kernel estimator with the zero bandwidth, we obtain the optimal expressions for the bandwidth. We show that the density of Berkson errors acts as a regularizer, so that the kernel estimator is unnecessary when the variance of Berkson errors lies above some threshold that depends on the on the shapes of the densities in the model and the number of observations.

math.ST↗

arXiv2018

Probabilistic Sparse Subspace Clustering Using Delayed Association

Discovering and clustering subspaces in high-dimensional data is a fundamental problem of machine learning with a wide range of applications in data mining, computer vision, and pattern recognition. Earlier methods divided the problem into two separate stages of finding the similarity matrix and finding clusters. Similar to some recent works, we integrate these two steps using a joint optimization approach. We make the following contributions: (i) we estimate the reliability of the cluster assignment for each point before assigning a point to a subspace. We group the data points into two groups of "certain" and "uncertain", with the assignment of latter group delayed until their subspace association certainty improves. (ii) We demonstrate that delayed association is better suited for clustering subspaces that have ambiguities, i.e. when subspaces intersect or data are contaminated with outliers/noise. (iii) We demonstrate experimentally that such delayed probabilistic association leads to a more accurate self-representation and final clusters. The proposed method has higher accuracy both for points that exclusively lie in one subspace, and those that are on the intersection of subspaces. (iv) We show that delayed association leads to huge reduction of computational cost, since it allows for incremental spectral clustering.

stat.ML↗

arXiv2018

Anisotropic functional Laplace deconvolution

In the present paper we consider the problem of estimating a three-dimensional function $f$ based on observations from its noisy Laplace convolution. Our study is motivated by the analysis of Dynamic Contrast Enhanced (DCE) imaging data. We construct an adaptive wavelet-Laguerre estimator of $f$, derive minimax lower bounds for the $L^2$-risk when $f$ belongs to a three-dimensional Laguerre-Sobolev ball and demonstrate that the wavelet-Laguerre estimator is adaptive and asymptotically near-optimal in a wide range of Laguerre-Sobolev spaces. We carry out a limited simulations study and show that the estimator performs well in a finite sample setting. Finally, we use the technique for the solution of the Laplace deconvolution problem on the basis of DCE Computerized Tomography data.

stat.ME↗

arXiv2018

Dynamic network models and graphon estimation

In the present paper we consider a dynamic stochastic network model. The objective is estimation of the tensor of connection probabilities $Λ$ when it is generated by a Dynamic Stochastic Block Model (DSBM) or a dynamic graphon. In particular, in the context of the DSBM, we derive a penalized least squares estimator $\widehatΛ$ of $Λ$ and show that $\widehatΛ$ satisfies an oracle inequality and also attains minimax lower bounds for the risk. We extend those results to estimation of $Λ$ when it is generated by a dynamic graphon function. The estimators constructed in the paper are adaptive to the unknown number of blocks in the context of the DSBM or to the smoothness of the graphon function. The technique relies on the vectorization of the model and leads to much simpler mathematical arguments than the ones used previously in the stationary set up. In addition, all results in the paper are non-asymptotic and allow a variety of extensions.

math.ST↗

arXiv2017

Solution of linear ill-posed problems by model selection and aggregation

We consider a general statistical linear inverse problem, where the solution is represented via a known (possibly overcomplete) dictionary that allows its sparse representation. We propose two different approaches. A model selection estimator selects a single model by minimizing the penalized empirical risk over all possible models. By contrast with direct problems, the penalty depends on the model itself rather than on its size only as for complexity penalties. A Q-aggregate estimator averages over the entire collection of estimators with properly chosen weights. Under mild conditions on the dictionary, we establish oracle inequalities both with high probability and in expectation for the two estimators. Moreover, for the latter estimator these inequalities are sharp. The proposed procedures are implemented numerically and their performance is assessed by a simulation study.

stat.ME↗

arXiv2017

Solution of linear ill-posed problems using random dictionaries

In the present paper we consider application of overcomplete dictionaries to solution of general ill-posed linear inverse problems. In the context of regression problems, there has been enormous amount of effort to recover an unknown function using such dictionaries. One of the most popular methods, lasso and its versions, is based on minimizing empirical likelihood and unfortunately, requires stringent assumptions on the dictionary, the, so called, compatibility conditions. Though compatibility conditions are hard to satisfy, it is well known that this can be accomplished by using random dictionaries. In the present paper, we show how one can apply random dictionaries to solution of ill-posed linear inverse problems. We put a theoretical foundation under the suggested methodology and study its performance via simulations.

math.ST↗

arXiv2017

Spectral clustering in the dynamic stochastic block model

In the present paper, we studied a Dynamic Stochastic Block Model (DSBM) under the assumptions that the connection probabilities, as functions of time, are smooth and that at most $s$ nodes can switch their class memberships between two consecutive time points. We estimate the edge probability tensor by a kernel-type procedure and extract the group memberships of the nodes by spectral clustering. The procedure is computationally viable, adaptive to the unknown smoothness of the functional connection probabilities, to the rate $s$ of membership switching and to the unknown number of clusters. In addition, it is accompanied by non-asymptotic guarantees for the precision of estimation and clustering.

stat.ME↗

arXiv2015

Estimation of delta-contaminated density of the random intensity of Poisson data

In the present paper, we constructed an estimator of a delta contaminated mixing density function $g(λ)$ of the intensity $λ$ of the Poisson distribution. The estimator is based on an expansion of the continuous portion $g_0(λ)$ of the unknown pdf over an overcomplete dictionary with the recovery of the coefficients obtained as solution of an optimization problem with Lasso penalty. In order to apply Lasso technique in the, so called, prediction setting where it requires virtually no assumptions on dictionary and, moreover, to ensure fast convergence of Lasso estimator, we use a novel formulation of the optimization problem based on inversion of the dictionary elements. The total estimator of the delta contaminated mixing pdf is obtained using a two-stage iterative procedure. We formulate conditions on the dictionary and the unknown mixing density that yield a sharp oracle inequality for the norm of the difference between $g_0 (λ)$ and its estimator and, thus, obtain a smaller error than in a minimax setting. Numerical simulations and comparisons with the Laguerre functions based estimator recently constructed by Comte and Genon-Catalot (2015) also show advantages of our procedure. At last, we apply the technique developed in the paper to estimation of a delta contaminated mixing density of the Poisson intensity of the Saturn's rings data.

stat.ME↗

arXiv2015

Minimax theory of estimation of linear functionals of the deconvolution density with or without sparsity

The present paper considers a problem of estimating a linear functional $Φ=\int_{-\infty}^\infty φ(x) f(x)dx$ of an unknown deconvolution density $f$ on the basis of i.i.d. observations $Y_i = θ_i + ξ_i$ where $ξ_i$ has a known pdf $g$ and $f$ is the pdf of $θ_i$. Although various aspects and particular cases of this problem have been treated by a number of authors, there are still many gaps. In particular, there are no minimax lower bounds for an estimator of $Φ$ for an arbitrary function $φ$. The general upper risk bounds cover only the case when the Fourier transform of $φ$ exists. Moreover, no theory exists for estimating $Φ$ when vector of observations is sparse. In addition, until now, the related problem of estimation of functionals $Φ_n = n^{-1} \sum_{i=1}^n φ(θ_i)$ in indirect observations have been treated as a separate problem with no connection to estimation of $Φ$. The objective of the present paper is to fill in the gaps and develop the general minimax theory of estimation of $Φ$ and $Φ_n$. We offer a general approach to estimation of $Φ$ (and $Φ_n$) and provide the upper and the minimax lower risk bounds in the case when function $φ$ is square integrable. Furthermore, we extend the theory to the case when Fourier transform of $φ$ does not exist and $Φ$ can be presented as a linear functional of the Fourier transform of $f$ and its derivatives. Finally, we generalize our results to handle the situation when vector $θ$ is sparse. As a direct application of the proposed theory, we obtain multiple new results and automatically recover existing ones for a variety of problems such as estimation of the $(2M+1)$-th absolute moment or a generalized moment of the deconvolution density, estimation of the mixing cdf or estimation of the mixing pdf with classical and Berkson errors.

math.ST↗

arXiv2015

Solution of linear ill-posed problems using overcomplete dictionaries

In the present paper we consider application of overcomplete dictionaries to solution of general ill-posed linear inverse problems. Construction of an adaptive optimal solution for such problems usually relies either on a singular value decomposition or representation of the solution via an orthonormal basis. The shortcoming of both approaches lies in the fact that, in many situations, neither the eigenbasis of the linear operator nor a standard orthonormal basis constitutes an appropriate collection of functions for sparse representation of the unknown function. In the context of regression problems, there have been an enormous amount of effort to recover an unknown function using an overcomplete dictionary. One of the most popular methods, Lasso, is based on minimizing the empirical likelihood and requires stringent assumptions on the dictionary, the, so called, compatibility conditions. While these conditions may be satisfied for the original dictionary functions, they usually do not hold for their images due to contraction imposed by the linear operator. In what follows, we bypass this difficulty by a novel approach which is based on inverting each of the dictionary functions and matching the resulting expansion to the true function, thus, avoiding unrealistic assumptions on the dictionary and using Lasso in a predictive setting. We examine both the white noise and the observational model formulations and also discuss how exact inverse images of the dictionary functions can be replaced by their approximate counterparts. Furthermore, we show how the suggested methodology can be extended to the problem of estimation of a mixing density in a continuous mixture. For all the situations listed above, we provide the oracle inequalities for the risk in a finite sample setting. Simulation studies confirm good computational properties of the Lasso-based technique.

stat.ME↗

arXiv2015

Laplace deconvolution on the basis of time domain data and its application to Dynamic Contrast Enhanced imaging

In the present paper we consider the problem of Laplace deconvolution with noisy discrete non-equally spaced observations on a finite time interval. We propose a new method for Laplace deconvolution which is based on expansions of the convolution kernel, the unknown function and the observed signal over Laguerre functions basis (which acts as a surrogate eigenfunction basis of the Laplace convolution operator) using regression setting. The expansion results in a small system of linear equations with the matrix of the system being triangular and Toeplitz. Due to this triangular structure, there is a common number $m$ of terms in the function expansions to control, which is realized via complexity penalty. The advantage of this methodology is that it leads to very fast computations, produces no boundary effects due to extension at zero and cut-off at $T$ and provides an estimator with the risk within a logarithmic factor of the oracle risk. We emphasize that, in the present paper, we consider the true observational model with possibly nonequispaced observations which are available on a finite interval of length $T$ which appears in many different contexts, and account for the bias associated with this model (which is not present when $T\rightarrow\infty$). The study is motivated by perfusion imaging using a short injection of contrast agent, a procedure which is applied for medical assessment of micro-circulation within tissues such as cancerous tumors. Presence of a tuning parameter $a$ allows to choose the most advantageous time units, so that both the kernel and the unknown right hand side of the equation are well represented for the deconvolution. The methodology is illustrated by an extensive simulation study and a real data example which confirms that the proposed technique is fast, efficient, accurate, usable from a practical point of view and very competitive.

stat.ME↗

arXiv2014

Sparse high-dimensional varying coefficient model: non-asymptotic minimax study

The objective of the present paper is to develop a minimax theory for the varying coefficient model in a non-asymptotic setting. We consider a high-dimensional sparse varying coefficient model where only few of the covariates are present and only some of those covariates are time dependent. Our analysis allows the time dependent covariates to have different degrees of smoothness and to be spatially inhomogeneous. We develop the minimax lower bounds for the quadratic risk and construct an adaptive estimator which attains those lower bounds within a constant (if all time-dependent covariates are spatially homogeneous) or logarithmic factor of the number of observations.

math.ST↗

arXiv2013

Spatially inhomogeneous linear inverse problems with possible singularities

The objective of the present paper is to introduce the concept of a spatially inhomogeneous linear inverse problem where the degree of ill-posedness of operator $Q$ depends not only on the scale but also on location. In this case, the rates of convergence are determined by the interaction of four parameters, the smoothness and spatial homogeneity of the unknown function $f$ and degrees of ill-posedness and spatial inhomogeneity of operator $Q$. Estimators obtained in the paper are based either on wavelet-vaguelette decomposition (if the norms of all vaguelettes are finite) or on a hybrid of wavelet-vaguelette decomposition and Galerkin method (if vaguelettes in the neighborhood of the singularity point have infinite norms). The hybrid estimator is a combination of a linear part in the vicinity of the singularity point and the nonlinear block thresholding wavelet estimator elsewhere. To attain adaptivity, an optimal resolution level for the linear, singularity affected, portion of the estimator is obtained using Lepski [Theory Probab. Appl. 35 (1990) 454-466 and 36 (1991) 682-697] method and is used subsequently as the lowest resolution level for the nonlinear wavelet estimator. We show that convergence rates of the hybrid estimator lie within a logarithmic factor of the optimal minimax convergence rates. The theory presented in the paper is supplemented by examples of deconvolution with a spatially inhomogeneous kernel and deconvolution in the presence of locally extreme noise or extremely inhomogeneous design. The first two problems are examined via a limited simulation study which demonstrates advantages of the hybrid estimator when the degree of spatial inhomogeneity is high. In addition, we apply the technique to recovery of a convolution signal transmitted via amplitude modulation.

math.ST↗

arXiv2013

Adaptive Nonparametric Empirical Bayes Estimation Via Wavelet Series: the Minimax Study

In the present paper, we derive lower bounds for the risk of the nonparametric empirical Bayes estimators. In order to attain the optimal convergence rate, we propose generalization of the linear empirical Bayes estimation method which takes advantage of the flexibility of the wavelet techniques. We present an empirical Bayes estimator as a wavelet series expansion and estimate coefficients by minimizing the prior risk of the estimator. As a result, estimation of wavelet coefficients requires solution of a well-posed low-dimensional sparse system of linear equations. The dimension of the system depends on the size of wavelet support and smoothness of the Bayes estimator. An adaptive choice of the resolution level is carried out using Lepski (1997) method. The method is computationally efficient and provides asymptotically optimal adaptive EB estimators. The theory is supplemented by numerous examples.

math.ST↗

arXiv2013

Anisotropic Denoising in Functional Deconvolution Model with Dimension-free Convergence Rates

In the present paper we consider the problem of estimating a periodic $(r+1)$-dimensional function $f$ based on observations from its noisy convolution. We construct a wavelet estimator of $f$, derive minimax lower bounds for the $L^2$-risk when $f$ belongs to a Besov ball of mixed smoothness and demonstrate that the wavelet estimator is adaptive and asymptotically near-optimal within a logarithmic factor, in a wide range of Besov balls. We prove in particular that choosing this type of mixed smoothness leads to rates of convergence which are free of the "curse of dimensionality" and, hence, are higher than usual convergence rates when $r$ is large. The problem studied in the paper is motivated by seismic inversion which can be reduced to solution of noisy two-dimensional convolution equations that allow to draw inference on underground layer structures along the chosen profiles. The common practice in seismology is to recover layer structures separately for each profile and then to combine the derived estimates into a two-dimensional function. By studying the two-dimensional version of the model, we demonstrate that this strategy usually leads to estimators which are less accurate than the ones obtained as two-dimensional functional deconvolutions. Indeed, we show that unless the function $f$ is very smooth in the direction of the profiles, very spatially inhomogeneous along the other direction and the number of profiles is very limited, the functional deconvolution solution has a much better precision compared to a combination of $M$ solutions of separate convolution equations. A limited simulation study in the case of $r=1$ confirms theoretical claims of the paper.

math.ST↗

arXiv2013

Multichannel Deconvolution with Long-Range Dependence: A Minimax Study

We consider the problem of estimating the unknown response function in the multichannel deconvolution model with long-range dependent Gaussian errors. We do not limit our consideration to a specific type of long-range dependence rather we assume that the errors should satisfy a general assumption in terms of the smallest and larger eigenvalues of their covariance matrices. We derive minimax lower bounds for the quadratic risk in the proposed multichannel deconvolution model when the response function is assumed to belong to a Besov ball and the blurring function is assumed to possess some smoothness properties, including both regular-smooth and super-smooth convolutions. Furthermore, we propose an adaptive wavelet estimator of the response function that is asymptotically optimal (in the minimax sense), or near-optimal within a logarithmic factor, in a wide range of Besov balls. It is shown that the optimal convergence rates depend on the balance between the smoothness parameter of the response function, the kernel parameters of the blurring function, the long memory parameters of the errors, and how the total number of observations is distributed among the total number of channels. Some examples of inverse problems in mathematical physics where one needs to recover initial or boundary conditions on the basis of observations from a noisy solution of a partial differential equation are used to illustrate the application of the theory we developed. The optimal convergence rates and the adaptive estimators we consider extend the ones studied by Pensky and Sapatinas (2009, 2010) for independent and identically distributed Gaussian errors to the case of long-range dependent Gaussian errors.

math.ST↗

arXiv2013

Non-asymptotic approach to varying coefficient model

In the present paper we consider the varying coefficient model which represents a useful tool for exploring dynamic patterns in many applications. Existing methods typically provide asymptotic evaluation of precision of estimation procedures under the assumption that the number of observations tends to infinity. In practical applications, however, only a finite number of measurements are available. In the present paper we focus on a non-asymptotic approach to the problem. We propose a novel estimation procedure which is based on recent developments in matrix estimation. In particular, for our estimator, we obtain upper bounds for the mean squared and the pointwise estimation errors. The obtained oracle inequalities are non-asymptotic and hold for finite sample size.

math.ST↗

arXiv2013

De-noising procedures for frame operators

The present paper provides a comprehensive study of de-noising properties of frames and, in particular, tight frames, which constitute one of the most popular tools in contemporary signal processing. The objective of the paper is to bridge the existing gap between mathematical and statistical theories on one hand and engineering practice on the other and explore how one can take advantage of a specific structure of a frame in contrast to an arbitrary collection of vectors or an orthonormal basis. For both the general and the tight frames, the paper presents a set of practically implementable de-noising techniques which take frame induced correlation structures into account. These results are supplemented by an examination of the case when the frame is constructed as a collection of orthonormal bases. In particular, recommendations are given for aggregation of the estimators at the stage of frame coefficients. The paper is concluded by a finite sample simulation study which confirms that taking frame structure and frame induced correlations into account indeed improves de-noising precision.

stat.ME↗