arXiv subjects

Deanna Needell

Publications and source records attributed to Deanna Needell.

At least 145 records · Page 8Linked to original sources

arXiv2018

Matrix Completion for Structured Observations

The need to predict or fill-in missing data, often referred to as matrix completion, is a common challenge in today's data-driven world. Previous strategies typically assume that no structural difference between observed and missing entries exists. Unfortunately, this assumption is woefully unrealistic in many applications. For example, in the classic Netflix challenge, in which one hopes to predict user-movie ratings for unseen films, the fact that the viewer has not watched a given movie may indicate a lack of interest in that movie, thus suggesting a lower rating than otherwise expected. We propose adjusting the standard nuclear norm minimization strategy for matrix completion to account for such structural differences between observed and unobserved entries by regularizing the values of the unobserved entries. We show that the proposed method outperforms nuclear norm minimization in certain settings.

math.NA↗

arXiv2018

An algebraic perspective on integer sparse recovery

Compressed sensing is a relatively new mathematical paradigm that shows a small number of linear measurements are enough to efficiently reconstruct a large dimensional signal under the assumption the signal is sparse. Applications for this technology are ubiquitous, ranging from wireless communications to medical imaging, and there is now a solid foundation of mathematical theory and algorithms to robustly and efficiently reconstruct such signals. However, in many of these applications, the signals of interest do not only have a sparse representation, but have other structure such as lattice-valued coefficients. While there has been a small amount of work in this setting, it is still not very well understood how such extra information can be utilized during sampling and reconstruction. Here, we explore the problem of integer sparse reconstruction, lending insight into when this knowledge can be useful, and what types of sampling designs lead to robust reconstruction guarantees. We use a combination of combinatorial, probabilistic and number-theoretic methods to discuss existence and some constructions of such sensing matrices with concrete examples. We also prove sparse versions of Minkowski's Convex Body and Linear Forms theorems that exhibit some limitations of this framework.

math.NA↗

arXiv2017

Biquasiles and Dual Graph Diagrams

We introduce \textit{dual graph diagrams} representing oriented knots and links. We use these combinatorial structures to define corresponding algebraic structures we call \textit{biquasiles} whose axioms are motivated by dual graph Reidemeister moves, generalizing the Dehn presentation of the knot group analogously to the way quandles and biquandles generalize the Wirtinger presentation. We use these structures to define invariants of oriented knots and links and provide examples.

math.GT↗

arXiv2017

Tolerant Compressed Sensing With Partially Coherent Sensing Matrices

Most of compressed sensing (CS) theory to date is focused on incoherent sensing, that is, columns from the sensing matrix are highly uncorrelated. However, sensing systems with naturally occurring correlations arise in many applications, such as signal detection, motion detection and radar. Moreover, in these applications it is often not necessary to know the support of the signal exactly, but instead small errors in the support and signal are tolerable. Despite the abundance of work utilizing incoherent sensing matrices, for this type of tolerant recovery we suggest that coherence is actually beneficial. We promote the use of coherent sampling when tolerant support recovery is acceptable, and demonstrate its advantages empirically. In addition, we provide a first step towards theoretical analysis by considering a specific reconstruction method for selected signal classes.

cs.IT↗

arXiv2017

Simple Classification using Binary Data

Binary, or one-bit, representations of data arise naturally in many applications, and are appealing in both hardware implementations and algorithm design. In this work, we study the problem of data classification from binary data and propose a framework with low computation and resource costs. We illustrate the utility of the proposed approach through stylized and realistic numerical experiments, and provide a theoretical analysis for a simple case. We hope that our framework and analysis will serve as a foundation for studying similar types of approaches.

cs.LG↗

arXiv2017

Rows vs. Columns: Randomized Kaczmarz or Gauss-Seidel for Ridge Regression

The Kaczmarz and Gauss-Seidel methods aim to solve a linear $m \times n$ system $\boldsymbol{X} \boldsymbolβ = \boldsymbol{y}$ by iteratively refining the solution estimate; the former uses random rows of $\boldsymbol{X}$ {to update $\boldsymbolβ$ given the corresponding equations} and the latter uses random columns of $\boldsymbol{X}$ {to update corresponding coordinates in $\boldsymbolβ$}. Interest in these methods was recently revitalized by a proof of Strohmer and Vershynin showing linear convergence in expectation for a \textit{randomized} Kaczmarz method variant (RK), and a similar result for the randomized Gauss-Seidel algorithm (RGS) was later proved by Lewis and Leventhal. Recent work unified the analysis of these algorithms for the overcomplete and undercomplete systems, showing convergence to the ordinary least squares (OLS) solution and the minimum Euclidean norm solution respectively. This paper considers the natural follow-up to the OLS problem, ridge regression, which solves $(\boldsymbol{X}^* \boldsymbol{X} + λ\boldsymbol{I}) \boldsymbolβ = \boldsymbol{X}^* \boldsymbol{y}$. We present particular variants of RK and RGS for solving this system and derive their convergence rates. We compare these to a recent proposal by Ivanov and Zhdanov to solve this system, that can be interpreted as randomly sampling both rows and columns, which we argue is often suboptimal. Instead, we claim that one should always use RGS (columns) when $m > n$ and RK (rows) when $m < n$. This difference in behavior is simply related to the minimum eigenvalue of two related positive semidefinite matrices, $\boldsymbol{X}^* \boldsymbol{X} + λ\boldsymbol{I}_n$ and $\boldsymbol{X} \boldsymbol{X}^* + λ\boldsymbol{I}_m$ when $m > n$ or $m < n$.

math.NA↗

arXiv2017

Batched Stochastic Gradient Descent with Weighted Sampling

We analyze a batched variant of Stochastic Gradient Descent (SGD) with weighted sampling distribution for smooth and non-smooth objective functions. We show that by distributing the batches computationally, a significant speedup in the convergence rate is provably possible compared to either batched sampling or weighted sampling alone. We propose several computationally efficient schemes to approximate the optimal weights, and compute proposed sampling distributions explicitly for the least squares and hinge loss problems. We show both analytically and experimentally that substantial gains can be obtained.

math.NA↗

arXiv2017

An Asynchronous Parallel Approach to Sparse Recovery

Asynchronous parallel computing and sparse recovery are two areas that have received recent interest. Asynchronous algorithms are often studied to solve optimization problems where the cost function takes the form $\sum_{i=1}^M f_i(x)$, with a common assumption that each $f_i$ is sparse; that is, each $f_i$ acts only on a small number of components of $x\in\mathbb{R}^n$. Sparse recovery problems, such as compressed sensing, can be formulated as optimization problems, however, the cost functions $f_i$ are dense with respect to the components of $x$, and instead the signal $x$ is assumed to be sparse, meaning that it has only $s$ non-zeros where $s\ll n$. Here we address how one may use an asynchronous parallel architecture when the cost functions $f_i$ are not sparse in $x$, but rather the signal $x$ is sparse. We propose an asynchronous parallel approach to sparse recovery via a stochastic greedy algorithm, where multiple processors asynchronously update a vector in shared memory containing information on the estimated signal support. We include numerical simulations that illustrate the potential benefits of our proposed asynchronous method.

cs.LG↗

arXiv2016

Weighted $\ell_1$-Minimization for Sparse Recovery under Arbitrary Prior Information

Weighted $\ell_1$-minimization has been studied as a technique for the reconstruction of a sparse signal from compressively sampled measurements when prior information about the signal, in the form of a support estimate, is available. In this work, we study the recovery conditions and the associated recovery guarantees of weighted $\ell_1$-minimization when arbitrarily many distinct weights are permitted. For example, such a setup might be used when one has multiple estimates for the support of a signal, and these estimates have varying degrees of accuracy. Our analysis yields an extension to existing works that assume only a single support estimate set upon which a constant weight is applied. We include numerical experiments, with both synthetic signals and real video data, that demonstrate the benefits of allowing non-uniform weights in the reconstruction procedure.

cs.IT↗

arXiv2016

A Practical Study of Longitudinal Reference Based Compressed Sensing for MRI

Compressed sensing (CS) is a new signal acquisition paradigm that enables the reconstruction of signals and images from a low number of samples. A particularly exciting application of CS is Magnetic Resonance Imaging (MRI), where CS significantly speeds up scan time by requiring far fewer measurements than standard MRI techniques. Such a reduction in sampling time leads to less power consumption, less need for patient sedation, and more accurate images. This accuracy increase is especially pronounced in pediatric MRI where patients have trouble being still for long scan periods. Although such gains are already significant, even further improvements can be made by utilizing past MRI scans of the same patient. Many patients require repeated scans over a period of time in order to track illnesses and the prior scans can be used as references for the current image. This allows samples to be taken adaptively, based on both the prior scan and the current measurements. Work by Weizman has shown that so-called reference based adaptive-weighted temporal Compressed Sensing MRI (LACS-MRI) requires far fewer samples than standard Compressed Sensing (CS) to achieve the same reconstruction signal-to-noise ratio (RSNR). The method uses a mixture of reference-based and adaptive-sampling. In this work, we test this methodology by using various adaptive sensing schemes, reconstruction methods, and image types. We create a thorough catalog of reconstruction behavior and success rates that is interesting from a mathematical point of view and is useful for practitioners. We also solve a grayscale compensation toy problem that supports the insensitivity of LACS-MRI to changes in MRI acquisition parameters and thus showcases the reliability of LACS-MRI in possible clinical situations.

cs.IT↗

arXiv2016

Lattices from tight equiangular frames

We consider the set of all linear combinations with integer coefficients of the vectors of a unit tight equiangular $(k,n)$ frame and are interested in the question whether this set is a lattice, that is, a discrete additive subgroup of the $k$-dimensional Euclidean space. We show that this is not the case if the cosine of the angle of the frame is irrational. We also prove that the set is a lattice for $n = k+1$ and that there are infinitely many $k$ such that a lattice emerges for $n = 2k$. We dispose of all cases in dimensions $k$ at most $9$. In particular, we show that a $(7,28)$ frame generates a strongly eutactic lattice and give an alternative proof of Roland Bacher's recent observation that this lattice is perfect.

math.FA↗

arXiv2016

Constrained adaptive sensing

Suppose that we wish to estimate a vector $\mathbf{x} \in \mathbb{C}^n$ from a small number of noisy linear measurements of the form $\mathbf{y} = \mathbf{A x} + \mathbf{z}$, where $\mathbf{z}$ represents measurement noise. When the vector $\mathbf{x}$ is sparse, meaning that it has only $s$ nonzeros with $s \ll n$, one can obtain a significantly more accurate estimate of $\mathbf{x}$ by adaptively selecting the rows of $\mathbf{A}$ based on the previous measurements provided that the signal-to-noise ratio (SNR) is sufficiently large. In this paper we consider the case where we wish to realize the potential of adaptivity but where the rows of $\mathbf{A}$ are subject to physical constraints. In particular, we examine the case where the rows of $\mathbf{A}$ are constrained to belong to a finite set of allowable measurement vectors. We demonstrate both the limitations and advantages of adaptive sensing in this constrained setting. We prove that for certain measurement ensembles, the benefits offered by adaptive designs fall far short of the improvements that are possible in the unconstrained adaptive setting. On the other hand, we also provide both theoretical and empirical evidence that in some scenarios adaptivity does still result in substantial improvements even in the constrained setting. To illustrate these potential gains, we propose practical algorithms for constrained adaptive sensing by exploiting connections to the theory of optimal experimental design and show that these algorithms exhibit promising performance in some representative applications.

cs.IT↗

arXiv2016

One-Bit Compressive Sensing of Dictionary-Sparse Signals

One-bit compressive sensing has extended the scope of sparse recovery by showing that sparse signals can be accurately reconstructed even when their linear measurements are subject to the extreme quantization scenario of binary samples---only the sign of each linear measurement is maintained. Existing results in one-bit compressive sensing rely on the assumption that the signals of interest are sparse in some fixed orthonormal basis. However, in most practical applications, signals are sparse with respect to an overcomplete dictionary, rather than a basis. There has already been a surge of activity to obtain recovery guarantees under such a generalized sparsity model in the classical compressive sensing setting. Here, we extend the one-bit framework to this important model, providing a unified theory of one-bit compressive sensing under dictionary sparsity. Specifically, we analyze several different algorithms---based on convex programming and on hard thresholding---and show that, under natural assumptions on the sensing matrix (satisfied by Gaussian matrices), these algorithms can efficiently recover analysis-dictionary-sparse signals in the one-bit model.

cs.IT↗

arXiv2016

Optimizing quantization for Lasso recovery

This letter is focused on quantized Compressed Sensing, assuming that Lasso is used for signal estimation. Leveraging recent work, we provide a framework to optimize the quantization function and show that the recovered signal converges to the actual signal at a quadratic rate as a function of the quantization level. We show that when the number of observations is high, this method of quantization gives a significantly better recovery rate than standard Lloyd-Max quantization. We support our theoretical analysis with numerical simulations.

cs.IT↗

arXiv2015

Methods for Quantized Compressed Sensing

In this paper, we compare and catalog the performance of various greedy quantized compressed sensing algorithms that reconstruct sparse signals from quantized compressed measurements. We also introduce two new greedy approaches for reconstruction: Quantized Compressed Sampling Matching Pursuit (QCoSaMP) and Adaptive Outlier Pursuit for Quantized Iterative Hard Thresholding (AOP-QIHT). We compare the performance of greedy quantized compressed sensing algorithms for a given bit-depth, sparsity, and noise level.

cs.IT↗

arXiv2015

A note on practical approximate projection schemes in signal space methods

Compressive sensing (CS) is a new technology which allows the acquisition of signals directly in compressed form, using far fewer measurements than traditional theory dictates. Recently, many so-called signal space methods have been developed to extend this body of work to signals sparse in arbitrary dictionaries rather than orthonormal bases. In doing so, CS can be utilized in a much broader array of practical settings. Often, such approaches often rely on the ability to optimally project a signal onto a small number of dictionary atoms. Such optimal, or even approximate, projections have been difficult to derive theoretically. Nonetheless, it has been observed experimentally that conventional CS approaches can be used for such projections, and still provide accurate signal recovery. In this letter, we summarize the empirical evidence and clearly demonstrate for what signal types certain CS methods may be used as approximate projections. In addition, we provide theoretical guarantees for such methods for certain sparse signal structures. Our theoretical results match those observed in experimental studies, and we thus establish both experimentally and theoretically that these CS methods can be used in this context. \end{abstract}

math.NA↗

arXiv2015

Randomized Block Kaczmarz Method with Projection for Solving Least Squares

The Kaczmarz method is an iterative method for solving overcomplete linear systems of equations Ax=b. The randomized version of the Kaczmarz method put forth by Strohmer and Vershynin iteratively projects onto a randomly chosen solution space given by a single row of the matrix A and converges exponentially in expectation to the solution of a consistent system. In this paper we analyze two block versions of the method each with a randomized projection, that converge in expectation to the least squares solution of inconsistent systems. Our approach utilizes a paving of the matrix A to guarantee exponential convergence, and suggests that paving yields a significant improvement in performance in certain regimes. The proposed method is an extension of the block Kaczmarz method analyzed by Needell and Tropp and the Randomized Extended Kaczmarz method of Zouzias and Freris. The contribution is thus two-fold; unlike the standard Kaczmarz method, our methods converge to the least-squares solution of inconsistent systems, and by using appropriate blocks of the matrix this convergence can be significantly accelerated. Numerical experiments suggest that the proposed algorithm can indeed lead to advantages in practice.

math.NA↗

arXiv2015

Compressive Sensing with Redundant Dictionaries and Structured Measurements

Consider the problem of recovering an unknown signal from undersampled measurements, given the knowledge that the signal has a sparse representation in a specified dictionary $D$. This problem is now understood to be well-posed and efficiently solvable under suitable assumptions on the measurements and dictionary, if the number of measurements scales roughly with the sparsity level. One sufficient condition for such is the $D$-restricted isometry property ($D$-RIP), which asks that the sampling matrix approximately preserve the norm of all signals which are sufficiently sparse in $D$. While many classes of random matrices are known to satisfy such conditions, such matrices are not representative of the structural constraints imposed by practical sensing systems. We close this gap in the theory by demonstrating that one can subsample a fixed orthogonal matrix in such a way that the $D$-RIP will hold, provided this basis is sufficiently incoherent with the sparsifying dictionary $D$. We also extend this analysis to allow for weighted sparse expansions. Consequently, we arrive at compressive sensing recovery guarantees for structured measurements and redundant dictionaries, opening the door to a wide array of practical applications.

cs.IT↗