arXiv subjects

Michael B. Wakin

Publications and source records attributed to Michael B. Wakin.

At least 55 records · Page 3Linked to original sources

arXiv2018

Global Optimality in Low-rank Matrix Optimization

This paper considers the minimization of a general objective function $f(X)$ over the set of rectangular $n\times m$ matrices that have rank at most $r$. To reduce the computational burden, we factorize the variable $X$ into a product of two smaller matrices and optimize over these two matrices instead of $X$. Despite the resulting nonconvexity, recent studies in matrix completion and sensing have shown that the factored problem has no spurious local minima and obeys the so-called strict saddle property (the function has a directional negative curvature at all critical points but local minima). We analyze the global geometry for a general and yet well-conditioned objective function $f(X)$ whose restricted strong convexity and restricted strong smoothness constants are comparable. In particular, we show that the reformulated objective function has no spurious local minima and obeys the strict saddle property. These geometric properties imply that a number of iterative optimization algorithms (such as gradient descent) can provably solve the factored problem with global convergence.

cs.IT↗

arXiv2017

Atomic Norm Minimization for Modal Analysis from Random and Compressed Samples

Modal analysis is the process of estimating a system's modal parameters such as its natural frequencies and mode shapes. One application of modal analysis is in structural health monitoring (SHM), where a network of sensors may be used to collect vibration data from a physical structure such as a building or bridge. There is a growing interest in developing automated techniques for SHM based on data collected in a wireless sensor network. In order to conserve power and extend battery life, however, it is desirable to minimize the amount of data that must be collected and transmitted in such a sensor network. In this paper, we highlight the fact that modal analysis can be formulated as an atomic norm minimization (ANM) problem, which can be solved efficiently and in some cases recover perfectly a structure's mode shapes and frequencies. We survey a broad class of sampling and compression strategies that one might consider in a physical sensor network, and we provide bounds on the sample complexity of these compressive schemes in order to recover a structure's mode shapes and frequencies via ANM. A main contribution of our paper is to establish a bound on the sample complexity of modal analysis with random temporal compression, and in this scenario we prove that the samples per sensor can actually decrease as the number of sensors increases. We also extend an atomic norm denoising problem to the multiple measurement vector (MMV) setting in the case of uniform sampling.

cs.IT↗

arXiv2017

MC^2: A Two-Phase Algorithm for Leveraged Matrix Completion

Leverage scores, loosely speaking, reflect the importance of the rows and columns of a matrix. Ideally, given the leverage scores of a rank-$r$ matrix $M\in\mathbb{R}^{n\times n}$, that matrix can be reliably completed from just $O(rn\log^{2}n)$ samples if the samples are chosen randomly from a nonuniform distribution induced by the leverage scores. In practice, however, the leverage scores are often unknown a priori. As such, the sample complexity in uniform matrix completion---using uniform random sampling---increases to $O(η(M)\cdot rn\log^{2}n)$, where $η(M)$ is the largest leverage score of $M$. In this paper, we propose a two-phase algorithm called MC$^2$ for matrix completion: in the first phase, the leverage scores are estimated based on uniform random samples, and then in the second phase the matrix is resampled nonuniformly based on the estimated leverage scores and then completed. For well-conditioned matrices, the total sample complexity of MC$^2$ is no worse than uniform matrix completion, and for certain classes of well-conditioned matrices---namely, reasonably coherent matrices whose leverage scores exhibit mild decay---MC$^2$ requires substantially fewer samples. Numerical simulations suggest that the algorithm outperforms uniform matrix completion in a broad class of matrices, and in particular, is much less sensitive to the condition number than our theory currently requires.

cs.IT↗

arXiv2017

The Fast Slepian Transform

The discrete prolate spheroidal sequences (DPSS's) provide an efficient representation for discrete signals that are perfectly timelimited and nearly bandlimited. Due to the high computational complexity of projecting onto the DPSS basis - also known as the Slepian basis - this representation is often overlooked in favor of the fast Fourier transform (FFT). We show that there exist fast constructions for computing approximate projections onto the leading Slepian basis elements. The complexity of the resulting algorithms is comparable to the FFT, and scales favorably as the quality of the desired approximation is increased. In the process of bounding the complexity of these algorithms, we also establish new nonasymptotic results on the eigenvalue distribution of discrete time-frequency localization operators. We then demonstrate how these algorithms allow us to efficiently compute the solution to certain least-squares problems that arise in signal processing. We also provide simulations comparing these fast, approximate Slepian methods to exact Slepian methods as well as the traditional FFT based methods.

math.NA↗

arXiv2017

Stabilizing Embedology: Geometry-Preserving Delay-Coordinate Maps

Delay-coordinate mapping is an effective and widely used technique for reconstructing and analyzing the dynamics of a nonlinear system based on time-series outputs. The efficacy of delay-coordinate mapping has long been supported by Takens' embedding theorem, which guarantees that delay-coordinate maps use the time-series output to provide a reconstruction of the hidden state space that is a one-to-one embedding of the system's attractor. While this topological guarantee ensures that distinct points in the reconstruction correspond to distinct points in the original state space, it does not characterize the quality of this embedding or illuminate how the specific parameters affect the reconstruction. In this paper, we extend Takens' result by establishing conditions under which delay-coordinate mapping is guaranteed to provide a stable embedding of a system's attractor. Beyond only preserving the attractor topology, a stable embedding preserves the attractor geometry by ensuring that distances between points in the state space are approximately preserved. In particular, we find that delay-coordinate mapping stably embeds an attractor of a dynamical system if the stable rank of the system is large enough to be proportional to the dimension of the attractor. The stable rank reflects the relation between the sampling interval and the number of delays in delay-coordinate mapping. Our theoretical findings give guidance to choosing system parameters, echoing the trade-off between irrelevancy and redundancy that has been heuristically investigated in the literature. Our initial result is stated for attractors that are smooth submanifolds of Euclidean space, with extensions provided for the case of strange attractors.

nlin.CD↗

arXiv2017

The Eigenvalue Distribution of Discrete Periodic Time-Frequency Limiting Operators

Bandlimiting and timelimiting operators play a fundamental role in analyzing bandlimited signals that are approximately timelimited (or vice versa). In this paper, we consider a time-frequency (in the discrete Fourier transform (DFT) domain) limiting operator whose eigenvectors are known as the periodic discrete prolate spheroidal sequences (PDPSSs). We establish new nonasymptotic results on the eigenvalue distribution of this operator. As a byproduct, we also characterize the eigenvalue distribution of a set of submatrices of the DFT matrix, which is of independent interest.

cs.IT↗

arXiv2017

What to Expect When You Are Expecting on the Grassmannian

Consider an incoming sequence of vectors, all belonging to an unknown subspace $\operatorname{S}$, and each with many missing entries. In order to estimate $\operatorname{S}$, it is common to partition the data into blocks and iteratively update the estimate of $\operatorname{S}$ with each new incoming measurement block. In this paper, we investigate a rather basic question: Is it possible to identify $\operatorname{S}$ by averaging the column span of the partially observed incoming measurement blocks on the Grassmannian? We show that in general the span of the incoming blocks is in fact a biased estimator of $\operatorname{S}$ when data suffers from erasures, and we find an upper bound for this bias. We reach this conclusion by examining the defining optimization program for the Fréchet expectation on the Grassmannian, and with the aid of a sharp perturbation bound and standard large deviation results.

cs.IT↗

arXiv2017

On the Asymptotic Equivalence of Circulant and Toeplitz Matrices

Any sequence of uniformly bounded $N\times N$ Hermitian Toeplitz matrices $\{\boldsymbol{H}_N\}$ is asymptotically equivalent to a certain sequence of $N\times N$ circulant matrices $\{\boldsymbol{C}_N\}$ derived from the Toeplitz matrices in the sense that $\left\| \boldsymbol{H}_N - \boldsymbol{C}_N \right\|_F = o(\sqrt{N})$ as $N\rightarrow \infty$. This implies that certain collective behaviors of the eigenvalues of each Toeplitz matrix are reflected in those of the corresponding circulant matrix and supports the utilization of the computationally efficient fast Fourier transform (instead of the Karhunen-Loève transform) in applications like coding and filtering. In this paper, we study the asymptotic performance of the individual eigenvalue estimates. We show that the asymptotic equivalence of the circulant and Toeplitz matrices implies the individual asymptotic convergence of the eigenvalues for certain types of Toeplitz matrices. We also show that these estimates asymptotically approximate the largest and smallest eigenvalues for more general classes of Toeplitz matrices.

cs.IT↗

arXiv2016

What Happens to a Manifold Under a Bi-Lipschitz Map?

We study geometric and topological properties of the image of a smooth submanifold of $\mathbb{R}^{n}$ under a bi-Lipschitz map to $\mathbb{R}^{m}$. In particular, we characterize how the dimension, diameter, volume, and reach of the embedded manifold relate to the original. Our main result establishes a lower bound on the reach of the embedded manifold in the case where $m \le n$ and the bi-Lipschitz map is linear. We discuss implications of this work in signal processing and machine learning, where bi-Lipschitz maps on low-dimensional manifolds have been constructed using randomized linear operators.

cs.IT↗

arXiv2016

Super-Resolution of Complex Exponentials from Modulations with Unknown Waveforms

Super-resolution is generally referred to as the task of recovering fine details from coarse information. Motivated by applications such as single-molecule imaging, radar imaging, etc., we consider parameter estimation of complex exponentials from their modulations with unknown waveforms, allowing for non-stationary blind super-resolution. This problem, however, is ill-posed since both the parameters associated with the complex exponentials and the modulating waveforms are unknown. To alleviate this, we assume that the unknown waveforms live in a common low-dimensional subspace. Using a lifting trick, we recast the blind super-resolution problem as a structured low-rank matrix recovery problem. Atomic norm minimization is then used to enforce the structured low-rankness, and is reformulated as a semidefinite program that is solvable in polynomial time. We show that, up to scaling ambiguities, exact recovery of both of the complex exponential parameters and the unknown waveforms is possible when the waveform subspace is random and the number of measurements is proportional to the number of degrees of freedom in the problem. Numerical simulations support our theoretical findings, showing that non-stationary blind super-resolution using atomic norm minimization is possible.

cs.IT↗

arXiv2016

Approximating Sampled Sinusoids and Multiband Signals Using Multiband Modulated DPSS Dictionaries

Many signal processing problems--such as analysis, compression, denoising, and reconstruction--can be facilitated by expressing the signal as a linear combination of atoms from a well-chosen dictionary. In this paper, we study possible dictionaries for representing the discrete vector one obtains when collecting a finite set of uniform samples from a multiband analog signal. By analyzing the spectrum of combined time- and multiband-limiting operations in the discrete-time domain, we conclude that the information level of the sampled multiband vectors is essentially equal to the time-frequency area. For representing these vectors, we consider a dictionary formed by concatenating a collection of modulated Discrete Prolate Spheroidal Sequences (DPSS's). We study the angle between the subspaces spanned by this dictionary and an optimal dictionary, and we conclude that the multiband modulated DPSS dictionary--which is simple to construct and more flexible than the optimal dictionary in practical applications--is nearly optimal for representing multiband sample vectors. We also show that the multiband modulated DPSS dictionary not only provides a very high degree of approximation accuracy in an MSE sense for multiband sample vectors (using a number of atoms comparable to the information level), but also that it can provide high-quality approximations of all sampled sinusoids within the bands of interest.

cs.IT↗

arXiv2016

Reconstruction of Frequency Hopping Signals From Multi-Coset Samples

Multi-Coset (MC) sampling is a well established, practically feasible scheme for sampling multiband analog signals below the Nyquist rate. MC sampling has gained renewed interest in the Compressive Sensing (CS) community, due partly to the fact that in the frequency domain, MC sampling bears a strong resemblance to other sub-Nyquist CS acquisition protocols. In this paper, we consider MC sampling of analog frequency hopping signals, which can be viewed as multiband signals with changing band positions. This nonstationarity motivates our consideration of a segment-based reconstruction framework, in which the sample stream is broken into short segments for reconstruction. In contrast, previous works focusing on the reconstruction of multiband signals have used a segment-less reconstruction framework such as the modified MUSIC algorithm. We outline the challenges associated with segment-based recovery of frequency hopping signals from MC samples, and we explain how these challenges can be addressed using conventional CS recovery techniques. We also demonstrate the utility of the Discrete Prolate Spheroidal Sequences (DPSS's) as an efficient dictionary for reducing the computational complexity of segment-based reconstruction.

cs.IT↗

arXiv2015

Greed is Super: A Fast Algorithm for Super-Resolution

We present a fast two-phase algorithm for super-resolution with strong theoretical guarantees. Given the low-frequency part of the spectrum of a sequence of impulses, Phase I consists of a greedy algorithm that roughly estimates the impulse positions. These estimates are then refined by local optimization in Phase II. In contrast to the convex relaxation proposed by Candès et al., our approach has a low computational complexity but requires the impulses to be separated by an additional logarithmic factor to succeed. The backbone of our work is the fundamental work of Slepian et al. involving discrete prolate spheroidal wave functions and their unique properties.

cs.IT↗

arXiv2015

Computing Active Subspaces Efficiently with Gradient Sketching

Active subspaces are an emerging set of tools for identifying and exploiting the most important directions in the space of a computer simulation's input parameters; these directions depend on the simulation's quantity of interest, which we treat as a function from inputs to outputs. To identify a function's active subspace, one must compute the eigenpairs of a matrix derived from the function's gradient, which presents challenges when the gradient is not available as a subroutine. We numerically study two methods for estimating the necessary eigenpairs using only linear measurements of the function's gradient. In practice, these measurements can be estimated by finite differences using only two function evaluations, regardless of the dimension of the function's input space.

math.NA↗

arXiv2014

Technical Report: Compressive Temporal Higher Order Cyclostationary Statistics

The application of nonlinear transformations to a cyclostationary signal for the purpose of revealing hidden periodicities has proven to be useful for applications requiring signal selectivity and noise tolerance. The fact that the hidden periodicities, referred to as cyclic moments, are often compressible in the Fourier domain motivates the use of compressive sensing (CS) as an efficient acquisition protocol for capturing such signals. In this work, we consider the class of Temporal Higher Order Cyclostationary Statistics (THOCS) estimators when CS is used to acquire the cyclostationary signal assuming compressible cyclic moments in the Fourier domain. We develop a theoretical framework for estimating THOCS using the low-rate nonuniform sampling protocol from CS and illustrate the performance of this framework using simulated data.

cs.IT↗

arXiv2014

A First Analysis of the Stability of Takens' Embedding

Takens' Embedding Theorem asserts that when the states of a hidden dynamical system are confined to a low-dimensional attractor, complete information about the states can be preserved in the observed time-series output through the delay coordinate map. However, the conditions for the theorem to hold ignore the effects of noise and time-series analysis in practice requires a careful empirical determination of the sampling time and number of delays resulting in a number of delay coordinates larger than the minimum prescribed by Takens' theorem. In this paper, we use tools and ideas in Compressed Sensing to provide a first theoretical justification for the choice of the number of delays in noisy conditions. In particular, we show that under certain conditions on the dynamical system, measurement function, number of delays and sampling time, the delay-coordinate map can be a stable embedding of the dynamical system's attractor.

math.DS↗

arXiv2014

New Analysis of Manifold Embeddings and Signal Recovery from Compressive Measurements

Compressive Sensing (CS) exploits the surprising fact that the information contained in a sparse signal can be preserved in a small number of compressive, often random linear measurements of that signal. Strong theoretical guarantees have been established concerning the embedding of a sparse signal family under a random measurement operator and on the accuracy to which sparse signals can be recovered from noisy compressive measurements. In this paper, we address similar questions in the context of a different modeling framework. Instead of sparse models, we focus on the broad class of manifold models, which can arise in both parametric and non-parametric signal families. Using tools from the theory of empirical processes, we improve upon previous results concerning the embedding of low-dimensional manifolds under random measurement operators. We also establish both deterministic and probabilistic instance-optimal bounds in $\ell_2$ for manifold-based signal recovery and parameter estimation from noisy compressive measurements. In line with analogous results for sparsity-based CS, we conclude that much stronger bounds are possible in the probabilistic setting. Our work supports the growing evidence that manifold-based models can be used with high accuracy in compressive signal processing.

cs.IT↗

arXiv2014

The Restricted Isometry Property for Random Block Diagonal Matrices

In Compressive Sensing, the Restricted Isometry Property (RIP) ensures that robust recovery of sparse vectors is possible from noisy, undersampled measurements via computationally tractable algorithms. It is by now well-known that Gaussian (or, more generally, sub-Gaussian) random matrices satisfy the RIP under certain conditions on the number of measurements. Their use can be limited in practice, however, due to storage limitations, computational considerations, or the mismatch of such matrices with certain measurement architectures. These issues have recently motivated considerable effort towards studying the RIP for structured random matrices. In this paper, we study the RIP for block diagonal measurement matrices where each block on the main diagonal is itself a sub-Gaussian random matrix. Our main result states that such matrices can indeed satisfy the RIP but that the requisite number of measurements depends on certain properties of the basis in which the signals are sparse. In the best case, these matrices perform nearly as well as dense Gaussian random matrices, despite having many fewer nonzero entries.

cs.IT↗