arXiv subjects

Namrata Vaswani

Publications and source records attributed to Namrata Vaswani.

At least 55 records · Page 3Linked to original sources

arXiv2016

Online (and Offline) Robust PCA: Novel Algorithms and Performance Guarantees

In this work, we study the online robust principal components' analysis (RPCA) problem. In recent work, RPCA has been defined as a problem of separating a low-rank matrix (true data), $L$, and a sparse matrix (outliers), $S$, from their sum, $M:=L + S$. A more general version of this problem is to recover $L$ and $S$ from $M:=L + S + W$ where $W$ is the matrix of unstructured small noise/corruptions. An important application where this problem occurs is in video analytics in trying to separate sparse foregrounds (e.g., moving objects) from slowly changing backgrounds. While there has been a large amount of recent work on solutions and guarantees for the batch RPCA problem, the online problem is largely open."Online" RPCA is the problem of doing the above on-the-fly with the extra assumptions that the initial subspace is accurately known and that the subspace from which $l_t$ is generated changes slowly over time. We develop and study a novel "online" RPCA algorithm based on the recently introduced Recursive Projected Compressive Sensing (ReProCS) framework. Our algorithm improves upon the original ReProCS algorithm and it also returns even more accurate offline estimates. The key contribution of this work is a correctness result (complete performance guarantee) for this algorithm under reasonably mild assumptions. By using extra assumptions -- accurate initial subspace knowledge, slow subspace change, and clustered eigenvalues -- we are able to remove one important limitation of batch RPCA results and two key limitations of a recent result for ReProCS for online RPCA. To our knowledge, this work is among the first few correctness results for online RPCA. Most earlier results were only partial results, i.e., they required an assumption on intermediate algorithm estimates.

cs.IT↗

arXiv2015

Online Matrix Completion and Online Robust PCA

This work studies two interrelated problems - online robust PCA (RPCA) and online low-rank matrix completion (MC). In recent work by Candès et al., RPCA has been defined as a problem of separating a low-rank matrix (true data), $L:=[\ell_1, \ell_2, \dots \ell_{t}, \dots , \ell_{t_{\max}}]$ and a sparse matrix (outliers), $S:=[x_1, x_2, \dots x_{t}, \dots, x_{t_{\max}}]$ from their sum, $M:=L+S$. Our work uses this definition of RPCA. An important application where both these problems occur is in video analytics in trying to separate sparse foregrounds (e.g., moving objects) and slowly changing backgrounds. While there has been a large amount of recent work on both developing and analyzing batch RPCA and batch MC algorithms, the online problem is largely open. In this work, we develop a practical modification of our recently proposed algorithm to solve both the online RPCA and online MC problems. The main contribution of this work is that we obtain correctness results for the proposed algorithms under mild assumptions. The assumptions that we need are: (a) a good estimate of the initial subspace is available (easy to obtain using a short sequence of background-only frames in video surveillance); (b) the $\ell_t$'s obey a `slow subspace change' assumption; (c) the basis vectors for the subspace from which $\ell_t$ is generated are dense (non-sparse); (d) the support of $x_t$ changes by at least a certain amount at least every so often; and (e) algorithm parameters are appropriately set

cs.IT↗

arXiv2015

A Correctness Result for Online Robust PCA

This work studies the problem of sequentially recovering a sparse vector $x_t$ and a vector from a low-dimensional subspace $l_t$ from knowledge of their sum $m_t = x_t + l_t$. If the primary goal is to recover the low-dimensional subspace where the $l_t$'s lie, then the problem is one of online or recursive robust principal components analysis (PCA). To the best of our knowledge, this is the first correctness result for online robust PCA. We prove that if the $l_t$'s obey certain denseness and slow subspace change assumptions, and the support of $x_t$ changes by at least a certain amount at least every so often, and some other mild assumptions hold, then with high probability, the support of $x_t$ will be recovered exactly, and the error made in estimating $x_t$ and $l_t$ will be small. An example of where such a problem might arise is in separating a sparse foreground and slowly changing dense background in a surveillance video.

cs.IT↗

arXiv2015

Time Invariant Error Bounds for Modified-CS based Sparse Signal Sequence Recovery

In this work, we obtain performance guarantees for modified-CS and for its improved version, modified-CS-Add-LS-Del, for recursive reconstruction of a time sequence of sparse signals from a reduced set of noisy measurements available at each time. Under mild assumptions, we show that the support recovery error of both algorithms is bounded by a time-invariant and small value at all times. The same is also true for the reconstruction error. Under a slow support change assumption, (i) the support recovery error bound is small compared to the support size; and (ii) our results hold under weaker assumptions on the number of measurements than what $\ell_1$ minimization for noisy data needs. We first give a general result that only assumes a bound on support size, number of support changes and number of small magnitude nonzero entries at each time. Later, we specialize the main idea of these results for two sets of signal change assumptions that model the class of problems in which a new element that is added to the support either gets added at a large initial magnitude or its magnitude slowly increases to a large enough value within a finite delay. Simulation experiments are shown to back up our claims.

cs.IT↗

arXiv2014

Robust PCA with Partial Subspace Knowledge

In recent work, robust Principal Components Analysis (PCA) has been posed as a problem of recovering a low-rank matrix $\mathbf{L}$ and a sparse matrix $\mathbf{S}$ from their sum, $\mathbf{M}:= \mathbf{L} + \mathbf{S}$ and a provably exact convex optimization solution called PCP has been proposed. This work studies the following problem. Suppose that we have partial knowledge about the column space of the low rank matrix $\mathbf{L}$. Can we use this information to improve the PCP solution, i.e. allow recovery under weaker assumptions? We propose here a simple but useful modification of the PCP idea, called modified-PCP, that allows us to use this knowledge. We derive its correctness result which shows that, when the available subspace knowledge is accurate, modified-PCP indeed requires significantly weaker incoherence assumptions than PCP. Extensive simulations are also used to illustrate this. Comparisons with PCP and other existing work are shown for a stylized real application as well. Finally, we explain how this problem naturally occurs in many applications involving time series data, i.e. in what is called the online or recursive robust PCA problem. A corollary for this case is also given.

cs.IT↗

arXiv2014

Recursive Robust PCA or Recursive Sparse Recovery in Large but Structured Noise

This work studies the recursive robust principal components' analysis(PCA) problem. Here, "robust" refers to robustness to both independent and correlated sparse outliers. If the outlier is the signal-of-interest, this problem can be interpreted as one of recursively recovering a time sequence of sparse vectors, St, in the presence of large but structured noise, Lt. The structure that we assume on Lt is that Lt is dense and lies in a low dimensional subspace that is either fixed or changes "slowly enough". A key application where this problem occurs is in video surveillance where the goal is to separate a slowly changing background (Lt) from moving foreground objects (St) on-the-fly. To solve the above problem, we introduce a novel solution called Recursive Projected CS (ReProCS). Under mild assumptions, we show that, with high probability (w.h.p.), ReProCS can exactly recover the support set of St at all times; and the reconstruction errors of both St and Lt are upper bounded by a time-invariant and small value at all times.

cs.IT↗

arXiv2014

An Online Algorithm for Separating Sparse and Low-dimensional Signal Sequences from their Sum

This paper designs and evaluates a practical algorithm, called practical recursive projected compressive sensing (Prac-ReProCS), for recovering a time sequence of sparse vectors $S_t$ and a time sequence of dense vectors $L_t$ from their sum, $M_t:= S_t + L_t$, when any subsequence of the $L_t$'s lies in a slowly changing low-dimensional subspace. A key application where this problem occurs is in video layering where the goal is to separate a video sequence into a slowly changing background sequence and a sparse foreground sequence that consists of one or more moving regions/objects. Prac-ReProCS is a practical modification of its theoretical counterpart which was analyzed in our recent work. Experimental comparisons demonstrating the advantage of the approach for both simulated and real videos are shown. Extension to the undersampled case is also developed.

cs.IT↗

arXiv2014

Performance Guarantees for ReProCS -- Correlated Low-Rank Matrix Entries Case

Online or recursive robust PCA can be posed as a problem of recovering a sparse vector, $S_t$, and a dense vector, $L_t$, which lies in a slowly changing low-dimensional subspace, from $M_t:= S_t + L_t$ on-the-fly as new data comes in. For initialization, it is assumed that an accurate knowledge of the subspace in which $L_0$ lies is available. In recent works, Qiu et al proposed and analyzed a novel solution to this problem called recursive projected compressed sensing or ReProCS. In this work, we relax one limiting assumption of Qiu et al's result. Their work required that the $L_t$'s be mutually independent over time. However this is not a practical assumption, e.g., in the video application, $L_t$ is the background image sequence and one would expect it to be correlated over time. In this work we relax this and allow the $L_t$'s to follow an autoregressive model. We are able to show that under mild assumptions and under a denseness assumption on the unestimated part of the changed subspace, with high probability (w.h.p.), ReProCS can exactly recover the support set of $S_t$ at all times; the reconstruction errors of both $S_t$ and $L_t$ are upper bounded by a time invariant and small value; and the subspace recovery error decays to a small value within a finite delay of a subspace change time. Because the last assumption depends on an algorithm estimate, this result cannot be interpreted as a correctness result but only a useful step towards it.

cs.IT↗

arXiv2014

Recursive Robust PCA or Recursive Sparse Recovery in Large but Structured Noise (parts 1 and 2 combined)

This work studies the recursive robust principal components analysis (PCA) problem. If the outlier is the signal-of-interest, this problem can be interpreted as one of recursively recovering a time sequence of sparse vectors, $S_t$, in the presence of large but structured noise, $L_t$. The structure that we assume on $L_t$ is that $L_t$ is dense and lies in a low dimensional subspace that is either fixed or changes "slowly enough". A key application where this problem occurs is in video surveillance where the goal is to separate a slowly changing background ($L_t$) from moving foreground objects ($S_t$) on-the-fly. To solve the above problem, in recent work, we introduced a novel solution called Recursive Projected CS (ReProCS). In this work we develop a simple modification of the original ReProCS idea and analyze it. This modification assumes knowledge of a subspace change model on the $L_t$'s. Under mild assumptions and a denseness assumption on the unestimated part of the subspace of $L_t$ at various times, we show that, with high probability (w.h.p.), the proposed approach can exactly recover the support set of $S_t$ at all times; and the reconstruction errors of both $S_t$ and $L_t$ are upper bounded by a time-invariant and small value. In simulation experiments, we observe that the last assumption holds as long as there is some support change of $S_t$ every few frames.

cs.IT↗

arXiv2013

Recursive Sparse Recovery in Large but Structured Noise - Part 2

We study the problem of recursively recovering a time sequence of sparse vectors, St, from measurements Mt := St + Lt that are corrupted by structured noise Lt which is dense and can have large magnitude. The structure that we require is that Lt should lie in a low dimensional subspace that is either fixed or changes "slowly enough"; and the eigenvalues of its covariance matrix are "clustered". We do not assume any model on the sequence of sparse vectors. Their support sets and their nonzero element values may be either independent or correlated over time (usually in many applications they are correlated). The only thing required is that there be some support change every so often. We introduce a novel solution approach called Recursive Projected Compressive Sensing with cluster-PCA (ReProCS-cPCA) that addresses some of the limitations of earlier work. Under mild assumptions, we show that, with high probability, ReProCS-cPCA can exactly recover the support set of St at all times; and the reconstruction errors of both St and Lt are upper bounded by a time-invariant and small value.

cs.IT↗

arXiv2012

Regularized Modified BPDN for Noisy Sparse Reconstruction with Partial Erroneous Support and Signal Value Knowledge

We study the problem of sparse reconstruction from noisy undersampled measurements when the following two things are available. (1) We are given partial, and partly erroneous, knowledge of the signal's support, denoted by $T$. (2) We are also given an erroneous estimate of the signal values on $T$, denoted by $(\hatμ)_T$. In practice, both these may be available from available prior knowledge. Alternatively, in recursive reconstruction applications, like real-time dynamic MRI, one can use the support estimate and the signal value estimate from the previous time instant as $T$ and $(\hatμ)_T$. In this work, we introduce regularized modified-BPDN (reg-mod-BPDN) and obtain computable bounds on its reconstruction error. Reg-mod-BPDN tries to find the signal that is sparsest outside the set $T$, while being "close enough" to $(\hatμ)_T$ on $T$ and while satisfying the data constraint. Corresponding results for modified-BPDN and BPDN follow as direct corollaries. A second key contribution is an approach to obtain computable error bounds that hold without any sufficient conditions. This makes it easy to compare the bounds for the various approaches. Empirical reconstruction error comparisons with many existing approaches are also provided.

cs.IT↗

arXiv2012

Exact Reconstruction Conditions for Regularized Modified Basis Pursuit

In this correspondence, we obtain exact recovery conditions for regularized modified basis pursuit (reg-mod-BP) and discuss when the obtained conditions are weaker than those for modified-CS or for basis pursuit (BP). The discussion is also supported by simulation comparisons. Reg-mod-BP provides a solution to the sparse recovery problem when both an erroneous estimate of the signal's support, denoted by $T$, and an erroneous estimate of the signal values on $T$ are available.

cs.IT↗

arXiv2011

ReProCS: A Missing Link between Recursive Robust PCA and Recursive Sparse Recovery in Large but Correlated Noise

This work studies the recursive robust principal components' analysis (PCA) problem. Here, "robust" refers to robustness to both independent and correlated sparse outliers, although we focus on the latter. A key application where this problem occurs is in video surveillance where the goal is to separate a slowly changing background from moving foreground objects on-the-fly. The background sequence is well modeled as lying in a low dimensional subspace, that can gradually change over time, while the moving foreground objects constitute the correlated sparse outliers. In this and many other applications, the foreground is an outlier for PCA but is actually the "signal of interest" for the application; where as the background is the corruption or noise. Thus our problem can also be interpreted as one of recursively recovering a time sequence of sparse signals in the presence of large but spatially correlated noise. This work has two key contributions. First, we provide a new way of looking at this problem and show how a key part of our solution strategy involves solving a noisy compressive sensing (CS) problem. Second, we show how we can utilize the correlation of the outliers to our advantage in order to even deal with very large support sized outliers. The main idea is as follows. The correlation model applied to the previous support estimate helps predict the current support. This prediction serves as "partial support knowledge" for solving the modified-CS problem instead of CS. The support estimate of the modified-CS reconstruction is, in turn, used to update the correlation model parameters using a Kalman filter (or any adaptive filter). We call the resulting approach "support-predicted modified-CS".

cs.IT↗

arXiv2011

Stability of Modified-CS and LS-CS for Recursive Reconstruction of Sparse Signal Sequences

In this work, we obtain sufficient conditions for the "stability" of our recently proposed algorithms, Least Squares Compressive Sensing residual (LS-CS) and modified-CS, for recursively reconstructing sparse signal sequences from noisy measurements. By "stability" we mean that the number of misses from the current support estimate and the number of extras in it remain bounded by a time-invariant value at all times. We show that, for a signal model with fixed signal power and support set size; support set changes allowed at every time; and gradual coefficient magnitude increase/decrease, "stability" holds under mild assumptions -- bounded noise, high enough minimum nonzero coefficient magnitude increase rate, and large enough number of measurements at every time. A direct corollary is that the reconstruction error is also bounded by a time-invariant value at all times. If the support set of the sparse signal sequence changes slowly over time, our results hold under weaker assumptions than what simple compressive sensing (CS) needs for the same error bound. Also, our support error bounds are small compared to the support size. Our discussion is backed up by Monte Carlo simulation based comparisons.

cs.IT↗

arXiv2011

Support-Predicted Modified-CS for Recursive Robust Principal Components' Pursuit

This work proposes a causal and recursive algorithm for solving the "robust" principal components' analysis (PCA) problem. We primarily focus on robustness to correlated outliers. In recent work, we proposed a new way to look at this problem and showed how a key part of its solution strategy involves solving a noisy compressive sensing(CS) problem. However, if the support size of the outliers becomes too large, for a given dimension of the current PC space, then the number of "measurements" available for CS may become too small. In this work, we show how to address this issue by utilizing the correlation of the outliers to predict their support at the current time; and using this as "partial support knowledge" for solving Modified-CS instead of CS.

cs.IT↗

arXiv2011

Real-time Robust Principal Components' Pursuit

In the recent work of Candes et al, the problem of recovering low rank matrix corrupted by i.i.d. sparse outliers is studied and a very elegant solution, principal component pursuit, is proposed. It is motivated as a tool for video surveillance applications with the background image sequence forming the low rank part and the moving objects/persons/abnormalities forming the sparse part. Each image frame is treated as a column vector of the data matrix made up of a low rank matrix and a sparse corruption matrix. Principal component pursuit solves the problem under the assumptions that the singular vectors of the low rank matrix are spread out and the sparsity pattern of the sparse matrix is uniformly random. However, in practice, usually the sparsity pattern and the signal values of the sparse part (moving persons/objects) change in a correlated fashion over time, for e.g., the object moves slowly and/or with roughly constant velocity. This will often result in a low rank sparse matrix. For video surveillance applications, it would be much more useful to have a real-time solution. In this work, we study the online version of the above problem and propose a solution that automatically handles correlated sparse outliers. The key idea of this work is as follows. Given an initial estimate of the principal directions of the low rank part, we causally keep estimating the sparse part at each time by solving a noisy compressive sensing type problem. The principal directions of the low rank part are updated every-so-often. In between two update times, if new Principal Components' directions appear, the "noise" seen by the Compressive Sensing step may increase. This problem is solved, in part, by utilizing the time correlation model of the low rank part. We call the proposed solution "Real-time Robust Principal Components' Pursuit".

cs.CV↗

arXiv2010

Modified-CS: Modifying Compressive Sensing for Problems with Partially Known Support

We study the problem of reconstructing a sparse signal from a limited number of its linear projections when a part of its support is known, although the known part may contain some errors. The ``known" part of the support, denoted T, may be available from prior knowledge. Alternatively, in a problem of recursively reconstructing time sequences of sparse spatial signals, one may use the support estimate from the previous time instant as the ``known" part. The idea of our proposed solution (modified-CS) is to solve a convex relaxation of the following problem: find the signal that satisfies the data constraint and is sparsest outside of T. We obtain sufficient conditions for exact reconstruction using modified-CS. These are much weaker than those needed for compressive sensing (CS) when the sizes of the unknown part of the support and of errors in the known part are small compared to the support size. An important extension called Regularized Modified-CS (RegModCS) is developed which also uses prior signal estimate knowledge. Simulation comparisons for both sparse and compressible signals are shown.

cs.IT↗

arXiv2010

Stability (over time) of Modified-CS and LS-CS for Recursive Causal Sparse Reconstruction

In this work, we obtain sufficient conditions for the ``stability" of our recently proposed algorithms, modified-CS (for noisy measurements) and Least Squares CS-residual (LS-CS), designed for recursive reconstruction of sparse signal sequences from noisy measurements. By ``stability" we mean that the number of misses from the current support estimate and the number of extras in it remain bounded by a time-invariant value at all times. The concept is meaningful only if the bound is small compared to the current signal support size. A direct corollary is that the reconstruction errors are also bounded by a time-invariant and small value.

cs.IT↗