arXiv subjects

Yonina C. Eldar

Publications and source records attributed to Yonina C. Eldar.

At least 469 records · Page 26Linked to original sources

arXiv2009

Uncertainty Relations for Shift-Invariant Analog Signals

The past several years have witnessed a surge of research investigating various aspects of sparse representations and compressed sensing. Most of this work has focused on the finite-dimensional setting in which the goal is to decompose a finite-length vector into a given finite dictionary. Underlying many of these results is the conceptual notion of an uncertainty principle: a signal cannot be sparsely represented in two different bases. Here, we extend these ideas and results to the analog, infinite-dimensional setting by considering signals that lie in a finitely-generated shift-invariant (SI) space. This class of signals is rich enough to include many interesting special cases such as multiband signals and splines. By adapting the notion of coherence defined for finite dictionaries to infinite SI representations, we develop an uncertainty principle similar in spirit to its finite counterpart. We demonstrate tightness of our bound by considering a bandlimited lowpass train that achieves the uncertainty principle. Building upon these results and similar work in the finite setting, we show how to find a sparse decomposition in an overcomplete dictionary by solving a convex optimization problem. The distinguishing feature of our approach is the fact that even though the problem is defined over an infinite domain with infinitely many variables and constraints, under certain conditions on the dictionary spectrum our algorithm can find the sparsest representation by solving a finite-dimensional problem.

cs.IT↗

arXiv2009

Recovering Signals from Lowpass Data

The problem of recovering a signal from its low frequency components occurs often in practical applications due to the lowpass behavior of many physical systems. Here we study in detail conditions under which a signal can be determined from its low-frequency content. We focus on signals in shift-invariant spaces generated by multiple generators. For these signals, we derive necessary conditions on the cutoff frequency of the lowpass filter as well as necessary and sufficient conditions on the generators such that signal recovery is possible. When the lowpass content is not sufficient to determine the signal, we propose appropriate pre-processing that can improve the reconstruction ability. In particular, we show that modulating the signal with one or more mixing functions prior to lowpass filtering, can ensure the recovery of the signal in many cases, and reduces the necessary bandwidth of the filter.

cs.IT↗

arXiv2009

Non-Invertible Gabor Transforms

Time-frequency analysis, such as the Gabor transform, plays an important role in many signal processing applications. The redundancy of such representations is often directly related to the computational load of any algorithm operating in the transform domain. To reduce complexity, it may be desirable to increase the time and frequency sampling intervals beyond the point where the transform is invertible, at the cost of an inevitable recovery error. In this paper we initiate the study of recovery procedures for non-invertible Gabor representations. We propose using fixed analysis and synthesis windows, chosen e.g. according to implementation constraints, and to process the Gabor coefficients prior to synthesis in order to shape the reconstructed signal. We develop three methods to tackle this problem. The first follows from the consistency requirement, namely that the recovered signal has the same Gabor representation as the input signal. The second, is based on the minimization of a worst-case error criterion. Last, we develop a recovery technique based on the assumption that the input signal lies in some subspace of $L_2$. We show that for each of the criteria, the manipulation of the transform coefficients amounts to a 2D twisted convolution operation, which we show how to perform using a filter-bank. When the under-sampling factor is an integer, the processing reduces to standard 2D convolution. We provide simulation results to demonstrate the advantages and weaknesses of each of the algorithms.

math.CA↗

arXiv2009

Compressed Sensing of Block-Sparse Signals: Uncertainty Relations and Efficient Recovery

We consider compressed sensing of block-sparse signals, i.e., sparse signals that have nonzero coefficients occurring in clusters. An uncertainty relation for block-sparse signals is derived, based on a block-coherence measure, which we introduce. We then show that a block-version of the orthogonal matching pursuit algorithm recovers block $k$-sparse signals in no more than $k$ steps if the block-coherence is sufficiently small. The same condition on block-coherence is shown to guarantee successful recovery through a mixed $\ell_2/\ell_1$-optimization approach. This complements previous recovery results for the block-sparse case which relied on small block-restricted isometry constants. The significance of the results presented in this paper lies in the fact that making explicit use of block-sparsity can provably yield better reconstruction properties than treating the signal as being sparse in the conventional sense, thereby ignoring the additional structure in the problem.

cs.IT↗

arXiv2009

A Lower Bound on the Bayesian MSE Based on the Optimal Bias Function

A lower bound on the minimum mean-squared error (MSE) in a Bayesian estimation problem is proposed in this paper. This bound utilizes a well-known connection to the deterministic estimation setting. Using the prior distribution, the bias function which minimizes the Cramer-Rao bound can be determined, resulting in a lower bound on the Bayesian MSE. The bound is developed for the general case of a vector parameter with an arbitrary probability distribution, and is shown to be asymptotically tight in both the high and low signal-to-noise ratio regimes. A numerical study demonstrates several cases in which the proposed technique is both simpler to compute and tighter than alternative methods.

cs.IT↗

arXiv2009

Average Case Analysis of Multichannel Sparse Recovery Using Convex Relaxation

In this paper, we consider recovery of jointly sparse multichannel signals from incomplete measurements. Several approaches have been developed to recover the unknown sparse vectors from the given observations, including thresholding, simultaneous orthogonal matching pursuit (SOMP), and convex relaxation based on a mixed matrix norm. Typically, worst-case analysis is carried out in order to analyze conditions under which the algorithms are able to recover any jointly sparse set of vectors. However, such an approach is not able to provide insights into why joint sparse recovery is superior to applying standard sparse reconstruction methods to each channel individually. Previous work considered an average case analysis of thresholding and SOMP by imposing a probability model on the measured signals. In this paper, our main focus is on analysis of convex relaxation techniques. In particular, we focus on the mixed l_2,1 approach to multichannel recovery. We show that under a very mild condition on the sparsity and on the dictionary characteristics, measured for example by the coherence, the probability of recovery failure decays exponentially in the number of channels. This demonstrates that most of the time, multichannel sparse recovery is indeed superior to single channel methods. Our probability bounds are valid and meaningful even for a small number of signals. Using the tools we develop to analyze the convex relaxation method, we also tighten the previous bounds for thresholding and SOMP.

cs.IT↗

arXiv2009

Robust Recovery of Signals From a Structured Union of Subspaces

Traditional sampling theories consider the problem of reconstructing an unknown signal $x$ from a series of samples. A prevalent assumption which often guarantees recovery from the given measurements is that $x$ lies in a known subspace. Recently, there has been growing interest in nonlinear but structured signal models, in which $x$ lies in a union of subspaces. In this paper we develop a general framework for robust and efficient recovery of such signals from a given set of samples. More specifically, we treat the case in which $x$ lies in a sum of $k$ subspaces, chosen from a larger set of $m$ possibilities. The samples are modelled as inner products with an arbitrary set of sampling functions. To derive an efficient and robust recovery algorithm, we show that our problem can be formulated as that of recovering a block-sparse vector whose non-zero elements appear in fixed blocks. We then propose a mixed $\ell_2/\ell_1$ program for block sparse recovery. Our main result is an equivalence condition under which the proposed convex algorithm is guaranteed to recover the original signal. This result relies on the notion of block restricted isometry property (RIP), which is a generalization of the standard RIP used extensively in the context of compressed sensing. Based on RIP we also prove stability of our approach in the presence of noise and modelling errors. A special case of our framework is that of recovering multiple measurement vectors (MMV) that share a joint sparsity pattern. Adapting our results to this context leads to new MMV recovery methods as well as equivalence conditions under which the entire set can be determined efficiently.

nlin.CG↗

arXiv2009

Compressed Sensing of Analog Signals in Shift-Invariant Spaces

A traditional assumption underlying most data converters is that the signal should be sampled at a rate exceeding twice the highest frequency. This statement is based on a worst-case scenario in which the signal occupies the entire available bandwidth. In practice, many signals are sparse so that only part of the bandwidth is used. In this paper, we develop methods for low-rate sampling of continuous-time sparse signals in shift-invariant (SI) spaces, generated by m kernels with period T. We model sparsity by treating the case in which only k out of the m generators are active, however, we do not know which k are chosen. We show how to sample such signals at a rate much lower than m/T, which is the minimal sampling rate without exploiting sparsity. Our approach combines ideas from analog sampling in a subspace with a recently developed block diagram that converts an infinite set of sparse equations to a finite counterpart. Using these two components we formulate our problem within the framework of finite compressed sensing (CS) and then rely on algorithms developed in that context. The distinguishing feature of our results is that in contrast to standard CS, which treats finite-length vectors, we consider sampling of analog signals for which no underlying finite-dimensional model exists. The proposed framework allows to extend much of the recent literature on CS to the analog domain.

cs.IT↗

arXiv2009

Covariance estimation in decomposable Gaussian graphical models

Graphical models are a framework for representing and exploiting prior conditional independence structures within distributions using graphs. In the Gaussian case, these models are directly related to the sparsity of the inverse covariance (concentration) matrix and allow for improved covariance estimation with lower computational complexity. We consider concentration estimation with the mean-squared error (MSE) as the objective, in a special type of model known as decomposable. This model includes, for example, the well known banded structure and other cases encountered in practice. Our first contribution is the derivation and analysis of the minimum variance unbiased estimator (MVUE) in decomposable graphical models. We provide a simple closed form solution to the MVUE and compare it with the classical maximum likelihood estimator (MLE) in terms of performance and complexity. Next, we extend the celebrated Stein's unbiased risk estimate (SURE) to graphical models. Using SURE, we prove that the MSE of the MVUE is always smaller or equal to that of the biased MLE, and that the MVUE itself is dominated by other approaches. In addition, we propose the use of SURE as a constructive mechanism for deriving new covariance estimators. Similarly to the classical MLE, all of our proposed estimators have simple closed form solutions but result in a significant reduction in MSE.

math.ST↗

arXiv2009

Efficient Sampling of Sparse Wideband Analog Signals

Periodic nonuniform sampling is a known method to sample spectrally sparse signals below the Nyquist rate. This strategy relies on the implicit assumption that the individual samplers are exposed to the entire frequency range. This assumption becomes impractical for wideband sparse signals. The current paper proposes an alternative sampling stage that does not require a full-band front end. Instead, signals are captured with an analog front end that consists of a bank of multipliers and lowpass filters whose cutoff is much lower than the Nyquist rate. The problem of recovering the original signal from the low-rate samples can be studied within the framework of compressive sampling. An appropriate parameter selection ensures that the samples uniquely determine the analog input. Moreover, the analog input can be stably reconstructed with digital algorithms. Numerical experiments support the theoretical analysis.

cs.IT↗

arXiv2008

Block-Sparsity: Coherence and Efficient Recovery

We consider compressed sensing of block-sparse signals, i.e., sparse signals that have nonzero coefficients occuring in clusters. Based on an uncertainty relation for block-sparse signals, we define a block-coherence measure and we show that a block-version of the orthogonal matching pursuit algorithm recovers block k-sparse signals in no more than k steps if the block-coherence is sufficiently small. The same condition on block-sparsity is shown to guarantee successful recovery through a mixed l2/l1 optimization approach. The significance of the results lies in the fact that making explicit use of block-sparsity can yield better reconstruction properties than treating the signal as being sparse in the conventional sense thereby ignoring the additional structure in the problem.

cs.IT↗

arXiv2008

Generalized SURE for Exponential Families: Applications to Regularization

Stein's unbiased risk estimate (SURE) was proposed by Stein for the independent, identically distributed (iid) Gaussian model in order to derive estimates that dominate least-squares (LS). In recent years, the SURE criterion has been employed in a variety of denoising problems for choosing regularization parameters that minimize an estimate of the mean-squared error (MSE). However, its use has been limited to the iid case which precludes many important applications. In this paper we begin by deriving a SURE counterpart for general, not necessarily iid distributions from the exponential family. This enables extending the SURE design technique to a much broader class of problems. Based on this generalization we suggest a new method for choosing regularization parameters in penalized LS estimators. We then demonstrate its superior performance over the conventional generalized cross validation approach and the discrepancy method in the context of image deblurring and deconvolution. The SURE technique can also be used to design estimates without predefining their structure. However, allowing for too many free parameters impairs the performance of the resulting estimates. To address this inherent tradeoff we propose a regularized SURE objective. Based on this design criterion, we derive a wavelet denoising strategy that is similar in sprit to the standard soft-threshold approach but can lead to improved MSE performance.

stat.ME↗

arXiv2008

Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors

The rapid developing area of compressed sensing suggests that a sparse vector lying in an arbitrary high dimensional space can be accurately recovered from only a small set of non-adaptive linear measurements. Under appropriate conditions on the measurement matrix, the entire information about the original sparse vector is captured in the measurements, and can be recovered using efficient polynomial methods. The vector model has been extended to a finite set of sparse vectors sharing a common non-zero location set. In this paper, we treat a broader framework in which the goal is to recover a possibly infinite set of jointly sparse vectors. Extending existing recovery methods to this model is difficult due to the infinite structure of the sparse vector set. Instead, we prove that the entire infinite set of sparse vectors can recovered by solving a single, reduced-size finite-dimensional problem, corresponding to recovery of a finite set of sparse vectors. We then show that the problem can be further reduced to the basic recovery of a single sparse vector by randomly combining the measurement vectors. Our approach results in exact recovery of both countable and uncountable sets as it does not rely on discretization or heuristic techniques. To efficiently recover the single sparse vector produced by the last reduction step, we suggest an empirical boosting strategy that improves the recovery ability of any given sub-optimal method for recovering a sparse vector. Numerical experiments on random data demonstrate that when applied to infinite sets our strategy outperforms discretization techniques in terms of both run time and empirical recovery rate. In the finite model, our boosting algorithm is characterized by fast run time and superior recovery rate than known popular methods.

nlin.CG↗

arXiv2007

Blind Minimax Estimation

We consider the linear regression problem of estimating an unknown, deterministic parameter vector based on measurements corrupted by colored Gaussian noise. We present and analyze blind minimax estimators (BMEs), which consist of a bounded parameter set minimax estimator, whose parameter set is itself estimated from measurements. Thus, one does not require any prior assumption or knowledge, and the proposed estimator can be applied to any linear regression problem. We demonstrate analytically that the BMEs strictly dominate the least-squares estimator, i.e., they achieve lower mean-squared error for any value of the parameter vector. Both Stein's estimator and its positive-part correction can be derived within the blind minimax framework. Furthermore, our approach can be readily extended to a wider class of estimation problems than Stein's estimator, which is defined only for white noise and non-transformed measurements. We show through simulations that the BMEs generally outperform previous extensions of Stein's technique.

math.ST↗

arXiv2007

Blind Multi-Band Signal Reconstruction: Compressed Sensing for Analog Signals

We address the problem of reconstructing a multi-band signal from its sub-Nyquist point-wise samples. To date, all reconstruction methods proposed for this class of signals assumed knowledge of the band locations. In this paper, we develop a non-linear blind perfect reconstruction scheme for multi-band signals which does not require the band locations. Our approach assumes an existing blind multi-coset sampling method. The sparse structure of multi-band signals in the continuous frequency domain is used to replace the continuous reconstruction with a single finite dimensional problem without the need for discretization. The resulting problem can be formulated within the framework of compressed sensing, and thus can be solved efficiently using known tractable algorithms from this emerging area. We also develop a theoretical lower bound on the average sampling rate required for blind signal reconstruction, which is twice the minimal rate of known-spectrum recovery. Our method ensures perfect reconstruction for a wide class of signals sampled at the minimal rate. Numerical experiments are presented demonstrating blind sampling and reconstruction with minimal sampling rate.

nlin.CG↗

arXiv2006

Optimal Encoding of Classical Information in a Quantum Medium

We investigate optimal encoding and retrieval of digital data, when the storage/communication medium is described by quantum mechanics. We assume an m-ary alphabet with arbitrary prior distribution, and an n-dimensional quantum system. Under these constraints, we seek an encoding-retrieval setup, comprised of code-states and a quantum measurement, which maximizes the probability of correct detection. In our development, we consider two cases. In the first, the measurement is predefined and we seek the optimal code-states. In the second, optimization is performed on both the code-states and the measurement. We show that one cannot outperform `pseudo-classical transmission', in which we transmit n symbols with orthogonal code-states, and discard the remaining symbols. However, such pseudo-classical transmission is not the only optimum. We fully characterize the collection of optimal setups, and briefly discuss the links between our findings and applications such as quantum key distribution and quantum computing. We conclude with a number of results concerning the design under an alternative optimality criterion, the worst-case posterior probability, which serves as a measure of the retrieval reliability.

quant-ph↗

arXiv2005

A constructive inversion framework for twisted convolution

In this paper we develop constructive invertibility conditions for the twisted convolution. Our approach is based on splitting the twisted convolution with rational parameters into a finite number of weighted convolutions, which can be interpreted as another twisted convolution on a finite cyclic group. In analogy with the twisted convolution of finite discrete signals, we derive an anti-homomorphism between the sequence space and a suitable matrix algebra which preserves the algebraic structure. In this way, the problem reduces to the analysis of finite matrices whose entries are sequences supported on corresponding cosets. The invertibility condition then follows from Cramer's rule and Wiener's lemma for this special class of matrices. The problem results from a well known approach of studying the invertibility properties of the Gabor frame operator in the rational case. The presented approach gives further insights into Gabor frames. In particular, it can be applied for both the continuous (on $\Rd$) and the finite discrete setting. In the latter case, we obtain algorithmic schemes for directly computing the inverse of Gabor frame-type matrices equivalent to those known in the literature.

math.FA↗

arXiv2005

Quantum Detection with Unknown States

We address the problem of distinguishing among a finite collection of quantum states, when the states are not entirely known. For completely specified states, necessary and sufficient conditions on a quantum measurement minimizing the probability of a detection error have been derived. In this work, we assume that each of the states in our collection is a mixture of a known state and an unknown state. We investigate two criteria for optimality. The first is minimization of the worst-case probability of a detection error. For the second we assume a probability distribution on the unknown states, and minimize of the expected probability of a detection error. We find that under both criteria, the optimal detectors are equivalent to the optimal detectors of an ``effective ensemble''. In the worst-case, the effective ensemble is comprised of the known states with altered prior probabilities, and in the average case it is made up of altered states with the original prior probabilities.

quant-ph↗