arXiv subjects

Felix Krahmer

Publications and source records attributed to Felix Krahmer.

At least 73 records · Page 4Linked to original sources

arXiv2016

Empirical Chaos Processes and Blind Deconvolution

This paper investigates conditions under which certain kinds of systems of bilinear equations have a unique structured solution. In particular, we look at when we can recover vectors $\boldsymbol{w},\boldsymbol{q}$ from observations of the form \[ y_{\ell} = <\boldsymbol{w},\boldsymbol{b}_{\ell}><\boldsymbol{c}_{\ell},\boldsymbol{q}>, \quad \ell = 1,\ldots,L, \] where $\boldsymbol{b}_\ell,\boldsymbol{c}_\ell$ are known. We show that if $\boldsymbol{w}\in\mathbb{C}^{M_1}$ and $\boldsymbol{q}\in\mathbb{C}^{M_2}$ are sparse, with no more than $K$ and $N$ nonzero entries, respectively, and the $\boldsymbol{b}_\ell,\boldsymbol{c}_\ell$ are generic, selected as independent Gaussian random vectors, then $\boldsymbol{w},\boldsymbol{q}$ are uniquely determined from \[ L \geq \mathrm{Const}\cdot (K+N)\log^5(M_1M_2) \] such equations with high probability. The key ingredient in our analysis is a uniform probabilistic bound on how far a random process of the form \[Z(\boldsymbol{X}) = \sum_{\ell=1}^L|\boldsymbol{b}_\ell^*\boldsymbol{X}\boldsymbol{c}_\ell|^2 \] deviates from its mean over a set of structured matrices $\boldsymbol{X}\in\mathcal{X}$. As both $\boldsymbol{b}_\ell$ and $\boldsymbol{c}_\ell$ are random, this is a specialized type of $4$th order chaos; we refer to $Z(\boldsymbol{X})$ as an {\em empirical chaos process}. Bounding this process yields a set of general conditions for when the map $\boldsymbol{X}\rightarrow \{\boldsymbol{b}_\ell^*\boldsymbol{X}\boldsymbol{c}_\ell\}_{\ell=1}^L$ is a restricted isometry over the set of matrices $\mathcal{X}$. The conditions are stated in terms of general geometric properties of the set $\mathcal{X}$, and are explicitly computed for the case where $\mathcal{X}$ is the set of matrices that are simultaneously sparse and low rank.

cs.IT↗

arXiv2016

The Homotopy Method Revisited: Computing Solution Paths of $\ell_1$-Regularized Problems

$ \ell_1 $-regularized linear inverse problems are frequently used in signal processing, image analysis, and statistics. The correct choice of the regularization parameter $ t \in \mathbb{R}_{\geq 0} $ is a delicate issue. Instead of solving the variational problem for a fixed parameter, the idea of the homotopy method is to compute a complete solution path $ u(t) $ as a function of $ t $. In a celebrated paper by Osborne, Presnell, and Turlach, it has been shown that the computational cost of this approach is often comparable to the cost of solving the corresponding least squares problem. Their analysis relies on the one-at-a-time condition, which requires that different indices enter or leave the support of the solution at distinct regularization parameters. In this paper, we introduce a generalized homotopy algorithm based on a nonnegative least squares problem, which does not require such a condition, and prove its termination after finitely many steps. At every point of the path, we give a full characterization of all possible directions. To illustrate our results, we discuss examples in which the standard homotopy method either fails or becomes infeasible. To the best of our knowledge, our algorithm is the first to provably compute a full solution path for an arbitrary combination of an input matrix and a data vector.

math.OC↗

arXiv2016

Optimal Injectivity Conditions for Bilinear Inverse Problems with Applications to Identifiability of Deconvolution Problems

We study identifiability for bilinear inverse problems under sparsity and subspace constraints. We show that, up to a global scaling ambiguity, almost all such maps are injective on the set of pairs of sparse vectors if the number of measurements $m$ exceeds $2(s_1+s_2)-2$, where $s_1$ and $s_2$ denote the sparsity of the two input vectors, and injective on the set of pairs of vectors lying in known subspaces of dimensions $n_1$ and $n_2$ if $m\geq 2(n_1+n_2)-4$. We also prove that both these bounds are tight in the sense that one cannot have injectivity for a smaller number of measurements. Our proof technique draws from algebraic geometry. As an application we derive optimal identifiability conditions for the deconvolution problem, thus improving on recent work of Li et al. [1].

cs.IT↗

arXiv2016

Improved Recovery Guarantees for Phase Retrieval from Coded Diffraction Patterns

In this work we analyze the problem of phase retrieval from Fourier measurements with random diffraction patterns. To this end, we consider the recently introduced PhaseLift algorithm, which expresses the problem in the language of convex optimization. We provide recovery guarantees which require O(log^2 d) different diffraction patterns, thus improving on recent results by Candes et al. [arXiv:1310.3240], which require O(log^4 d) different patterns.

cs.IT↗

arXiv2015

An arithmetic-geometric mean inequality for products of three matrices

Consider the following noncommutative arithmetic-geometric mean inequality: given positive-semidefinite matrices $\mathbf{A}_1, \dots, \mathbf{A}_n$, the following holds for each integer $m \leq n$: $$ \frac{1}{n^m}\sum_{j_1, j_2, \dots, j_m = 1}^{n} ||| \mathbf{A}_{j_1} \mathbf{A}_{j_2} \dots \mathbf{A}_{j_m} ||| \geq \frac{(n-m)!}{n!} \sum_{\substack{j_1, j_2, \dots, j_m = 1 \\ \text{all distinct}}}^{n} ||| \mathbf{A}_{j_1} \mathbf{A}_{j_2} \dots \mathbf{A}_{j_m} |||,$$ where $||| \cdot |||$ denotes a unitarily invariant norm, including the operator norm and Schatten p-norms as special cases. While this inequality in full generality remains a conjecture, we prove that the inequality holds for products of up to three matrices, $m \leq 3$. The proofs for $m = 1,2$ are straightforward; to derive the proof for $m=3$, we appeal to a variant of the classic Araki-Lieb-Thirring inequality for permutations of matrix products.

math.SP↗

arXiv2015

Compressive Sensing with Redundant Dictionaries and Structured Measurements

Consider the problem of recovering an unknown signal from undersampled measurements, given the knowledge that the signal has a sparse representation in a specified dictionary $D$. This problem is now understood to be well-posed and efficiently solvable under suitable assumptions on the measurements and dictionary, if the number of measurements scales roughly with the sparsity level. One sufficient condition for such is the $D$-restricted isometry property ($D$-RIP), which asks that the sampling matrix approximately preserve the norm of all signals which are sufficiently sparse in $D$. While many classes of random matrices are known to satisfy such conditions, such matrices are not representative of the structural constraints imposed by practical sensing systems. We close this gap in the theory by demonstrating that one can subsample a fixed orthogonal matrix in such a way that the $D$-RIP will hold, provided this basis is sufficiently incoherent with the sparsifying dictionary $D$. We also extend this analysis to allow for weighted sparse expansions. Consequently, we arrive at compressive sensing recovery guarantees for structured measurements and redundant dictionaries, opening the door to a wide array of practical applications.

cs.IT↗

arXiv2015

A unified framework for linear dimensionality reduction in L1

For a family of interpolation norms $\| \cdot \|_{1,2,s}$ on $\mathbb{R}^n$, we provide a distribution over random matrices $Φ_s \in \mathbb{R}^{m \times n}$ parametrized by sparsity level $s$ such that for a fixed set $X$ of $K$ points in $\mathbb{R}^n$, if $m \geq C s \log(K)$ then with high probability, $\frac{1}{2} \| x \|_{1,2,s} \leq \| Φ_s (x) \|_1 \leq 2 \| x\|_{1,2,s}$ for all $x\in X$. Several existing results in the literature reduce to special cases of this result at different values of $s$: for $s=n$, $\| x\|_{1,2,n} \equiv \| x \|_{1}$ and we recover that dimension reducing linear maps can preserve the $\ell_1$-norm up to a distortion proportional to the dimension reduction factor, which is known to be the best possible such result. For $s=1$, $\|x \|_{1,2,1} \equiv \| x \|_{2}$, and we recover an $\ell_2 / \ell_1$ variant of the Johnson-Lindenstrauss Lemma for Gaussian random matrices. Finally, if $x$ is $s$-sparse, then $\| x \|_{1,2,s} = \| x \|_1$ and we recover that $s$-sparse vectors in $\ell_1^n$ embed into $\ell_1^{\mathcal{O}(s \log(n))}$ via sparse random matrix constructions.

cs.DS↗

arXiv2015

Noise-shaping Quantization Methods for Frame-based and Compressive Sampling Systems

Noise shaping refers to an analog-to-digital conversion methodology in which quantization error is arranged to lie mostly outside the signal spectrum by means of oversampling and feedback. Recently it has been successfully applied to more general redundant linear sampling and reconstruction systems associated with frames as well as non-linear systems associated with compressive sampling. This chapter reviews some of the recent progress in this subject.

cs.IT↗

arXiv2015

A Novel Compressed Sensing Scheme for Photoacoustic Tomography

Speeding up the data acquisition is one of the central aims to advance tomographic imaging. On the one hand, this reduces motion artifacts due to undesired movements, and on the other hand this decreases the examination time for the patient. In this article, we propose a new scheme for speeding up the data collection process in photoacoustic tomography. Our proposal is based on compressed sensing and reduces acquisition time and system costs while maintaining image quality. As measurement data we use random combinations of pressure values that we use to recover a complete set of pressure data prior to the actual image reconstruction. We obtain theoretical recovery guarantees for our compressed sensing scheme and support the theory by reconstruction results on simulated data as well as on experimental data.

math.NA↗

arXiv2014

Quantization and Compressive Sensing

Quantization is an essential step in digitizing signals, and, therefore, an indispensable component of any modern acquisition system. This book chapter explores the interaction of quantization and compressive sensing and examines practical quantization strategies for compressive acquisition systems. Specifically, we first provide a brief overview of quantization and examine fundamental performance bounds applicable to any quantization approach. Next, we consider several forms of scalar quantizers, namely uniform, non-uniform, and 1-bit. We provide performance bounds and fundamental analysis, as well as practical quantizer designs and reconstruction algorithms that account for quantization. Furthermore, we provide an overview of Sigma-Delta ($ΣΔ$) quantization in the compressed sensing context, and also discuss implementation issues, recovery algorithms and performance bounds. As we demonstrate, proper accounting for quantization and careful quantizer design has significant impact in the performance of a compressive acquisition system.

cs.IT↗

arXiv2014

Structured random measurements in signal processing

Compressed sensing and its extensions have recently triggered interest in randomized signal acquisition. A key finding is that random measurements provide sparse signal reconstruction guarantees for efficient and stable algorithms with a minimal number of samples. While this was first shown for (unstructured) Gaussian random measurement matrices, applications require certain structure of the measurements leading to structured random measurement matrices. Near optimal recovery guarantees for such structured measurements have been developed over the past years in a variety of contexts. This article surveys the theory in three scenarios: compressed sensing (sparse recovery), low rank matrix recovery, and phaseless estimation. The random measurement matrices to be considered include random partial Fourier matrices, partial random circulant matrices (subsampled convolutions), matrix completion, and phase estimation from magnitudes of Fourier type measurements. The article concludes with a brief discussion of the mathematical techniques for the analysis of such structured random measurements.

cs.IT↗

arXiv2014

An RIP-based approach to $ΣΔ$ quantization for compressed sensing

In this paper, we provide a new approach to estimating the error of reconstruction from $ΣΔ$ quantized compressed sensing measurements. Our method is based on the restricted isometry property (RIP) of a certain projection of the measurement matrix. Our result yields simple proofs and a slight generalization of the best-known reconstruction error bounds for Gaussian and subgaussian measurement matrices.

cs.IT↗

arXiv2013

Fast Subspace Approximation via Greedy Least-Squares

In this note, we develop fast and deterministic dimensionality reduction techniques for a family of subspace approximation problems. Let $P\subset \mathbbm{R}^N$ be a given set of $M$ points. The techniques developed herein find an $O(n \log M)$-dimensional subspace that is guaranteed to always contain a near-best fit $n$-dimensional hyperplane $\mathcal{H}$ for $P$ with respect to the cumulative projection error $(\sum_{{\bf x} \in P} \| {\bf x} - Π_\mathcal{H} {\bf x} \|^p_2)^{1/p}$, for any chosen $p > 2$. The deterministic algorithm runs in $\tilde{O} (MN^2)$-time, and can be randomized to run in only $\tilde{O} (MNn)$-time while maintaining its error guarantees with high probability. In the case $p = \infty$ the dimensionality reduction techniques can be combined with efficient algorithms for computing the John ellipsoid of a data set in order to produce an $n$-dimensional subspace whose maximum $\ell_2$-distance to any point in the convex hull of $P$ is minimized. The resulting algorithm remains $\tilde{O} (MNn)$-time. In addition, the dimensionality reduction techniques developed herein can also be combined with other existing subspace approximation algorithms for $2 < p \leq \infty$ - including more accurate algorithms based on convex programming relaxations - in order to reduce their runtimes.

cs.CG↗

arXiv2013

Stable and robust sampling strategies for compressive imaging

In many signal processing applications, one wishes to acquire images that are sparse in transform domains such as spatial finite differences or wavelets using frequency domain samples. For such applications, overwhelming empirical evidence suggests that superior image reconstruction can be obtained through variable density sampling strategies that concentrate on lower frequencies. The wavelet and Fourier transform domains are not incoherent because low-order wavelets and low-order frequencies are correlated, so compressive sensing theory does not immediately imply sampling strategies and reconstruction guarantees. In this paper we turn to a more refined notion of coherence -- the so-called local coherence -- measuring for each sensing vector separately how correlated it is to the sparsity basis. For Fourier measurements and Haar wavelet sparsity, the local coherence can be controlled and bounded explicitly, so for matrices comprised of frequencies sampled from a suitable inverse square power-law density, we can prove the restricted isometry property with near-optimal embedding dimensions. Consequently, the variable-density sampling strategy we provide allows for image reconstructions that are stable to sparsity defects and robust to measurement noise. Our results cover both reconstruction by $\ell_1$-minimization and by total variation minimization. The local coherence framework developed in this paper should be of independent interest in sparse recovery problems more generally, as it implies that for optimal sparse recovery results, it suffices to have bounded \emph{average} coherence from sensing basis to sparsity basis -- as opposed to bounded maximal coherence -- as long as the sampling strategy is adapted accordingly.

cs.CV↗

arXiv2013

Local sampling and approximation of operators with bandlimited Kohn-Nirenberg symbols

Recent sampling theorems allow for the recovery of operators with bandlimited Kohn-Nirenberg symbols from their response to a single discretely supported identifier signal. The available results are inherently non-local. For example, we show that in order to recover a bandlimited operator precisely, the identifier cannot decay in time nor in frequency. Moreover, a concept of local and discrete representation is missing from the theory. In this paper, we develop tools that address these shortcomings. We show that to obtain a local approximation of an operator, it is sufficient to test the operator on a truncated and mollified delta train, that is, on a compactly supported Schwarz class function. To compute the operator numerically, discrete measurements can be obtained from the response function which are localized in the sense that a local selection of the values yields a local approximation of the operator. Central to our analysis is to conceptualize the meaning of localization for operators with bandlimited Kohn-Nirenberg symbol.

math.FA↗

arXiv2013

Suprema of Chaos Processes and the Restricted Isometry Property

We present a new bound for suprema of a special type of chaos processes indexed by a set of matrices, which is based on a chaining method. As applications we show significantly improved estimates for the restricted isometry constants of partial random circulant matrices and time-frequency structured random matrices. In both cases the required condition on the number $m$ of rows in terms of the sparsity $s$ and the vector length $n$ is $m \gtrsim s \log^2 s \log^2 n$.

math.PR↗

arXiv2013

Sigma-Delta quantization of sub-Gaussian frame expansions and its application to compressed sensing

Suppose that the collection $\{e_i\}_{i=1}^m$ forms a frame for $\R^k$, where each entry of the vector $e_i$ is a sub-Gaussian random variable. We consider expansions in such a frame, which are then quantized using a Sigma-Delta scheme. We show that an arbitrary signal in $\R^k$ can be recovered from its quantized frame coefficients up to an error which decays root-exponentially in the oversampling rate $m/k$. Here the quantization scheme is assumed to be chosen appropriately depending on the oversampling rate and the quantization alphabet can be coarse. The result holds with high probability on the draw of the frame uniformly for all signals. The crux of the argument is a bound on the extreme singular values of the product of a deterministic matrix and a sub-Gaussian frame. For fine quantization alphabets, we leverage this bound to show polynomial error decay in the context of compressed sensing. Our results extend previous results for structured deterministic frame expansions and Gaussian compressed sensing measurements.

cs.IT↗

arXiv2012

Sparsity and spectral properties of dual frames

We study sparsity and spectral properties of dual frames of a given finite frame. We show that any finite frame has a dual with no more than $n^2$ non-vanishing entries, where $n$ denotes the ambient dimension, and that for most frames no sparser dual is possible. Moreover, we derive an expression for the exact sparsity level of the sparsest dual for any given finite frame using a generalized notion of spark. We then study the spectral properties of dual frames in terms of singular values of the synthesis operator. We provide a complete characterization for which spectral patterns of dual frames are possible for a fixed frame. For many cases, we provide simple explicit constructions for dual frames with a given spectrum, in particular, if the constraint on the dual is that it be tight.

math.FA↗