arXiv subjects

Felix Krahmer

Publications and source records attributed to Felix Krahmer.

At least 37 records · Page 2Linked to original sources

arXiv2023

Uncertainty quantification for sparse Fourier recovery

One of the most prominent methods for uncertainty quantification in high-dimen-sional statistics is the desparsified LASSO that relies on unconstrained $\ell_1$-minimization. The majority of initial works focused on real (sub-)Gaussian designs. However, in many applications, such as magnetic resonance imaging (MRI), the measurement process possesses a certain structure due to the nature of the problem. The measurement operator in MRI can be described by a subsampled Fourier matrix. The purpose of this work is to extend the uncertainty quantification process using the desparsified LASSO to design matrices originating from a bounded orthonormal system, which naturally generalizes the subsampled Fourier case and also allows for the treatment of the case where the sparsity basis is not the standard basis. In particular we construct honest confidence intervals for every pixel of an MR image that is sparse in the standard basis provided the number of measurements satisfies $n \gtrsim\max\{ s\log^2 s\log p, s \log^2 p \}$ or that is sparse with respect to the Haar Wavelet basis provided a slightly larger number of measurements.

cs.IT↗

arXiv2023

Approximating Positive Homogeneous Functions with Scale Invariant Neural Networks

We investigate to what extent it is possible to solve linear inverse problems with $ReLu$ networks. Due to the scaling invariance arising from the linearity, an optimal reconstruction function $f$ for such a problem is positive homogeneous, i.e., satisfies $f(λx) = λf(x)$ for all non-negative $λ$. In a $ReLu$ network, this condition translates to considering networks without bias terms. We first consider recovery of sparse vectors from few linear measurements. We prove that $ReLu$- networks with only one hidden layer cannot even recover $1$-sparse vectors, not even approximately, and regardless of the width of the network. However, with two hidden layers, approximate recovery with arbitrary precision and arbitrary sparsity level $s$ is possible in a stable way. We then extend our results to a wider class of recovery problems including low-rank matrix recovery and phase retrieval. Furthermore, we also consider the approximation of general positive homogeneous functions with neural networks. Extending previous work, we establish new results explaining under which conditions such functions can be approximated with neural networks. Our results also shed some light on the seeming contradiction between previous works showing that neural networks for inverse problems typically have very large Lipschitz constants, but still perform very well also for adversarial noise. Namely, the error bounds in our expressivity results include a combination of a small constant term and a term that is linear in the noise level, indicating that robustness issues may occur only for very small noise levels.

cs.LG↗

arXiv2023

How robust is randomized blind deconvolution via nuclear norm minimization against adversarial noise?

In this paper, we study the problem of recovering two unknown signals from their convolution, which is commonly referred to as blind deconvolution. Reformulation of blind deconvolution as a low-rank recovery problem has led to multiple theoretical recovery guarantees in the past decade due to the success of the nuclear norm minimization heuristic. In particular, in the absence of noise, exact recovery has been established for sufficiently incoherent signals contained in lower-dimensional subspaces. However, if the convolution is corrupted by additive bounded noise, the stability of the recovery problem remains much less understood. In particular, existing reconstruction bounds involve large dimension factors and therefore fail to explain the empirical evidence for dimension-independent robustness of nuclear norm minimization. Recently, theoretical evidence has emerged for ill-posed behavior of low-rank matrix recovery for sufficiently small noise levels. In this work, we develop improved recovery guarantees for blind deconvolution with adversarial noise which exhibit square-root scaling in the noise level. Hence, our results are consistent with existing counterexamples which speak against linear scaling in the noise level as demonstrated for related low-rank matrix recovery problems.

cs.IT↗

arXiv2022

Non-intrusive surrogate modelling using sparse random features with applications in crashworthiness analysis

Efficient surrogate modelling is a key requirement for uncertainty quantification in data-driven scenarios. In this work, a novel approach of using Sparse Random Features for surrogate modelling in combination with self-supervised dimensionality reduction is described. The method is compared to other methods on synthetic and real data obtained from crashworthiness analyses. The results show a superiority of the here described approach over state of the art surrogate modelling techniques, Polynomial Chaos Expansions and Neural Networks.

cs.LG↗

arXiv2022

Riemannian thresholding methods for row-sparse and low-rank matrix recovery

In this paper, we present modifications of the iterative hard thresholding (IHT) method for recovery of jointly row-sparse and low-rank matrices. In particular a Riemannian version of IHT is considered which significantly reduces computational cost of the gradient projection in the case of rank-one measurement operators, which have concrete applications in blind deconvolution. Experimental results are reported that show near-optimal recovery for Gaussian and rank-one measurements, and that adaptive stepsizes give crucial improvement. A Riemannian proximal gradient method is derived for the special case of unknown sparsity.

math.OC↗

arXiv2022

On recovery guarantees for angular synchronization

The angular synchronization problem of estimating a set of unknown angles from their known noisy pairwise differences arises in various applications. It can be reformulated as a optimization problem on graphs involving the graph Laplacian matrix. We consider a general, weighted version of this problem, where the impact of the noise differs between different pairs of entries and some of the differences are erased completely; this version arises for example in ptychography. We study two common approaches for solving this problem, namely eigenvector relaxation and semidefinite convex relaxation. Although some recovery guarantees are available for both methods, their performance is either unsatisfying or restricted to the unweighted graphs. We close this gap, deriving recovery guarantees for the weighted problem that are completely analogous to the unweighted version.

math.OC↗

arXiv2022

Enhanced Digital Halftoning via Weighted Sigma-Delta Modulation

In this paper, we study error diffusion techniques for digital halftoning from the perspective of 1-bit Sigma-Delta quantization. We introduce a method to generate Sigma-Delta schemes for two-dimensional signals as a weighted combination of its one-dimensional counterparts and show that various error diffusion schemes proposed in the literature can be represented in this framework via Sigma-Delta schemes of first order. Under the model of two-dimensional bandlimited signals, which is motivated by a mathematical model of human visual perception, we derive quantitative error bounds for such weighted Sigma-Delta schemes. We see these bounds as a step towards a mathematical understanding of the good empirical performance of error diffusion, even though they are formulated in the supremum norm, which is known to not fully capture the visual similarity of images. Motivated by the correspondence between existing error diffusion algorithms and first-order Sigma-Delta schemes, we study the performance of the analogous weighted combinations of second-order Sigma-Delta schemes and show that they exhibit a superior performance in terms of guaranteed error decay for two-dimensional bandlimited signals. In extensive numerical simulations for real world images, we demonstrate that with some modifications to enhance stability this superior performance also translates to the problem of digital halftoning. More concretely, we find that certain second-order weighted Sigma-Delta schemes exhibit competitive performance for digital halftoning of real world images in terms of the Feature Similarity Index (FSIM), a state-of-the-art measure for image quality assessment.

math.NA↗

arXiv2021

The Surprising Benefits of Hysteresis in Unlimited Sampling: Theory, Algorithms and Experiments

The Unlimited Sensing Framework (USF) was recently introduced to overcome the sensor saturation bottleneck in conventional digital acquisition systems. At its core, the USF allows for high-dynamic-range (HDR) signal reconstruction by converting a continuous-time signal into folded, low-dynamic-range (LDR), modulo samples. HDR reconstruction is then carried out by algorithmic unfolding of the folded samples. In hardware, however, implementing an ideal modulo folding requires careful calibration, analog design and high precision. At the interface of theory and practice, this paper explores a computational sampling strategy that relaxes strict hardware requirements by compensating them via a novel, mathematically guaranteed recovery method. Our starting point is a generalized model for USF. The generalization relies on two new parameters modeling hysteresis and folding transients} in addition to the modulo threshold. Hysteresis accounts for the mismatch between the reset threshold and the amplitude displacement at the folding time and we refer to a continuous transition period in the implementation of a reset as folding transient. Both these effects are motivated by our hardware experiments and also occur in previous, domain-specific applications. We show that the effect of hysteresis is beneficial for the USF and we leverage it to derive the first recovery guarantees in the context of our generalized USF model. Additionally, we show how the proposed recovery can be directly generalized for the case of lower sampling rates. Our theoretical work is corroborated by hardware experiments that are based on a hysteresis enabled, modulo ADC testbed comprising off-the-shelf electronic components. Thus, by capitalizing on a collaboration between hardware and algorithms, our paper enables an end-to-end pipeline for HDR sampling allowing more flexible hardware implementations.

cs.IT↗

arXiv2021

The Hanson-Wright Inequality for Random Tensors

We provide moment bounds for expressions of the type $(X^{(1)} \otimes \dots \otimes X^{(d)})^T A (X^{(1)} \otimes \dots \otimes X^{(d)})$ where $\otimes$ denotes the Kronecker product and $X^{(1)}, \dots, X^{(d)}$ are random vectors with independent, mean 0, variance 1, subgaussian entries. The bounds are tight up to constants depending on $d$ for the case of Gaussian random vectors. Our proof also provides a decoupling inequality for expressions of this type. Using these bounds, we obtain new, improved concentration inequalities for expressions of the form $\|B (X^{(1)} \otimes \dots \otimes X^{(d)})\|_2$.

math.PR↗

arXiv2021

Johnson-Lindenstrauss Embeddings with Kronecker Structure

We prove the Johnson-Lindenstrauss property for matrices $ΦD_ξ$ where $Φ$ has the restricted isometry property and $D_ξ$ is a diagonal matrix containing the entries of a Kronecker product $ξ= ξ^{(1)} \otimes \dots \otimes ξ^{(d)}$ of $d$ independent Rademacher vectors. Such embeddings have been proposed in recent works for a number of applications concerning compression of tensor structured data, including the oblivious sketching procedure by Ahle et al. for approximate tensor computations. For preserving the norms of $p$ points simultaneously, our result requires $Φ$ to have the restricted isometry property for sparsity $C(d) (\log p)^d$. In the case of subsampled Hadamard matrices, this can improve the dependence of the embedding dimension on $p$ to $(\log p)^d$ while the best previously known result required $(\log p)^{d + 1}$. That is, for the case of $d=2$ at the core of the oblivious sketching procedure by Ahle et al., the scaling improves from cubic to quadratic. We provide a counterexample to prove that the scaling established in our result is optimal under mild assumptions.

cs.DS↗

arXiv2021

Proof methods for robust low-rank matrix recovery

Low-rank matrix recovery problems arise naturally as mathematical formulations of various inverse problems, such as matrix completion, blind deconvolution, and phase retrieval. Over the last two decades, a number of works have rigorously analyzed the reconstruction performance for such scenarios, giving rise to a rather general understanding of the potential and the limitations of low-rank matrix models in sensing problems. In this article, we compare the two main proof techniques that have been paving the way to a rigorous analysis, discuss their potential and limitations, and survey their successful applications. On the one hand, we review approaches based on descent cone analysis, showing that they often lead to strong guarantees even in the presence of adversarial noise, but face limitations when it comes to structured observations. On the other hand, we discuss techniques using approximate dual certificates and the golfing scheme, which are often better suited to deal with practical measurement structures, but sometimes lead to weaker guarantees. Lastly, we review recent progress towards analyzing descent cones also for structured scenarios -- exploiting the idea of splitting the cones into multiple parts that are analyzed via different techniques.

cs.IT↗

arXiv2021

Sketching with Kerdock's crayons: Fast sparsifying transforms for arbitrary linear maps

Given an arbitrary matrix $A\in\mathbb{R}^{n\times n}$, we consider the fundamental problem of computing $Ax$ for any $x\in\mathbb{R}^n$ such that $Ax$ is $s$-sparse. While fast algorithms exist for particular choices of $A$, such as the discrete Fourier transform, there is currently no $o(n^2)$ algorithm that treats the unstructured case. In this paper, we devise a randomized approach to tackle the unstructured case. Our method relies on a representation of $A$ in terms of certain real-valued mutually unbiased bases derived from Kerdock sets. In the preprocessing phase of our algorithm, we compute this representation of $A$ in $O(n^3\log n)$ operations. Next, given any unit vector $x\in\mathbb{R}^n$ such that $Ax$ is $s$-sparse, our randomized fast transform uses this representation of $A$ to compute the entrywise $ε$-hard threshold of $Ax$ with high probability in only $O(sn + ε^{-2}\|A\|_{2\to\infty}^2n\log n)$ operations. In addition to a performance guarantee, we provide numerical results that demonstrate the plausibility of real-world implementation of our algorithm.

cs.CC↗

arXiv2021

Unlimited Sampling from Theory to Practice: Fourier-Prony Recovery and Prototype ADC

Following the Unlimited Sampling strategy to alleviate the omnipresent dynamic range barrier, we study the problem of recovering a bandlimited signal from point-wise modulo samples, aiming to connect theoretical guarantees with hardware implementation considerations. Our starting point is a class of non-idealities that we observe in prototyping an unlimited sampling based analog-to-digital converter. To address these non-idealities, we provide a new Fourier domain recovery algorithm. Our approach is validated both in theory and via extensive experiments on our prototype analog-to-digital converter, providing the first demonstration of unlimited sampling for data arising from real hardware, both for the current and previous approaches. Advantages of our algorithm include that it is agnostic to the modulo threshold and it can handle arbitrary folding times. We expect that the end-to-end realization studied in this paper will pave the path for exploring the unlimited sampling methodology in a number of real world applications.

cs.IT↗

arXiv2021

The Modulo Radon Transform: Theory, Algorithms and Applications

Recently, experiments have been reported where researchers were able to perform high dynamic range (HDR) tomography in a heuristic fashion, by fusing multiple tomographic projections. This approach to HDR tomography has been inspired by HDR photography and inherits the same disadvantages. Taking a computational imaging approach to the HDR tomography problem, we here suggest a new model based on the Modulo Radon Transform (MRT), which we rigorously introduce and analyze. By harnessing a joint design between hardware and algorithms, we present a single-shot HDR tomography approach, which to our knowledge, is the only approach that is backed by mathematical guarantees. On the hardware front, instead of recording the Radon Transform projections that my potentially saturate, we propose to measure modulo values of the same. This ensures that the HDR measurements are folded into a lower dynamic range. On the algorithmic front, our recovery algorithms reconstruct the HDR images from folded measurements. Beyond mathematical aspects such as injectivity and inversion of the MRT for different scenarios including band-limited and approximately compactly supported images, we also provide a first proof-of-concept demonstration. To do so, we implement MRT by experimentally folding tomographic measurements available as an open source data set using our custom designed modulo hardware. Our reconstruction clearly shows the advantages of our approach for experimental data. In this way, our MRT based solution paves a path for HDR acquisition in a number of related imaging problems.

cs.IT↗

arXiv2020

On Unlimited Sampling and Reconstruction

Shannon's sampling theorem is one of the cornerstone topics that is well understood and explored, both mathematically and algorithmically. That said, practical realization of this theorem still suffers from a severe bottleneck due to the fundamental assumption that the samples can span an arbitrary range of amplitudes. In practice, the theorem is realized using so-called analog-to-digital converters (ADCs) which clip or saturate whenever the signal amplitude exceeds the maximum recordable ADC voltage thus leading to a significant information loss. In this paper, we develop an alternative paradigm for sensing and recovery, called the Unlimited Sampling Framework. It is based on the observation that when a signal is mapped to an appropriate bounded interval via a modulo operation before entering the ADC, the saturation problem no longer exists, but one rather encounters a different type of information loss due to the modulo operation. Such an alternative setup can be implemented, for example, via so-called folding or self-reset ADCs, as they have been proposed in various contexts in the circuit design literature. The key task that we need to accomplish in order to cope with this new type of information loss is to recover a bandlimited signal from its modulo samples. In this paper we derive conditions when perfect recovery is possible and complement them with a stable recovery algorithm. The sampling density required to guarantee recovery is independent of the maximum recordable ADC voltage and depends on the signal bandwidth only. Our recovery guarantees extend to measurements affected by bounded noise, which includes the case of round-off quantization. Numerical experiments validate our approach. Applications of the unlimited sampling paradigm can be found in a number of fields such as signal processing, communication and imaging.

cs.IT↗

arXiv2020

On the robustness of noise-blind low-rank recovery from rank-one measurements

We prove new results about the robustness of well-known convex noise-blind optimization formulations for the reconstruction of low-rank matrices from underdetermined linear measurements. Our results are applicable for symmetric rank-one measurements as used in a formulation of the phase retrieval problem. We obtain these results by establishing that with high probability rank-one measurement operators defined by i.i.d. Gaussian vectors exhibit the so-called Schatten-1 quotient property, which corresponds to a lower bound for the inradius of their image of the nuclear norm (Schatten-1) unit ball. We complement our analysis by numerical experiments comparing the solutions of noise-blind and noise-aware formulations. These experiments confirm that noise-blind optimization methods exhibit comparable robustness to noise-aware formulations. Keywords: low-rank matrix recovery, phase retrieval, quotient property, noise-blind, robustness, nuclear norm minimization

cs.IT↗

arXiv2020

High order low-bit Sigma-Delta quantization for fusion frames

We construct high order low-bit Sigma-Delta $(ΣΔ)$ quantizers for the vector-valued setting of fusion frames. We prove that these $ΣΔ$ quantizers can be stably implemented to quantize fusion frame measurements on subspaces $W_n$ using $\log_2( {\rm dim}(W_n)+1)$ bits per measurement. Signal reconstruction is performed using a version of Sobolev duals for fusion frames, and numerical experiments are given to validate the overall performance.

cs.IT↗

arXiv2020

On Recovery Guarantees for One-Bit Compressed Sensing on Manifolds

This paper studies the problem of recovering a signal from one-bit compressed sensing measurements under a manifold model; that is, assuming that the signal lies on or near a manifold of low intrinsic dimension. We provide a convex recovery method based on the Geometric Multi-Resolution Analysis and prove recovery guarantees with a near-optimal scaling in the intrinsic manifold dimension. Our method is the first tractable algorithm with such guarantees for this setting. The results are complemented by numerical experiments confirming the validity of our approach.

cs.IT↗