arXiv subjects

Sebastian Neumayer

Publications and source records attributed to Sebastian Neumayer.

At least 19 recordsLinked to original sources

arXiv2026

Data-driven Regularization for Inverse Problems

In many applications, we are interested in studying an object of interest $x$. However, its properties are often not directly measurable. Instead, we only observe $y = F(x)$, where the (known) function $F$ is referred to as the forward mapping. To infer about $x$ from the observation $y$, we seek an (approximate) inverse mapping $x \approx F^{-1}(y)$. This process is known as solving the inverse problem. In practice, the inverse mapping $F^{-1}$ may be ill-defined or discontinuous, which forces us to choose a certain solution from many possible ones based on prior assumptions on $x$. Such assumptions about $x$ are typically incorporated into the inversion process through regularization. Therefore, the first part of this course focuses on the classical regularization theory of inverse problems. In the second part, we transition to algorithmic approaches for solving inverse problems. The foundation for this lies in the variational formulation of inverse problems, where reconstructions are obtained by minimizing specific functionals. Here, we will explore key optimization principles, discuss their practical implementation, and examine how they contribute to robust reconstructions. Today, classical methods are often modified to incorporate data-driven machine learning techniques, which improve the empirical performance. However, this often comes at the cost of explainability or convergence guarantees. Such properties are critical, for example, in medical applications, where one wants to be confident that reconstructed abnormalities are of physical nature and not phantoms resulting from the method. For these reasons, we discuss a number of machine learning methods for inverse problems, such as unrolling techniques, learned regularizers and plug-and-play methods, and explore their performance guarantees.

math.NA↗

arXiv2026

Random nodes with positive weights: Richter-Tchakaloff rules and optimal positive quadrature

We show that independent random points support exact quadrature rules with positive weights. For any $n$-dimensional space $V$ of integrable functions a constant multiple of $n\log n$ points sampled from a suitable density yields, with high probability, positive weights which integrate the functions in the space exactly. The weights stay close to the natural importance sampling weights according to the Lewis density with respect to $V$. They are the result of a weight adaptation procedure which requires an explicit $L_1$-norm discretization result that is proved on the way. As an application, we study numerical integration in reproducing kernel Hilbert spaces. Combining the random rules with a subsampling step, we show that positive quadrature rules with constantly oversampled nodes achieve the best possible worst-case error bound in terms of the singular numbers of the embedding into $L_2$.

math.NA↗

arXiv2026

High-dimensional sparse recovery from function samples - Decoders, guarantees and instance optimality

We investigate the reconstruction of multivariate functions from samples using sparse recovery techniques. For Square Root Lasso, Orthogonal Matching Pursuit, and Compressive Sampling Matching Pursuit, we demonstrate both theoretically and empirically that they allow us to recover functions from a small number of random samples. In contrast to Basis Pursuit Denoising, the deployed decoders only require a search space $V_J$ spanned by dictionary elements indexed by $J$ and a sparsity parameter $n$ to guarantee an $L_2$-approximation error decaying no worse than a best $n$-term approximation error and the truncation error with respect to the search space $V_J$ and the uniform norm. We show that this happens simultaneously for all admissible functions if the number of samples scales as $n\log^2 n\log |J|$, coming from known bounds for the RIP for matrices built upon bounded orthonormal systems. As a consequence, we obtain bounds for sampling widths in function classes. In addition, we establish lower bounds on the required sample complexity, which show that the log-factor in $\vert J \vert$ is indeed necessary to obtain such {\em instance-optimal} error guarantees. Finally, we conduct several numerical experiments to show that our theoretical bounds are reasonable and compare the discussed decoders in practice.

math.NA↗

arXiv2026

Weakly Convex Ridge Regularization for 3D Non-Cartesian MRI Reconstruction

While highly accelerated non-Cartesian acquisition protocols significantly reduce scan time, they often entail long reconstruction delays. Deep learning based reconstruction methods can alleviate this, but often lack stability and robustness to distribution shifts. As an alternative, we train a rotation invariant weakly convex ridge regularizer (WCRR). The resulting variational reconstruction approach is benchmarked against state of the art methods on retrospectively simulated data and (out of distribution) on prospective GoLF SPARKLING and CAIPIRINHA acquisitions. Our approach consistently outperforms widely used baselines and achieves performance comparable to Plug and Play reconstruction with a state of the art 3D DRUNet denoiser, while offering substantially improved computational efficiency and robustness to acquisition changes. In summary, WCRR unifies the strengths of principled variational methods and modern deep learning based approaches.

cs.CV↗

arXiv2026

Quasi Linear Kernel Attention with Infinite Capacity

The evaluation cost of transformers with softmax attention scales quadratically with sequence length. Kernel attention addresses this by replacing softmax with a more general kernel function. In this paper, we aim to identify kernels that retain the expressivity of attention while enabling quasi linear computation. To quantify expressivity, we introduce a capacity for each kernel, measuring the maximum sequence length for which the attention matrix can approximate the identity. A higher capacity thus indicates greater expressivity. We show that expressive kernels like softmax, Gauss, and Laplace have infinite capacity. In contrast, common quasi linear kernels, such as those derived from finite dimensional feature maps, exhibit finite capacity. As a solution, we propose additive kernels constructed from univariate spline and polynomial exponential kernels. We prove that these maintain infinite capacity while allowing quasi linear computation via sorting. Finally, we implement additive sorting kernels efficiently and benchmark them against modern softmax backends, demonstrating advantages for long sequences.

cs.LG↗

arXiv2026

CLEAR: Complex Learned Explicit Analytical Regularization for Ultra-Accelerated 4D Flow CMR Reconstruction

While compressed-sensing regularizers enable interpretable reconstruction of 4D Flow CMR through transparent variational objectives, their hand-crafted nature is too restrictive under high acceleration. State-of-the-art learning-based approaches mitigate this, but typically encode regularization implicitly through unrolled network modules, which limits their interpretability. To address this limitation, we propose CLEAR, designed to combine the interpretability of compressed sensing with the flexibility of learned models. To the best of our knowledge, it is the first learned regularizer for a 4D reconstruction task. In the ultra-accelerated $10\times$--$50\times$ regime of the CMRx4DFlow2026 challenge, CLEAR outperforms compressed sensing locally low-rank (LLR) and the popular variational network FlowVN, while using less than 10k parameters and preserving an interpretable regularization structure.

cs.CV↗

arXiv2026

A sharp bound for sampling widths in the uniform norm: kernel $D$-optimal designs and oversampling

For every reproducing kernel Hilbert space $\mathcal{H}_K$ with bounded kernel $K$, we prove that the linear sampling widths $g_m^{\text{lin}}$ and Gelfand widths $c_n$ in the uniform norm satisfy $$ g_m^{\text{lin}}(B_{\mathcal H_K})_\infty \leq \frac{m+1}{m-n+1}\, c_n(B_{\mathcal H_K})_\infty\quad , \quad m\ge n. $$ In a certain sense, the leading factor $(m+1)/(m-n+1)$ on the right-hand side is sharp. We transfer the problem into the selection of a maximal volume subset of kernel translates (kernel $D$-optimal design). Apart from basic linear algebra, our proof employs low rank approximation techniques and a version of the Eckart-Young-Mirsky theorem. The same argument gives corresponding bounds for the selection of a reduced basis in Hilbert spaces.

math.FA↗

arXiv2026

Fast Gauss Sums via Flash Attention

Gaussian kernel sums are the computational core of maximum mean discrepancies (MMDs), kernel gradient flows, Stein variational gradient descent (SVGD), and many other kernel methods. At the same time, softmax attention has received an extraordinary amount of hardware-aware code engineering, culminating in flash attention. We show that Gauss kernel sums with arbitrary, signed weights can be evaluated via flash attention: two small input augmentations turn the normalized softmax reduction into the unnormalized Gauss sum, without writing a single line of custom GPU code. For feature dimension D>8 in fp16, this approach beats compiled PyTorch code as well as PyKeOps kernels (often significantly) in speed, memory-overhead and accuracy. Indeed, its memory scaling remains linear.

cs.LG↗

arXiv2026

Greedy sampling designs via reduced basis methods: optimal recovery in the uniform norm

We study optimal sampling recovery in reproducing kernel Hilbert spaces (RKHS) in the uniform norm. For every RKHS with bounded kernel, we establish new comparisons between linear sampling widths and Gelfand widths that overcome the known square-root gap, without requiring a measure or a Christoffel-type condition. Our bounds rely on nested sampling designs obtained by kernel interpolation at (weak) P-greedy points. Under additional (polynomial) decay assumptions the decay rate of the Gelfand widths directly transfers to the sampling widths. With either a logarithmic oversampling or passing to the square root of the Gelfand widths we obtain a direct comparison (requiring no decay assumption) between them. This is particularly effective for super-polynomial decay, such as in Paley-Wiener spaces. Our results follow from representations of both widths in terms of kernel translates and yield, in the opposite direction, a new existence result for a sharp reduced basis selection. Numerical experiments for Legendre, mixed-Sobolev, and Paley-Wiener kernels illustrate our findings.

math.NA↗

arXiv2026

Learning solution operators of PDEs with sparse approximation methods

We investigate the approximation of solution operators for partial differential equations (PDEs) using sparse high-dimensional techniques. Building on a dimension-incremental framework, we combine product basis expansions with sparse recovery methods, specifically orthogonal matching pursuit (OMP), to substantially reduce the required sample size compared with a previously considered cubature-based approach. We evaluate the resulting method numerically on several examples, comparing it against both cubature-based sparse approximation and Fourier neural operators in terms of accuracy, runtime, and sample size. The experiments show that our approach considerably reduces the number of required PDE solves relative to its predecessor while maintaining competitive accuracy, particularly when the solution admits a sparse representation in the chosen basis. Furthermore, the recovered sparse index sets yield interpretable insights into the relevant variables and parameter interactions.

math.NA↗

arXiv2026

A Stability Benchmark of Generative Regularizers for Inverse Problems

Generative (diffusion) priors demonstrate remarkable performance in addressing inverse problems in imaging. Yet, for scientific and medical imaging, it is crucial that reconstruction techniques remain stable and reliable under imperfect settings. Typical definitions of stability encompass the notion of ''convergent regularization'', robustness to out-of-distribution data, and to inaccuracies in the forward operator or noise model. We evaluate these properties numerically. Furthermore, we benchmark generative approaches against modern optimization-based methods inspired by the widely used variational techniques. Our results give insights for which settings and applications generative priors can deliver state-of-the-art reconstructions, and on those in which they fall short or may even be problematic.

eess.IV↗

arXiv2026

A remark on an error analysis for classical and learned Tikhonov regularization schemes

This paper presents an error analysis of classical and learned Tikhonov regularization schemes for inverse problems. We first demonstrate, both theoretically and numerically, that using a fixed regularization parameter across varying noise levels-which is a common miss-specification in practice-has only a mild impact on the reconstruction error. As a special case, we then investigate scenarios where the true data resides in an unknown finite-dimensional subspace. Here, our results lead to an empirically supported strategy for estimating the unknown dimension based on numerical experiments. Finally, we examine the approach that motivated this study: a method where a sparsity-promoting term is learned from denoising tasks and subsequently applied to general inverse problems via a simple heuristic parameter selection. The corresponding error analysis is initially developed using classical concepts and subsequently refined through a more detailed investigation of the discretized setting.

math.NA↗

arXiv2026

Learning Regularization Functionals for Inverse Problems: A Comparative Study

In recent years, a variety of learned regularization frameworks for solving inverse problems in imaging have emerged. These offer flexible modeling together with mathematical insights. The proposed methods differ in their architectural design and training strategies, making direct comparison challenging due to non-modular implementations. We address this gap by collecting and unifying the available code into a common framework. This unified view allows us to systematically compare the approaches and highlight their strengths and limitations, providing valuable insights into their future potential. We also provide concise descriptions of each method, complemented by practical guidelines.

cs.LG↗

arXiv2025

Numerical Methods for Kernel Slicing

Kernels are key in machine learning for modeling interactions. Unfortunately, brute-force computation of the related kernel sums scales quadratically with the number of samples. Recent Fourier-slicing methods lead to an improved linear complexity, provided that the kernel can be sliced and its Fourier coefficients are known. To obtain these coefficients, we view the slicing relation as an inverse problem and present two algorithms for their recovery. Extensive numerical experiments demonstrate the speed and accuracy of our methods.

math.NA↗

arXiv2025

Generative Feature Training of Thin 2-Layer Networks

We consider the approximation of functions by 2-layer neural networks with a small number of hidden weights based on the squared loss and small datasets. Due to the highly non-convex energy landscape, gradient-based training often suffers from local minima. As a remedy, we initialize the hidden weights with samples from a learned proposal distribution, which we parameterize as a deep generative model. To train this model, we exploit the fact that with fixed hidden weights, the optimal output weights solve a linear equation. After learning the generative model, we refine the sampled weights with a gradient-based post-processing in the latent space. Here, we also include a regularization scheme to counteract potential noise. Finally, we demonstrate the effectiveness of our approach by numerical examples.

cs.LG↗

arXiv2025

DeepInverse: A Python package for solving imaging inverse problems with deep learning

DeepInverse is an open-source PyTorch-based library for solving imaging inverse problems. The library covers all crucial steps in image reconstruction from the efficient implementation of forward operators (e.g., optics, MRI, tomography), to the definition and resolution of variational problems and the design and training of advanced neural network architectures. In this paper, we describe the main functionality of the library and discuss the main design choices.

eess.IV↗

arXiv2025

Wasserstein Gradient Flows for Moreau Envelopes of f-Divergences in Reproducing Kernel Hilbert Spaces

Commonly used $f$-divergences of measures, e.g., the Kullback-Leibler divergence, are subject to limitations regarding the support of the involved measures. A remedy is regularizing the $f$-divergence by a squared maximum mean discrepancy (MMD) associated with a characteristic kernel $K$. We use the kernel mean embedding to show that this regularization can be rewritten as the Moreau envelope of some function on the associated reproducing kernel Hilbert space. Then, we exploit well-known results on Moreau envelopes in Hilbert spaces to analyze the MMD-regularized $f$-divergences, particularly their gradients. Subsequently, we use our findings to analyze Wasserstein gradient flows of MMD-regularized $f$-divergences. We provide proof-of-the-concept numerical examples for flows starting from empirical measures. Here, we cover $f$-divergences with infinite and finite recession constants. Lastly, we extend our results to the tight variational formulation of $f$-divergences and numerically compare the resulting flows.

stat.ML↗

arXiv2025

Learning of Patch-Based Smooth-Plus-Sparse Models for Image Reconstruction

We aim at the solution of inverse problems in imaging, by combining a penalized sparse representation of image patches with an unconstrained smooth one. This allows for a straightforward interpretation of the reconstruction. We formulate the optimization as a bilevel problem. The inner problem deploys classical algorithms while the outer problem optimizes the dictionary and the regularizer parameters through supervised learning. The process is carried out via implicit differentiation and gradient-based optimization. We evaluate our method for denoising, super-resolution, and compressed-sensing magnetic-resonance imaging. We compare it to other classical models as well as deep-learning-based methods and show that it always outperforms the former and also the latter in some instances.

eess.IV↗