arXiv subjects

Gabriele Steidl

Publications and source records attributed to Gabriele Steidl.

At least 73 records · Page 4Linked to original sources

arXiv2022

On Assignment Problems Related to Gromov-Wasserstein Distances on the Real Line

Let $x_1 < \dots < x_n$ and $y_1 < \dots < y_n$, $n \in \mathbb N$, be real numbers. We show by an example that the assignment problem $$ \max_{σ\in S_n} F_σ(x,y) := \frac12 \sum_{i,k=1}^n |x_i - x_k|^α\, |y_{σ(i)} - y_{σ(k)}|^α, \quad α>0, $$ is in general neither solved by the identical permutation (id) nor the anti-identical permutation (a-id) if $n > 2 +2^α$. Indeed the above maximum can be, depending on the number of points, arbitrary far away from $F_\text{id}(x,y)$ and $F_\text{a-id}(x,y)$. The motivation to deal with such assignment problems came from their relation to Gromov-Wasserstein divergences which have recently attained a lot of attention.

math.NA↗

arXiv2022

Stochastic Normalizing Flows for Inverse Problems: a Markov Chains Viewpoint

To overcome topological constraints and improve the expressiveness of normalizing flow architectures, Wu, Köhler and Noé introduced stochastic normalizing flows which combine deterministic, learnable flow transformations with stochastic sampling methods. In this paper, we consider stochastic normalizing flows from a Markov chain point of view. In particular, we replace transition densities by general Markov kernels and establish proofs via Radon-Nikodym derivatives which allows to incorporate distributions without densities in a sound way. Further, we generalize the results for sampling from posterior distributions as required in inverse problems. The performance of the proposed conditional stochastic normalizing flow is demonstrated by numerical examples.

cs.LG↗

arXiv2021

Sparse Mixture Models inspired by ANOVA Decompositions

Inspired by the analysis of variance (ANOVA) decomposition of functions we propose a Gaussian-Uniform mixture model on the high-dimensional torus which relies on the assumption that the function we wish to approximate can be well explained by limited variable interactions. We consider three approaches, namely wrapped Gaussians, diagonal wrapped Gaussians and products of von Mises distributions. The sparsity of the mixture model is ensured by the fact that its summands are products of Gaussian-like density functions acting on low dimensional spaces and uniform probability densities defined on the remaining directions. To learn such a sparse mixture model from given samples, we propose an objective function consisting of the negative log-likelihood function of the mixture model and a regularizer that penalizes the number of its summands. For minimizing this functional we combine the Expectation Maximization algorithm with a proximal step that takes the regularizer into account. To decide which summands of the mixture model are important, we apply a Kolmogorov-Smirnov test. Numerical examples demonstrate the performance of our approach.

math.ST↗

arXiv2021

An Image Registration Model in Electron Backscatter Diffraction

Recently, variational methods were successfully applied for computing the optical flow in gray and RGB-valued image sequences. A crucial assumption in these models is that pixel-values do not change under transformations. Nowadays, modern image acquisition techniques such as electron backscatter tomography (EBSD), which is used in material sciences, can capture images with values in nonlinear spaces. Here, the image values belong to the quotient space $\text{SO}(3)/ \mathcal S$ of the special orthogonal group modulo the discrete symmetry group of the crystal. For such data, the assumption that pixel-values remain unchanged under transformations appears to be no longer valid. Hence, we propose a variational model for determining the optical flow in $\text{SO}(3)/\mathcal S$-valued image sequences, taking the dependence of pixel-values on the transformation into account. More precisely, the data is transformed according to the rotation part in the polar decomposition of the Jacobian of the transformation. To model non-smooth transformations without obtaining so-called staircasing effects, we propose to use a total generalized variation like prior. Then, we prove existence of a minimizer for our model and explain how it can be discretized and minimized by a primal-dual algorithm. Numerical examples illustrate the performance of our method.

math.OC↗

arXiv2021

On the Dynamical System of Principal Curves in $\mathbb R^d$

Principal curves are natural generalizations of principal lines arising as first principal components in the Principal Component Analysis. They can be characterized from a stochastic point of view as so-called self-consistent curves based on the conditional expectation and from the variational-calculus point of view as saddle points of the expected difference of a random variable and its projection onto some curve, where the current curve acts as argument of the energy functional. Beyond that, Duchamp and Stützle (1993,1996) showed that planar curves can by computed as solutions of a system of ordinary differential equations. The aim of this paper is to generalize this characterization of principal curves to $\mathbb R^d$ with $d \ge 3$. Having derived such a dynamical system, we provide several examples for principal curves related to uniform distribution on certain domains in $\mathbb R^3$.

math.DS↗

arXiv2021

PCA Reduced Gaussian Mixture Models with Applications in Superresolution

Despite the rapid development of computational hardware, the treatment of large and high dimensional data sets is still a challenging problem. This paper provides a twofold contribution to the topic. First, we propose a Gaussian Mixture Model in conjunction with a reduction of the dimensionality of the data in each component of the model by principal component analysis, called PCA-GMM. To learn the (low dimensional) parameters of the mixture model we propose an EM algorithm whose M-step requires the solution of constrained optimization problems. Fortunately, these constrained problems do not depend on the usually large number of samples and can be solved efficiently by an (inertial) proximal alternating linearized minimization algorithm. Second, we apply our PCA-GMM for the superresolution of 2D and 3D material images based on the approach of Sandeep and Jacob. Numerical results confirm the moderate influence of the dimensionality reduction on the overall superresolution result.

stat.ML↗

arXiv2021

Transfer Operators from Optimal Transport Plans for Coherent Set Detection

The topic of this study lies in the intersection of two fields. One is related with analyzing transport phenomena in complicated flows.For this purpose, we use so-called coherent sets: non-dispersing, possibly moving regions in the flow's domain. The other is concerned with reconstructing a flow field from observing its action on a measure, which we address by optimal transport. We show that the framework of optimal transport is well suited for delivering the formal requirements on which a coherent-set analysis can be based on. The necessary noise-robustness requirement of coherence can be matched by the computationally efficient concept of unbalanced regularized optimal transport. Moreover, the applied regularization can be interpreted as an optimal way of retrieving the full dynamics given the extremely restricted information of an initial and a final distribution of particles moving according to Brownian motion.

math.NA↗

arXiv2021

Fourier reconstruction for diffraction tomography of an object rotated into arbitrary orientations

In this paper, we study the mathematical imaging problem of optical diffraction tomography (ODT) for the scenario of a microscopic rigid particle rotating in a trap created, for instance, by acoustic or optical forces. Under the influence of the inhomogeneous forces the particle carries out a time-dependent smooth, but complicated motion described by a set of affine transformations. The rotation of the particle enables one to record optical images from a wide range of angles, which largely eliminates the "missing cone problem" in optics. This advantage, however, comes at the price that the rotation axis in this scenario is not fixed, but continuously undergoes some variations, and that the rotation angles are not equally spaced, which is in contrast to standard tomographic reconstruction assumptions. In the present work, we assume that the time-dependent motion parameters are known, and that the particle's scattering potential is compatible with making the first order Born or Rytov approximation. We prove a Fourier diffraction theorem and derive novel backprojection formulae for the reconstruction of the scattering potential, which depends on the refractive index distribution inside the object, taking its complicated motion into account. This provides the basis for solving the ODT problem with an efficient non-uniform discrete Fourier transform.

math.NA↗

arXiv2021

On $α$-Firmly Nonexpansive Operators in $r$-Uniformly Convex Spaces

We introduce the class of $α$-firmly nonexpansive and quasi $α$-firmly nonexpansive operators on $r$-uniformly convex Banach spaces. This extends the existing notion from Hilbert spaces, where $α$-firmly nonexpansive operators coincide with so-called $α$-averaged operators. For our more general setting, we show that $α$-averaged operators form a subset of $α$-firmly nonexpansive operators. We develop some basic calculus rules for (quasi) $α$-firmly nonexpansive operators. In particular, we show that their compositions and convex combinations are again (quasi) $α$-firmly nonexpansive. Moreover, we will see that quasi $α$-firmly nonexpansive operators enjoy the asymptotic regularity property. Then, based on Browder's demiclosedness principle, we prove for $r$-uniformly convex Banach spaces that the weak cluster points of the iterates $x_{n+1}:=Tx_{n}$ belong to the fixed point set $\text{Fix} T$ whenever the operator $T$ is nonexpansive and quasi $α$-firmly. If additionally the space has a Fréchet differentiable norm or satisfies Opial's property then these iterates converge weakly to some element in $\text{Fix} T$. Further, the projections $P_{\text{Fix} T}x_n$ converge strongly to this weak limit point. Finally, we give three illustrative examples, where our theory can be applied, namely from infinite dimensional neural networks, semigroup theory, and contractive projections in $L_p$, $p \in (1,\infty) \backslash \{2\}$ spaces on probability measure spaces.

math.FA↗

arXiv2021

On approximate operator representations of sequences in Banach spaces

Generalizing results by Halperin et al., Grivaux recently showed that any linearly independent sequence $\{f_k\}_{k=1}^\infty$ in a separable Banach space $X$ can be represented as a suborbit $\{T^{α(k)}φ\}_{k=1}^\infty$ of some bounded operator $T: X\to X.$ In general, the operator $T$ and the powers $α(k)$ are not known explicitly. In this paper we consider approximate representations $\{f_k\}_{k=1}^\infty \approx \{T^{α(k)}φ\}_{k=1}^\infty$ of certain types of sequences $\{f_k\}_{k=1}^\infty.$ In contrast to the results in the literature we are able to be very explicit about the operator $T$ and suitable powers $α(k),$ and we do not need to assume that the sequences are linearly independent. The exact meaning of approximation is defined in a way such that $\{T^{α(k)}φ\}_{k=1}^\infty$ keeps essential features of $\{f_k\}_{k=1}^\infty,$ e.g., in the setting of atomic decompositions and Banach frames. We will present two different approaches. The first approach is universal, in the sense that it applies in general Banach spaces; the technical conditions are typically easy to verify in sequence spaces, but are more complicated in function spaces. For this reason we present a second approach, directly tailored to the setting of Banach function spaces. A number of examples prove that the results apply in arbitrary weighted $\ell^p$-spaces and $L^p$-spaces.

math.FA↗

arXiv2021

Invertible Neural Networks versus MCMC for Posterior Reconstruction in Grazing Incidence X-Ray Fluorescence

Grazing incidence X-ray fluorescence is a non-destructive technique for analyzing the geometry and compositional parameters of nanostructures appearing e.g. in computer chips. In this paper, we propose to reconstruct the posterior parameter distribution given a noisy measurement generated by the forward model by an appropriately learned invertible neural network. This network resembles the transport map from a reference distribution to the posterior. We demonstrate by numerical comparisons that our method can compete with established Markov Chain Monte Carlo approaches, while being more efficient and flexible in applications.

cs.LG↗

arXiv2021

Super-Resolution for Doubly-Dispersive Channel Estimation

In this work we consider the problem of identification and reconstruction of doubly-dispersive channel operators which are given by finite linear combinations of time-frequency shifts. Such operators arise as time-varying linear systems for example in radar and wireless communications. In particular, for information transmission in highly non-stationary environments the channel needs to be estimated quickly with identification signals of short duration and for vehicular application simultaneous high-resolution radar is desired as well. We consider the time-continuous setting and prove an exact resampling reformulation of the involved channel operator when applied to a trigonometric polynomial as identifier in terms of sparse linear combinations of real-valued atoms. Motivated by recent works of Heckel et al. we present an exact approach for off-the-grid superresolution which allows to perform the identification with realizable signals having compact support. Then we show how an alternating descent conditional gradient algorithm can be adapted to solve the reformulated problem. Numerical examples demonstrate the performance of this algorithm, in particular in comparison with a simple adaptive grid refinement strategy and an orthogonal matching pursuit algorithm.

cs.IT↗

arXiv2021

Curve Based Approximation of Measures on Manifolds by Discrepancy Minimization

The approximation of probability measures on compact metric spaces and in particular on Riemannian manifoldsby atomic or empirical ones is a classical task in approximation and complexity theory with a wide range of applications. Instead of point measures we are concerned with the approximation by measures supported on Lipschitz curves. Special attention is paid to push-forward measures of Lebesgue measures on the interval by such curves. Using the discrepancy as distance between measures, we prove optimal approximation rates in terms of Lipschitz constants of curves. Having established the theoretical convergence rates, we are interested in the numerical minimization of the discrepancy between a given probability measure and the set of push-forward measures of Lebesgue measures on the interval by Lipschitz curves. We present numerical examples for measures on the 2- and 3-dimensional torus, the 2-sphere, the rotation group on $\mathbb R^3$ and the Grassmannian of all 2-dimensional linear subspaces of $\mathbb{R}^4$. Our algorithm of choice is a conjugate gradient method on these manifolds which incorporates second-oder information. For efficiently computing the gradients and the Hessians within the algorithm, we approximate the given measures by truncated Fourier series and use fast Fourier transform techniques on these manifolds.

math.OC↗

arXiv2020

Inertial Stochastic PALM (iSPALM) and Applications in Machine Learning

Inertial algorithms for minimizing nonsmooth and nonconvex functions as the inertial proximal alternating linearized minimization algorithm (iPALM) have demonstrated their superiority with respect to computation time over their non inertial variants. In many problems in imaging and machine learning, the objective functions have a special form involving huge data which encourage the application of stochastic algorithms. While algorithms based on stochastic gradient descent are still used in the majority of applications, recently also stochastic algorithms for minimizing nonsmooth and nonconvex functions were proposed. In this paper, we derive an inertial variant of a stochastic PALM algorithm with variance-reduced gradient estimator, called iSPALM, and prove linear convergence of the algorithm under certain assumptions. Our inertial approach can be seen as generalization of momentum methods widely used to speed up and stabilize optimization algorithms, in particular in machine learning, to nonsmooth problems. Numerical experiments for learning the weights of a so-called proximal neural network and the parameters of Student-t mixture models show that our new algorithm outperforms both stochastic PALM and its deterministic counterparts.

math.OC↗

arXiv2020

Convolutional Proximal Neural Networks and Plug-and-Play Algorithms

In this paper, we introduce convolutional proximal neural networks (cPNNs), which are by construction averaged operators. For filters of full length, we propose a stochastic gradient descent algorithm on a submanifold of the Stiefel manifold to train cPNNs. In case of filters with limited length, we design algorithms for minimizing functionals that approximate the orthogonality constraints imposed on the operators by penalizing the least squares distance to the identity operator. Then, we investigate how scaled cPNNs with a prescribed Lipschitz constant can be used for denoising signals and images, where the achieved quality depends on the Lipschitz constant. Finally, we apply cPNN based denoisers within a Plug-and-Play (PnP) framework and provide convergence results for the corresponding PnP forward-backward splitting algorithm based on an oracle construction.

math.OC↗

arXiv2020

From Optimal Transport to Discrepancy

A common way to quantify the ,,distance'' between measures is via their discrepancy, also known as maximum mean discrepancy (MMD). Discrepancies are related to Sinkhorn divergences $S_\varepsilon$ with appropriate cost functions as $\varepsilon \to \infty$. In the opposite direction, if $\varepsilon \to 0$, Sinkhorn divergences approach another important distance between measures, namely the Wasserstein distance or more generally optimal transport ,,distance''. In this chapter, we investigate the limiting process for arbitrary measures on compact sets and Lipschitz continuous cost functions. In particular, we are interested in the behavior of the corresponding optimal potentials $\hat φ_\varepsilon$, $\hat ψ_\varepsilon$ and $\hat φ_K$ appearing in the dual formulation of the Sinkhorn divergences and discrepancies, respectively. While part of the results are known, we provide rigorous proofs for some relations which we have not found in this generality in the literature. Finally, we demonstrate the limiting process by numerical examples and show the behavior of the distances when used for the approximation of measures by point measures in a process called dithering.

math.OC↗

arXiv2020

Robust PCA via Regularized REAPER with a Matrix-Free Proximal Algorithm

Principal component analysis (PCA) is known to be sensitive to outliers, so that various robust PCA variants were proposed in the literature. A recent model, called REAPER, aims to find the principal components by solving a convex optimization problem. Usually the number of principal components must be determined in advance and the minimization is performed over symmetric positive semi-definite matrices having the size of the data, although the number of principal components is substantially smaller. This prohibits its use if the dimension of the data is large which is often the case in image processing. In this paper, we propose a regularized version of REAPER which enforces the sparsity of the number of principal components by penalizing the nuclear norm of the corresponding orthogonal projector. This has the advantage that only an upper bound on the number of principal components is required. Our second contribution is a matrix-free algorithm to find a minimizer of the regularized REAPER which is also suited for high dimensional data. The algorithm couples a primal-dual minimization approach with a thick-restarted Lanczos process. As a side result, we discuss the topic of the bias in robust PCA. Numerical examples demonstrate the performance of our algorithm.

math.NA↗

arXiv2020

Parseval Proximal Neural Networks

The aim of this paper is twofold. First, we show that a certain concatenation of a proximity operator with an affine operator is again a proximity operator on a suitable Hilbert space. Second, we use our findings to establish so-called proximal neural networks (PNNs) and stable tight frame proximal neural networks. Let $\mathcal H$ and $\mathcal K$ be real Hilbert spaces, $b\in\mathcal K$ and $T\in\mathcal{B}(\mathcal H,\mathcal K)$ have closed range and Moore-Penrose inverse $T^\dagger$. Based on the well-known characterization of proximity operators by Moreau, we prove that for any proximity operator $\text{Prox}\colon\mathcal K\to\mathcal K$ the operator $T^\dagger\,\text{Prox} (T\cdot +b)$ is a proximity operator on $\mathcal H$ equipped with a suitable norm. In particular, it follows for the frequently applied soft shrinkage operator $\text{Prox} = S_λ\colon\ell_2 \rightarrow\ell_2$ and any frame analysis operator $T\colon\mathcal H\to\ell_2$ that the frame shrinkage operator $T^\dagger\, S_λ\,T$ is a proximity operator on a suitable Hilbert space. The concatenation of proximity operators on $\mathbb R^d$ equipped with different norms establishes a PNN. If the network arises from tight frame analysis or synthesis operators, then it forms an averaged operator. Hence, it has Lipschitz constant 1 and belongs to the class of so-called Lipschitz networks, which were recently applied to defend against adversarial attacks. Moreover, due to its averaging property, PNNs can be used within so-called Plug-and-Play algorithms with convergence guarantee. In case of Parseval frames, we call the networks Parseval proximal neural networks (PPNNs). Then, the involved linear operators are in a Stiefel manifold and corresponding minimization methods can be applied for training. Finally, some proof-of-the concept examples demonstrate the performance of PPNNs.

math.NA↗