arXiv subjects

Stefan Sommer

Publications and source records attributed to Stefan Sommer.

69 records · Page 4Linked to original sources

arXiv2018

Latent Space Non-Linear Statistics

Given data, deep generative models, such as variational autoencoders (VAE) and generative adversarial networks (GAN), train a lower dimensional latent representation of the data space. The linear Euclidean geometry of data space pulls back to a nonlinear Riemannian geometry on the latent space. The latent space thus provides a low-dimensional nonlinear representation of data and classical linear statistical techniques are no longer applicable. In this paper we show how statistics of data in their latent space representation can be performed using techniques from the field of nonlinear manifold statistics. Nonlinear manifold statistics provide generalizations of Euclidean statistical notions including means, principal component analysis, and maximum likelihood fits of parametric probability distributions. We develop new techniques for maximum likelihood inference in latent space, and adress the computational complexity of using geometric algorithms with high-dimensional data by training a separate neural network to approximate the Riemannian metric and cometric tensor capturing the shape of the learned data manifold.

cs.CV↗

arXiv2017

Differential geometry and stochastic dynamics with deep learning numerics

In this paper, we demonstrate how deterministic and stochastic dynamics on manifolds, as well as differential geometric constructions can be implemented concisely and efficiently using modern computational frameworks that mix symbolic expressions with efficient numerical computations. In particular, we use the symbolic expression and automatic differentiation features of the python library Theano, originally developed for high-performance computations in deep learning. We show how various aspects of differential geometry and Lie group theory, connections, metrics, curvature, left/right invariance, geodesics and parallel transport can be formulated with Theano using the automatic computation of derivatives of any order. We will also show how symbolic stochastic integrators and concepts from non-linear statistics can be formulated and optimized with only a few lines of code. We will then give explicit examples on low-dimensional classical manifolds for visualization and demonstrate how this approach allows both a concise implementation and efficient scaling to high dimensional problems.

cs.CG↗

arXiv2017

Stochastic metamorphosis with template uncertainties

In this paper, we investigate two stochastic perturbations of the metamorphosis equations of image analysis, in the geometrical context of the Euler-Poincaré theory. In the metamorphosis of images, the Lie group of diffeomorphisms deforms a template image that is undergoing its own internal dynamics as it deforms. This type of deformation allows more freedom for image matching and has analogies with complex fluids when the template properties are regarded as order parameters (coset spaces of broken symmetries). The first stochastic perturbation we consider corresponds to uncertainty due to random errors in the reconstruction of the deformation map from its vector field. We also consider a second stochastic perturbation, which compounds the uncertainty in of the deformation map with the uncertainty in the reconstruction of the template position from its velocity field. We apply this general geometric theory to several classical examples, including landmarks, images, and closed curves, and we discuss its use for functional data analysis.

cs.CV↗

arXiv2017

Computational Anatomy in Theano

To model deformation of anatomical shapes, non-linear statistics are required to take into account the non-linear structure of the data space. Computer implementations of non-linear statistics and differential geometry algorithms often lead to long and complex code sequences. The aim of the paper is to show how the Theano framework can be used for simple and concise implementation of complex differential geometry algorithms while being able to handle complex and high-dimensional data structures. We show how the Theano framework meets both of these requirements. The framework provides a symbolic language that allows mathematical equations to be directly translated into Theano code, and it is able to perform both fast CPU and GPU computations on high-dimensional data. We show how different concepts from non-linear statistics and differential geometry can be implemented in Theano, and give examples of the implemented theory visualized on landmark representations of Corpus Callosum shapes.

cs.OH↗

arXiv2017

Bridge Simulation and Metric Estimation on Landmark Manifolds

We present an inference algorithm and connected Monte Carlo based estimation procedures for metric estimation from landmark configurations distributed according to the transition distribution of a Riemannian Brownian motion arising from the Large Deformation Diffeomorphic Metric Mapping (LDDMM) metric. The distribution possesses properties similar to the regular Euclidean normal distribution but its transition density is governed by a high-dimensional PDE with no closed-form solution in the nonlinear case. We show how the density can be numerically approximated by Monte Carlo sampling of conditioned Brownian bridges, and we use this to estimate parameters of the LDDMM kernel and thus the metric structure by maximum likelihood.

cs.CV↗

arXiv2017

A Statistical Model for Simultaneous Template Estimation, Bias Correction, and Registration of 3D Brain Images

Template estimation plays a crucial role in computational anatomy since it provides reference frames for performing statistical analysis of the underlying anatomical population variability. While building models for template estimation, variability in sites and image acquisition protocols need to be accounted for. To account for such variability, we propose a generative template estimation model that makes simultaneous inference of both bias fields in individual images, deformations for image registration, and variance hyperparameters. In contrast, existing maximum a posterori based methods need to rely on either bias-invariant similarity measures or robust image normalization. Results on synthetic and real brain MRI images demonstrate the capability of the model to capture heterogeneity in intensities and provide a reliable template estimation from registration.

cs.CV↗

arXiv2017

Most Likely Separation of Intensity and Warping Effects in Image Registration

This paper introduces a class of mixed-effects models for joint modeling of spatially correlated intensity variation and warping variation in 2D images. Spatially correlated intensity variation and warp variation are modeled as random effects, resulting in a nonlinear mixed-effects model that enables simultaneous estimation of template and model parameters by optimization of the likelihood function. We propose an algorithm for fitting the model which alternates estimation of variance parameters and image registration. This approach avoids the potential estimation bias in the template estimate that arises when treating registration as a preprocessing step. We apply the model to datasets of facial images and 2D brain magnetic resonance images to illustrate the simultaneous estimation and prediction of intensity and warp effects.

cs.CV↗

arXiv2017

Stochastic Development Regression on Non-Linear Manifolds

We introduce a regression model for data on non-linear manifolds. The model describes the relation between a set of manifold valued observations, such as shapes of anatomical objects, and Euclidean explanatory variables. The approach is based on stochastic development of Euclidean diffusion processes to the manifold. Defining the data distribution as the transition distribution of the mapped stochastic process, parameters of the model, the non-linear analogue of design matrix and intercept, are found via maximum likelihood. The model is intrinsically related to the geometry encoded in the connection of the manifold. We propose an estimation procedure which applies the Laplace approximation of the likelihood function. A simulation study of the performance of the model is performed and the model is applied to a real dataset of Corpus Callosum shapes.

cs.OH↗

arXiv2016

A Stochastic Large Deformation Model for Computational Anatomy

In the study of shapes of human organs using computational anatomy, variations are found to arise from inter-subject anatomical differences, disease-specific effects, and measurement noise. This paper introduces a stochastic model for incorporating random variations into the Large Deformation Diffeomorphic Metric Mapping (LDDMM) framework. By accounting for randomness in a particular setup which is crafted to fit the geometrical properties of LDDMM, we formulate the template estimation problem for landmarks with noise and give two methods for efficiently estimating the parameters of the noise fields from a prescribed data set. One method directly approximates the time evolution of the variance of each landmark by a finite set of differential equations, and the other is based on an Expectation-Maximisation algorithm. In the second method, the evaluation of the data likelihood is achieved without registering the landmarks, by applying bridge sampling using a stochastically perturbed version of the large deformation gradient flow algorithm. The method and the estimation algorithms are experimentally validated on synthetic examples and shape data of human corpora callosa.

cs.CV↗

arXiv2016

Anisotropically Weighted and Nonholonomically Constrained Evolutions on Manifolds

We present evolution equations for a family of paths that results from anisotropically weighting curve energies in non-linear statistics of manifold valued data. This situation arises when performing inference on data that have non-trivial covariance and are anisotropic distributed. The family can be interpreted as most probable paths for a driving semi-martingale that through stochastic development is mapped to the manifold. We discuss how the paths are projections of geodesics for a sub-Riemannian metric on the frame bundle of the manifold, and how the curvature of the underlying connection appears in the sub-Riemannian Hamilton-Jacobi equations. Evolution equations for both metric and cometric formulations of the sub-Riemannian metric are derived. We furthermore show how rank-deficient metrics can be mixed with an underlying Riemannian metric, and we relate the paths to geodesics and polynomials in Riemannian geometry. Examples from the family of paths are visualized on embedded surfaces, and we explore computational representations on finite dimensional landmark manifolds with geometry induced from right-invariant metrics on diffeomorphism groups.

math.DG↗

arXiv2016

Modelling Anisotropic Covariance using Stochastic Development and Sub-Riemannian Frame Bundle Geometry

We discuss the geometric foundation behind the use of stochastic processes in the frame bundle of a smooth manifold to build stochastic models with applications in statistical analysis of non-linear data. The transition densities for the projection to the manifold of Brownian motions developed in the frame bundle lead to a family of probability distributions on the manifold. We explain how data mean and covariance can be interpreted as points in the frame bundle or, more precisely, in the bundle of symmetric positive definite 2-tensors analogously to the parameters describing Euclidean normal distributions. We discuss a factorization of the frame bundle projection map through this bundle, the natural sub-Riemannian structure of the frame bundle, the effect of holonomy, and the existence of subbundles where the Hormander condition is satisfied such that the Brownian motions have smooth transition densities. We identify the most probable paths for the underlying Euclidean Brownian motion and discuss small time asymptotics of the transition densities on the manifold. The geometric setup yields an intrinsic approach to the estimation of mean and covariance in non-linear spaces.

math.DG↗

arXiv2015

Higher-order Spatial Accuracy in Diffeomorphic Image Registration

We discretize a cost functional for image registration problems by deriving Taylor expansions for the matching term. Minima of the discretized cost functionals can be computed with no spatial discretization error, and the optimal solutions are equivalent to minimal energy curves in the space of $k$-jets. We show that the solutions convergence to optimal solutions of the original cost functional as the number of particles increases with a convergence rate of $O(h^{d+k})$ where $h$ is a resolution parameter. The effect of this approach over traditional particle methods is illustrated on synthetic examples and real images.

cs.CV↗

arXiv2014

Symmetry in Image Registration and Deformation Modeling

We survey the role of symmetry in diffeomorphic registration of landmarks, curves, surfaces, images and higher-order data. The infinite dimensional problem of finding correspondences between objects can for a range of concrete data types be reduced resulting in compact representations of shape and spatial structure. This reduction is possible because the available data is incomplete in encoding the full deformation model. Using reduction by symmetry, we describe the reduced models in a common theoretical framework that draws on links between the registration problem and geometric mechanics. Symmetry also arises in reduction to the Lie algebra using particle relabeling symmetry allowing the equations of motion to be written purely in terms of Eulerian velocity field. Reduction by symmetry has recently been applied for jet-matching and higher-order discrete approximations of the image matching problem. We outline these constructions and further cases where reduction by symmetry promises new approaches to registration of complex data types.

cs.CV↗

arXiv2013

Optimization over Geodesics for Exact Principal Geodesic Analysis

In fields ranging from computer vision to signal processing and statistics, increasing computational power allows a move from classical linear models to models that incorporate non-linear phenomena. This shift has created interest in computational aspects of differential geometry, and solving optimization problems that incorporate non-linear geometry constitutes an important computational task. In this paper, we develop methods for numerically solving optimization problems over spaces of geodesics using numerical integration of Jacobi fields and second order derivatives of geodesic families. As an important application of this optimization strategy, we compute exact Principal Geodesic Analysis (PGA), a non-linear version of the PCA dimensionality reduction procedure. By applying the exact PGA algorithm to synthetic data, we exemplify the differences between the linearized and exact algorithms caused by the non-linear geometry. In addition, we use the numerically integrated Jacobi fields to determine sectional curvatures and provide upper bounds for injectivity radii.

cs.CG↗

arXiv2012

Higher-Order Momentum Distributions and Locally Affine LDDMM Registration

To achieve sparse parametrizations that allows intuitive analysis, we aim to represent deformation with a basis containing interpretable elements, and we wish to use elements that have the description capacity to represent the deformation compactly. To accomplish this, we introduce in this paper higher-order momentum distributions in the LDDMM registration framework. While the zeroth order moments previously used in LDDMM only describe local displacement, the first-order momenta that are proposed here represent a basis that allows local description of affine transformations and subsequent compact description of non-translational movement in a globally non-rigid deformation. The resulting representation contains directly interpretable information from both mathematical and modeling perspectives. We develop the mathematical construction of the registration framework with higher-order momenta, we show the implications for sparse image registration and deformation description, and we provide examples of how the parametrization enables registration with a very low number of parameters. The capacity and interpretability of the parametrization using higher-order momenta lead to natural modeling of articulated movement, and the method promises to be useful for quantifying ventricle expansion and progressing atrophy during Alzheimer's disease.

cs.CV↗