arXiv subjects

Michael Breuß

Publications and source records attributed to Michael Breuß.

33 records · Page 2Linked to original sources

arXiv2022

Adaptive neural domain refinement for solving time-dependent differential equations

A classic approach for solving differential equations with neural networks builds upon neural forms, which employ the differential equation with a discretisation of the solution domain. Making use of neural forms for time-dependent differential equations, one can apply the recently developed method of domain fragmentation. That is, the domain may be split into several subdomains, on which the optimisation problem is solved. In classic adaptive numerical methods, the mesh as well as the domain may be refined or decomposed, respectively, in order to improve accuracy. Also the degree of approximation accuracy may be adapted. It would be desirable to transfer such important and successful strategies to the field of neural network based solutions. In the present work, we propose a novel adaptive neural approach to meet this aim for solving time-dependent problems. To this end, each subdomain is reduced in size until the optimisation is resolved up to a predefined training accuracy. In addition, while the neural networks employed are by default small, we propose a means to adjust also the number of neurons in an adaptive way. We introduce conditions to automatically confirm the solution reliability and optimise computational parameters whenever it is necessary. Results are provided for several initial value problems that illustrate important computational properties of the method alongside. In total, our approach not only allows to analyse in high detail the relation between network error and numerical accuracy. The new approach also allows reliable neural network solutions over large computational domains.

cs.NE↗

arXiv2021

Computational characteristics of feedforward neural networks for solving a stiff differential equation

Feedforward neural networks offer a promising approach for solving differential equations. However, the reliability and accuracy of the approximation still represent delicate issues that are not fully resolved in the current literature. Computational approaches are in general highly dependent on a variety of computational parameters as well as on the choice of optimisation methods, a point that has to be seen together with the structure of the cost function. The intention of this paper is to make a step towards resolving these open issues. To this end we study here the solution of a simple but fundamental stiff ordinary differential equation modelling a damped system. We consider two computational approaches for solving differential equations by neural forms. These are the classic but still actual method of trial solutions defining the cost function, and a recent direct construction of the cost function related to the trial solution method. Let us note that the settings we study can easily be applied more generally, including solution of partial differential equations. By a very detailed computational study we show that it is possible to identify preferable choices to be made for parameters and methods. We also illuminate some interesting effects that are observable in the neural network simulations. Overall we extend the current literature in the field by showing what can be done in order to obtain reliable and accurate results by the neural network approach. By doing this we illustrate the importance of a careful choice of the computational setup.

cs.NE↗

arXiv2021

Collocation Polynomial Neural Forms and Domain Fragmentation for solving Initial Value Problems

Several neural network approaches for solving differential equations employ trial solutions with a feedforward neural network. There are different means to incorporate the trial solution in the construction, for instance one may include them directly in the cost function. Used within the corresponding neural network, the trial solutions define the so-called neural form. Such neural forms represent general, flexible tools by which one may solve various differential equations. In this article we consider time-dependent initial value problems, which require to set up the neural form framework adequately. The neural forms presented up to now in the literature for such a setting can be considered as first order polynomials. In this work we propose to extend the polynomial order of the neural forms. The novel collocation-type construction includes several feedforward neural networks, one for each order. Additionally, we propose the fragmentation of the computational domain into subdomains. The neural forms are solved on each subdomain, whereas the interfacing grid points overlap in order to provide initial values over the whole fragmentation. We illustrate in experiments that the combination of collocation neural forms of higher order and the domain fragmentation allows to solve initial value problems over large domains with high accuracy and reliability.

cs.NE↗

arXiv2019

Model Order Reduction for Efficient Descriptor-Based Shape Analysis

In order to investigate correspondences between 3D shapes, many methods rely on a feature descriptor which is invariant under almost isometric transformations. An interesting class of models for such descriptors relies on partial differential equations (PDEs) based on the Laplace-Beltrami operator for constructing intrinsic shape signatures. In order to conduct the construction, not only a variety of PDEs but also several ways to solve them have been considered in previous works. In particular, spectral methods have been used derived from the series expansion of analytic solutions of the PDEs, and alternatively numerical integration schemes have been proposed. In this paper we show how to define a computational framework by model order reduction (MOR) that yields efficient PDE integration and much more accurate shape signatures as in previous works. Within the construction of our framework we introduce some technical novelties that contribute to these advances, and in doing this we present some improvements for virtually all considered methods. As part of the main contributions, we show for the first time an extensive and detailed comparison between the spectral and integration techniques, which is possible by the advances documented in this paper. We also propose here to employ soft correspondences in the context of the MOR methods which turns out to be highly beneficial with this approach.

math.NA↗

arXiv2018

Theoretical Foundation of the Weighted Laplace Inpainting Problem

Laplace interpolation is a popular approach in image inpainting using partial differential equations. The classic approach considers the Laplace equation with mixed boundary conditions. Recently a more general formulation has been proposed where the differential operator consists of a point-wise convex combination of the Laplacian and the known image data. We provide the first detailed analysis on existence and uniqueness of solutions for the arising mixed boundary value problem. Our approach considers the corresponding weak formulation and aims at using the Theorem of Lax-Milgram to assert the existence of a solution. To this end we have to resort to weighted Sobolev spaces. Our analysis shows that solutions do not exist unconditionally. The weights need some regularity and fulfil certain growth conditions. The results from this work complement findings which were previously only available for a discrete setup.

math.AP↗

arXiv2017

Optimisation of photometric stereo methods by non-convex variational minimisation

Estimating shape and appearance of a three dimensional object from a given set of images is a classic research topic that is still actively pursued. Among the various techniques available, PS is distinguished by the assumption that the underlying input images are taken from the same point of view but under different lighting conditions. The most common techniques provide the shape information in terms of surface normals. In this work, we instead propose to minimise a much more natural objective function, namely the reprojection error in terms of depth. Minimising the resulting non-trivial variational model for PS allows to recover the depth of the photographed scene directly. As a solving strategy, we follow an approach based on a recently published optimisation scheme for non-convex and non-smooth cost functions. The main contributions of our paper are of theoretical nature. A technical novelty in our framework is the usage of matrix differential calculus. We supplement our approach by a detailed convergence analysis of the resulting optimisation algorithm and discuss possibilities to ease the computational complexity. At hand of an experimental evaluation we discuss important properties of the method. Overall, our strategy achieves more accurate results than competing approaches. The experiments also highlights some practical aspects of the underlying optimisation algorithm that may be of interest in a more general context.

cs.CV↗

arXiv2017

Photometric stereo for strong specular highlights

Photometric stereo (PS) is a fundamental technique in computer vision known to produce 3-D shape with high accuracy. The setting of PS is defined by using several input images of a static scene taken from one and the same camera position but under varying illumination. The vast majority of studies in this 3-D reconstruction method assume orthographic projection for the camera model. In addition, they mainly consider the Lambertian reflectance model as the way that light scatters at surfaces. So, providing reliable PS results from real world objects still remains a challenging task. We address 3-D reconstruction by PS using a more realistic set of assumptions combining for the first time the complete Blinn-Phong reflectance model and perspective projection. To this end, we will compare two different methods of incorporating the perspective projection into our model. Experiments are performed on both synthetic and real world images. Note that our real-world experiments do not benefit from laboratory conditions. The results show the high potential of our method even for complex real world applications such as medical endoscopy images which may include high amounts of specular highlights.

cs.CV↗

arXiv2017

Clustering-Based Quantisation for PDE-Based Image Compression

Finding optimal data for inpainting is a key problem in the context of partial differential equation based image compression. The data that yields the most accurate reconstruction is real-valued. Thus, quantisation models are mandatory to allow an efficient encoding. These can also be understood as challenging data clustering problems. Although clustering approaches are well suited for this kind of compression codecs, very few works actually consider them. Each pixel has a global impact on the reconstruction and optimal data locations are strongly correlated with their corresponding colour values. These facts make it hard to predict which feature works best. In this paper we discuss quantisation strategies based on popular methods such as k-means. We are lead to the central question which kind of feature vectors are best suited for image compression. To this end we consider choices such as the pixel values, the histogram or the colour map. Our findings show that the number of colours can be reduced significantly without impacting the reconstruction quality. Surprisingly, these benefits do not directly translate to a good image compression performance. The gains in the compression ratio are lost due to increased storage costs. This suggests that it is integral to evaluate the clustering on both, the reconstruction error and the final file size.

cs.CV↗

arXiv2016

Fast and Accurate Surface Normal Integration on Non-Rectangular Domains

The integration of surface normals for the purpose of computing the shape of a surface in 3D space is a classic problem in computer vision. However, even nowadays it is still a challenging task to devise a method that combines the flexibility to work on non-trivial computational domains with high accuracy, robustness and computational efficiency. By uniting a classic approach for surface normal integration with modern computational techniques we construct a solver that fulfils these requirements. Building upon the Poisson integration model we propose to use an iterative Krylov subspace solver as a core step in tackling the task. While such a method can be very efficient, it may only show its full potential when combined with a suitable numerical preconditioning and a problem-specific initialisation. We perform a thorough numerical study in order to identify an appropriate preconditioner for our purpose. To address the issue of a suitable initialisation we propose to compute this initial state via a recently developed fast marching integrator. Detailed numerical experiments illuminate the benefits of this novel combination. In addition, we show on real-world photometric stereo datasets that the developed numerical framework is flexible enough to tackle modern computer vision applications.

math.NA↗

arXiv2016

Highly Robust Clustering of GPS Driver Data for Energy Efficient Driving Style Modelling

This paper presents a novel approach to distinguish driving styles with respect to their energy efficiency. A distinct property of our method is that it relies exclusively on Global Positioning System (GPS) logs of drivers. This setting is highly relevant in practice as these data can easily be acquired. Relying on positional data alone means that all derived features will be correlated, so we strive to find a single quantity that allows us to perform the driving style analysis. To this end we consider a robust variation of the so called jerk of a movement. We show that our feature choice outperforms other more commonly used jerk-based formulations and we discuss the handling of noisy, inconsistent, and incomplete data as this is a notorious problem when dealing with real-world GPS logs. Our solving strategy relies on an agglomerative hierarchical clustering combined with an L-term heuristic to determine the relevant number of clusters. It can easily be implemented and performs fast, even on very large, real-world data sets. Experiments show that our approach is robust against noise and able to discern different driving styles.

cs.OH↗

arXiv2016

A Numerical Study of Newton Interpolation with Extremely High Degrees

In current textbooks the use of Chebyshev nodes with Newton interpolation is advocated as the most efficient numerical interpolation method in terms of approximation accuracy and computational effort. However, we show numerically that the approximation quality obtained by Newton interpolation with Fast Leja (FL) points is competitive to the use of Chebyshev nodes, even for extremely high degree interpolation. This is an experimental account of the analytic result that the limit distribution of FL points and Chebyshev nodes is the same when letting the number of points go to infinity. Since the FL construction is easy to perform and allows to add interpolation nodes on the fly in contrast to the use of Chebyshev nodes, our study suggests that Newton interpolation with FL points is currently the most efficient numerical technique for polynomial interpolation. Moreover, we give numerical evidence that any reasonable function can be approximated up to machine accuracy by Newton interpolation with FL points if desired, which shows the potential of this method.

math.NA↗

arXiv2016

Implicit monotone difference methods for scalar conservation laws with source terms

In this article, a concept of implicit methods for scalar conservation laws in one or more spatial dimensions allowing also for source terms of various types is presented. This material is a significant extension of previous work of the first author [3]. Implicit notions are developed that are centered around a monotonicity criterion. We demonstrate a connection between a numerical scheme and a discrete entropy inequality, which is based on a classical approach by Crandall and Majda. Additionally, three implicit methods are investigated using the developed notions. Next, we conduct a convergence proof which is not based on a classical compactness argument. Finally, the theoretical results are confirmed by various numerical tests.

math.AP↗

arXiv2016

Bregman Iteration for Correspondence Problems: A Study of Optical Flow

Bregman iterations are known to yield excellent results for denoising, deblurring and compressed sensing tasks, but so far this technique has rarely been used for other image processing problems. In this paper we give a thorough description of the Bregman iteration, unifying thereby results of different authors within a common framework. Then we show how to adapt the split Bregman iteration, originally developed by Goldstein and Osher for image restoration purposes, to optical flow which is a fundamental correspondence problem in computer vision. We consider some classic and modern optical flow models and present detailed algorithms that exhibit the benefits of the Bregman iteration. By making use of the results of the Bregman framework, we address the issues of convergence and error estimation for the algorithms. Numerical examples complement the theoretical part.

math.OC↗

arXiv2016

Sparse l1 Regularisation of Matrix Valued Models for Acoustic Source Characterisation

We present a strategy for the recovery of a sparse solution of a common problem in acoustic engineering, which is the reconstruction of sound source levels and locations applying microphone array measurements. The considered task bears similarities to the basis pursuit formalism but also relies on additional model assumptions that are challenging from a mathematical point of view. Our approach reformulates the original task as a convex optimisation model. The sought solution shall be a matrix with a certain desired structure. We enforce this structure through additional constraints. By combining popular splitting algorithms and matrix differential theory in a novel framework we obtain a numerically efficient strategy. Besides a thorough theoretical consideration we also provide an experimental setup that certifies the usability of our strategy. Finally, we also address practical issues, such as the handling of inaccuracies in the measurement and corruption of the given data. We provide a post processing step that is capable of yielding an almost perfect solution in such circumstances.

math.OC↗

arXiv2015

Direct Variational Perspective Shape from Shading with Cartesian Depth Parametrisation

Most of today's state-of-the-art methods for perspective shape from shading are modelled in terms of partial differential equations (PDEs) of Hamilton-Jacobi type. To improve the robustness of such methods w.r.t. noise and missing data, first approaches have recently been proposed that seek to embed the underlying PDE into a variational framework with data and smoothness term. So far, however, such methods either make use of a radial depth parametrisation that makes the regularisation hard to interpret from a geometrical viewpoint or they consider indirect smoothness terms that require additional consistency constraints to provide valid solutions. Moreover the minimisation of such frameworks is an intricate task, since the underlying energy is typically non-convex. In our paper we address all three of the aforementioned issues. First, we propose a novel variational model that operates directly on the Cartesian depth. In this context, we also point out a common mistake in the derivation of the surface normal. Moreover, we employ a direct second-order regulariser with edge-preservation property. This direct regulariser yields by construction valid solutions without requiring additional consistency constraints. Finally, we also propose a novel coarse-to-fine minimisation framework based on an alternating explicit scheme. This framework allows us to avoid local minima during the minimisation and thus to improve the accuracy of the reconstruction. Experiments show the good quality of our model as well as the usefulness of the proposed numerical scheme.

cs.CV↗