arXiv subjects

Clemens Elster

Publications and source records attributed to Clemens Elster.

13 recordsLinked to original sources

arXiv2024

JCGM 101-compliant uncertainty evaluation using virtual experiments

Virtual experiments (VEs), a modern tool in metrology, can be used to help perform an uncertainty evaluation for the measurand. Current guidelines in metrology do not cover the many possibilities to incorporate VEs into an uncertainty evaluation, and it is often difficult to assess if the intended use of a VE complies with said guidelines. In recent work, it was shown that a VE can be used in conjunction with real measurement data and a Monte Carlo procedure to produce equal results to a supplement of the Guide to the Expression of Uncertainty in Measurement. However, this was shown only for linear measurement models. In this work, we extend this Monte Carlo approach to a common class of non-linear measurement models and more complex VEs, providing a reference approach for suitable uncertainty evaluations involving VEs. Numerical examples are given to show that the theoretical derivations hold in a practical scenario.

stat.AP↗

arXiv2024

Bayesian uncertainty evaluation applied to the tilted-wave interferometer

The tilted-wave interferometer is a promising technique for the development of a reference measurement system for the highly accurate form measurement of aspheres and freeform surfaces. The technique combines interferometric measurements, acquired with a special setup, and sophisticated mathematical evaluation procedures. To determine the form of the surface under test, a computational model is required that closely mimics the measurement process of the physical measurement instruments. The parameters of the computational model, comprising the surface under test sought, are then tuned by solving an inverse problem. Due to this embedded structure of the real experiment and computational model and the overall complexity, a thorough uncertainty evaluation is challenging. In this work, a Bayesian approach is proposed to tackle the inverse problem, based on a statistical model derived from the computational model of the tilted-wave interferometer. Such a procedure naturally allows for uncertainty quantification to be made. We present an approximate inference scheme to efficiently sample quantities of the posterior using Monte Carlo sampling involving the statistical model. In particular, the methodology derived is applied to the tilted-wave interferometer to obtain an estimate and corresponding uncertainty of the pixel-by-pixel form of the surface under test for two typical surfaces taking into account a number of key influencing factors. A statistical analysis using experimental design is employed to identify main influencing factors and a subsequent analysis confirms the efficacy of the method derived.

physics.optics↗

arXiv2023

Aleatoric uncertainty for Errors-in-Variables models in deep regression

A Bayesian treatment of deep learning allows for the computation of uncertainties associated with the predictions of deep neural networks. We show how the concept of Errors-in-Variables can be used in Bayesian deep regression to also account for the uncertainty associated with the input of the employed neural network. The presented approach thereby exploits a relevant, but generally overlooked, source of uncertainty and yields a decomposition of the predictive uncertainty into an aleatoric and epistemic part that is more complete and, in many cases, more consistent from a statistical perspective. We discuss the approach along various simulated and real examples and observe that using an Errors-in-Variables model leads to an increase in the uncertainty while preserving the prediction performance of models without Errors-in-Variables. For examples with known regression function we observe that this ground truth is substantially better covered by the Errors-in-Variables model, indicating that the presented approach leads to a more reliable uncertainty estimation.

cs.LG↗

arXiv2022

Generative models and Bayesian inversion using Laplace approximation

The Bayesian approach to solving inverse problems relies on the choice of a prior. This critical ingredient allows the formulation of expert knowledge or physical constraints in a probabilistic fashion and plays an important role for the success of the inference. Recently, Bayesian inverse problems were solved using generative models as highly informative priors. Generative models are a popular tool in machine learning to generate data whose properties closely resemble those of a given database. Typically, the generated distribution of data is embedded in a low-dimensional manifold. For the inverse problem, a generative model is trained on a database that reflects the properties of the sought solution, such as typical structures of the tissue in the human brain in magnetic resonance (MR) imaging. The inference is carried out in the low-dimensional manifold determined by the generative model which strongly reduces the dimensionality of the inverse problem. However, this proceeding produces a posterior that admits no Lebesgue density in the actual variables and the accuracy reached can strongly depend on the quality of the generative model. For linear Gaussian models we explore an alternative Bayesian inference based on probabilistic generative models which is carried out in the original high-dimensional space. A Laplace approximation is employed to analytically derive the required prior probability density function induced by the generative model. Properties of the resulting inference are investigated. Specifically, we show that derived Bayes estimates are consistent, in contrast to the approach employing the low-dimensional manifold of the generative model. The MNIST data set is used to construct numerical experiments which confirm our theoretical findings.

stat.ML↗

arXiv2022

Assessment of sub-sampling schemes for compressive nano-FTIR imaging

Nano-FTIR imaging is a powerful scanning-based technique at nanometer spatial resolution which combines Fourier transform infrared spectroscopy (FTIR) and scattering-type scanning near-field optical microscopy (s-SNOM). However, recording large spatial areas with nano-FTIR is limited by long measurement times due to its sequential data acquisition. Several mathematical approaches have been proposed to tackle this problem. All of them have in common that only a small fraction of randomly chosen measurements is required. However, choosing the fraction of measurements in a random fashion poses practical challenges for scanning procedures and does not lead to time savings as large as desired. We consider different, practically relevant sub-sampling schemes assuring a faster acquisition. It is demonstrated that results for almost all considered sub-sampling schemes, namely original Lissajous, triangle Lissajous, and random reflection sub-sampling, at a sub-sampling rate of 10%, are comparable to results when using a random sub-sampling of 10%. This implies that random sub-sampling is not required for efficient data acquisition.

physics.optics↗

arXiv2021

Deep Ensembles from a Bayesian Perspective

Deep ensembles can be considered as the current state-of-the-art for uncertainty quantification in deep learning. While the approach was originally proposed as a non-Bayesian technique, arguments supporting its Bayesian footing have been put forward as well. We show that deep ensembles can be viewed as an approximate Bayesian method by specifying the corresponding assumptions. Our findings lead to an improved approximation which results in an enlarged epistemic part of the uncertainty. Numerical examples suggest that the improved approximation can lead to more reliable uncertainties. Analytical derivations ensure easy calculation of results.

cs.LG↗

arXiv2021

A framework for benchmarking uncertainty in deep regression

We propose a framework for the assessment of uncertainty quantification in deep regression. The framework is based on regression problems where the regression function is a linear combination of nonlinear functions. Basically, any level of complexity can be realized through the choice of the nonlinear functions and the dimensionality of their domain. Results of an uncertainty quantification for deep regression are compared against those obtained by a statistical reference method. The reference method utilizes knowledge of the underlying nonlinear functions and is based on a Bayesian linear regression using a reference prior. Reliability of uncertainty quantification is assessed in terms of coverage probabilities, and accuracy through the size of calculated uncertainties. We illustrate the proposed framework by applying it to current approaches for uncertainty quantification in deep regression. The flexibility, together with the availability of a reference solution, makes the framework suitable for defining benchmark sets for uncertainty quantification.

cs.LG↗

arXiv2021

Hypothesis-based acceptance sampling for modules F and F1 of the European Measuring Instruments Directive

Millions of measuring instruments are verified each year before being placed on the markets worldwide. In the EU, such initial conformity assessments are regulated by the Measuring Instruments Directive (MID). The MID modules F and F1 on product verification allow for statistical acceptance sampling, whereby only random subsets of instruments need to be inspected. This paper re-interprets the acceptance sampling conditions formulated by the MID. The new interpretation is contrasted with the one advanced in WELMEC guide 8.10, and three advantages have become apparent. Firstly, an economic advantage of the new interpretation is a producers' risk bounded from above, such that measuring instruments with sufficient quality are accepted with a guaranteed probability of no less than 95 %. Secondly, a conceptual advantage is that the new MID interpretation fits into the well-known, formal framework of statistical hypothesis testing. Thirdly, the new interpretation applies unambiguously to finite-sized lots, even very small ones. We conclude that the new interpretation is to be preferred and suggest re-formulating the statistical sampling conditions in the MID. Re-interpreting the MID conditions implies that currently available sampling plans are either not admissible or not optimal. We derive a new acceptance sampling scheme and recommend its application.

stat.AP↗

arXiv2021

Uncertainty Quantification by Ensemble Learning for Computational Optical Form Measurements

Uncertainty quantification by ensemble learning is explored in terms of an application from computational optical form measurements. The application requires to solve a large-scale, nonlinear inverse problem. Ensemble learning is used to extend a recently developed deep learning approach for this application in order to provide an uncertainty quantification of its predicted solution to the inverse problem. By systematically inserting out-of-distribution errors as well as noisy data the reliability of the developed uncertainty quantification is explored. Results are encouraging and the proposed application exemplifies the ability of ensemble methods to make trustworthy predictions on high dimensional data in a real-world application.

cs.LG↗

arXiv2020

Deep Neural Networks for Computational Optical Form Measurements

Deep neural networks have been successfully applied in many different fields like computational imaging, medical healthcare, signal processing, or autonomous driving. In a proof-of-principle study, we demonstrate that computational optical form measurement can also benefit from deep learning. A data-driven machine learning approach is explored to solve an inverse problem in the accurate measurement of optical surfaces. The approach is developed and tested using virtual measurements with known ground truth.

eess.IV↗

arXiv2020

Detecting unusual input to neural networks

Evaluating a neural network on an input that differs markedly from the training data might cause erratic and flawed predictions. We study a method that judges the unusualness of an input by evaluating its informative content compared to the learned parameters. This technique can be used to judge whether a network is suitable for processing a certain input and to raise a red flag that unexpected behavior might lie ahead. We compare our approach to various methods for uncertainty evaluation from the literature for various datasets and scenarios. Specifically, we introduce a simple, effective method that allows to directly compare the output of such metrics for single input points even if these metrics live on different scales.

cs.LG↗

arXiv2020

The variation of the posterior variance and Bayesian sample size determination

We consider Bayesian sample size determination using a criterion that utilizes the first two moments of the expected posterior variance. We study the resulting sample size in dependence on the chosen prior and explore the success rate for bounding the posterior variance below a prescribed limit under the true sampling distribution. Compared with sample size determination based on the expected average of the posterior variance the proposed criterion leads to an increase in sample size and significantly improved success rates. Generic asymptotic properties are proven, such as an asymptotic expression for the sample size and a sort of phase transition. Our study is illustrated using two real world datasets with Poisson and normally distributed data. Based on our results some recommendations are given.

math.ST↗

arXiv2019

Inspecting adversarial examples using the Fisher information

Adversarial examples are slight perturbations that are designed to fool artificial neural networks when fed as an input. In this work the usability of the Fisher information for the detection of such adversarial attacks is studied. We discuss various quantities whose computation scales well with the network size, study their behavior on adversarial examples and show how they can highlight the importance of single input neurons, thereby providing a visual tool for further analyzing (un-)reasonable behavior of a neural network. The potential of our methods is demonstrated by applications to the MNIST, CIFAR10 and Fruits-360 datasets.

cs.LG↗