arXiv subjects

Werner G. Müller

Publications and source records attributed to Werner G. Müller.

14 recordsLinked to original sources

arXiv2026

An Introduction to Copulas: a Complement

For many years I have taught an advanced statistical inference course for master's students using the text of Casella and Berger (2002). The book gives a comprehensive treatment of the core topics at a level that avoids measure theory while remaining mathematically precise, but it does not cover the increasingly important concept of copulas. The present notes are intended to complement the book by adding two sections on copulas in a style that is as close as possible to that of the original text. Numbering of definitions, theorems, examples, and exercises is consistent with Casella and Berger (2002), but the material may also be read as a brief, stand-alone introduction to copula theory.

stat.OT↗

arXiv2025

Green LIME: Improving AI Explainability through Design of Experiments

In artificial intelligence (AI), the complexity of many models and processes surpasses human understanding, making it challenging to determine why a specific prediction is made. This lack of transparency is particularly problematic in critical fields like healthcare, where trust in a model's predictions is paramount. As a result, the explainability of machine learning (ML) and other complex models has become a key area of focus. Efforts to improve model explainability often involve experimenting with AI systems and approximating their behavior through interpretable surrogate mechanisms. However, these procedures can be resource-intensive. Optimal design of experiments, which seeks to maximize the information obtained from a limited number of observations, offers promising methods for improving the efficiency of these explainability techniques. To demonstrate this potential, we explore Local Interpretable Model-agnostic Explanations (LIME), a widely used method introduced by Ribeiro et al. (2016). LIME provides explanations by generating new data points near the instance of interest and passing them through the model. While effective, this process can be computationally expensive, especially when predictions are costly or require many samples. LIME is highly versatile and can be applied to a wide range of models and datasets. In this work, we focus on models involving tabular data, regression tasks, and linear models as interpretable local approximations. By utilizing optimal design of experiments' techniques, we reduce the number of function evaluations of the complex model, thereby reducing the computational effort of LIME by a significant amount. We consider this modified version of LIME to be energy-efficient or "green".

stat.ML↗

arXiv2025

Practical aspects of the virtual noise convex optimum design approach for correlated responses

In this paper we present several practically-oriented extensions and considerations for the virtual noise method in optimal design under correlation. First we introduce a slightly modified virtual noise representation which further illuminates the parallels to the classical design approach for uncorrelated observations. We suggest more efficient algorithms to obtain the design measures. Furthermore, we show that various convex relaxation methods used for sensor selection are special cases of our approach and can be solved within our framework. Finally, we provide practical guidelines on how to generally approach a design problem with correlated observations and demonstrate how to utilize the virtual noise method in this context in a meaningful way.

stat.ME↗

arXiv2024

A criterion and incremental design construction for simultaneous kriging predictions

In this paper, we further investigate the problem of selecting a set of design points for universal kriging, which is a widely used technique for spatial data analysis. Our goal is to select the design points in order to make simultaneous predictions of the random variable of interest at a finite number of unsampled locations with maximum precision. Specifically, we consider as response a correlated random field given by a linear model with an unknown parameter vector and a spatial error correlation structure. We propose a new design criterion that aims at simultaneously minimizing the variation of the prediction errors at various points. We also present various efficient techniques for incrementally building designs for that criterion scaling well for high dimensions. Thus the method is particularly suitable for big data applications in areas of spatial data analysis such as mining, hydrogeology, natural resource monitoring, and environmental sciences or equivalently for any computer simulation experiments. We have demonstrated the effectiveness of the proposed designs through two illustrative examples: one by simulation and another based on real data from Upper Austria.

stat.ME↗

arXiv2022

Active Discrimination Learning for Gaussian Process Models

The paper covers the design and analysis of experiments to discriminate between two Gaussian process models, such as those widely used in computer experiments, kriging, sensor location and machine learning. Two frameworks are considered. First, we study sequential constructions, where successive design (observation) points are selected, either as additional points to an existing design or from the beginning of observation. The selection relies on the maximisation of the difference between the symmetric Kullback Leibler divergences for the two models, which depends on the observations, or on the mean squared error of both models, which does not. Then, we consider static criteria, such as the familiar log-likelihood ratios and the Fréchet distance between the covariance functions of the two models. Other distance-based criteria, simpler to compute than previous ones, are also introduced, for which, considering the framework of approximate design, a necessary condition for the optimality of a design measure is provided. The paper includes a study of the mathematical links between different criteria and numerical illustrations are provided.

stat.ME↗

arXiv2021

Bayesian design for minimising prediction uncertainty in bivariate spatial responses with applications to air quality monitoring

Model-based geostatistical design involves the selection of locations to collect data to minimise an expected loss function over a set of all possible locations. The loss function is specified to reflect the aim of data collection, which, for geostatistical studies, could be to minimise the prediction uncertainty at unobserved locations. In this paper, we propose a new approach to design such studies via a loss function derived through considering the entropy about the model predictions and the parameters of the model. The approach also includes a multivariate extension to generalised linear spatial models, and thus can be used to design experiments with more than one response. Unfortunately, evaluating our proposed loss function is computationally expensive so we provide an approximation such that our approach can be adopted to design realistically sized geostatistical studies. This is demonstrated through a simulated study and through designing an air quality monitoring program in Queensland, Australia. The results show that our designs remain highly efficient in achieving each experimental objective individually, providing an ideal compromise between the two objectives. Accordingly, we advocate that our approach could be adopted more generally in model-based geostatistical design.

stat.CO↗

arXiv2021

A convex approach to optimum design of experiments with correlated observations

Optimal design of experiments for correlated processes is an increasingly relevant and active research topic. Present methods have restricted possibilities to judge their quality. To fill this gap, we complement the virtual noise approach by a convex formulation leading to an equivalence theorem comparable to the uncorrelated case and to an algorithm giving an upper performance bound against which alternative design methods can be judged. Moreover, a method for generating exact designs follows naturally. We exclusively consider estimation problems on a finite design space with a fixed number of elements. A comparison on some classical examples from the literature as well as a real application is provided.

math.ST↗

arXiv2021

Impact of the error structure on the design and analysis of enzyme kinetic models

The statistical analysis of enzyme kinetic reactions usually involves models of the response functions which are well defined on the basis of Michaelis-Menten type equations. The error structure however is often without good reason assumed as additive Gaussian noise. This simple assumption may lead to undesired properties of the analysis, particularly when simulations are involved and consequently negative simulated reaction rates may occur. In this study we investigate the effect of assuming multiplicative lognormal errors instead. While there is typically little impact on the estimates, the experimental designs and their efficiencies are decisively affected, particularly when it comes to model discrimination problems.

q-bio.MN↗

arXiv2018

A design criterion for symmetric model discrimination based on nominal confidence sets

Experimental design applications for discriminating between models have been hampered by the assumption to know beforehand which model is the true one, which is counter to the very aim of the experiment. Previous approaches to alleviate this requirement were either symmetrizations of asymmetric techniques, or Bayesian, minimax and sequential approaches. Here we present a genuinely symmetric criterion based on a linearized distance between mean-value surfaces and the newly introduced tool of flexible nominal confidence sets. We demonstrate the computational efficiency of the approach using the proposed criterion and provide a Monte-Carlo evaluation of its discrimination performance on the basis of the likelihood ratio. An application for a pair of competing models in enzyme kinetics is given.

stat.ME↗

arXiv2016

Optimal discrimination design for copula models

Optimum experimental design theory has recently been extended for parameter estimation in copula models. However, the choice of the correct dependence structure still requires wider analyses. In this work the issue of copula selection is treated by using discrimination design techniques. The new proposed approach consists in the use of $D_s$-optimality following an extension of corresponding equivalence theory. We also present some examples and highlight the strength of such a criterion as a way to discriminate between various classes of dependences.

stat.ME↗

arXiv2015

Privacy sets for constrained space-filling

The paper provides typology for space filling into what we call "soft" and "hard" methods along with introducing the central notion of privacy sets for dealing with the latter. A heuristic algorithm based on this notion is presented and we compare its performance on some well-known examples.

stat.ME↗

arXiv2014

Optimal Designs for Copula Models

Copula modelling has in the past decade become a standard tool in many areas of applied statistics. However, a largely neglected aspect concerns the design of related experiments. Particularly the issue of whether the estimation of copula parameters can be enhanced by optimizing experimental conditions and how robust all the parameter estimates for the model are with respect to the type of copula employed. In this paper an equivalence theorem for (bivariate) copula models is provided that allows formulation of efficient design algorithms and quick checks of whether designs are optimal or at least efficient. Some examples illustrate that in practical situations considerable gains in design efficiency can be achieved. A natural comparison between different copula models with respect to design efficiency is provided as well.

stat.ME↗

arXiv2013

Likelihood-free Simulation-based Optimal Design

Simulation-based optimal design techniques are a convenient tool for solving a particular class of optimal design problems. The goal is to find the optimal configuration of factor settings with respect to an expected utility criterion. This criterion depends on the specified probability model for the data and on the assumed prior distribution for the model parameters. We develop new simulation-based optimal design methods which incorporate likelihood-free approaches and utilize them in novel applications. Most simulation-based design strategies solve the intractable expected utility integral at a specific design point by using Monte Carlo simulations from the probability model. Optimizing the criterion over the design points is carried out in a separate step. Müller (1999) introduces an MCMC algorithm which simultaneously addresses the simulation as well as the optimization problem. In principle, the optimal design can be found by detecting the utility mode of the sampled design points. Several improvements have been suggested to facilitate this task for multidimensional design problems (see e.g. Amzal et al. 2006). We aim to extend this simulation-based design methodology to design problems where the likelihood of the probability model is of an unknown analytical form but it is possible to simulate from the probability model. We further assume that prior observations are available. In such a setting it is seems natural to employ approximate Bayesian computation (ABC) techniques in order to be able to simulate from the conditional probability model. We provide a thorough review of adjacent literature and we investigate the benefits and the limitations of our design methodology for a particular paradigmatic example.

stat.ME↗

arXiv2013

Efficient Prediction Designs for Random Fields

For estimation and predictions of random fields it is increasingly acknowledged that the kriging variance may be a poor representative of true uncertainty. Experimental designs based on more elaborate criteria that are appropriate for empirical kriging are then often non-space-filling and very costly to determine. In this paper, we investigate the possibility of using a compound criterion inspired by an equivalence theorem type relation to build designs quasi-optimal for the empirical kriging variance, when space-filling designs become unsuitable. Two algorithms are proposed, one relying on stochastic optimization to explicitly identify the Pareto front, while the second uses the surrogate criteria as local heuristic to chose the points at which the (costly) true Empirical Kriging variance is effectively computed. We illustrate the performance of the algorithms presented on both a simple simulated example and a real oceanographic dataset.

stat.ME↗