arXiv subjects

Henri Pesonen

Publications and source records attributed to Henri Pesonen.

12 recordsLinked to original sources

arXiv2025

Surrogate-based ABC matches generalized Bayesian inference under specific discrepancy and kernel choices

Generalized Bayesian inference (GBI) is an alternative inference framework motivated by robustness to modeling errors, where a specific loss function is used to link the model parameters with observed data, instead of the log-likelihood used in standard Bayesian inference. Approximate Bayesian Computation (ABC) refers in turn to a family of methods approximating the posterior distribution via a discrepancy function between the observed and simulated data instead of using the likelihood. In this paper we discuss the connection between ABC and GBI, when the loss function is defined as an expected discrepancy between the observed and simulated data from the model under consideration. We show that the resulting generalized posterior corresponds to an ABC-posterior when the latter is obtained under a Gaussian process -based surrogate model. We illustrate the behavior of the approximations as a function of specific discrepancy and kernel choices to provide insights of the relationships between these different approximate inference paradigms.

stat.ME↗

arXiv2023

Stratified distance space improves the efficiency of sequential samplers for approximate Bayesian computation

Approximate Bayesian computation (ABC) methods are standard tools for inferring parameters of complex models when the likelihood function is analytically intractable. A popular approach to improving the poor acceptance rate of the basic rejection sampling ABC algorithm is to use sequential Monte Carlo (ABC SMC) to produce a sequence of proposal distributions adapting towards the posterior, instead of generating values from the prior distribution of the model parameters. Proposal distribution for the subsequent iteration is typically obtained from a weighted set of samples, often called particles, of the current iteration of this sequence. Current methods for constructing these proposal distributions treat all the particles equivalently, regardless of the corresponding value generated by the sampler, which may lead to inefficiency when propagating the information across iterations of the algorithm. To improve sampler efficiency, we introduce a modified approach called stratified distance ABC SMC. Our algorithm stratifies particles based on their distance between the corresponding synthetic and observed data, and then constructs distinct proposal distributions for all the strata. Taking into account the distribution of distances across the particle space leads to substantially improved acceptance rate of the rejection sampling. We further show that efficiency can be gained by introducing a novel stopping rule for the sequential process based on the stratified posterior samples and demonstrate these advances by several examples.

stat.ME↗

arXiv2023

An Extendable Python Implementation of Robust Optimisation Monte Carlo

Performing inference in statistical models with an intractable likelihood is challenging, therefore, most likelihood-free inference (LFI) methods encounter accuracy and efficiency limitations. In this paper, we present the implementation of the LFI method Robust Optimisation Monte Carlo (ROMC) in the Python package ELFI. ROMC is a novel and efficient (highly-parallelizable) LFI framework that provides accurate weighted samples from the posterior. Our implementation can be used in two ways. First, a scientist may use it as an out-of-the-box LFI algorithm; we provide an easy-to-use API harmonized with the principles of ELFI, enabling effortless comparisons with the rest of the methods included in the package. Additionally, we have carefully split ROMC into isolated components for supporting extensibility. A researcher may experiment with novel method(s) for solving part(s) of ROMC without reimplementing everything from scratch. In both scenarios, the ROMC parts can run in a fully-parallelized manner, exploiting all CPU cores. We also provide helpful functionalities for (i) inspecting the inference process and (ii) evaluating the obtained samples. Finally, we test the robustness of our implementation on some typical LFI examples.

cs.LG↗

arXiv2023

Prior knowledge elicitation: The past, present, and future

Specification of the prior distribution for a Bayesian model is a central part of the Bayesian workflow for data analysis, but it is often difficult even for statistical experts. In principle, prior elicitation transforms domain knowledge of various kinds into well-defined prior distributions, and offers a solution to the prior specification problem. In practice, however, we are still fairly far from having usable prior elicitation tools that could significantly influence the way we build probabilistic models in academia and industry. We lack elicitation methods that integrate well into the Bayesian workflow and perform elicitation efficiently in terms of costs of time and effort. We even lack a comprehensive theoretical framework for understanding different facets of the prior elicitation problem. Why are we not widely using prior elicitation? We analyse the state of the art by identifying a range of key aspects of prior knowledge elicitation, from properties of the modelling task and the nature of the priors to the form of interaction with the expert. The existing prior elicitation literature is reviewed and categorized in these terms. This allows recognizing under-studied directions in prior elicitation research, finally leading to a proposal of several new avenues to improve prior elicitation methodology.

stat.ME↗

arXiv2023

Likelihood-Free Inference in State-Space Models with Unknown Dynamics

Likelihood-free inference (LFI) has been successfully applied to state-space models, where the likelihood of observations is not available but synthetic observations generated by a black-box simulator can be used for inference instead. However, much of the research up to now have been restricted to cases, in which a model of state transition dynamics can be formulated in advance and the simulation budget is unrestricted. These methods fail to address the problem of state inference when simulations are computationally expensive and the Markovian state transition dynamics are undefined. The approach proposed in this manuscript enables LFI of states with a limited number of simulations by estimating the transition dynamics, and using state predictions as proposals for simulations. In the experiments with non-stationary user models, the proposed method demonstrates significant improvement in accuracy for both state inference and prediction, where a multi-output Gaussian process is used for LFI of states, and a Bayesian Neural Network as a surrogate model of transition dynamics.

cs.LG↗

arXiv2022

High-Q guided-mode resonance of a crossed grating with near-flat dispersion

Guided-mode resonances in diffraction gratings are manifested as peaks (dips) in reflection (transmission) spectra. Smaller resonance line widths (higher Q-factors) ensure stronger light-matter interactions and are beneficial for field-dependent physical processes. However, strong angular and spectral dispersion are inherent to such high-Q resonances. We demonstrate that a class of high-Q resonant modes (Q-factor >1000) exhibiting extraordinarily weak dispersion can be excited in crossed gratings simultaneously with the modes with well-known nearly linear dispersion. Furthermore, we show that the polarization of the incoming light can be adjusted to engineer the dispersion of these modes, and strong to near-flat dispersion or vice-versa can be achieved by switching between two mutually orthogonal linear polarization states. We introduce a semi-analytical model to explain the underlying physics behind these observations and perform full-wave numerical simulations and experiments to support our theoretical conjecture. The results presented here will benefit all applications that rely on resonances in free-space-coupled geometries.

physics.optics↗

arXiv2022

ABC of the Future

Approximate Bayesian computation (ABC) has advanced in two decades from a seminal idea to a practically applicable inference tool for simulator-based statistical models, which are becoming increasingly popular in many research domains. The computational feasibility of ABC for practical applications has been recently boosted by adopting techniques from machine learning to build surrogate models for the approximate likelihood or posterior and by the introduction of a general-purpose software platform with several advanced features, including automated parallelization. Here we demonstrate the strengths of the advances in ABC by going beyond the typical benchmark examples and considering real applications in astronomy, infectious disease epidemiology, personalised cancer therapy and financial prediction. We anticipate that the emerging success of ABC in producing actual added value and quantitative insights in the real world will continue to inspire a plethora of further applications across different fields of science, social science and technology.

stat.AP↗

arXiv2021

Likelihood-Free Inference with Deep Gaussian Processes

In recent years, surrogate models have been successfully used in likelihood-free inference to decrease the number of simulator evaluations. The current state-of-the-art performance for this task has been achieved by Bayesian Optimization with Gaussian Processes (GPs). While this combination works well for unimodal target distributions, it is restricting the flexibility and applicability of Bayesian Optimization for accelerating likelihood-free inference more generally. We address this problem by proposing a Deep Gaussian Process (DGP) surrogate model that can handle more irregularly behaved target distributions. Our experiments show how DGPs can outperform GPs on objective functions with multimodal distributions and maintain a comparable performance in unimodal cases. This confirms that DGPs as surrogate models can extend the applicability of Bayesian Optimization for likelihood-free inference (BOLFI), while adding computational overhead that remains negligible for computationally intensive simulators.

cs.LG↗

arXiv2020

Generalised Bayes Updates with $f$-divergences through Probabilistic Classifiers

A stream of algorithmic advances has steadily increased the popularity of the Bayesian approach as an inference paradigm, both from the theoretical and applied perspective. Even with apparent successes in numerous application fields, a rising concern is the robustness of Bayesian inference in the presence of model misspecification, which may lead to undesirable extreme behavior of the posterior distributions for large sample sizes. Generalized belief updating with a loss function represents a central principle to making Bayesian inference more robust and less vulnerable to deviations from the assumed model. Here we consider such updates with $f$-divergences to quantify a discrepancy between the assumed statistical model and the probability distribution which generated the observed data. Since the latter is generally unknown, estimation of the divergence may be viewed as an intractable problem. We show that the divergence becomes accessible through the use of probabilistic classifiers that can leverage an estimate of the ratio of two probability distributions even when one or both of them is unknown. We demonstrate the behavior of generalized belief updates for various specific choices under the $f$-divergence family. We show that for specific divergence functions such an approach can even improve on methods evaluating the correct model likelihood function analytically.

stat.ME↗

arXiv2020

Probabilistic elicitation of expert knowledge through assessment of computer simulations

We present a new method for probabilistic elicitation of expert knowledge using binary responses of human experts assessing simulated data from a statistical model, where the parameters are subject to uncertainty. The binary responses describe either the absolute realism of individual simulations or the relative realism of a pair of simulations in the two alternative versions of out approach. Each version provides a nonparametric representation of the expert belief distribution over the values of a model parameter, without demanding the assertion of any opinion on the parameter values themselves. Our framework also integrates the use of active learning to efficiently query the experts, with the possibility to additionally provide a useful misspecification diagnostic. We validate both methods on an automatic expert judging a binomial distribution, and on human experts judging the distribution of voters across political parties in the United States and Norway. Both methods provide flexible and meaningful representations of the human experts' beliefs, correctly identifying the higher dispersion of voters between parties in Norway.

stat.ME↗

arXiv2020

Misspecification-robust likelihood-free inference in high dimensions

Likelihood-free inference for simulator-based statistical models has developed rapidly from its infancy to a useful tool for practitioners. However, models with more than a handful of parameters still generally remain a challenge for the Approximate Bayesian Computation (ABC) based inference. To advance the possibilities for performing likelihood-free inference in higher dimensional parameter spaces, we introduce an extension of the popular Bayesian optimisation based approach to approximate discrepancy functions in a probabilistic manner which lends itself to an efficient exploration of the parameter space. Our approach achieves computational scalability for higher dimensional parameter spaces by using separate acquisition functions and discrepancies for each parameter. The efficient additive acquisition structure is combined with exponentiated loss -likelihood to provide a misspecification-robust characterisation of the marginal posterior distribution for all model parameters. The method successfully performs computationally efficient inference in a 100-dimensional space on canonical examples and compares favourably to existing modularised ABC methods. We further illustrate the potential of this approach by fitting a bacterial transmission dynamics model to a real data set, which provides biologically coherent results on strain competition in a 30-dimensional parameter space.

stat.CO↗

arXiv2019

Approximate Bayesian Computation via Population Monte Carlo and Classification

Approximate Bayesian computation (ABC) methods can be used to sample from posterior distributions when the likelihood function is unavailable or intractable, as is often the case in biological systems. ABC methods suffer from inefficient particle proposals in high dimensions, and subjectivity in the choice of summary statistics, discrepancy measure, and error tolerance. Sequential Monte Carlo (SMC) methods have been combined with ABC to improve the efficiency of particle proposals, but suffer from subjectivity and require many simulations from the likelihood function. Likelihood-Free Inference by Ratio Estimation (LFIRE) leverages classification to estimate the posterior density directly but does not explore the parameter space efficiently. This work proposes a classification approach that approximates population Monte Carlo (PMC), where model class probabilities from classification are used to update particle weights. This approach, called Classification-PMC, blends adaptive proposals and classification, efficiently producing samples from the posterior without subjectivity. We show through a simulation study that Classification-PMC outperforms two state-of-the-art methods: ratio estimation and SMC ABC when it is computationally difficult to simulate from the likelihood.

stat.ML↗