arXiv subjects

Julien Bect

Publications and source records attributed to Julien Bect.

At least 37 records · Page 2Linked to original sources

arXiv2017

Sequential design of experiments to estimate a probability of exceeding a threshold in a multi-fidelity stochastic simulator

In this article, we consider a stochastic numerical simulator to assess the impact of some factors on a phenomenon. The simulator is seen as a black box with inputs and outputs. The quality of a simulation, hereafter referred to as fidelity, is assumed to be tunable by means of an additional input of the simulator (e.g., a mesh size parameter): high-fidelity simulations provide more accurate results, but are time-consuming. Using a limited computation-time budget, we want to estimate, for any value of the physical inputs, the probability that a certain scalar output of the simulator will exceed a given critical threshold at the highest fidelity level. The problem is addressed in a Bayesian framework, using a Gaussian process model of the multi-fidelity simulator. We consider a Bayesian estimator of the probability, together with an associated measure of uncertainty, and propose a new multi-fidelity sequential design strategy, called Maximum Speed of Uncertainty Reduction (MSUR), to select the value of physical inputs and the fidelity level of new simulations. The MSUR strategy is tested on an example.

stat.CO↗

arXiv2017

Bayesian subset simulation

We consider the problem of estimating a probability of failure $α$, defined as the volume of the excursion set of a function $f:\mathbb{X} \subseteq \mathbb{R}^{d} \to \mathbb{R}$ above a given threshold, under a given probability measure on $\mathbb{X}$. In this article, we combine the popular subset simulation algorithm (Au and Beck, Probab. Eng. Mech. 2001) and our sequential Bayesian approach for the estimation of a probability of failure (Bect, Ginsbourger, Li, Picheny and Vazquez, Stat. Comput. 2012). This makes it possible to estimate $α$ when the number of evaluations of $f$ is very limited and $α$ is very small. The resulting algorithm is called Bayesian subset simulation (BSS). A key idea, as in the subset simulation algorithm, is to estimate the probabilities of a sequence of excursion sets of $f$ above intermediate thresholds, using a sequential Monte Carlo (SMC) approach. A Gaussian process prior on $f$ is used to define the sequence of densities targeted by the SMC algorithm, and drive the selection of evaluation points of $f$ to estimate the intermediate probabilities. Adaptive procedures are proposed to determine the intermediate thresholds and the number of evaluations to be carried out at each stage of the algorithm. Numerical experiments illustrate that BSS achieves significant savings in the number of function evaluations with respect to other Monte Carlo approaches.

stat.CO↗

arXiv2016

Design of a commercial aircraft environment control system using Bayesian optimization techniques

In this paper, we present the application of a recently developed algorithm for Bayesian multi-objective optimization to the design of a commercial aircraft environment control system (ECS). In our model, the ECS is composed of two cross-flow heat exchangers, a centrifugal compressor and a radial turbine, the geometries of which are simultaneously optimized to achieve minimal weight and entropy generation of the system. While both objectives impact the overall performance of the aircraft, they are shown to be antagonistic and a set of trade-off design solutions is identified. The algorithm used for optimizing the system implements a Bayesian approach to the multi-objective optimization problem in the presence of non-linear constraints and the emphasis is on conducting the optimization using a limited number of system simulations. Noteworthy features of this particular application include a non-hypercubic design domain and the presence of hidden constraints due to simulation failures.

math.OC↗

arXiv2016

Gaussian process modeling for stochastic multi-fidelity simulators, with application to fire safety

To assess the possibility of evacuating a building in case of a fire, a standard method consists in simulating the propagation of fire, using finite difference methods and takes into account the random behavior of the fire, so that the result of a simulation is non-deterministic. The mesh fineness tunes the quality of the numerical model, and its computational cost. Depending on the mesh fineness, one simulation can last anywhere from a few minutes to several weeks. In this article, we focus on predicting the behavior of the fire simulator at fine meshes, using cheaper results, at coarser meshes. In the literature of the design and analysis of computer experiments, such a problem is referred to as multi-fidelity prediction. Our contribution is to extend to the case of stochastic simulators the Bayesian multi-fidelity model proposed by Picheny and Ginsbourger (2013) and Tuo et al. (2014).

stat.ME↗

arXiv2016

A Bayesian approach to constrained single- and multi-objective optimization

This article addresses the problem of derivative-free (single- or multi-objective) optimization subject to multiple inequality constraints. Both the objective and constraint functions are assumed to be smooth, non-linear and expensive to evaluate. As a consequence, the number of evaluations that can be used to carry out the optimization is very limited, as in complex industrial design optimization problems. The method we propose to overcome this difficulty has its roots in both the Bayesian and the multi-objective optimization literatures. More specifically, an extended domination rule is used to handle objectives and constraints in a unified way, and a corresponding expected hyper-volume improvement sampling criterion is proposed. This new criterion is naturally adapted to the search of a feasible point when none is available, and reduces to existing Bayesian sampling criteria---the classical Expected Improvement (EI) criterion and some of its constrained/multi-objective extensions---as soon as at least one feasible point is available. The calculation and optimization of the criterion are performed using Sequential Monte Carlo techniques. In particular, an algorithm similar to the subset simulation method, which is well known in the field of structural reliability, is used to estimate the criterion. The method, which we call BMOO (for Bayesian Multi-Objective Optimization), is compared to state-of-the-art algorithms for single- and multi-objective constrained optimization.

stat.CO↗

arXiv2016

Quantifying uncertainties on excursion sets under a Gaussian random field prior

We focus on the problem of estimating and quantifying uncertainties on the excursion set of a function under a limited evaluation budget. We adopt a Bayesian approach where the objective function is assumed to be a realization of a Gaussian random field. In this setting, the posterior distribution on the objective function gives rise to a posterior distribution on excursion sets. Several approaches exist to summarize the distribution of such sets based on random closed set theory. While the recently proposed Vorob'ev approach exploits analytical formulae, further notions of variability require Monte Carlo estimators relying on Gaussian random field conditional simulations. In the present work we propose a method to choose Monte Carlo simulation points and obtain quasi-realizations of the conditional field at fine designs through affine predictors. The points are chosen optimally in the sense that they minimize the posterior expected distance in measure between the excursion set and its reconstruction. The proposed method reduces the computational costs due to Monte Carlo simulations and enables the computation of quasi-realizations on fine designs in large dimensions. We apply this reconstruction approach to obtain realizations of an excursion set on a fine grid which allow us to give a new measure of uncertainty based on the distance transform of the excursion set. Finally we present a safety engineering test case where the simulation method is employed to compute a Monte Carlo estimate of a contour line.

math.ST↗

arXiv2015

The Informational Approach to Global Optimization in presence of very noisy evaluation results. Application to the optimization of renewable energy integration strategies

We consider the problem of global optimization of a function f from very noisy evaluations. We adopt a Bayesian sequential approach: evaluation points are chosen so as to reduce the uncertainty about the position of the global optimum of f, as measured by the entropy of the corresponding random variable (Informational Approach to Global Optimization, Villemonteix et al., 2009). When evaluations are very noisy, the error coming from the estimation of the entropy using conditional simulations becomes non negligible compared to its variations on the input domain. We propose a solution to this problem by choosing evaluation points as if several evaluations were going to be made at these points. The method is applied to the optimization of a strategy for the integration of renewable energies into an electrical distribution network.

stat.CO↗

arXiv2014

A new integral loss function for Bayesian optimization

We consider the problem of maximizing a real-valued continuous function $f$ using a Bayesian approach. Since the early work of Jonas Mockus and Antanas Žilinskas in the 70's, the problem of optimization is usually formulated by considering the loss function $\max f - M_n$ (where $M_n$ denotes the best function value observed after $n$ evaluations of $f$). This loss function puts emphasis on the value of the maximum, at the expense of the location of the maximizer. In the special case of a one-step Bayes-optimal strategy, it leads to the classical Expected Improvement (EI) sampling criterion. This is a special case of a Stepwise Uncertainty Reduction (SUR) strategy, where the risk associated to a certain uncertainty measure (here, the expected loss) on the quantity of interest is minimized at each step of the algorithm. In this article, assuming that $f$ is defined over a measure space $(\mathbb{X}, λ)$, we propose to consider instead the integral loss function $\int_{\mathbb{X}} (f - M_n)_{+}\, dλ$, and we show that this leads, in the case of a Gaussian process prior, to a new numerically tractable sampling criterion that we call $\rm EI^2$ (for Expected Integrated Expected Improvement). A numerical experiment illustrates that a SUR strategy based on this new sampling criterion reduces the error on both the value and the location of the maximizer faster than the EI-based strategy.

stat.CO↗

arXiv2013

Relabeling and Summarizing Posterior Distributions in Signal Decomposition Problems when the Number of Components is Unknown

This paper addresses the problems of relabeling and summarizing posterior distributions that typically arise, in a Bayesian framework, when dealing with signal decomposition problems with an unknown number of components. Such posterior distributions are defined over union of subspaces of differing dimensionality and can be sampled from using modern Monte Carlo techniques, for instance the increasingly popular RJ-MCMC method. No generic approach is available, however, to summarize the resulting variable-dimensional samples and extract from them component-specific parameters. We propose a novel approach, named Variable-dimensional Approximate Posterior for Relabeling and Summarizing (VAPoRS), to this problem, which consists in approximating the posterior distribution of interest by a "simple"---but still variable-dimensional---parametric distribution. The distance between the two distributions is measured using the Kullback-Leibler divergence, and a Stochastic EM-type algorithm, driven by the RJ-MCMC sampler, is proposed to estimate the parameters. Two signal decomposition problems are considered, to show the capability of VAPoRS both for relabeling and for summarizing variable dimensional posterior distributions: the classical problem of detecting and estimating sinusoids in white Gaussian noise on the one hand, and a particle counting problem motivated by the Pierre Auger project in astrophysics on the other hand.

stat.CO↗

arXiv2012

Note on the computation of the Metropolis-Hastings ratio for Birth-or-Death moves in trans-dimensional MCMC algorithms for signal decomposition problems

Reversible jump MCMC (RJ-MCMC) sampling techniques, which allow to jointly tackle model selection and parameter estimation problems in a coherent Bayesian framework, have become increasingly popular in the signal processing literature since the seminal paper of Andrieu and Doucet (IEEE Trans. Signal Process., 47(10), 1999). Crucial to the implementation of any RJ-MCMC sampler is the computation of the so-called Metropolis-Hastings-Green (MHG) ratio, which determines the acceptance probability for the proposed moves. It turns out that the expression of the MHG ratio that was given in the paper of Andrieu and Doucet for "Birth-or-Death" moves---the simplest kind of trans-dimensional move, used in virtually all applications of RJ-MCMC to signal decomposition problems---was erroneous. Unfortunately, this mistake has been reproduced in many subsequent papers dealing with RJ-MCMC sampling in the signal processing literature. This note discusses the computation of the MHG ratio, with a focus on the case where the proposal kernel can be decomposed as a mixture of simpler kernels, for which the MHG ratio is easy to compute. We provide sufficient conditions under which the MHG ratio of the mixture can be deduced from the MHG ratios of the elementary kernels of which it is composed. As an application, we consider the case of Birth-or-Death moves, and provide a corrected expression for the erroneous ratio in the paper of Andrieu and Doucet.

stat.CO↗

arXiv2012

Bayesian Subset Simulation: a kriging-based subset simulation algorithm for the estimation of small probabilities of failure

The estimation of small probabilities of failure from computer simulations is a classical problem in engineering, and the Subset Simulation algorithm proposed by Au & Beck (Prob. Eng. Mech., 2001) has become one of the most popular method to solve it. Subset simulation has been shown to provide significant savings in the number of simulations to achieve a given accuracy of estimation, with respect to many other Monte Carlo approaches. The number of simulations remains still quite high however, and this method can be impractical for applications where an expensive-to-evaluate computer model is involved. We propose a new algorithm, called Bayesian Subset Simulation, that takes the best from the Subset Simulation algorithm and from sequential Bayesian methods based on kriging (also known as Gaussian process modeling). The performance of this new algorithm is illustrated using a test case from the literature. We are able to report promising results. In addition, we provide a numerical study of the statistical properties of the estimator.

stat.CO↗

arXiv2012

Sequential design of computer experiments for the estimation of a probability of failure

This paper deals with the problem of estimating the volume of the excursion set of a function $f:\mathbb{R}^d \to \mathbb{R}$ above a given threshold, under a probability measure on $\mathbb{R}^d$ that is assumed to be known. In the industrial world, this corresponds to the problem of estimating a probability of failure of a system. When only an expensive-to-simulate model of the system is available, the budget for simulations is usually severely limited and therefore classical Monte Carlo methods ought to be avoided. One of the main contributions of this article is to derive SUR (stepwise uncertainty reduction) strategies from a Bayesian-theoretic formulation of the problem of estimating a probability of failure. These sequential strategies use a Gaussian process model of $f$ and aim at performing evaluations of $f$ as efficiently as possible to infer the value of the probability of failure. We compare these strategies to other strategies also based on a Gaussian process model for estimating a probability of failure.

stat.CO↗

arXiv2011

Summarizing posterior distributions in signal decomposition problems when the number of components is unknown

This paper addresses the problem of summarizing the posterior distributions that typically arise, in a Bayesian framework, when dealing with signal decomposition problems with unknown number of components. Such posterior distributions are defined over union of subspaces of differing dimensionality and can be sampled from using modern Monte Carlo techniques, for instance the increasingly popular RJ-MCMC method. No generic approach is available, however, to summarize the resulting variable-dimensional samples and extract from them component-specific parameters. We propose a novel approach to this problem, which consists in approximating the complex posterior of interest by a "simple"---but still variable-dimensional---parametric distribution. The distance between the two distributions is measured using the Kullback-Leibler divergence, and a Stochastic EM-type algorithm, driven by the RJ-MCMC sampler, is proposed to estimate the parameters. The proposed algorithm is illustrated on the fundamental signal processing example of joint detection and estimation of sinusoids in white Gaussian noise.

stat.CO↗

arXiv2011

Bayesian optimization using sequential Monte Carlo

We consider the problem of optimizing a real-valued continuous function $f$ using a Bayesian approach, where the evaluations of $f$ are chosen sequentially by combining prior information about $f$, which is described by a random process model, and past evaluation results. The main difficulty with this approach is to be able to compute the posterior distributions of quantities of interest which are used to choose evaluation points. In this article, we decide to use a Sequential Monte Carlo (SMC) approach.

math.OC↗

arXiv2011

Sequential search based on kriging: convergence analysis of some algorithms

Let $\FF$ be a set of real-valued functions on a set $\XX$ and let $S:\FF \to \GG$ be an arbitrary mapping. We consider the problem of making inference about $S(f)$, with $f\in\FF$ unknown, from a finite set of pointwise evaluations of $f$. We are mainly interested in the problems of approximation and optimization. In this article, we make a brief review of results concerning average error bounds of Bayesian search methods that use a random process prior about $f$.

math.ST↗

arXiv2010

A unifying formulation of the Fokker-Planck-Kolmogorov equation for general stochastic hybrid systems (extended version)

This paper has been withdrawn from the arXiv. It is now published by Elsevier in Nonlinear Analysis: Hybrid Systems, see http://dx.doi.org/10.1016/j.nahs.2009.07.008 . A general formulation of the Fokker-Planck-Kolmogorov (FPK) equation for stochastic hybrid systems is presented, within the framework of Generalized Stochastic Hybrid Systems (GSHS). The FPK equation describes the time evolution of the probability law of the hybrid state. Our derivation is based on the concept of mean jump intensity, which is related to both the usual stochastic intensity (in the case of spontaneous jumps) and the notion of probability current (in the case of forced jumps). This work unifies all previously known instances of the FPK equation for stochastic hybrid systems, and provides GSHS practitioners with a tool to derive the correct evolution equation for the probability law of the state in any given example.

math.PR↗

arXiv2010

Convergence properties of the expected improvement algorithm

This paper has been withdrawn from the arXiv. It is now published by Elsevier in the Journal of Statistical Planning and Inference, under the modified title "Convergence properties of the expected improvement algorithm with fixed mean and covariance functions". See http://dx.doi.org/10.1016/j.jspi.2010.04.018 An author-generated post-print version is available from the HAL repository of SUPELEC at http://hal-supelec.archives-ouvertes.fr/hal-00217562 Abstract : "This paper deals with the convergence of the expected improvement algorithm, a popular global optimization algorithm based on a Gaussian process model of the function to be optimized. The first result is that under some mild hypotheses on the covariance function k of the Gaussian process, the expected improvement algorithm produces a dense sequence of evaluation points in the search domain, when the function to be optimized is in the reproducing kernel Hilbert space generated by k. The second result states that the density property also holds for P-almost all continuous functions, where P is the (prior) probability distribution induced by the Gaussian process."

stat.CO↗

arXiv2009

Pointwise consistency of the kriging predictor with known mean and covariance functions

This paper deals with several issues related to the pointwise consistency of the kriging predictor when the mean and the covariance functions are known. These questions are of general importance in the context of computer experiments. The analysis is based on the properties of approximations in reproducing kernel Hilbert spaces. We fix an erroneous claim of Yakowitz and Szidarovszky (J. Multivariate Analysis, 1985) that the kriging predictor is pointwise consistent for all continuous sample paths under some assumptions.

math.ST↗