Two-stage Design for Failure Probability Estimation with Gaussian Process Surrogates

Abstract

We tackle the problem of quantifying failure probabilities for expensive deterministic computer experiments with stochastic inputs under a fixed budget. The computational cost of the computer simulation prohibits direct Monte Carlo (MC) and necessitates a surrogate model, which may facilitate either a "surrogate MC" estimator or a surrogate-informed importance sampling estimator. We embrace the former, finding importance sampling too variable when budgets are limited, and propose a novel design strategy to effectively train a surrogate for the purpose of failure probability estimation. Existing works exhaust the entire evaluation budget on active learning through sequential contour location (CL), attempting to balance exploration with exploitation of the failure contour throughout the design, but we find exhaustive CL to be suboptimal. Instead we propose a novel two-stage surrogate design. In Stage 1, we conduct sequential CL to locate the failure contour. In Stage 2, once surrogate learning has saturated, we use a solely exploitative strategy -- allocating the remaining evaluation budget to MC samples with the highest classification entropy to ensure they are classified correctly. We propose a stopping criterion to determine the transition between stages without any tuning. Our two-stage design outperforms alternatives, including exhaustive CL and surrogate-informed importance sampling, on a variety of benchmark exercises. With these tools, we are able to effectively estimate small failure probabilities with only hundreds of simulator evaluations, showcasing functionality with both shallow and deep Gaussian process surrogates, and deploying our method on a simulation of fluid flow around an airfoil.