arXiv subjects

Victor Picheny

Publications and source records attributed to Victor Picheny.

32 records · Page 2Linked to original sources

arXiv2020

Targeting Solutions in Bayesian Multi-Objective Optimization: Sequential and Batch Versions

Multi-objective optimization aims at finding trade-off solutions to conflicting objectives. These constitute the Pareto optimal set. In the context of expensive-to-evaluate functions, it is impossible and often non-informative to look for the entire set. As an end-user would typically prefer a certain part of the objective space, we modify the Bayesian multi-objective optimization algorithm which uses Gaussian Processes to maximize the Expected Hypervolume Improvement, to focus the search in the preferred region. The cumulated effects of the Gaussian Processes and the targeting strategy lead to a particularly efficient convergence to the desired part of the Pareto set. To take advantage of parallel computing, a multi-point extension of the targeting criterion is proposed and analyzed.

stat.ML↗

arXiv2020

A Review on Quantile Regression for Stochastic Computer Experiments

We report on an empirical study of the main strategies for quantile regression in the context of stochastic computer experiments. To ensure adequate diversity, six metamodels are presented, divided into three categories based on order statistics, functional approaches, and those of Bayesian inspiration. The metamodels are tested on several problems characterized by the size of the training set, the input dimension, the signal-to-noise ratio and the value of the probability density function at the targeted quantile. The metamodels studied reveal good contrasts in our set of experiments, enabling several patterns to be extracted. Based on our results, guidelines are proposed to allow users to select the best method for a given problem.

stat.ML↗

arXiv2019

Ordinal Bayesian Optimisation

Bayesian optimisation is a powerful tool to solve expensive black-box problems, but fails when the stationary assumption made on the objective function is strongly violated, which is the case in particular for ill-conditioned or discontinuous objectives. We tackle this problem by proposing a new Bayesian optimisation framework that only considers the ordering of variables, both in the input and output spaces, to fit a Gaussian process in a latent space. By doing so, our approach is agnostic to the original metrics on the original spaces. We propose two algorithms, respectively based on an optimistic strategy and on Thompson sampling. For the optimistic strategy we prove an optimal performance under the measure of regret in the latent space. We illustrate the capability of our framework on several challenging toy problems.

stat.ML↗

arXiv2019

The Kalai-Smorodinski solution for many-objective Bayesian optimization

An ongoing aim of research in multiobjective Bayesian optimization is to extend its applicability to a large number of objectives. While coping with a limited budget of evaluations, recovering the set of optimal compromise solutions generally requires numerous observations and is less interpretable since this set tends to grow larger with the number of objectives. We thus propose to focus on a specific solution originating from game theory, the Kalai-Smorodinsky solution, which possesses attractive properties. In particular, it ensures equal marginal gains over all objectives. We further make it insensitive to a monotonic transformation of the objectives by considering the objectives in the copula space. A novel tailored algorithm is proposed to search for the solution, in the form of a Bayesian optimization algorithm: sequential sampling decisions are made based on acquisition functions that derive from an instrumental Gaussian process prior. Our approach is tested on four problems with respectively four, six, eight, and nine objectives. The method is available in the Rpackage GPGame available on CRAN at https://cran.r-project.org/package=GPGame.

math.OC↗

arXiv2019

Budgeted Multi-Objective Optimization with a Focus on the Central Part of the Pareto Front -- Extended Version

Optimizing nonlinear systems involving expensive computer experiments with regard to conflicting objectives is a common challenge. When the number of experiments is severely restricted and/or when the number of objectives increases, uncovering the whole set of Pareto optimal solutions is out of reach, even for surrogate-based approaches: the proposed solutions are sub-optimal or do not cover the front well. As non-compromising optimal solutions have usually little point in applications, this work restricts the search to solutions that are close to the Pareto front center. The article starts by characterizing this center, which is defined for any type of front. Next, a Bayesian multi-objective optimization method for directing the search towards it is proposed. Targeting a subset of the Pareto front allows an improved optimality of the solutions and a better coverage of this zone, which is our main concern. A criterion for detecting convergence to the center is described. If the criterion is triggered, a widened central part of the Pareto front is targeted such that sufficiently accurate convergence to it is forecasted within the remaining budget. Numerical experiments show how the resulting algorithm, C-EHI, better locates the central part of the Pareto front when compared to state-of-the-art Bayesian algorithms.

stat.ML↗

arXiv2018

Interpretable sparse SIR for functional data

This work focuses on the issue of variable selection in functional regression. Unlike most work in this framework, our approach does not select isolated points in the definition domain of the predictors, nor does it rely on the expansion of the predictors in a given functional basis. It provides an approach to select full intervals made of consecutive points. This feature improves the interpretability of the estimated coefficients and is desirable in the functional framework for which small shifts are frequent when comparing one predictor (curve) to another. Our method is described in a semiparametric framework based on Sliced Inverse Regression (SIR). SIR is an effective method for dimension reduction of high-dimensional data which computes a linear projection of the predictors in a low-dimensional space, without loss on regression information. We extend the approaches of variable selection developed for multidimensional SIR to select intervals rather than separated evaluation points in the definition domain of the functional predictors. Different and equivalent formulations of SIR are combined in a shrinkage approach with a group-LASSO-like penalty. Finally, a fully automated iterative procedure is also proposed to find the critical (interpretable) intervals. The approach is proved efficient on simulated and real data.

math.ST↗

arXiv2018

A Bayesian optimization approach to find Nash equilibria

Game theory finds nowadays a broad range of applications in engineering and machine learning. However, in a derivative-free, expensive black-box context, very few algorithmic solutions are available to find game equilibria. Here, we propose a novel Gaussian-process based approach for solving games in this context. We follow a classical Bayesian optimization framework, with sequential sampling decisions based on acquisition functions. Two strategies are proposed, based either on the probability of achieving equilibrium or on the Stepwise Uncertainty Reduction paradigm. Practical and numerical aspects are discussed in order to enhance the scalability and reduce computation time. Our approach is evaluated on several synthetic game problems with varying number of players and decision space dimensions. We show that equilibria can be found reliably for a fraction of the cost (in terms of black-box evaluations) compared to classical, derivative-based algorithms. The method is available in the R package GPGame available on CRAN at https://cran.r-project.org/package=GPGame.

stat.ML↗

arXiv2017

Inferring large graphs using l1-penalized likelihood

We address the issue of recovering the structure of large sparse directed acyclic graphs from noisy observations of the system. We propose a novel procedure based on a specific formulation of the l1-norm regularized maximum likelihood, which decomposes the graph estimation into two optimization sub-problems: topological structure and node order learning. We provide oracle inequalities for the graph estimator, as well as an algorithm to solve the induced optimization problem, in the form of a convex program embedded in a genetic algorithm. We apply our method to various data sets (including data from the DREAM4 challenge) and show that it compares favorably to state-of-the-art methods.

math.ST↗

arXiv2017

Using numerical plant models and phenotypic correlation space to design achievable ideotypes

Numerical plant models can predict the outcome of plant traits modifications resulting from genetic variations, on plant performance, by simulating physiological processes and their interaction with the environment. Optimization methods complement those models to design ideotypes, i.e. ideal values of a set of plant traits resulting in optimal adaptation for given combinations of environment and management, mainly through the maximization of a performance criteria (e.g. yield, light interception). As use of simulation models gains momentum in plant breeding, numerical experiments must be carefully engineered to provide accurate and attainable results, rooting them in biological reality. Here, we propose a multi-objective optimization formulation that includes a metric of performance, returned by the numerical model, and a metric of feasibility, accounting for correlations between traits based on field observations. We applied this approach to two contrasting models: a process-based crop model of sunflower and a functional-structural plant model of apple trees. In both cases, the method successfully characterized key plant traits and identified a continuum of optimal solutions, ranging from the most feasible to the most efficient. The present study thus provides successful proof of concept for this enhanced modeling approach, which identified paths for desirable trait modification, including direction and intensity.

q-bio.QM↗

arXiv2016

Bayesian optimization under mixed constraints with a slack-variable augmented Lagrangian

An augmented Lagrangian (AL) can convert a constrained optimization problem into a sequence of simpler (e.g., unconstrained) problems, which are then usually solved with local solvers. Recently, surrogate-based Bayesian optimization (BO) sub-solvers have been successfully deployed in the AL framework for a more global search in the presence of inequality constraints; however, a drawback was that expected improvement (EI) evaluations relied on Monte Carlo. Here we introduce an alternative slack variable AL, and show that in this formulation the EI may be evaluated with library routines. The slack variables furthermore facilitate equality as well as inequality constraints, and mixtures thereof. We show how our new slack "ALBO" compares favorably to the original. Its superiority over conventional alternatives is reinforced on several mixed constraint examples.

stat.CO↗

arXiv2015

Sunflower phenotype optimization under climatic uncertainties using crop models

Accounting for the annual climatic variability is a well-known issue for simulation-based studies of environmental models. It often requires intensive sampling (e.g., averaging the simulation outputs over many climatic series), which hinders many sequential processes, in particular optimization algorithms. We propose here an approach based on a subset selection of a large basis of climatic series, using an ad-hoc similarity function and clustering. A non-parametric reconstruction technique is introduced to estimate accurately the distribution of the output of interest using only the subset sampling. The proposed strategy is non-intrusive and generic (i.e. transposable to most models with climatic data inputs), and can be combined to most "off-the-shelf" optimization solvers. We apply our approach to sunflower phenotype optimization using the crop model SUNFLO. The underlying optimization problem is formulated as multi-objective to account for risk-aversion. Our approach achieves good performances even for limited computational budgets, outperforming significantly more "naive" strategies.

math.OC↗

arXiv2014

Increased genetic diversity improves crop yield stability under climate variability: a computational study on sunflower

A crop can be represented as a biotechnical system in which components are either chosen (cultivar, management) or given (soil, climate) and whose combination generates highly variable stress patterns and yield responses. Here, we used modeling and simulation to predict the crop phenotypic plasticity resulting from the interaction of plant traits (G), climatic variability (E) and management actions (M). We designed two in silico experiments that compared existing and virtual sunflower cultivars (Helianthus annuus L.) in a target population of cropping environments by simulating a range of indicators of crop performance. Optimization methods were then used to search for GEM combinations that matched desired crop specifications. Computational experiments showed that the fit of particular cultivars in specific environments is gradually increasing with the knowledge of pedo-climatic conditions. At the regional scale, tuning the choice of cultivar impacted crop performance the same magnitude as the effect of yearly genetic progress made by breeding. When considering virtual genetic material, designed by recombining plant traits, cultivar choice had a greater positive impact on crop performance and stability. Results suggested that breeding for key traits conferring plant plasticity improved cultivar global adaptation capacity whereas increasing genetic diversity allowed to choose cultivars with distinctive traits that were more adapted to specific conditions. Consequently, breeding genetic material that is both plastic and diverse may improve yield stability of agricultural systems exposed to climatic variability. We argue that process-based modeling could help enhancing spatial management of cultivated genetic diversity and could be integrated in functional breeding approaches.

q-bio.PE↗

arXiv2013

Multiobjective optimization using Gaussian process emulators via stepwise uncertainty reduction

Optimization of expensive computer models with the help of Gaussian process emulators in now commonplace. However, when several (competing) objectives are considered, choosing an appropriate sampling strategy remains an open question. We present here a new algorithm based on stepwise uncertainty reduction principles to address this issue. Optimization is seen as a sequential reduction of the volume of the excursion sets below the current best solutions, and our sampling strategy chooses the points that give the highest expected reduction. Closed-form formulae are provided to compute the sampling criterion, avoiding the use of cumbersome simulations. We test our method on numerical examples, showing that it provides an efficient trade-off between exploration and intensification.

math.OC↗

arXiv2012

Sequential design of computer experiments for the estimation of a probability of failure

This paper deals with the problem of estimating the volume of the excursion set of a function $f:\mathbb{R}^d \to \mathbb{R}$ above a given threshold, under a probability measure on $\mathbb{R}^d$ that is assumed to be known. In the industrial world, this corresponds to the problem of estimating a probability of failure of a system. When only an expensive-to-simulate model of the system is available, the budget for simulations is usually severely limited and therefore classical Monte Carlo methods ought to be avoided. One of the main contributions of this article is to derive SUR (stepwise uncertainty reduction) strategies from a Bayesian-theoretic formulation of the problem of estimating a probability of failure. These sequential strategies use a Gaussian process model of $f$ and aim at performing evaluations of $f$ as efficiently as possible to infer the value of the probability of failure. We compare these strategies to other strategies also based on a Gaussian process model for estimating a probability of failure.

stat.CO↗