arXiv subjects

Sébastien Le Digabel

Publications and source records attributed to Sébastien Le Digabel.

28 records · Page 2Linked to original sources

arXiv2021

NOMAD version 4: Nonlinear optimization with the MADS algorithm

NOMAD is software for optimizing blackbox problems. In continuous development since 2001, it constantly evolved with the integration of new algorithmic features published in scientific publications. These features are motivated by real applications encountered by industrial partners. The latest major release of NOMAD, version 3, dates from 2008. Minor releases are produced as new features are incorporated. The present work describes NOMAD 4, a complete redesign of the previous version, with a new architecture providing more flexible code, added functionalities and reusable code. We introduce algorithmic components, which are building blocks for more complex algorithms, and can initiate other components, launch nested algorithms, or perform specialized tasks. They facilitate the implementation of new ideas, including the MegaSearchPoll component, warm and hot restarts, and a revised version of the PSD-MADS algorithm. Another main improvement of NOMAD 4 is the usage of parallelism, to simultaneously compute multiple blackbox evaluations, and to maximize usage of available cores. Running different algorithms, tuning their parameters, and comparing their performance for optimization is simpler than before, while overall optimization performance is maintained between versions 3 and 4. NOMAD is freely available at www.gerad.ca/nomad and the whole project is visible at github.com/bbopt/nomad.

math.OC↗

arXiv2021

Use of static surrogates in hyperparameter optimization

Optimizing the hyperparameters and architecture of a neural network is a long yet necessary phase in the development of any new application. This consuming process can benefit from the elaboration of strategies designed to quickly discard low quality configurations and focus on more promising candidates. This work aims at enhancing HyperNOMAD, a library that adapts a direct search derivative-free optimization algorithm to tune both the architecture and the training of a neural network simultaneously, by targeting two keys steps of its execution and exploiting cheap approximations in the form of static surrogates to trigger the early stopping of the evaluation of a configuration and the ranking of pools of candidates. These additions to HyperNOMAD are shown to improve on its resources consumption without harming the quality of the proposed solutions.

math.OC↗

arXiv2021

CONICOPF: Conic relaxations for AC optimal power flow computations

Computational speed and global optimality are key needs for practical algorithms for the optimal power flow problem. Two convex relaxations offer a favorable trade-off between the standard second-order cone and the standard semidefinite relaxations for large-scale meshed networks in terms of optimality gap and computation time: the tight-and-cheap relaxation (TCR) and the quadratic convex relaxation (QCR). We compare these relaxations on 60 PGLib-OPF test cases with up to 1,354 buses under three operating conditions and show that TCR dominates QCR on all 20 typical (TYP) test cases, on 18 out of 20 active power increase (API), and 12 out of 20 small angle difference (SAD). Selected state-of-the-art conic relaxations are implemented in the new MATLAB-based package CONICOPF available on GitHub.

math.OC↗

arXiv2020

Constrained stochastic blackbox optimization using a progressive barrier and probabilistic estimates

This work introduces the StoMADS-PB algorithm for constrained stochastic blackbox optimization, which is an extension of the mesh adaptive direct-search (MADS) method originally developed for deterministic blackbox optimization under general constraints. The values of the objective and constraint functions are provided by a noisy blackbox, i.e., they can only be computed with random noise whose distribution is unknown. As in MADS, constraint violations are aggregated into a single constraint violation function. Since all functions values are numerically unavailable, StoMADS-PB uses estimates and introduces so-called probabilistic bounds for the violation. Such estimates and bounds obtained from stochastic observations are required to be accurate and reliable with high but fixed probabilities. The proposed method, which allows intermediate infeasible iterates, accepts new points using sufficient decrease conditions and imposing a threshold on the probabilistic bounds. Using Clarke nonsmooth calculus and martingale theory, Clarke stationarity convergence results for the objective and the violation function are derived with probability one.

math.OC↗

arXiv2019

Optimization of noisy blackboxes with adaptive precision

In derivative-free and blackbox optimization, the objective function is often evaluated through the execution of a computer program seen as a blackbox. It can be noisy, in the sense that its outputs are contaminated by random errors. Sometimes, the source of these errors is identified and controllable, in the sense that it is possible to reduce the standard deviation of the stochastic noise it generates. A common strategy to deal with such a situation is to monotonically diminish this standard deviation, to asymptotically make it converge to zero and ensure convergence of algorithms because the noise is dismantled. This work presents MpMads, an algorithm which follows this approach. However, in practice a reduction of the standard deviation increases the computation time, and makes the optimization process long. Therefore, a second algorithm called DpMads is introduced to explore another strategy, which does not force the standard deviation to monotonically diminish. Although these strategies are proved to be theoretically equivalents, tests on analytical problems and an industrial blackbox are presented to illustrate practical differences.

math.OC↗

arXiv2019

StoMADS: Stochastic blackbox optimization using probabilistic estimates

This work introduces StoMADS, a stochastic variant of the mesh adaptive direct-search (MADS) algorithm originally developed for deterministic blackbox optimization. StoMADS considers the unconstrained optimization of an objective function f whose values can be computed only through a blackbox corrupted by some random noise following an unknown distribution. The proposed method is based on an algorithmic framework similar to that of MADS and uses random estimates of function values obtained from stochastic observations since the exact deterministic computable version of f is not available. Such estimates are required to be accurate with a sufficiently large but fixed probability and satisfy a variance condition. The ability of the proposed algorithm to generate an asymptotically dense set of search directions is then exploited to show convergence to a Clarke stationary point of f with probability one, using martingale theory.

math.OC↗

arXiv2019

Tight-and-cheap conic relaxation for the AC optimal power flow problem

The classical alternating current optimal power flow problem is highly nonconvex and generally hard to solve. Convex relaxations, in particular semidefinite, second-order cone, convex quadratic, and linear relaxations, have recently attracted significant interest. The semidefinite relaxation is the strongest among them and is exact for many cases. However, the computational efficiency for solving large-scale semidefinite optimization is lower than for second-order cone optimization. We propose a conic relaxation obtained by combining semidefinite optimization with the reformulation-linearization technique, commonly known as RLT. The proposed relaxation is stronger than the second-order cone relaxation and nearly as tight as the standard semidefinite relaxation. Computational experiments using standard test cases with up to 6515 buses show that the time to solve the new conic relaxation is up to one order of magnitude lower than for the chordal relaxation, a semidefinite relaxation technique that exploits the sparsity of power networks.

math.OC↗

arXiv2019

HyperNOMAD: Hyperparameter optimization of deep neural networks using mesh adaptive direct search

The performance of deep neural networks is highly sensitive to the choice of the hyperparameters that define the structure of the network and the learning process. When facing a new application, tuning a deep neural network is a tedious and time consuming process that is often described as a "dark art". This explains the necessity of automating the calibration of these hyperparameters. Derivative-free optimization is a field that develops methods designed to optimize time consuming functions without relying on derivatives. This work introduces the HyperNOMAD package, an extension of the NOMAD software that applies the MADS algorithm [7] to simultaneously tune the hyperparameters responsible for both the architecture and the learning process of a deep neural network (DNN), and that allows for an important flexibility in the exploration of the search space by taking advantage of categorical variables. This new approach is tested on the MNIST and CIFAR-10 data sets and achieves results comparable to the current state of the art.

cs.LG↗

arXiv2019

Tight-and-cheap conic relaxation for the optimal reactive power dispatch problem

The optimal reactive power dispatch (ORPD) problem is an alternating current optimal power flow (ACOPF) problem where discrete control devices for regulating the reactive power, such as shunt elements and tap changers, are considered. The ORPD problem is modelled as a mixed-integer nonlinear optimization problem and its complexity is increased compared to the ACOPF problem, which is highly nonconvex and generally hard to solve. Recently, convex relaxations of the ACOPF problem have attracted a significant interest since they can lead to global optimality. We propose a tight conic relaxation of the ORPD problem and show that a round-off technique applied with this relaxation leads to near-global optimal solutions with very small guaranteed optimality gaps, unlike with the nonconvex continuous relaxation. We report computational results on selected MATPOWER test cases with up to 3375 buses.

math.OC↗

arXiv2015

A Taxonomy of Constraints in Simulation-Based Optimization

The types of constraints encountered in black-box and simulation-based optimization problems differ significantly from those treated in nonlinear programming. We introduce a characterization of constraints to address this situation. We provide formal definitions for several constraint classes and present illustrative examples in the context of the resulting taxonomy. This taxonomy, denoted QRAK, is useful for modeling and problem formulation, as well as optimization software development and deployment. It can also be used as the basis for a dialog with practitioners in moving problems to increasingly solvable branches of optimization.

math.OC↗