arXiv subjects

Charles Audet

Publications and source records attributed to Charles Audet.

At least 19 recordsLinked to original sources

arXiv2026

Adaptive direct search algorithms with relaxable and quantifiable constraints

This work introduces ADS-PB, an extension of the Adaptive Direct Search (ADS) framework for solving constrained blackbox optimization problems. With ADS, iterates progress without relying on mesh structures or sufficient decrease conditions on the objective function value. Unlike the extreme barrier approach used in ADS, where only unrelaxable constraints are considered, the proposed method also handles quantifiable and relaxable constraints using a Progressive Barrier (PB) mechanism that exploits both constraint and objective function values. A convergence analysis of the proposed framework under mild assumptions is presented. The performance of the proposed method is assessed using sets of analytical and simulation-based constrained test problems and is compared with state-of-the-art blackbox optimization solvers, including the PB approach within the Mesh Adaptive Direct Search (MADS) framework.

math.OC↗

arXiv2026

Benchmarking Bilevel Derivative-Free Optimization Algorithms

Bilevel optimization involves an upper-level and a lower-level decision maker. The lower-level optimization problem is nested within the constraints of the upper-level one. A point is said to be admissible for the bilevel problem if it satisfies all constraints and is optimal for the lower-level decision-maker. Bilevel derivative-free optimization (BL-DFO) algorithms address bilevel optimization problems in which either the upper-level or the lower-level problem is solved using a derivative-free optimization method. In this context, existing BL-DFO benchmarking techniques often do not rigorously validate the admissibility of proposed solutions, and do not adequately account for the computational effort deployed by the upper- and lower-level solvers. This work proposes a benchmarking methodology for BL-DFO algorithms. A post-optimization procedure, named refereeing procedure, is introduced to discard non-admissible points and ensure a fair comparison between the algorithms. The computational effort deployed by upper- and lower-level solvers are also taken into account into the overall computational cost. Numerical experiments illustrate the benchmarking methodology.

math.OC↗

arXiv2026

Surrogate-based categorical neighborhoods for mixed-variable blackbox optimization

In simulation-based engineering, design choices are often obtained following the optimization of complex blackbox models. These models frequently involve mixed-variable domains with quantitative and categorical variables. Unlike quantitative variables, categorical variables lack an inherent structure, which makes them difficult to handle, especially in the presence of constraints. This work proposes a systematic approach to structure and model categorical variables in constrained mixed-variable blackbox optimization. Surrogate models of the objective and constraint functions are used to induce problem-specific categorical distances. From these distances, surrogate-based neighborhoods are constructed using notions of dominance from bi-objective optimization, jointly accounting for information from both the objective and the constraint functions. This study addresses the lack of automatic and constraint-aware categorical neighborhood construction in mixed-variable blackbox optimization. As a proof of concept, these neighborhoods are employed within CatMADS, an extension of the MADS algorithm for categorical variables. The surrogate models are Gaussian processes, and the resulting method is called CatMADS-GP. The method is benchmarked on the Cat-Suite collection of 60 mixed-variable optimization problems and compared against state-of-the-art solvers. Data profiles indicate that CatMADS-GP achieves superior performance for both unconstrained and constrained problems.

math.OC↗

arXiv2026

A penalty-interior point method combined with MADS for equality and inequality constrained optimization

This work introduces MADS-PIP, an efficient framework that integrates a penalty-interior point strategy into the mesh adaptive direct search (MADS) algorithm for solving nonsmooth blackbox optimization problems with general inequality and equality constraints. Inequality constraints are partitioned into two subsets: one treated via a logarithmic barrier applied to an aggregated interior constraint violation, and the other handled through an exterior quadratic penalty. All equality constraints are treated by the exterior penalty. A merit function defines a sequence of unconstrained subproblems, which are solved approximately using MADS, while a carefully designed update rule drives the penalty-barrier parameter to zero. In the nonsmooth setting, we establish convergence results ensuring feasibility for general constraints as well as Clarke stationarity for inequality-constrained problems. Computational experiments on both analytical test sets and challenging blackbox problems demonstrate that the proposed MADS-PIP algorithm is competitive with, and often outperforms, MADS with the progressive barrier strategy, particularly in the presence of equality constraints.

math.OC↗

arXiv2026

Multi-fidelity constraints in blackbox optimization

This work studies constrained blackbox optimization problems that cannot be solved in reasonable time due to prohibitive computational costs. This challenge is especially prevalent in industrial applications, where blackbox evaluations are costly. However, constraints can be evaluated at various fidelities at a lower computational cost. More specifically, this work targets situations in which the infeasibility of each individual constraint can be detected at lower fidelities, and where a large discrete number of fidelities are available. Moreover, highly discontinuous problems which may fail to evaluate are considered, such that direct search methods are preferred to model-based ones. To this effect, the Interruptible Direct Search (IDS) and the Dynamic Interruptible Direct Search (DIDS) algorithms are proposed to leverage feasibility assessments from various fidelity levels to avoid high cost evaluations. The results show highly increased performances from NOMAD when it is paired with IDS or DIDS.

math.OC↗

arXiv2025

Adaptive direct search algorithms for constrained optimization

Two families of directional direct search methods have emerged in derivative-free and blackbox optimization (DFO and BBO), each based on distinct principles: Mesh Adaptive Direct Search (MADS) and Sufficient Decrease Direct Search (SDDS). MADS restricts trial points to a mesh and accepts any improvement, ensuring none are missed, but at the cost of restraining the placement of trial points. SDDS allows greater freedom by evaluating points anywhere in the space, but accepts only those yielding a sufficient decrease in the objective function value, which may lead to discarding improving points. This work introduces a new class of methods, Adaptive Direct Search (ADS), which uses a novel acceptance rule based on the so-called punctured space, avoiding both meshes and sufficient decrease conditions. ADS enables flexible search while addressing the limitations of MADS and SDDS, and retains the theoretical foundations of directional direct search. Computational results in constrained and unconstrained settings highlight its performance compared to both MADS and SDDS.

math.OC↗

arXiv2025

CatMADS: Mesh Adaptive Direct Search for constrained blackbox optimization with categorical variables

Solving optimization problems in which functions are blackboxes and variables involve different types poses significant theoretical and algorithmic challenges. Nevertheless, such settings frequently occur in simulation-based engineering design and machine learning. This paper extends the Mesh Adaptive Direct Search (MADS) algorithm to address mixed-variable problems with categorical, integer and continuous variables. MADS is a robust derivative-free optimization framework with a well-established convergence analysis for constrained quantitative problems. CatMADS generalizes MADS by incorporating categorical variables through distance-induced neighborhoods. A detailed convergence analysis of CatMADS is provided, with flexible choices balancing computational cost and local optimality strength. Four types of mixed-variable local minima are introduced, corresponding to progressively stronger notions of local optimality. CatMADS integrates the progressive barrier strategy for handling constraints, and ensures Clarke stationarity. An instance of \catmads employs cross-validation to construct problem-specific categorical distances. This instance is compared to state-of-the-art solvers on 32 mixed-variable problems, half of which are constrained. Data profiles show that CatMADS achieves the best results, demonstrating that the framework is empirically efficient in addition to having strong theoretical foundations.

math.OC↗

arXiv2024

Blackbox optimization for origami-inspired bistable structures

Bistable mechanical systems exhibit two stable configurations where the elastic energy is locally minimized. To realize such systems, origami techniques have been proposed as a versatile platform to design deployable structures with both compact and functional stable states. Conceptually, a bistable origami motif is composed of two-dimensional surfaces connected by one-dimensional fold lines. This leads to stable configurations exhibiting zero-energy local minima. Physically, origami-inspired structures are three-dimensional, comprising facets and hinges fabricated in a distinct stable state where residual stresses are minimized. This leads to the dominance of one stable state over the other. To improve mechanical performance, one can solve the constrained optimization problem of maximizing the bistability of origami structures, defined as the amount of elastic energy required to switch between stable states, while ensuring materials used for the facets and hinges remain within their elastic regime. In this study, the Mesh Adaptive Direct Search (MADS) algorithm, a blackbox optimization technique, is used to solve the constrained optimization problem. The bistable waterbomb-base origami motif is selected as a case-study to present the methodology. The elastic energy of this origami pattern under deployment is calculated via Finite Element simulations which serve as the blackbox in the MADS optimization loop. To validate the results, optimized waterbomb-base geometries are built via Fused Filament Fabrication and their response under loading is characterized experimentally on a Uniaxial Test Machine. Ultimately, our method offers a general framework for optimizing bistability in mechanical systems, presenting opportunities for advancement across various engineering applications.

math.OC↗

arXiv2024

A Partitioned Optimization Framework for Structure-Aware Problems

This work tackles a class of optimization problems in which fixing some well-chosen combinations of the variables makes the problem substantially easier to solve. We consider that the variables space may be partitioned into subsets that fix these combinations to given values, so that the restriction of the problem to any of the partition sets admits a tractable solution. Then, we exhibit a reformulation of the problem that consists in searching for the partition set index that minimizes the objective value of the solution to the restricted problem. We name partitioned optimization framework (POf) the formalization of this class of problems and this reformulation process. As we prove in this work, the POf allows solving the original problem by focusing on the reformulated problem: all solutions to the reformulated problem are partition indices for which the solution to the associated restricted problem is also a solution to the original problem. Second, we introduce a derivative-free partitioned optimization method (DFPOm) to efficiently solve problems that fit in the POf. We prove that the reformulated problem is nicely handled by derivative-free optimization (DFO) algorithms with a covering step. Then the DFPOm consists in solving the reformulated problem using such DFO algorithm with a covering step to obtain an optimal partition index, and to return the solution to the associated restricted problem as a solution to the initial problem. Finally, we illustrate how the \pof allows solving some classes of problems. We first focus on an infinite-dimensional case, by solving analytically an optimal control problem that challenges standard methods from the literature. Then, we apply the DFPOm on a class of finite-dimensional problems called composite greybox problems, and we highlight the gain in numerical performance provided by the DFPOm by comparing it to two popular DFO solvers.

math.OC↗

arXiv2024

solar: A solar thermal power plant simulator for blackbox optimization benchmarking

This work introduces solar, a collection of ten optimization problem instances for benchmarking blackbox optimization solvers. The instances present different design aspects of a concentrated solar power plant simulated by blackbox numerical models. The type of variables (discrete or continuous), dimensionality, and number and types of constraints (including hidden constraints) differ across instances. Some are deterministic, others are stochastic with possibilities to execute several replications to control stochasticity. Most instances offer variable fidelity surrogates, two are biobjective and one is unconstrained. The solar plant model takes into account various subsystems: a heliostats field, a central cavity receiver (the receiver), a molten salt thermal energy storage, a steam generator and an idealized power block. Several numerical methods are implemented throughout the solar code and most of the executions are time-consuming. Great care was applied to guarantee reproducibility across platforms. The solar tool encompasses most of the characteristics that can be found in industrial and real-life blackbox optimization problems, all in an open-source and stand-alone code.

math.OC↗

arXiv2024

A distance for mixed-variable and hierarchical domains with meta variables

Heterogeneous datasets emerge in various machine learning and optimization applications that feature different input sources, types or formats. Most models or methods do not natively tackle heterogeneity. Hence, such datasets are often partitioned into smaller and simpler ones, which may limit the generalizability or performance, especially when data is limited. The first main contribution of this work is a modeling framework that generalizes hierarchical, tree-structured, variable-size or conditional search frameworks. The framework models mixed-variable and hierarchical domains in which variables may be continuous, integer, or categorical, with some identified as meta when they influence the structure of the problem. The second main contribution is a novel distance that compares any pair of mixed-variable points that do not share the same variables, allowing to use whole heterogeneous datasets that reside in mixed-variable and hierarchical domains with meta variables. The contributions are illustrated through regression and classification experiments using simple distance-based models applied to datasets of hyperparameters with corresponding performance scores.

stat.ML↗

arXiv2024

The cosine measure relative to a subspace

The cosine measure was introduced in 2003 to quantify the richness of a finite positive spanning sets of directions in the context of derivative-free directional methods. A positive spanning set is a set of vectors whose nonnegative linear combinations span the whole space. The present work extends the definition of cosine measure. In particular, the paper studies cosine measures relative to a subspace, and proposes a deterministic algorithm to compute it. The paper also studies the situation in which the set of vectors is infinite. The extended definition of the cosine measure might be useful for subspace decomposition methods.

math.OC↗

arXiv2024

Convergence towards a local minimum by direct search methods with a covering step

This paper introduces a new step to the Direct Search Method (DSM) to strengthen its convergence analysis. By design, this so-called covering step may ensure that for all refined points of the sequence of incumbent solutions generated by the resulting cDSM (covering DSM), the set of all evaluated trial points is dense in a neighborhood of that refined point. We prove that this additional property guarantees that all refined points are local solutions to the optimization problem. This new result holds true even for discontinuous objective function, under a mild assumption that we discuss in details. We also provide a practical construction scheme for the covering step that works at low additional cost per iteration. Finally, we show that the covering step may be adapted to classes of algorithms differing from the DSM.

math.OC↗

arXiv2023

Fidelity and interruption control for expensive constrained multi-fidelity blackbox optimization

This work introduces a novel blackbox optimization algorithm for computationally expensive constrained multi-fidelity problems. When applying a direct search method to such problems, the scarcity of feasible points may lead to numerous costly evaluations spent on infeasible points. Our proposed fidelity and interruption controlled optimization algorithm addresses this issue by leveraging multi-fidelity information, allowing for premature interruption of an evaluation when a point is estimated to be infeasible. These estimations are controlled by a biadjacency matrix, for which we propose a construction. The proposed method acts as an intermediary component bridging any non multi-fidelity direct search solver and a multi-fidelity blackbox problem, giving the user freedom of choice for the solver. A series of computational tests are conducted to validate the approach. The results show a significant improvement in solution quality when an initial feasible starting point is provided. When this condition is not met, the outcomes are contingent upon specific properties of the blackbox.

math.OC↗

arXiv2023

Risk averse constrained blackbox optimization under mixed aleatory/epistemic uncertainties

This paper addresses risk averse constrained optimization problems where the objective and constraint functions can only be computed by a blackbox subject to unknown uncertainties. To handle mixed aleatory/epistemic uncertainties, the problem is transformed into a conditional value-at-risk (CVaR) constrained optimization problem. General inequality constraints are managed through Lagrangian relaxation. A convolution between a truncated Gaussian density and the Lagrangian function is used to smooth the problem. A gradient estimator of the smooth Lagrangian function is derived, possessing attractive properties: it estimates the gradient with only two outputs of the blackbox, regardless of dimension, and evaluates the blackbox only within the bound constraints. This gradient estimator is then utilized in a multi-timescale stochastic approximation algorithm to solve the smooth problem. Under mild assumptions, this algorithm almost surely converges to a feasible point of the CVaR-constrained problem whose objective function value is arbitrarily close to that of a local solution. Finally, numerical experiments are conducted to serve three purposes. Firstly, they provide insights on how to set the hyperparameter values of the algorithm. Secondly, they demonstrate the effectiveness of the algorithm when a truncated Gaussian gradient estimator is used. Lastly, they show its ability to handle mixed aleatory/epistemic uncertainties in practical applications.

math.OC↗

arXiv2023

Sequential stochastic blackbox optimization with zeroth-order gradient estimators

This work considers stochastic optimization problems in which the objective function values can only be computed by a blackbox corrupted by some random noise following an unknown distribution. The proposed method is based on sequential stochastic optimization (SSO): the original problem is decomposed into a sequence of subproblems. Each of these subproblems is solved using a zeroth order version of a sign stochastic gradient descent with momentum algorithm (ZO-Signum) and with an increasingly fine precision. This decomposition allows a good exploration of the space while maintaining the efficiency of the algorithm once it gets close to the solution. Under Lipschitz continuity assumption on the blackbox, a convergence rate in expectation is derived for the ZO-signum algorithm. Moreover, if the blackbox is smooth and convex or locally convex around its minima, a convergence rate to an $\epsilon$-optimal point of the problem may be obtained for the SSO algorithm. Numerical experiments are conducted to compare the SSO algorithm with other state-of-the-art algorithms and to demonstrate its competitiveness.

math.OC↗

arXiv2022

Erratum, counterexample and an additional revealing poll step for a result of "Analysis of direct searches for discontinuous functions''

This note provides a counterexample to a theorem announced in the last part of the paper ''Analysis of direct searches for discontinuous functions'', Mathematical Programming Vol. 133, pp.~299--325, 2012. The counterexample involves an objective function $f: \mathbb{R} \to \mathbb{R}$ which satisfies all the assumptions required by the theorem but contradicts some of its conclusions. A corollary of this theorem is also affected by this counterexample. The main flaw revealed by the counterexample is the possibility that a directional direct search method (dDSM) generates a sequence of trial points $(x_k)_k$ converging to a point $x_*$ where $f$ is discontinuous and whose objective function value $f(x_*)$ is strictly less than $\lim_{k\to\infty} f(x_k)$. Moreover the dDSM generates no trial point in one of the two branches of $f$ near $x_*$. This note also investigates the proof of the theorem to highlight the inexact statements in the original paper. Finally this work concludes with a modification of the dDSM that allows to recover the properties broken by the counterexample.

math.OC↗

arXiv2022

A general mathematical framework for constrained mixed-variable blackbox optimization problems with meta and categorical variables

A mathematical framework for modelling constrained mixed-variable optimization problems is presented in a blackbox optimization context. The framework introduces a new notation and allows solution strategies. The notation framework allows meta and categorical variables to be explicitly and efficiently modelled, which facilitates the solution of such problems. The new term meta variables is used to describe variables that influence which variables are acting or nonacting: meta variables may affect the number of variables and constraints. The flexibility of the solution strategies supports the main blackbox mixed-variable optimization approaches: direct search methods and surrogate-based methods (Bayesian optimization). The notation system and solution strategies are illustrated through an example of a hyperparameter optimization problem from the machine learning community.

math.OC↗