arXiv subjects

Edward Hallé-Hannan

Publications and source records attributed to Edward Hallé-Hannan.

6 recordsLinked to original sources

arXiv2026

NomadBBO: A direct-search framework for constrained mixed-variable blackbox optimization in Python

Mixed-variable blackbox optimization arises in simulation-based engineering and machine learning, where objective and constraint functions are expensive to evaluate and variables may be continuous, integer, binary, or categorical. Several software libraries are available for derivative-free optimization, but few combine ease of use, computational efficiency, flexibility, and theoretical convergence guarantees. This work presents NomadBBO, a user-friendly Python library for inequality-constrained mixed-variable blackbox optimization built around Nomad. Its core optimization framework is CatADS, a new direct-search method that extends Adaptive Direct Search (ADS) to mixed-variable problems. CatADS combines the mechanisms of ADS for quantitative variables with neighborhoods for handling categorical variables. These neighborhoods use Gaussian process-based or empirical Wasserstein distances derived from available data. Inequality constraints are handled through the progressive barrier. The mixed-variable and surrogate-based mechanisms are implemented in Python and connected to the efficient C++ backend of Nomad via Cython. This integration allows Nomad to handle categorical variables. It also enables the use of external libraries and hybrid optimization strategies, including Gaussian process models from SMT 2.0 and Bayesian optimization. Numerical experiments compare NomadBBO with other solvers on constrained and unconstrained mixed-variable problems from the Cat-Suite benchmark collection. The beta release is available at https://test.pypi.org/project/NomadBBO/

math.OC↗

arXiv2026

Surrogate-based categorical neighborhoods for mixed-variable blackbox optimization

In simulation-based engineering, design choices are often obtained following the optimization of complex blackbox models. These models frequently involve mixed-variable domains with quantitative and categorical variables. Unlike quantitative variables, categorical variables lack an inherent structure, which makes them difficult to handle, especially in the presence of constraints. This work proposes a systematic approach to structure and model categorical variables in constrained mixed-variable blackbox optimization. Surrogate models of the objective and constraint functions are used to induce problem-specific categorical distances. From these distances, surrogate-based neighborhoods are constructed using notions of dominance from bi-objective optimization, jointly accounting for information from both the objective and the constraint functions. This study addresses the lack of automatic and constraint-aware categorical neighborhood construction in mixed-variable blackbox optimization. As a proof of concept, these neighborhoods are employed within CatMADS, an extension of the MADS algorithm for categorical variables. The surrogate models are Gaussian processes, and the resulting method is called CatMADS-GP. The method is benchmarked on the Cat-Suite collection of 60 mixed-variable optimization problems and compared against state-of-the-art solvers. Data profiles indicate that CatMADS-GP achieves superior performance for both unconstrained and constrained problems.

math.OC↗

arXiv2026

Modeling Hierarchical Spaces: A Review and Unified Framework for Surrogate-Based Architecture Design

Simulation-based problems involving mixed-variable inputs frequently feature domains that are hierarchical, conditional, heterogeneous, or tree-structured. These characteristics pose challenges for data representation, modeling, and optimization. This paper reviews extensive literature on these structured input spaces and proposes a unified framework that generalizes existing approaches. In this framework, input variables may be continuous, integer, or categorical. A variable is described as meta if its value governs the presence of other decreed variables, enabling the modeling of conditional and hierarchical structures. We further introduce the concept of partially-decreed variables, whose activation depends on contextual conditions. To capture these inter-variable hierarchical relationships, we introduce design space graphs, combining principles from feature modeling and graph theory. This allows the definition of general hierarchical domains suitable for describing complex system architectures. Our framework defines hierarchical distances and kernels to enable surrogate modeling and optimization on hierarchical domains. We demonstrate its effectiveness on complex system design problems, including a neural network and a green-aircraft case study. Our methods are available in the open-source Surrogate Modeling Toolbox (SMT 2.0).

cs.LG↗

arXiv2025

CatMADS: Mesh Adaptive Direct Search for constrained blackbox optimization with categorical variables

Solving optimization problems in which functions are blackboxes and variables involve different types poses significant theoretical and algorithmic challenges. Nevertheless, such settings frequently occur in simulation-based engineering design and machine learning. This paper extends the Mesh Adaptive Direct Search (MADS) algorithm to address mixed-variable problems with categorical, integer and continuous variables. MADS is a robust derivative-free optimization framework with a well-established convergence analysis for constrained quantitative problems. CatMADS generalizes MADS by incorporating categorical variables through distance-induced neighborhoods. A detailed convergence analysis of CatMADS is provided, with flexible choices balancing computational cost and local optimality strength. Four types of mixed-variable local minima are introduced, corresponding to progressively stronger notions of local optimality. CatMADS integrates the progressive barrier strategy for handling constraints, and ensures Clarke stationarity. An instance of \catmads employs cross-validation to construct problem-specific categorical distances. This instance is compared to state-of-the-art solvers on 32 mixed-variable problems, half of which are constrained. Data profiles show that CatMADS achieves the best results, demonstrating that the framework is empirically efficient in addition to having strong theoretical foundations.

math.OC↗

arXiv2025

A distance for mixed-variable and hierarchical domains with meta variables

Heterogeneous datasets emerge in various machine learning and optimization applications that feature different input sources, types or formats. Most models or methods do not natively tackle heterogeneity. Hence, such datasets are often partitioned into smaller and simpler ones, which may limit the generalizability or performance, especially when data is limited. The first main contribution of this work is a modeling framework that generalizes hierarchical, tree-structured, variable-size or conditional search frameworks. The framework models mixed-variable and hierarchical domains in which variables may be continuous, integer, or categorical, with some identified as meta when they influence the structure of the problem. The second main contribution is a novel distance that compares any pair of mixed-variable points that do not share the same variables, allowing to use whole heterogeneous datasets that reside in mixed-variable and hierarchical domains with meta variables. The contributions are illustrated through regression and classification experiments using simple distance-based models applied to datasets of hyperparameters with corresponding performance scores.

stat.ML↗

arXiv2022

A general mathematical framework for constrained mixed-variable blackbox optimization problems with meta and categorical variables

A mathematical framework for modelling constrained mixed-variable optimization problems is presented in a blackbox optimization context. The framework introduces a new notation and allows solution strategies. The notation framework allows meta and categorical variables to be explicitly and efficiently modelled, which facilitates the solution of such problems. The new term meta variables is used to describe variables that influence which variables are acting or nonacting: meta variables may affect the number of variables and constraints. The flexibility of the solution strategies supports the main blackbox mixed-variable optimization approaches: direct search methods and surrogate-based methods (Bayesian optimization). The notation system and solution strategies are illustrated through an example of a hyperparameter optimization problem from the machine learning community.

math.OC↗