arXiv subjects

Youssef Diouane

Publications and source records attributed to Youssef Diouane.

At least 19 recordsLinked to original sources

arXiv2026

Adaptive direct search algorithms with relaxable and quantifiable constraints

This work introduces ADS-PB, an extension of the Adaptive Direct Search (ADS) framework for solving constrained blackbox optimization problems. With ADS, iterates progress without relying on mesh structures or sufficient decrease conditions on the objective function value. Unlike the extreme barrier approach used in ADS, where only unrelaxable constraints are considered, the proposed method also handles quantifiable and relaxable constraints using a Progressive Barrier (PB) mechanism that exploits both constraint and objective function values. A convergence analysis of the proposed framework under mild assumptions is presented. The performance of the proposed method is assessed using sets of analytical and simulation-based constrained test problems and is compared with state-of-the-art blackbox optimization solvers, including the PB approach within the Mesh Adaptive Direct Search (MADS) framework.

math.OC↗

arXiv2026

Benchmarking Bilevel Derivative-Free Optimization Algorithms

Bilevel optimization involves an upper-level and a lower-level decision maker. The lower-level optimization problem is nested within the constraints of the upper-level one. A point is said to be admissible for the bilevel problem if it satisfies all constraints and is optimal for the lower-level decision-maker. Bilevel derivative-free optimization (BL-DFO) algorithms address bilevel optimization problems in which either the upper-level or the lower-level problem is solved using a derivative-free optimization method. In this context, existing BL-DFO benchmarking techniques often do not rigorously validate the admissibility of proposed solutions, and do not adequately account for the computational effort deployed by the upper- and lower-level solvers. This work proposes a benchmarking methodology for BL-DFO algorithms. A post-optimization procedure, named refereeing procedure, is introduced to discard non-admissible points and ensure a fair comparison between the algorithms. The computational effort deployed by upper- and lower-level solvers are also taken into account into the overall computational cost. Numerical experiments illustrate the benchmarking methodology.

math.OC↗

arXiv2026

Efficient multidisciplinary design via Bayesian optimization

This study introduces SEGOMOE, a Bayesian optimization tool for optimizing complex, computationally expensive systems, especially in aeronautics. It efficiently handles mixed design variables (continuous, discrete, categorical, hierarchical) using adaptive Gaussian process models. SEGOMOE combines expert models to address nonlinearities in objectives and constraints, leveraging the open-source Surrogate Modeling Toolbox (SMT). The tool supports multi-fidelity data and solves both single- and multi-objective problems, including hidden constraints and high-dimensional decomposition. Validated through benchmarks and real-world aeronautical applications, SEGOMOE proves to be robust and versatile for tackling multidisciplinary challenges.

math.OC↗

arXiv2026

Online Sketched Newton-Raphson

In online convex optimization (OCO), a decision-maker is confronted with an unknown environment and seeks to play an optimal sequence of decisions on a short time-scale using only past information. Recent advances in second-order OCO methods have demonstrated tighter regret bounds and improved empirical performance over traditional first-order methods. However, this performance comes at a cost: a matrix inversion is now required, which scales with the cube of the size of the problem. In this work, we propose sketching to mitigate this limitation. Specifically, we present the online sketched Newton-Raphson method (OSNR) which preserves the tight regret bounds obtained with second-order methods while presenting a strict computational improvement in terms of complexity. We discuss three application scenarios of OSNR: online root finding, unconstrained OCO, and time-varying equality-constrained OCO, and present their respective regret and a constraint violation bound for the latter. In all three applications, OSNR achieves sublinear dynamic regret bounds. For the equality-constrained case, the extension OSNR with equality constraints OSNR-EC is shown to yield sublinear cumulative constraint violation. Finally, we illustrate the performance of OSNR and OSNR-EC on two numerical examples, viz., online position tracking and optimal power flow, and observe that OSNR and OSNR-EC exhibit high performance even at low sampling rates.

math.OC↗

arXiv2026

Bayesian Algorithm for Collaborative Optimization with Application to Aircraft Design

Collaborative Optimization (CO) is a multidisciplinary design optimization (MDO) framework that decomposes large-scale engineering problems into parallel, independently solvable subsystems coordinated by a system-level optimizer. Its practical utility is limited by the high frequency of expensive black-box disciplinary evaluations arising from the bi-level consistency constraints. This paper introduces BACO, a Bayesian Algorithm for Collaborative Optimization, which replaces the direct black-box calls at both levels with Gaussian process (GP) surrogates and acquisition function maximization. At the subsystem level, an acquisition function subject to GP-predicted feasibility constraints identifies the next evaluation point. At the system level, the same surrogate framework enforces consistency through predicted discrepancy constraints. This architecture reduces the number of true black-box evaluations required per major iteration. BACO is benchmarked against state-of-the-art CO variants on a Scalable MDO problem over 50 randomized instances. On this problem, BACO consistently achieves lower objective values and drives both constraint violation and interdisciplinary discrepancy to near-zero within the evaluation budget, outperforming all three CO variants across all tested DoE sizes. Further validation is conducted on a coupled aero-structural wing optimization problem based on the Common Research Model (CRM) geometry, where BACO identifies a feasible solution within 886 of 1000 allocated evaluations, recovering results physically consistent with active bending stress and tip deflection constraints. The BACO software, the state-of-the-art CO solvers, as well as standard MDO benchmarking problems are open-source and publicly available at https://moebehfn.github.io/mdotoolbox/.

math.OC↗

arXiv2026

A Spectral Preconditioner for the Conjugate Gradient Method with Iteration Budget

We study the solution of large symmetric positive-definite linear systems in a matrix-free setting with a limited iteration budget. We focus on the preconditioned conjugate gradient (PCG) method with spectral preconditioning. Spectral preconditioners map a subset of eigenvalues to a positive cluster via a scaling parameter, and leave the remainder of the spectrum unchanged, in hopes to reduce the number of iterations to convergence. We formulate the design of the spectral preconditioners as a constrained optimization problem. The optimal cluster placement is defined to minimize the error in energy norm at a fixed iteration. This optimality criterion provides new insight into the design of efficient spectral preconditioners when PCG is stopped short of convergence. We propose practical strategies for selecting the scaling parameter, hence the cluster position, that incur negligible computational cost. Numerical experiments highlight the importance of cluster placement and demonstrate significant improvements in terms of error in energy norm, particularly during the initial iterations.

math.NA↗

arXiv2026

Multi-fidelity approaches for general constrained Bayesian optimization with application to aircraft design

Aircraft design relies heavily on solving challenging and computationally expensive Multidisciplinary Design Optimization problems. In this context, there has been growing interest in multi-fidelity models for Bayesian optimization to improve the MDO process by balancing computational cost and accuracy through the combination of high- and low-fidelity simulation models, enabling efficient exploration of the design process at a minimal computational effort. In the existing literature, fidelity selection focuses only on the objective function to decide how to integrate multiple fidelity levels, balancing precision and computational cost using variance reduction criteria. In this work, we propose novel multi-fidelity selection strategies. Specifically, we demonstrate how incorporating information from both the objective and the constraints can further reduce computational costs without compromising the optimality of the solution. We validate the proposed multi-fidelity optimization strategy by applying it to four analytical test cases, showcasing its effectiveness. The proposed method is used to efficiently solve a challenging aircraft wing aero-structural design problem. The proposed setting uses a linear vortex lattice method and a finite element method for the aerodynamic and structural analysis respectively. We show that employing our proposed multi-fidelity approach leads to $86\%$ to $200\%$ more constraint compliant solutions given a limited budget compared to the state-of-the-art approach.

math.OC↗

arXiv2026

Transfer Learning in Bayesian Optimization for Aircraft Design

The use of transfer learning within Bayesian optimization addresses the disadvantages of the so-called \textit{cold start} problem by using source data to aid in the optimization of a target problem. We present a method that leverages an ensemble of surrogate models using transfer learning and integrates it in a constrained Bayesian optimization framework. We identify challenges particular to aircraft design optimization related to heterogeneous design variables and constraints. We propose the use of a partial-least-squares dimension reduction algorithm to address design space heterogeneity, and a \textit{meta} data surrogate selection method to address constraint heterogeneity. Numerical benchmark problems and an aircraft conceptual design optimization problem are used to demonstrate the proposed methods. Results show significant improvement in convergence in early optimization iterations compared to standard Bayesian optimization, with improved prediction accuracy for both objective and constraint surrogate models.

math.OC↗

arXiv2026

Surrogate-based categorical neighborhoods for mixed-variable blackbox optimization

In simulation-based engineering, design choices are often obtained following the optimization of complex blackbox models. These models frequently involve mixed-variable domains with quantitative and categorical variables. Unlike quantitative variables, categorical variables lack an inherent structure, which makes them difficult to handle, especially in the presence of constraints. This work proposes a systematic approach to structure and model categorical variables in constrained mixed-variable blackbox optimization. Surrogate models of the objective and constraint functions are used to induce problem-specific categorical distances. From these distances, surrogate-based neighborhoods are constructed using notions of dominance from bi-objective optimization, jointly accounting for information from both the objective and the constraint functions. This study addresses the lack of automatic and constraint-aware categorical neighborhood construction in mixed-variable blackbox optimization. As a proof of concept, these neighborhoods are employed within CatMADS, an extension of the MADS algorithm for categorical variables. The surrogate models are Gaussian processes, and the resulting method is called CatMADS-GP. The method is benchmarked on the Cat-Suite collection of 60 mixed-variable optimization problems and compared against state-of-the-art solvers. Data profiles indicate that CatMADS-GP achieves superior performance for both unconstrained and constrained problems.

math.OC↗

arXiv2026

A penalty-interior point method combined with MADS for equality and inequality constrained optimization

This work introduces MADS-PIP, an efficient framework that integrates a penalty-interior point strategy into the mesh adaptive direct search (MADS) algorithm for solving nonsmooth blackbox optimization problems with general inequality and equality constraints. Inequality constraints are partitioned into two subsets: one treated via a logarithmic barrier applied to an aggregated interior constraint violation, and the other handled through an exterior quadratic penalty. All equality constraints are treated by the exterior penalty. A merit function defines a sequence of unconstrained subproblems, which are solved approximately using MADS, while a carefully designed update rule drives the penalty-barrier parameter to zero. In the nonsmooth setting, we establish convergence results ensuring feasibility for general constraints as well as Clarke stationarity for inequality-constrained problems. Computational experiments on both analytical test sets and challenging blackbox problems demonstrate that the proposed MADS-PIP algorithm is competitive with, and often outperforms, MADS with the progressive barrier strategy, particularly in the presence of equality constraints.

math.OC↗

arXiv2025

A Probabilistic U-Net Approach to Downscaling Climate Simulations

Climate models are limited by heavy computational costs, often producing outputs at coarse spatial resolutions, while many climate change impact studies require finer scales. Statistical downscaling bridges this gap, and we adapt the probabilistic U-Net for this task, combining a deterministic U-Net backbone with a variational latent space to capture aleatoric uncertainty. We evaluate four training objectives, afCRPS and WMSE-MS-SSIM with three settings for downscaling precipitation and temperature from $16\times$ coarser resolution. Our main finding is that WMSE-MS-SSIM performs well for extremes under certain settings, whereas afCRPS better captures spatial variability across scales.

cs.LG↗

arXiv2025

Adaptive direct search algorithms for constrained optimization

Two families of directional direct search methods have emerged in derivative-free and blackbox optimization (DFO and BBO), each based on distinct principles: Mesh Adaptive Direct Search (MADS) and Sufficient Decrease Direct Search (SDDS). MADS restricts trial points to a mesh and accepts any improvement, ensuring none are missed, but at the cost of restraining the placement of trial points. SDDS allows greater freedom by evaluating points anywhere in the space, but accepts only those yielding a sufficient decrease in the objective function value, which may lead to discarding improving points. This work introduces a new class of methods, Adaptive Direct Search (ADS), which uses a novel acceptance rule based on the so-called punctured space, avoiding both meshes and sufficient decrease conditions. ADS enables flexible search while addressing the limitations of MADS and SDDS, and retains the theoretical foundations of directional direct search. Computational results in constrained and unconstrained settings highlight its performance compared to both MADS and SDDS.

math.OC↗

arXiv2025

System-of-systems Modeling and Optimization: An Integrated Framework for Intermodal Mobility

For developing innovative systems architectures, modeling and optimization techniques have been central to frame the architecting process and define the optimization and modeling problems. In this context, for system-of-systems the use of efficient dedicated approaches (often physics-based simulations) is highly recommended to reduce the computational complexity of the targeted applications. However, exploring novel architectures using such dedicated approaches might pose challenges for optimization algorithms, including increased evaluation costs and potential failures. To address these challenges, surrogate-based optimization algorithms, such as Bayesian optimization utilizing Gaussian process models have emerged.

cs.AI↗

arXiv2025

Modeling Hierarchical Spaces: A Review and Unified Framework for Surrogate-Based Architecture Design

Simulation-based problems involving mixed-variable inputs frequently feature domains that are hierarchical, conditional, heterogeneous, or tree-structured. These characteristics pose challenges for data representation, modeling, and optimization. This paper reviews extensive literature on these structured input spaces and proposes a unified framework that generalizes existing approaches. In this framework, input variables may be continuous, integer, or categorical. A variable is described as meta if its value governs the presence of other decreed variables, enabling the modeling of conditional and hierarchical structures. We further introduce the concept of partially-decreed variables, whose activation depends on contextual conditions. To capture these inter-variable hierarchical relationships, we introduce design space graphs, combining principles from feature modeling and graph theory. This allows the definition of general hierarchical domains suitable for describing complex system architectures. Our framework defines hierarchical distances and kernels to enable surrogate modeling and optimization on hierarchical domains. We demonstrate its effectiveness on complex system design problems, including a neural network and a green-aircraft case study. Our methods are available in the open-source Surrogate Modeling Toolbox (SMT 2.0).

cs.LG↗

arXiv2025

CatMADS: Mesh Adaptive Direct Search for constrained blackbox optimization with categorical variables

Solving optimization problems in which functions are blackboxes and variables involve different types poses significant theoretical and algorithmic challenges. Nevertheless, such settings frequently occur in simulation-based engineering design and machine learning. This paper extends the Mesh Adaptive Direct Search (MADS) algorithm to address mixed-variable problems with categorical, integer and continuous variables. MADS is a robust derivative-free optimization framework with a well-established convergence analysis for constrained quantitative problems. CatMADS generalizes MADS by incorporating categorical variables through distance-induced neighborhoods. A detailed convergence analysis of CatMADS is provided, with flexible choices balancing computational cost and local optimality strength. Four types of mixed-variable local minima are introduced, corresponding to progressively stronger notions of local optimality. CatMADS integrates the progressive barrier strategy for handling constraints, and ensures Clarke stationarity. An instance of \catmads employs cross-validation to construct problem-specific categorical distances. This instance is compared to state-of-the-art solvers on 32 mixed-variable problems, half of which are constrained. Data profiles show that CatMADS achieves the best results, demonstrating that the framework is empirically efficient in addition to having strong theoretical foundations.

math.OC↗

arXiv2025

Multi-objective Bayesian Optimization With Mixed-categorical Design Variables for Expensive-to-evaluate Aeronautical Applications

This work aims at developing new methodologies to optimize computational costly complex systems (e.g., aeronautical engineering systems). The proposed surrogate-based method (often called Bayesian optimization) uses adaptive sampling to promote a trade-off between exploration and exploitation. Our in-house implementation, called SEGOMOE, handles a high number of design variables (continuous, discrete or categorical) and nonlinearities by combining mixtures of experts for the objective and/or the constraints. Additionally, the method handles multi-objective optimization settings, as it allows the construction of accurate Pareto fronts with a minimal number of function evaluations. Different infill criteria have been implemented to handle multiple objectives with or without constraints. The effectiveness of the proposed method was tested on practical aeronautical applications within the context of the European Project AGILE 4.0 and demonstrated favorable results. A first example concerns a retrofitting problem where a comparison between two optimizers have been made. A second example introduces hierarchical variables to deal with architecture system in order to design an aircraft family. The third example increases drastically the number of categorical variables as it combines aircraft design, supply chain and manufacturing process. In this article, we show, on three different realistic problems, various aspects of our optimization codes thanks to the diversity of the treated aircraft problems.

cs.LG↗

arXiv2025

Bayesian optimization for mixed variables using an adaptive dimension reduction process: applications to aircraft design

Multidisciplinary design optimization methods aim at adapting numerical optimization techniques to the design of engineering systems involving multiple disciplines. In this context, a large number of mixed continuous, integer and categorical variables might arise during the optimization process and practical applications involve a large number of design variables. Recently, there has been a growing interest in mixed variables constrained Bayesian optimization but most existing approaches severely increase the number of the hyperparameters related to the surrogate model. In this paper, we address this issue by constructing surrogate models using less hyperparameters. The reduction process is based on the partial least squares method. An adaptive procedure for choosing the number of hyperparameters is proposed. The performance of the proposed approach is confirmed on analytical tests as well as two real applications related to aircraft design. A significant improvement is obtained compared to genetic algorithms.

stat.ME↗

arXiv2025

Regularized infill criteria for multi-objective Bayesian optimization with application to aircraft design

Bayesian optimization is an advanced tool to perform ecient global optimization It consists on enriching iteratively surrogate Kriging models of the objective and the constraints both supposed to be computationally expensive of the targeted optimization problem Nowadays efficient extensions of Bayesian optimization to solve expensive multiobjective problems are of high interest The proposed method in this paper extends the super efficient global optimization with mixture of experts SEGOMOE to solve constrained multiobjective problems To cope with the illposedness of the multiobjective inll criteria different enrichment procedures using regularization techniques are proposed The merit of the proposed approaches are shown on known multiobjective benchmark problems with and without constraints The proposed methods are then used to solve a biobjective application related to conceptual aircraft design with ve unknown design variables and three nonlinear inequality constraints The preliminary results show a reduction of the total cost in terms of function evaluations by a factor of 20 compared to the evolutionary algorithm NSGA-II.

cs.LG↗