arXiv subjects

Thibaut Vidal

Publications and source records attributed to Thibaut Vidal.

At least 19 recordsLinked to original sources

arXiv2026

Trustworthy Machine Learning through the Lens of Combinatorial Optimization: Survey and Research Perspectives

Modern machine learning (ML) increasingly relies on complex models whose behavior is difficult to characterize beyond empirical performance metrics. Across a wide range of tasks, including prediction, generation, and decision-making, models with similar empirical performance can exhibit markedly different properties in terms of their transparency, interpretability, robustness, fairness, privacy, and certifiability. This survey highlights how optimization- and certification-oriented reasoning can provide a useful framework for reasoning about such differences, supporting tasks ranging from model training and selection to auditing and certification. We review and synthesize recent advances at the intersection of combinatorial optimization (CO) and trustworthy ML, covering both training and post-training tasks, including interpretable model learning, explanation generation, robustness analysis, fairness auditing, model compression, and privacy attacks and protections. Across these domains, CO formulations offer additional capabilities over purely heuristic approaches, e.g., gradient-based ones, notably global guarantees, formal certificates, and explicit treatment of trade-offs. While scalability remains an important challenge, continued progress in solvers and hybrid algorithms suggests a growing role for CO in the design and deployment of trustworthy ML systems.

cs.LG↗

arXiv2026

Intermediate Bilevel Optimization: Modeling Endogenous Follower Tie-Breaking Behavior

In bilevel optimization, optimistic and pessimistic follower behaviors are the most commonly used forms to define how the follower ties-breaks among multiple optimal solutions. In this work, we go beyond these extreme tie-breaking behaviors and investigate the intermediate bilevel optimization program (I-BO), where the follower's selected optimal response is a decision-dependent random event, with a probability measure influenced by the leader's decision. We formally introduce a class of such endogenous measures, including the special case of strong-weak decision-dependent I-BO. We reformulate the I-BO as a Transformed I-BO (T-I-BO) with exogenous uncertainty by defining inverse and Markov-chain transformations, which represent the follower's response as a function of the leader's decision and exogenous randomness. We handle the T-I-BO's uncertainty via sample-average approximation (SAA), and we propose tailored approaches for its SAA program according to the chosen transformation. Computationally, our methods solve reasonable-sized instances efficiently and outperform the deterministic equivalent when available. Furthermore, experiments stress the critical need to accurately model follower tie-breaking behavior, particularly depending on its alignment with the leader's objective, as misspecification leads to suboptimal leader decisions.

math.OC↗

arXiv2026

When Interpretability Is Unequally Distributed: Fairness in Hybrid Interpretable Models

Hybrid interpretable models combine a transparent component with a black-box model by assigning some examples to the former and deferring the rest to the latter. While this design enables flexible tradeoffs between accuracy and interpretability, it also raises a distinct procedural fairness concern: some demographic groups may systematically receive interpretable decisions, while others are disproportionately routed to a black box. We formalize this issue as Interpretability Coverage Disparity (ICD), a demographic-parity-style measure applied to the routing decision of hybrid interpretable models. Using tools from predictive multiplicity, we study ICD across four hybrid interpretable learning methods, three standard fairness benchmark datasets, and multiple sensitive attributes. Our experiments reveal substantial ICD in intermediate transparency regimes, where both the interpretable and black-box components are actively used. We further show that simple coverage-disparity constraints can significantly reduce ICD in exact hybrid learning methods, with marginal impact on accuracy and sparsity. In several settings, ICD mitigation also improves standard algorithmic fairness metrics. These results show that hybrid interpretable models should be audited not only for predictive fairness, but also for how they allocate interpretability across individuals and groups.

cs.LG↗

arXiv2026

PACE: Prune-And-Compress Ensemble Models

Ensemble models achieve state-of-the-art performance on prediction tasks, but usually require aggregating a large number of weak learners. This can hinder deployment, interpretability, and downstream tasks such as robustness verification. Remedies to this issue fall into two main camps: pruning, which discards redundant learners, and compression, which generates new ones from scratch. We introduce PACE, a framework that interleaves these paradigms in a two-phase strategy. First, new learners are actively generated via a theoretically grounded procedure to enhance the diversity of the initial ensemble. When no more relevant learners can be found, a second phase of pruning is performed on this enriched ensemble. During both operations, PACE allows fine control on the faithfulness to the original ensemble. Experiments show that our method outperforms prior pruning and compression methods while offering principled control of faithfulness guarantees.

cs.LG↗

arXiv2026

Optimal Counterfactual Search in Tree Ensembles: A Study Across Modeling and Solution Paradigms

Trust in counterfactual explanations depends critically on whether their recommended changes are truly minimal: suboptimal explanations may vastly overshoot the actual changes needed to alter a decision, and heuristic errors can affect individuals unevenly, giving some users relevant recourse while assigning others unnecessarily costly recommendations. Consequently, we study the problem of computing optimal counterfactual explanations for tree ensembles under plausibility and actionability constraints. This is a combinatorial problem: for a fixed model, counterfactual search boils down to selecting consistent branching decisions and threshold-defined regions under a distance objective. We exploit this structure through CPCF, a constraint programming (CP) formulation in which numerical features are encoded as interval domains induced by split thresholds, while discrete features retain native finite-domain representations. This yields a compact finite-domain formulation that supports multiple distance objectives without continuous split-boundary search. We then place CPCF in a broader comparison across mathematical programming paradigms: we extend a maximum Boolean satisfiability (MaxSAT) formulation, originally designed for hard-voting random forests, to soft-voting ensembles, and compare against the current state-of-the-art mixed-integer linear programming (MILP) optimal approach. Across ten datasets and three types of tree ensembles, we analyze scalability, anytime performance, and sensitivity to distance metrics. We observe that CP achieves the best overall performance. More importantly, our results identify regimes in which the specific strengths of each paradigm make it best suited: CP is most versatile overall, MaxSAT handles hard-voting ensembles particularly well, and MILP remains competitive in amortized inference settings with a moderate number of split levels.

cs.LG↗

arXiv2026

Linear Decision Tree Policies for Integer Linear Programs

We study optimal decision policies, represented as linear decision trees, for integer linear programs with a fixed feasible set and varying cost vectors. Once synthesized for a given feasible set, they return an optimal solution for any queried cost vector through a sequence of linear tests. We show that there exists a policy performing this operation in a polynomial number of arithmetic operations in the worst case. In contrast, deciding whether there exists an exact policy with a prescribed maximum number of leaves is $\Sigma_2^p$-complete. Alongside these theoretical results, we develop a practical construction framework to synthesize policies within a specific subclass of linear decision trees. Our computational experiments show that, although policy synthesis can be time-intensive, it allows one to retrieve optimal solutions orders of magnitude faster than classical and specialized solution methods on repeated queries. Overall, this paradigm provides a different perspective on the solution of integer linear programs and offers a principled offline-online approach for repeated optimization.

math.OC↗

arXiv2026

Counterfactual Maps: What They Are and How to Find Them

Counterfactual explanations are a central tool in interpretable machine learning, yet computing them exactly for complex models remains challenging. For tree ensembles, predictions are piecewise constant over a large collection of axis-aligned hyperrectangles, implying that an optimal counterfactual for a point corresponds to its projection onto the nearest rectangle with an alternative label under a chosen metric. Existing methods largely overlook this geometric structure, relying either on heuristics with no optimality guarantees or on mixed-integer programming formulations that do not scale to interactive use. In this work, we revisit counterfactual generation through the lens of nearest-region search and introduce counterfactual maps, a global representation of recourse for tree ensembles. Leveraging the fact that any tree ensemble can be compressed into an equivalent partition of labeled hyperrectangles, we cast counterfactual search as the problem of identifying the generalized Voronoi cell associated with the nearest rectangle of an alternative label. This leads to an exact, amortized algorithm based on volumetric k-dimensional (KD) trees, which performs branch-and-bound nearest-region queries with explicit optimality certificates and sublinear average query time after a one-time preprocessing phase. Our experimental analyses on several real datasets drawn from high-stakes application domains show that this approach delivers globally optimal counterfactual explanations with millisecond-level latency, achieving query times that are orders of magnitude faster than existing exact, cold-start optimization methods.

cs.LG↗

arXiv2026

The XL Instances and the CVRPLib Best Known Solution Challenge

This paper introduces the XL set, a new collection of large-scale benchmark instances for the capacitated vehicle routing problem (CVRP). The set extends previous benchmarks by covering instances with 1,000 to 10,000 customers and a wide range of structural characteristics, following established generation principles from prior CVRP studies. To provide strong reference solutions, we conducted an extensive computational study with several state-of-the-art algorithms and retained the best solutions obtained as the starting point for a community-driven BKS challenge hosted on the CVRPLib website. The XL instances are publicly available to support the experimental evaluation and comparison of future solution methods. The post-competition results demonstrate the impact of the challenge: over 30 days, participating teams submitted 1,932 BKS improvements, substantially refining the initial solution set and highlighting promising research directions for solving large-scale CVRPs, notably through LLM-assisted algorithm discovery.

math.OC↗

arXiv2025

The Branch-and-Bound Tree Closure

This paper investigates the a-posteriori analysis of Branch-and-Bound~(BB) trees to extract structural information about the feasible region of mixed-binary linear programs. We introduce three novel outer approximations of the feasible region, systematically constructed from a BB tree. These are: a tight formulation based on disjunctive programming, a branching-based formulation derived from the tree's branching logic, and a mixing-set formulation derived from the on-off properties inside the tree. We establish an inclusion hierarchy, which ranks the approximations by their theoretical strength \wrt to the original feasible region. The analysis is extended to the generation of valid inequalities, revealing a separation-time hierarchy that mirrors the inclusion hierarchy in reverse. This highlights a trade-off between the tightness of an approximation and the computational cost of generating cuts from it. Motivated by the computational expense of the stronger approximations, we introduce a new family of valid inequalities called star tree inequalities. Although their closure forms the weakest of the proposed approximations, their practical appeal lies in an efficient, polynomial-time combinatorial separation algorithm. A computational study on multi-dimensional knapsack and set-covering problems empirically validates the theoretical findings. Moreover, these experiments confirm that computationally useful valid inequalities can be generated from BB trees obtained by solving optimization problems considered in practice.

math.OC↗

arXiv2025

Optimization by Directional Attacks: Solving Problems with Neural Network Surrogates

This paper tackles optimization problems whose objective and constraints involve a trained Neural Network (NN), where the goal is to maximize $f(\Phi(x))$ subject to $c(\Phi(x)) \leq 0$, with $f$ smooth, $c$ general and non-stringent, and $\Phi$ a NN already trained and not given as a glass-box. We address two challenges regarding this problem: identifying ascent directions for local search, and ensuring reliable convergence towards relevant local solutions. To this end, we re-purpose the notion of directional NN attacks as efficient optimization subroutines, since directional NN attacks are designed to efficiently compute perturbations of $x$ that steer $\Phi(x)$ in prescribed directions. Precisely, we develop an attack operator that computes attacks of $\Phi$ at any $x$ along the direction $\nabla f(\Phi(x))$. Then, we propose a hybrid algorithm combining the attack operator with derivative-free optimization (DFO) techniques, designed for numerical reliability by remaining oblivious to the structure of the problem. We consider the Covering Direct Search Method (CDSM), which offers asymptotic guarantees to converge to a local solution under mild assumptions on the problem. The resulting method alternates between attack-based steps for heuristic yet fast local improvements of the current incumbent solution and CDSM steps for certified convergence and numerical reliability. Experiments on three problems show that this hybrid approach outperforms standard baselines.

math.OC↗

arXiv2025

Boosting Revisited: Benchmarking and Advancing LP-Based Ensemble Methods

Despite their theoretical appeal, totally corrective boosting methods based on linear programming have received limited empirical attention. In this paper, we conduct the first large-scale experimental study of six LP-based boosting formulations, including two novel methods, NM-Boost and QRLP-Boost, across 20 diverse datasets. We evaluate the use of both heuristic and optimal base learners within these formulations, and analyze not only accuracy, but also ensemble sparsity, margin distribution, anytime performance, and hyperparameter sensitivity. We show that totally corrective methods can outperform or match state-of-the-art heuristics like XGBoost and LightGBM when using shallow trees, while producing significantly sparser ensembles. We further show that these methods can thin pre-trained ensembles without sacrificing performance, and we highlight both the strengths and limitations of using optimal decision trees in this context.

cs.LG↗

arXiv2025

Two-Stage Stochastic Capacity Expansion in Stable Matching under Truthful or Strategic Preference Uncertainty

Recent studies on many-to-one matching markets have explored agents with flexible capacity and truthful preference reporting, focusing on mechanisms that jointly design capacities and select a matching. However, in real-world applications such as school choice and residency matching, preferences are revealed after capacity decisions are made, with matching occurring afterward; uncertainty about agents' preferences must be considered during capacity planning. Moreover, even under strategy-proof mechanisms, agents may strategically misreport preferences based on beliefs about admission chances. We introduce a two-stage stochastic matching problem with uncertain preferences, using school choice as a case study. In the first stage, the clearinghouse expands schools' capacities before observing students' reported preferences. Students either report their true preferences, producing exogenous uncertainty, or act strategically, submitting reported preferences based on their true preferences and admission chances (which depend on capacities), introducing endogenous uncertainty. In the second stage, the clearinghouse computes the student-optimal stable matching based on schools' priorities and students' reported preferences. In strategic cases, endogenous reported preferences are utility-maximizing transformations of capacity decisions and exogenous true preferences; we handle uncertainty using sample average approximation(SAA). We develop behavior-based mathematical formulations and, due to problem complexity, propose Lagrangian- and local-search-based behavior-specific heuristics for near-optimal solutions. Our SAA-based approaches outperform the average scenario approach on students' matching preferences and admission outcomes, emphasizing the impact of stochastic preferences on capacity decisions. Student behavior notably influences capacity design, stressing the need to consider misreports.

cs.GT↗

arXiv2025

Large Neighborhood and Hybrid Genetic Search for Inventory Routing Problems

The inventory routing problem (IRP) focuses on jointly optimizing inventory and distribution operations from a supplier to retailers over multiple days. Compared to other problems from the vehicle routing family, the interrelations between inventory and routing decisions render IRP optimization more challenging and call for advanced solution techniques. A few studies have focused on developing large neighborhood search approaches for this class of problems, but this remains a research area with vast possibilities due to the challenges related to the integration of inventory and routing decisions. In this study, we advance this research area by developing a new large neighborhood search operator tailored for the IRP. Specifically, the operator optimally removes and reinserts all visits to a specific retailer while minimizing routing and inventory costs. We propose an efficient tailored dynamic programming algorithm that exploits preprocessing and acceleration strategies. The operator is used to build an effective local search routine, and included in a state-of-the-art routing algorithm, i.e., Hybrid Genetic Search (HGS). Through extensive computational experiments, we demonstrate that the resulting heuristic algorithm leads to solutions of unmatched quality up to this date, especially on large-scale benchmark instances.

cs.NE↗

arXiv2025

Fairness and Sparsity within Rashomon sets: Enumeration-Free Exploration and Characterization

We introduce an enumeration-free method based on mathematical programming to precisely characterize various properties such as fairness or sparsity within the set of "good models", known as Rashomon set. This approach is generically applicable to any hypothesis class, provided that a mathematical formulation of the model learning task exists. It offers a structured framework to define the notion of business necessity and evaluate how fairness can be improved or degraded towards a specific protected group, while remaining within the Rashomon set and maintaining any desired sparsity level. We apply our approach to two hypothesis classes: scoring systems and decision diagrams, leveraging recent mathematical programming formulations for training such models. As seen in our experiments, the method comprehensively and certifiably quantifies trade-offs between predictive performance, sparsity, and fairness. We observe that a wide range of fairness values are attainable, ranging from highly favorable to significantly unfavorable for a protected group, while staying within less than 1% of the best possible training accuracy for the hypothesis class. Additionally, we observe that sparsity constraints limit these trade-offs and may disproportionately harm specific subgroups. As we evidenced, thoroughly characterizing the tensions between these key aspects is critical for an informed and accountable selection of models.

cs.LG↗

arXiv2025

Training Set Reconstruction from Differentially Private Forests: How Effective is DP?

Recent research has shown that structured machine learning models such as tree ensembles are vulnerable to privacy attacks targeting their training data. To mitigate these risks, differential privacy (DP) has become a widely adopted countermeasure, as it offers rigorous privacy protection. In this paper, we introduce a reconstruction attack targeting state-of-the-art $\epsilon$-DP random forests. By leveraging a constraint programming model that incorporates knowledge of the forest's structure and DP mechanism characteristics, our approach formally reconstructs the most likely dataset that could have produced a given forest. Through extensive computational experiments, we examine the interplay between model utility, privacy guarantees and reconstruction accuracy across various configurations. Our results reveal that random forests trained with meaningful DP guarantees can still leak portions of their training data. Specifically, while DP reduces the success of reconstruction attacks, the only forests fully robust to our attack exhibit predictive performance no better than a constant classifier. Building on these insights, we also provide practical recommendations for the construction of DP random forests that are more resilient to reconstruction attacks while maintaining a non-trivial predictive performance.

cs.LG↗

arXiv2025

From Counterfactuals to Trees: Competitive Analysis of Model Extraction Attacks

The advent of Machine Learning as a Service (MLaaS) has heightened the trade-off between model explainability and security. In particular, explainability techniques, such as counterfactual explanations, inadvertently increase the risk of model extraction attacks, enabling unauthorized replication of proprietary models. In this paper, we formalize and characterize the risks and inherent complexity of model reconstruction, focusing on the "oracle'' queries required for faithfully inferring the underlying prediction function. We present the first formal analysis of model extraction attacks through the lens of competitive analysis, establishing a foundational framework to evaluate their efficiency. Focusing on models based on additive decision trees (e.g., decision trees, gradient boosting, and random forests), we introduce novel reconstruction algorithms that achieve provably perfect fidelity while demonstrating strong anytime performance. Our framework provides theoretical bounds on the query complexity for extracting tree-based model, offering new insights into the security vulnerabilities of their deployment.

cs.LG↗

arXiv2024

DistrictNet: Decision-aware learning for geographical districting

Districting is a complex combinatorial problem that consists in partitioning a geographical area into small districts. In logistics, it is a major strategic decision determining operating costs for several years. Solving districting problems using traditional methods is intractable even for small geographical areas and existing heuristics often provide sub-optimal results. We present a structured learning approach to find high-quality solutions to real-world districting problems in a few minutes. It is based on integrating a combinatorial optimization layer, the capacitated minimum spanning tree problem, into a graph neural network architecture. To train this pipeline in a decision-aware fashion, we show how to construct target solutions embedded in a suitable space and learn from target solutions. Experiments show that our approach outperforms existing methods as it can significantly reduce costs on real-world cities.

cs.LG↗

arXiv2024

Free Lunch in the Forest: Functionally-Identical Pruning of Boosted Tree Ensembles

Tree ensembles, including boosting methods, are highly effective and widely used for tabular data. However, large ensembles lack interpretability and require longer inference times. We introduce a method to prune a tree ensemble into a reduced version that is "functionally identical" to the original model. In other words, our method guarantees that the prediction function stays unchanged for any possible input. As a consequence, this pruning algorithm is lossless for any aggregated metric. We formalize the problem of functionally identical pruning on ensembles, introduce an exact optimization model, and provide a fast yet highly effective method to prune large ensembles. Our algorithm iteratively prunes considering a finite set of points, which is incrementally augmented using an adversarial model. In multiple computational experiments, we show that our approach is a "free lunch", significantly reducing the ensemble size without altering the model's behavior. Thus, we can preserve state-of-the-art performance at a fraction of the original model's size.

cs.LG↗