arXiv subjects

Abdoulaye Sakho

Publications and source records attributed to Abdoulaye Sakho.

3 recordsLinked to original sources

arXiv2026

OT-FairBoost: Optimal Transport-Guided Gradient Boosting for Fairness Regularization on Tabular Data

Although neural-based machine learning models have received a lot of attention recently, tree-based models such as gradient boosting are competitive for tabular data and therefore remain widely used in various applications of AI. As when using other machine learning predictive models, they can however yield discriminative predictions across demographic groups, due to so-called algorithmic biases. These undesirable phenomena have motivated the emergence of new regulatory frameworks and various AI fairness strategies. While several pre-and post-processing methodologies exist to mitigate such bias on gradient boosting models, only a few in-processing methods have been proposed. To bridge this gap, we introduce OT-FairBoost, a novel in-processing framework that incorporates a Wasserstein-2 distance penalty directly into the objective function of gradient-boosted trees. This OT-based mitigation strategy has been shown to efficiently optimize group fairness criteria such as Demographic Parity and Equalized Odds on neural-based predictions. To adapt this approach for gradient boosting, we extend the sample-wise gradient estimation of the Wasserstein-2 distance between group predictions to discrete distributions and hessian diagonals. We then integrate our approach into the LightGBM training procedure and evaluate it across binary classification, regression, and multi-group sensitive attribute settings. Experimental results in each of these settings demonstrate that OT-FairBoost achieves best accuracy-fairness trade-offs against alternatives.

math.ST↗

arXiv2026

Do we need rebalancing strategies? A theoretical and empirical study around SMOTE and its variants

Synthetic Minority Oversampling Technique (SMOTE) is a common rebalancing strategy for handling imbalanced tabular data sets. However, few works analyze SMOTE theoretically. In this paper, we derive several non-asymptotic upper bound on SMOTE density. From these results, we prove that SMOTE (with default parameter) tends to copy the original minority samples asymptotically. We confirm and illustrate empirically this first theoretical behavior on a real-world data-set.bFurthermore, we prove that SMOTE density vanishes near the boundary of the support of the minority class distribution. We then adapt SMOTE based on our theoretical findings to introduce two new variants. These strategies are compared on 13 tabular data sets with 10 state-of-the-art rebalancing procedures, including deep generative and diffusion models. One of our key findings is that, for most data sets, applying no rebalancing strategy is competitive in terms of predictive performances, would it be with LightGBM, tuned random forests or logistic regression. However, when the imbalance ratio is artificially augmented, one of our two modifications of SMOTE leads to promising predictive performances compared to SMOTE and other state-of-the-art strategies.

stat.ML↗

arXiv2025

Harnessing Mixed Features for Imbalance Data Oversampling: Application to Bank Customers Scoring

This study investigates rare event detection on tabular data within binary classification. Standard techniques to handle class imbalance include SMOTE, which generates synthetic samples from the minority class. However, SMOTE is intrinsically designed for continuous input variables. In fact, despite SMOTE-NC-its default extension to handle mixed features (continuous and categorical variables)-very few works propose procedures to synthesize mixed features. On the other hand, many real-world classification tasks, such as in banking sector, deal with mixed features, which have a significant impact on predictive performances. To this purpose, we introduce MGS-GRF, an oversampling strategy designed for mixed features. This method uses a kernel density estimator with locally estimated full-rank covariances to generate continuous features, while categorical ones are drawn from the original samples through a generalized random forest. Empirically, contrary to SMOTE-NC, we show that MGS-GRF exhibits two important properties: (i) the coherence i.e. the ability to only generate combinations of categorical features that are already present in the original dataset and (ii) association, i.e. the ability to preserve the dependence between continuous and categorical features. We also evaluate the predictive performances of LightGBM classifiers trained on data sets, augmented with synthetic samples from various strategies. Our comparison is performed on simulated and public real-world data sets, as well as on a private data set from a leading financial institution. We observe that synthetic procedures that have the properties of coherence and association display better predictive performances in terms of various predictive metrics (PR and ROC AUC...), with MGS-GRF being the best one. Furthermore, our method exhibits promising results for the private banking application, with development pipeline being compliant with regulatory constraints.

cs.LG↗