arXiv subjects

Federico D'Onofrio

Publications and source records attributed to Federico D'Onofrio.

4 recordsLinked to original sources

arXiv2026

Feature Selection in Nonlinear SVMs via Local Search and Submodular Optimization

Embedded feature selection is a classical approach to interpretable machine learning, aiming to iden- tify the most informative variables while simultaneously training the predictive model. We address the problem of enforcing an exact limit on the number of features used in nonlinear Support Vector Machines (SVMs) by means of a hard cardinality constraint. Whereas existing hard-constraint formulations apply only to the primal formulation of linear SVMs, we embed the cardinality constraint directly into the dual formulation of a nonlinear SVM. This ensures strict control over the number of selected features while still leveraging kernelization. We model the problem as a mixed-integer nonlinear program (MINLP). Our first contribution is a local-search metaheuristic applicable to any nonlinear kernel, which exploits the decomposable structure of the formulation to efficiently explore the binary decision space. Our second and main contribution is a decomposition framework that alternates optimization over continuous and binary variables. For polynomial kernels, we reformulate the binary subproblem as the maximization of a monotone submodular function under a cardinality constraint, enabling the integration of scalable greedy algorithms into the alternating scheme. Computational experiments on benchmark datasets show that the proposed algorithms significantly outperform existing MINLP solvers and standard heuristics, delivering higher-quality solutions within practical runtimes.

math.OC↗

arXiv2024

Feature selection in linear SVMs via a hard cardinality constraint: a scalable SDP decomposition approach

In this paper, we study the embedded feature selection problem in linear Support Vector Machines (SVMs), in which a cardinality constraint is employed, leading to an interpretable classification model. The problem is NP-hard due to the presence of the cardinality constraint, even though the original linear SVM amounts to a problem solvable in polynomial time. To handle the hard problem, we first introduce two mixed-integer formulations for which novel semidefinite relaxations are proposed. Exploiting the sparsity pattern of the relaxations, we decompose the problems and obtain equivalent relaxations in a much smaller cone, making the conic approaches scalable. To make the best usage of the decomposed relaxations, we propose heuristics using the information of its optimal solution. Moreover, an exact procedure is proposed by solving a sequence of mixed-integer decomposed semidefinite optimization problems. Numerical results on classical benchmarking datasets are reported, showing the efficiency and effectiveness of our approach.

math.OC↗

arXiv2023

Margin Optimal Classification Trees

In recent years, there has been growing attention to interpretable machine learning models which can give explanatory insights on their behaviour. Thanks to their interpretability, decision trees have been intensively studied for classification tasks and, due to the remarkable advances in mixed integer programming (MIP), various approaches have been proposed to formulate the problem of training an Optimal Classification Tree (OCT) as a MIP model. We present a novel mixed integer quadratic formulation for the OCT problem, which exploits the generalization capabilities of Support Vector Machines for binary classification. Our model, denoted as Margin Optimal Classification Tree (MARGOT), encompasses maximum margin multivariate hyperplanes nested in a binary tree structure. To enhance the interpretability of our approach, we analyse two alternative versions of MARGOT, which include feature selection constraints inducing sparsity of the hyperplanes' coefficients. First, MARGOT has been tested on non-linearly separable synthetic datasets in a 2-dimensional feature space to provide a graphical representation of the maximum margin approach. Finally, the proposed models have been tested on benchmark datasets from the UCI repository. The MARGOT formulation turns out to be easier to solve than other OCT approaches, and the generated tree better generalizes on new observations. The two interpretable versions effectively select the most relevant features, maintaining good prediction quality.

math.OC↗

arXiv2023

The Incremental Knapsack Problem with Monotone Submodular All-or-Nothing Profits

We study incremental knapsack problems with profits given by a special class of monotone submodular functions, that we dub all-or-nothing. We show that these problems are not harder to approximate than a less general class of modular incremental knapsack problems, that have been investigated in the literature. We also show that certain extensions to more general submodular functions are APX-hard.

cs.DS↗