arXiv subjects

Laura Palagi

Publications and source records attributed to Laura Palagi.

18 recordsLinked to original sources

arXiv2025

Feature Selection in Nonlinear SVMs via Local Search and Submodular Optimization

Embedded feature selection is a classical approach to interpretable machine learning, aiming to iden- tify the most informative variables while simultaneously training the predictive model. We address the problem of enforcing an exact limit on the number of features used in nonlinear Support Vector Machines (SVMs) by means of a hard cardinality constraint. Whereas existing hard-constraint formulations apply only to the primal formulation of linear SVMs, we embed the cardinality constraint directly into the dual formulation of a nonlinear SVM. This ensures strict control over the number of selected features while still leveraging kernelization. We model the problem as a mixed-integer nonlinear program (MINLP). Our first contribution is a local-search metaheuristic applicable to any nonlinear kernel, which exploits the decomposable structure of the formulation to efficiently explore the binary decision space. Our second and main contribution is a decomposition framework that alternates optimization over continuous and binary variables. For polynomial kernels, we reformulate the binary subproblem as the maximization of a monotone submodular function under a cardinality constraint, enabling the integration of scalable greedy algorithms into the alternating scheme. Computational experiments on benchmark datasets show that the proposed algorithms significantly outperform existing MINLP solvers and standard heuristics, delivering higher-quality solutions within practical runtimes.

math.OC↗

arXiv2024

Beyond adaptive gradient: Fast-Controlled Minibatch Algorithm for large-scale optimization

Adaptive gradient methods have been increasingly adopted by deep learning community due to their fast convergence and reduced sensitivity to hyper-parameters. However, these methods come with limitations, such as increased memory requirements for elements like moving averages and a poorly understood convergence theory. To overcome these challenges, we introduce F-CMA, a Fast-Controlled Mini-batch Algorithm with a random reshuffling method featuring a sufficient decrease condition and a line-search procedure to ensure loss reduction per epoch, along with its deterministic proof of global convergence to a stationary point. To evaluate the F-CMA, we integrate it into conventional training protocols for classification tasks involving both convolutional neural networks and vision transformer models, allowing for a direct comparison with popular optimizers. Computational tests show significant improvements, including a decrease in the overall training time by up to 68%, an increase in per-epoch efficiency by up to 20%, and in model accuracy by up to 5%.

cs.LG↗

arXiv2024

Computational issues in Optimization for Deep networks

The paper aims to investigate relevant computational issues of deep neural network architectures with an eye to the interaction between the optimization algorithm and the classification performance. In particular, we aim to analyze the behaviour of state-of-the-art optimization algorithms in relationship to their hyperparameters setting in order to detect robustness with respect to the choice of a certain starting point in ending on different local solutions. We conduct extensive computational experiments using nine open-source optimization algorithms to train deep Convolutional Neural Network architectures on an image multi-class classification task. Precisely, we consider several architectures by changing the number of layers and neurons per layer, in order to evaluate the impact of different width and depth structures on the computational optimization performance.

math.OC↗

arXiv2024

Feature selection in linear SVMs via a hard cardinality constraint: a scalable SDP decomposition approach

In this paper, we study the embedded feature selection problem in linear Support Vector Machines (SVMs), in which a cardinality constraint is employed, leading to an interpretable classification model. The problem is NP-hard due to the presence of the cardinality constraint, even though the original linear SVM amounts to a problem solvable in polynomial time. To handle the hard problem, we first introduce two mixed-integer formulations for which novel semidefinite relaxations are proposed. Exploiting the sparsity pattern of the relaxations, we decompose the problems and obtain equivalent relaxations in a much smaller cone, making the conic approaches scalable. To make the best usage of the decomposed relaxations, we propose heuristics using the information of its optimal solution. Moreover, an exact procedure is proposed by solving a sequence of mixed-integer decomposed semidefinite optimization problems. Numerical results on classical benchmarking datasets are reported, showing the efficiency and effectiveness of our approach.

math.OC↗

arXiv2024

The forgotten pillar of sustainability: development of the S-assessment tool to evaluate Organizational Social Sustainability

Pursuing sustainable development has become a global imperative, underscored adopting of the 2030 Agenda for Sustainable Development and its 17 Sustainable Development Goals (SDG). At the heart of this agenda lies the recognition of social sustainability as a pivotal component, emphasizing the need for inclusive societies where every individual can thrive. Despite its significance, social sustainability remains a "forgotten pillar," often overshadowed by environmental concerns. In response, this paper presents the development and validation of the S-Assessment Tool for Social Sustainability, a comprehensive questionnaire designed to evaluate organizations' performance across critical dimensions such as health and wellness, gender equality, decent work, and economic growth, reducing inequalities, and responsible production and consumption. The questionnaire was constructed on the critical dimensions identified through a systematic and narrative hybrid approach to the analysis of peer-reviewed literature. The framework has been structured around the values of the SDGs. It aims to empower organizations to better understand and address their social impact, fostering positive change and contributing to the collective effort towards a more equitable and sustainable future. Through collaborative partnerships and rigorous methodology, this research underscores the importance of integrating social sustainability into organizational practices and decision-making processes, ultimately advancing the broader agenda of sustainable development.

econ.GN↗

arXiv2024

Benders decomposition for congested partial set covering location with uncertain demand

In this paper, we introduce a mixed integer quadratic formulation for the congested variant of the partial set covering location problem, which involves determining a subset of facility locations to open and efficiently allocating customers to these facilities to minimize the combined costs of facility opening and congestion while ensuring target coverage. To enhance the resilience of the solution against demand fluctuations, we address the case under uncertain customer demand using $\Gamma$-robustness. We formulate the deterministic problem and its robust counterpart as mixed-integer quadratic problems. We investigate the effect of the protection level in adapted instances from the literature to provide critical insights into how sensitive the planning is to the protection level. Moreover, since the size of the robust counterpart grows with the number of customers, which could be significant in real-world contexts, we propose the use of Benders decomposition to effectively reduce the number of variables by projecting out of the master problem all the variables dependent on the number of customers. We illustrate how to incorporate our Benders approach within a mixed-integer second-order cone programming (MISOCP) solver, addressing explicitly all the ingredients that are instrumental for its success. We discuss single-tree and multi-tree approaches and introduce a perturbation technique to deal with the degeneracy of the Benders subproblem efficiently. Our tailored Benders approaches outperform the perspective reformulation solved using the state-of-the-art MISOCP solver Gurobi on adapted instances from the literature.

math.OC↗

arXiv2023

A graph neural network-based model with Out-of-Distribution Robustness for enhancing Antiretroviral Therapy Outcome Prediction for HIV-1

Predicting the outcome of antiretroviral therapies (ART) for HIV-1 is a pressing clinical challenge, especially when the ART includes drugs with limited effectiveness data. This scarcity of data can arise either due to the introduction of a new drug to the market or due to limited use in clinical settings, resulting in clinical dataset with highly unbalanced therapy representation. To tackle this issue, we introduce a novel joint fusion model, which combines features from a Fully Connected (FC) Neural Network and a Graph Neural Network (GNN) in a multi-modality fashion. Our model uses both tabular data about genetic sequences and a knowledge base derived from Stanford drug-resistance mutation tables, which serve as benchmark references for deducing in-vivo treatment efficacy based on the viral genetic sequence. By leveraging this knowledge base structured as a graph, the GNN component enables our model to adapt to imbalanced data distributions and account for Out-of-Distribution (OoD) drugs. We evaluated these models' robustness against OoD drugs in the test set. Our comprehensive analysis demonstrates that the proposed model consistently outperforms the FC model. These results underscore the advantage of integrating Stanford scores in the model, thereby enhancing its generalizability and robustness, but also extending its utility in contributing in more informed clinical decisions with limited data availability. The source code is available at https://github.com/federicosiciliano/graph-ood-hiv

q-bio.QM↗

arXiv2023

Incorporating temporal dynamics of mutations to enhance the prediction capability of antiretroviral therapy's outcome for HIV-1

Motivation: In predicting HIV therapy outcomes, a critical clinical question is whether using historical information can enhance predictive capabilities compared with current or latest available data analysis. This study analyses whether historical knowledge, which includes viral mutations detected in all genotypic tests before therapy, their temporal occurrence, and concomitant viral load measurements, can bring improvements. We introduce a method to weigh mutations, considering the previously enumerated factors and the reference mutation-drug Stanford resistance tables. We compare a model encompassing history (H) with one not using it (NH). Results: The H-model demonstrates superior discriminative ability, with a higher ROC-AUC score (76.34%) than the NH-model (74.98%). Significant Wilcoxon test results confirm that incorporating historical information improves consistently predictive accuracy for treatment outcomes. The better performance of the H-model might be attributed to its consideration of latent HIV reservoirs, probably obtained when leveraging historical information. The findings emphasize the importance of temporal dynamics in mutations, offering insights into HIV infection complexities. However, our result also shows that prediction accuracy remains relatively high even when no historical information is available. Supplementary information: Supplementary material is available.

cs.LG↗

arXiv2023

CMA Light: a novel Minibatch Algorithm for large-scale non convex finite sum optimization

The supervised training of a deep neural network on a given dataset consists in the unconstrained minimization of the finite sum of continuously differentiable functions, commonly referred to as loss with respect to the samples. These functions depend on the network parameters and most of the times are non-convex. We develop CMA Light, a globally convergent mini-batch gradient method to tackle this problem. We consider the recently introduced Controlled Minibatch Algorithm (CMA) framework and we overcome its main bottleneck, removing the need for at least one evaluation of the whole objective function per iteration. We prove globally convergence of CMA Light under mild assumptions and we discuss extensive computational results on the same experimental test-bed used for CMA, showing that CMA Light requires less computational effort than most of the state-of-the-art optimizers. Eventually, we present early results on a large-scale Image Classification task.

math.OC↗

arXiv2023

A computational study of off-the-shelf MINLP solvers on a benchmark set of congested capacitated facility location problems

This paper analyzes the performance of five well-known off-the-shelf optimization solvers on a set of congested capacitated facility location problems formulated as mixed-integer conic programs (MICPs). We aim to compare the computational efficiency of the solvers and examine the solution strategies they adopt when solving instances with different sizes and complexity. The solvers we compare are Gurobi, Cplex, Mosek, Xpress, and Scip. We run extensive numerical tests on a testbed of 30 instances from the literature. Our results show that Mosek and Gurobi are the most competitive solvers, as they achieve better time and gap performance, solving most instances within the time limit. Mosek outperforms Gurobi in large-size problems and provides more accurate solutions in terms of feasibility. Xpress solves to optimality about half of the instances tested within the time limit, and in this half, it achieves performance similar to that of Gurobi and Mosek. Cplex and Scip emerge as the least competitive solvers. The results provide guidelines on how each solver behaves on this class of problems and highlight the importance of choosing a solver suited to the problem type.

math.OC↗

arXiv2023

Unboxing Tree Ensembles for interpretability: a hierarchical visualization tool and a multivariate optimal re-built tree

The interpretability of models has become a crucial issue in Machine Learning because of algorithmic decisions' growing impact on real-world applications. Tree ensemble methods, such as Random Forests or XgBoost, are powerful learning tools for classification tasks. However, while combining multiple trees may provide higher prediction quality than a single one, it sacrifices the interpretability property resulting in "black-box" models. In light of this, we aim to develop an interpretable representation of a tree-ensemble model that can provide valuable insights into its behavior. First, given a target tree-ensemble model, we develop a hierarchical visualization tool based on a heatmap representation of the forest's feature use, considering the frequency of a feature and the level at which it is selected as an indicator of importance. Next, we propose a mixed-integer linear programming (MILP) formulation for constructing a single optimal multivariate tree that accurately mimics the target model predictions. The goal is to provide an interpretable surrogate model based on oblique hyperplane splits, which uses only the most relevant features according to the defined forest's importance indicators. The MILP model includes a penalty on feature selection based on their frequency in the forest to further induce sparsity of the splits. The natural formulation has been strengthened to improve the computational performance of {mixed-integer} software. Computational experience is carried out on benchmark datasets from the UCI repository using a state-of-the-art off-the-shelf solver. Results show that the proposed model is effective in yielding a shallow interpretable tree approximating the tree-ensemble decision function.

math.OC↗

arXiv2022

Convergence of ease-controlled Random Reshuffling gradient Algorithms under Lipschitz smoothness

In this work, we consider minimizing the average of a very large number of smooth and possibly non-convex functions, and we focus on two widely used minibatch frameworks to tackle this optimization problem: Incremental Gradient (IG) and Random Reshuffling (RR). We define ease-controlled modifications of the IG/RR schemes, which require a light additional computational effort {but} can be proved to converge under {weak} and standard assumptions. In particular, we define two algorithmic schemes in which the IG/RR iteration is controlled by using a watchdog rule and a derivative-free linesearch that activates only sporadically to guarantee convergence. The two schemes differ in the watchdog and the linesearch, which are performed using either a monotonic or a non-monotonic rule. The two schemes also allow controlling the updating of the stepsize used in the main IG/RR iteration, avoiding the use of pre-set rules that may drive the stepsize to zero too fast, reducing the effort in designing effective updating rules of the stepsize. We prove convergence under the mild assumption of Lipschitz continuity of the gradients of the component functions and perform extensive computational analysis using different deep neural architectures and a benchmark of varying-size datasets. We compare our implementation with both a full batch gradient method (i.e. L-BFGS) and an implementation of IG/RR methods, proving that our algorithms require a similar computational effort compared to the other online algorithms and that the control on the learning rate may allow a faster decrease of the objective function.

math.OC↗

arXiv2022

Margin Optimal Classification Trees

In recent years, there has been growing attention to interpretable machine learning models which can give explanatory insights on their behaviour. Thanks to their interpretability, decision trees have been intensively studied for classification tasks and, due to the remarkable advances in mixed integer programming (MIP), various approaches have been proposed to formulate the problem of training an Optimal Classification Tree (OCT) as a MIP model. We present a novel mixed integer quadratic formulation for the OCT problem, which exploits the generalization capabilities of Support Vector Machines for binary classification. Our model, denoted as Margin Optimal Classification Tree (MARGOT), encompasses maximum margin multivariate hyperplanes nested in a binary tree structure. To enhance the interpretability of our approach, we analyse two alternative versions of MARGOT, which include feature selection constraints inducing sparsity of the hyperplanes' coefficients. First, MARGOT has been tested on non-linearly separable synthetic datasets in a 2-dimensional feature space to provide a graphical representation of the maximum margin approach. Finally, the proposed models have been tested on benchmark datasets from the UCI repository. The MARGOT formulation turns out to be easier to solve than other OCT approaches, and the generated tree better generalizes on new observations. The two interpretable versions effectively select the most relevant features, maintaining good prediction quality.

math.OC↗

arXiv2022

Speeding up the solution of the Site and Power Assignment Problem in Wireless Networks

This paper addresses the optimal design of wireless networks through the site and power assignment problem. Given a set of candidate transmitters, this problem involves choosing optimal transmitter locations and powers to provide service coverage over a target area. In the modern context of increasing traffic, establishing suitable locations and power emissions for the transmitters in wireless networks is a relevant and challenging task due to heavy radio spectrum congestion. Traditional network design formulations are very ill-conditioned and suffer from numerical inaccuracies and limited applicability to large-scale practical scenarios. Our contribution consists of speeding up the solution of the problem under consideration by addressing its drawbacks from a modeling point of view. We propose valid cutting planes and various presolve operations to reduce the problem size and strengthen existing formulations, along with a reduction scheme based on reduced cost fixing to reduce the sources of numerical inaccuracies. Our proposals prove effective, allowing us to achieve optimality on large-scale instances obtained from a real 4G LTE network in solution times aligning well with planning windows.

cs.DM↗

arXiv2022

Solving the vehicle routing problem with deep reinforcement learning

Recently, the applications of the methodologies of Reinforcement Learning (RL) to NP-Hard Combinatorial optimization problems have become a popular topic. This is essentially due to the nature of the traditional combinatorial algorithms, often based on a trial-and-error process. RL aims at automating this process. At this regard, this paper focuses on the application of RL for the Vehicle Routing Problem (VRP), a famous combinatorial problem that belongs to the class of NP-Hard problems. In this work, first, the problem is modeled as a Markov Decision Process (MDP) and then the PPO method (which belongs to the Actor-Critic class of Reinforcement learning methods) is applied. In a second phase, the neural architecture behind the Actor and Critic has been established, choosing to adopt a neural architecture based on the Convolutional neural networks, both for the Actor and the Critic. This choice resulted in effectively addressing problems of different sizes. Experiments performed on a wide range of instances show that the algorithm has good generalization capabilities and can reach good solutions in a short time. Comparisons between the algorithm proposed and the state-of-the-art solver OR-TOOLS show that the latter still outperforms the Reinforcement learning algorithm. However, there are future research perspectives, that aim to upgrade the current performance of the algorithm proposed.

math.OC↗

arXiv2022

AI-based Data Preparation and Data Analytics in Healthcare: The Case of Diabetes

The Associazione Medici Diabetologi (AMD) collects and manages one of the largest worldwide-available collections of diabetic patient records, also known as the AMD database. This paper presents the initial results of an ongoing project whose focus is the application of Artificial Intelligence and Machine Learning techniques for conceptualizing, cleaning, and analyzing such an important and valuable dataset, with the goal of providing predictive insights to better support diabetologists in their diagnostic and therapeutic choices.

cs.LG↗

arXiv2020

Block Layer Decomposition schemes for training Deep Neural Networks

Deep Feedforward Neural Networks' (DFNNs) weights estimation relies on the solution of a very large nonconvex optimization problem that may have many local (no global) minimizers, saddle points and large plateaus. As a consequence, optimization algorithms can be attracted toward local minimizers which can lead to bad solutions or can slow down the optimization process. Furthermore, the time needed to find good solutions to the training problem depends on both the number of samples and the number of variables. In this work, we show how Block Coordinate Descent (BCD) methods can be applied to improve performance of state-of-the-art algorithms by avoiding bad stationary points and flat regions. We first describe a batch BCD method ables to effectively tackle the network's depth and then we further extend the algorithm proposing a \textit{minibatch} BCD framework able to scale with respect to both the number of variables and the number of samples by embedding a BCD approach into a minibatch framework. By extensive numerical results on standard datasets for several architecture networks, we show how the application of BCD methods to the training phase of DFNNs permits to outperform standard batch and minibatch algorithms leading to an improvement on both the training phase and the generalization performance of the networks.

math.OC↗

arXiv2015

A Class of Parallel Decomposition Algorithms for SVMs Training

The training of Support Vector Machines may be a very difficult task when dealing with very large datasets. The memory requirement and the time consumption of the SVMs algorithms grow rapidly with the increase of the data. To overcome these drawbacks, we propose a parallel decomposition algorithmic scheme for SVMs training for which we prove global convergence under suitable conditions. We outline how these assumptions can be satisfied in practice and we suggest various specific implementations exploiting the adaptable structure of the algorithmic model.

math.OC↗