arXiv subjects

S. Ilker Birbil

Publications and source records attributed to S. Ilker Birbil.

13 recordsLinked to original sources

arXiv2026

Optimization with Dynamic Constraint Learning (DCL)

We propose Dynamic Constraint Learning (DCL), a data-driven framework for constrained optimization when constraint functions are unknown and cannot be queried during optimization. At each iteration, the method learns a local surrogate from nearby data and solves a subproblem within a data-supported trust region. Compared with offline global constraint learning, the approach uses local surrogates that adapt to the data distribution during optimization and can achieve solution quality comparable to that of global models while using simpler local models and smaller optimization subproblems. We demonstrate the performance of DCL on a synthetic test problem and two case studies from the literature.

cs.LG↗

arXiv2026

The Role of Feature Interactions in Graph-based Tabular Deep Learning

Accurate predictions on tabular data rely on capturing complex, dataset-specific feature interactions. Attention-based methods and graph neural networks, referred to as graph-based tabular deep learning (GTDL), aim to improve predictions by modeling these interactions as a graph. In this work, we analyze how these methods model the feature interactions. Current GTDL approaches primarily focus on optimizing predictive accuracy, often neglecting the accurate modeling of the underlying graph structure. Using synthetic datasets with known ground-truth graph structures, we find that current GTDL methods fail to recover meaningful feature interactions, as their edge recovery is close to random. This suggests that the attention mechanism and message-passing schemes used in GTDL do not effectively capture feature interactions. Furthermore, when we impose the true interaction structure, we find that the predictive accuracy improves. This highlights the need for GTDL methods to prioritize accurate modeling of the graph structure, as it leads to better predictions.

cs.LG↗

arXiv2025

Scalable Bayesian Structure Learning for Gaussian Graphical Models Using Marginal Pseudo-likelihood

Bayesian methods for learning Gaussian graphical models offer a principled framework for quantifying model uncertainty and incorporating prior knowledge. However, their scalability is constrained by the computational cost of jointly exploring graph structures and precision matrices. To address this challenge, we perform inference directly on the graph by integrating out the precision matrix. We adopt a marginal pseudo-likelihood approach, eliminating the need to compute intractable normalizing constants and perform computationally intensive precision matrix sampling. Building on this framework, we develop continuous-time (birth-death) and discrete-time (reversible jump) Markov chain Monte Carlo (MCMC) algorithms that efficiently explore the posterior over graph space. We establish theoretical guarantees for posterior contraction, convergence, and graph selection consistency. The algorithms scale to large graph spaces, enabling parallel exploration for graphs with over 1,000 nodes, while providing uncertainty quantification and supporting flexible prior specification over the graph space. Extensive simulations show substantial computational gains over state-of-the-art Bayesian approaches without sacrificing graph recovery accuracy. Applications to human and mouse gene expression datasets demonstrate the ability of our approach to recover biologically meaningful structures and quantify uncertainty in complex networks. An implementation is available in the R package BDgraph.

stat.ME↗

arXiv2025

Coherent Local Explanations for Mathematical Optimization

The surge of explainable artificial intelligence methods seeks to enhance transparency and explainability in machine learning models. At the same time, there is a growing demand for explaining decisions taken through complex algorithms used in mathematical optimization. However, current explanation methods do not take into account the structure of the underlying optimization problem, leading to unreliable outcomes. In response to this need, we introduce Coherent Local Explanations for Mathematical Optimization (CLEMO). CLEMO provides explanations for multiple components of optimization models, the objective value and decision variables, which are coherent with the underlying model structure. Our sampling-based procedure can provide explanations for the behavior of exact and heuristic solution algorithms. The effectiveness of CLEMO is illustrated by experiments for the shortest path problem, the knapsack problem, and the vehicle routing problem.

math.OC↗

arXiv2025

Modeling Alzheimer's Disease: Bayesian Copula Graphical Model from Demographic, Cognitive, and Neuroimaging Data

The early detection of Alzheimer's disease (AD) requires an understanding of the relationships between a wide range of features. Conditional independencies and partial correlations are suitable measures for these relationships, because they can identify the effects of confounding and mediating variables. This article presents a Bayesian approach to Gaussian copula graphical models (GCGMs) in order to estimate these conditional dependencies and partial correlations. This approach has two key advantages. First, it includes binary, discrete, and continuous variables. Second, it quantifies the uncertainty of the estimates. Despite these advantages, Bayesian GCGMs have not been applied to AD research yet. In this study, we design a GCGM to find the conditional dependencies and partial correlations among brain-region specific gray matter volume and glucose uptake, amyloid-beta levels, demographic information, and cognitive test scores. We applied our model to 1022 participants, including healthy and cognitively impaired, across different stages of AD. We found that aging reduces cognition through three indirect pathways: hippocampal volume loss, posterior cingulate cortex (PCC) volume loss, and amyloid-beta accumulation. We found a positive partial correlation between being woman and cognition, but also discovered four indirect pathways that dampen this association in women: lower hippocampal volume, lower PCC volume, more amyloid-beta accumulation, and less education. We found limited relations between brain-region specific glucose uptake and cognition, but discovered that the hippocampus and PCC volumes are related to cognition. These results show that the use of GCGMs offers valuable insights into AD pathogenesis.

stat.AP↗

arXiv2024

Bayesian Structure Learning in Undirected Gaussian Graphical Models: Literature Review with Empirical Comparison

Gaussian graphical models provide a powerful framework to reveal the conditional dependency structure between multivariate variables. The process of uncovering the conditional dependency network is known as structure learning. Bayesian methods can measure the uncertainty of conditional relationships and include prior information. However, frequentist methods are often preferred due to the computational burden of the Bayesian approach. Over the last decade, Bayesian methods have seen substantial improvements, with some now capable of generating accurate estimates of graphs up to a thousand variables in mere minutes. Despite these advancements, a comprehensive review or empirical comparison of all recent methods has not been conducted. This paper delves into a wide spectrum of Bayesian approaches used for structure learning and evaluates their efficacy through a comprehensive simulation study. We also demonstrate how to apply Bayesian structure learning to a real-world data set and provide directions for future research. This study gives an exhaustive overview of this dynamic field for newcomers, practitioners, and experts.

stat.ME↗

arXiv2024

Bolstering Stochastic Gradient Descent with Model Building

Stochastic gradient descent method and its variants constitute the core optimization algorithms that achieve good convergence rates for solving machine learning problems. These rates are obtained especially when these algorithms are fine-tuned for the application at hand. Although this tuning process can require large computational costs, recent work has shown that these costs can be reduced by line search methods that iteratively adjust the step length. We propose an alternative approach to stochastic line search by using a new algorithm based on forward step model building. This model building step incorporates second-order information that allows adjusting not only the step length but also the search direction. Noting that deep learning model parameters come in groups (layers of tensors), our method builds its model and calculates a new step for each parameter group. This novel diagonalization approach makes the selected step lengths adaptive. We provide convergence rate analysis, and experimentally show that the proposed algorithm achieves faster convergence and better generalization in well-known test problems. More precisely, SMB requires less tuning, and shows comparable performance to other adaptive methods.

cs.LG↗

arXiv2024

Differentially Private Linear Optimization for Multi-Party Resource Sharing

This study examines a resource-sharing problem involving multiple parties that agree to use a set of capacities together. We start with modeling the whole problem as a mathematical program, where all parties are required to exchange information to obtain the optimal objective function value. This information bears private data from each party in terms of coefficients used in the mathematical program. Moreover, the parties also consider the individual optimal solutions as private. In this setting, the concern for the parties is the privacy of their data and their optimal allocations. We propose a two-step approach to meet the privacy requirements of the parties. In the first step, we obtain a reformulated model that is amenable to a decomposition scheme. Although this scheme eliminates almost all data exchanges, it does not provide a formal privacy guarantee. In the second step, we provide this guarantee with a locally differentially private algorithm, which does not need a trusted aggregator, at the expense of deviating slightly from the optimality. We provide bounds on this deviation and discuss the consequences of these theoretical results. We also propose a novel modification to increase the efficiency of the algorithm in terms of reducing the theoretical optimality gap. The study ends with a numerical experiment on a planning problem that demonstrates an application of the proposed approach. As we work with a general linear optimization model, our analysis and discussion can be used in different application areas including production planning, logistics, and revenue management.

math.OC↗

arXiv2023

Mixed-Integer Optimization with Constraint Learning

We establish a broad methodological foundation for mixed-integer optimization with learned constraints. We propose an end-to-end pipeline for data-driven decision making in which constraints and objectives are directly learned from data using machine learning, and the trained models are embedded in an optimization formulation. We exploit the mixed-integer optimization-representability of many machine learning methods, including linear models, decision trees, ensembles, and multi-layer perceptrons, which allows us to capture various underlying relationships between decisions, contextual variables, and outcomes. We also introduce two approaches for handling the inherent uncertainty of learning from data. First, we characterize a decision trust region using the convex hull of the observations, to ensure credible recommendations and avoid extrapolation. We efficiently incorporate this representation using column generation and propose a more flexible formulation to deal with low-density regions and high-dimensional datasets. Then, we propose an ensemble learning approach that enforces constraint satisfaction over multiple bootstrapped estimators or multiple algorithms. In combination with domain-driven components, the embedded models and trust region define a mixed-integer optimization problem for prescription generation. We implement this framework as a Python package (OptiCL) for practitioners. We demonstrate the method in both World Food Programme planning and chemotherapy optimization. The case studies illustrate the framework's ability to generate high-quality prescriptions as well as the value added by the trust region, the use of ensembles to control model robustness, the consideration of multiple machine learning methods, and the inclusion of multiple learned constraints.

math.OC↗

arXiv2021

Masking Primal and Dual Models for Data Privacy in Network Revenue Management

We study a collaborative revenue management problem where multiple decentralized parties agree to share some of their capacities. This collaboration is performed by constructing a large mathematical programming model available to all parties. The parties then use the solution of this model in their own capacity control systems. In this setting, however, the major concern for the parties is the privacy of their input data along with their individual optimal solutions. We first reformulate a general linear programming model that can be used for a wide-range of network revenue management problems. Then, we address the data-privacy concern of the reformulated model and propose an approach based on solving an equivalent data-private model constructed with input masking via random transformations. Our main result shows that after solving the data-private model, each party can safely access only its own optimal capacity control decisions. We also discuss the security of the transformed problem in the considered multi-party setting. We conduct simulation experiments to support our results and evaluate the computational efficiency of the proposed data-private model. Our work provides an analytical approach and insights on how to manage shared resources in a network problem while ensuring data privacy. Constructing and solving the collaborative network problem requires information exchange between parties which may not be possible in practice. Including data-privacy in decentralized collaborative network revenue management problems with capacity sharing is new to the literature and relevant to practice.

math.OC↗

arXiv2020

Rule Covering for Interpretation and Boosting

We propose two algorithms for interpretation and boosting of tree-based ensemble methods. Both algorithms make use of mathematical programming models that are constructed with a set of rules extracted from an ensemble of decision trees. The objective is to obtain the minimum total impurity with the least number of rules that cover all the samples. The first algorithm uses the collection of decision trees obtained from a trained random forest model. Our numerical results show that the proposed rule covering approach selects only a few rules that could be used for interpreting the random forest model. Moreover, the resulting set of rules closely matches the accuracy level of the random forest model. Inspired by the column generation algorithm in linear programming, our second algorithm uses a rule generation scheme for boosting decision trees. We use the dual optimal solutions of the linear programming models as sample weights to obtain only those rules that would improve the accuracy. With a computational study, we observe that our second algorithm performs competitively with the other well-known boosting methods. Our implementations also demonstrate that both algorithms can be trivially coupled with the existing random forest and decision tree packages.

cs.LG↗

arXiv2020

Low-dimensional Interpretable Kernels with Conic Discriminant Functions for Classification

Kernels are often developed and used as implicit mapping functions that show impressive predictive power due to their high-dimensional feature space representations. In this study, we gradually construct a series of simple feature maps that lead to a collection of interpretable low-dimensional kernels. At each step, we keep the original features and make sure that the increase in the dimension of input data is extremely low, so that the resulting discriminant functions remain interpretable and amenable to fast training. Despite our persistence on interpretability, we obtain high accuracy results even without in-depth hyperparameter tuning. Comparison of our results against several well-known kernels on benchmark datasets show that the proposed kernels are competitive in terms of prediction accuracy, while the training times are significantly lower than those obtained with state-of-the-art kernel implementations.

cs.LG↗

arXiv2013

Trade-offs Computing Minimum Hub Cover toward Optimized Graph Query Processing

As techniques for graph query processing mature, the need for optimization is increasingly becoming an imperative. Indices are one of the key ingredients toward efficient query processing strategies via cost-based optimization. Due to the apparent absence of a common representation model, it is difficult to make a focused effort toward developing access structures, metrics to evaluate query costs, and choose alternatives. In this context, recent interests in covering-based graph matching appears to be a promising direction of research. In this paper, our goal is to formally introduce a new graph representation model, called Minimum Hub Cover, and demonstrate that this representation offers interesting strategic advantages, facilitates construction of candidate graphs from graph fragments, and helps leverage indices in novel ways for query optimization. However, similar to other covering problems, minimum hub cover is NP-hard, and thus is a natural candidate for optimization. We claim that computing the minimum hub cover leads to substantial cost reduction for graph query processing. We present a computational characterization of minimum hub cover based on integer programming to substantiate our claim and investigate its computational cost on various graph types.

cs.DB↗