arXiv subjects

Sven Leyffer

Publications and source records attributed to Sven Leyffer.

At least 19 recordsLinked to original sources

arXiv2025

An Alternating Direction Method of Multipliers for Topology Optimization

We consider a class of integer-constrained optimization problems governed by partial differential equation (PDE) constraints and regularized via total variation (TV) in the context of topology optimization. The presence of discrete design variables, nonsmooth regularization, and non-convex objective renders the problem computationally challenging. To address this, we adopt the alternating direction method of multipliers (ADMM) framework, which enables a decomposition of the original problem into simpler subproblems that can be solved efficiently. The augmented Lagrangian formulation ensures consistency across variable updates while facilitating convergence under appropriate conditions.

math.OC↗

arXiv2025

Optimal Experimental Design of a Moving Sensor for Linear Bayesian Inverse Problems

We optimize the path of a mobile sensor to minimize the posterior uncertainty of a Bayesian inverse problem. Along its path, the sensor continuously takes measurements of the state, which is a physical quantity modeled as the solution of a partial differential equation (PDE) with uncertain parameters. Considering linear PDEs specifically, we derive the closed-form expression of the posterior covariance matrix of the model parameters as a function of the path, and formulate the optimal experimental design problem for minimizing the posterior's uncertainty. We discretize the problem such that the cost function remains consistent under temporal refinement. Additional constraints ensure that the path avoids obstacles and remains physically interpretable through a control parameterization. The constrained optimization problem is solved using an interior-point method. We present computational results for a convection-diffusion equation with unknown initial condition.

cs.CE↗

arXiv2025

Optimizing Paths for Adaptive Fly-Scan Microscopy: An Extended Version

In x-ray microscopy, traditional raster-scanning techniques are used to acquire a microscopic image in a series of step-scans. Alternatively, scanning the x-ray probe along a continuous path, called a fly-scan, reduces scan time and increases scan efficiency. However, not all regions of an image are equally important. Currently used fly-scan methods do not adapt to the characteristics of the sample during the scan, often wasting time in uniform, uninteresting regions. One approach to avoid unnecessary scanning in uniform regions for raster step-scans is to use deep learning techniques to select a shorter optimal scan path instead of a traditional raster scan path, followed by reconstructing the entire image from the partially scanned data. However, this approach heavily depends on the quality of the initial sampling, requires a large dataset for training, and incurs high computational costs. We propose leveraging the fly-scan method along an optimal scanning path, focusing on regions of interest (ROIs) and using image completion techniques to reconstruct details in non-scanned areas. This approach further shortens the scanning process and potentially decreases x-ray exposure dose while maintaining high-quality and detailed information in critical regions. To achieve this, we introduce a multi-iteration fly-scan framework that adapts to the scanned image. Specifically, in each iteration, we define two key functions: (1) a score function to generate initial anchor points and identify potential ROIs, and (2) an objective function to optimize the anchor points for convergence to an optimal set. Using these anchor points, we compute the shortest scanning path between optimized anchor points, perform the fly-scan, and subsequently apply image completion based on the acquired information in preparation for the next scan iteration.

eess.IV↗

arXiv2025

Augmentation Algorithms for Integer Programs with Total Variation-like Regularization

We address a class of integer optimization programs with a total variation-like regularizer and convex, separable constraints on a graph. Our approach makes use of the Graver basis, an optimality certificate for integer programs, which we characterize as corresponding to the collection of induced connected subgraphs of our graph. We demonstrate how to use this basis to craft an exact global optimization algorithm for the unconstrained problem recovering a method first shown by Kolmogorov and Shioura in 2009. We then address the problem with an additional budget constraint with a randomized heuristic algorithm that samples improving moves from the Graver basis in a randomized variant of the simplex algorithm. Through comprehensive experiments, we demonstrate that this randomized algorithm is competitive with and often outperforms state-of-the-art integer program solvers.

math.OC↗

arXiv2025

Sampling Imbalanced Data with Multi-objective Bilevel Optimization

Two-class classification problems are often characterized by an imbalance between the number of majority and minority datapoints resulting in poor classification of the minority class in particular. Traditional approaches, such as reweighting the loss function or na\"ive resampling, risk overfitting and subsequently fail to improve classification because they do not consider the diversity between majority and minority datasets. Such consideration is infeasible because there is no metric that can measure the impact of imbalance on the model. To obviate these challenges, we make two key contributions. First, we introduce MOODS~(Multi-Objective Optimization for Data Sampling), a novel multi-objective bilevel optimization framework that guides both synthetic oversampling and majority undersampling. Second, we introduce a validation metric -- `$\epsilon/ \delta$ non-overlapping diversification metric' -- that quantifies the goodness of a sampling method towards model performance. With this metric we experimentally demonstrate state-of-the-art performance with improvement in diversity driving a $1-15 \%$ increase in $F1$ scores.

cs.LG↗

arXiv2025

A Globally Convergent Method for Computing B-stationary Points of Mathematical Programs with Equilibrium Constraints

This paper introduces a computationally efficient method that converges globally to B-stationary points of mathematical programs with equilibrium constraints (MPECs). B-stationarity is necessary for optimality and means that no feasible first-order direction can improve the objective. It can be certified by solving a linear program with equilibrium constraints (LPEC) constructed at a given feasible point. The proposed method solves a finite sequence of LPECs, which either certify B-stationarity or provide an active set estimate for the complementarity constraints, along with branch nonlinear programs (BNLPs) obtained by fixing the complementarity active set in the MPEC. In particular, the method proceeds in two phases: the first identifies a feasible BNLP or a stationary point of a constraint infeasibility minimization problem, and the second solves a sequence of BNLPs until a B-stationary point of the MPEC is found. We prove that under the MPEC-MFCQ, the method requires solving only a finite number of BNLPs and LPECs for convergence. Moreover, we show that, unless the current iterate is B-stationary, the combinatorial LPECs need not be solved to optimality. For convergence, it suffices to compute a nonzero feasible point, which in practice often requires solving a single linear program, yielding significant computational savings. Numerical experiments show that the proposed method is more robust and faster than relaxation-based methods and mixed-integer NLP reformulations (which, in contrast to the proposed approach, do not provide a certificate of B-stationarity), even on medium- to large-scale instances.

math.OC↗

arXiv2024

A Bilevel Optimization Framework for Imbalanced Data Classification

Data rebalancing techniques, including oversampling and undersampling, are a common approach to addressing the challenges of imbalanced data. To tackle unresolved problems related to both oversampling and undersampling, we propose a new undersampling approach that: (i) avoids the pitfalls of noise and overlap caused by synthetic data and (ii) avoids the pitfall of under-fitting caused by random undersampling. Instead of undersampling majority data randomly, our method undersamples datapoints based on their ability to improve model loss. Using improved model loss as a proxy measurement for classification performance, our technique assesses a datapoint's impact on loss and rejects those unable to improve it. In so doing, our approach rejects majority datapoints redundant to datapoints already accepted and, thereby, finds an optimal subset of majority training data for classification. The accept/reject component of our algorithm is motivated by a bilevel optimization problem uniquely formulated to identify the optimal training set we seek. Experimental results show our proposed technique with F1 scores up to 10% higher than state-of-the-art methods.

cs.LG↗

arXiv2024

A Unified Funnel Restoration SQP Algorithm

We consider nonlinearly constrained optimization problems and discuss a generic double-loop framework consisting of four algorithmic ingredients that unifies a broad range of nonlinear optimization solvers. This framework has been implemented in the open-source solver Uno, a Swiss Army knife-like C++ optimization framework that unifies many nonlinearly constrained nonconvex optimization solvers. We illustrate the framework with a sequential quadratic programming (SQP) algorithm that maintains an acceptable upper bound on the constraint violation, called a funnel, that is monotonically decreased to control the feasibility of the iterates. Infeasible quadratic subproblems are handled by a feasibility restoration strategy. Globalization is controlled by a line search or a trust-region method. We prove global convergence of the trust-region funnel SQP method, building on known results from filter methods. We implement the algorithm in Uno, and we provide extensive test results for the trust-region line-search funnel SQP on small CUTEst instances.

math.OC↗

arXiv2024

Modeling and solving cascading failures across interdependent infrastructure systems

Physical infrastructure systems supply crucial resources to residential, commercial, and industrial activities. These infrastructure systems generally consist of multiple types of infrastructure assets that are interdependent. In the event of a disaster, some of the infrastructure assets can be damaged and disabled, creating failures that propagate to other assets that depend on the disabled assets and cause a cascade of failures that may lead to a potential system collapse. We present a bilevel interdiction model in this paper to study this problem of cascading failures in a system of interdependent infrastructure systems with a nondeterministic dependency graph. We also propose a computationally tractable reformulation of the proposed bilevel model and utilize a Benders-type decomposition algorithm to solve the resulting formulation. Computational experiments are performed using infrastructure networks generated from anonymized real-world data to validate the performance of this algorithm.

math.OC↗

arXiv2024

Implementing a unified solver for nonlinearly constrained optimization

SQP and interior-point methods (also referred to as Lagrange-Newton methods) typically share key algorithmic components, such as strategies for computing descent directions and mechanisms that promote global convergence. Building on this insight, we introduce a unifying framework with eight building blocks that abstracts the workflows of Lagrange-Newton methods. We then present Uno, a modular C++ solver that implements our unifying framework and allows the automatic combination of a wide range of strategies with no programming effort from the user. Uno is meant to (1) organize mathematical optimization strategies into a coherent hierarchy; (2) offer a wide range of efficient and robust methods that can be compared for a given instance; (3) enable researchers to experiment with novel optimization strategies; and (4) reduce the cost of development and maintenance of multiple optimization solvers. Uno's software design allows user to compose new customized solvers for emerging optimization areas such as robust optimization or optimization problems with complementarity constraints, while building on reliable nonlinear optimization techniques. We demonstrate that Uno is highly competitive against state-of-the-art solvers filterSQP, IPOPT, SNOPT, MINOS, LANCELOT, LOQO, and CONOPT on a subset of 429 small problems from the CUTE collection. Uno is available as open-source software under the MIT license at https://github.com/cvanaret/Uno and via its C, Julia, Python, Fortran, and AMPL interfaces.

math.OC↗

arXiv2024

McCormick envelopes in mixed-integer PDE-constrained optimization

McCormick envelopes are a standard tool for deriving convex relaxations of optimization problems that involve polynomial terms. Such McCormick relaxations provide lower bounds, for example, in branch-and-bound procedures for mixed-integer nonlinear programs but have not gained much attention in PDE-constrained optimization so far. This lack of attention may be due to the distributed nature of such problems, which on the one hand leads to infinitely many linear constraints (generally state constraints that may be difficult to handle) in addition to the state equation for a pointwise formulation of the McCormick envelopes and renders bound-tightening procedures that successively improve the resulting convex relaxations computationally intractable. We analyze McCormick envelopes for a problem class that is governed by a semilinear PDE involving a bilinearity and integrality constraints. We approximate the nonlinearity by averaging the involved terms over the cells of a partition of the computational domain on which the PDE is defined. This yields convex relaxations that underestimate the original problem up to an a priori error estimate that depends on the mesh size of the discretization. These approximate McCormick relaxations can be improved by means of an optimization-based bound-tightening procedure. We show that their minimizers converge to minimizers to a limit problem with a pointwise formulation of the McCormick envelopes when driving the mesh size to zero. We provide a computational example, for which we certify all of our imposed assumptions. The results point to both the potential of the methodology and the gaps in the research that need to be closed.

math.OC↗

arXiv2024

Binary Quantum Control Optimization with Uncertain Hamiltonians

Optimizing the controls of quantum systems plays a crucial role in advancing quantum technologies. The time-varying noises in quantum systems and the widespread use of inhomogeneous quantum ensembles raise the need for high-quality quantum controls under uncertainties. In this paper, we consider a stochastic discrete optimization formulation of a binary optimal quantum control problem involving Hamiltonians with predictable uncertainties. We propose a sample-based reformulation that optimizes both risk-neutral and risk-averse measurements of control policies, and solve these with two gradient-based algorithms using sum-up-rounding approaches. Furthermore, we discuss the differentiability of the objective function and prove upper bounds of the gaps between the optimal solutions to binary control problems and their continuous relaxations. We conduct numerical studies on various sized problem instances based of two applications of quantum pulse optimization; we evaluate different strategies to mitigate the impact of uncertainties in quantum systems. We demonstrate that the controls of our stochastic optimization model achieve significantly higher quality and robustness compared to the controls of a deterministic model.

quant-ph↗

arXiv2023

Heuristic Algorithms for Placing Geomagnetically Induced Current Blocking Devices

We propose a new heuristic approach for solving the challenge of determining optimal placements for geomagnetically induced current blocking devices on electrical grids. Traditionally, these determinations are approached by formulating the problem as mixed-integer nonlinear programming models and solving them using optimization solvers based on the spatial branch-and-bound algorithm.However, computing an optimal solution using the solvers often demands substantial computational time due to their inability to leverage the inherent problem structure. Therefore, in this work we propose a new heuristic approach based on a three-block alternating direction method of multipliers algorithm, and we compare it with an existing stochastic learning algorithm. Both heuristics exploit the structure of the problem of interest. We test these heuristic approaches through extensive numerical experiments conducted on the EPRI-21 and UIUC-150 test systems. The outcomes showcase the superior performance of our methodologies in terms of both solution quality and computational speed when compared with conventional solvers.

math.OC↗

arXiv2023

Switching Time Optimization for Binary Quantum Optimal Control

Quantum optimal control is a technique for controlling the evolution of a quantum system and has been applied to a wide range of problems in quantum physics. We study a binary quantum control optimization problem, where control decisions are binary-valued and the problem is solved in diverse quantum algorithms. In this paper, we utilize classical optimization and computing techniques to develop an algorithmic framework that sequentially optimizes the number of control switches and the duration of each control interval on a continuous time horizon. Specifically, we first solve the continuous relaxation of the binary control problem based on time discretization and then use a heuristic to obtain a controller sequence with a penalty on the number of switches. Then, we formulate a switching time optimization model and apply sequential least-squares programming with accelerated time-evolution simulation to solve the model. We demonstrate that our computational framework can obtain binary controls with high-quality performance and also reduce computational time via solving a family of quantum control instances in various quantum physics applications.

quant-ph↗

arXiv2023

On Convergence of Binary Trust-Region Steepest Descent

Binary trust-region steepest descent (BTR) and combinatorial integral approximation (CIA) are two recently investigated approaches for the solution of optimization problems with distributed binary-/discrete-valued variables (control functions). We show improved convergence results for BTR by imposing a compactness assumption that is similar to the convergence theory of CIA. As a corollary we conclude that BTR also constitutes a descent algorithm on the continuous relaxation and its iterates converge weakly-$^*$ to stationary points of the latter. We provide computational results that validate our findings. In addition, we observe a regularizing effect of BTR, which we explore by means of a hybridization of CIA and BTR.

math.OC↗

arXiv2023

Robust A-Optimal Experimental Design for Bayesian Inverse Problems

Optimal design of experiments for Bayesian inverse problems has recently gained wide popularity and attracted much attention, especially in the computational science and Bayesian inversion communities. An optimal design maximizes a predefined utility function that is formulated in terms of the elements of an inverse problem, an example being optimal sensor placement for parameter identification. The state-of-the-art algorithmic approaches following this simple formulation generally overlook misspecification of the elements of the inverse problem, such as the prior or the measurement uncertainties. This work presents an efficient algorithmic approach for designing optimal experimental design schemes for Bayesian inverse problems such that the optimal design is robust to misspecification of elements of the inverse problem. Specifically, we consider a worst-case scenario approach for the uncertain or misspecified parameters, formulate robust objectives, and propose an algorithmic approach for optimizing such objectives. Both relaxation and stochastic solution approaches are discussed with detailed analysis and insight into the interpretation of the problem and the proposed algorithmic approach. Extensive numerical experiments to validate and analyze the proposed approach are carried out for sensor placement in a parameter identification problem.

math.OC↗

arXiv2023

Polytopic Superset Algorithm for Nonlinear Robust Optimization

Nonlinear robust optimization (NRO) is widely used in different applications, including energy, control, and economics, to make robust decisions under uncertainty. One of the classical solution methods in NRO is an outer approximation method that iteratively solves a sample-based nonlinear problem and updates the sample set by solving an auxiliary problem subject to the uncertainty set. Although it guarantees convergence under certain assumptions, its solution iterates are generally infeasible in the original NRO problem, and it provides only a lower bound on the optimal objective value. We propose a new algorithm for a class of NRO problems that generates feasible solution iterates and provides both lower and upper bounds to the optimal objective value. In each iteration, the algorithm solves the reformulation of an NRO subproblem with respect to the polytopic supersets of the original uncertainty set and uses a cutting plane method to improve the supersets over iteration. If the NRO subproblem is infeasible, we provide a feasibility restoration step to detect whether the original NRO problem is infeasible or construct a new superset to restore the feasibility of the NRO subproblem. Further, we prove that our superset algorithm converges to the optimal solution of the original NRO problem. In numerical studies, we use application instances from portfolio optimization and production cost minimization and compare the performance between the superset algorithm and an outer approximation method called Polak's algorithm. Our result shows that the superset algorithm is more advantageous than Polak's algorithm when the number of robust constraints is large.

math.OC↗

arXiv2022

Binary Control Pulse Optimization for Quantum Systems

Quantum control aims to manipulate quantum systems toward specific quantum states or desired operations. Designing highly accurate and effective control steps is vitally important to various quantum applications, including energy minimization and circuit compilation. In this paper we focus on discrete binary quantum control problems and apply different optimization algorithms and techniques to improve computational efficiency and solution quality. Specifically, we develop a generic model and extend it in several ways. We introduce a squared $L_2$-penalty function to handle additional side constraints, to model requirements such as allowing at most one control to be active. We introduce a total variation (TV) regularizer to reduce the number of switches in the control. We modify the popular gradient ascent pulse engineering (GRAPE) algorithm, develop a new alternating direction method of multipliers (ADMM) algorithm to solve the continuous relaxation of the penalized model, and then apply rounding techniques to obtain binary control solutions. We propose a modified trust-region method to further improve the solutions. Our algorithms can obtain high-quality control results, as demonstrated by numerical studies on diverse quantum control examples.

quant-ph↗