arXiv subjects

François Pacaud

Publications and source records attributed to François Pacaud.

At least 19 recordsLinked to original sources

arXiv2026

ExaModels.jl: an Algebraic Modeling System for Nonlinear Programming on GPUs

Large-scale nonlinear programs almost always exhibit partially separable and repetitive structure, yet most existing algebraic modeling systems do not take advantage of it. A nonlinear optimization solver queries the objective, the constraints, and their derivatives at every iteration, so the speed of these evaluations bears directly on the overall solution time. We present ExaModels.jl, a Julia-based algebraic modeling system that exploits this structure to evaluate the objective, the constraints, and their derivatives in parallel. At its core is a single-instruction, multiple-data abstraction that represents a nonlinear program as a small number of algebraic patterns, each repeated over many data points. Because the patterns are visible at compile time, a specialized model and derivative evaluation kernel is compiled for each pattern. Applying that kernel independently across the data points maps naturally onto GPU parallelism and, with sufficiently many threads, yields O(1) evaluation time regardless of the number of data points. On the largest instances of the Luksan-Vlcek library, GPU execution speeds up sparse Hessian evaluation by 76x over single-threaded CPU evaluation, and by 30x on COPS and 7.3x on PGLIB-OPF.

math.OC↗

arXiv2026

Condensed Interior-Point Methods for Scalable Nonlinear Programming on GPUs

This paper explores two variants of condensed-space interior-point methods designed for GPUs - HyKKT and LiftedKKT - by analyzing their numerical properties through error analysis and assessing their real-world performance via extensive numerical experiments with a fully GPU-resident software implementation. Traditional implementations of interior-point methods (IPMs) involve solving indefinite augmented KKT systems repeatedly by utilizing direct sparse solvers based on the LBL factorization with sophisticated numerical pivoting strategies. While this method achieves high performance and robustness on CPUs, the serial nature of numerical pivoting presents challenges for effective implementation on GPUs. Recently, multiple condensed-space IPM strategies have emerged to address this issue by transforming the KKT system into a symmetric positive-definite matrix, which is more suitable for factorization on GPUs. In this study, we demonstrate that although the condensed systems show increased ill-conditioning, the inherent structures of the condensed KKT system effectively counterbalance potential accuracy loss in the IPM. Furthermore, we provide numerical results that thoroughly assess the capabilities of a fully GPU-resident nonlinear programming software stack, comprising MadNLP (a filter line-search IPM solver), cuDSS (a direct sparse solver leveraging Cholesky factorization), and ExaModels (a modeling framework), by benchmarking their performance against the pglib-opf and CUTEst libraries. Our findings suggest that the GPU framework holds promise for solving highly sparse large-scale nonlinear programs, such as optimal power flow instances, except for diminished robustness and limited speedups for edge cases observed within CUTEst instances.

math.OC↗

arXiv2026

CCOpt: an Open-Source Solver for Large-Scale Mathematical Programs with Complementarity Constraints

This paper presents the Julia package CCOpt, built on top of the interior-point solver MadNLP. CCOpt implements a suite of algorithms for Mathematical Programs with Complementarity Constraints (MPCCs). The solver additionally comes with interfaces for use in Matlab, Python, and C++. MPCCs have recently gained renewed attention in engineering optimization, as complementarity provides a powerful modeling tool for nonsmooth functions and logical conditions. These problems are inherently challenging since their nonlinear programming reformulations violate classical regularity conditions at all feasible points, complicating both theoretical analysis and numerical treatment. Consequently, specialized algorithms are required to handle this degeneracy, and several approaches have been proposed. We implement a toolbox of methods, including relaxation and penalty approaches, as well as a crossover to recently proposed active-set methods. Our solver is based on nonlinear interior-point algorithms that couple the relaxation or penalty parameter with the barrier parameter, yielding substantial speedups compared to standard implementations. Both monotone and nonmonotone strategies for updating this joint parameter update are proposed and investigated. In addition, we propose regularization techniques that improve the conditioning of the KKT system for small relaxation parameters, enhancing robustness and computational efficiency. The implementation is validated on the classical MacMPEC benchmark, large-scale problems in security-constrained optimal power flow, optimal control of nonsmooth systems, as well as on quadratic programs with complementarity constraints arising in model predictive control. This benchmarking reveals an algorithmically driven improvement of often an entire order of magnitude over other methods, including commercial solvers.

math.OC↗

arXiv2026

ExaModelsPower.jl: A GPU-Compatible Modeling Library for Nonlinear Power System Optimization

As GPU-accelerated mathematical programming techniques mature, there is growing interest in utilizing them to address the computational challenges of power system optimization. This paper introduces ExaModelsPower.jl, an open-source modeling library for creating GPU-compatible nonlinear AC optimal power flow models. Built on ExaModels.jl, ExaModelsPower.jl provides a high-level interface that automatically generates all necessary callback functions for GPU solvers. The library is designed for large-scale problem instances, which may include multiple time periods and security constraints. Using ExaModelsPower.jl, we benchmark GPU and CPU solvers on open-source test cases. Our results show that GPU solvers can deliver up to two orders of magnitude speedups compared to alternative tools on CPU for problems with more than 20,000 variables and a solution precision of up to $10^{-4}$, while performance for smaller instances or tighter tolerances may vary.

eess.SY↗

arXiv2025

Optimal Operation and Valuation of Electricity Storages in Intraday Markets

This paper applies computational techniques of convex stochastic optimization to optimal operation and valuation of electricity storages in the face of uncertain electricity prices. Our valuations are based on the indifference pricing principle, which builds on optimal trading strategies and calibrates to the user's financial position, market views and risk preferences. The underlying optimization problem is solved with the Stochastic Dual Dynamic Programming algorithm which is applicable to various specifications of storages, and it allows for e.g. hard constraints on storage capacity and charging speed. We illustrate the approach in intraday trading where the agent charges or discharges a battery over a finite number of delivery periods, and the electricity prices are subject to bid-ask spreads and significant uncertainty. Optimal strategies are found in a matter of minutes on a regular PC. We find that the corresponding trading strategies and battery valuations vary consistently with respect to the agent's risk preferences as well as the physical characteristics of the battery.

math.OC↗

arXiv2025

GPU Implementation of Second-Order Linear and Nonlinear Programming Solvers

In recent years, GPU-accelerated optimization solvers based on second-order methods (e.g., interior-point methods) have gained momentum with the advent of mature and efficient GPU-accelerated direct sparse linear solvers, such as cuDSS. This paper provides an overview of the state of the art in GPU-based second-order solvers, focusing on pivoting-free interior-point methods for large and sparse linear and nonlinear programs. We begin by highlighting the capabilities and limitations of the currently available GPU-accelerated sparse linear solvers. Next, we discuss different formulations of the Karush-Kuhn-Tucker systems for second-order methods and evaluate their suitability for pivoting-free GPU implementations. We also discuss strategies for computing sparse Jacobians and Hessians on GPUs for nonlinear programming. Finally, we present numerical experiments demonstrating the scalability of GPU-based optimization solvers. We observe speedups often exceeding 10x compared to comparable CPU implementations on large-scale instances when solved up to medium precision. Additionally, we examine the current limitations of existing approaches.

math.OC↗

arXiv2025

A General and Streamlined Differentiable Optimization Framework

Differentiating through constrained optimization problems is increasingly central to learning, control, and large-scale decision-making systems, yet practical integration remains challenging due to solver specialization and interface mismatches. This paper presents a general and streamlined framework-an updated DiffOpt.jl-that unifies modeling and differentiation within the Julia optimization stack. The framework computes forward - and reverse-mode solution and objective sensitivities for smooth, potentially nonconvex programs by differentiating the KKT system under standard regularity assumptions. A first-class, JuMP-native parameter-centric API allows users to declare named parameters and obtain derivatives directly with respect to them - even when a parameter appears in multiple constraints and objectives - eliminating brittle bookkeeping from coefficient-level interfaces. We illustrate these capabilities on convex and nonconvex models, including economic dispatch, mean-variance portfolio selection with conic risk constraints, and nonlinear robot inverse kinematics. Two companion studies further demonstrate impact at scale: gradient-based iterative methods for strategic bidding in energy markets and Sobolev-style training of end-to-end optimization proxies using solver-accurate sensitivities. Together, these results demonstrate that differentiable optimization can be deployed as a routine tool for experimentation, learning, calibration, and design-without deviating from standard JuMP modeling practices and while retaining access to a broad ecosystem of solvers.

cs.LG↗

arXiv2025

An Augmented Lagrangian Method on GPU for Security-Constrained AC Optimal Power Flow

We present a new algorithm for solving large-scale security-constrained optimal power flow in polar form (AC-SCOPF). The method builds on Nonlinearly Constrained augmented Lagrangian (NCL), an augmented Lagrangian method in which the subproblems are solved using an interior-point method. NCL has two key advantages for large-scale SC-OPF. First, NCL handles difficult problems such as infeasible ones or models with complementarity constraints. Second, the augmented Lagrangian term naturally regularizes the Newton linear systems within the interior-point method, enabling to solve the Newton systems with a pivoting-free factorization that can be efficiently parallelized on GPUs. We assess the performance of our implementation, called MadNCL, on large-scale corrective AC-SCOPFs, with complementarity constraints modeling the corrective actions. Numerical results show that MadNCL can solve AC-SCOPF with 500 buses and 256 contingencies fully on the GPU in less than 3 minutes, whereas Knitro takes more than 3 hours to find an equivalent solution.

math.OC↗

arXiv2025

MadNCL: A GPU Implementation of Algorithm NCL for Large-Scale, Degenerate Nonlinear Programs

We present a GPU implementation of Algorithm NCL, an augmented Lagrangian method for solving large-scale and degenerate nonlinear programs. Although interior-point methods and sequential quadratic programming are widely used for solving nonlinear programs, the augmented Lagrangian method is known to offer superior robustness against constraint degeneracies and can rapidly detect infeasibility. We introduce several enhancements to Algorithm NCL, including fusion of the inner and outer loops and use of extrapolation steps, which improve both efficiency and convergence stability. Further, NCL has the key advantage of being well-suited for GPU architectures because of the regularity of the KKT systems provided by quadratic penalty terms. In particular, the NCL subproblem formulation allows the KKT systems to be naturally expressed as either stabilized or condensed KKT systems, whereas the interior-point approach requires aggressive reformulations or relaxations to make it suitable for GPUs. Both systems can be efficiently solved on GPUs using sparse \ldlt factorization with static pivoting, as implemented in NVIDIA cuDSS. Building on these advantages, we examine the KKT systems arising from NCL subproblems. We present an optimized GPU implementation of Algorithm NCL by leveraging MadNLP as an interior-point subproblem solver and utilizing the stabilized and condensed formulations of the KKT systems for computing Newton steps. Numerical experiments on various large-scale and degenerate NLPs, including optimal power flow, COPS benchmarks, and security-constrained optimal power flow, demonstrate that MadNCL operates efficiently on GPUs while effectively managing problem degeneracy, including MPCC constraints.

math.OC↗

arXiv2025

Sensitivity analysis for parametric nonlinear programming: A tutorial

This tutorial provides an overview of the current state-of-the-art in the sensitivity analysis for nonlinear programming. Building upon the fundamental work of Fiacco, it derives the sensitivity of primal-dual solutions for regular nonlinear programs and explores the extent to which Fiacco's framework can be extended to degenerate nonlinear programs with non-unique dual solutions. The survey ends with a discussion on how to adapt the sensitivity analysis for conic programs and approximate solutions obtained from interior-point algorithms.

math.OC↗

arXiv2025

Optimal Operation and Valuation of Electricity Storages

This paper applies computational techniques of convex stochastic optimization to optimal operation and valuation of electricity storages in the face of uncertain electricity prices. Our approach is applicable to various specifications of storages, and it allows for e.g.\ hard constraints on storage capacity and charging speed. Our valuations are based on the indifference pricing principle, which builds on optimal trading strategies and calibrates to the user's initial position, market views and risk preferences. We illustrate the effects of storage capacity and charging speed by numerically computing the valuations using stochastic dual dynamic programming.

math.OC↗

arXiv2025

Recovering sparse DFT from missing signals via interior point method on GPU

We propose a method to recover the sparse discrete Fourier transform (DFT) of a signal that is both noisy and potentially incomplete, with missing values. The problem is formulated as a penalized least-squares minimization based on the inverse discrete Fourier transform (IDFT) with an $\ell_1$-penalty term, reformulated to be solvable using a primal-dual interior point method (IPM). Although Krylov methods are not typically used to solve Karush-Kuhn-Tucker (KKT) systems arising in IPMs due to their ill-conditioning, we employ a tailored preconditioner and establish new asymptotic bounds on the condition number of preconditioned KKT matrices. Thanks to this dedicated preconditioner -- and the fact that FFT and IFFT operate as linear operators without requiring explicit matrix materialization -- KKT systems can be solved efficiently at large scales in a matrix-free manner. Numerical results from a Julia implementation leveraging GPU-accelerated interior point methods, Krylov methods, and FFT toolkits demonstrate the scalability of our approach on problems with hundreds of millions of variables, inclusive of real data obtained from the diffuse scattering from a slightly disordered Molybdenum Vanadium Dioxide crystal.

math.OC↗

arXiv2024

Scalable Primal Decomposition Schemes for Large-Scale Infrastructure Networks

The operation of large-scale infrastructure networks requires scalable optimization schemes. To guarantee safe system operation, a high degree of feasibility in a small number of iterations is important. Decomposition schemes can help to achieve scalability. In terms of feasibility, however, classical approaches such as the alternating direction method of multipliers (ADMM) often converge slowly. In this work, we present primal decomposition schemes for hierarchically structured strongly convex QPs. These schemes offer high degrees of feasibility in a small number of iterations in combination with global convergence guarantees. We benchmark their performance against the centralized off-the-shelf interior-point solver Ipopt and ADMM on problems with up to 300,000 decision variables and constraints. We find that the proposed approaches solve problems as fast as Ipopt, but with reduced communication and without requiring a full model exchange. Moreover, the proposed schemes achieve a higher accuracy than ADMM.

eess.SY↗

arXiv2024

GPU-accelerated dynamic nonlinear optimization with ExaModels and MadNLP

We investigate the potential of Graphics Processing Units (GPUs) to solve large-scale nonlinear programs with a dynamic structure. Using ExaModels, a GPU-accelerated automatic differentiation tool, and the interior-point solver MadNLP, we significantly reduce the time to solve dynamic nonlinear optimization problems. The sparse linear systems formulated in the interior-point method is solved on the GPU using a hybrid solver combining an iterative method with a sparse Cholesky factorization, which harness the newly released NVIDIA cuDSS solver. Our results on the classical distillation column instance show that despite a significant pre-processing time, the hybrid solver allows to reduce the time per iteration by a factor of 25 for the largest instance.

math.OC↗

arXiv2024

Accelerating Optimal Power Flow with GPUs: SIMD Abstraction of Nonlinear Programs and Condensed-Space Interior-Point Methods

This paper introduces a framework for solving alternating current optimal power flow (ACOPF) problems using graphics processing units (GPUs). While GPUs have demonstrated remarkable performance in various computing domains, their application in ACOPF has been limited due to challenges associated with porting sparse automatic differentiation (AD) and sparse linear solver routines to GPUs. We address these issues with two key strategies. First, we utilize a single-instruction, multiple-data abstraction of nonlinear programs. This approach enables the specification of model equations while preserving their parallelizable structure and, in turn, facilitates the parallel AD implementation. Second, we employ a condensed-space interior-point method (IPM) with an inequality relaxation. This technique involves condensing the Karush--Kuhn--Tucker (KKT) system into a positive definite system. This strategy offers the key advantage of being able to factorize the KKT matrix without numerical pivoting, which has hampered the parallelization of the IPM algorithm. By combining these strategies, we can perform the majority of operations on GPUs while keeping the data residing in the device memory only. Comprehensive numerical benchmark results showcase the advantage of our approach. Remarkably, our implementations -- MadNLP.jl and ExaModels.jl -- running on NVIDIA GPUs achieve an order of magnitude speedup compared with state-of-the-art tools running on contemporary CPUs.

math.OC↗

arXiv2023

Parallel Interior-Point Solver for Block-Structured Nonlinear Programs on SIMD/GPU Architectures

We investigate how to port the standard interior-point method to new exascale architectures for block-structured nonlinear programs with state equations. Computationally, we decompose the interior-point algorithm into two successive operations: the evaluation of the derivatives and the solution of the associated Karush-Kuhn-Tucker (KKT) linear system. Our method accelerates both operations using two levels of parallelism. First, we distribute the computations on multiple processes using coarse parallelism. Second, each process uses a SIMD/GPU accelerator locally to accelerate the operations using fine-grained parallelism. The KKT system is reduced by eliminating the inequalities and the state variables from the corresponding equations, to a dense matrix encoding the sensitivities of the problem's degrees of freedom, drastically minimizing the memory exchange. We demonstrate the method's capability on the supercomputer Polaris, a testbed for the future exascale Aurora system. Each node is equipped with four GPUs, a setup amenable to our two-level approach. Our experiments on the stochastic optimal power flow problem show that the method can achieve a 50x speed-up compared to the state-of-the-art method.

math.OC↗

arXiv2022

Exploiting GPU/SIMD Architectures for Solving Linear-Quadratic MPC Problems

We report numerical results on solving constrained linear-quadratic model predictive control (MPC) problems by exploiting graphics processing units (GPUs). The presented method reduces the MPC problem by eliminating the state variables and applies a condensed-space interior-point method to remove the inequality constraints in the KKT system. The final condensed matrix is positive definite and can be efficiently factorized in parallel on GPU/SIMD architectures. In addition, the size of the condensed matrix depends only on the number of controls in the problem, rendering the method particularly effective when the problem has many states but few inputs and moderate horizon length. Our numerical results for PDE-constrained problems show that the approach is an order of magnitude faster than a standard CPU implementation. We also provide an open-source Julia framework that facilitates modeling (DynamicNLPModels.jl) and solution (MadNLP.jl) of MPC problems on GPUs.

math.OC↗

arXiv2022

Constrained Policy Optimization for Stochastic Optimal Control under Nonstationary Uncertainties

This article presents a constrained policy optimization approach for the optimal control of systems under nonstationary uncertainties. We introduce an assumption that we call Markov embeddability that allows us to cast the stochastic optimal control problem as a policy optimization problem over the augmented state space. Then, the infinite-dimensional policy optimization problem is approximated as a finite-dimensional nonlinear program by applying function approximation, deterministic sampling, and temporal truncation. The approximated problem is solved by using automatic differentiation and condensed-space interior-point methods. We formulate several conceptual and practical open questions regarding the asymptotic exactness of the approximation and the solution strategies for the approximated problem. As a proof of concept, we provide a numerical example demonstrating the performance of the control policy obtained by the proposed method.

math.OC↗