arXiv subjects

Jonathan Frey

Publications and source records attributed to Jonathan Frey.

At least 19 recordsLinked to original sources

arXiv2026

Anderson Acceleration for Linearly Converging SQP-Type Methods

Although Anderson acceleration (AA) is known to speed up fixed-point iterations, it is rarely applied in constrained optimization, in particular sequential quadratic programming (SQP). We show that the local convergence behavior of a general family of (inexact) SQP-type methods can benefit from AA and introduce a simple heuristic to alleviate slower convergence farther from the solution. The method is implemented in the software framework acados. Numerical examples from optimal control illustrate consistent improvements in convergence of different SQP-type methods.

math.OC↗

arXiv2025

Riccati-ZORO: An efficient algorithm for heuristic online optimization of internal feedback laws in robust and stochastic model predictive control

We present Riccati-ZORO, an algorithm for tube-based optimal control problems (OCP). Tube OCPs predict a tube of trajectories to capture predictive uncertainty. The tube induces a constraint tightening via additional backoff terms. This backoff can significantly affect the performance, and thus implicitly defines a cost of uncertainty. Optimizing the feedback law used to predict the tube can significantly reduce the backoffs, but its online computation is challenging. Riccati-ZORO jointly optimizes the nominal trajectory and uncertainty tube based on a heuristic uncertainty cost design. The algorithm alternates between two subproblems: (i) a nominal OCP with fixed backoffs, (ii) an unconstrained tube OCP, which optimizes the feedback gains for a fixed nominal trajectory. For the tube optimization, we propose a cost function informed by the proximity of the nominal trajectory to constraints, prioritizing reduction of the corresponding backoffs. These ideas are developed for ellipsoidal tubes under linear state feedback. In this case, the decomposition into the two subproblems substantially reduces the computational complexity with respect to the state dimension from $\mathcal{O}(n_x^6)$ to $\mathcal{O}(n_x^3)$, i.e., the complexity of a nominal OCP. We investigate the algorithm in numerical experiments, and provide two open-source implementations: a prototyping version in CasADi and a high-performance implementation integrated into the acados OCP solver.

math.OC↗

arXiv2025

Differentiable Nonlinear Model Predictive Control

The efficient computation of parametric solution sensitivities is a key challenge in the integration of learning-enhanced methods with nonlinear model predictive control (MPC), as their availability is crucial for many learning algorithms. This paper discusses the computation of solution sensitivities of general nonlinear programs (NLPs) using the implicit function theorem (IFT) and smoothed optimality conditions treated in interior-point methods (IPM). We detail sensitivity computation within a sequential quadratic programming (SQP) method which employs an IPM for the quadratic subproblems. Previous works presented in the machine learning community are limited to convex or unconstrained formulations, or lack an implementation for efficient sensitivity evaluation. The publication is accompanied by an efficient open-source implementation within the acados framework, providing both forward and adjoint sensitivities for general optimal control problems, achieving speedups exceeding 3x over the state-of-the-art solvers mpc.pytorch and cvxpygen.

math.OC↗

arXiv2025

MPC4RL -- A Software Package for Reinforcement Learning based on Model Predictive Control

In this paper, we present an early software integrating Reinforcement Learning (RL) with Model Predictive Control (MPC). Our aim is to make recent theoretical contributions from the literature more accessible to both the RL and MPC communities. We combine standard software tools developed by the RL community, such as Gymnasium, stable-baselines3, or CleanRL with the acados toolbox, a widely-used software package for efficient MPC algorithms. Our core contribution is MPC4RL, an open-source Python package that supports learning-enhanced MPC schemes for existing acados implementations. The package is designed to be modular, extensible, and user-friendly, facilitating the tuning of MPC algorithms for a broad range of control problems. It is available on GitHub.

eess.SY↗

arXiv2024

L4acados: Learning-based models for acados, applied to Gaussian process-based predictive control

Incorporating learning-based models, such as artificial neural networks or Gaussian processes, into model predictive control (MPC) strategies can significantly improve control performance and online adaptation capabilities for real-world applications. Still, enabling state-of-the-art implementations of learning-based models for MPC is complicated by the challenge of interfacing machine learning frameworks with real-time optimal control software. This work aims at filling this gap by incorporating external sensitivities in sequential quadratic programming solvers for nonlinear optimal control. To this end, we provide L4acados, a general framework for incorporating Python-based dynamics models in the real-time optimal control software acados. By computing external sensitivities via a user-defined Python module, L4acados enables the implementation of MPC controllers with learning-based residual models in acados, while supporting parallelization of sensitivity computations when preparing the quadratic subproblems. We demonstrate significant speed-ups and superior scaling properties of L4acados compared to available software using a neural-network-based control example. Last, we provide an efficient and modular real-time implementation of Gaussian process-based MPC using L4acados, which is applied to two hardware examples: autonomous miniature racing, as well as motion control of a full-scale autonomous vehicle for an ISO lane change maneuver.

eess.SY↗

arXiv2024

Multi-Phase Optimal Control Problems for Efficient Nonlinear Model Predictive Control with acados

Computationally efficient nonlinear model predictive control relies on elaborate discrete-time optimal control problem (OCP) formulations trading off accuracy with respect to the continuous-time problem and associated computational burden. Such formulations, however, are in general not easy to implement within specialized software frameworks tailored to numerical optimal control. This paper introduces a new multi-phase OCP interface for the open-source software acados allowing to conveniently formulate such problems and generate fast solvers that can be used for nonlinear model predictive control (NMPC). While multi-phase OCP (MOCP) formulations occur naturally in many applications, this work focuses on MOCP formulations that can be used to efficiently approximate standard continuous-time OCPs in the context of NMPC. To this end, the paper discusses advanced control parametrizations, such as closed-loop costing and piecewise polynomials with varying degree, as well as partial tightening and formulations that leverage models of different fidelity. An introductory example is presented to showcase the usability of the new interface. Finally, three numerical experiments demonstrate that NMPC controllers based on multi-phase formulations can efficiently trade-off computation time and control performance.

math.OC↗

arXiv2024

Fast Generation of Feasible Trajectories in Direct Optimal Control

This paper examines the question of finding feasible points to discrete-time optimal control problems. The optimization problem of finding a feasible trajectory is transcribed to an unconstrained optimal control problem. An efficient algorithm, called FP-DDP, is proposed that solves the resulting problem using Differential Dynamic Programming preserving feasibility with respect to the system dynamics in every iteration. Notably, FP-DDP admits global and rapid local convergence properties induced by a combination of a Levenberg-Marquardt method and an Armijo-type line search. The efficiency of FP-DDP is demonstrated against established methods such as Direct Multiple Shooting, Direct Single Shooting, and state-of-the-art solvers.

math.OC↗

arXiv2024

Advanced-Step Real-time Iterations with Four Levels -- New Error Bounds and Fast Implementation in acados

The Real-Time Iteration (RTI) is an online nonlinear model predictive control algorithm that performs a single Sequential Quadratic Programming (SQP) per sampling time. The algorithm is split into a preparation and a feedback phase, where the latter one performs as little computations as possible solving a single prepared quadratic program. To further improve the accuracy of this method, the Advanced-Step RTI (AS-RTI) performs additional Multi-Level Iterations (MLI) in the preparation phase, such as inexact or zero-order SQP iterations on a problem with a predicted state estimate. This paper extends and streamlines the existing local convergence analysis of AS-RTI, such as analyzing MLI of level A and B for the first time, and significantly simplifying the proofs for levels C and D. Moreover, this paper provides an efficient open-source implementation in acados, making it widely accessible to practitioners.

math.OC↗

arXiv2023

Efficient Zero-Order Robust Optimization for Real-Time Model Predictive Control with acados

Robust and stochastic optimal control problem (OCP) formulations allow a systematic treatment of uncertainty, but are typically associated with a high computational cost. The recently proposed zero-order robust optimization (zoRO) algorithm mitigates the computational cost of uncertainty-aware MPC by propagating the uncertainties outside of the MPC problem. This paper details the combination of zoRO with the real-time iteration (RTI) scheme and presents an efficient open-source implementation in acados, utilizing BLASFEO for the linear algebra operations. In addition to the scaling advantages posed by the zoRO algorithm, the efficient implementation drastically reduces the computational overhead, and, combined with an RTI scheme, enables the use of tube-based MPC for a wider range of applications. The flexibility, usability and effectiveness of the proposed implementation is demonstrated on two examples. On the practical example of a differential drive robot, the proposed implementation results in a tenfold reduction of computation time with respect to the previously available zoRO implementation.

math.OC↗

arXiv2023

Gauss-Newton Runge-Kutta Integration for Efficient Discretization of Optimal Control Problems with Long Horizons and Least-Squares Costs

This work proposes an efficient treatment of continuous-time optimal control problem (OCP) with long horizons and nonlinear least-squares costs. The Gauss-Newton Runge-Kutta (GNRK) integrator is presented which provides a high-order cost integration. Crucially, the Hessian of the cost terms required within an SQP-type algorithm is approximated with a Gauss-Newton Hessian. Moreover, L2 penalty formulations for constraints are shown to be particularly effective for optimization with GNRK. An efficient implementation of GNRK is provided in the open-source software framework acados. We demonstrate the effectiveness of the proposed approach and its implementation on an illustrative example showing a reduction of relative suboptimality by a factor greater than 10 while increasing the runtime by only 10 %.

math.OC↗

arXiv2023

Finite Elements with Switch Detection for Numerical Optimal Control of Nonsmooth Dynamical Systems with Set-Valued Heaviside Step Functions

This paper develops high-accuracy methods for numerically solving optimal control problems subject to nonsmooth differential equations with set-valued step functions. A notable subclass of these systems are Filippov systems. The set-valued step functions are here written as the solution map of a linear program. Using the optimality conditions of this problem we rewrite the initial nonsmooth system into a equivalent dynamic complementarity systems (DCS). We extend the Finite Elements with Switch Detection (FESD) method [Nurkanovi\'c et al., 2024], initially developed for Filippov systems transformed via Stewart's reformulation into DCS [Stewart, 1990], to the class of nonsmooth systems with set-valued step functions. The key ideas are to start with a standard Runge-Kutta method for the obtained DCS and to let the integration step sizes to be degrees of freedom. Next, we introduce additional conditions to enable implicit but exact switch detection and to remove possible spurious degrees of freedom if no switches occur. The theoretical properties of the method are studied. Its favorable properties are illustrated on numerical simulation and optimal control examples. All methods introduced in this paper are implemented in the open-source software package NOSNOC.

math.OC↗

arXiv2023

Collision-free Motion Planning for Mobile Robots by Zero-order Robust Optimization-based MPC

This paper presents an implementation of robust model predictive control (MPC) for collision-free reference trajectory tracking for mobile robots. The presented approach considers the robot motion to be subject to process noise bounded by ellipsoidal sets. In order to efficiently handle the evolution of the disturbance ellipsoids within the MPC, the zero-order robust optimization (zoRO) scheme is applied. The idea is to fix the disturbance ellipsoids within one optimization iteration and solve the problem repeatedly with updated disturbance ellipsoid trajectories. The zero-order approach is suboptimal in general. However, we show that it does not impair convergence to the reference trajectory in the absence of obstacles. The experiments on an industrial mobile robot prototype demonstrate the performance of the controller.

eess.SY↗

arXiv2023

FESD-J: Finite Elements with Switch Detection for Numerical Optimal Control of Rigid Bodies with Impacts and Coulomb Friction

The Finite Elements with Switch Detection (FESD) is a high-accuracy method for the numerical simulation and solution of optimal control problems subject to discontinuous ODEs. In this article, we extend the FESD method [Nurkanovi\'c et al., 2022] to the dynamic equations of multiple rigid bodies that exhibit state jumps due to impacts and Coulomb friction. This new method is referred to as FESD with Jumps (FESD-J). Starting from the standard Runge-Kutta equations, we let the integration step sizes be degrees of freedom. Additional constraints are introduced to ensure exact switch detection and to remove spurious degrees of freedom if no switches occur. Moreover, at the boundaries of each finite element, we impose the impact equations in their complementarity form, at both the position and velocity level. They compute the normal and tangential impulses in case of contact making. Otherwise, they are reduced to the continuity conditions for the velocities. FESD-J treats multiple contacts, where each contact can have a different coefficient of restitution and friction. All methods introduced in this paper are implemented in the open-source software package NOSNOC. We illustrate the use of FESD-J in both simulation and optimal control examples.

math.OC↗

arXiv2023

Imitation Learning from Nonlinear MPC via the Exact Q-Loss and its Gauss-Newton Approximation

This work presents a novel loss function for learning nonlinear Model Predictive Control policies via Imitation Learning. Standard approaches to Imitation Learning neglect information about the expert and generally adopt a loss function based on the distance between expert and learned controls. In this work, we present a loss based on the Q-function directly embedding the performance objectives and constraint satisfaction of the associated Optimal Control Problem (OCP). However, training a Neural Network with the Q-loss requires solving the associated OCP for each new sample. To alleviate the computational burden, we derive a second Q-loss based on the Gauss-Newton approximation of the OCP resulting in a faster training time. We validate our losses against Behavioral Cloning, the standard approach to Imitation Learning, on the control of a nonlinear system with constraints. The final results show that the Q-function-based losses significantly reduce the amount of constraint violations while achieving comparable or better closed-loop costs.

cs.LG↗

arXiv2023

Finite Elements with Switch Detection for Direct Optimal Control of Nonsmooth Systems with Set-Valued Step Functions

This paper extends the Finite Elements with Switch Detection (FESD) method [Nurkanovi\'c et al., 2022] to optimal control problems with nonsmooth systems involving set-valued step functions. Logical relations and common nonsmooth functions within a dynamical system can be expressed using linear and nonlinear expressions involving step functions. A prominent subclass of these systems are Filippov systems. The set-valued step function can be expressed by the solution map of a linear program, and using its KKT conditions allows one to transform the initial system into an equivalent dynamic complementarity system (DCS). Standard Runge-Kutta (RK) methods applied to DCS have only first-order accuracy. The FESD discretization makes the step sizes degrees of freedom and adds further constraints that ensure exact switch detection to recover the high-accuracy properties that RK methods have for smooth ODEs. We use the novel FESD method for the direct transcription of optimal control problems. All methods and examples in this paper are implemented in the open-source software package NOSNOC.

math.OC↗

arXiv2022

Frenet-Cartesian Model Representations for Automotive Obstacle Avoidance within Nonlinear MPC

In recent years, nonlinear model predictive control (NMPC) has been extensively used for solving automotive motion control and planning tasks. In order to formulate the NMPC problem, different coordinate systems can be used with different advantages. We propose and compare formulations for the NMPC related optimization problem, involving a Cartesian and a Frenet coordinate frame (CCF/ FCF) in a single nonlinear program (NLP). We specify costs and collision avoidance constraints in the more advantageous coordinate frame, derive appropriate formulations and compare different obstacle constraints. With this approach, we exploit the simpler formulation of opponent vehicle constraints in the CCF, as well as road aligned costs and constraints related to the FCF. Comparisons to other approaches in a simulation framework highlight the advantages of the proposed approaches.

eess.SY↗

arXiv2022

Fast integrators with sensitivity propagation for use in CasADi

Efficient integrators with sensitivity propagation are an essential ingredient for the numerical solution of optimal control problems. This paper gives an overview on the acados integrators, their Python interface and presents a workflow that allows using them with their sensitivities within a nonlinear programming (NLP) solver interfaced by CasADi. The implementation is discussed, demonstrated and provided as open-source software. The computation times of the proposed integrator and its sensitivity computation are compared to the native CasADi collocation integrator, CVODES and IDAS on different examples. A speedup of one order of magnitude for simulation and of up to three orders of magnitude for the forward sensitivity propagation is shown for an airborne wind energy system model.

math.OC↗

arXiv2022

Active Learning of Discrete-Time Dynamics for Uncertainty-Aware Model Predictive Control

Model-based control requires an accurate model of the system dynamics for precisely and safely controlling the robot in complex and dynamic environments. Moreover, in the presence of variations in the operating conditions, the model should be continuously refined to compensate for dynamics changes. In this paper, we present a self-supervised learning approach that actively models the dynamics of nonlinear robotic systems. We combine offline learning from past experience and online learning from current robot interaction with the unknown environment. These two ingredients enable a highly sample-efficient and adaptive learning process, capable of accurately inferring model dynamics in real-time even in operating regimes that greatly differ from the training distribution. Moreover, we design an uncertainty-aware model predictive controller that is heuristically conditioned to the aleatoric (data) uncertainty of the learned dynamics. This controller actively chooses the optimal control actions that (i) optimize the control performance and (ii) improve the efficiency of online learning sample collection. We demonstrate the effectiveness of our method through a series of challenging real-world experiments using a quadrotor system. Our approach showcases high resilience and generalization capabilities by consistently adapting to unseen flight conditions, while it significantly outperforms classical and adaptive control baselines.

cs.RO↗