arXiv subjects

Andrea Ghezzi

Publications and source records attributed to Andrea Ghezzi.

7 recordsLinked to original sources

arXiv2025

A Comparative Study of MINLP and MPVC Formulations for Solving Complex Nonlinear Decision-Making Problems in Aerospace Applications

High-level decision-making for dynamical systems often involves performance and safety specifications that are activated or deactivated depending on conditions related to the system state and commands. Such decision-making problems can be naturally formulated as optimization problems where these conditional activations are regulated by discrete variables. However, solving these problems can be challenging numerically, even on powerful computing platforms, especially when the dynamics are nonlinear. In this work, we consider decision-making for nonlinear systems where certain constraints, as well as possible terms in the cost function, are activated or deactivated depending on the system state and commands. We show that these problems can be formulated either as mixed-integer nonlinear programs (MINLPs) or as mathematical programs with vanishing constraints (MPVCs), where the former formulation involves discrete decision variables, whereas the latter relies on continuous variables subject to structured nonconvex constraints. We discuss the different solution methods available for both formulations and demonstrate them on optimal trajectory planning problems in various aerospace applications. Finally, we compare the strengths and weaknesses of the MINLP and MPVC approaches through a focused case study on powered descent guidance with divert-feasible regions.

math.OC↗

arXiv2025

Rollout Then Optimize: A One-Step Newton Refinement of Learned Policies for Nonlinear Model Predictive Control

We propose a computationally efficient rollout-then-optimize method to improve a learned control policy at deployment time. A learned policy provides a nominal trajectory, which is refined online by a single Newton step implemented via a Riccati recursion within a model predictive control (MPC) scheme. This refinement combines model knowledge with the learned policy at minimal additional computational cost. We establish bounds on the approximation error of the learned policy relative to the MPC policy and show that one Newton step reduces the suboptimality of the learned rollout quadratically in the policy approximation error. The proposed controller is validated in simulation on a constrained trajectory-tracking task for a quadcopter with nonlinear dynamics. Results highlight that the Newton step significantly improves the learned policy, achieving performance close to a fully converged MPC solution while requiring roughly half of the computational time. The code is available at https://github.com/aghezz1/rl-riccati.

math.OC↗

arXiv2024

AC4MPC: Actor-Critic Reinforcement Learning for Nonlinear Model Predictive Control

\Ac{MPC} and \ac{RL} are two powerful control strategies with, arguably, complementary advantages. In this work, we show how actor-critic \ac{RL} techniques can be leveraged to improve the performance of \ac{MPC}. The \ac{RL} critic is used as an approximation of the optimal value function, and an actor roll-out provides an initial guess for primal variables of the \ac{MPC}. A parallel control architecture is proposed where each \ac{MPC} instance is solved twice for different initial guesses. Besides the actor roll-out initialization, a shifted initialization from the previous solution is used. Thereafter, the actor and the critic are again used to approximately evaluate the infinite horizon cost of these trajectories. The control actions from the lowest-cost trajectory are applied to the system at each time step. We establish that the proposed algorithm is guaranteed to outperform the original \ac{RL} policy plus an error term that depends on the accuracy of the critic and decays with the horizon length of the \ac{MPC} formulation. Moreover, we do not require globally optimal solutions for these guarantees to hold. The approach is demonstrated on an illustrative toy example and an \ac{AD} overtaking scenario.

eess.SY↗

arXiv2024

A Sequential Benders-based Mixed-Integer Quadratic Programming Algorithm and Its Implementation in the CAMINO Toolbox

Sequential quadratic programming and sequential convex programming efficiently solve nonlinear programs (NLPs) by linearizing inner nonlinearities while preserving the outer convex structure. This paper introduces a sequential mixed-integer quadratic programming (MIQP) algorithm to extend this methodology to mixed-integer nonlinear problems (MINLPs), leveraging the efficiency of modern MIQP solvers. The algorithm uses a three-step iterative process. First, the MINLP is linearized around the current iterate. Second, an MIQP is formulated and solved, with its feasible region restricted to a specific area around the linearization point. This region is defined using objective values and derivatives from previous iterations, drawing on concepts from generalized Benders' decomposition. Third, the integer variables from the MIQP solution are fixed, and an NLP involving only the continuous variables is solved. The best solution among all iterates becomes the linearization point for the next iteration. A fallback strategy based on a mixed-integer linear program (MILP) is used when MIQP progress stalls. This guarantees convergence to the global optimal solution for convex MINLPs. For nonconvex problems, the algorithm functions as a heuristic without global optimality guarantees. Numerical experiments show its competitiveness with other MINLP solvers on benchmark problems. In addition, the algorithm was successfully applied to mixed-integer optimal control problems, demonstrating its effectiveness in handling challenging nonlinear equality constraints. The proposed algorithm is publicly available at https://github.com/minlp-toolbox/CAMINO with the name s-b-miqp.

math.OC↗

arXiv2023

Imitation Learning from Nonlinear MPC via the Exact Q-Loss and its Gauss-Newton Approximation

This work presents a novel loss function for learning nonlinear Model Predictive Control policies via Imitation Learning. Standard approaches to Imitation Learning neglect information about the expert and generally adopt a loss function based on the distance between expert and learned controls. In this work, we present a loss based on the Q-function directly embedding the performance objectives and constraint satisfaction of the associated Optimal Control Problem (OCP). However, training a Neural Network with the Q-loss requires solving the associated OCP for each new sample. To alleviate the computational burden, we derive a second Q-loss based on the Gauss-Newton approximation of the OCP resulting in a faster training time. We validate our losses against Behavioral Cloning, the standard approach to Imitation Learning, on the control of a nonlinear system with constraints. The final results show that the Q-function-based losses significantly reduce the amount of constraint violations while achieving comparable or better closed-loop costs.

cs.LG↗

arXiv2023

An Efficient Method for the Joint Estimation of System Parameters and Noise Covariances for Linear Time-Variant Systems

We present an optimization-based method for the joint estimation of system parameters and noise covariances of linear time-variant systems. Given measured data, this method maximizes the likelihood of the parameters. We solve the optimization problem of interest via a novel structure-exploiting solver. We present the advantages of the proposed approach over commonly used methods in the framework of Moving Horizon Estimation. Finally, we show the performance of the method through numerical simulations on a realistic example of a thermal system. In this example, the method can successfully estimate the model parameters in a short computational time.

math.OC↗

arXiv2022

An Implicit and Explicit Dual Model Predictive Control Formulation for a Steel Recycling Process

We present a formulation for both implicit and explicit dual model predictive control for a steel recycling process. The process consists in the production of new steel by choosing a combination of several different steel scraps with unknown pollutant content. The pollutant content can only be measured after a scrap combination is molten, allowing for inference on the pollutants in the different scrap heaps. The production cost should be minimized while ensuring high quality of the product through constraining the maximum amount of pollutant. The dual control formulation allows to achieve the optimal explore-exploit trade-off between uncertainty reduction and cost minimization for the examined problem. Specifically, the dual effect is obtained by considering the dependence of the future pollutant uncertainties on the scrap selection in the predictions. The implicit formulation promotes uncertainty reduction indirectly via the impact of active constraints on the objective, while the explicit formulation adds a heuristic cost on uncertainty to encourage active exploration. We compare the formulations by numerical simulations of a simplified but representative industrial steel recycling process. The results demonstrate the superiority of the two dual formulations with respect to a robustified but non-dual formulation.

math.OC↗