Searcharxiv⌕ Search

arXiv subjects

Panagiotis Patrinos

Publications and source records attributed to Panagiotis Patrinos.

At least 19 recordsLinked to original sources

FanCyqlone: A Parallelized Solver for Multi-Horizon Optimal Control

Multi-horizon optimal control aims to select optimal control sequences simultaneously for several possible future scenarios, ensuring consistency at the first stage. While this enables advanced model predictive control schemes that explicitly handle prediction uncertainty, solving these problems tends to be costly. We introduce FanCyqlone, a linear solver that exploits parallelism across both time steps and scenario branches. The method builds on Cyqlone to compute the contribution of each branch and combines these results through a shared root problem. We integrate FanCyqlone into QPALM to develop FanCyQPALM, a solver for multi-horizon optimal control problems with general linear stage-wise constraints. We compare its performance with state-of-the-art solvers on spring-mass and quadruped control benchmarks.

math.OC↗

Dual-Space Preconditioning for Variational Inequalities and Root-Finding Problems

This paper develops a dual-space preconditioning framework for variational inequalities and root-finding problems beyond classical cocoercivity and Lipschitz continuity assumptions. To this end, we introduce both a novel cocoercivity-type condition and a relaxed Lipschitz continuity condition, and derive convergence of deterministic and stochastic methods under these generalized conditions. For isotropic preconditioners, the resulting algorithms can be interpreted as variable-stepsize variants of classical methods, extending recent analyses based on $(L_0, L_1)$-type growth conditions while allowing simpler proofs and larger stepsize ranges. We also show that the framework naturally handles variational inequalities through weak Minty-type arguments. Finally, we study the associated continuous-time dynamics and derive a variational and optimal control interpretation of the preconditioned flow. Overall, the results show that dual-space preconditioning provides a flexible mechanism for designing and analyzing operator methods beyond the standard Euclidean Lipschitz setting.

math.OC↗

Regret-Based Scenario-Tree Construction for Multistage Stochastic Programs with Deterministic Constraints

Scenario trees are widely used to discretize multistage stochastic programs into forms that are tractable for numerical optimization. Yet, traditional scenario-tree construction methods are typically based on statistical similarity between the original uncertainty process and its discretization, and may fail to preserve the aspects of the uncertainty that matter most for decision-making. This paper shows that, under a fixed probability measure and deterministic feasibility constraints, the gap between the optimal value of the original multistage stochastic program and that of its scenario-tree approximation decomposes exactly into a sum of nodewise regret terms, each measuring the optimality loss at a particular node. This decomposition provides a clear target for scenario-tree construction: minimizing total regret across the tree. Motivated by this result, we propose a practical two-phase construction method for repeated stochastic optimization. In an offline phase, computationally inexpensive surrogates for decision regret are learned from data, and in the online phase, these surrogates guide scenario-tree construction. The resulting framework retains the structured optimization and constraint-handling capabilities of multistage stochastic programming while exploiting learned information about the uncertainty to make more effective use of a limited scenario-tree budget. Numerical experiments on multistage stochastic storage and production-planning problems show that, for a fixed tree size, the proposed method constructs higher-quality scenario trees and achieves better closed-loop performance than traditional scenario-tree construction methods.

math.OC↗

A Convex Feasibility Problem for Which Douglas-Rachford Converges Only Weakly

We provide two closed, convex sets with nonempty intersection and an initial point for which the Douglas-Rachford algorithm converges weakly but not strongly. This resolves an open question as noted earlier by Aragón Artacho, Borwein and Tam (2014, Remark 2.2). Our construction is a simple lift of a recent construction by Bauschke and Tung concerning firmly nonexpansive operators.

math.OC↗

A Smoothed Augmented Lagrangian Method for Mathematical Programs with Vertical Complementarity Constraints

We propose a smoothed augmented Lagrangian method (ALM) for solving mathematical programs with vertical complementarity constraints (VCCs). The augmented Lagrangian function involving the VCCs is smoothed by the Lasry--Lions double envelope, which is an extension of the Moreau envelope that enjoys enhanced smoothness properties. This construction yields a smooth unconstrained subproblem within the ALM framework that can be efficiently solved by many fast optimization algorithms. In the limiting setting, we establish global convergence to Mordukhovich- and Clarke-stationary points. For the convergence rate, we show that, under appropriate assumptions, the smoothed ALM achieves Q-linear convergence. We further provide a worst-case iteration complexity analysis for computing an approximate stationary point of the original problem or of an associated feasibility problem. Preliminary numerical experiments demonstrate the effectiveness of the proposed method.

math.OC↗

Scaled relative graphs for pairs of operators beyond classical monotonicity

We introduce a generalization of the scaled relative graph (SRG) to pairs of operators, enabling the visualization of their relative incremental properties. This novel SRG framework provides the geometric counterpart for the study of nonlinear resolvents based on paired monotonicity conditions. We demonstrate that these conditions apply to linear operators composed with monotone mappings, a class that notably includes NPN transistors, allowing us to compute the response of multivalued, nonsmooth, and highly nonmonotone electrical circuits.

math.OC↗

Active learning-based Bayesian optimization in the realm of copper slag-blended cement systems

Accelerated mix design optimization is critical for deploying low-carbon supplementary cementitious materials (SCMs) because traditional experimental approaches require extensive testing campaigns. The authors demonstrate that Bayesian optimization (BO) can identify near-optimal blended cement formulations using an AI-driven approach with minimal experimental data. Starting with only 10 initial experiments and a 2:1 data-to-variable ratio reflecting realistic laboratory constraints, the authors optimized copper slag, limestone, Portland cement systems for 32.5N strength class with respect to 2-day compressive strength, cost, and CO2 emissions. Gaussian process surrogate models guided sequential experimentation through Expected Improvement and Upper Confidence Bound acquisition functions. Within 2-6 iterations, BO identified Pareto non-dominated solutions meeting strength requirements (greater than 10 MPa at 2 days, greater than 32.5 MPa at 28 days) while achieving CO2 emissions below 500 kg CO2/ton; consistent with industry decarbonization targets. A conservative update strategy incorporating uncertainty bounds for 28-day strength enabled rapid iteration without waiting 28 days per cycle. Comparative analysis revealed that linear kernels outperformed nonlinear alternatives in predictive accuracy, though radial basis function kernels were preferred for active learning due to superior uncertainty quantification. This work demonstrates BO as a practical decision-support tool for cement research under severe data constraints.

math.OC↗

A localized consensus-based sampling algorithm

We propose a localized consensus-based method for sampling from non-Gaussian distributions, a task that frequently arises when solving Bayesian inverse problems. Our method arises from an alternative derivation of consensus-based sampling (CBS). Starting from ensemble-preconditioned Langevin dynamics, we replace the potential by its Moreau envelope -- a smoother approximation -- in order to replace the gradient in the Langevin equation with a proximal operator. We then approximate this operator by a weighted mean. In the limit of infinitely smoothing the potential to a quadratic function, this procedure recovers the standard CBS dynamics. In addition, outside this limit, we retrieve a refined variant of polarized CBS. We call the resulting algorithm localized consensus-based sampling, since particles interact more with nearby particles than with faraway ones. Our method is affine-invariant, exact for Gaussian targets in the mean-field limit, and demonstrates improved robustness over polarized CBS in numerical experiments. Like other consensus-based methods, localized CBS is gradient-free and easily parallelizable.

math.NA↗

A Lasry-Lions envelope approach for mathematical programs with complementarity constraints

We propose a homotopy method for solving mathematical programs with complementarity constraints (CCs). The indicator function of the CCs is relaxed by the Lasry--Lions double envelope, an extension of the Moreau envelope that enjoys an additional smoothness property, making it amenable to fast optimization algorithms. The proposed algorithm mimics the behavior of homotopy methods for systems of nonlinear equations or penalty methods for constrained optimization: it solves a sequence of smooth subproblems that progressively approximate the original problem, using the solution of each subproblem as the starting point for the next one. In the limiting setting, we establish the convergence to Mordukhovich and Clarke stationary points. We also provide a worst-case complexity analysis for computing an approximate stationary point. Preliminary numerical results on a suite of benchmark problems demonstrate the effectiveness of the proposed approach.

math.OC↗

Constrained Stochastic Spectral Preconditioning Converges for Nonconvex Objectives

In this work, we develop proximal preconditioned gradient methods with a focus on spectral gradient methods providing a proximal extension to the Muon and Scion optimizers. We introduce a family of stochastic algorithms that can handle a wide variety of convex and nonconvex constraints and study its convergence under heavy-tailed noise, through a novel analysis tailored to the geometry of the proposed methods. We further propose a variance-reduced version, which achieves faster convergence under standard noise assumptions. Finally, we show that the polynomial iterations used in Muon are more accurately captured by a nonlinear preconditioner than by the ideal matrix sign, leading to a convergence analysis that more faithfully reflects practical implementations.

math.OC↗

Newton methods beyond Hessian Lipschitz continuity: A nonlinear preconditioning approach

Newton-type methods are typically analyzed under Lipschitz continuity of the Hessian, an assumption that can fail for objectives with higher-order or polynomial growth. We introduce a class of nonlinearly preconditioned Newton methods that apply Newton's root-finding scheme to a transformed optimality mapping, thereby extending recent nonlinear preconditioning ideas from first-order methods to the second-order setting. The resulting methods are naturally analyzed under Lipschitz continuity of a preconditioned Hessian, a condition that significantly relaxes the classical Hessian Lipschitz continuity assumption. Under this generalized smoothness model, we establish local superlinear and quadratic convergence guarantees, and develop a globalization strategy for the nonregularized method despite the fact that the preconditioned Newton direction need not be a descent direction. We further propose a regularized variant for isotropic preconditioners, and show that it attains an $O(\varepsilon^{-3/2})$ iteration complexity. An adaptive version removes the need to know the smoothness constant and allows inexact subproblem solutions while preserving the same complexity order.

math.OC↗

Nonlinearly preconditioned gradient flows

We study a continuous-time dynamical system which arises as the limit of a broad class of nonlinearly preconditioned gradient methods. Under mild assumptions, we establish existence of global solutions and derive Lyapunov-based convergence guarantees. For convex costs, we prove a sublinear decay in a geometry induced by some reference function, and under a generalized gradient-dominance condition we obtain exponential convergence. We further uncover a duality connection with mirror descent, and use it to establish that the flow of interest solves an infinite-horizon optimal-control problem of which the value function is the Bregman divergence generated by the cost. These results clarify the structure and optimization behavior of nonlinearly preconditioned gradient flows and connect them to known continuous-time models in non-Euclidean optimization.

math.OC↗

PANOC-lite: A simpler and more efficient algorithm for composite minimization

This work introduces a simple and efficient linesearch method for composite minimization that accelerates proximal-gradient iterations with fast Newton-type directions. Our algorithm is based on simple operations and only requires the standard proximal-gradient oracle, similar to PANOC and ZeroFPR, provided that the nonsmooth term is convex. Noteworthy improvements include a cheaper backtracking procedure, in the sense that no additional gradients need to be evaluated, and an enlarged range of permitted stepsizes. Global subsequential convergence and local superlinear convergence are established under conventional assumptions by considering a novel merit function which is less expensive to evaluate than alternatives like the forward-backward envelope. Finally, the proposed approach is validated on model predictive control problems with collision avoidance constraints, as well as on the LIBSVM and CUTEst benchmarks.

math.OC↗

Parametric Nonconvex Optimization via Convex Surrogates

This paper presents a novel learning-based approach to construct a surrogate problem that approximates a given parametric nonconvex optimization problem. The surrogate function is designed to be the minimum of a finite set of functions, given by the composition of convex and monotonic terms, so that the surrogate problem can be solved directly through parallel convex optimization. As a proof of concept, numerical experiments on a nonconvex path tracking problem confirm the approximation quality of the proposed method.

math.OC↗

On the Regularity of Generalized Conjugate Functions

We investigate regularity properties of generalized conjugate functions induced by a general coupling function and the associated generalized proximal mapping. Our main results provide verifiable conditions ensuring local single-valuedness, continuity, Lipschitz continuity, and differentiability of the generalized proximal mapping, and transfer these properties to generalized conjugates providing explicit derivative formulas. These results are based on a nonsmooth implicit function theorem for generalized equations, relying on graphical localizations and second-order variational tools. Beyond first-order regularity, we also derive conditions under which generalized conjugates are strictly twice differentiable.

math.OC↗

Cyqlone: A Parallel, High-Performance Linear Solver for Optimal Control

We present Cyqlone, a solver for linear systems with a stage-wise optimal control structure that fully exploits the various levels of parallelism available in modern hardware. Cyqlone unifies algorithms based on the sequential Riccati recursion, parallel Schur complement methods, and cyclic reduction methods, thereby minimizing the required number of floating-point operations, while allowing parallelization across a configurable number of processors. Given sufficient parallelism, the solver run time scales with the logarithm of the horizon length (in contrast to the linear scaling of sequential Riccati-based methods), enabling real-time solution of long-horizon problems. Beyond multithreading on multi-core processors, implementations of Cyqlone can also leverage vectorization using batched linear algebra routines. Such batched routines exploit data parallelism using single instruction, multiple data (SIMD) operations, and expose a higher degree of instruction-level parallelism than their non-batched counterparts. This enables them to significantly outperform BLAS and BLASFEO for the small matrices that arise in optimal control. Building on this high-performance linear solver, we develop CyQPALM, a parallel and optimal-control-specific variant of the QPALM quadratic programming solver. It combines the parallel and vectorized linear algebra operations from Cyqlone with a parallel line search and parallel factorization updates, resulting in order-of-magnitude speedups over the state-of-the-art HPIPM solver. Open-source C++ implementations of Cyqlone and CyQPALM are available at https://github.com/kul-optec/cyqlone

math.OC↗

Exploiting Parallelism in a QPALM-based Solver for Optimal Control

We discuss the opportunities for parallelization in the recently proposed QPALM-OCP algorithm, a solver tailored to quadratic programs arising in optimal control. A significant part of the computational work can be carried out independently for the different stages in the optimal control problem. We exploit this specific structure to apply parallelization and vectorization techniques in an optimized C++ implementation of the method. Results for optimal control benchmark problems and comparisons to the original QPALM method are provided.

math.OC↗

Exact worst-case convergence rates of gradient descent: a complete analysis for all constant stepsizes over nonconvex and convex functions

We consider gradient descent with constant stepsizes and derive exact worst-case convergence rates on the minimum gradient norm of the iterates. Our analysis covers all possible stepsizes and arbitrary upper/lower bounds on the curvature of the objective function, thus including convex, strongly convex and weakly convex (hypoconvex) objective functions. Among the challenging parts of the analysis, we note the necessity to exploit dependencies between non-consecutive iterates. While this complicates the proofs to some extent, it enables us to achieve an exact full-range analysis of gradient descent for any constant stepsize (covering, in particular, normalized stepsizes greater than one), whereas the literature contained only conjectured rates of this type. In the nonconvex case, allowing arbitrary bounds on upper and lower curvatures extends existing partial results that are valid only for gradient Lipschitz functions (i.e., where lower and upper bounds on curvature are equal), leading to improved rates for weakly convex functions. From our exact worst-case performance bounds, we deduce the optimal constant stepsize for gradient descent. Leveraging our analysis, we also introduce a new variant of gradient descent based on a unique, fixed sequence of variable stepsizes, demonstrating its superiority in the worst-case over any constant stepsize schedule.

math.OC↗