arXiv subjects

Andreas Themelis

Publications and source records attributed to Andreas Themelis.

At least 19 recordsLinked to original sources

arXiv2026

PANOC-lite: A simpler and more efficient algorithm for composite minimization

This work introduces a simple and efficient linesearch method for composite minimization that accelerates proximal-gradient iterations with fast Newton-type directions. Our algorithm is based on simple operations and only requires the standard proximal-gradient oracle, similar to PANOC and ZeroFPR, provided that the nonsmooth term is convex. Noteworthy improvements include a cheaper backtracking procedure, in the sense that no additional gradients need to be evaluated, and an enlarged range of permitted stepsizes. Global subsequential convergence and local superlinear convergence are established under conventional assumptions by considering a novel merit function which is less expensive to evaluate than alternatives like the forward-backward envelope. Finally, the proposed approach is validated on model predictive control problems with collision avoidance constraints, as well as on the LIBSVM and CUTEst benchmarks.

math.OC↗

arXiv2025

Linesearch-free adaptive Bregman proximal gradient for convex minimization under local relative smoothness

This paper introduces adaptive Bregman proximal gradient algorithms for solving convex composite minimization problems without relying on global relative smoothness or strong convexity assumptions. Building upon recent advances in adaptive stepsize selections, the proposed methods generate stepsizes based on local curvature estimates, entirely eliminating the need for backtracking linesearch. A key tool in our analysis is a Bregman generalization of Young's inequality, which allows the control of a critical inner product in terms of the same Bregman distances used in the updates. Our theory applies to problems where the differentiable term is merely \emph{locally} smooth relative to a distance-generating function, without requiring the existence of global moduli or symmetry coefficients. Numerical experiments demonstrate their competitive performance compared to existing approaches across various problem classes.

math.OC↗

arXiv2025

A Lasry-Lions envelope approach for mathematical programs with complementarity constraints

We propose a homotopy method for solving mathematical programs with complementarity constraints (CCs). The indicator function of the CCs is relaxed by the Lasry--Lions double envelope, an extension of the Moreau envelope that enjoys an additional smoothness property, making it amenable to fast optimization algorithms. The proposed algorithm mimics the behavior of homotopy methods for systems of nonlinear equations or penalty methods for constrained optimization: it solves a sequence of smooth subproblems that progressively approximate the original problem, using the solution of each subproblem as the starting point for the next one. In the limiting setting, we establish the convergence to Mordukhovich and Clarke stationary points. We also provide a worst-case complexity analysis for computing an approximate stationary point. Preliminary numerical results on a suite of benchmark problems demonstrate the effectiveness of the proposed approach.

math.OC↗

arXiv2025

Bregman level proximal subdifferentials and new characterizations of Bregman proximal operators

Classic subdifferentials in variational analysis may fail to fully represent the Bregman proximal operator in the absence of convexity. In this paper, we fill this gap by introducing the left and right \emph{Bregman level proximal subdifferentials} and investigate them systematically. Every Bregman proximal operator turns out to be the resolvent of a Bregman level proximal subdifferential under a standard range assumption, even without convexity. Aided by this pleasant feature, we establish new correspondences among useful properties of the Bregman proximal operator, the underlying function, and the (left) Bregman level proximal subdifferential, generalizing classical equivalences in the Euclidean case. Unlike the classical setting, asymmetry and duality gap emerge as natural consequences of the Bregman distance. Along the way, we improve results by Kan and Song and by Wang and Bauschke on Bregman proximal operators. We also characterize the existence and single-valuedness of the Bregman level proximal subdifferential, investigate coincidence results, and make an interesting connection to relative smoothness. Abundant examples are provided to justify the necessity of our assumptions. We also introduce \emph{anisotropic firm nonexpansiveness}, a new notion that is complementary to \emph{Bregman} firm nonexpansiveness and is shown to characterize relative smooth convex functions and convex envelopes via properties of gradient and proximal operators.

math.OC↗

arXiv2025

On the natural domain of Bregman operators

The Bregman proximal mapping and Bregman-Moreau envelope are traditionally studied for functions defined on the entire space $\mathbb{R}^n$, even though these constructions depend only on the values of the function within (the interior of) the domain of the distance-generating function (dgf). While this convention is largely harmless in the convex setting, it leads to substantial limitations in the nonconvex case, as it fails to embrace important classes of functions such as relatively weakly convex ones. In this work, we revisit foundational aspects of Bregman analysis by adopting a domain-aware perspective: we define functions on the natural domain induced by the dgf and impose properties only relative to this set. This framework not only generalizes existing results but also rectifies and simplifies their statements and proofs. Several examples illustrate both the necessity of our assumptions and the advantages of this refined approach.

math.OC↗

arXiv2024

Adaptive proximal gradient methods are universal without approximation

We show that adaptive proximal gradient methods for convex problems are not restricted to traditional Lipschitzian assumptions. Our analysis reveals that a class of linesearch-free methods is still convergent under mere local Hölder gradient continuity, covering in particular continuously differentiable semi-algebraic functions. To mitigate the lack of local Lipschitz continuity, popular approaches revolve around $\varepsilon$-oracles and/or linesearch procedures. In contrast, we exploit plain Hölder inequalities not entailing any approximation, all while retaining the linesearch-free nature of adaptive schemes. Furthermore, we prove full sequence convergence without prior knowledge of local Hölder constants nor of the order of Hölder continuity. Numerical experiments make comparisons with baseline methods on diverse tasks from machine learning covering both the locally and the globally Hölder setting.

math.OC↗

arXiv2024

A penalty barrier framework for nonconvex constrained optimization

We consider minimization problems with structured objective function and smooth constraints, and present a flexible framework that combines the beneficial regularization effects of (exact) penalty and interior-point methods. In the fully nonconvex setting, a pure barrier approach requires careful steps when approaching the infeasible set, thus hindering convergence. We show how a tight integration with a penalty scheme mitigates this issue and enables the construction of subproblems whose domain is independent of the explicit constraints. This decoupling allows us to leverage efficient solvers designed for unconstrained or suitably structured optimization tasks. The key behind all this is a marginalization step: closely related to a conjugacy operation, this step effectively merges (exact) penalty and barrier into a smooth, full domain functional object. When the penalty exactness takes effect, the generated subproblems do not suffer the ill-conditioning typical of barrier methods, nor do they exhibit the nonsmoothness of exact penalty terms. We provide a theoretical characterization of the algorithm and its asymptotic properties, deriving convergence results for fully nonconvex problems. Stronger conclusions are available for the convex setting, where optimality can be guaranteed. Illustrative examples and numerical simulations demonstrate the wide range of problems our theory and algorithm are able to cover.

math.OC↗

arXiv2024

On the convergence of adaptive first order methods: proximal gradient and alternating minimization algorithms

Building upon recent works on linesearch-free adaptive proximal gradient methods, this paper proposes adaPG$^{q,r}$, a framework that unifies and extends existing results by providing larger stepsize policies and improved lower bounds. Different choices of the parameters $q$ and $r$ are discussed and the efficacy of the resulting methods is demonstrated through numerical simulations. In an attempt to better understand the underlying theory, its convergence is established in a more general setting that allows for time-varying parameters. Finally, an adaptive alternating minimization algorithm is presented by exploring the dual setting. This algorithm not only incorporates additional adaptivity, but also expands its applicability beyond standard strongly convex settings.

math.OC↗

arXiv2024

Safeguarding adaptive methods: global convergence of Barzilai-Borwein and other stepsize choices

Leveraging on recent advancements on adaptive methods for convex minimization problems, this paper provides a linesearch-free proximal gradient framework for globalizing the convergence of popular stepsize choices such as Barzilai-Borwein and one-dimensional Anderson acceleration. This framework can cope with problems in which the gradient of the differentiable function is merely locally H\"older continuous. Our analysis not only encompasses but also refines existing results upon which it builds. The theory is corroborated by numerical evidence that showcases the synergetic interplay between fast stepsize selections and adaptive methods.

math.OC↗

arXiv2024

Adaptive proximal algorithms for convex optimization under local Lipschitz continuity of the gradient

Backtracking linesearch is the de facto approach for minimizing continuously differentiable functions with locally Lipschitz gradient. In recent years, it has been shown that in the convex setting it is possible to avoid linesearch altogether, and to allow the stepsize to adapt based on a local smoothness estimate without any backtracks or evaluations of the function value. In this work we propose an adaptive proximal gradient method, adaPG, that uses novel estimates of the local smoothness modulus which leads to less conservative stepsize updates and that can additionally cope with nonsmooth terms. This idea is extended to the primal-dual setting where an adaptive three-term primal-dual algorithm, adaPD, is proposed which can be viewed as an extension of the PDHG method. Moreover, in this setting the "essentially" fully adaptive variant adaPD$^+$ is proposed that avoids evaluating the linear operator norm by invoking a backtracking procedure, that, remarkably, does not require extra gradient evaluations. Numerical simulations demonstrate the effectiveness of the proposed algorithms compared to the state of the art.

math.OC↗

arXiv2024

On the convergence of proximal gradient methods for convex simple bilevel optimization

This paper studies proximal gradient iterations for solving simple bilevel optimization problems where both the upper and the lower level cost functions are split as the sum of differentiable and (possibly nonsmooth) proximable functions. We develop a novel convergence recipe for iteration varying stepsizes that relies on Barzilai-Borwein type local estimates for the differentiable terms. Leveraging the convergence recipe, under global Lipschitz gradient continuity, we establish convergence for a nonadaptive stepsize sequence, without requiring any strong convexity or linesearch. In the locally Lipschitz differentiable setting, we develop an adaptive linesearch method that introduces a systematic adaptive scheme enabling large and nonmonotonic stepsize sequences while being insensitive to the choice of hyperparameters and initialization. Numerical simulations are provided showcasing favorable convergence speed of our methods.

math.OC↗

arXiv2024

An interior proximal gradient method for nonconvex optimization

We consider structured minimization problems subject to smooth inequality constraints and present a flexible algorithm that combines interior point (IP) and proximal gradient schemes. While traditional IP methods cannot cope with nonsmooth objective functions and proximal algorithms cannot handle complicated constraints, their combined usage is shown to successfully compensate the respective shortcomings. We provide a theoretical characterization of the algorithm and its asymptotic properties, deriving convergence results for fully nonconvex problems, thus bridging the gap with previous works that successfully addressed the convex case. Our interior proximal gradient algorithm benefits from warm starting, generates strictly feasible iterates with decreasing objective value, and returns after finitely many iterations a primal-dual pair approximately satisfying suitable optimality conditions. As a byproduct of our analysis of proximal gradient iterations we demonstrate that a slight refinement of traditional backtracking techniques waives the need for upper bounding the stepsize sequence, as required in existing results for the nonconvex setting.

math.OC↗

arXiv2024

SPIRAL: A superlinearly convergent incremental proximal algorithm for nonconvex finite sum minimization

We introduce SPIRAL, a SuPerlinearly convergent Incremental pRoximal ALgorithm, for solving nonconvex regularized finite sum problems under a relative smoothness assumption. Each iteration of SPIRAL consists of an inner and an outer loop. It combines incremental gradient updates with a linesearch that has the remarkable property of never being triggered asymptotically, leading to superlinear convergence under mild assumptions at the limit point. Simulation results with L-BFGS directions on different convex, nonconvex, and non-Lipschitz differentiable problems show that our algorithm, as well as its adaptive variant, are competitive to the state of the art.

math.OC↗

arXiv2023

Dualities for non-Euclidean smoothness and strong convexity under the light of generalized conjugacy

Relative smoothness and strong convexity have recently gained considerable attention in optimization. These notions are generalizations of the classical Euclidean notions of smoothness and strong convexity that are known to be dual to each other. However, conjugate dualities for non-Euclidean relative smoothness and strong convexity remain an open problem as noted earlier by Lu, Freund and Nesterov [SIAM J. Optim., 28 (2018), pp. 333-354]. In this paper we address this question by introducing the notions of anisotropic strong convexity and smoothness as the respective dual counterparts. The dualities are developed under the light of generalized conjugacy which leads us embed the anticipated dual notions within the superclasses of certain upper and lower envelopes. In contrast to the Euclidean case these inclusions are proper in general as showcased by means of counterexamples.

math.OC↗

arXiv2023

Optimal Grid Layouts for Hybrid Offshore Assets in the North Sea under Different Market Designs

This work examines the Generation and Transmission Expansion (GATE) planning problem of offshore grids under different market clearing mechanisms: a Home Market Design (HMD), a zonal cleared Offshore Bidding Zone (zOBZ) and a nodal cleared Offshore Bidding Zone (nOBZ). It aims at answering two questions. 1) Is knowing the market structure a priori necessary for effective generation and transmission expansion planning? 2) Which market mechanism results in the highest overall social welfare? To this end a multi-period, stochastic GATE planning formulation is developed for both nodal and zonal market designs. The approach considers the costs and benefits among stake-holders of Hybrid Offshore Assets (HOA) as well as gross consumer surplus (GCS). The methodology is demonstrated on a North Sea test grid based on projects from the European Network of Transmission System Operators' (ENTSO-E) Ten Year Network Development Plan (TYNDP). An upper bound on potential social welfare in zonal market designs is calculated and it is concluded that from a generation and transmission perspective, planning under the assumption of an nOBZ results in the best risk adjusted return.

eess.SP↗

arXiv2022

Gauss-Newton meets PANOC: A fast and globally convergent algorithm for nonlinear optimal control

PANOC is an algorithm for nonconvex optimization that has recently gained popularity in real-time control applications due to its fast, global convergence. The present work proposes a variant of PANOC that makes use of Gauss-Newton directions to accelerate the method. Furthermore, we show that when applied to optimal control problems, the computation of this Gauss-Newton step can be cast as a linear quadratic regulator (LQR) problem, allowing for an efficient solution through the Riccati recursion. Finally, we demonstrate that the proposed algorithm is more than twice as fast as the traditional L-BFGS variant of PANOC when applied to an optimal control benchmark problem, and that the performance scales favorably with increasing horizon length.

math.OC↗

arXiv2022

A mirror inertial forward-reflected-backward splitting: Global convergence and linesearch extension beyond convexity and Lipschitz smoothness

This work investigates a Bregman and inertial extension of the forward-reflected-backward algorithm [Y. Malitsky and M. Tam, SIAM J. Optim., 30 (2020), pp. 1451--1472] applied to structured nonconvex minimization problems under relative smoothness. To this end, the proposed algorithm hinges on two key features: taking inertial steps in the dual space, and allowing for possibly negative inertial values. Our analysis begins with studying an associated envelope function that takes inertial terms into account through a novel product space formulation. Such construction substantially differs from similar objects in the literature and could offer new insights for extensions of splitting algorithms. Global convergence and rates are obtained by appealing to the generalized concave Kurdyka-Lojasiewicz (KL) property, which allows us to describe a sharp upper bound on the total length of iterates. Finally, a linesearch extension is given to enhance the proposed method.

math.OC↗

arXiv2022

Flock navigation with dynamic hierarchy and subjective weights using nonlinear MPC

We propose a model predictive control (MPC) based approach to a flock control problem with obstacle avoidance capability in a leader-follower framework, utilizing the future trajectory prediction computed by each agent. We employ the traditional Reynolds' flocking rules (cohesion, separation, and alignment) as a basis, and tailor the model to fit a navigation (as opposed to formation) purpose. In particular, we introduce several concepts such as the credibility and the importance of the gathered information from neighbors, and dynamic trade-offs between references. They are based on the observations that near-future predictions are more reliable, agents closer to leaders are implicit carriers of more educated information, and the predominance of either cohesion or alignment is dictated by the distance between the agent and its neighbors. These features are incorporated in the MPC formulation, and their advantages are discussed through numerical simulations.

math.OC↗