Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.23206

Synthesis of the Transient Behaviour of Optimization Algorithms for Strongly Convex Objective Functions

Adel Abdelsamed·Dennis Gramlich·Christian Ebenbauer·Carsten W. Scherer
View original source ↗Read source document ↗Export metadata ↓

Abstract

We study the transient behavior of first-order optimization methods for unconstrained minimization of smooth and strongly convex objective functions. These methods are modelled as linear systems over finite-time intervals embedded in feedback with the gradient nonlinearity. By characterizing the function class through discrete-time Zames-Falb integral quadratic constraints, we employ a robust $\mathcal{H}_2$ control framework to obtain convex semidefinite programs certifying tight worst-case bounds on the terminal distance to the optimizer. Building on the same formulation, we derive a convex synthesis procedure for a transient-optimal method that achieves the optimal finite-horizon bounds and guarantees error reduction over any prescribed number of steps. The framework naturally extends to delayed gradient measurements, enabling both transient performance analysis and synthesis of optimal algorithms under feedback delays.

Explore related subjects

math.OC

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Adel Abdelsamed, Dennis Gramlich, Christian Ebenbauer, Carsten W. Scherer. 2026-10-16. Synthesis of the Transient Behaviour of Optimization Algorithms for Strongly Convex Objective Functions. https://arxiv.org/abs/2610.23206

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

A Behavioral Model for Exploration vs. Exploitation: Theoretical Framework and Experimental Evidence

How do people navigate the exploration-exploitation (EE) trade-off when making repeated choices with unknown rewards? We study this question through the lens of multi-armed bandit problems and introduce a novel behavioral model, Quantal Choice with Adaptive Reduction of Exploration (QCARE). It generalizes Thompson Sampling, allowing for a principled way to quantify the EE trade-off and reflect human decision-making patterns. The model adaptively reduces exploration as information accumulates, with the reduction rate serving as a parameter to quantify the EE trade-off dynamics. We theoretically analyze how varying reduction rates influence decision quality, shedding light on the distinct effects of ``over-exploration'' and ``under-exploration.'' Empirically, we validate QCARE using behavioral data collected from human participants. QCARE not only captures critical behavioral patterns in the EE trade-off but also performs competitively with, and often outperforms, alternative models in out-of-sample prediction. Our analysis reveals a behavioral tendency toward over-exploration.

math.OC↗
arXiv2026

Active-Set Identification in Noisy and Stochastic Optimization

Identifying active constraints from a point near a local minimizer is important both theoretically and practically in constrained continuous optimization as it can help identify optimal Lagrange multipliers and essentially reduce an inequality-constrained problem to an equality-constrained one. Traditional active-set identification guarantees have been proved under various types of constraint qualifications. Importantly, they also assume exact function and derivative values. This work extends such results to settings when both objective and constraint function and derivative values have deterministic or stochastic noise. Two strategies are analyzed. For each strategy, it is shown that, under constraint qualifications and other standard conditions, the active set of a local minimizer can be identified correctly when a point is close enough to the minimizer and the noise is sufficiently small. It is also shown that regularity of a minimizer is critical. Under strict complementarity, a fixed noise level suffices, whereas in the presence of a weakly active constraint the noise must vanish at least linearly in the distance to the minimizer in general, a rate that is also shown to be sufficient for correct identification. Guarantees are also stated for the use of active-set identification strategies within a stochastic algorithm, with explicit sample-size conditions. The theoretical findings are demonstrated with two simple illustrative examples and a more realistic constrained neural-network training task.

math.OC↗
arXiv2026

Regularization in Data-driven Predictive Control: A Convex Relaxation Perspective

This paper explores the role of regularization in data-driven predictive control (DDPC) through the lens of convex relaxation. Using a bi-level optimization framework, we model system identification as an inner problem and predictive control as an outer problem. Within this framework, we show that several regularized DDPC formulations, including l1-norm penalties, projection-based regularizers, and a newly introduced causality-based regularizer, can be viewed as convex relaxations of their respective bi-level problems. This perspective clarifies the conceptual links between direct and indirect data-driven control and highlights how regularization implicitly enforces system identification. We further propose an optimality-based variant, A-DDPC, which approximately solves the inner problem with all identification constraints via an iterative algorithm. Numerical experiments demonstrate that A-DDPC outperforms existing regularized DDPC by reducing both bias and variance errors. These results indicate that further benefits may be obtained by applying system identification techniques to pre-process the trajectory library in nonlinear settings. Overall, our analysis contributes to a unified convex relaxation view of regularization in DDPC and sheds light on its strong empirical performance beyond linear time-invariant systems.

math.OC↗