Jonathan Chirinos-Rodríguez

arXiv2026

On The Linear Convergence of Bregman Proximal Gradient Methods with Applications to Kullback--Leibler regression

Bregman Proximal Gradient methods (BPGM) exploit the underlying geometry of the objective function through a carefully chosen mirror map. In this work, we introduce a novel notion of strong convexity, termed Restricted Relative Strong Convexity, and establish linear convergence rates for BPGM under this condition. We then exploit the proposed theoretical framework to provide an in-depth analysis of the convergence of BPGM for (regularized) Kullback--Leibler regression problems, covering scenarios with both unique and non-unique minimizers, as well as regularized and unregularized formulations. Specifically, we demonstrate that using the popular Burg's entropy as a distance-generating function may only yield linear convergence for certain KL regression problems. In contrast, we show that employing a smoothed version of the Burg's entropy induces the suitable geometry required to guarantee linear convergence. We conclude with numerical experiments that nicely align with our theoretical findings.

math.OC↗

arXiv2026

Optimization landscape of $\ell_0$-Bregman relaxations

In this paper, we study (noisy) linear systems, and their $\ell_0$-regularized optimization problems, coupled with general data fidelity terms. Recent approaches for solving this class of problems have proposed to consider non-convex exact continuous relaxations that preserve global minimizers while reducing the number of local minimizers. Within this framework, we consider the class of $\ell_0$-Bregman relaxations, and establish sufficient conditions under which a critical point is isolated in terms of sparsity, in the sense that any other critical point has a strictly larger cardinality. In this way, we ensure a form of uniqueness in the solution structure. Furthermore, we analyze the exact recovery properties of such exact relaxations. To that end, we derive conditions under which the oracle solution (i.e., the one sharing the same support as the ground-truth) is the unique global minimizer of the relaxed problem, and is isolated in terms of sparsity. Our analysis is primarily built upon a novel property we introduce, termed the Bregman Restricted Strong Convexity. Finally, we specialize our general results to both sparse Gaussian (least-squares) and Poisson ((generalized) Kullback-Leibler divergence) regression problems. In particular, we show that our general analysis sharpens existing bounds for the LS setting, while providing an entirely new result for the KL case.

math.OC↗