arXiv subjects

Guillaume Olikier

Publications and source records attributed to Guillaume Olikier.

13 recordsLinked to original sources

arXiv2026

The tangent cone to the real determinantal variety: various expressions and a proof

The set of real matrices of upper-bounded rank is a real algebraic variety called the real generic determinantal variety. An explicit description of the tangent cone to that variety is given in Theorem 3.2 of Schneider and Uschmajew [SIAM J. Optim., 25 (2015), pp. 622-646]. The present paper shows that the proof therein is incomplete and provides a proof. It also reviews equivalent descriptions of the tangent cone to that variety. Moreover, it shows that the tangent cone and the algebraic tangent cone to that variety coincide, which is not true for all real algebraic varieties.

math.OC↗

arXiv2026

Low-rank optimization methods based on projected projected-gradient descent that accumulate at Bouligand stationary points

This paper considers the problem of minimizing a differentiable function with locally Lipschitz continuous gradient on the algebraic variety of real matrices of upper-bounded rank. This problem is known to enable the formulation of various machine learning or signal processing tasks such as dimensionality reduction, collaborative filtering, and signal recovery. Several definitions of stationarity exist for this nonconvex problem. Among them, Bouligand stationarity is the strongest necessary condition for local optimality. This paper proposes two first-order methods that generate a sequence in the variety whose accumulation points are Bouligand stationary. The first method combines the well-known projected projected-gradient descent map with a rank reduction mechanism. The second method is a hybrid of projected gradient descent and projected projected-gradient descent. Both methods stand out in the field of low-rank optimization methods when considering their convergence properties, their streamlined design, their typical computational cost per iteration, and their empirically observed numerical performance. The theoretical framework used to analyze the proposed methods is of independent interest.

math.OC↗

arXiv2025

Eigengap Sparsity for Covariance Parsimony

Covariance estimation is a central problem in statistics. An important issue is that there are rarely enough samples $n$ to accurately estimate the $p (p+1) / 2$ coefficients in dimension $p$. Parsimonious covariance models are therefore preferred, but the discrete nature of model selection makes inference computationally challenging. In this paper, we propose a relaxation of covariance parsimony termed "eigengap sparsity" and motivated by the good accuracy-parsimony tradeoffs of eigenvalue-equalization in covariance matrices. This penalty can be included in a penalized-likelihood framework that we propose to solve with a projected gradient descent on a monotone cone. The algorithm turns out to resemble an isotonic regression of mutually-attracted sample eigenvalues, drawing an interesting link between covariance parsimony and shrinkage.

stat.ME↗

arXiv2025

Projected gradient descent accumulates at Bouligand stationary points

This paper considers the projected gradient descent (PGD) algorithm for the problem of minimizing a continuously differentiable function on a nonempty closed subset of a Euclidean vector space. Without further assumptions, this problem is intractable and algorithms are only expected to find a stationary point. PGD generates a sequence in the set whose accumulation points are known to be Mordukhovich stationary. In this paper, these accumulation points are proven to be Bouligand stationary, and even proximally stationary if the gradient is locally Lipschitz continuous. These are the strongest stationarity properties that can be expected for the considered problem.

math.OC↗

arXiv2025

Gauss-Southwell type descent methods for low-rank matrix optimization

We consider gradient-related methods for low-rank matrix optimization with a smooth cost function. The methods operate on single factors of the low-rank factorization and share aspects of both alternating and Riemannian optimization. Two possible choices for the search directions based on Gauss-Southwell type selection rules are compared: one using the gradient of a factorized non-convex formulation, the other using the Riemannian gradient. While both methods provide gradient convergence guarantees that are similar to the unconstrained case, numerical experiments on a quadratic cost function indicate that the version based on the Riemannian gradient is significantly more robust with respect to small singular values and the condition number of the cost function. As a side result of our approach, we also obtain new convergence results for the alternating least squares method.

math.OC↗

arXiv2024

Computing Bouligand stationary points efficiently in low-rank optimization

This paper considers the problem of minimizing a differentiable function with locally Lipschitz continuous gradient on the algebraic variety of all $m$-by-$n$ real matrices of rank at most $r$. Several definitions of stationarity exist for this nonconvex problem. Among them, Bouligand stationarity is the strongest necessary condition for local optimality. Only a handful of algorithms generate a sequence in the variety whose accumulation points are provably Bouligand stationary. Among them, the most parsimonious with (truncated) singular value decompositions (SVDs) or eigenvalue decompositions can still require a truncated SVD of a matrix whose rank can be as large as $\min\{m, n\}-r+1$ if the gradient does not have low rank, which is computationally prohibitive in the typical case where $r \ll \min\{m, n\}$. This paper proposes a first-order algorithm that generates a sequence in the variety whose accumulation points are Bouligand stationary while requiring SVDs of matrices whose smaller dimension is always at most $r$. A standard measure of Bouligand stationarity converges to zero along the bounded subsequences at a rate at least $O(1/\sqrt{i+1})$, where $i$ is the iteration counter. Furthermore, a rank-increasing scheme based on the proposed algorithm is presented, which can be of interest if the parameter $r$ is potentially overestimated.

math.OC↗

arXiv2024

Retractions on closed sets

On a manifold or a closed subset of a Euclidean vector space, a retraction enables to move in the direction of a tangent vector while staying on the set. Retractions are a versatile tool to perform computational tasks such as optimization, interpolation, and numerical integration. This paper studies two known definitions of retraction on a closed subset of a Euclidean vector space, one being weaker than the other. Specifically, it shows that, in the context of constrained optimization, the weaker definition should be preferred as it inherits the main property of the other while being less restrictive.

math.OC↗

arXiv2023

Rank Estimation for Third-Order Tensor Completion in the Tensor-Train Format

We propose a numerical method to obtain an adequate value for the upper bound on the rank for the tensor completion problem on the variety of third-order tensors of bounded tensor-train rank. The method is inspired by the parametrization of the tangent cone derived by Kutschan (2018). A proof of the adequacy of the upper bound for a related low-rank tensor approximation problem is given and an estimated rank is defined to extend the result to the low-rank tensor completion problem. Some experiments on synthetic data illustrate the approach and show that the method is very robust, e.g., to noise on the data.

math.OC↗

arXiv2023

An Approximate Projection onto the Tangent Cone to the Variety of Third-Order Tensors of Bounded Tensor-Train Rank

An approximate projection onto the tangent cone to the variety of third-order tensors of bounded tensor-train rank is proposed and proven to satisfy a better angle condition than the one proposed by Kutschan (2019). Such an approximate projection enables, e.g., to compute gradient-related directions in the tangent cone, as required by algorithms aiming at minimizing a continuously differentiable function on the variety, a problem appearing notably in tensor completion. A numerical experiment is presented which indicates that, in practice, the angle condition satisfied by the proposed approximate projection is better than both the one satisfied by the approximate projection introduced by Kutschan and the proven theoretical bound.

math.OC↗

arXiv2023

First-order optimization on stratified sets

We consider the problem of minimizing a differentiable function with locally Lipschitz continuous gradient on a stratified set and present a first-order algorithm designed to find a stationary point of that problem. Our assumptions on the stratified set are satisfied notably by the determinantal variety (i.e., matrices of bounded rank), its intersection with the cone of positive-semidefinite matrices, and the set of nonnegative sparse vectors. The iteration map of the proposed algorithm applies a step of projected-projected gradient descent with backtracking line search, as proposed by Schneider and Uschmajew (2015), to its input but also to a projection of the input onto each of the lower strata to which it is considered close, and outputs a point among those thereby produced that maximally reduces the cost function. Under our assumptions on the stratified set, we prove that this algorithm produces a sequence whose accumulation points are stationary, and therefore does not follow the so-called apocalypses described by Levin, Kileel, and Boumal (2022). We illustrate the apocalypse-free property of our method through a numerical experiment on the determinantal variety.

math.OC↗

arXiv2022

An apocalypse-free first-order low-rank optimization algorithm with at most one rank reduction attempt per iteration

We consider the problem of minimizing a differentiable function with locally Lipschitz continuous gradient over the real determinantal variety, and present a first-order algorithm designed to find stationary points of that problem. This algorithm applies steps of a retraction-free descent method proposed by Schneider and Uschmajew (2015), while taking the numerical rank into account to attempt rank reductions. We prove that this algorithm produces a sequence of iterates the accumulation points of which are stationary, and therefore does not follow the so-called apocalypses described by Levin, Kileel, and Boumal (2022). Moreover, the rank reduction mechanism of this algorithm requires at most one rank reduction attempt per iteration, in contrast with the one of the $\mathrm{P}^2\mathrm{GDR}$ algorithm introduced by Olikier, Gallivan, and Absil (2022) which can require a number of rank reduction attempts equal to the rank of the iterate in the worst-case scenario.

math.OC↗

arXiv2022

Comparison of an Apocalypse-Free and an Apocalypse-Prone First-Order Low-Rank Optimization Algorithm

We compare two first-order low-rank optimization algorithms, namely $\text{P}^2\text{GD}$ (Schneider and Uschmajew, 2015), which has been proven to be apocalypse-prone (Levin et al., 2021), and its apocalypse-free version $\text{P}^2\text{GDR}$ obtained by equipping $\text{P}^2\text{GD}$ with a suitable rank reduction mechanism (Olikier et al., 2022). Here an apocalypse refers to the situation where the stationarity measure goes to zero along a convergent sequence whereas it is nonzero at the limit. The comparison is conducted on two simple examples of apocalypses, the original one (Levin et al., 2021) and a new one. We also present a potential side effect of the rank reduction mechanism of $\text{P}^2\text{GDR}$ and discuss the choice of the rank reduction parameter.

math.OC↗

arXiv2022

On the continuity of the tangent cone to the determinantal variety

Tangent and normal cones play an important role in constrained optimization to describe admissible search directions and, in particular, to formulate optimality conditions. They notably appear in various recent algorithms for both smooth and nonsmooth low-rank optimization where the feasible set is the set $\mathbb{R}_{\leq r}^{m \times n}$ of all $m \times n$ real matrices of rank at most $r$. In this paper, motivated by the convergence analysis of such algorithms, we study, by computing inner and outer limits, the continuity of the correspondence that maps each $X \in \mathbb{R}_{\leq r}^{m \times n}$ to the tangent cone to $\mathbb{R}_{\leq r}^{m \times n}$ at $X$. We also deduce results about the continuity of the corresponding normal cone correspondence. Finally, we show that our results include as a particular case the $a$-regularity of the Whitney stratification of $\mathbb{R}_{\leq r}^{m \times n}$ following from the fact that this set is a real algebraic variety, called the real determinantal variety.

math.OC↗