arXiv subjects

Geoffroy Leconte

Publications and source records attributed to Geoffroy Leconte.

3 recordsLinked to original sources

arXiv2024

An interior-point trust-region method for nonsmooth regularized bound-constrained optimization

We develop an interior-point method for nonsmooth regularized bound-constrained optimization problems. Our method consists of iteratively solving a sequence of unconstrained nonsmooth barrier subproblems. We use a variant of the proximal quasi-Newton trust-region algorithm TR of arXiv:2103.15993v3 to solve the barrier subproblems, with additional assumptions inspired from well-known smooth interior-point trust-region methods. We show global convergence of our algorithm with respect to the criticality measure of arXiv:2103.15993v3. Under an additional assumption linked to the convexity of the nonsmooth term in the objective, we present an alternative interior-point algorithm with a slightly modified criticality measure, which performs better in practice. Numerical experiments show that our algorithm performs better than the trust-region method TR, the trust-region method with diagonal hessian approximations TRDH of arXiv:2309.08433, and the quadratic regularization method R2 of arXiv:2103.15993v3 for two out of four tested bound-constrained problems. On those two problems, our algorithm obtains smaller objective values than the other solvers using fewer objective and gradient evaluations. On the two other problems, it performs similarly to TR, R2 and TRDH.

math.OC↗

arXiv2023

Complexity of trust-region methods with unbounded Hessian approximations for smooth and nonsmooth optimization

We develop a worst-case evaluation complexity bound for trust-region methods in the presence of unbounded Hessian approximations. We use the algorithm of arXiv:2103.15993v3 as a model, which is designed for nonsmooth regularized problems, but applies to unconstrained smooth problems as a special case. Our analysis assumes that the growth of the Hessian approximation is controlled by the number of successful iterations. We show that the best known complexity bound of $\epsilon^{-2}$ deteriorates to $\epsilon^{-2/(1-p)}$, where $0 \le p < 1$ is a parameter that controls the growth of the Hessian approximation. The faster the Hessian approximation grows, the more the bound deteriorates. We construct an objective that satisfies all of our assumptions and for which our complexity bound is attained, which establishes that our bound is sharp. To the best of our knowledge, our complexity result is the first to consider potentially unbounded Hessians and is a first step towards addressing a conjecture of Powell [38] that trust-region methods may require an exponential number of iterations in such a case. Numerical experiments conducted in double precision arithmetic are consistent with the analysis.

math.OC↗

arXiv2023

The Indefinite Proximal Gradient Method

We introduce a variant of the proximal gradient method in which the quadratic term is diagonal but may be indefinite, and is safeguarded by a trust region. Our method is a special case of the proximal quasi-Newton trust-region method of arXiv:2103.15993v3. We provide closed-form solution of the step computation in certain cases where the nonsmooth term is separable and the trust region is defined in infinity norm, so that no iterative subproblem solver is required. Our analysis expands upon that of arXiv:2103.15993v3 by generalizing the trust-region approach to problems with bound constraints. We provide an efficient open-source implementation of our method, named TRDH, in the Julia language in which Hessians approximations are given by diagonal quasi-Newton updates. TRDH evaluates one standard proximal operator and one indefinite proximal operator per iteration. We also analyze and implement a variant named iTRDH that performs a single indefinite proximal operator evaluation per iteration. We establish that iTRDH enjoys the same asymptotic worst-case iteration complexity as TRDH. We report numerical experience on unconstrained and bound-constrained problems, where TRDH and iTRDH are used both as standalone and subproblem solvers. Our results illustrate that, as standalone solvers, TRDH and iTRDH improve upon the quadratic regularization method R2 of arXiv:2103.15993v3 but also sometimes upon their quasi-Newton trust-region method, referred to here as TR-R2, in terms of smooth objective value and gradient evaluations. On challenging nonnegative matrix factorization, binary classification and data fitting problems, TRDH and iTRDH used as subproblem solvers inside TR improve upon TR-R2 for at least one choice of diagonal approximation.

math.OC↗