arXiv subjects

Lahcen El Bourkhissi

Publications and source records attributed to Lahcen El Bourkhissi.

6 recordsLinked to original sources

arXiv2025

Monotone and nonmonotone linearized block coordinate descent methods for nonsmooth composite optimization problems

In this paper, we introduce both monotone and nonmonotone variants of LiBCoD, a \textbf{Li}nearized \textbf{B}lock \textbf{Co}ordinate \textbf{D}escent method for solving composite optimization problems. At each iteration, a random block is selected, and the smooth components of the objective are linearized along the chosen block in a Gauss-Newton approach. For the monotone variant, we establish a global sublinear convergence rate to a stationary point under the assumption of bounded iterates. For the nonmonotone variant, we derive a global sublinear convergence rate without requiring global Lipschitz continuity or bounded iterates. Preliminary numerical experiments highlight the promising performance of the proposed approach.

math.OC↗

arXiv2025

Complexity of a linearized augmented Lagrangian method for nonconvex minimization with nonlinear equality constraints

In this paper, we consider a nonconvex optimization problem with nonlinear equality constraints. We assume that both, the objective function and the functional constraints are locally smooth. For solving this problem, we propose a linearized augmented Lagrangian method, i.e., we linearize the objective function and the functional constraints in a Gauss-Newton fashion at the current iterate within the augmented Lagrangian function and add a quadratic regularization, yielding a subproblem that is easy to solve, and whose solution is the next primal iterate. The update of the dual multipliers is also based on the linearization of functional constraints. Under a novel dynamic regularization parameter choice, we prove boundedness and global asymptotic convergence of the iterates to a first-order solution of the problem. We also derive convergence guarantees for the iterates of our method to an $ε$-first-order solution in $\mathcal{O}(\sqrtρ ε^{-2})$ Jacobian evaluations, where $ρ$ is the penalty parameter. Moreover, when the problem exhibits a benign nonconvex property, we derive improved convergence results to an $ε$-second-order solution. Finally, we validate the performance of the proposed algorithm by numerically comparing it with the existing methods and software from the literature.

math.OC↗

arXiv2025

Convergence analysis of linearized $\ell_q$ penalty methods for nonconvex optimization with nonlinear equality constraints

In this paper, we consider nonconvex optimization problems with nonlinear equality constraints. We assume that the objective function and the functional constraints are locally smooth. To solve this problem, we introduce a linearized $\ell_q$ penalty based method, where $q \in (1,2]$ is the parameter defining the norm used in the construction of the penalty function. Our method involves linearizing the objective function and functional constraints in a Gauss-Newton fashion at the current iteration in the penalty formulation and introduces a quadratic regularization. This approach yields an easily solvable subproblem, whose solution becomes the next iterate. By using a novel dynamic rule for the choice of the regularization parameter, we establish that the iterates of our method converge to an $ε$-first-order solution in $\mathcal{O}(1/{ε^{2+ (q-1)/q}})$ outer iterations. Finally, we put theory into practice and evaluate the performance of the proposed algorithm by making numerical comparisons with existing methods from literature.

math.OC↗

arXiv2025

Complexity of Linearized Perturbed Augmented Lagrangian Methods for Nonsmooth Nonconvex Optimization with Nonlinear Equality Constraints

This paper addresses a class of general nonsmooth and nonconvex composite optimization problems subject to nonlinear equality constraints. We assume that a part of the objective function and the functional constraints exhibit local smoothness. To tackle this challenging class of problems, we propose a novel linearized perturbed augmented Lagrangian method. This method incorporates a perturbation in the augmented Lagrangian function by scaling the dual variable with a sub-unitary parameter. Furthermore, we linearize the smooth components of the objective and the constraints within the perturbed Lagrangian function at the current iterate, while preserving the nonsmooth components. This approach, inspired by prox-linear (or Gauss-Newton) methods, results in a convex subproblem that is typically easy to solve. The solution of this subproblem then serves as the next primal iterate, followed by a perturbed ascent step to update the dual variables. Under a newly introduced constraint qualification condition, we establish the boundedness of the dual iterates. We derive convergence guarantees for the primal iterates, proving convergence to an $ε$-first-order optimal solution within $\mathcal{O}(ε^{-3})$ evaluations of the problem's functions and their first derivatives. Moreover, when the problem exhibits for example a semialgebraic property, we derive improved local convergence results. Finally, we validate the theoretical findings and assess the practical performance of our proposed algorithm through numerical comparisons with existing state-of-the-art methods.

math.OC↗

arXiv2025

Convergence rates for an inexact linearized ADMM for nonsmooth nonconvex optimization with nonlinear equality constraints

In this paper, we consider nonconvex optimization problems with nonsmooth nonconvex objective function and nonlinear equality constraints. We assume that both the objective function and the functional constraints can be separated into 2 blocks. To solve this problem, we introduce a new inexact linearized alternating direction method of multipliers (ADMM) algorithm. Specifically, at each iteration, we linearize the smooth part of the objective function and the nonlinear part of the functional constraints within the augmented Lagrangian and add a dynamic quadratic regularization. We then compute the new iterate of the block associated with nonlinear constraints inexactly. This strategy yields subproblems that are easily solvable and their (inexact) solutions become the next iterates. Using Lyapunov arguments, we establish convergence guarantees for the iterates of our method toward an $ε$-first-order solution within $\mathcal{O}(ε^{-2})$ iterations. Moreover, we demonstrate that in cases where the problem data exhibit e.g., semi-algebraic properties or more general the KL condition, the entire sequence generated by our algorithm converges, and we provide convergence rates. To validate both the theory and the performance of our algorithm, we conduct numerical simulations for several nonlinear model predictive control and matrix factorization problems.

math.OC↗

arXiv2024

Complexity of linearized quadratic penalty for optimization with nonlinear equality constraints

In this paper we consider a nonconvex optimization problem with nonlinear equality constraints. We assume that both, the objective function and the functional constraints, are locally smooth. For solving this problem, we propose a linearized quadratic penalty method, i.e., we linearize the objective function and the functional constraints in the penalty formulation at the current iterate and add a quadratic regularization, thus yielding a subproblem that is easy to solve, and whose solution is the next iterate. Under a new adaptive regularization parameter choice, we provide convergence guarantees for the iterates of this method to an $ε$ first-order optimal solution in $\mathcal{O}({ε^{-2.5}})$ iterations. Finally, we show that when the problem data satisfy Kurdyka-Lojasiewicz property, e.g., are semialgebraic, the whole sequence generated by the proposed algorithm converges and we derive improved local convergence rates depending on the KL parameter. We validate the theory and the performance of the proposed algorithm by numerically comparing it with some existing methods from the literature.

math.OC↗