SearcharxivSearch

arXiv subjects

Yassine Nabou

Publications and source records attributed to Yassine Nabou.

9 recordsLinked to original sources

Dynamic inverse problems: Single-loop online algorithms

We study efficient online methods for dynamic inverse problems with infinite time horizon. We concentrate, in particular, on problems whose forward model arises from a PDE. Our motivating application is flow monitoring with Electrical Impedance Tomography (EIT). The idea of such online methods is to take single steps of of standard optimisation algorithms, on each time index; each data frame. A predictor, based on problem dynamics, is used to transfer iterates one from time index to the next one. If we monitor a fast flow with a correspondingly fast measurement modality, such as EIT, basic methods are unable to solve the PDE before new data arrives. Our idea, then, is to not solve it, and instead, on each iteration, each time index, take single or few steps of standard iterative solvers towards the solution of both the PDE and an adjoint PDE. This is what ``single loop'' refers to. To the overall problem, we apply standard online optimisation methods, at the outside developed for exact gradients $\nabla E_k(x^k)$ of the iteration-dependent data fidelity $E_k$ that incorporates the PDE. We replace the gradient by a single-loop estimate $\tilde\grad E_k(x^k)$ that satisfies standard smoothness properties with summable errors. This allows standard regret proofs to go through. Our numerical experiments on dynamic EIT validate the theoretical predictions and highlight the potential of the proposed approach for the real-time solution of PDE-constrained dynamic inverse problems.

math.OC

Dynamic inverse problems: Online regularisation theory

We develop regularisation theory for dynamic inverse problems, solved using online methods with an infinite time horizon. Using concepts of subregularity to treat nonsmooth regularisers, we prove that time-averaged reconstruction errors converge to zero as noise, algorithmic errors, and regularisation vanish as the horizon grows. We illustrate the theory numerically with a dynamic electrical impedance tomography example.

math.NA

Monotone and nonmonotone linearized block coordinate descent methods for nonsmooth composite optimization problems

In this paper, we introduce both monotone and nonmonotone variants of LiBCoD, a \textbf{Li}nearized \textbf{B}lock \textbf{Co}ordinate \textbf{D}escent method for solving composite optimization problems. At each iteration, a random block is selected, and the smooth components of the objective are linearized along the chosen block in a Gauss-Newton approach. For the monotone variant, we establish a global sublinear convergence rate to a stationary point under the assumption of bounded iterates. For the nonmonotone variant, we derive a global sublinear convergence rate without requiring global Lipschitz continuity or bounded iterates. Preliminary numerical experiments highlight the promising performance of the proposed approach.

math.OC

Modified projected Gauss-Newton method for constrained nonlinear least-squares: application to power flow analysis

In this paper, we consider a modified projected Gauss-Newton method for solving constrained nonlinear least-squares problems. We assume that the functional constraints are smooth and the the other constraints are represented by a simple closed convex set. We formulate the nonlinear least-squares problem as an optimization problem using the Euclidean norm as a merit function. In our method, at each iteration we linearize the functional constraints inside the merit function at the current point and add a quadratic regularization, yielding a strongly convex subproblem that is easy to solve, whose solution is the next iterate. We present global convergence guarantees for the proposed method under mild assumptions. In particular, we prove stationary point convergence guarantees and under Kurdyka-Lojasiewicz (KL) property for the objective function we derive convergence rates depending on the KL parameter. Finally, we show the efficiency of this method on the power flow analysis problem using several IEEE bus test cases.

math.OC

Regularized higher-order Taylor approximation methods for nonlinear least-squares

In this paper, we develop a regularized higher-order Taylor based method for solving composite (e.g., nonlinear least-squares) problems. At each iteration, we replace each smooth component of the objective function by a higher-order Taylor approximation with an appropriate regularization, leading to a regularized higher-order Taylor approximation (RHOTA) algorithm. We derive global convergence guarantees for RHOTA algorithm. In particular, we prove stationary point convergence guarantees for the iterates generated by RHOTA, and leveraging a Kurdyka-Łojasiewicz (KL) type property of the objective function, we derive improved rates depending on the KL parameter. When the Taylor approximation is of order $2$, we present an efficient implementation of RHOTA algorithm, demonstrating that the resulting nonconvex subproblem can be effectively solved utilizing standard convex programming tools. Furthermore, we extend the scope of our investigation to include the behavior and efficacy of RHOTA algorithm in handling systems of nonlinear equations and optimization problems with nonlinear equality constraints deriving new rates under improved constraint qualifications conditions. Finally, we consider solving the phase retrieval problem with a higher-order proximal point algorithm, showcasing its rapid convergence rate for this particular application. Numerical simulations on phase retrieval and output feedback control problems also demonstrate the efficacy and performance of the proposed methods when compared to some state-of-the-art optimization methods and software.

math.OC

Moving higher-order Taylor approximations method for smooth constrained minimization problems

In this paper we develop a higher-order method for solving composite (non)convex minimization problems with smooth (non)convex functional constraints. At each iteration our method approximates the smooth part of the objective function and of the constraints by higher-order Taylor approximations, leading to a moving Taylor approximation method (MTA). We present convergence guarantees for MTA algorithm for both, nonconvex and convex problems. In particular, when the objective and the constraints are nonconvex functions, we prove that the sequence generated by MTA algorithm converges globally to a KKT point. Moreover, we derive convergence rates in the iterates when the problem data satisfy the Kurdyka-Lojasiewicz (KL) property. Further, when the objective function is (uniformly) convex and the constraints are also convex, we provide (linear/superlinear) sublinear convergence rates for our algorithm. Finally, we present an efficient implementation of the proposed algorithm and compare it with existing methods from the literature.

math.OC

Nonmonotone higher-order Taylor approximation methods for composite problems

We study composite optimization problems in which the smooth part of the objective function is \( p \)-times continuously differentiable, where \( p \geq 1 \) is an integer. Higher-order methods are known to be effective for solving such problems, as they speed up convergence rates. These methods often require, or implicitly ensure, a monotonic decrease in the objective function across iterations. Maintaining this monotonicity typically requires that the \( p \)-th derivative of the smooth part of the objective function is globally Lipschitz or that the generated iterates remain bounded. In this paper, we propose nonmonotone higher-order Taylor approximation (NHOTA) method for composite problems. Our method achieves the same nice global and rate of convergence properties as traditional higher-order methods while eliminating the need for global Lipschitz continuity assumptions, strict descent condition, or explicit boundedness of the iterates. Specifically, for nonconvex composite problems, we derive global convergence rate to a stationary point of order \( \mathcal{O}(k^{-\frac{p}{p+1}}) \), where \( k \) is the iteration counter. Moreover, when the objective function satisfies the Kurdyka-Łojasiewicz (KL) property, we obtain improved rates that depend on the KL parameter. Furthermore, for convex composite problems, our method achieves sublinear convergence rate of order \( \mathcal{O}(k^{-p}) \) in function values. Finally, preliminary numerical experiments on nonconvex phase retrieval problems highlight the promising performance of the proposed approach.

math.OC

Efficiency of higher-order algorithms for minimizing composite functions

Composite minimization involves a collection of functions which are aggregated in a nonsmooth manner. It covers, as a particular case, smooth approximation of minimax games, minimization of max-type functions, and simple composite minimization problems, where the objective function has a nonsmooth component. We design a higher-order majorization algorithmic framework for fully composite problems (possibly nonconvex). Our framework replaces each component with a higher-order surrogate such that the corresponding error function has a higher-order Lipschitz continuous derivative. We present convergence guarantees for our method for composite optimization problems with (non)convex and (non)smooth objective function. In particular, we prove stationary point convergence guarantees for general nonconvex (possibly nonsmooth) problems and under Kurdyka-Lojasiewicz (KL) property of the objective function we derive improved rates depending on the KL parameter. For convex (possibly nonsmooth) problems we also provide sublinear convergence rates.

math.OC

Proximal gradient methods with inexact oracle of degree q for composite optimization

We introduce the concept of inexact first-order oracle of degree q for a possibly nonconvex and nonsmooth function, which naturally appears in the context of approximate gradient, weak level of smoothness and other situations. Our definition is less conservative than those found in the existing literature, and it can be viewed as an interpolation between fully exact and the existing inexact first-order oracle definitions. We analyze the convergence behavior of a (fast) inexact proximal gradient method using such an oracle for solving (non)convex composite minimization problems. We derive complexity estimates and study the dependence between the accuracy of the oracle and the desired accuracy of the gradient or of the objective function. Our results show that better rates can be obtained both theoretically and in numerical simulations when q is large.

math.OC