arXiv subjects

Ion Necoara

Publications and source records attributed to Ion Necoara.

At least 19 recordsLinked to original sources

arXiv2026

Distributed Fast Fixed-Point Algorithms for Composite Monotone Inclusions over Networks

This paper aims to develop new and efficient distributed algorithms for solving a class of monotone inclusions, $0 \in \sum_{i=1}^n (G_ix + T_ix)$, over a connected network of $n$ agents, where the single-valued operator $G_i$ and the possibly multivalued operator $T_i$ remain private to agent $i$. Existing distributed algorithms for this problem class are primarily non-accelerated, and their exact convergence rates in the original primal space are largely unexplored. To bridge this gap, we propose two Decentralized Fast Fixed-Point-based algorithms, \texttt{ND-DFFP} and \texttt{NI-DFFP}, which integrate Nesterov-type acceleration with primal-dual techniques under two prominent settings: (i) \textit{Lipschitz continuity of $G_i$ and maximal monotonicity of $G_i+T_i$}; and (ii) \textit{co-coercivity of $G_i$ and maximal monotonicity of $T_i$}. While \texttt{ND-DFFP} utilizes a homogeneous network-dependent stepsize, \texttt{NI-DFFP} reformulates the problem into a three-operator inclusion to decouple the network topology, enabling heterogeneous network-independent stepsizes. Under appropriate assumptions, we establish an $\mathcal{O}(1/k)$ convergence rate for the consensus error and an $\mathcal{O}(1/k)$ rate for both the restricted gap function and the squared forward-backward splitting residual, with the latter two metrics evaluated at the network-average iterate or its projection onto the effective domain. Finally, numerical experiments on distributed bilinear matrix games and a virtual power plant problem demonstrate the competitive performance and computational efficiency of our methods over recent decentralized baselines in the literature.

math.OC↗

arXiv2026

Regularized coordinate minimization for nonconvex composite optimization with application to quantized image compression

This paper presents a regularized cyclic coordinate minimization method for solving nonconvex composite optimization problems having the objective function formed as the sum of two terms, one is twice continuously differentiable and the second term is simple and separable. We analyze the convergence behaviour of our coordinate minimization method, in particular we provide convergence rates to a first-order optimality criterion and objective residual depending on the assumptions on the problem. Then, we show that our algorithmic framework can be efficiently applied for solving quantized matrix factorization problems that arise in e.g., lossy image compression. More specifically, on KODAK and CLIC 2024 datasets, our method notably outperforms JPEG at low bit rates, achieving savings in bits per pixel without excessive degradation and remains comparable at higher bit rates. Moreover, we evaluate both the original and quantized compressed images in a classification task using 3 well-known convolutional networks AlexNet, ResNet50 and MobileNetV2 based on floating point and integer arithmetic representations. Remarkably, when comparing the two numerical representations of the convolutional networks, there is no substantially accuracy degradation on ImageNet dataset. The numerical results on image compression and classification using real data show the flexibility and efficiency of our algorithm compared to well-established methods from the literature.

math.OC↗

arXiv2026

On the Worst-Case Analysis of Cyclic Block Coordinate Descent type Algorithms

We study the worst-case behavior of Block Coordinate Descent (BCD) type algorithms for unconstrained minimization of coordinate-wise smooth convex functions. This behavior is indeed not completely understood, and the practical success of these algorithms is not fully explained by current convergence analyses. We extend the recently proposed Performance Estimation Problem (PEP) approach to convex coordinate-wise smooth functions by proposing necessary interpolation conditions. We then exploit this to obtain improved numerical upper bounds on the worst-case convergence rate of three different BCD algorithms, namely Cyclic Coordinate Descent (CCD), Alternating Minimization (AM), and a Cyclic version of the Random Accelerated Coordinate Descent introduced in Fercoq and Richtárik (2015) (CACD), substantially outperforming the best current bounds in some situations. In addition, we show the convergence of the CCD algorithm with more natural assumptions in the context of convex optimization than those typically made in the literature. Our methodology uncovers a number of phenomena, some of which can be formally established. These include a scale-invariance property of the worst case of CCD with respect to the coordinate-wise smoothness constants and a lower bound on the worst-case performance of CCD which is equal to the number of blocks times the worst-case of full gradient descent over the class of smooth convex functions. We also adapt our framework to the analysis of random BCD algorithms, and present numerical results showing that the standard acceleration scheme in Fercoq and Richtárik (2015) appears to be inefficient for deterministic algorithms.

math.OC↗

arXiv2026

Hyperspectral Unmixing Hierarchies

Unmixing reveals the spatial distribution and spectral details of different constituents, called endmembers, in a hyperspectral image. Because unmixing has limited ground truth requirements, can accommodate mixed pixels, and is closely tied to light propagation, it is a uniquely powerful tool for analyzing hyperspectral images. However, spectral variability inhibits unmixing performance, the proper way to determine the number of endmembers is ambiguous, and the clarity of the endmembers degrades as more are included. Hierarchical structure is a possible solution to all three problems. Here, hierarchical unmixing is defined by imposing a hierarchical abundance sum constraint on Deep Nonnegative Matrix Factorization. Binary Linear Unmixing Tactile Hierarchies (BLUTHs) solve the hierarchical unmixing problem with a simple network architecture. Sparsity modulation unmixing growth tailors the topology of a BLUTH to each scene. The structure imposed by BLUTHs allows endmembers with varying levels of spectral contrast to be revealed, mitigating the challenge of spectral variability. The performance of BLUTHs exceeds state-of-the-art unmixing algorithms on laboratory scenes, particularly with regard to abundance estimation, while their performance remains competitive on remote sensing scenes. In addition, ocean color unmixing by BLUTHs is demonstrated on hyperspectral scenes from the HYPSO and PACE satellites.

cs.CV↗

arXiv2025

Complexity of a linearized augmented Lagrangian method for nonconvex minimization with nonlinear equality constraints

In this paper, we consider a nonconvex optimization problem with nonlinear equality constraints. We assume that both, the objective function and the functional constraints are locally smooth. For solving this problem, we propose a linearized augmented Lagrangian method, i.e., we linearize the objective function and the functional constraints in a Gauss-Newton fashion at the current iterate within the augmented Lagrangian function and add a quadratic regularization, yielding a subproblem that is easy to solve, and whose solution is the next primal iterate. The update of the dual multipliers is also based on the linearization of functional constraints. Under a novel dynamic regularization parameter choice, we prove boundedness and global asymptotic convergence of the iterates to a first-order solution of the problem. We also derive convergence guarantees for the iterates of our method to an $ε$-first-order solution in $\mathcal{O}(\sqrtρ ε^{-2})$ Jacobian evaluations, where $ρ$ is the penalty parameter. Moreover, when the problem exhibits a benign nonconvex property, we derive improved convergence results to an $ε$-second-order solution. Finally, we validate the performance of the proposed algorithm by numerically comparing it with the existing methods and software from the literature.

math.OC↗

arXiv2025

Coordinate projected gradient descent minimization and its application to orthogonal nonnegative matrix factorization

In this paper we consider large-scale composite nonconvex optimization problems having the objective function formed as a sum of three terms, first has block coordinate-wise Lipschitz continuous gradient, second is twice differentiable but nonseparable and third is the indicator function of some separable closed convex set. Under these general settings we derive and analyze a new cyclic coordinate descent method, which uses the partial gradient of the differentiable part of the objective, yielding a coordinate gradient descent scheme with a novel adaptive stepsize rule. We prove that this stepsize rule makes the coordinate gradient scheme a descent method, provided that additional assumptions hold for the second term in the objective function. We also present a worst-case complexity analysis for this new method in the nonconvex settings. Numerical results on orthogonal nonnegative matrix factorization problem also confirm the efficiency of our algorithm.

math.OC↗

arXiv2025

Modified projected Gauss-Newton method for constrained nonlinear least-squares: application to power flow analysis

In this paper, we consider a modified projected Gauss-Newton method for solving constrained nonlinear least-squares problems. We assume that the functional constraints are smooth and the the other constraints are represented by a simple closed convex set. We formulate the nonlinear least-squares problem as an optimization problem using the Euclidean norm as a merit function. In our method, at each iteration we linearize the functional constraints inside the merit function at the current point and add a quadratic regularization, yielding a strongly convex subproblem that is easy to solve, whose solution is the next iterate. We present global convergence guarantees for the proposed method under mild assumptions. In particular, we prove stationary point convergence guarantees and under Kurdyka-Lojasiewicz (KL) property for the objective function we derive convergence rates depending on the KL parameter. Finally, we show the efficiency of this method on the power flow analysis problem using several IEEE bus test cases.

math.OC↗

arXiv2025

An accelerated randomized Bregman-Kaczmarz method for strongly convex linearly constraint optimization

In this paper, we propose a randomized accelerated method for the minimization of a strongly convex function under linear constraints. The method is of Kaczmarz-type, i.e. it only uses a single linear equation in each iteration. To obtain acceleration we build on the fact that the Kaczmarz method is dual to a coordinate descent method. We use a recently proposed acceleration method for the randomized coordinate descent and transfer it to the primal space. This method inherits many of the attractive features of the accelerated coordinate descent method, including its worst-case convergence rates. A theoretical analysis of the convergence of the proposed method is given. Numerical experiments show that the proposed method is more efficient and faster than the existing methods for solving the same problem.

math.OC↗

arXiv2025

A stochastic perturbed augmented Lagrangian method for smooth convex constrained minimization

This paper considers smooth convex optimization problems with many functional constraints. To solve this general class of problems we propose a new stochastic perturbed augmented Lagrangian method, called SGDPA, where a perturbation is introduced in the augmented Lagrangian function by multiplying the dual variables with a subunitary parameter. Essentially, we linearize the objective and one randomly chosen functional constraint within the perturbed augmented Lagrangian at the current iterate and add a quadratic regularization that leads to a stochastic gradient descent update for the primal variables, followed by a perturbed random coordinate ascent step to update the dual variables. We provide a convergence analysis in both optimality and feasibility criteria for the iterates of SGDPA algorithm using basic assumptions on the problem. In particular, when the dual updates are assumed to be bounded, we prove sublinear rates of convergence for the iterates of algorithm SGDPA of order $\mathcal{O} (k^{-1/2})$ when the objective is convex and of order $\mathcal{O} (k^{-1})$ when the objective is strongly convex, where $k$ is the iteration counter. Under some additional assumptions, we prove that the dual iterates are bounded and in this case we obtain convergence rates of order $\mathcal{O} (k^{-1/4})$ and $\mathcal{O} (k^{-1/2})$ when the objective is convex and strongly convex, respectively. Preliminary numerical experiments on problems with many quadratic constraints demonstrate the viability and performance of our method when compared to some existing state-of-the-art optimization methods and software.

math.OC↗

arXiv2025

Convergence analysis of linearized $\ell_q$ penalty methods for nonconvex optimization with nonlinear equality constraints

In this paper, we consider nonconvex optimization problems with nonlinear equality constraints. We assume that the objective function and the functional constraints are locally smooth. To solve this problem, we introduce a linearized $\ell_q$ penalty based method, where $q \in (1,2]$ is the parameter defining the norm used in the construction of the penalty function. Our method involves linearizing the objective function and functional constraints in a Gauss-Newton fashion at the current iteration in the penalty formulation and introduces a quadratic regularization. This approach yields an easily solvable subproblem, whose solution becomes the next iterate. By using a novel dynamic rule for the choice of the regularization parameter, we establish that the iterates of our method converge to an $ε$-first-order solution in $\mathcal{O}(1/{ε^{2+ (q-1)/q}})$ outer iterations. Finally, we put theory into practice and evaluate the performance of the proposed algorithm by making numerical comparisons with existing methods from literature.

math.OC↗

arXiv2025

Regularized higher-order Taylor approximation methods for nonlinear least-squares

In this paper, we develop a regularized higher-order Taylor based method for solving composite (e.g., nonlinear least-squares) problems. At each iteration, we replace each smooth component of the objective function by a higher-order Taylor approximation with an appropriate regularization, leading to a regularized higher-order Taylor approximation (RHOTA) algorithm. We derive global convergence guarantees for RHOTA algorithm. In particular, we prove stationary point convergence guarantees for the iterates generated by RHOTA, and leveraging a Kurdyka-Łojasiewicz (KL) type property of the objective function, we derive improved rates depending on the KL parameter. When the Taylor approximation is of order $2$, we present an efficient implementation of RHOTA algorithm, demonstrating that the resulting nonconvex subproblem can be effectively solved utilizing standard convex programming tools. Furthermore, we extend the scope of our investigation to include the behavior and efficacy of RHOTA algorithm in handling systems of nonlinear equations and optimization problems with nonlinear equality constraints deriving new rates under improved constraint qualifications conditions. Finally, we consider solving the phase retrieval problem with a higher-order proximal point algorithm, showcasing its rapid convergence rate for this particular application. Numerical simulations on phase retrieval and output feedback control problems also demonstrate the efficacy and performance of the proposed methods when compared to some state-of-the-art optimization methods and software.

math.OC↗

arXiv2025

A systematic approach to general higher-order majorization-minimization algorithms for (non)convex optimization

Majorization-minimization algorithms consist of successively minimizing a sequence of upper bounds of the objective function so that along the iterations the objective function decreases. Such a simple principle allows to solve a large class of optimization problems, even nonconvex and nonsmooth. We propose a general higher-order majorization-minimization algorithmic framework for minimizing an objective function that admits an approximation (surrogate) such that the corresponding error function has a higher-order Lipschitz continuous derivative. We present convergence guarantees for our new method for general optimization problems with (non)convex and/or (non)smooth objective function. For convex (possibly nonsmooth) problems we provide global sublinear convergence rates, while for problems with uniformly convex objective function we obtain locally faster superlinear convergence rates. We also prove global stationary point guarantees for general nonconvex (possibly nonsmooth) problems and under Kurdyka-Lojasiewicz property of the objective function we derive local convergence rates ranging from sublinear to superlinear for our majorization-minimization algorithm. Moreover, for unconstrained nonconvex problems we derive convergence rates in terms of first- and second-order optimality conditions.

math.OC↗

arXiv2025

Moving higher-order Taylor approximations method for smooth constrained minimization problems

In this paper we develop a higher-order method for solving composite (non)convex minimization problems with smooth (non)convex functional constraints. At each iteration our method approximates the smooth part of the objective function and of the constraints by higher-order Taylor approximations, leading to a moving Taylor approximation method (MTA). We present convergence guarantees for MTA algorithm for both, nonconvex and convex problems. In particular, when the objective and the constraints are nonconvex functions, we prove that the sequence generated by MTA algorithm converges globally to a KKT point. Moreover, we derive convergence rates in the iterates when the problem data satisfy the Kurdyka-Lojasiewicz (KL) property. Further, when the objective function is (uniformly) convex and the constraints are also convex, we provide (linear/superlinear) sublinear convergence rates for our algorithm. Finally, we present an efficient implementation of the proposed algorithm and compare it with existing methods from the literature.

math.OC↗

arXiv2025

An adaptive forward-backward-forward splitting algorithm for solving pseudo-monotone inclusions

In this paper, we propose an adaptive forward-backward-forward splitting algorithm for finding a zero of a pseudo-monotone operator which is split as a sum of three operators: the first is continuous single-valued, the second is Lipschitzian, and the third is maximally monotone. This setting covers, in particular, constrained minimization scenarios, such as problems having smooth and convex functional constraints (e.g., quadratically constrained quadratic programs) or problems with a pseudo-convex objective function minimized over a simple closed convex set (e.g., quadratic over linear fractional programs). For the general problem, we design a forward-backward-forward splitting type method based on novel adaptive stepsize strategies. Under an additional generalized Lipschitz property of the first operator, sublinear convergence rate is derived for the sequence generated by our adaptive algorithm. Moreover, if the sum is uniformly pseudo-monotone, linear/sublinear rates are derived depending on the parameter of uniform pseudo-monotonicity. Preliminary numerical experiments demonstrate the good performance of our method when compared to some existing optimization methods and software.

math.OC↗

arXiv2025

Efficiency of higher-order algorithms for minimizing composite functions

Composite minimization involves a collection of functions which are aggregated in a nonsmooth manner. It covers, as a particular case, smooth approximation of minimax games, minimization of max-type functions, and simple composite minimization problems, where the objective function has a nonsmooth component. We design a higher-order majorization algorithmic framework for fully composite problems (possibly nonconvex). Our framework replaces each component with a higher-order surrogate such that the corresponding error function has a higher-order Lipschitz continuous derivative. We present convergence guarantees for our method for composite optimization problems with (non)convex and (non)smooth objective function. In particular, we prove stationary point convergence guarantees for general nonconvex (possibly nonsmooth) problems and under Kurdyka-Lojasiewicz (KL) property of the objective function we derive improved rates depending on the KL parameter. For convex (possibly nonsmooth) problems we also provide sublinear convergence rates.

math.OC↗

arXiv2025

Complexity of Linearized Perturbed Augmented Lagrangian Methods for Nonsmooth Nonconvex Optimization with Nonlinear Equality Constraints

This paper addresses a class of general nonsmooth and nonconvex composite optimization problems subject to nonlinear equality constraints. We assume that a part of the objective function and the functional constraints exhibit local smoothness. To tackle this challenging class of problems, we propose a novel linearized perturbed augmented Lagrangian method. This method incorporates a perturbation in the augmented Lagrangian function by scaling the dual variable with a sub-unitary parameter. Furthermore, we linearize the smooth components of the objective and the constraints within the perturbed Lagrangian function at the current iterate, while preserving the nonsmooth components. This approach, inspired by prox-linear (or Gauss-Newton) methods, results in a convex subproblem that is typically easy to solve. The solution of this subproblem then serves as the next primal iterate, followed by a perturbed ascent step to update the dual variables. Under a newly introduced constraint qualification condition, we establish the boundedness of the dual iterates. We derive convergence guarantees for the primal iterates, proving convergence to an $ε$-first-order optimal solution within $\mathcal{O}(ε^{-3})$ evaluations of the problem's functions and their first derivatives. Moreover, when the problem exhibits for example a semialgebraic property, we derive improved local convergence results. Finally, we validate the theoretical findings and assess the practical performance of our proposed algorithm through numerical comparisons with existing state-of-the-art methods.

math.OC↗

arXiv2025

Convergence rates for an inexact linearized ADMM for nonsmooth nonconvex optimization with nonlinear equality constraints

In this paper, we consider nonconvex optimization problems with nonsmooth nonconvex objective function and nonlinear equality constraints. We assume that both the objective function and the functional constraints can be separated into 2 blocks. To solve this problem, we introduce a new inexact linearized alternating direction method of multipliers (ADMM) algorithm. Specifically, at each iteration, we linearize the smooth part of the objective function and the nonlinear part of the functional constraints within the augmented Lagrangian and add a dynamic quadratic regularization. We then compute the new iterate of the block associated with nonlinear constraints inexactly. This strategy yields subproblems that are easily solvable and their (inexact) solutions become the next iterates. Using Lyapunov arguments, we establish convergence guarantees for the iterates of our method toward an $ε$-first-order solution within $\mathcal{O}(ε^{-2})$ iterations. Moreover, we demonstrate that in cases where the problem data exhibit e.g., semi-algebraic properties or more general the KL condition, the entire sequence generated by our algorithm converges, and we provide convergence rates. To validate both the theory and the performance of our algorithm, we conduct numerical simulations for several nonlinear model predictive control and matrix factorization problems.

math.OC↗

arXiv2024

Stochastic halfspace approximation method for convex optimization with nonsmooth functional constraints

In this work, we consider convex optimization problems with smooth objective function and nonsmooth functional constraints. We propose a new stochastic gradient algorithm, called Stochastic Halfspace Approximation Method (SHAM), to solve this problem, where at each iteration we first take a gradient step for the objective function and then we perform a projection step onto one halfspace approximation of a randomly chosen constraint. We propose various strategies to create this stochastic halfspace approximation and we provide a unified convergence analysis that yields new convergence rates for SHAM algorithm in both optimality and feasibility criteria evaluated at some average point. In particular, we derive convergence rates of order $\mathcal{O} (1/\sqrt{k})$, when the objective function is only convex, and $\mathcal{O} (1/k)$ when the objective function is strongly convex. The efficiency of SHAM is illustrated through detailed numerical simulations.

math.OC↗