arXiv subjects

Moslem Zamani

Publications and source records attributed to Moslem Zamani.

16 recordsLinked to original sources

arXiv2026

Difference-of-Convex Optimization via Inexact Smoothing Descent Methods: Difference of High-Order Moreau Envelopes

This paper studies difference-of-convex (DC) optimization problems through smoothing descent techniques. In particular, we introduce the difference of high-order Moreau envelopes (HOME-DC) and establish its fundamental and differential properties. Approximating the underlying proximal points, we generate an inexact first-order oracle for HOME-DC and characterize its accuracy guarantees. Building upon this oracle, we propose a class of inexact descent methods for minimizing DC functions and provide a convergence analysis. The proposed framework extends the applicability of envelope-based optimization techniques to a broad class of structured nonconvex problems while accommodating inexact solutions to subproblems. Preliminary numerical experiments on a sparse clustering problem demonstrate the approach's practical potential and support the theoretical findings.

math.OC↗

arXiv2025

On the convergence rate of the Douglas-Rachford splitting algorithm

This work is concerned with the convergence rate analysis of the Douglas-Rachford splitting (DRS) method for finding a zero of the sum of two maximally monotone operators. We obtain an exact rate of convergence for the DRS algorithm and demonstrate its sharpness in the setting of convex feasibility problems. Furthermore, we investigate the linear convergence of the DRS algorithm, providing both necessary and sufficient conditions that characterize this behavior. We further examine the performance of the DRS method when applied to convex composite optimization problems. The paper concludes with several conjectures on the convergence behavior of the DRS algorithm for this class of problems.

math.OC↗

arXiv2025

Are Convex Optimization Curves Convex?

In this paper, we study when we might expect the optimization curve induced by gradient descent to be \emph{convex} -- precluding, for example, an initial plateau followed by a sharp decrease, making it difficult to decide when optimization should stop. Although such undesirable behavior can certainly occur when optimizing general functions, might it also occur in the benign and well-studied case of smooth convex functions? As far as we know, this question has not been tackled in previous work. We show, perhaps surprisingly, that the answer crucially depends on the choice of the step size. In particular, for the range of step sizes which are known to result in monotonic convergence to an optimal value, we characterize a regime where the optimization curve will be provably convex, and a regime where the curve can be non-convex. We also extend our results to gradient flow, and to the closely-related but different question of whether the gradient norm decreases monotonically.

math.OC↗

arXiv2024

Exact Convergence rate of the subgradient method by using Polyak step size

This paper studies the last iterate of subgradient method with Polyak step size when applied to the minimization of a nonsmooth convex function with bounded subgradients. We show that the subgradient method with Polyak step size achieves a convergence rate $\mathcal{O}\left(\tfrac{1}{\sqrt[4]{N}}\right)$ in terms of the final iterate. An example is provided to show that this rate is exact and cannot be improved. We introduce an adaptive Polyak step size for which the subgradient method enjoys a convergence rate $\mathcal{O}\left(\tfrac{1}{\sqrt{N}}\right)$ for the last iterate. Its convergence rate matches exactly the lower bound on the performance of any black-box method on the considered problem class. Additionally, we propose an adaptive Polyak method with a momentum term, where the step sizes are independent of the number of iterates. We establish that the algorithm also attains the optimal convergence rate. We investigate the alternating projection method. We derive a convergence rate $\left( \frac{2N }{ 2N+1 } \right)^N\tfrac{R}{\sqrt{2N+1}}$ for the last iterate, where $R$ is a bound on the distance between the initial iterate and a solution. An example is also provided to illustrate the exactness of the rate.

math.OC↗

arXiv2024

On the Set of Possible Minimizers of a Sum of Convex Functions

Consider a sum of convex functions, where the only information known about each individual summand is the location of a minimizer. In this work, we give an exact characterization of the set of possible minimizers of the sum. Our results cover several types of assumptions on the summands, such as smoothness or strong convexity. Our main tool is the use of necessary and sufficient conditions for interpolating the considered function classes, which leads to shorter and more direct proofs in comparison with previous work. We also address the setting where each summand minimizer is assumed to lie in a unit ball, and prove a tight bound on the norm of any minimizer of the sum.

math.OC↗

arXiv2023

Exact convergence rate of the last iterate in subgradient methods

We study the convergence of the last iterate in subgradient methods applied to the minimization of a nonsmooth convex function with bounded subgradients. We first introduce a proof technique that generalizes the standard analysis of subgradient methods. It is based on tracking the distance between the current iterate and a different reference point at each iteration. Using this technique, we obtain the exact worst-case convergence rate for the objective accuracy of the last iterate of the projected subgradient method with either constant step sizes or constant step lengths. Tightness is shown with a worst-case instance matching the established convergence rate. We also derive the value of the optimal constant step size when performing $N$ iterations, for which we find that the last iterate accuracy is smaller than $B R \sqrt{1+\log(N)/4}/{\sqrt{N+1}}$ %$\frac{B R \log N}{\sqrt{N+1}}$ , where $B$ is a bound on the subgradient norm and $R$ is a bound on the distance between the initial iterate and a minimizer. Finally, we introduce a new optimal subgradient method that achieves the best possible last-iterate accuracy after a given number $N$ of iterations. Its convergence rate ${B R}/{\sqrt{N+1}}$ matches exactly the lower bound on the performance of any black-box method on the considered problem class. We also show that there is no universal sequence of step sizes that simultaneously achieves this optimal rate at each iteration, meaning that the dependence of the step size sequence in $N$ is unavoidable.

math.OC↗

arXiv2023

The exact worst-case convergence rate of the alternating direction method of multipliers

Recently, semidefinite programming performance estimation has been employed as a strong tool for the worst-case performance analysis of first order methods. In this paper, we derive new non-ergodic convergence rates for the alternating direction method of multipliers (ADMM) by using performance estimation. We give some examples which show the exactness of the given bounds. We also study the linear and R-linear convergence of ADMM. We establish that ADMM enjoys a global linear convergence rate if and only if the dual objective satisfies the Polyak-Lojasiewicz (PL)inequality in the presence of strong convexity. In addition, we give an explicit formula for the linear convergence rate factor. Moreover, we study the R-linear convergence of ADMM under two new scenarios.

math.OC↗

arXiv2023

On the rate of convergence of the Difference-of-Convex Algorithm (DCA)

In this paper, we study the convergence rate of the DCA (Difference-of-Convex Algorithm), also known as the convex-concave procedure, with two different termination criteria that are suitable for smooth and nonsmooth decompositions respectively. The DCA is a popular algorithm for difference-of-convex (DC) problems, and known to converge to a stationary point of the objective under some assumptions. We derive a worst-case convergence rate of $O(1/\sqrt{N})$ after $N$ iterations of the objective gradient norm for certain classes of DC problems, without assuming strong convexity in the DC decomposition, and give an example which shows the convergence rate is exact. We also provide a new convergence rate of $O(1/N)$ for the DCA with the second termination criterion. %In addition, we investigate the DCA with regularization. Moreover, we derive a new linear convergence rate result for the DCA under the assumption of the Polyak-Łojasiewicz inequality. The novel aspect of our analysis is that it employs semidefinite programming performance estimation.

math.OC↗

arXiv2022

Convergence rate analysis of randomized and cyclic coordinate descent for convex optimization through semidefinite programming

In this paper, we study randomized and cyclic coordinate descent for convex unconstrained optimization problems. We improve the known convergence rates in some cases by using the numerical semidefinite programming performance estimation method. As a spin-off we provide a method to analyse the worst-case performance of the Gauss-Seidel iterative method for linear systems where the coefficient matrix is positive semidefinite with a positive diagonal.

math.OC↗

arXiv2022

Convergence rate analysis of the gradient descent-ascent method for convex-concave saddle-point problems

In this paper, we study the gradient descent-ascent method for convex-concave saddle-point problems. We derive a new non-asymptotic global convergence rate in terms of distance to the solution set by using the semidefinite programming performance estimation method. The given convergence rate incorporates most parameters of the problem and it is exact for a large class of strongly convex-strongly concave saddle-point problems for one iteration. We also investigate the algorithm without strong convexity and we provide some necessary and sufficient conditions under which the gradient descent-ascent enjoys linear convergence.

math.OC↗

arXiv2022

Conditions for linear convergence of the gradient method for non-convex optimization

In this paper, we derive a new linear convergence rate for the gradient method with fixed step lengths for non-convex smooth optimization problems satisfying the Polyak-Lojasiewicz (PL) inequality. We establish that the PL inequality is a necessary and sufficient condition for linear convergence to the optimal value for this class of problems. We list some related classes of functions for which the gradient method may enjoy linear convergence rate. Moreover, we investigate their relationship with the PL inequality.

math.OC↗

arXiv2021

The exact worst-case convergence rate of the gradient method with fixed step lengths for L-smooth functions

In this paper, we study the convergence rate of the gradient (or steepest descent) method with fixed step lengths for finding a stationary point of an $L$-smooth function. We establish a new convergence rate, and show that the bound may be exact in some cases, in particular when all step lengths lie in the interval $(0,1/L]$. In addition, we derive an optimal step length with respect to the new bound.

math.OC↗

arXiv2020

Error bounds and a condition number for the absolute value equations

Absolute value equations, due to their relation to the linear complementarity problem, have been intensively studied recently. In this paper, we present error bounds for absolute value equations. Along with the error bounds, we introduce an appropriate condition number. We consider general scaled matrix p-norms, as well as particular p-norms. We discuss basic properties of the condition number, its computational complexity, its bounds and also exact values for special classes of matrices. We consider also matrices that appear based on the transformation from the linear complementarity problem.

math.OC↗

arXiv2019

Proper efficiency, scalarization and transformation in multi-objective optimization: Unified approaches

In this paper, we investigate the relationships between proper efficiency and the solutions of a general scalarization problem in multi-objective optimization. We provide some conditions under which the solutions of the dealt with scalar program are properly efficient and vice versa. We also show that, under some conditions, if the considered general scalar problem is unbounded, then the original multi-objective problem does not have any properly efficient solution. In another part of the work, we investigate a general transformation of the objective functions which preserves proper efficiency. We show that several important results existing in the literature are direct consequences of the results of the present paper.

math.OC↗

arXiv2019

New bounds for nonconvex quadratically constrained quadratic programming

In this paper, we study some bounds for nonconvex quadratically constrained quadratic programs. We propose two types of bounds for quadratically constrained quadratic programs, quadratic and cubic bounds. For quadratic bounds, we use affine functions as Lagrange multipliers. We demonstrate that most semi-definite relaxations can be obtained as the dual of a quadratic bound. In addition, we study bounds obtained by changing the ground set. For cubic bounds, in addition to affine multipliers we employ quadratic functions. We provide a comparison between the proposed cubic bound and typical bounds for standard quadratic programs. Moreover, we report comparison results of a quadratic and a cubic bound for some non-convex quadratically constrained quadratic programs.

math.OC↗

arXiv2019

A new dual for quadratic programming and its applications

The main outcomes of the paper are divided into two parts. First, we present a new dual for quadratic programs, in which, the dual variables are affine functions, and we prove strong duality. Since the new dual is intractable, we consider a modified version by restricting the feasible set. This leads to a new bound for quadratic programs. We demonstrate that the dual of the bound is a semi-definite relaxation of quadratic programs. In addition, we probe the relationship between this bound and the well-known bounds. In the second part, thanks to the new bound, we propose a branch and cut algorithm for concave quadratic programs. We establish that the algorithm enjoys global convergence. The effectiveness of the method is illustrated for numerical problem instances.

math.OC↗