arXiv subjects

Saman Babaie-Kafaki

Publications and source records attributed to Saman Babaie-Kafaki.

3 recordsLinked to original sources

arXiv2026

A family of spectral conjugate gradient algorithms derived by least-squares approximations based on a modified quasi--Newton update with application to a revised robust binary classification model

We develop a spectral three-term modification of the classic Hestenes--Stiefel conjugate gradient algorithm, preserving its anti-jamming characteristic and, simultaneously, taking care of the sufficient descent property. We discuss how a modified secant equation can be extracted from our modification scheme, yielding a memoryless BFGS updating formula. Then, the spectral parameter of our method is obtained by steering its direction toward the given BFGS direction within a least-squares context. Using our technical improvements, we outline the general framework of our algorithm and discuss its theoretical features, including the descent and convergence properties, without the convexity assumption. We put our algorithm to the test in comparison with the three other conjugate gradient algorithms on a set of CUTEr unconstrained optimization test models, comparing the outputs using the Dolan--More measure. Next, we provide a concise evaluation of the results, highlighting the practical advantages of our algorithm. As a real-world case study, we introduce a reduced quadratic surface SVM with the rescaled loss for robust binary classification and apply the proposed algorithm to assess its accuracy and training time against several other SVMs.

math.OC↗

arXiv2026

An Approximate Conjugate Subgradient Algorithm with Matrix Parameter for Derivative-Free Nonsmooth Optimization Problems

We propose a derivative-free matrix conjugate-subgradient method for unconstrained nonsmooth optimization of locally Lipschitz functions. The method constructs discrete gradients using only function values and forms a finite sampled model of the Goldstein subdifferential. A minimal-norm element of the convex hull of the sampled discrete gradients is then computed and used both as a stationarity measure and as the reference vector for generating descent-oriented directions. To improve robustness beyond the basic steepest-descent direction, we introduce a matrix memory correction together with coefficient damping, diagonal scaling, bounded-angle correction, and matrix-stability safeguards. A two-point line-search procedure with enrichment is used to obtain either a serious step or an improved local model. Under suitable consistency assumptions on the discrete-gradient approximation and line-search sampling, the method generates directions satisfying a safeguarded descent property and computes approximate Goldstein stationary points. Numerical experiments on nonsmooth test problems with dimensions up to 1000 show that both proposed variants are robust for lower and medium accuracy requirements, while the matrix conjugate-subgradient variant remains the most reliable under the strictest tolerance.

math.OC↗

arXiv2026

An efficient penalty decomposition algorithm for minimization over sparse symmetric sets

This paper proposes an improved quasi-Newton penalty decomposition algorithm for the minimization of continuously differentiable functions, possibly nonconvex, over sparse symmetric sets. The method solves a sequence of penalty subproblems approximately via a two-block decomposition scheme: the first subproblem admits a closed-form solution without sparsity constraints, while the second subproblem is handled through an efficient sparse projection over the symmetric feasible set. Under a new assumption on the gradient of the objective function, weaker than global Lipschitz continuity from the origin, we establish that accumulation points of the outer iterates are basic feasible and cardinality-constrained Mordukhovich stationarity points. To ensure robustness and efficiency in finite-precision arithmetic, the algorithm incorporates several practical enhancements, including an enhanced line search strategy based on either backtracking or extrapolation, and four inexpensive diagonal Hessian approximations derived from differences of previous iterates and gradients or from eigenvalue-distribution information. Numerical experiments on a diverse benchmark of $30$ synthetic and data-driven test problems, including machine-learning datasets from the UCI repository and sparse symmetric instances with dimensions ranging from $10$ to $500$, demonstrate that the proposed algorithm is competitive with several state-of-the-art methods in terms of efficiency, robustness, and strong stationarity.

math.OC↗