arXiv subjects

Saman Babaie--Kafaki

Publications and source records attributed to Saman Babaie--Kafaki.

2 recordsLinked to original sources

arXiv2026

From a Scalar to a Matrix Setting for the Dai--Liao Parameter

As is well known, both the numerical performance and the theoretical properties of the Dai--Liao conjugate gradient algorithm are highly dependent on the adjustment of its key parameter. Here, we first employ the well-known Harmonic--Geometric--Arithmetic--Quadratic mean inequality to reinforce the optimality of two previously proposed scalar adaptive settings of the Dai--Liao parameter. In other words, we show how these parameter choices are capable of enhancing the well-conditioning of the Dai--Liao search direction matrix by shrinking the intervals containing its singular values. A similar analysis is also carried out for scaled memoryless quasi--Newton updating formulas in order to further justify the optimality of two classical scaling parameters associated with these updates. Then, as the main contribution of this work, we move from the classical scalar setting of the Dai--Liao parameter to a matrix setting aimed at enhancing flexibility and diversity within optimization methods. In particular, we show that, under such a matrix formulation of the parameter, a well-known three-term conjugate gradient algorithm emerges as a member of the proposed extended Dai--Liao class of algorithms while enjoying several computationally attractive properties. Among these, the well-conditioning of the associated search direction matrix is especially noteworthy, as well as the ability to make more explicit use of the second-order information of the model. Finally, to provide practical evidence supporting the proposed matrix setting of the Dai--Liao parameter, we conduct a series of numerical experiments on standard benchmark test problems and report the results in detail. Generally speaking, the proposed framework is shown to retain both theoretical soundness and computational reliability.

math.OC↗

arXiv2026

Diagonal Hessian Approximation Based on Conjugacy Condition for Noisy Derivative-Free Optimization Problems in High Dimensions

We consider large-scale noisy derivative-free optimization (DFO) problems in which only function values are available and gradient or subgradient information cannot be reliably estimated. Matrix-adaptation evolution strategies (MAES) and their limited-memory variants are among the most robust DFO methods under noise; however, their performance may deteriorate when the noise level is large. In such regimes, sorting and selection may misidentify informative sampled points, making the recombination step less reliable and weakening the scaling information used by affine or matrix-adaptation mechanisms. This can substantially reduce the efficiency of MAES-type methods, especially in high-dimensional settings. To address this limitation, we propose a DFO method that replaces the full affine-scaling matrix with a diagonal approximation constructed from conjugacy-type conditions. The proposed mechanism does not attempt to estimate gradients, subgradients, or interpolation models, nor does it learn dense covariance information from noisy rankings. Instead, it uses consecutive normalized recombination displacements in a conservative diagonal update, thereby limiting the influence of unreliable selection information while preserving the derivative-free structure of the underlying evolutionary framework. As a result, the method is computationally cheaper than full matrix-adaptation schemes and limited-memory affine-scaling variants, while providing a stable scaling mechanism in noisy environments. Numerical experiments on noisy benchmark problems show that the proposed method is competitive with, and often more efficient than, MAES-type baselines, particularly when the noise level is large and ranking-based selection becomes unreliable.

math.OC↗