arXiv subjects

Taha El Bakkali

Publications and source records attributed to Taha El Bakkali.

3 recordsLinked to original sources

arXiv2026

Noisy Pairwise-Comparison Random Search for Smooth Nonconvex Optimization

We study smooth nonconvex optimization using only noisy pairwise comparisons, without access to gradients or function values. We propose Noisy-Comparison Random Search (NCRS), a simple direct-search method that samples random directions and performs accept/reject updates from comparison feedback. Under a low-dimensional active-subspace structure, NCRS adapts to the intrinsic dimension $k\le d$ rather than the ambient dimension $d$. For a uniform-margin comparison oracle with advantage $p$, NCRS achieves $ε$-first-order stationarity with comparison complexity $\mathcal{O}(k/(p^2ε^2))$. We also introduce a gap-dependent confidence model, where comparison reliability decreases as the objective-value gap between the two candidates becomes small, and analyze a confidence-weighted voting variant of NCRS. For this oracle, the method achieves $ε$-first-order stationarity with total comparison complexity $\mathcal{O}(k^2/ε^4)$. These results provide intrinsic-dimension convergence guarantees for noisy comparison-based random search in smooth nonconvex optimization.

math.OC↗

arXiv2026

Stochastic Zeroth-Order Optimization Under Heavy-Tailed Noise

We study stochastic zeroth-order (ZO) optimization of smooth nonconvex objectives under heavy-tailed sample-gradient noise. This regime is motivated by empirical evidence that gradient noise in modern machine learning can violate the bounded-variance assumptions used in classical ZO theory. While first-order methods have optimal rates under bounded $p$-th moment noise for $p\in(1,2]$, analogous high-probability guarantees for nonconvex ZO methods are much less understood. The ZO setting is not a direct corollary of first-order theory. First-order methods observe stochastic gradients, whereas derivative-free methods only query noisy function values and build finite-difference estimates. Thus, weak-$L_p$ control of $\nabla F(x;ξ)-\nabla f(x)$ must first be transferred to scalar directional estimates. We propose the Robust Scalar-Clipped Zeroth-Order method (RSC-ZO), a two-point method that clips each scalar directional derivative before aggregation. Under sample-wise smoothness and a weak-$L_p$ tail condition on the sample-gradient noise, RSC-ZO finds an $\varepsilon$-stationary point with high probability using $$ \widetilde{O}\!\left( d^{\frac{p}{2(p-1)}}\varepsilon^{-\frac{3p-2}{p-1}} \right) $$ noisy function evaluations. This matches the optimal first-order $\varepsilon$-dependence. At $p=2$, the bound becomes $\widetilde{O}(d\varepsilon^{-4})$, matching the classical stochastic ZO dimension--accuracy dependence, but with a high-probability guarantee and under a weaker weak-$L_2$ condition that can allow infinite variance. We also analyze a momentum variant and quantify its batch-size/stepsize tradeoff.

math.OC↗

arXiv2026

Nonsmooth Optimization with Zeroth Order Comparison Feedback

We study unconstrained optimization problems of nonsmooth, nonconvex Lipschitz functions, using only noisy pairwise comparisons governed by a known link function. Our goal is to compute a $(δ,\varepsilon)$-Goldstein stationary point. We combine randomized smoothing with a novel unbiased reduction from comparisons to local value differences. By leveraging a Russian-roulette truncation on the Bernoulli-product expansion of the inverse link, we construct an exactly unbiased estimator for directional differences. This estimator has finite expected cost and variance scaling quadratically with the function gap, $\mathcal{O}(B^2)$, under mild conditions. Plugging this into the smoothed gradient identity enables a standard nonconvex SGD analysis, yielding explicit comparison-complexity bounds for common symmetric links such as logistic, probit, and cauchit.

math.OC↗