A counterexample to a three-step convergence bound for BFGS with exact line search

Jiawei Xu; Jiang Hu

Home/Archive/arXiv

arXiv · 2611.02719

A counterexample to a three-step convergence bound for BFGS with exact line search

Jiawei Xu·Jiang Hu

View original source ↗Read source document ↗Export metadata ↓

Abstract

We disprove a three-step error estimate for the Broyden--Fletcher--Goldfarb--Shanno {(BFGS)} method with exact line searches. We construct a globally strongly convex function $F\in C^\infty(\mathbb R^3)$ and a single nonterminating trajectory $x_k\to0$ for which $\limsup_{k\to\infty}\|x_{k+3}\|/(\|x_{k+2}\|\|x_k\|)=+\infty$. The Hessian of $F$ lies between $\tfrac12 I$ and $\tfrac32 I$ everywhere, and $F(x)-\tfrac12\|x\|^2$ is compactly supported with all derivatives vanishing at $0$. The proof first constructs point-gradient pairs satisfying the exact BFGS step identities. A five-step cycle produces a large three-step quotient and restores an invariant plane for the actual updated matrix, allowing the cycle to be repeated with arbitrarily small gradient errors. Disjoint smooth corrections to a quadratic then realize all the pairs by one objective. Strong convexity identifies the prescribed points with the unique exact line-search minimizers. Thus smoothness and strong convexity do not imply the proposed product estimate, even for an objective that agrees with a quadratic to every order at its minimizer.

Explore related subjects

math.OC

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Jiawei Xu, Jiang Hu. 2026-11-02. A counterexample to a three-step convergence bound for BFGS with exact line search. https://arxiv.org/abs/2611.02719

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Common Noise by Random Measures: Constructing Mean-Field Equilibria for Competitive Investment and Hedging

We construct Nash equilibria in mean-field portfolio games of optimal investment and hedging under relative performance concerns with exponential (CARA) utility preferences. Common noise dynamics are modeled by integer-valued random measures, for instance Poisson random measures, in addition to Brownian motions. Agents differ in individual risk aversions, competition weights, and initial capital endowments, while their contingent claim liabilities depend on both common and idiosyncratic risk factors. Mean-field equilibria are characterized by solutions to McKean-Vlasov backward stochastic differential equations with jumps, for which we prove existence and uniqueness of solutions, without assuming mean-field interaction to be small. Moreover, we show how the equilibrium can be constructed from the optimal strategy of a single-agent optimization problem (without mean-field interaction) via an appropriate projection. Using successive changes of measure, our derivation provides a decomposition of the equilibrium strategy into three components with clear interpretations. Finally, we show how a limiting mean-field game of quadratic (instead of utility-based) hedging with relative performance concerns arises for vanishing risk aversion.

math.OC↗

arXiv2026

Riemannian EXTRA: Communication-efficient decentralized optimization over compact submanifolds with data heterogeneity

We consider decentralized optimization over a compact Riemannian submanifold in a network of $n$ nodes, where each node $i$ owns a smooth nonconvex local objective $f_i$ defined by its private data, and the goal is to minimize the global objective $F(x)=\frac{1}{n}\sum_{i=1}^n f_i(x)$ using only local communications. A naive Riemannian extension of EXTRA leaves the auxiliary normal-space error uncontrolled and does not inherit the Euclidean convergence guarantees. We develop projection-based Riemannian EXTRA (REXTRA) and a retraction-based counterpart that incorporate a normal correction into both the communicated variable and the auxiliary update, while using only one local-variable-sized message per optimization iteration. By jointly controlling consensus, tangent, and normal errors, our analysis interprets the update as a network-based Riemannian inexact-gradient step. For projection-based REXTRA, a sufficiently small constant stepsize and a local initialization yield vanishing consensus, stationary accumulation points, and an $\mathcal O(1/k)$ best-iterate stationarity--consensus bound. For the retraction-based method, we prove that its difference from the projection update is uniformly second order and establish the same convergence properties and rate under a local initialization. Numerical experiments on representative manifold-valued learning tasks demonstrate the communication efficiency of the REXTRA updates.

math.OC↗

arXiv2026

Quantification of Tail-trimming Effects for Interval CVaR Based Nonlinear Regression

The tail trimming strategy is widely adopted for robustness against outlier contamination in statistical learning and operations research. However, it generally lacks the thorough understanding that to what extent the tail-trimming technique is effective for nonlinear regression. In this paper, we address the robust nonlinear regression based on the so-called interval conditional value-at-risk (In-CVaR), which is introduced to guard against outliers by trimming extreme losses. We first analyze the trimming effect of the In-CVaR based estimator under distributional perturbation. Our first finding is that the In-CVaR based estimator is qualitatively robust under distributional perturbation in terms of the Prokhorov metric if and only if the largest portion of losses is trimmed. Furthermore, under sample contamination, we derive a probably approximately correct (PAC) bound for the In-CVaR based estimator, showing that the preliminary covariate filtering that is customary in the literature is not needed for In-CVaR based regression. The nonconvexity of the In-CVaR objective, together with unboundedness of covariates, responses and parameter space create the technical challenges for our developed theory, and thus necessitate the new treatment of the qualitative robustness and PAC analysis. Overall, this study rigorously analyzes robustness properties of In-CVaR based nonlinear regression models, which illustrates to what extent the right tail-trimming of the loss by In-CVaR risk measure is essential and effective for robust nonlinear regression. These theoretical findings are further supported by numerical experiments.

math.OC↗