Complexity analysis of trust-region methods under $(α,L_0,L_1)$-smoothness

Abstract

Generalized smoothness assumptions have attracted growing attention in recent years, motivated in part by machine learning problems in which the gradient of the objective function may not be Lipschitz continuous. Among the most prominent of these is the $(α, L_0, L_1)$-smoothness assumption. Existing methods that attain the best known complexity bounds require knowledge of, or upper bounds on, $α$, $L_0$ and $L_1$, while the few parameter-agnostic methods available do not recover those bounds. In this paper, we show that trust-region methods attain the best known bounds for $(α, L_0, L_1)$-smooth objective functions without prior knowledge of these parameters. Establishing these results requires new analytical tools, beyond those used in classical trust-region complexity analyses. We further show that our working model assumption allows the use of general model Hessian approximations and, in particular, accommodates limited-memory quasi-Newton updates. Finally, we show that our complexity bound is sharp in the nonconvex setting.