Searcharxiv⌕ Search

arXiv subjects

Adrian S. Lewis

Publications and source records attributed to Adrian S. Lewis.

33 records · Page 2Linked to original sources
arXiv2018

Rescaling nonsmooth optimization using BFGS and Shor updates

The BFGS quasi-Newton methodology, popular for smooth minimization, has also proved surprisingly effective in nonsmooth optimization. Through a variety of simple examples and computational experiments, we explore how the BFGS matrix update improves the local metric associated with a convex function even in the absence of smoothness and without using a line search. We compare the behavior of the BFGS and Shor r-algorithm updates.

math.OC↗
arXiv2016

Nonsmooth optimization using Taylor-like models: error bounds, convergence, and termination criteria

We consider optimization algorithms that successively minimize simple Taylor-like models of the objective function. Methods of Gauss-Newton type for minimizing the composition of a convex function and a smooth map are common examples. Our main result is an explicit relationship between the step-size of any such algorithm and the slope of the function at a nearby point. Consequently, we (1) show that the step-sizes can be reliably used to terminate the algorithm, (2) prove that as long as the step-sizes tend to zero, every limit point of the iterates is stationary, and (3) show that conditions, akin to classical quadratic growth, imply that the step-sizes linearly bound the distance of the iterates to the solution set. The latter so-called error bound property is typically used to establish linear (or faster) convergence guarantees. Analogous results hold when the step-size is replaced by the square root of the decrease in the model's value. We complete the paper with extensions to when the models are minimized only inexactly.

math.OC↗
arXiv2016

Error bounds, quadratic growth, and linear convergence of proximal methods

The proximal gradient algorithm for minimizing the sum of a smooth and a nonsmooth convex function often converges linearly even without strong convexity. One common reason is that a multiple of the step length at each iteration may linearly bound the "error" -- the distance to the solution set. We explain the observed linear convergence intuitively by proving the equivalence of such an error bound to a natural quadratic growth condition. Our approach generalizes to linear convergence analysis for proximal methods (of Gauss-Newton type) for minimizing compositions of nonsmooth functions with smooth mappings. We observe incidentally that short step-lengths in the algorithm indicate near-stationarity, suggesting a reliable termination criterion.

math.OC↗
arXiv2013

Orthogonal Invariance and Identifiability

Orthogonally invariant functions of symmetric matrices often inherit properties from their diagonal restrictions: von Neumann's theorem on matrix norms is an early example. We discuss the example of "identifiability", a common property of nonsmooth functions associated with the existence of a smooth manifold of approximate critical points. Identifiability (or its synonym, "partial smoothness") is the key idea underlying active set methods in optimization. Polyhedral functions, in particular, are always partly smooth, and hence so are many standard examples from eigenvalue optimization.

math.OC↗
arXiv2012

Clarke subgradients for directionally Lipschitzian stratifiable functions

Using a geometric argument, we show that under a reasonable continuity condition, the Clarke subdifferential of a semi-algebraic (or more generally stratifiable) directionally Lipschitzian function admits a simple form: the normal cone to the domain and limits of gradients generate the entire Clarke subdifferential. The characterization formula we obtain unifies various apparently disparate results that have appeared in the literature. Our techniques also yield a simplified proof that closed semialgebraic functions on $\R^n$ have a limiting subdifferential graph of uniform local dimension $n$.

math.OC↗
arXiv2012

Optimality, identifiability, and sensitivity

Around a solution of an optimization problem, an "identifiable" subset of the feasible region is one containing all nearby solutions after small perturbations to the problem. A quest for only the most essential ingredients of sensitivity analysis leads us to consider identifiable sets that are "minimal". This new notion lays a broad and intuitive variational-analytic foundation for optimality conditions, sensitivity, and active set methods.

math.OC↗
arXiv2011

The dimension of semialgebraic subdifferential graphs

Examples exist of extended-real-valued closed functions on ${\bf R}^n$ whose subdifferentials (in the standard, limiting sense) have large graphs. By contrast, if such a function is semi-algebraic, then its subdifferential graph must have everywhere constant local dimension $n$. This result is related to a celebrated theorem of Minty, and surprisingly may fail for the Clarke subdifferential.

math.OC↗
arXiv2010

Level set methods for finding critical points of mountain pass type

Computing mountain passes is a standard way of finding critical points. We describe a numerical method for finding critical points that is convergent in the nonsmooth case and locally superlinearly convergent in the smooth finite dimensional case. We apply these techniques to describe a strategy for the Wilkinson problem of calculating the distance of a matrix to a closest matrix with repeated eigenvalues. Finally, we relate critical points of mountain pass type to nonsmooth and metric critical point theory.

math.NA↗
arXiv2010

Generic nondegeneracy in convex optimization

We show that minimizers of convex functions subject to almost all linear perturbations are nondegenerate. An analogous result holds more generally, for lower-C^2 functions.

math.OC↗
arXiv2009

Identifying Activity

Identification of active constraints in constrained optimization is of interest from both practical and theoretical viewpoints, as it holds the promise of reducing an inequality-constrained problem to an equality-constrained problem, in a neighborhood of a solution. We study this issue in the more general setting of composite nonsmooth minimization, in which the objective is a composition of a smooth vector function c with a lower semicontinuous function h, typically nonsmooth but structured. In this setting, the graph of the generalized gradient of h can often be decomposed into a union (nondisjoint) of simpler subsets. "Identification" amounts to deciding which subsets of the graph are "active" in the criticality conditions at a given solution. We give conditions under which any convergent sequence of approximate critical points finitely identifies the activity. Prominent among these properties is a condition akin to the Mangasarian-Fromovitz constraint qualification, which ensures boundedness of the set of multiplier vectors that satisfy the optimality conditions at the solution.

math.OC↗
arXiv2008

Lipschitz behavior of the robust regularization

To minimize or upper-bound the value of a function "robustly", we might instead minimize or upper-bound the "epsilon-robust regularization", defined as the map from a point to the maximum value of the function within an epsilon-radius. This regularization may be easy to compute: convex quadratics lead to semidefinite-representable regularizations, for example, and the spectral radius of a matrix leads to pseudospectral computations. For favorable classes of functions, we show that the robust regularization is Lipschitz around any given point, for all small epsilon > 0, even if the original function is nonlipschitz (like the spectral radius). One such favorable class consists of the semi-algebraic functions. Such functions have graphs that are finite unions of sets defined by finitely-many polynomial inequalities, and are commonly encountered in applications.

math.OC↗
arXiv2004

The Lax conjecture is true

In 1958 Lax conjectured that hyperbolic polynomials in three variables are determinants of linear combinations of three symmetric matrices. This conjecture is equivalent to a recent observation of Helton and Vinnikov.

math.OC↗
arXiv2003

The structured distance to ill-posedness for conic systems

An important measure of conditioning of a conic linear system is the size of the smallest structured perturbation making the system ill-posed. We show that this measure is unchanged if we restrict to perturbations of low rank. We thereby derive a broad generalization of the classical Eckart-Young result characterizing the distance to ill-posedness for a linear map.

math.OC↗