arXiv subjects

Warren Hare

Publications and source records attributed to Warren Hare.

34 records · Page 2Linked to original sources

arXiv2020

A deterministic algorithm to compute the cosine measure of a finite positive spanning set

Originally developed in 1954, positive bases and positive spanning sets have been found to be a valuable concept in derivative-free optimization (DFO). The quality of a positive basis (or positive spanning set) can be quantified via the {\em cosine measure} and convergence properties of certain DFO algorithms are intimately linked to the value of this measure. However, it is unclear how to compute the cosine measure for a positive basis from the definition. In this paper, a deterministic algorithm to compute the cosine measure of any positive basis or finite positive spanning set is provided. The algorithm is proven to return the exact value of the cosine measure in finite time.

math.OC↗

arXiv2019

Parametrically prox-regular functions

Prox-regularity is a generalization of convexity that includes all C2, lower-C2, strongly amenable and primal-lower-nice functions. The study of prox-regular functions provides insight on a broad spectrum of important functions. Parametrically prox-regular (para-prox-regular) functions are a further extension of this family, produced by adding a parameter. Such functions have been shown to play a key role in understanding stability of minimizers in optimization problems. This document discusses para-prox-regular functions in Rn: we begin with some basic examples of para-prox-regular functions and move on to the more complex examples of the convex and nonconvex proximal average. We develop an alternate representation of a para-prox-regular function, related to the monotonicity of an f-attentive epsilon-localization as has been done for prox-regular functions. This extends a result of Levy, who used an alternate approach to show one implication of the relationship (we provide a characterization). We analyze two common forms of parametrized functions that appear in optimization: finite parametrized sum of functions and finite parametrized max of functions. The example of strongly amenable functions by Poliquin and Rockafellar is given and a relaxation of its necessary conditions is presented.

math.FA↗

arXiv2019

The chain rule for VU-decompositions of nonsmooth functions

In Variational Analysis, VU-theory provides a set of tools that is helpful for understanding and exploiting the structure of nonsmooth functions. The theory takes advantage of the fact that at any point, the space can be separated into two orthogonal subspaces: one that describes the direction of nonsmoothness of the function, and the other on which the function behaves smoothly and has a gradient. For a composite function, this work establishes a chain rule that facilitates the computation of such gradients and characterizes the smooth subspace under reasonable conditions. From the chain rule presented, formulas for the separation, smooth perturbation and sum of functions are provided. Several nonsmooth examples are explored, including norm functions, max-of-quadratic functions and LASSO-type regularizations.

math.OC↗

arXiv2019

The NC-proximal average for multiple functions

The NC-proximal average is a parametrized function used to continuously transform one proper, lsc, prox-bounded function into another. Until now, it has been defined for two functions. The purpose of this article is to redefine it so that any finite number of functions may be used. The layout generally follows that of [11], extending those results to the more general case and in some instances giving alternate proofs by using techniques developed after the publication of that paper. We conclude with an example examining the discontinuity of the minimizers of the NC-proximal average.

math.FA↗

arXiv2019

A derivative-free $\mathcal{VU}$-algorithm for convex finite-max problems

The $\mathcal{VU}$-algorithm is a superlinearly convergent method for minimizing nonsmooth, convex functions. At each iteration, the algorithm works with a certain $\mathcal{V}$-space and its orthogonal $\U$-space, such that the nonsmoothness of the objective function is concentrated on its projection onto the $\mathcal{V}$-space, and on the $\mathcal{U}$-space the projection is smooth. This structure allows for an alternation between a Newton-like step where the function is smooth, and a proximal-point step that is used to find iterates with promising $\mathcal{VU}$-decompositions. We establish a derivative-free variant of the $\mathcal{VU}$-algorithm for convex finite-max objective functions. We show global convergence and provide numerical results from a proof-of-concept implementation, which demonstrates the feasibility and practical value of the approach. We also carry out some tests using nonconvex functions and discuss the results.

math.OC↗

arXiv2018

"Active-set complexity" of proximal gradient: How long does it take to find the sparsity pattern?

Proximal gradient methods have been found to be highly effective for solving minimization problems with non-negative constraints or L1-regularization. Under suitable nondegeneracy conditions, it is known that these algorithms identify the optimal sparsity pattern for these types of problems in a finite number of iterations. However, it is not known how many iterations this may take. We introduce the notion of the "active-set complexity", which in these cases is the number of iterations before an algorithm is guaranteed to have identified the final sparsity pattern. We further give a bound on the active-set complexity of proximal gradient methods in the common case of minimizing the sum of a strongly-convex smooth function and a separable convex non-smooth function.

math.OC↗

arXiv2017

Visualization of the ε-Subdifferential of Piecewise Linear-Quadratic Functions

Computing explicitly the ε-subdifferential of a proper function amounts to computing the level set of a convex function namely the conjugate minus a linear function. The resulting theoretical algorithm is applied to the the class of (convex univariate) piecewise linear-quadratic functions for which existing numerical libraries allow practical computations. We visualize the results in a primal, dual, and subdifferential views through several numerical examples. We also provide a visualization of the Brøndsted-Rockafellar Theorem.

math.OC↗

arXiv2017

Best practices for comparing optimization algorithms

Comparing, or benchmarking, of optimization algorithms is a complicated task that involves many subtle considerations to yield a fair and unbiased evaluation. In this paper, we systematically review the benchmarking process of optimization algorithms, and discuss the challenges of fair comparison. We provide suggestions for each step of the comparison process and highlight the pitfalls to avoid when evaluating the performance of optimization algorithms. We also discuss various methods of reporting the benchmarking results. Finally, some suggestions for future research are presented to improve the current benchmarking process.

math.OC↗

arXiv2017

Multi-Haul Quasi Network Flow Model for Vertical Alignment Optimization

The vertical alignment optimization problem for road design aims to generate a vertical alignment of a new road with a minimum cost, while satisfying safety and design constraints. We present a new model called multi-haul quasi network flow (MH-QNF) for vertical alignment optimization that improves the accuracy and reliability of previous mixed integer linear programming models. We evaluate the performance of the new model compared to two state-of-the-art models in the field: the complete transportation graph (CTG) and the quasi network flow (QNF) models. The numerical results show that, within a 1% relative error, the proposed model is robust and solves more than 93% of test problems compared to 82% for the CTG and none for the QNF. Moreover, the MH-QNF model solves the problems approximately 8 times faster than the CTG model.

math.OC↗

arXiv2016

Compositions of Convex Functions and Fully Linear Models

Derivative-free optimization (DFO) is the mathematical study of the optimization algorithms that do not use derivatives. One branch of DFO focuses on model-based DFO methods, where an approximation of the objective function is used to guide the optimization algorithm. Proving convergence of such methods often applies an assumption that the approximations form {\em fully linear models} -- an assumption that requires the true objective function to be smooth. However, some recent methods have loosened this assumption and instead worked with functions that are compositions of smooth functions with simple convex functions (the max-function or the $\ell_1$ norm). In this paper, we examine the error bounds resulting from the composition of a convex lower semi-continuous function with a smooth vector-valued function when it is possible to provide fully linear models for each component of the vector-valued function. We derive error bounds for the resulting function values and subgradient vectors.

math.NA↗

arXiv2016

Computing proximal points of convex functions with inexact subgradients

Locating proximal points is a component of numerous minimization algorithms. This work focuses on developing a method to find the proximal point of a convex function at a point, given an inexact oracle. Our method assumes that exact function values are at hand, but exact subgradients are either not available or not useful. We use approximate subgradients to build a model of the objective function, and prove that the method converges to the true prox-point within acceptable tolerance. The subgradient $g_k$ used at each step $k$ is such that the distance from $g_k$ to the true subdifferential of the objective function at the current iteration point is bounded by some fixed $\varepsilon>0.$ The algorithm includes a novel tilt-correct step applied to the approximate subgradient.

math.OC↗

arXiv2016

Thresholds of Prox-Boundedness of PLQ functions

Introduced in the 1960s, the Moreau envelope has grown to become a key tool in non\-smooth analysis and optimization. Essentially an infimal convolution with a parametrized norm squared, the Moreau envelope is used in many applications and optimization algorithms. An important aspect in applying the Moreau envelope to nonconvex functions is determining if the function is prox-bounded, that is, if there exists a point $x$ and a parameter $r$ such that the Moreau envelope is finite. The infimum of all such $r$ is called the threshold of prox-boundedness (prox-threshold) of the function $f.$ In this paper, we seek to understand the prox-thresholds of piecewise linear-quadratic (PLQ) functions. (A PLQ function is a function whose domain is a union of finitely many polyhedral sets, and that is linear or quadratic on each piece.) The main result provides a computational technique for determining the prox-threshold for a PLQ function, and further analyzes the behavior of the Moreau envelope of the function using the prox-threshold. We provide several examples to illustrate the techniques and challenges.

math.OC↗

arXiv2016

Algorithmic construction of the subdifferential from directional derivatives

The subdifferential of a function is a generalization for nonsmooth functions of the concept of gradient. It is frequently used in variational analysis, particularly in the context of nonsmooth optimization. The present work proposes algorithms to reconstruct a polyhedral subdifferential of a function from the computation of finitely many directional derivatives. We provide upper bounds on the required number of directional derivatives when the space is $\R^1$ and $\R^2$, as well as in $\R^n$ where subdifferential is known to possess at most three vertices.

math.OC↗

arXiv2015

Multiple-Path Selection for new Highway Alignments using Discrete Algorithms

This paper addresses the problem of finding multiple near-optimal, spatially-dissimilar paths that can be considered as alternatives in the decision making process, for finding optimal corridors in which to construct a new road. We further consider combinations of techniques for reducing the costs associated with the computation and increasing the accuracy of the cost formulation. Numerical results for five algorithms to solve the dissimilar multipath problem show that a "bidirectional approach" yields the fastest running times and the most robust algorithm. Further modifications of the algorithms to reduce the running time were tested and it is shown that running time can be reduced by an average of 56 percent without compromising the quality of the results.

cs.DS↗

arXiv2015

Discussion: The NoMax Strategy and Correlated Outputs

This is a discussion of the paper "Modeling an Augmented Lagrangian for Improved Blackbox Constrained Optimization," (Gramacy, R.~B., Gray, G.~A., Digabel, S.~L., Lee, H.~K.~H., Ranjan, P., Wells, G., and Wild, S.~M., Technometrics, 61, 1--38, 2015).

math.OC↗

arXiv2015

Optimizing horizontal alignment of roads in a specified corridor

Finding an optimal alignment connecting two end-points in a specified corridor is a complex problem that requires solving three interrelated sub-problems, namely the horizontal alignment, vertical alignment and earthwork optimization problems. In this research, we developed a novel bi-level optimization model combining those three problems. In the outer level of the model, we optimize the horizontal alignment and in the inner level of the model a vertical alignment optimization problem considering earthwork allocation is solved for a fixed horizontal alignment. Derivative-free optimization algorithms are used to solve the outer problem. The result of our model gives an optimal horizontal alignment in the form of a linear-circular curve and an optimal vertical alignment in the form of a quadratic spline. Our model is tested on real-life data. The numerical results show that our approach improves the road alignment designed by civil engineers by 27\% on average, resulting in potentially millions of dollars of savings.

math.OC↗