arXiv subjects

William W. Hager

Publications and source records attributed to William W. Hager.

At least 19 recordsLinked to original sources

arXiv2025

Integral Form of Legendre-Gauss-Lobatto Collocation for Optimal Control

A new method is described for solving optimal control problems using direct collocation at Legendre-Gauss-Lobatto points. The approach of this paper employs a polynomial approximation of the right-hand side vector field of the differential equations and leads to the following important outcomes. First, the first-order optimality conditions of the LGL integral form are derived, which lead to a full-rank transformed adjoint system and novel costate estimate. Next, a derivative-like form of the LGL collocation method is obtained by multiplying the system by the inverse of an appropriate full-rank block of the integration matrix. The first-order optimality conditions of the LGL derivative-like form are then derived, leading to an equivalent full-rank transformed adjoint system and secondary novel costate estimate which is related to the costate estimate of the integral form via a linear transformation. Then, it is shown that a second integral form can be constructed by including an additional noncollocated support point, but such a point is superfluous and has no impact on the solution to the nonlinear programming problem. Finally, the method is demonstrated on two benchmark problems: a one-dimensional initial value optimal control problem with an analytic solution and a time-variant orbit raising optimal control problem.

math.OC↗

arXiv2024

Second-order sufficient optimality conditions in the calculus of variations

Some classic second-order sufficient optimality conditions in the calculus of variations are shown to be equivalent, while also introducing a new equivalent second-order condition which is extremely easy to apply: simply integrate a linear second-order initial value problem and check that the solution is positive over the problem domain.

math.OC↗

arXiv2023

Extension of Switch Point Algorithm to Boundary-Value Problems

In an earlier paper (https://doi.org/10.1137/21M1393315), the Switch Point Algorithm was developed for solving optimal control problems whose solutions are either singular or bang-bang or both singular and bang-bang, and which possess a finite number of jump discontinuities in an optimal control at the points in time where the solution structure changes. The class of control problems that were considered had a given initial condition, but no terminal constraint. The theory is now extended to include problems with both initial and terminal constraints, a structure that often arises in boundary-value problems. Substantial changes to the theory are needed to handle this more general setting. Nonetheless, the derivative of the cost with respect to a switch point is again the jump in the Hamiltonian at the switch point.

math.OC↗

arXiv2022

A Gradient-Based Implementation of the Polyhedral Active Set Algorithm

The Polyhedral Active Set Algorithm (PASA) is designed to optimize a general nonlinear function over a polyhedron. Phase one of the algorithm is a nonmonotone gradient projection algorithm, while phase two is an active set algorithm that explores faces of the constraint polyhedron. A gradient-based implementation is presented, where a projected version of the conjugate gradient algorithm is employed in phase two. Asymptotically, only phase two is performed. Comparisons are given with IPOPT using polyhedral constrained problems from CUTEst and the Maros/Meszaros quadratic programming test set.

math.OC↗

arXiv2021

NPASA: An algorithm for nonlinear programming -- Local Convergence

In this paper, we provide local convergence analysis for the two phase Nonlinear Polyhedral Active Set Algorithm (NPASA) designed to solve nonlinear programs. In particular, we establish local quadratic convergence of the primal iterates and global error estimator for NPASA under reasonable assumptions. Additionally, under the same set of assumptions we prove that only phase two of NPASA is executed after finitely many iterations. This paper is companion to a paper that provides motivation and global convergence analysis for NPASA.

math.OC↗

arXiv2021

NPASA: An algorithm for nonlinear programming -- Motivation and Global Convergence

In this paper, we present a two phase method for solving nonlinear programming problems called Nonlinear Polyhedral Active Set Algorithm (NPASA) that has global and local convergence guarantees under reasonable assumptions. The first phase consists of an augmented Lagrangian method to ensure global convergence while the second phase is designed to promote fast local convergence by performing a balanced reduction of two error estimators for nonlinear programs. After presenting error estimators for nonlinear programs and our algorithm NPASA, we establish global convergence properties for NPASA. Local quadratic convergence of NPASA is established in a companion paper.

math.OC↗

arXiv2021

The Switch Point Algorithm

The Switch Point Algorithm is a new approach for solving optimal control problems whose solutions are either singular or bang-bang or both singular and bang-bang, and which possess a finite number of jump discontinuities in an optimal control at the points in time where the solution structure changes. Problems in this class can often be reduced to an optimization over the switching points. Formulas are derived for the derivative of the objective with respect to the switch points, the initial costate, and the terminal time. All these derivatives can be computed simultaneously in just one integration of the state and costate dynamics. Hence, gradient-based unconstrained optimization techniques, including the conjugate gradient method or quasi-Newton methods, can be used to compute an optimal control. The performance of the algorithm is illustrated using test problems with known solutions and comparisons with other algorithms from the literature.

math.OC↗

arXiv2021

A Newton-Type Active Set Method for Nonlinear Optimization with Polyhedral Constraints

A Newton-type active set algorithm for large-scale minimization subject to polyhedral constraints is proposed. The algorithm consists of a gradient projection step, a second-order Newton-type step in the null space of the constraint matrix, and a set of rules for branching between the two steps. We show that the proposed method asymptotically takes the Newton step when the active constraints are linearly independent and a strong second-order sufficient optimality condition holds. We also show that the method has a quadratic rate of convergence under standard conditions. Numerical experiments are presented illustrating the performance of the algorithm on the CUTEst and on a specific class of problems for which finding second-order stationary points is critical.

math.OC↗

arXiv2020

Modified Legendre-Gauss-Radau Collocation Method for Solving Optimal Control Problems with Nonsmooth Solutions

A new method is developed for solving optimal control problems whose solutions are nonsmooth. The method developed in this paper employs a modified form of the Legendre-Gauss-Radau orthogonal direct collocation method. This modified Legendre-Gauss-Radau method adds two variables and two constraints at the end of a mesh interval when compared with a previously developed standard Legendre-Gauss-Radau collocation method. The two additional variables are the time at the interface between two mesh intervals and the control at the end of each mesh interval. The two additional constraints are a collocation condition for those differential equations that depend upon the control and an inequality constraint on the control at the endpoint of each mesh interval. The additional constraints modify the search space of the nonlinear programming problem such that an accurate approximation to the location of the nonsmoothness is obtained. The transformed adjoint system of the modified Legendre-Gauss-Radau method is then developed. Using this transformed adjoint system, a method is developed to transform the Lagrange multipliers of the nonlinear programming problem to the costate of the optimal control problem. Furthermore, it is shown that the costate estimate satisfies one of the Weierstrass-Erdmann optimality conditions. Finally, the method developed in this paper is demonstrated on an example whose solution is nonsmooth.

math.OC↗

arXiv2020

An Inexact Accelerated Stochastic ADMM for Separable Convex Optimization

An inexact accelerated stochastic Alternating Direction Method of Multipliers (AS-ADMM) scheme is developed for solving structured separable convex optimization problems with linear constraints. The objective function is the sum of a possibly nonsmooth convex function and a smooth function which is an average of many component convex functions. Problems having this structure often arise in machine learning and data mining applications. AS-ADMM combines the ideas of both ADMM and the stochastic gradient methods using variance reduction techniques. One of the ADMM subproblems employs a linearization technique while a similar linearization could be introduced for the other subproblem. For a specified choice of the algorithm parameters, it is shown that the objective error and the constraint violation are $\mathcal{O}(1/k)$ relative to the number of outer iterations $k$. Under a strong convexity assumption, the expected iterate error converges to zero linearly. A linearized variant of AS-ADMM and incremental sampling strategies are also discussed. Numerical experiments with both stochastic and deterministic ADMM algorithms show that AS-ADMM can be particularly effective for structured optimization arising in big data applications.

math.OC↗

arXiv2020

Convergence rates for an inexact ADMM applied to separable convex optimization

Convergence rates are established for an inexact accelerated alternating direction method of multipliers (I-ADMM) for general separable convex optimization with a linear constraint. Both ergodic and non-ergodic iterates are analyzed. Relative to the iteration number k, the convergence rate is O(1/k) in a convex setting and O(1/k^2) in a strongly convex setting. When an error bound condition holds, the algorithm is 2-step linearly convergent. The I-ADMM is designed so that the accuracy of the inexact iteration preserves the global convergence rates of the exact iteration, leading to better numerical performance in the test problems.

math.NA↗

arXiv2019

Mesh Refinement Method for Solving Bang-Bang Optimal Control Problems Using Direct Collocation

A mesh refinement method is developed for solving bang-bang optimal control problems using direct collocation. The method starts by finding a solution on a coarse mesh. Using this initial solution, the method then determines automatically if the Hamiltonian is linear with respect to the control, and, if so, estimates the locations of the discontinuities in the control. The switch times are estimated by determining the roots of the switching functions, where the switching functions are determined using estimates of the state and costate obtained from the collocation method. The accuracy of the switch times is then improved on subsequent meshes by dividing the original optimal control problem into multiple domains and including variables that define the locations of the switch times. While in principle any collocation method can be used, in this research the previously developed Legendre-Gauss-Radau collocation method is employed because it provides an accurate approximation of the costate which in turn improves the approximation of the switching functions. The method of this paper is designed to be used with a previously developed mesh refinement method in order to accurately approximate the solution in segments where the solution is smooth. The method is demonstrated on three examples where it is shown to accurately determine the switching structure of a bang-bang optimal control problem. When compared with previously developed mesh refinement methods, the results demonstrate that the method developed in this paper improves computational efficiency when solving bang-bang optimal control problems.

math.OC↗

arXiv2019

Projection Algorithms for Non-Convex Minimization with Application to Sparse Principal Component Analysis

We consider concave minimization problems over non-convex sets.Optimization problems with this structure arise in sparse principal component analysis. We analyze both a gradient projection algorithm and an approximate Newton algorithm where the Hessian approximation is a multiple of the identity. Convergence results are established. In numerical experiments arising in sparse principal component analysis, it is seen that the performance of the gradient projection algorithm is very similar to that of the truncated power method and the generalized power method. In some cases, the approximate Newton algorithm with a Barzilai-Borwein (BB) Hessian approximation can be substantially faster than the other algorithms, and can converge to a better solution.

math.NA↗

arXiv2018

Convergence Rate for a Radau hp Collocation Method Applied to Constrained Optimal Control

For unconstrained control problems, a local convergence rate is established for an $hp$-method based on collocation at the Radau quadrature points in each mesh interval of the discretization. If the continuous problem has a sufficiently smooth solution and the Hamiltonian satisfies a strong convexity condition, then the discrete problem possesses a local minimizer in a neighborhood of the continuous solution, and as either the number of collocation points or the number of mesh intervals increase, the discrete solution convergences to the continuous solution in the sup-norm. The convergence is exponentially fast with respect to the degree of the polynomials on each mesh interval, while the error is bounded by a polynomial in the mesh spacing. An advantage of the $hp$-scheme over global polynomials is that there is a convergence guarantee when the mesh is sufficiently small, while the convergence result for global polynomials requires that a norm of the linearized dynamics is sufficiently small. Numerical examples explore the convergence theory.

math.NA↗

arXiv2017

Convergence rate for a Gauss collocation method applied to constrained optimal control

A local convergence rate is established for a Gauss orthogonal collocation method applied to optimal control problems with control constraints. If the Hamiltonian possesses a strong convexity property, then the theory yields convergence for problems whose optimal state and costate possess two square integrable derivatives. The convergence theory is based on a stability result for the sup-norm change in the solution of a variational inequality relative to a 2-norm perturbation, and on a Sobolev space bound for the error in interpolation at the Gauss quadrature points and the additional point -1. The tightness of the convergence theory is examined using a numerical example.

math.NA↗

arXiv2016

A Multilevel Bilinear Programming Algorithm For the Vertex Separator Problem

The Vertex Separator Problem for a graph is to find the smallest collection of vertices whose removal breaks the graph into two disconnected subsets that satisfy specified size constraints. In the paper 10.1016/j.ejor.2014.05.042, the Vertex Separator Problem was formulated as a continuous (non-concave/non-convex) bilinear quadratic program. In this paper, we develop a more general continuous bilinear program which incorporates vertex weights, and which applies to the coarse graphs that are generated in a multilevel compression of the original Vertex Separator Problem. A Mountain Climbing Algorithm is used to find a stationary point of the continuous bilinear quadratic program, while second-order optimality conditions and perturbation techniques are used to escape from either a stationary point or a local maximizer. The algorithms for solving the continuous bilinear program are employed during the solution and refinement phases in a multilevel scheme. Computational results and comparisons demonstrate the advantage of the proposed algorithm.

cs.DS↗

arXiv2016

An Active Set Algorithm for Nonlinear Optimization with Polyhedral Constraints

A polyhedral active set algorithm PASA is developed for solving a nonlinear optimization problem whose feasible set is a polyhedron. Phase one of the algorithm is the gradient projection method, while phase two is any algorithm for solving a linearly constrained optimization problem. Rules are provided for branching between the two phases. Global convergence to a stationary point is established, while asymptotically PASA performs only phase two when either a nondegeneracy assumption holds, or the active constraints are linearly independent and a strong second-order sufficient optimality condition holds.

math.OC↗

arXiv2016

Inexact alternating direction multiplier methods for separable convex optimization

Inexact alternating direction multiplier methods (ADMMs) are developed for solving general separable convex optimization problems with a linear constraint and with an objective that is the sum of smooth and nonsmooth terms. The approach involves linearized subproblems, a back substitution step, and either gradient or accelerated gradient techniques. Global convergence is established. The methods are particularly useful when the ADMM subproblems do not have closed form solution or when the solution of the subproblems is expensive. Numerical experiments based on image reconstruction problems show the effectiveness of the proposed methods.

math.OC↗