arXiv subjects

I. M. Ross

Publications and source records attributed to I. M. Ross.

15 recordsLinked to original sources

arXiv2026

On The Mathematics of the Natural Physics of Optimization

A number of optimization algorithms have been inspired by the physics of Newtonian motion. Here, we ask the question: do algorithms themselves obey some ``natural laws of motion,'' and can they be derived by an application of these laws? We explore this question by positing the theory that optimization algorithms may be considered as some manifestation of hidden algorithm primitives that obey certain universal non-Newtonian dynamics. This natural physics of optimization is developed by equating the terminal transversality conditions of an optimal control problem to the generalized Karush/John-Kuhn-Tucker conditions of an optimization problem. Through this equivalence formulation, the data functions of a given constrained optimization problem generate a natural vector field that permeates an entire hidden space with information on the optimality conditions. An ``action-at-a-distance'' operation via a Pontryagin-type minimum principle produces a local action to deliver a globalized result by way of a Hamilton-Jacobi inequality. An inverse-optimal algorithm is generated by performing control jumps that dissipate quantized ``energy'' defined by a search Lyapunov function. Illustrative applications of the proposed theory show that a large number of algorithms can be generated and explained in terms of the new mathematical physics of optimization.

math.OC↗

arXiv2025

A Review of Pseudospectral Optimal Control: From Theory to Flight

The home space for optimal control is a Sobolev space. The home space for pseudospectral theory is also a Sobolev space. It thus seems natural to combine pseudospectral theory with optimal control theory and construct ``pseudospectral optimal control theory,'' a term coined by Ross. In this paper, we review key theoretical results in pseudospectral optimal control that have proven to be critical for a successful flight. Implementation details of flight demonstrations onboard NASA spacecraft are discussed along with emerging trends and techniques in both theory and practice. The 2011 launch of pseudospectral optimal control in embedded platforms is changing the way in which we see solutions to challenging control problems in aerospace and autonomous systems.

math.OC↗

arXiv2025

Hessians in Birkhoff-Theoretic Trajectory Optimization

This paper derives various Hessians associated with Birkhoff-theoretic methods for trajectory optimization. According to a theorem proved in this paper, approximately 80% of the eigenvalues are contained in the narrow interval [-2, 4] for all Birkhoff-discretized optimal control problems. A preliminary analysis of computational complexity is also presented with further discussions on the grand challenge of solving a million point trajectory optimization problem.

math.OC↗

arXiv2025

A Universal Birkhoff Theory for Fast Trajectory Optimization

Over the last two decades, pseudospectral methods based on Lagrange interpolants have flourished in solving trajectory optimization problems and their flight implementations. In a seemingly unjustified departure from these highly successful methods, a new starting point for trajectory optimization is proposed. This starting point is based on the recently-developed concept of universal Birkhoff interpolants. The new approach offers a substantial computational upgrade to the Lagrange theory in completely flattening the rapid growth of the condition numbers from O(N2) to O(1), where N is the number of grid points. In addition, the Birkhoff-specific primal-dual computations are isolated to a well-conditioned linear system even for nonlinear, nonconvex problems. This is part I of a two-part paper. In part I, a new theory is developed on the basis of two hypotheses. Other than these hypotheses, the theoretical development makes no assumptions on the choices of basis functions or the selection of grid points. Several covector mapping theorems are proved to establish the mathematical equivalence between direct and indirect Birkhoff methods. In part II of this paper (with Proulx), it is shown that a select family of Gegenbauer grids satisfy the two hypotheses required for the theory to hold. Numerical examples in part II illustrate the power and utility of the new theory.

math.OC↗

arXiv2025

A Million-Point Fast Trajectory Optimization Solver

One might argue that solving a trajectory optimization problem over a million grid points is preposterous. How about solving such a problem at an incredibly fast computational time? On a small form-factor processor? Algorithmic details that make possible this trifecta of breakthroughs are presented in this paper. The computational mathematics that deliver these advancements are: (i) a Birkhoff-theoretic discretization of optimal control problems, (ii) matrix-free linear algebra leveraging Krylov-subspace methods, and (iii) a near-perfect Birkhoff preconditioner that helps achieve $\mathcal{O}(1)$ iteration speed with respect to the grid size,~$N$. A key enabler of this high performance is the computation of Birkhoff matrix-vector products at $\mathcal{O}(N\log(N))$ time using fast Fourier transform techniques that eliminate traditional computational bottlenecks. A numerical demonstration of this unprecedented scale and speed is illustrated for a practical astrodynamics problem.

math.NA↗

arXiv2024

Unscented Trajectory Optimization

In a nutshell, unscented trajectory optimization is the generation of optimal trajectories through the use of an unscented transform. Although unscented trajectory optimization was introduced by the authors about a decade ago, it is reintroduced in this paper as a special instantiation of tychastic optimal control theory. Tychastic optimal control theory (from \textit{Tyche}, the Greek goddess of chance) avoids the use of a Brownian motion and the resulting Itô calculus even though it uses random variables across the entire spectrum of a problem formulation. This approach circumvents the enormous technical and numerical challenges associated with stochastic trajectory optimization. Furthermore, it is shown how a tychastic optimal control problem that involves nonlinear transformations of the expectation operator can be quickly instantiated using an unscented transform. These nonlinear transformations are particularly useful in managing trajectory dispersions be it associated with path constraints or targeted values of final-time conditions. This paper also presents a systematic and rapid process for formulating and computing the most desirable tychastic trajectory using an unscented transform. Numerical examples are used to illustrate how unscented trajectory optimization may be used for risk reduction and mission recovery caused by uncertainties and failures.

math.OC↗

arXiv2023

Derivation of Coordinate Descent Algorithms from Optimal Control Theory

Recently, it was posited that disparate optimization algorithms may be coalesced in terms of a central source emanating from optimal control theory. Here we further this proposition by showing how coordinate descent algorithms may be derived from this emerging new principle. In particular, we show that basic coordinate descent algorithms can be derived using a maximum principle and a collection of max functions as "control" Lyapunov functions. The convergence of the resulting coordinate descent algorithms is thus connected to the controlled dissipation of their corresponding Lyapunov functions. The operational metric for the search vector in all cases is given by the Hessian of the convex objective function.

math.OC↗

arXiv2023

Implementations of the Universal Birkhoff Theory for Fast Trajectory Optimization

This is part II of a two-part paper. Part I presented a universal Birkhoff theory for fast and accurate trajectory optimization. The theory rested on two main hypotheses. In this paper, it is shown that if the computational grid is selected from any one of the Legendre and Chebyshev family of node points, be it Lobatto, Radau or Gauss, then, the resulting collection of trajectory optimization methods satisfy the hypotheses required for the universal Birkhoff theory to hold. All of these grid points can be generated at an $\mathcal{O}(1)$ computational speed. Furthermore, all Birkhoff-generated solutions can be tested for optimality by a joint application of Pontryagin's- and Covector-Mapping Principles, where the latter was developed in Part~I. More importantly, the optimality checks can be performed without resorting to an indirect method or even explicitly producing the full differential-algebraic boundary value problem that results from an application of Pontryagin's Principle. Numerical problems are solved to illustrate all these ideas. The examples are chosen to particularly highlight three practically useful features of Birkhoff methods: (1) bang-bang optimal controls can be produced without suffering any Gibbs phenomenon, (2) discontinuous and even Dirac delta covector trajectories can be well approximated, and (3) extremal solutions over dense grids can be computed in a stable and efficient manner.

math.OC↗

arXiv2022

A Derivation of Nesterov's Accelerated Gradient Algorithm from Optimal Control Theory

Nesterov's accelerated gradient algorithm is derived from first principles. The first principles are founded on the recently-developed optimal control theory for optimization. This theory frames an optimization problem as an optimal control problem whose trajectories generate various continuous-time algorithms. The algorithmic trajectories satisfy the necessary conditions for optimal control. The necessary conditions produce a controllable dynamical system for accelerated optimization. Stabilizing this system via a quadratic control Lyapunov function generates an ordinary differential equation. An Euler discretization of the resulting differential equation produces Nesterov's algorithm. In this context, this result solves the purported mystery surrounding the algorithm.

math.OC↗

arXiv2022

An Optimal Control Theory for Accelerated Optimization

The first-order optimality conditions for a generic nonlinear optimization problem are generated as part of the terminal transversality conditions of an optimal control problem. It is shown that the Lagrangian of the optimization problem is connected to the Hamiltonian of the optimal control problem via a zero-Hamiltonian, infinite-order, singular arc. The necessary conditions for the singular optimal control problem are used to produce an auxiliary controllable dynamical system whose trajectories generate algorithm primitives for the optimization problem. A three-step iterative map for a generic algorithm is designed by a semi-discretization step. Neither the feedback control law nor the differential equation governing the algorithm need be derived explicitly. A search direction is produced by a proximal-aiming-type method that dissipates a control Lyapunov function. New step size procedures based on minimizing control Lyapunov functions along a search vector complete the design of the accelerated algorithms.

math.OC↗

arXiv2020

A Direct Shooting Method is Equivalent to an Indirect Method

We show that a direct shooting method is mathematically equivalent to an indirect method in the sense of certain first-order conditions. Specific mathematical formulas pertaining to the equivalence of a direct shooting method with an indirect method are derived. We also show that a theoretical equivalence does not necessarily translate to practical equivalence if the parameterized optimal control problem is simply patched to a nonlinear programming solver. A mathematical explanation is provided for the successes and failures of such patched nonlinear programming methods. In order to generate the correct solution more consistently, the nonlinear programming solver used in a traditional direct method must be replaced or augmented by a Hamiltonian programming method. The theoretical results derived in this paper further strengthen the connections between computational optimal control, deep learning and automatic differentiation.

math.OC↗

arXiv2020

Enhancements to the DIDO Optimal Control Toolbox

In 2020, DIDO$^©$ turned 20! The software package emerged in 2001 as a basic, user-friendly MATLAB$^\circledR$ teaching-tool to illustrate the various nuances of Pontryagin's Principle but quickly rose to prominence in 2007 after NASA announced it had executed a globally optimal maneuver using DIDO. Since then, the toolbox has grown in applications well beyond its aerospace roots: from solving problems in quantum control to ushering rapid, nonlinear sensitivity-analysis in designing high-performance automobiles. Most recently, it has been used to solve continuous-time traveling-salesman problems. Over the last two decades, DIDO's algorithms have evolved from their simple use of generic nonlinear programming solvers to a multifaceted engagement of fast spectral Hamiltonian programming techniques. A description of the internal enhancements to DIDO that define its mathematics and algorithms are described in this paper. A challenge example problem from robotics is included to showcase how the latest version of DIDO is capable of escaping the trappings of a ``local minimum'' that ensnare many other trajectory optimization methods.

math.OC↗

arXiv2020

An Optimal Control Theory for the Traveling Salesman Problem and Its Variants

We show that the traveling salesman problem (TSP) and its many variants may be modeled as functional optimization problems over a graph. In this formulation, all vertices and arcs of the graph are functionals; i.e., a mapping from a space of measurable functions to the field of real numbers. Many variants of the TSP, such as those with neighborhoods, with forbidden neighborhoods, with time-windows and with profits, can all be framed under this construct. In sharp contrast to their discrete-optimization counterparts, the modeling constructs presented in this paper represent a fundamentally new domain of analysis and computation for TSPs and their variants. Beyond its apparent mathematical unification of a class of problems in graph theory, the main advantage of the new approach is that it facilitates the modeling of certain application-specific problems in their home space of measurable functions. Consequently, certain elements of economic system theory such as dynamical models and continuous-time cost/profit functionals can be directly incorporated in the new optimization problem formulation. Furthermore, subtour elimination constraints, prevalent in discrete optimization formulations, are naturally enforced through continuity requirements. The price for the new modeling framework is nonsmooth functionals. Although a number of theoretical issues remain open in the proposed mathematical framework, we demonstrate the computational viability of the new modeling constructs over a sample set of problems to illustrate the rapid production of end-to-end TSP solutions to extensively-constrained practical problems.

math.OC↗

arXiv2019

Fast Mesh Refinement in Pseudospectral Optimal Control

Mesh refinement in pseudospectral (PS) optimal control is embarrassingly easy --- simply increase the order $N$ of the Lagrange interpolating polynomial and the mathematics of convergence automates the distribution of the grid points. Unfortunately, as $N$ increases, the condition number of the resulting linear algebra increases as $N^2$; hence, spectral efficiency and accuracy are lost in practice. In this paper, we advance Birkhoff interpolation concepts over an arbitrary grid to generate well-conditioned PS optimal control discretizations. We show that the condition number increases only as $\sqrt{N}$ in general, but is independent of $N$ for the special case of one of the boundary points being fixed. Hence, spectral accuracy and efficiency are maintained as $N$ increases. The effectiveness of the resulting fast mesh refinement strategy is demonstrated by using \underline{polynomials of over a thousandth order} to solve a low-thrust, long-duration orbit transfer problem.

math.OC↗

arXiv2018

Scaling and Balancing for High-Performance Computation of Optimal Controls

It is well-known that proper scaling can increase the efficiency of computational problems. In this paper we define and show that a balancing technique can substantially improve the computational efficiency of optimal control algorithms. We also show that non-canonical scaling and balancing procedures may be used quite effectively to reduce the computational difficulty of some hard problems. These results have been used successfully for several flight and field operations at NASA and DoD. A surprising aspect of our analysis shows that it may be inadvisable to use auto-scaling procedures employed in some software packages. The new results are agnostic to the specifics of the computational method; hence, they can be used to enhance the utility of any existing algorithm or software.

math.OC↗