SearcharxivSearch

arXiv subjects

Yaguang Yang

Publications and source records attributed to Yaguang Yang.

At least 19 recordsLinked to original sources

Combining Reinforcement Learning with Arc-search Interior-Point Method for Path Planning

Path planning in environments containing obstacles has numerous practical applications. The problem is challenging because it is inherently nonlinear and nonconvex. Consequently, a variety of techniques have been developed to address this problem, among which machine learning and optimal control (or optimization) have emerged as two prominent approaches. In general, machine learning methods do not require a high-fidelity model, and a trained agent can often generate a feasible path in real time. However, the resulting path is not necessarily optimal with respect to performance objectives such as minimizing path length or travel time. In contrast, optimal control and optimization methods typically rely on high-fidelity models and often require computational effort that may not satisfy real-time constraints. Nevertheless, these methods are more likely to produce optimal or near-optimal solutions. To overcome the limitations of each approach while exploiting their respective strengths, this paper proposes a framework that combines reinforcement learning with an arc-search interior-point method for path planning. Numerical simulations demonstrate that the proposed approach effectively integrates the real-time decision-making capability of reinforcement learning with the optimization performance of the arc-search interior-point method, resulting in improved path-planning performance.

math.OC

Path Planning Using Deep Deterministic Policy Gradient: A Reinforcement Learning Approach

Path-planning for autonomous vehicles in threat-laden environments is a fundamental challenge because the problem is nonlinear and nonconvex even in simplest scenarios. While traditional optimal control methods can be used to find ideal paths, the computational time is often too slow for real-time decision-making. To solve this challenge, we propose a method based on Deep Deterministic Policy Gradient (DDPG) and model the threat as possibly multiple circular 'no-go' zones. A mission is regarded as a failure if the vehicle enters this restricted zone at any time or does not reach a neighborhood of the destination. The DDPG agent is trained through trial and error in a simulated environment, learning a direct mapping from its current state (position and heading) to a series of feasible actions that guide the agent to safely reach its destination. The reword function has three parts: (a) an attractive field centered at the final destination, (b) some repulsive fields centered at the origins of circular obstacles, and (c) a penalty of control energy consumption (the magnitude of heading change) that indirectly in favor for straight path. The DDPG trains the agent using these incentives to find the largest possible set of starting points wherein a safe path to the destination is guaranteed. This provides critical information for mission planning, showing beforehand whether a task is achievable from a given starting point, assisting pre-mission planning activities. The approach is validated in simulation. A comparison between the DDPG method and a traditional optimal control (pseudo-spectral) method is carried out. The results show that the learning-based agent produces effective paths while being significantly faster, making it a better fit for real-time applications.

cs.RO

An arc-search BFGS algorithm for unconstrained nonlinear optimization problems

The classical BFGS algorithm performs excellently for convex optimization problems. However, for non-convex problems, the classical BFGS method may fail to converge reliably. To overcome this limitation, researchers have developed modified BFGS methods that are applicable to both convex and non-convex optimization problems. Among these methods, a robust BFGS algorithm has been shown to achieve global convergence and fast local convergence, with a superlinear convergence rate, for both convex and non-convex nonlinear optimization problems under mild assumptions. In this paper, we propose an arc-search BFGS algorithm that aims to further improve the computational efficiency of the robust BFGS method while preserving its desirable convergence properties. Numerical experiments are carried out, and performance comparisons between the proposed algorithm and state-of-the-art algorithms are reported to demonstrate the advantages of the arc-search BFGS algorithm.

math.OC

On the facet pivot simplex method for linear programming

Dantzig's vertex pivot simplex method has been published for more than seven decades. Amazingly, it remains one of the most efficient methods to solve linear programming (LP) problem after numerous efforts trying to find some better methods. In this paper, we propose a facet pivot simplex method and demonstrate by numerical testing that the new method is very promising compared to the vertex pivot method. Since there is no polynomial pivot simplex algorithm for linear programming problems after many decades of effort, we hope that this new type of pivot algorithm will give us some hope to find a polynomial pivot simplex method for linear programming problems. A Matlab implementation of the facet pivot algorithm and Netlib benchmark test problems are available in Matlab file exchange website.

math.OC

A Comparison of Reinforcement Learning and Optimal Control Methods for Path Planning

Path-planning for autonomous vehicles in threat-laden environments is a fundamental challenge. While traditional optimal control methods can find ideal paths, the computational time is often too slow for real-time decision-making. To solve this challenge, we propose a method based on Deep Deterministic Policy Gradient (DDPG) and model the threat as a simple, circular `no-go' zone. A mission failure is claimed if the vehicle enters this `no-go' zone at any time or does not reach a neighborhood of the destination. The DDPG agent is trained to learn a direct mapping from its current state (position and velocity) to a series of feasible actions that guide the agent to safely reach its goal. A reward function and two neural networks, critic and actor, are used to describe the environment and guide the control efforts. The DDPG trains the agent to find the largest possible set of starting points (``feasible set'') wherein a safe path to the goal is guaranteed. This provides critical information for mission planning, showing beforehand whether a task is achievable from a given starting point, assisting pre-mission planning activities. The approach is validated in simulation. A comparison between the DDPG method and a traditional optimal control (pseudo-spectral) method is carried out. The results show that the learning-based agent may produce effective paths while being significantly faster, making it a better fit for real-time applications. However, there are areas (``infeasible set'') where the DDPG agent cannot find paths to the destination, and the paths in the feasible set may not be optimal. These preliminary results guide our future research: (1) improve the reward function to enlarge the DDPG feasible set, (2) examine the feasible set obtained by the pseudo-spectral method, and (3) investigate the arc-search IPM method for the path planning problem.

math.OC

A robust BFGS algorithm for unconstrained nonlinear optimization problems

In this paper, a modified BFGS algorithm is proposed. The modified BFGS matrix estimates a modified Hessian matrix which is a convex combination of an identity matrix for the steepest descent algorithm and a Hessian matrix for the Newton algorithm. The coefficient of the convex combination in the modified BFGS algorithm is dynamically chosen in every iteration. It is proved that, for any twice differentiable nonlinear function (convex or non-convex), the algorithm is globally convergent to a stationary point. If the stationary point is a local optimizer where the Hessian is strongly positive definite in a neighborhood of the optimizer, the iterates will eventually enter and stay in the neighborhood, and the modified BFGS algorithm reduces to the BFGS algorithm in this neighborhood. Therefore, the modified BFGS algorithm is super-linearly convergent. Moreover, the computational cost of the modified BFGS in each iteration is almost the same as the cost of the BFGS. Numerical test on the CUTE test set is reported. The performance of the modified BFGS algorithm implemented in our MATLAB function is compared to the BFGS algorithm implemented in the MATLAB Optimization Toolbox function, a limited memory BFGS implemented as L-BFGS, a descent conjugate gradient algorithm implemented as CG-Descent 5.3, and a limited memory, descent and conjugate algorithm implemented as L-CG-Descent. This result shows that the modified BFGS algorithm may be very effective.

math.OC

On the facet pivot simplex method for linear programming II: a linear iteration bound

The Hirsch Conjecture stated that any $d$-dimensional polytope with n facets has a diameter at most equal to $n - d$. This conjecture was disproved by Santos (A counterexample to the Hirsch Conjecture, Annals of Mathematics, 172(1) 383-412, 2012). The implication of Santos' work is that all {\it vertex} pivot algorithms cannot solve the linear programming problem in the worst case in $n - d$ vertex pivot iterations. In the first part of this series of papers, we proposed a {\it facet} pivot method. In this paper, we show that the proposed facet pivot method can solve the canonical linear programming problem in the worst case in at most $n-d$ facet pivot iterations. This work was inspired by Smale's Problem 9 (Mathematical problems for the next century, In Arnold, V. I.; Atiyah, M.; Lax, P.; Mazur, B. Mathematics: frontiers and perspectives, American Mathematical Society, 271-294, 1999).

math.OC

On optimal LISA orbit design

The ESA/NASA joint LISA (laser interferometer space antenna) mission is designed to detect gravitational waves, which relies crucially on maintaining three-spacecraft constellation as close to an equilateral triangle with a designed distance as possible. Efforts have been made to achieve this goal by using various simplified models to make it easy to approximately solve the complex problem. In this paper, the problem is formulated as a nonlinear optimization problem using exact nonlinear Kepler's orbit equations. It is shown that the optimal solution based on the exact nonlinear Kepler's orbit equations gives a better solution than the previously obtained ones.

gr-qc

A Facet Enumeration Algorithm for Convex Polytopes

This paper proposes a novel and simple algorithm of facet enumeration for convex polytopes. The complexity of the algorithm is discussed. The algorithm is implemented in Matlab. Some simple polytopes with known H-representations and V-representations are used as the test examples. Numerical test shows the effectiveness and efficiency of the proposed algorithm. Due to the duality between the vertex enumeration problem and facet enumeration problem, we expect that this method can also be used to solve the vertex enumeration problem.

math.OC

A computationally efficient arc-search interior-point algorithm for nonlinear constrained optimization

This paper proposes an arc-search interior-point algorithm for the nonlinear constrained optimization problem. The proposed algorithm uses the second-order derivatives to construct a search arc that approaches the optimizer. Because the arc stays in the interior set longer than any straight line, it is expected that the scheme will generate a better new iterate than a line search method. The computation of the second-order derivatives requires to solve the second linear system of equations, but the coefficient matrix of the second linear system of equations is the same as the first linear system of equations. Therefore, the matrix decomposition obtained while solving the first linear system of equations can be reused. In addition, most elements of the right-hand side vector of the second linear system of equations are already computed when the coefficient matrix is assembled. Therefore, the computation cost for solving the second linear system of equations is insignificant and the benefit of having a better search scheme is well justified. The convergence of the proposed algorithm is established. Some preliminary test results are reported to demonstrate the merit of the proposed algorithm.

math.OC

A Systematic Methodology for Modeling and Attitude Control of Multi-body Space Telescopes

This paper derives a symbolic multi-body rigid nonlinear model for a space telescope using Stoneking's implementation of Kane's method. This symbolic nonlinear model is linearized using Matlab symbolic functions {\tt diff} and {\tt inv} because the analytic linearization is intractable for manual derivation. The linearized system model is then used to design the controllers using both linear quadratic regulator (LQR) and robust pole assignment methods. The closed-loop systems for the two designs are simulated using both the rigid model as well as a second model containing flexible modes. The performances of the two designs are compared based on the simulation testing results. Our conclusion is that the robust pole assignment design offers better performance than that of the LQR system in terms of actuator usage and pointing accuracy. However, the LQR approach remains an effective first design step that can inform the selection of real eigenvalues for robust pole assignment. The proposed method may be used for the modeling and controller designs for various multi-body systems.

math.OC

A double-pivot degenerate-robust simplex algorithm for linear programming

A double pivot algorithm that combines features of two recently published papers by these authors is proposed. The proposed algorithm is implemented in MATLAB. The MATLAB code is tested, along with a MATLAB implementation of Dantzig's algorithm, for several test sets, including a set of cycling LP problems, Klee-Minty's problems, randomly generated linear programming (LP) problems, and Netlib benchmark problems. The test result shows that the proposed algorithm is (a) degenerate-tolerance as we expected, and (b) more efficient than Dantzig's algorithm for large size randomly generated LP problems but less efficient for Netlib benchmark problems and small size randomly generated problems in terms of CPU time.

math.OC

A polynomial time infeasible interior-point arc-search algorithm for convex optimization

This paper proposes an infeasible interior-point algorithm for the convex optimization problem using arc-search techniques. The proposed algorithm simultaneously selects the centering parameter and the step size, aiming at optimizing the performance in every iteration. Analytic formulas for the arc-search are provided to make the arc-search method very efficient. The convergence of the algorithm is proved and a polynomial bound of the algorithm is established. The preliminary numerical test results indicate that the algorithm is efficient and effective.

math.OC

Cycling problems in linear programming

This paper provides a set of cycling problems in linear programming. These problems should be useful for researchers to develop and test new simplex algorithms. As matter of the fact, this set of problems is used to test a recently proposed double pivot simplex algorithm for linear programming.

math.OC

A note on the diameter of convex polytope

This short note extends a recent result (Bonifas et al, On sub-determinants and the diameter of polyhedra, Discrete Computational Geometry, 52, 2014) of an upper bound of the diameter of a convex polytope defined by an integer matrix to a similar upper bound of the diameter of a convex polytope defined by a real matrix. It also shows, by an example, that the new bound may be better than the ones of Bonifas et al.

math.MG

An Infeasible Interior-point Arc-search Algorithm for Nonlinear Constrained Optimization

In this paper, we propose an infeasible arc-search interior-point algorithm for solving nonlinear programming problems. Most algorithms based on interior-point methods are categorized as line search, since they compute a next iterate on a straight line determined by a search direction which approximates the central path.The proposed arc-search interior-point algorithm uses an arc for the approximation.We discuss convergence properties of the proposed algorithm.We also conduct numerical experiments on the CUTEst benchmark problems and compare the performance of the proposed arc-search algorithm with that of a line-search algorithm. Numerical results indicate that the proposed arc-search algorithm reaches the optimal solution using less iterations but longer time than a line-search algorithm. A modification that leads to a faster arc-search algorithm is also discussed.

math.OC

A double-pivot simplex algorithm and its upper bounds of the iteration numbers

In this paper, a double-pivot simplex method is proposed. Two upper bounds of iteration numbers are derived. Applying one of the bounds to some special linear programming (LP) problems, such as LP with a totally unimodular matrix and Markov Decision Problem (MDP) with a fixed discount rate, indicates that the double-pivot simplex method solves these problems in a strongly polynomial time. A variant of Klee-Minty cube is used to show that the estimated bounds of the iteration numbers are very tight. Numerical test on three variants of Klee-Minty cubes is performed for the problems with sizes as big as $200$ constraints and $400$ variables. Dantzig's simplex method cannot handle Klee-Minty cube problem with $200$ constraints because it needs about $2^{200} \approx 10^{60}$ iterations. But the proposed algorithm performs extremely good for all three variants.

math.OC

Two computationally efficient polynomial-iteration infeasible interior-point algorithms for linear programming

Since the beginning of the development of interior-point methods, there exists a puzzling gap between the results in theory and the observations in numerical experience, i.e., algorithms with good polynomial bound are not computationally efficient and algorithms demonstrated efficiency in computation do not have a good or any polynomial bound. Todd raised a question in 2002: "Can we find a theoretically and practically efficient way to reoptimize?" This paper is an effort to close the gap. We propose two arc-search infeasible interior-point algorithms with infeasible central path neighborhood wider than all existing infeasible interior-point algorithms that are proved to be convergent. We show that the first algorithm is polynomial and its simplified version, if it terminates in finite iterations, has a complexity bound equal to the best known complexity bound for all (feasible or infeasible) interior-point algorithms. We demonstrate the computational efficiency of the proposed algorithms by testing all Netlib linear programming problems in standard form and comparing the numerical results to those obtained by Mehrotra's predictor-corrector algorithm and a recently developed more efficient arc-search algorithm (the convergence of these two algorithms is unknown). We conclude that the newly proposed algorithms are not only polynomial but also computationally competitive comparing to both Mehrotra's predictor-corrector algorithm and the efficient arc-search algorithm.

math.OC