arXiv subjects

Richard Linares

Publications and source records attributed to Richard Linares.

At least 73 records · Page 4Linked to original sources

arXiv2020

Random Finite Set Theory and Centralized Control of Large Collaborative Swarms

Controlling large swarms of robotic agents presents many challenges including, but not limited to, computational complexity due to a large number of agents, uncertainty in the functionality of each agent in the swarm, and uncertainty in the swarm's configuration. This work generalizes the swarm state using Random Finite Set (RFS) theory and solves a centralized control problem with a Quasi-Newton optimization through the use of Model Predictive Control (MPC) to overcome the aforementioned challenges. This work uses the RFS formulation to control the distribution of agents assuming an unknown or unspecified number of agents. Computationally efficient solutions are also obtained via the MPC version of the Iterative Linear Quadratic Regulator (ILQR), a variant of Differential Dynamic Programming (DDP). Information divergence is used to define the distance between the swarm RFS and the desired swarm configuration through the use of the modified $L_2^2$ distance. Simulation results using MPC and ILQR show that the swarm intensity converges to the desired intensity. Additionally, the RFS control formulation is shown to be very flexible in terms of the number of agents in the swarm and configuration of the desired Gaussian mixtures. Lastly, the ILQR and the Gaussian Mixture Probability Hypothesis Density filter are used in conjunction to solve a spacecraft relative motion problem with imperfect information to show the viability of centralized RFS control for this real-world scenario.

eess.SY↗

arXiv2020

A set of orbital elements to fully represent the zonal harmonics around an oblate celestial body

This work introduces a new set of orbital elements to fully represent the zonal harmonics problem around an oblate celestial body. This new set of orbital elements allows to obtain a complete linear system for the unperturbed problem and, in addition, a complete polynomial system when considering the perturbation produced by the zonal harmonics from the gravitational force of an oblate celestial body. These orbital elements present no singularities and are able to represent any kind of orbit, including elliptic, parabolic and hyperbolic orbits. In addition, an application to this formulation of the Poincaré-Lindstedt perturbation method is included to obtain an approximate first order solution of the problem for the case of the J2 perturbation.

astro-ph.EP↗

arXiv2020

Autonomous Six-Degree-of-Freedom Spacecraft Docking Maneuvers via Reinforcement Learning

A policy for six-degree-of-freedom docking maneuvers is developed through reinforcement learning and implemented as a feedback control law. Reinforcement learning provides a potential framework for robust, autonomous maneuvers in uncertain environments with low on-board computational cost. Specifically, proximal policy optimization is used to produce a docking policy that is valid over a portion of the six-degree-of-freedom state-space while striving to minimize performance and control costs. Experiments using the simulated Apollo transposition and docking maneuver exhibit the policy's capabilities and provide a comparison with standard optimal control techniques. Furthermore, specific challenges and work-arounds, as well as a discussion on the benefits and disadvantages of reinforcement learning for docking policies, are discussed to facilitate future research. As such, this work will serve as a foundation for further investigation of learning-based control laws for spacecraft proximity operations in uncertain environments.

eess.SY↗

arXiv2020

Motion Planning and Control for On-Orbit Assembly using LQR-RRT* and Nonlinear MPC

Deploying large, complex space structures is of great interest to the modern scientific world as it can provide new capabilities in obtaining scientific, communicative, and observational information. However, many theoretical mission designs contain complexities that must be constrained by the requirements of the launch vehicle, such as volume and mass. To mitigate such constraints, the use of on-orbit additive manufacturing and robotic assembly allows for the flexibility of building large complex structures including telescopes, space stations, and communication satellites. The contribution of this work is to develop motion planning and control algorithms using the linear quadratic regulator and rapidly-exploring randomized trees (LQR-RRT*), path smoothing, and tracking the trajectory using a closed-loop nonlinear receding horizon control optimizer for a robotic Astrobee free-flyer. By obtaining controlled trajectories that consider obstacle avoidance and dynamics of the vehicle and manipulator, the free-flyer rapidly considers and plans the construction of space structures. The approach is a natural generalization to repairing, refueling, and re-provisioning space structure components while providing optimal collision-free trajectories during operation.

cs.RO↗

arXiv2020

Atmospheric Density Uncertainty Quantification for Satellite Conjunction Assessment

Conjunction assessment requires knowledge of the uncertainty in the predicted orbit. Errors in the atmospheric density are a major source of error in the prediction of low Earth orbits. Therefore, accurate estimation of the density and quantification of the uncertainty in the density is required. Most atmospheric density models, however, do not provide an estimate of the uncertainty in the density. In this work, we present a new approach to quantify uncertainties in the density and to include these for calculating the probability of collision Pc. For this, we employ a recently developed dynamic reduced-order density model that enables efficient prediction of the thermospheric density. First, the model is used to obtain accurate estimates of the density and of the uncertainty in the estimates. Second, the density uncertainties are propagated forward simultaneously with orbit propagation to include the density uncertainties for Pc calculation. For this, we account for the effect of cross-correlation in position uncertainties due to density errors on the Pc. Finally, the effect of density uncertainties and cross-correlation on the Pc is assessed. The presented approach provides the distinctive capability to quantify the uncertainty in atmospheric density and to include this uncertainty for conjunction assessment while taking into account the dependence of the density errors on location and time. In addition, the results show that it is important to consider the effect of cross-correlation on the Pc, because ignoring this effect can result in severe underestimation of the collision probability.

astro-ph.EP↗

arXiv2020

Reinforcement Meta-Learning for Interception of Maneuvering Exoatmospheric Targets with Parasitic Attitude Loop

We use Reinforcement Meta-Learning to optimize an adaptive integrated guidance, navigation, and control system suitable for exoatmospheric interception of a maneuvering target. The system maps observations consisting of strapdown seeker angles and rate gyro measurements directly to thruster on-off commands. Using a high fidelity six degree-of-freedom simulator, we demonstrate that the optimized policy can adapt to parasitic effects including seeker angle measurement lag, thruster control lag, the parasitic attitude loop resulting from scale factor errors and Gaussian noise on angle and rotational velocity measurements, and a time varying center of mass caused by fuel consumption and slosh. Importantly, the optimized policy gives good performance over a wide range of challenging target maneuvers. Unlike previous work that enhances range observability by inducing line of sight oscillations, our system is optimized to use only measurements available from the seeker and rate gyros. Through extensive Monte Carlo simulation of randomized exoatmospheric interception scenarios, we demonstrate that the optimized policy gives performance close to that of augmented proportional navigation with perfect knowledge of the full engagement state. The optimized system is computationally efficient and requires minimal memory, and should be compatible with today's flight processors.

eess.SY↗

arXiv2020

Efficient search of optimal Flower Constellations

We derive an analytical closed expression to compute the minimum distance (quantified by the angle of separation measured from the center of the Earth) between any two satellites located at the same altitude and in circular orbits. We also exploit several properties of Flower Constellations (FCs) that, combined with our formula for the distance, give an efficient method to compute the minimum angular distance between satellites, for all possible FCs with up to a given number of satellites.

astro-ph.IM↗

arXiv2020

Decentralized Control of Large Collaborative Swarms using Random Finite Set Theory

Controlling large swarms of robotic agents presents many challenges including, but not limited to, computational complexity due to a large number of agents, uncertainty in the functionality of each agent in the swarm, and uncertainty in the swarm's configuration. The contribution of this work is to decentralize Random Finite Set (RFS) control of large collaborative swarms for control of individual agents. The RFS control formulation assumes that the topology underlying the swarm control is complete and uses the complete graph in a centralized manner. To generalize the control topology in a localized or decentralized manner, sparse LQR is used to sparsify the RFS control gain matrix obtained using iterative LQR. This allows agents to use information of agents near each other (localized topology) or only the agent's own information (decentralized topology) to make a control decision. Sparsity and performance for decentralized RFS control are compared for different degrees of localization in feedback control gains which show that the stability and performance compared to centralized control do not degrade significantly in providing RFS control for large collaborative swarms.

eess.SY↗

arXiv2020

Adaptive Generalized ZEM-ZEV Feedback Guidance for Planetary Landing via a Deep Reinforcement Learning Approach

Precision landing on large and small planetary bodies is a technology of utmost importance for future human and robotic exploration of the solar system. In this context, the Zero-Effort-Miss/Zero-Effort-Velocity (ZEM/ZEV) feedback guidance algorithm has been studied extensively and is still a field of active research. The algorithm, although powerful in terms of accuracy and ease of implementation, has some limitations. Therefore with this paper we present an adaptive guidance algorithm based on classical ZEM/ZEV in which machine learning is used to overcome its limitations and create a closed loop guidance algorithm that is sufficiently lightweight to be implemented on board spacecraft and flexible enough to be able to adapt to the given constraint scenario. The adopted methodology is an actor-critic reinforcement learning algorithm that learns the parameters of the above-mentioned guidance architecture according to the given problem constraints.

eess.SY↗

arXiv2020

Six Degree-of-Freedom Body-Fixed Hovering over Unmapped Asteroids via LIDAR Altimetry and Reinforcement Meta-Learning

We optimize a six degrees of freedom hovering policy using reinforcement meta-learning. The policy maps flash LIDAR measurements directly to on/off spacecraft body-frame thrust commands, allowing hovering at a fixed position and attitude in the asteroid body-fixed reference frame. Importantly, the policy does not require position and velocity estimates, and can operate in environments with unknown dynamics, and without an asteroid shape model or navigation aids. Indeed, during optimization the agent is confronted with a new randomly generated asteroid for each episode, insuring that it does not learn an asteroid's shape, texture, or environmental dynamics. This allows the deployed policy to generalize well to novel asteroid characteristics, which we demonstrate in our experiments. Moreover, our experiments show that the optimized policy adapts to actuator failure and sensor noise. Although the policy is optimized using randomly generated synthetic asteroids, it is tested on two shape models from actual asteroids: Bennu and Itokawa. We find that the policy generalizes well to these shape models. The hovering controller has the potential to simplify mission planning by allowing asteroid body-fixed hovering immediately upon the spacecraft's arrival to an asteroid. This in turn simplifies shape model generation and allows resource mapping via remote sensing immediately upon arrival at the target asteroid.

eess.SY↗

arXiv2019

Space Objects Maneuvering Prediction via Maximum Causal Entropy Inverse Reinforcement Learning

Inverse Reinforcement Learning (RL) can be used to determine the behavior of Space Objects (SOs) by estimating the reward function that an SO is using for control. The approach discussed in this work can be used to analyze maneuvering of SOs from observational data. The inverse RL problem is solved using maximum causal entropy. This approach determines the optimal reward function that a SO is using while maneuvering with random disturbances by assuming that the observed trajectories are optimal with respect to the SO's own reward function. Lastly, this paper develops results for scenarios involving Low Earth Orbit (LEO) station-keeping and Geostationary Orbit (GEO) station-keeping.

eess.SY↗

arXiv2019

Reinforcement Learning for Angle-Only Intercept Guidance of Maneuvering Targets

We present a novel guidance law that uses observations consisting solely of seeker line of sight angle measurements and their rate of change. The policy is optimized using reinforcement meta-learning and demonstrated in a simulated terminal phase of a mid-course exo-atmospheric interception. Importantly, the guidance law does not require range estimation, making it particularly suitable for passive seekers. The optimized policy maps stabilized seeker line of sight angles and their rate of change directly to commanded thrust for the missile's divert thrusters. The use of reinforcement meta-learning allows the optimized policy to adapt to target acceleration, and we demonstrate that the policy performs as well as augmented zero-effort miss guidance with perfect target acceleration knowledge. The optimized policy is computationally efficient and requires minimal memory, and should be compatible with today's flight processors.

eess.SY↗

arXiv2019

Real-Time Thermospheric Density Estimation Via Two-Line-Element Data Assimilation

Inaccurate estimates of the thermospheric density are a major source of error in low Earth orbit prediction. To improve orbit prediction, real-time density estimation is required. In this work, we develop a reduced-order dynamic model for the thermospheric density by computing the main spatial modes of the atmosphere and deriving a linear model for the dynamics. The model is then used to estimate the density using two-line element (TLE) data by simultaneously estimating the reduced-order modes and the orbits and ballistic coefficients of several objects using an unscented Kalman filter. Accurate density estimation using the TLEs of 17 objects is demonstrated and validated against CHAMP and GRACE accelerometer-derived densities. Finally, the use of the model for density forecasting is shown.

physics.space-ph↗

arXiv2019

Seeker based Adaptive Guidance via Reinforcement Meta-Learning Applied to Asteroid Close Proximity Operations

Current practice for asteroid close proximity maneuvers requires extremely accurate characterization of the environmental dynamics and precise spacecraft positioning prior to the maneuver. This creates a delay of several months between the spacecraft's arrival and the ability to safely complete close proximity maneuvers. In this work we develop an adaptive integrated guidance, navigation, and control system that can complete these maneuvers in environments with unknown dynamics, with initial conditions spanning a large deployment region, and without a shape model of the asteroid. The system is implemented as a policy optimized using reinforcement meta-learning. The spacecraft is equipped with an optical seeker that locks to either a terrain feature, back-scattered light from a targeting laser, or an active beacon, and the policy maps observations consisting of seeker angles and LIDAR range readings directly to engine thrust commands. The policy implements a recurrent network layer that allows the deployed policy to adapt real time to both environmental forces acting on the agent and internal disturbances such as actuator failure and center of mass variation. We validate the guidance system through simulated landing maneuvers in a six degrees-of-freedom simulator. The simulator randomizes the asteroid's characteristics such as solar radiation pressure, density, spin rate, and nutation angle, requiring the guidance and control system to adapt to the environment. We also demonstrate robustness to actuator failure, sensor bias, and changes in the spacecraft's center of mass and inertia tensor. Finally, we suggest a concept of operations for asteroid close proximity maneuvers that is compatible with the guidance system.

eess.SY↗

arXiv2019

Combining Parameter Identification and Trajectory Optimization: Real-time Planning for Information Gain

Robotic systems often operate with uncertainties in their dynamics, for example, unknown inertial properties. Broadly, there are two approaches for controlling uncertain systems: design robust controllers in spite of uncertainty, or characterize a system before attempting to control it. This paper proposes a middle-ground approach, making trajectory progress while also accounting for gaining information about the system. More specifically, it combines excitation trajectories which are usually intended to optimize information gain for an estimator, with goal-driven trajectory optimization metrics. For this purpose, a measure of information gain is incorporated (using the Fisher Information Matrix) in a real-time planning framework to produce trajectories favorable for estimation. At the same time, the planner receives stable parameter updates from the estimator, enhancing the system model. An implementation of this learn-as-you-go approach utilizing an Unscented Kalman Filter (UKF) and Nonlinear Model Predictive Controller (NMPC) is demonstrated in simulation. Results for cases with and without information gain and online parameter updates in the system model are presented.

cs.RO↗

arXiv2019

Adaptive Guidance and Integrated Navigation with Reinforcement Meta-Learning

This paper proposes a novel adaptive guidance system developed using reinforcement meta-learning with a recurrent policy and value function approximator. The use of recurrent network layers allows the deployed policy to adapt real time to environmental forces acting on the agent. We compare the performance of the DR/DV guidance law, an RL agent with a non-recurrent policy, and an RL agent with a recurrent policy in four challenging environments with unknown but highly variable dynamics. These tasks include a safe Mars landing with random engine failure and a landing on an asteroid with unknown environmental dynamics. We also demonstrate the ability of a RL meta-learning optimized policy to implement a guidance law using observations consisting of only Doppler radar altimeter readings in a Mars landing environment, and LIDAR altimeter readings in an asteroid landing environment, thus integrating guidance and navigation.

eess.SY↗

arXiv2019

Learning Accurate Extended-Horizon Predictions of High Dimensional Trajectories

We present a novel predictive model architecture based on the principles of predictive coding that enables open loop prediction of future observations over extended horizons. There are two key innovations. First, whereas current methods typically learn to make long-horizon open-loop predictions using a multi-step cost function, we instead run the model open loop in the forward pass during training. Second, current predictive coding models initialize the representation layer's hidden state to a constant value at the start of an episode, and consequently typically require multiple steps of interaction with the environment before the model begins to produce accurate predictions. Instead, we learn a mapping from the first observation in an episode to the hidden state, allowing the trained model to immediately produce accurate predictions. We compare the performance of our architecture to a standard predictive coding model and demonstrate the ability of the model to make accurate long horizon open-loop predictions of simulated Doppler radar altimeter readings during a six degree of freedom Mars landing. Finally, we demonstrate a 2X reduction in sample complexity by using the model to implement a Dyna style algorithm to accelerate policy learning with proximal policy optimization.

cs.LG↗

arXiv2019

Adaptive Guidance with Reinforcement Meta-Learning

This paper proposes a novel adaptive guidance system developed using reinforcement meta-learning with a recurrent policy and value function approximator. The use of recurrent network layers allows the deployed policy to adapt real time to environmental forces acting on the agent. We compare the performance of the DR/DV guidance law, an RL agent with a non-recurrent policy, and an RL agent with a recurrent policy in four difficult tasks with unknown but highly variable dynamics. These tasks include a safe Mars landing with random engine failure and a landing on an asteroid with unknown environmental dynamics. We also demonstrate the ability of a recurrent policy to navigate using only Doppler radar altimeter returns, thus integrating guidance and navigation.

eess.SY↗