arXiv subjects

Panagiotis Tsiotras

Publications and source records attributed to Panagiotis Tsiotras.

At least 109 records · Page 6Linked to original sources

arXiv2022

Hierarchical Decompositions of Stochastic Pursuit-Evasion Games

In this work we present a hierarchical framework for solving discrete stochastic pursuit-evasion games (PEGs) in large grid worlds. With a partition of the grid world into superstates (e.g., "rooms"), the proposed approach creates a two-resolution decision-making process, which consists of a set of local PEGs at the original state level and an aggregated PEG at the superstate level. Having much smaller cardinality, both the local games and the aggregated game can be easily solved to a Nash equilibrium. To connect the decision-making at the two resolutions, we use the Nash values of the local PEGs as the rewards for the aggregated game. Through numerical simulations, we show that the proposed hierarchical framework significantly reduces the computation overhead, while still maintaining a satisfactory level of performance when competing against the flat Nash policies.

cs.MA↗

arXiv2022

A Linear Programming Approach for Resource-Aware Information-Theoretic Tree Abstractions

In this chapter, an integer linear programming formulation for the problem of obtaining task-relevant, multi-resolution, environment abstractions for resource-constrained autonomous agents is presented. The formulation leverages concepts from information-theoretic signal compression, specifically, the information bottleneck (IB) method, to pose an abstraction problem as an optimal encoder search over the space of multi-resolution trees. The abstractions emerge in a task-relevant manner as a function of agent information-processing constraints. We detail our formulation, and show how hierarchical tree structures, signal encoders, and information-theoretic methods for signal compression can be unified under a common theme. A discussion delineating the benefits and drawbacks of our formulation is presented, as well as a detailed explanation how our approach can be interpreted within the context of generating abstractions for resource-constrained autonomous systems. It is shown that the resulting information-theoretic abstraction problem over the space of multi-resolution trees can be formulated as a integer linear programming (ILP) problem. We demonstrate the approach on a number of examples, and provide a discussion detailing the differences of the proposed framework compared to existing methods. Lastly, we consider a linear program relaxation of the ILP problem, thereby demonstrating that multi-resolution information-theoretic tree abstractions can be obtained by solving a convex program.

cs.IT↗

arXiv2022

AstroVision: Towards Autonomous Feature Detection and Description for Missions to Small Bodies Using Deep Learning

Missions to small celestial bodies rely heavily on optical feature tracking for characterization of and relative navigation around the target body. While deep learning has led to great advancements in feature detection and description, training and validating data-driven models for space applications is challenging due to the limited availability of large-scale, annotated datasets. This paper introduces AstroVision, a large-scale dataset comprised of 115,970 densely annotated, real images of 16 different small bodies captured during past and ongoing missions. We leverage AstroVision to develop a set of standardized benchmarks and conduct an exhaustive evaluation of both handcrafted and data-driven feature detection and description methods. Next, we employ AstroVision for end-to-end training of a state-of-the-art, deep feature detection and description network and demonstrate improved performance on multiple benchmarks. The full benchmarking pipeline and the dataset will be made publicly available to facilitate the advancement of computer vision algorithms for space applications.

astro-ph.IM↗

arXiv2022

Shaping Large Population Agent Behaviors Through Entropy-Regularized Mean-Field Games

Mean-field games (MFG) were introduced to efficiently analyze approximate Nash equilibria in large population settings. In this work, we consider entropy-regularized mean-field games with a finite state-action space in a discrete time setting. We show that entropy regularization provides the necessary regularity conditions, that are lacking in the standard finite mean field games. Such regularity conditions enable us to design fixed-point iteration algorithms to find the unique mean-field equilibrium (MFE). Furthermore, the reference policy used in the regularization provides an extra parameter, through which one can control the behavior of the population. We first consider a stochastic game with a large population of $N$ homogeneous agents. We establish conditions for the existence of a Nash equilibrium in the limiting case as $N$ tends to infinity, and we demonstrate that the Nash equilibrium for the infinite population case is also an $ε$-Nash equilibrium for the $N$-agent system, where the sub-optimality $ε$ is of order $\mathcal{O}\big(1/\sqrt{N}\big)$. Finally, we verify the theoretical guarantees through a resource allocation example and demonstrate the efficacy of using a reference policy to control the behavior of a large population.

cs.GT↗

arXiv2022

Trajectory Distribution Control for Model Predictive Path Integral Control using Covariance Steering

This paper presents a novel control approach for autonomous systems operating under uncertainty. We combine Model Predictive Path Integral (MPPI) control with Covariance Steering (CS) theory to obtain a robust controller for general nonlinear systems. The proposed Covariance-Controlled Model Predictive Path Integral (CC-MPPI) controller addresses the performance degradation observed in some MPPI implementations owing to unexpected disturbances and uncertainties. Namely, in cases where the environment changes too fast or the simulated dynamics during the MPPI rollouts do not capture the noise and uncertainty in the actual dynamics, the baseline MPPI implementation may lead to divergence. The proposed CC-MPPI controller avoids divergence by controlling the dispersion of the rollout trajectories at the end of the prediction horizon. Furthermore, the CC-MPPI has adjustable trajectory sampling distributions that can be changed according to the environment to achieve efficient sampling. Numerical examples using a ground vehicle navigating in challenging environments demonstrate the proposed approach.

cs.RO↗

arXiv2022

CBS-Budget (CBSB): A Complete and Bounded Suboptimal Search for Multi-Agent Path Finding

Multi-Agent Path Finding (MAPF) is the problem of finding a collection of collision-free paths for a team of multiple agents while minimizing some global cost, such as the sum of the time travelled by all agents, or the time travelled by the last agent. Conflict Based Search (CBS) is a leading complete and optimal MAPF solver which lazily explores the joint agent state space, using an admissible heuristic joint plan. Such an admissible heuristic joint plan is computed by combining individual shortest paths found without considering inter-agent conflicts, and which becomes gradually more informed as constraints are added to individual agents' path planning problems to avoid discovered conflicts. In this paper, we seek to speedup CBS by finding a more informed heuristic joint plan which is bounded from above. We first propose the budgeted Class-Ordered A* (bCOA*), a novel algorithm that finds the shortest path with minimal number of conflicts that is upper bounded in terms of length. Then, we propose a novel bounded-cost variant of CBS, called CBS-Budget (CBSB) by using a bCOA* search at the low-level search of the CBS and by using a modified focal search at the high-level search of the CBS. We prove that CBSB is complete and bounded-suboptimal. In our numerical experiments, CBSB finds a near optimal solution for hundreds of agents within a fraction of a second. CBSB shows state-of-the-art performance, comparable to Explicit Estimation CBS (EECBS), an enhanced recent version of CBS. On the other hand, CBSB is easier to implement than EECBS, since only two priority queues at the high-level search are needed as in Enhanced CBS (ECBS).

cs.MA↗

arXiv2022

CARNet: A Dynamic Autoencoder for Learning Latent Dynamics in Autonomous Driving Tasks

Autonomous driving has received a lot of attention in the automotive industry and is often seen as the future of transportation. Passenger vehicles equipped with a wide array of sensors (e.g., cameras, front-facing radars, LiDARs, and IMUs) capable of continuous perception of the environment are becoming increasingly prevalent. These sensors provide a stream of high-dimensional, temporally correlated data that is essential for reliable autonomous driving. An autonomous driving system should effectively use the information collected from the various sensors in order to form an abstract description of the world and maintain situational awareness. Deep learning models, such as autoencoders, can be used for that purpose, as they can learn compact latent representations from a stream of incoming data. However, most autoencoder models process the data independently, without assuming any temporal interdependencies. Thus, there is a need for deep learning models that explicitly consider the temporal dependence of the data in their architecture. This work proposes CARNet, a Combined dynAmic autoencodeR NETwork architecture that utilizes an autoencoder combined with a recurrent neural network to learn the current latent representation and, in addition, also predict future latent representations in the context of autonomous driving. We demonstrate the efficacy of the proposed model in both imitation and reinforcement learning settings using both simulated and real datasets. Our results show that the proposed model outperforms the baseline state-of-the-art model, while having significantly fewer trainable parameters.

cs.LG↗

arXiv2022

Simultaneous Control and Trajectory Estimation for Collision Avoidance of Autonomous Robotic Spacecraft Systems

We propose factor graph optimization for simultaneous planning, control, and trajectory estimation for collision-free navigation of autonomous systems in environments with moving objects. The proposed online probabilistic motion planning and trajectory estimation navigation technique generates optimal collision-free state and control trajectories for autonomous vehicles when the obstacle motion model is both unknown and known. We evaluate the utility of the algorithm to support future autonomous robotic space missions.

cs.RO↗

arXiv2022

Chance-Constrained Covariance Steering in a Gaussian Random Field via Successive Convex Programming

The problem of optimizing affine feedback laws that explicitly steer the mean and covariance of an uncertain system state in the presence of a Gaussian random field is considered. Spatially-dependent disturbances are successively approximated with respect to a nominal trajectory by a sequence of jointly Gaussian random vectors. Sequential updates to the nominal control inputs are computed via convex optimization that includes the effect of affine state feedback, the perturbing effects of spatial disturbances, and chance constraints on the closed-loop state and control. The developed method is applied to solve for an affine feedback law to minimize the 99th percentile of $Δv$ required to complete an aerocapture mission around a planet with a randomly disturbed atmosphere.

eess.SY↗

arXiv2022

Stochastic Entry Guidance

In this paper, closed-loop entry guidance in a randomly perturbed atmosphere, using bank angle control, is posed as a stochastic optimal control problem. The entry trajectory, as well as the closed-loop controls, are both modeled as random processes with statistics determined by the entry dynamics, the entry guidance, and the probabilistic structure of altitude-dependent atmospheric density variations. The entry guidance, which is parameterized as a sequence of linear feedback gains, is designed to steer the probability distribution of the entry trajectories while satisfying bounds on the allowable control inputs and on the maximum allowable state errors. Numerical simulations of a Mars entry scenario demonstrate improved range targeting performance with approximately 50% lower 1st and 99th percentile final range errors when using the developed stochastic guidance scheme as compared to the existing Apollo final phase algorithm.

eess.SY↗

arXiv2021

Optimal Controller and Quantizer Selection for Partially Observable Linear-Quadratic-Gaussian Systems

In networked control systems, often the sensory signals are quantized before being transmitted to the controller. Consequently, performance is affected by the coarseness of this quantization process. Modern communication technologies allow users to obtain resolution-varying quantized measurements based on the prices paid. In this paper, we consider joint optimal controller synthesis and quantizer scheduling for a partially observed Quantized-Feedback Linear-Quadratic-Gaussian (QF-LQG) system, where the measurements are quantized before being sent to the controller. The system is presented with several choices of quantizers, along with the cost of using each quantizer. The objective is to jointly select the quantizers and synthesize the controller to strike an optimal balance between control performance and quantization cost. When the innovation signal is quantized instead of the measurement, the problem is decoupled into two optimization problems: one for optimal controller synthesis, and the other for optimal quantizer selection. The optimal controller is found by solving a Riccati equation and the optimal quantizer selection policy is found by solving a linear program (LP)- both of which can be solved offline.

eess.SY↗

arXiv2021

Distribution Steering for Discrete-Time Linear Systems with General Disturbances using Characteristic Functions

We propose to solve a constrained distribution steering problem, i.e., steering a stochastic linear system from an initial distribution to some final, desired distribution subject to chance constraints. We do so by characterizing the cumulative distribution function in the chance constraints and by using the absolute distance between two probability density functions using the corresponding characteristic functions. We consider discrete-time, time-varying linear systems with affine feedback. We demonstrate the proposed approach on a 2D double-integrator perturbed by various disturbances and initial conditions.

math.OC↗

arXiv2021

Value Function Estimators for Feynman-Kac Forward-Backward SDEs in Stochastic Optimal Control

Two novel numerical estimators are proposed for solving forward-backward stochastic differential equations (FBSDEs) appearing in the Feynman-Kac representation of the value function in stochastic optimal control problems. In contrast to the current numerical approaches which are based on the discretization of the continuous-time FBSDE, we propose a converse approach, namely, we obtain a discrete-time approximation of the on-policy value function, and then we derive a discrete-time estimator that resembles the continuous-time counterpart. The proposed approach allows for the construction of higher accuracy estimators along with error analysis. The approach is applied to the policy improvement step in reinforcement learning. Numerical results and error analysis are demonstrated using (i) a scalar nonlinear stochastic optimal control problem and (ii) a four-dimensional linear quadratic regulator (LQR) problem. The proposed estimators show significant improvement in terms of accuracy in both cases over Euler-Maruyama-based estimators used in competing approaches. In the case of LQR problems, we demonstrate that our estimators result in near machine-precision level accuracy, in contrast to previously proposed methods that can potentially diverge on the same problems.

math.OC↗

arXiv2021

Accelerating Kinodynamic RRT* Through Dimensionality Reduction

Sampling-based motion planning algorithms such as RRT* are well-known for their ability to quickly find an initial solution and then converge to the optimal solution asymptotically. However, the convergence rate can be slow for highdimensional planning problems, particularly for dynamical systems where the sampling space is not just the configuration space but the full state space. In this paper, we introduce the idea of using a partial-final-state-free (PFF) optimal controller in kinodynamic RRT* [1] to reduce the dimensionality of the sampling space. Instead of sampling the full state space, the proposed accelerated kinodynamic RRT*, called Kino-RRT*, only samples part of the state space, while the rest of the states are selected by the PFF optimal controller. We also propose a delayed and intermittent update of the optimal arrival time of all the edges in the RRT* tree to decrease the computation complexity of the algorithm. We tested the proposed algorithm using 4-D and 10-D state-space linear systems and showed that Kino-RRT* converges much faster than the kinodynamic RRT* algorithm.

cs.RO↗

arXiv2021

Learning Nash Equilibria in Zero-Sum Stochastic Games via Entropy-Regularized Policy Approximation

We explore the use of policy approximations to reduce the computational cost of learning Nash equilibria in zero-sum stochastic games. We propose a new Q-learning type algorithm that uses a sequence of entropy-regularized soft policies to approximate the Nash policy during the Q-function updates. We prove that under certain conditions, by updating the regularized Q-function, the algorithm converges to a Nash equilibrium. We also demonstrate the proposed algorithm's ability to transfer previous training experiences, enabling the agents to adapt quickly to new environments. We provide a dynamic hyper-parameter scheduling scheme to further expedite convergence. Empirical results applied to a number of stochastic games verify that the proposed algorithm converges to the Nash equilibrium, while exhibiting a major speed-up over existing algorithms.

cs.LG↗

arXiv2021

Lazy Lifelong Planning for Efficient Replanning in Graphs with Expensive Edge Evaluation

We present an incremental search algorithm, called Lifelong-GLS, which combines the vertex efficiency of Lifelong Planning A* (LPA*) and the edge efficiency of Generalized Lazy Search (GLS) for efficient replanning on dynamic graphs where edge evaluation is expensive. We use a lazily evaluated LPA* to repair the cost-to-come inconsistencies of the relevant region of the current search tree based on the previous search results, and then we restrict the expensive edge evaluations only to the current shortest subpath as in the GLS framework. The proposed algorithm is complete and correct in finding the optimal solution in the current graph, if one exists. We also show that the search returns a bounded suboptimal solution, if an inflated heuristic edge weight is used and the tree repairing propagation is truncated early for faster search. Finally, we show the efficiency of the proposed algorithm compared to the standard LPA* and the GLS algorithms over consecutive search episodes in a dynamic environment. For each search, the proposed algorithm reduces the edge evaluations by a significant amount compared to the LPA*. Both the number of vertex expansions and the number of edge evaluations are reduced substantially compared to GLS, as the proposed algorithm utilizes previous search results to facilitate the new search.

cs.RO↗

arXiv2021

Belief Space Planning: A Covariance Steering Approach

A new belief space planning algorithm, called covariance steering Belief RoadMap (CS-BRM), is introduced, which is a multi-query algorithm for motion planning of dynamical systems under simultaneous motion and observation uncertainties. CS-BRM extends the probabilistic roadmap (PRM) approach to belief spaces and is based on the recently developed theory of covariance steering (CS) that enables guaranteed satisfaction of terminal belief constraints in finite-time. The nodes in the CS-BRM are sampled in belief space and represent distributions of the system states. A covariance steering controller steers the system from one BRM node to another, thus acting as an edge controller of the corresponding belief graph that ensures belief constraint satisfaction. After the edge controller is computed, a specific edge cost is assigned to that edge. The CS-BRM algorithm allows the sampling of non-stationary belief nodes, and thus is able to explore the velocity space and find efficient motion plans. The performance of CS-BRM is evaluated and compared to a previous belief space planning method, demonstrating the benefits of the proposed approach.

cs.RO↗

arXiv2021

Covariance Steering with Optimal Risk Allocation

This paper extends the optimal covariance steering problem for linear stochastic systems subject to chance constraints to account for optimal risk allocation. Previous works have assumed a uniform risk allocation to cast the optimal control problem as a semi-definite program (SDP), which can be solved efficiently using standard SDP solvers. We adopt an Iterative Risk Allocation (IRA) formalism, which uses a two-stage approach to solve the optimal risk allocation problem for covariance steering. The upper-stage of IRA optimizes the risk, which is proved to be a convex problem, while the lower-stage optimizes the controller with the new constraints. This is done iteratively so as to find the optimal risk allocation that achieves the lowest total cost. The proposed framework results in solutions that tend to maximize the terminal covariance, while still satisfying the chance constraints, thus leading to less conservative solutions than previous methodologies. We also introduce two novel convex relaxation methods to approximate quadratic chance constraints as second-order cone constraints. We finally demonstrate the approach to a spacecraft rendezvous problem and compare the results.

math.OC↗