arXiv subjects

Efstathios Bakolas

Publications and source records attributed to Efstathios Bakolas.

51 records · Page 3Linked to original sources

arXiv2021

Koopman Operator Based Modeling for Quadrotor Control on $SE(3)$

In this paper, we propose a Koopman operator based approach to describe the nonlinear dynamics of a quadrotor on SE(3) in terms of an infinite-dimensional linear system which evolves in the space of observable functions (lifted space) and which is more appropriate for control design purposes. The major challenge when using the Koopman operator is the characterization of a set of observable functions that can span the lifted space. Recent methods either use tools from machine learning to learn the observable functions or guess a suitable set of observables that best describes the nonlinear dynamics. Instead of guessing or learning the observables, in this work we derive them in a systematic way for the quadrotor dynamics on SE(3). In addition, we prove that the proposed sequence of observable functions converges pointwise to the zero function, which allows us to select only a finite set of observable functions to form (an approximation of) the lifted space. Our theoretical analysis is also confirmed by numerical simulations which demonstrate that by increasing the dimension of the lifted space, the derived linear state space model can approximate the nonlinear quadrotor dynamics more accurately.

eess.SY↗

arXiv2021

Nonlinear Covariance Steering using Variational Gaussian Process Predictive Models

In this work, we consider the problem of steering the first two moments of the uncertain state of an unknown discrete-time stochastic nonlinear system to a given terminal distribution in finite time. Toward that goal, first, a non-parametric predictive model is learned from a set of available training data points using stochastic variational Gaussian process regression: a powerful and scalable machine learning tool for learning distributions over arbitrary nonlinear functions. Second, we formulate a tractable nonlinear covariance steering algorithm that utilizes the Gaussian process predictive model to compute a feedback policy that will drive the distribution of the state of the system close to the goal distribution. In particular, we implement a greedy covariance steering control policy that linearizes at each time step the Gaussian process model around the latest predicted mean and covariance, solves the linear covariance steering control problem, and applies only the first control law. The state uncertainty under the latest feedback control policy is then propagated using the unscented transform with the learned Gaussian process predictive model and the algorithm proceeds to the next time step. Numerical simulations illustrating the main ideas of this paper are also presented.

math.OC↗

arXiv2021

Covariance Control of Discrete-Time Gaussian Linear Systems Using Affine Disturbance Feedback Control Policies

In this paper, we present a new control policy parametrization for the finite-horizon covariance steering problem for discrete-time Gaussian linear systems (DTGLS) which can reduce the latter stochastic optimal control problem to a tractable optimization problem. The covariance steering problem seeks for a feedback control policy that will steer the state covariance of a DTGLS to a desired positive definite matrix in finite time. We consider two different formulations of the covariance steering problem, one with hard terminal LMI constraints (hard-constrained covariance steering) and another one with soft terminal constraints in the form of a terminal cost which corresponds to the squared Wasserstein distance between the actual terminal state (Gaussian) distribution and the desired one (soft-constrained covariance steering). We propose a solution approach that relies on the affine disturbance feedback parametrization for both problem formulations. We show that this particular parametrization allows us to reduce the hard-constrained covariance steering problem into a semi-definite program (SDP) and the soft-constrained covariance steering problem into a difference of convex functions program(DCP). Finally, we show the advantages of our approach over other covariance steering algorithms in terms of computational complexity and computation time by means of theoretical analysis and numerical simulations.

math.OC↗

arXiv2021

On the Convexity of Discrete Time Covariance Steering in Stochastic Linear Systems with Wasserstein Terminal Cost

In this work, we analyze the properties of the solution to the covariance steering problem for discrete time Gaussian linear systems with a squared Wasserstein distance terminal cost. In our previous work, we have shown that by utilizing the state feedback control policy parametrization, this stochastic optimal control problem can be associated with a difference of convex functions program. Here, we revisit the same covariance control problem but this time we focus on the analysis of the problem. Specifically, we establish the existence of solutions to the optimization problem and derive the first and second order conditions for optimality. We provide analytic expressions for the gradient and the Hessian of the performance index by utilizing specialized tools from matrix calculus. Subsequently, we prove that the optimization problem always admits a global minimizer, and finally, we provide a sufficient condition for the performance index to be a strictly convex function (under the latter condition, the problem admits a unique global minimizer). In particular, we show that when the terminal state covariance is upper bounded, with respect to the Löwner partial order, by the covariance matrix of the desired terminal normal distribution, then our problem admits a unique global minimizing state feedback gain. The results of this paper set the stage for the development of specialized control design tools that exploit the structure of the solution to the covariance steering problem with a squared Wasserstein distance terminal cost.

math.OC↗

arXiv2020

Minimum Variance and Covariance Steering Based on Affine Disturbance Feedback Control Parameterization

The goal of this paper is to address finite-horizon minimum variance and covariance steering problems for discrete-time stochastic (Gaussian) linear systems. On the one hand, the minimum variance problem seeks for a control policy that will steer the state mean of an uncertain system to a prescribed quantity while minimizing the trace of its terminal state covariance (or variance). On the other hand, the covariance steering problem seeks for a control policy that will steer the covariance of the terminal state to a prescribed positive definite matrix. We propose a solution approach that relies on the stochastic version of the affine disturbance feedback control parametrization according to which the control input at each stage can be expressed as an affine function of the history of disturbances that have acted upon the system. Our analysis reveals that this particular parametrization allows one to reduce the stochastic optimal control problems considered herein into tractable convex programs with essentially the same decision variables. This is in contrast with other control policy parametrizations, such as the state feedback parametrization, in which the decision variables of the convex program do not coincide with the controller's parameters of the stochastic optimal control problem. In addition, we propose a variation of the control parametrization which relies on truncated histories of past disturbances. We show that by selecting the length of the truncated sequences appropriately, we can design suboptimal controllers which can strike the desired balance between performance and computational cost.

math.OC↗

arXiv2020

Greedy Finite-Horizon Covariance Steering for Discrete-Time Stochastic Nonlinear Systems Based on the Unscented Transform

In this work, we consider the problem of steering the first two moments of the uncertain state of a discrete time nonlinear stochastic system to prescribed goal quantities at a given final time. In principle, the latter problem can be formulated as a density tracking problem, which seeks for a feedback policy that will keep the probability density function of the state of the system close, in terms of an appropriate metric, to the goal density. The solution to the latter infinite-dimensional problem can be, however, a complex and computationally expensive task. Instead, we propose a more tractable and intuitive approach which relies on a greedy control policy. The latter control policy is comprised of the first elements of the control policies that solve a sequence of corresponding linearized covariance steering problems. Each of these covariance steering problems relies only on information available about the state mean and state covariance at the current stage and can be formulated as a tractable (finite-dimensional) convex program. At each stage, the information on the state statistics is updated by computing approximations of the predicted state mean and covariance of the resulting closed-loop nonlinear system at the next stage by utilizing the (scaled) unscented transform. Numerical simulations that illustrate the key ideas of our approach are also presented.

math.OC↗

arXiv2020

Relay Pursuit of an Evader by a Heterogeneous Group of Pursuers Using Potential Games

We propose a decentralized solution for a pursuit-evasion game involving a heterogeneous group of rational (selfish) pursuers and a single evader based on the framework of potential games. In the proposed game, the evader aims to delay (or, if possible, avoid) capture by any of the pursuers whereas each pursuer tries to capture the latter only if this is to his best interest. Our approach resembles in principle the so-called relay pursuit strategy introduced in [1], in which only the pursuer that can capture the evader faster than the others is active. In sharp contrast with the latter approach, the active pursuer herein is not determined by a reactive ad-hoc rule but from the solution of a corresponding potential game. We assume that each pursuer has different capabilities and his decision whether to go after the evader or not is based on the maximization of his individual utility (conditional on the choices and actions of the other pursuers). The pursuers' utilities depend on both the rewards that they will receive by capturing the evader and the time of capture (cost of capturing the evader) so that a pursuer should only seek capture when the incurred cost is relatively small. The determination of the active pursuer-evader assignments (in other words, which pursuers should be active) is done iteratively by having the pursuers exchange information and updating their own actions by executing a learning algorithm for games known as Spatial Adaptive Play (SAP). We illustrate the performance of our algorithm by means of extensive numerical simulations.

eess.SY↗

arXiv2020

Far-Field Minimum-Fuel Spacecraft Rendezvous using Koopman Operator and $\ell_2/\ell_1$ Optimization

We propose a method to compute approximate solutions to the minimum-fuel far-field rendezvous problem for thrust-vectoring spacecraft. It is well-known that the use of linearized spacecraft rendezvous equations may not give sufficiently accurate results for far-field rendezvous. In particular, as the distance between the active and the target spacecraft becomes significantly greater than the distance between the target spacecraft and the center of gravity of the planet, the accuracy of linearization-based control design approaches may decline substantially. In this paper, we use a nonlinear state space model which corresponds to more accurate description of dynamics than linearized models but at the same time poses the known challenges of nonlinear control design. To overcome these challenges, we utilize a Koopman operator based approach with which the nonlinear spacecraft rendezvous dynamics is lifted into a higher dimensional space over which the nonlinear dynamics can be approximated by a linear system which is more suitable for control design purposes than the original nonlinear model. An Iteratively Recursive Least Squares (IRLS) algorithm from compressive sensing is then used to solve the minimum fuel control problem based on the lifted linear system. Numerical simulations are performed to show the efficacy of the proposed Koopman operator based approach.

eess.SY↗

arXiv2020

Covariance Steering of Discrete-Time Stochastic Linear Systems Based on Distribution Distance Terminal Costs

We consider a class of stochastic optimal control problems for discrete-time stochastic linear systems which seek for control policies that will steer the probability distribution of the terminal state of the system close to a desired Gaussian distribution. In our problem formulation, the closeness between the terminal state distribution and the desired (goal) distribution is measured in terms of the squared Wasserstein distance which is associated with a corresponding terminal cost term. We recast the stochastic optimal control problem as a finite-dimensional nonlinear program and we show that its performance index can be expressed as the difference of two convex functions. This representation of the performance index allows us to find local minimizers of the original nonlinear program via the so-called convex-concave procedure [1]. Subsequently, we consider a similar problem but this time we use a terminal cost that corresponds to the KL divergence. Finally, we present non-trivial numerical simulations to demonstrate the proposed techniques and compare them in terms of computation time.

math.OC↗

arXiv2020

Decentralized Game-Theoretic Control for Dynamic Task Allocation Problems for Multi-Agent Systems

We propose a decentralized game-theoretic framework for dynamic task allocation problems for multi-agent systems. In our problem formulation, the agents' utilities depend on both the rewards and the costs associated with the successful completion of the tasks assigned to them. The rewards reflect how likely is for the agents to accomplish their assigned tasks whereas the costs reflect the effort needed to complete these tasks (this effort is determined by the solution of corresponding optimal control problems). The task allocation problem considered herein corresponds to a dynamic game whose solution depends on the states of the agents in contrast with classic static (or single-act) game formulations. We propose a greedy solution approach in which the agents negotiate with each other to find a mutually agreeable (or individually rational) task assignment profile based on evaluations of the task utilities that reflect their current states. We illustrate the main ideas of this work by means of extensive numerical simulations.

cs.MA↗

arXiv2020

MPC-Based Hierarchical Task Space Control of Underactuated and Constrained Robots for Execution of Multiple Tasks

This paper proposes an MPC-based controller to efficiently execute multiple hierarchical tasks for underactuated and constrained robotic systems. Existing task-space controllers or whole-body controllers solve instantaneous optimization problems given task trajectories and the robot plant dynamics. However, the task-space control method we propose here relies on the prediction of future state trajectories and the corresponding costs-to-go terms over a finite time-horizon for computing control commands. We employ acceleration energy error as the performance index for the optimization problem and extend it over the finite-time horizon of our MPC. Our approach employs quadratically constrained quadratic programming, which includes quadratic constraints to handle multiple hierarchical tasks, and is computationally more efficient than nonlinear MPC-based approaches that rely on nonlinear programming. We validate our approach using numerical simulations of a new type of robot manipulator system, which contains underactuated and constrained mechanical structures.

cs.RO↗

arXiv2020

Workspace Partitioning and Topology Discovery Algorithms for Heterogeneous Multi-Agent Networks

In this paper, we consider a class of workspace partitioning problems that arise in the context of area coverage and spatial load balancing for spatially distributed heterogeneous multi-agent networks. It is assumed that each agent has certain directions of motion or directions for sensing and exploration that are more preferable than others. These preferences are measured by means of convex and anisotropic (direction-dependent) quadratic proximity metrics which are, in general, different for each agent. These proximity metrics induce Voronoi-like partitions of the network's workspace that are comprised of cells which may not always be convex (or even connected) sets but are necessarily contained in ellipsoids that are known to their corresponding agents. The main contributions of this work are 1) a distributed algorithm for the computation of a Voronoi-like partition of the workspace of a heterogeneous multi-agent network and 2) a systematic process to discover the network topology induced by the latter Voronoi-like partition. Numerical simulations that illustrate the efficacy of the proposed algorithms are also presented.

eess.SY↗

arXiv2020

Min-Max Q-Learning for Multi-Player Pursuit-Evasion Games

In this paper, we address a pursuit-evasion game involving multiple players by utilizing tools and techniques from reinforcement learning and matrix game theory. In particular, we consider the problem of steering an evader to a goal destination while avoiding capture by multiple pursuers, which is a high-dimensional and computationally intractable problem in general. In our proposed approach, we first formulate the multi-agent pursuit-evasion game as a sequence of discrete matrix games. Next, in order to simplify the solution process, we transform the high-dimensional state space into a low-dimensional manifold and the continuous action space into a feature-based space, which is a discrete abstraction of the original space. Based on these transformed state and action spaces, we subsequently employ min-max Q-learning, to generate the entries of the payoff matrix of the game, and subsequently obtain the optimal action for the evader at each stage. Finally, we present extensive numerical simulations to evaluate the performance of the proposed learning-based evading strategy in terms of the evader's ability to reach the desired target location without being captured, as well as computational efficiency.

eess.SY↗

arXiv2019

Efficient Trajectory Generation for Robotic Systems Constrained by Contact Forces

In this work, we propose a trajectory generation method for robotic systems with contact force constraint based on optimal control and reachability analysis. Normally, the dynamics and constraints of the contact-constrained robot are nonlinear and coupled to each other. Instead of linearizing the model and constraints, we directly solve the optimal control problem to obtain the feasible state trajectory and the control input of the system. A tractable optimal control problem is formulated which is addressed by dual approaches, which are sampling-based dynamic programming and rigorous reachability analysis. The sampling-based method and Partially Observable Markov Decision Process (POMDP) are used to break down the end-to-end trajectory generation problem via sample-wise optimization in terms of given conditions. The result generates sequential pairs of subregions to be passed to reach the final goal. The reachability analysis ensures that we will find at least one trajectory starting from a given initial state and going through a sequence of subregions. The distinctive contributions of our method are to enable handling the intricate contact constraint coupled with system's dynamics due to the reduction of computational complexity of the algorithm. We validate our method using extensive numerical simulations with a legged robot.

cs.RO↗

arXiv2018

Trajectory Generation for Robotic Systems with Contact Force Constraints

This paper presents a trajectory generation method for contact-constrained robotic systems such as manipulators and legged robots. Contact-constrained systems are affected by the interaction forces between the robot and the environment. In turn, these forces determine and constrain state reachability of the robot parts or end effectors. Our study subdivides the trajectory generation problem and the supporting reachability analysis into tractable subproblems consisting of a sampling problem, a convex optimization problem, and a nonlinear programming problem. Our method leads to significant reduction of computational cost. The proposed approach is validated using a realistic simulated contact-constrained robotic system.

cs.RO↗