arXiv subjects

Adam J. Thorpe

Publications and source records attributed to Adam J. Thorpe.

25 records · Page 2Linked to original sources

arXiv2021

State-Based Confidence Bounds for Data-Driven Stochastic Reachability Using Hilbert Space Embeddings

In this paper, we compute finite sample bounds for data-driven approximations of the solution to stochastic reachability problems. Our approach uses a nonparametric technique known as kernel distribution embeddings, and provides probabilistic assurances of safety for stochastic systems in a model-free manner. By implicitly embedding the stochastic kernel of a Markov control process in a reproducing kernel Hilbert space, we can approximate the safety probabilities for stochastic systems with arbitrary stochastic disturbances as simple matrix operations and inner products. We present finite sample bounds for point-based approximations of the safety probabilities through construction of probabilistic confidence bounds that are state- and input-dependent. One advantage of this approach is that the bounds are responsive to non-uniformly sampled data, meaning that tighter bounds are feasible in regions of the state- and input-space with more observations. We numerically evaluate the approach, and demonstrate its efficacy on a neural network-controlled pendulum system.

math.OC↗

arXiv2021

Stochastic Optimal Control via Hilbert Space Embeddings of Distributions

Kernel embeddings of distributions have recently gained significant attention in the machine learning community as a data-driven technique for representing probability distributions. Broadly, these techniques enable efficient computation of expectations by representing integral operators as elements in a reproducing kernel Hilbert space. We apply these techniques to the area of stochastic optimal control theory and present a method to compute approximately optimal policies for stochastic systems with arbitrary disturbances. Our approach reduces the optimization problem to a linear program, which can easily be solved via the Lagrangian dual, without resorting to gradient-based optimization algorithms. We focus on discrete-time dynamic programming, and demonstrate our proposed approach on a linear regulation problem, and on a nonlinear target tracking problem. This approach is broadly applicable to a wide variety of optimal control problems, and provides a means of working with stochastic systems in a data-driven setting.

math.OC↗

arXiv2021

SReachTools Kernel Module: Data-Driven Stochastic Reachability Using Hilbert Space Embeddings of Distributions

We present algorithms for performing data-driven stochastic reachability as an addition to SReachTools, an open-source stochastic reachability toolbox. Our method leverages a class of machine learning techniques known as kernel embeddings of distributions to approximate the safety probabilities for a wide variety of stochastic reachability problems. By representing the probability distributions of the system state as elements in a reproducing kernel Hilbert space, we can learn the "best fit" distribution via a simple regularized least-squares problem, and then compute the stochastic reachability safety probabilities as simple linear operations. This technique admits finite sample bounds and has known convergence in probability. We implement these methods as part of SReachTools, and demonstrate their use on a double integrator system, on a million-dimensional repeated planar quadrotor system, and a cart-pole system with a black-box neural network controller.

math.OC↗

arXiv2020

Learning Approximate Forward Reachable Sets Using Separating Kernels

We present a data-driven method for computing approximate forward reachable sets using separating kernels in a reproducing kernel Hilbert space. We frame the problem as a support estimation problem, and learn a classifier of the support as an element in a reproducing kernel Hilbert space using a data-driven approach. Kernel methods provide a computationally efficient representation for the classifier that is the solution to a regularized least squares problem. The solution converges almost surely as the sample size increases, and admits known finite sample bounds. This approach is applicable to stochastic systems with arbitrary disturbances and neural network verification problems by treating the network as a dynamical system, or by considering neural network controllers as part of a closed-loop system. We present our technique on several examples, including a spacecraft rendezvous and docking problem, and two nonlinear system benchmarks with neural network controllers.

math.OC↗

arXiv2020

Approximate Stochastic Reachability for High Dimensional Systems

We present a method to compute the stochastic reachability safety probabilities for high-dimensional stochastic dynamical systems. Our approach takes advantage of a nonparametric learning technique known as conditional distribution embeddings to model the stochastic kernel using a data-driven approach. By embedding the dynamics and uncertainty within a reproducing kernel Hilbert space, it becomes possible to compute the safety probabilities for stochastic reachability problems as simple matrix operations and inner products. We employ a convergent approximation technique, random Fourier features, in order to alleviate the increased computational requirements for high-dimensional systems. This technique avoids the curse of dimensionality, and enables the computation of safety probabilities for high-dimensional systems without prior knowledge of the structure of the dynamics or uncertainty. We validate this approach on a double integrator system, and demonstrate its capabilities on a million-dimensional, nonlinear, non-Gaussian, repeated planar quadrotor system.

eess.SY↗

arXiv2020

Trust-based user-interface design for human-automation systems

We present a method for dynamics-driven, user-interface design for a human-automation system via sensor selection. We define the user-interface to be the output of a MIMO LTI system, and formulate the design problem as one of selecting an output matrix from a given set of candidate output matrices. Sufficient conditions for situation awareness are captured as additional constraints on the selection of the output matrix. These constraints depend upon the level of trust the human has in the automation. We show that the resulting user-interface design problem is a combinatorial, set-cardinality minimization problem with set function constraints. We propose tractable algorithms to compute optimal or sub-optimal solutions with suboptimality bounds. Our approaches exploit monotonicity and submodularity present in the design problem, and rely on constraint programming and submodular maximization. We apply this method to the IEEE 118-bus, to construct correct-by-design interfaces under various operating scenarios.

eess.SY↗

arXiv2020

Model-Free Stochastic Reachability Using Kernel Distribution Embeddings

We present a solution to the terminal-hitting stochastic reach-avoid problem for a Markov control process. This solution takes advantage of a nonparametric representation of the stochastic kernel as a conditional distribution embedding within a reproducing kernel Hilbert space (RKHS). Because the disturbance is modeled as a data-driven stochastic process, this representation avoids intractable integrals in the dynamic recursion of the reach-avoid problem since the expectations can be calculated as an inner product within the RKHS. We demonstrate this approach on a high-dimensional chain of integrators and on Clohessy-Wiltshire-Hill dynamics.

math.OC↗