arXiv subjects

Meeko M. K. Oishi

Publications and source records attributed to Meeko M. K. Oishi.

At least 19 recordsLinked to original sources

arXiv2024

Stochastic Reachability of Uncontrolled Systems via Probability Measures: Approximation via Deep Neural Networks

This paper poses a theoretical characterization of the stochastic reachability problem in terms of probability measures, capturing the probability measure of the state of the system that satisfies the reachability specification for all probabilities over a finite horizon. We achieve this by constructing the level sets of the probability measure for all probability values and, since our approach is only for autonomous systems, we can determine the level sets via forward simulations of the system from a point in the state space at some time step in the finite horizon to estimate the reach probability. We devise a training procedure which exploits this forward simulation and employ it to design a deep neural network (DNN) to predict the reach probability provided the current state and time step. We validate the effectiveness of our approach through three examples.

math.OC↗

arXiv2024

Forward Reachability for Discrete-Time Nonlinear Stochastic Systems via Mixed-Monotonicity and Stochastic Order

We present a method to overapproximate forward stochastic reach sets of discrete-time, stochastic nonlinear systems with interval geometry. This is made possible by extending the theory of mixed-monotone systems to incorporate stochastic orders, and a concentration inequality result that lower-bounds the probability the state resides within an interval through a monotone mapping. Then, we present an algorithm to compute the overapproximations of forward reachable set and the probability the state resides within it. We present our approach on two aerospace examples to show its efficacy.

eess.SY↗

arXiv2023

Physics-Informed Kernel Embeddings: Integrating Prior System Knowledge with Data-Driven Control

Data-driven control algorithms use observations of system dynamics to construct an implicit model for the purpose of control. However, in practice, data-driven techniques often require excessive sample sizes, which may be infeasible in real-world scenarios where only limited observations of the system are available. Furthermore, purely data-driven methods often neglect useful a priori knowledge, such as approximate models of the system dynamics. We present a method to incorporate such prior knowledge into data-driven control algorithms using kernel embeddings, a nonparametric machine learning technique based in the theory of reproducing kernel Hilbert spaces. Our proposed approach incorporates prior knowledge of the system dynamics as a bias term in the kernel learning problem. We formulate the biased learning problem as a least-squares problem with a regularization term that is informed by the dynamics, that has an efficiently computable, closed-form solution. Through numerical experiments, we empirically demonstrate the improved sample efficiency and out-of-sample generalization of our approach over a purely data-driven baseline. We demonstrate an application of our method to control through a target tracking problem with nonholonomic dynamics, and on spring-mass-damper and F-16 aircraft state prediction tasks.

eess.SY↗

arXiv2022

Probabilistic Verification of ReLU Neural Networks via Characteristic Functions

Verifying the input-output relationships of a neural network so as to achieve some desired performance specification is a difficult, yet important, problem due to the growing ubiquity of neural nets in many engineering applications. We use ideas from probability theory in the frequency domain to provide probabilistic verification guarantees for ReLU neural networks. Specifically, we interpret a (deep) feedforward neural network as a discrete dynamical system over a finite horizon that shapes distributions of initial states, and use characteristic functions to propagate the distribution of the input data through the network. Using the inverse Fourier transform, we obtain the corresponding cumulative distribution function of the output set, which can be used to check if the network is performing as expected given any random point from the input set. The proposed approach does not require distributions to have well-defined moments or moment generating functions. We demonstrate our proposed approach on two examples, and compare its performance to related approaches.

cs.LG↗

arXiv2022

Data-Driven Stochastic Optimal Control Using Kernel Gradients

We present an empirical, gradient-based method for solving data-driven stochastic optimal control problems using the theory of kernel embeddings of distributions. By embedding the integral operator of a stochastic kernel in a reproducing kernel Hilbert space, we can compute an empirical approximation of stochastic optimal control problems, which can then be solved efficiently using the properties of the RKHS. Existing approaches typically rely upon finite control spaces or optimize over policies with finite support to enable optimization. In contrast, our approach uses kernel-based gradients computed using observed data to approximate the cost surface of the optimal control problem, which can then be optimized using gradient descent. We apply our technique to the area of data-driven stochastic optimal control, and demonstrate our proposed approach on a linear regulation problem for comparison and on a nonlinear target tracking problem.

math.OC↗

arXiv2022

SOCKS: A Stochastic Optimal Control and Reachability Toolbox Using Kernel Methods

We present SOCKS, a data-driven stochastic optimal control toolbox based in kernel methods. SOCKS is a collection of data-driven algorithms that compute approximate solutions to stochastic optimal control problems with arbitrary cost and constraint functions, including stochastic reachability, which seeks to determine the likelihood that a system will reach a desired target set while respecting a set of pre-defined safety constraints. Our approach relies upon a class of machine learning algorithms based in kernel methods, a nonparametric technique which can be used to represent probability distributions in a high-dimensional space of functions known as a reproducing kernel Hilbert space. As a nonparametric technique, kernel methods are inherently data-driven, meaning that they do not place prior assumptions on the system dynamics or the structure of the uncertainty. This makes the toolbox amenable to a wide variety of systems, including those with nonlinear dynamics, black-box elements, and poorly characterized stochastic disturbances. We present the main features of SOCKS and demonstrate its capabilities on several benchmarks.

cs.LG↗

arXiv2022

Data-Driven Chance Constrained Control using Kernel Distribution Embeddings

We present a data-driven algorithm for efficiently computing stochastic control policies for general joint chance constrained optimal control problems. Our approach leverages the theory of kernel distribution embeddings, which allows representing expectation operators as inner products in a reproducing kernel Hilbert space. This framework enables approximately reformulating the original problem using a dataset of observed trajectories from the system without imposing prior assumptions on the parameterization of the system dynamics or the structure of the uncertainty. By optimizing over a finite subset of stochastic open-loop control trajectories, we relax the original problem to a linear program over the control parameters that can be efficiently solved using standard convex optimization techniques. We demonstrate our proposed approach in simulation on a system with nonlinear non-Markovian dynamics navigating in a cluttered environment.

eess.SY↗

arXiv2021

State-Based Confidence Bounds for Data-Driven Stochastic Reachability Using Hilbert Space Embeddings

In this paper, we compute finite sample bounds for data-driven approximations of the solution to stochastic reachability problems. Our approach uses a nonparametric technique known as kernel distribution embeddings, and provides probabilistic assurances of safety for stochastic systems in a model-free manner. By implicitly embedding the stochastic kernel of a Markov control process in a reproducing kernel Hilbert space, we can approximate the safety probabilities for stochastic systems with arbitrary stochastic disturbances as simple matrix operations and inner products. We present finite sample bounds for point-based approximations of the safety probabilities through construction of probabilistic confidence bounds that are state- and input-dependent. One advantage of this approach is that the bounds are responsive to non-uniformly sampled data, meaning that tighter bounds are feasible in regions of the state- and input-space with more observations. We numerically evaluate the approach, and demonstrate its efficacy on a neural network-controlled pendulum system.

math.OC↗

arXiv2021

Distribution Steering for Discrete-Time Linear Systems with General Disturbances using Characteristic Functions

We propose to solve a constrained distribution steering problem, i.e., steering a stochastic linear system from an initial distribution to some final, desired distribution subject to chance constraints. We do so by characterizing the cumulative distribution function in the chance constraints and by using the absolute distance between two probability density functions using the corresponding characteristic functions. We consider discrete-time, time-varying linear systems with affine feedback. We demonstrate the proposed approach on a 2D double-integrator perturbed by various disturbances and initial conditions.

math.OC↗

arXiv2021

Stochastic Optimal Control via Hilbert Space Embeddings of Distributions

Kernel embeddings of distributions have recently gained significant attention in the machine learning community as a data-driven technique for representing probability distributions. Broadly, these techniques enable efficient computation of expectations by representing integral operators as elements in a reproducing kernel Hilbert space. We apply these techniques to the area of stochastic optimal control theory and present a method to compute approximately optimal policies for stochastic systems with arbitrary disturbances. Our approach reduces the optimization problem to a linear program, which can easily be solved via the Lagrangian dual, without resorting to gradient-based optimization algorithms. We focus on discrete-time dynamic programming, and demonstrate our proposed approach on a linear regulation problem, and on a nonlinear target tracking problem. This approach is broadly applicable to a wide variety of optimal control problems, and provides a means of working with stochastic systems in a data-driven setting.

math.OC↗

arXiv2021

SReachTools Kernel Module: Data-Driven Stochastic Reachability Using Hilbert Space Embeddings of Distributions

We present algorithms for performing data-driven stochastic reachability as an addition to SReachTools, an open-source stochastic reachability toolbox. Our method leverages a class of machine learning techniques known as kernel embeddings of distributions to approximate the safety probabilities for a wide variety of stochastic reachability problems. By representing the probability distributions of the system state as elements in a reproducing kernel Hilbert space, we can learn the "best fit" distribution via a simple regularized least-squares problem, and then compute the stochastic reachability safety probabilities as simple linear operations. This technique admits finite sample bounds and has known convergence in probability. We implement these methods as part of SReachTools, and demonstrate their use on a double integrator system, on a million-dimensional repeated planar quadrotor system, and a cart-pole system with a black-box neural network controller.

math.OC↗

arXiv2020

Fast, Convexified Stochastic Optimal Open-Loop Control For Linear Systems Using Empirical Characteristic Functions

We consider the problem of stochastic optimal control in the presence of an unknown disturbance. We characterize the disturbance via empirical characteristic functions, and employ a chance constrained approach. By exploiting properties of characteristic functions and underapproximating cumulative distribution functions, we can reformulate a nonconvex problem by a conic, convex under-approximation. This results in extremely fast solutions that are assured to maintain probabilistic constraints. We construct algorithms for optimal open-loop control using piecewise linear approximations of the empirical characteristic function, and demonstrate our approach on two examples.

math.OC↗

arXiv2020

Stochastic reachability of a target tube: Theory and computation

Probabilistic guarantees of safety and performance are important in constrained dynamical systems with stochastic uncertainty. We consider the stochastic reachability problem, which maximizes the probability that the state remains within time-varying state constraints (i.e., a ``target tube''), despite bounded control authority. This problem subsumes the stochastic viability and terminal hitting-time stochastic reach-avoid problems. Of special interest is the stochastic reach set, the set of all initial states from which it is possible to stay in the target tube with a probability above a desired threshold. We provide sufficient conditions under which the stochastic reach set is closed, compact, and convex, and provide an underapproximative interpolation technique for stochastic reach sets. Utilizing convex optimization, we propose a scalable and grid-free algorithm that computes a polytopic underapproximation of the stochastic reach set and synthesizes an open-loop controller. This algorithm is anytime, i.e., it produces a valid output even on early termination. We demonstrate the efficacy and scalability of our approach on several numerical examples, and show that our algorithm outperforms existing software tools for verification of linear systems.

math.OC↗

arXiv2020

Learning Approximate Forward Reachable Sets Using Separating Kernels

We present a data-driven method for computing approximate forward reachable sets using separating kernels in a reproducing kernel Hilbert space. We frame the problem as a support estimation problem, and learn a classifier of the support as an element in a reproducing kernel Hilbert space using a data-driven approach. Kernel methods provide a computationally efficient representation for the classifier that is the solution to a regularized least squares problem. The solution converges almost surely as the sample size increases, and admits known finite sample bounds. This approach is applicable to stochastic systems with arbitrary disturbances and neural network verification problems by treating the network as a dynamical system, or by considering neural network controllers as part of a closed-loop system. We present our technique on several examples, including a spacecraft rendezvous and docking problem, and two nonlinear system benchmarks with neural network controllers.

math.OC↗

arXiv2020

Approximate Stochastic Reachability for High Dimensional Systems

We present a method to compute the stochastic reachability safety probabilities for high-dimensional stochastic dynamical systems. Our approach takes advantage of a nonparametric learning technique known as conditional distribution embeddings to model the stochastic kernel using a data-driven approach. By embedding the dynamics and uncertainty within a reproducing kernel Hilbert space, it becomes possible to compute the safety probabilities for stochastic reachability problems as simple matrix operations and inner products. We employ a convergent approximation technique, random Fourier features, in order to alleviate the increased computational requirements for high-dimensional systems. This technique avoids the curse of dimensionality, and enables the computation of safety probabilities for high-dimensional systems without prior knowledge of the structure of the dynamics or uncertainty. We validate this approach on a double integrator system, and demonstrate its capabilities on a million-dimensional, nonlinear, non-Gaussian, repeated planar quadrotor system.

eess.SY↗

arXiv2020

Convexified Open-Loop Stochastic Optimal Control for Linear Non-Gaussian Systems

We consider stochastic optimal control of linear dynamical systems with additive non-Gaussian disturbance. We propose a novel, sampling-free approach, based on Fourier transformations and convex optimization, to cast the stochastic optimal control problem as a difference-of-convex program. In contrast to existing moment based approaches, our approach invokes higher moments, resulting in less conservatism. We employ piecewise affine approximations and the well-known convex-concave procedure, to efficiently solve the resulting optimization problem via standard conic solvers. We demonstrate that the proposed approach is computationally faster than existing particle based and moment based approaches, without compromising probabilistic safety constraints.

math.OC↗

arXiv2020

Trust-based user-interface design for human-automation systems

We present a method for dynamics-driven, user-interface design for a human-automation system via sensor selection. We define the user-interface to be the output of a MIMO LTI system, and formulate the design problem as one of selecting an output matrix from a given set of candidate output matrices. Sufficient conditions for situation awareness are captured as additional constraints on the selection of the output matrix. These constraints depend upon the level of trust the human has in the automation. We show that the resulting user-interface design problem is a combinatorial, set-cardinality minimization problem with set function constraints. We propose tractable algorithms to compute optimal or sub-optimal solutions with suboptimality bounds. Our approaches exploit monotonicity and submodularity present in the design problem, and rely on constraint programming and submodular maximization. We apply this method to the IEEE 118-bus, to construct correct-by-design interfaces under various operating scenarios.

eess.SY↗

arXiv2020

Model-Free Stochastic Reachability Using Kernel Distribution Embeddings

We present a solution to the terminal-hitting stochastic reach-avoid problem for a Markov control process. This solution takes advantage of a nonparametric representation of the stochastic kernel as a conditional distribution embedding within a reproducing kernel Hilbert space (RKHS). Because the disturbance is modeled as a data-driven stochastic process, this representation avoids intractable integrals in the dynamic recursion of the reach-avoid problem since the expectations can be calculated as an inner product within the RKHS. We demonstrate this approach on a high-dimensional chain of integrators and on Clohessy-Wiltshire-Hill dynamics.

math.OC↗