arXiv subjects

Yiming Meng

Publications and source records attributed to Yiming Meng.

30 records · Page 2Linked to original sources

arXiv2024

Physics-Informed Neural Network Policy Iteration: Algorithms, Convergence, and Verification

Solving nonlinear optimal control problems is a challenging task, particularly for high-dimensional problems. We propose algorithms for model-based policy iterations to solve nonlinear optimal control problems with convergence guarantees. The main component of our approach is an iterative procedure that utilizes neural approximations to solve linear partial differential equations (PDEs), ensuring convergence. We present two variants of the algorithms. The first variant formulates the optimization problem as a linear least square problem, drawing inspiration from extreme learning machine (ELM) for solving PDEs. This variant efficiently handles low-dimensional problems with high accuracy. The second variant is based on a physics-informed neural network (PINN) for solving PDEs and has the potential to address high-dimensional problems. We demonstrate that both algorithms outperform traditional approaches, such as Galerkin methods, by a significant margin. We provide a theoretical analysis of both algorithms in terms of convergence of neural approximations towards the true optimal solutions in a general setting. Furthermore, we employ formal verification techniques to demonstrate the verifiable stability of the resulting controllers.

eess.SY↗

arXiv2023

Stochastic Control Barrier Functions with Bayesian Inference for Unknown Stochastic Differential Equations

Control barrier functions are widely used to synthesize safety-critical controls. However, the presence of Gaussian-type noise in dynamical systems can generate unbounded signals and potentially result in severe consequences. Although research has been conducted in the field of safety-critical control for stochastic systems, in many real-world scenarios, we do not have precise knowledge about the stochastic dynamics. In this paper, we delve into the safety-critical control for stochastic systems where both the drift and diffusion components are unknown. We employ Bayesian inference as a data-driven approach to approximate the system. To be more specific, we utilize Bayesian linear regression along with the central limit theorem to estimate the drift term, and employ Bayesian inference to approximate the diffusion term. Through simulations, we verify our findings by applying them to a nonlinear dynamical model and an adaptive cruise control model.

eess.SY↗

arXiv2023

Almost Sure Asymptotic Stability of Parabolic SPDEs with Small Multiplicative Noise

A better understanding of the instability margin will eventually optimize the operational range for safety-critical industries. In this paper, we investigate the almost-sure exponential asymptotic stability of the trivial solution of a parabolic semilinear stochastic partial differential equation (SPDE) driven by multiplicative noise near the deterministic Hopf bifurcation point. We show the existence and uniqueness of the invariant measure under appropriate assumptions, and approximate the exponential growth rate via asymptotic expansion, given that the strength of the noise is small. This approximate quantity can readily serve as a robust indicator of the change of almost-sure stability.

math.DS↗

arXiv2023

Towards Learning and Verifying Maximal Neural Lyapunov Functions

The search for Lyapunov functions is a crucial task in the analysis of nonlinear systems. In this paper, we present a physics-informed neural network (PINN) approach to learning a Lyapunov function that is nearly maximal for a given stable set. A Lyapunov function is considered nearly maximal if its sub-level sets can be made arbitrarily close to the boundary of the domain of attraction. We use Zubov's equation to train a maximal Lyapunov function defined on the domain of attraction. Additionally, we propose conditions that can be readily verified by satisfiability modulo theories (SMT) solvers for both local and global stability. We provide theoretical guarantees on the existence of maximal Lyapunov functions and demonstrate the effectiveness of our computational approach through numerical examples.

math.OC↗

arXiv2023

Robustly Complete Finite-State Abstractions for Control Synthesis of Stochastic Systems

The essential step of abstraction-based control synthesis for nonlinear systems to satisfy a given specification is to obtain a finite-state abstraction of the original systems. The complexity of the abstraction is usually the dominating factor that determines the efficiency of the algorithm. For the control synthesis of discrete-time nonlinear stochastic systems modelled by nonlinear stochastic difference equations, recent literature has demonstrated the soundness of abstractions in preserving robust probabilistic satisfaction of ω-regular lineartime properties. However, unnecessary transitions exist within the abstractions, which are difficult to quantify, and the completeness of abstraction-based control synthesis in the stochastic setting remains an open theoretical question. In this paper, we address this fundamental question from the topological view of metrizable space of probability measures, and propose constructive finite-state abstractions for control synthesis of probabilistic linear temporal specifications. Such abstractions are both sound and approximately complete. That is, given a concrete discrete-time stochastic system and an arbitrarily small L1-perturbation of this system, there exists a family of finite-state controlled Markov chains that both abstracts the concrete system and is abstracted by the slightly perturbed system. In other words, given an arbitrarily small prescribed precision, an abstraction always exists to decide whether a control strategy exists for the concrete system to satisfy the probabilistic specification.

eess.SY↗

arXiv2022

Lyapunov-Barrier Characterization of Robust Reach-Avoid-Stay Specifications for Hybrid Systems

Stability, reachability, and safety are crucial properties of dynamical systems. While verification and control synthesis of reach-avoid-stay objectives can be effectively handled by abstraction-based formal methods, such approaches can be computationally expensive due to the use of state-space discretization. In contrast, Lyapunov methods qualitatively characterize stability and safety properties without any state-space discretization. Recent work on converse Lyapunov-barrier theorems also demonstrates an approximate completeness or verifying reach-avoid-stay specifications of systems modelled by nonlinear differential equations. In this paper, based on the topology of hybrid arcs, we extend the Lyapunov-barrier characterization to more general hybrid systems described by differential and difference inclusions. We show that Lyapunov-barrier functions are not only sufficient to guarantee reach-avoid-stay specifications for well-posed hybrid systems, but also necessary for arbitrarily slightly perturbed systems under mild conditions. Numerical examples are provided to illustrate the main results.

math.DS↗

arXiv2022

Sufficient Conditions for Robust Probabilistic Reach-Avoid-Stay Specifications using Stochastic Lyapunov-Barrier Functions

Stability and safety are crucial in safety-critical control of dynamical systems. The reach-avoid-stay objectives for deterministic dynamical systems can be effectively handled by formal methods as well as Lyapunov methods with soundness and approximate completeness guarantees. However, for continuous-time stochastic dynamical systems, probabilistic reach-avoid-stay problems are viewed as challenging tasks. Motivated by the recent surge of applications in characterizing safety-critical properties using Lyapunov-barrier functions, we aim to provide a stochastic version for the probabilistic reach-avoid-stay problems in consideration of robustness. To this end, we first establish a connection between stochastic stability with safety constraints and reach-avoid-stay specifications. We then prove that stochastic Lyapunov-barrier functions provide sufficient conditions for the target objectives. We apply Lyapunovbarrier conditions in control synthesis for reach-avoid-stay specifications, and show its effectiveness in a case study.

math.DS↗

arXiv2022

Data-Driven Learning of Safety-Critical Control with Stochastic Control Barrier Functions

Control barrier functions are widely used to synthesize safety-critical controls. The existence of Gaussian-type noise may lead to unsafe actions and result in severe consequences. While studies are widely done in safety-critical control for stochastic systems, in many real-world applications, we do not have the knowledge of the stochastic component of the dynamics. In this paper, we study safety-critical control of stochastic systems with an unknown diffusion part and propose a data-driven method to handle these scenarios. More specifically, we propose a data-driven stochastic control barrier function (DDSCBF) framework and use supervised learning to learn the unknown stochastic dynamics via the DDSCBF scheme. Under some reasonable assumptions, we provide guarantees that the DDSCBF scheme can approximate the Itô derivative of the stochastic control barrier function (SCBF) under partially unknown dynamics using the universal approximation theorem. We also show that we can achieve the same safety guarantee using the DDSCBF scheme as with SCBF in previous work without requiring the knowledge of stochastic dynamics. We use two non-linear stochastic systems to validate our theory in simulations.

eess.SY↗

arXiv2022

Robustly Complete Finite-State Abstractions for Verification of Stochastic Systems

In this paper, we focus on discrete-time stochastic systems modelled by nonlinear stochastic difference equations and propose robust abstractions for verifying probabilistic linear temporal specifications. The current literature focuses on developing sound abstraction techniques for stochastic dynamics without perturbations. However, soundness thus far has only been shown for preserving the satisfaction probability of certain types of temporal-logic specification. We present constructive finite-state abstractions for verifying probabilistic satisfaction of general ω-regular linear-time properties of more general nonlinear stochastic systems. Instead of imposing stability assumptions, we analyze the probabilistic properties from the topological view of metrizable space of probability measures. Such abstractions are both sound and approximately complete. That is, given a concrete discrete-time stochastic system and an arbitrarily small L1-perturbation of this system, there exists a family of finite-state Markov chains whose set of satisfaction probabilities contains that of the original system and meanwhile is contained by that of the slightly perturbed system. A direct consequence is that, given a probabilistic linear-time specification, initializing within the winning/losing region of the abstraction system can guarantee a satisfaction/dissatisfaction for the original system. We make an interesting observation that, unlike the deterministic case, point-mass (Dirac) perturbations cannot fulfill the purpose of robust completeness.

math.PR↗

arXiv2021

Smooth Converse Lyapunov-Barrier Theorems for Asymptotic Stability with Safety Constraints and Reach-Avoid-Stay Specifications

Stability and safety are two important aspects in safety-critical control of dynamical systems. It has been a well established fact in control theory that stability properties can be characterized by Lyapunov functions. Reachability properties can also be naturally captured by Lyapunov functions for finite-time stability. Motivated by safety-critical control applications, such as in autonomous systems and robotics, there has been a recent surge of interests in characterizing safety properties using barrier functions. Lyapunov and barrier functions conditions, however, are sometimes viewed as competing objectives. In this paper, we provide a unified theoretical treatment of Lyapunov and barrier functions in terms of converse theorems for stability properties with safety guarantees and reach-avoid-stay type specifications. We show that if a system (modeled as a dynamical system with measurable perturbations) possesses a stability with safety property, then there exists a smooth Lyapunov function to certify such a property. This Lyapunov function is shown to be defined on the entire set of initial conditions from which solutions satisfy this property. A similar but slightly weaker statement is made for reach-avoid-stay specifications. We show by a simple example that the latter statement cannot be strengthened without additional assumptions. We further extend the results for systems with control inputs and prove existence of converse Lyapunov-barrier functions for reach-and-avoid specifications. While the converse Lyapunov-barrier theorems are not constructive, as with classical converse Lyapunov theorems, we believe that the unified necessary and sufficient conditions with a single Lyapunov-barrier function are of theoretical interest and can hopefully shed some light on computational approaches.

eess.SY↗

arXiv2021

Safety-Critical Control of Stochastic Systems using Stochastic Control Barrier Functions

Control barrier functions have been widely used for synthesizing safety-critical controls, often via solving quadratic programs. However, the existence of Gaussian-type noise may lead to unsafe actions and result in severe consequences. In this paper, we study systems modeled by stochastic differential equations (SDEs) driven by Brownian motions. We propose a notion of stochastic control barrier functions (SCBFs)and show that SCBFs can significantly reduce the control efforts, especially in the presence of noise, compared to stochastic reciprocal control barrier functions (SRCBFs), and offer a less conservative estimation of safety probability, compared to stochastic zeroing control barrier functions (SZCBFs). Based on this less conservative probabilistic estimation for the proposed notion of SCBFs, we further extend the results to handle high relative degree safety constraints using high-order SCBFs. We demonstrate that the proposed SCBFs achieve good trade-offs of performance and control efforts, both through theoretical analysis and numerical simulations.

eess.SY↗

arXiv2020

Learning Control Barrier Functions with High Relative Degree for Safety-Critical Control

Control barrier functions have shown great success in addressing control problems with safety guarantees. These methods usually find the next safe control input by solving an online quadratic programming problem. However, model uncertainty is a big challenge in synthesizing controllers. This may lead to the generation of unsafe control actions, resulting in severe consequences. In this paper, we develop a learning framework to deal with system uncertainty. Our method mainly focuses on learning the dynamics of the control barrier function, especially for high relative degree with respect to a system. We show that for each order, the time derivative of the control barrier function can be separated into the time derivative of the nominal control barrier function and a remainder. This implies that we can use a neural network to learn the remainder so that we can approximate the dynamics of the real control barrier function. We show by simulation that our method can generate safe trajectories under parametric uncertainty using a differential drive robot model.

eess.SY↗