arXiv subjects

Matthew Abate

Publications and source records attributed to Matthew Abate.

13 recordsLinked to original sources

arXiv2022

Robust Training and Verification of Implicit Neural Networks: A Non-Euclidean Contractive Approach

This paper proposes a theoretical and computational framework for training and robustness verification of implicit neural networks based upon non-Euclidean contraction theory. The basic idea is to cast the robustness analysis of a neural network as a reachability problem and use (i) the $\ell_{\infty}$-norm input-output Lipschitz constant and (ii) the tight inclusion function of the network to over-approximate its reachable sets. First, for a given implicit neural network, we use $\ell_{\infty}$-matrix measures to propose sufficient conditions for its well-posedness, design an iterative algorithm to compute its fixed points, and provide upper bounds for its $\ell_\infty$-norm input-output Lipschitz constant. Second, we introduce a related embedded network and show that the embedded network can be used to provide an $\ell_\infty$-norm box over-approximation of the reachable sets of the original network. Moreover, we use the embedded network to design an iterative algorithm for computing the upper bounds of the original system's tight inclusion function. Third, we use the upper bounds of the Lipschitz constants and the upper bounds of the tight inclusion functions to design two algorithms for the training and robustness verification of implicit neural networks. Finally, we apply our algorithms to train implicit neural networks on the MNIST dataset and compare the robustness of our models with the models trained via existing approaches in the literature.

cs.LG↗

arXiv2022

Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks

We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs). INNs are a class of implicit learning models that use implicit equations as layers and have been shown to exhibit several notable benefits over traditional deep neural networks. We first establish that tight inclusion functions of neural networks, which provide the tightest rectangular over-approximation of an input-output map, lead to sharper robustness guarantees than the well-studied robustness measures of local Lipschitz constants. Like Lipschitz constants, tight inclusions functions are computationally challenging to obtain, and we thus propose using mixed monotonicity and contraction theory to obtain computationally efficient estimates of tight inclusion functions for INNs. We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs. We design a novel optimization problem for training robust INNs and we provide empirical evidence that suitably-trained INNs can be more robust than comparably-trained feedforward networks.

cs.LG↗

arXiv2021

Robustness Certificates for Implicit Neural Networks: A Mixed Monotone Contractive Approach

Implicit neural networks are a general class of learning models that replace the layers in traditional feedforward models with implicit algebraic equations. Compared to traditional learning models, implicit networks offer competitive performance and reduced memory consumption. However, they can remain brittle with respect to input adversarial perturbations. This paper proposes a theoretical and computational framework for robustness verification of implicit neural networks; our framework blends together mixed monotone systems theory and contraction theory. First, given an implicit neural network, we introduce a related embedded network and show that, given an $\ell_\infty$-norm box constraint on the input, the embedded network provides an $\ell_\infty$-norm box overapproximation for the output of the given network. Second, using $\ell_{\infty}$-matrix measures, we propose sufficient conditions for well-posedness of both the original and embedded system and design an iterative algorithm to compute the $\ell_{\infty}$-norm box robustness margins for reachability and classification problems. Third, of independent value, we propose a novel relative classifier variable that leads to tighter bounds on the certified adversarial robustness in classification problems. Finally, we perform numerical simulations on a Non-Euclidean Monotone Operator Network (NEMON) trained on the MNIST dataset. In these simulations, we compare the accuracy and run time of our mixed monotone contractive approach with the existing robustness verification approaches in the literature for estimating the certified adversarial robustness.

cs.LG↗

arXiv2021

Run Time Assurance for Safety-Critical Systems: An Introduction to Safety Filtering Approaches for Complex Control Systems

Run Time Assurance (RTA) Systems are online verification mechanisms that filter an unverified primary controller output to ensure system safety. The primary control may come from a human operator, an advanced control approach, or an autonomous control approach that cannot be verified to the same level as simpler control systems designs. The critical feature of RTA systems is their ability to alter unsafe control inputs explicitly to assure safety. In many cases, RTA systems can functionally be described as containing a monitor that watches the state of the system and output of a primary controller, and a backup controller that replaces or modifies control input when necessary to assure safety. An important quality of an RTA system is that the assurance mechanism is constructed in a way that is entirely agnostic to the underlying structure of the primary controller. By effectively decoupling the enforcement of safety constraints from performance-related objectives, RTA offers a number of useful advantages over traditional (offline) verification. This article provides a tutorial on developing RTA systems.

eess.SY↗

arXiv2021

Improving the Fidelity of Mixed-Monotone Reachable Set Approximations via State Transformations

Mixed-monotone systems are separable via a decomposition function into increasing and decreasing components, and this decomposition function allows for embedding the system dynamics in a higher-order monotone embedding system. Embedding the system dynamics in this way facilitates the efficient over-approximation of reachable sets with hyperrectangles, however, unlike the monotonicity property, which can be applied to compute, e.g., the tightest hyperrectangle containing a reachable set, the application of the mixed-monotonicity property generally results in conservative reachable set approximations. In this work, explore conservatism in the method and we consider, in particular, embedding systems that are monotone with respect to an alternative partial order. This alternate embedding system is constructed with a decomposition function for a related system, formed via a linear transformation of the initial state-space. We show how these alternate embedding systems allow for computing reachable sets with improved fidelity, i.e., reduced conservatism.

eess.SY↗

arXiv2020

A Numerical Method to Compute Stability Margins of Switching Linear Systems

Stability margins for linear time-varying (LTV) and switched-linear systems are traditionally computed via quadratic Lyapunov functions, and these functions certify the stability of the system under study. In this work, we show how the more general class of homogeneous polynomial Lyapunov functions is used to compute stability margins with reduced conservatism, and we show how these Lyapunov functions aid in the search for periodic trajectories for marginally stable LTV systems. Our work is premised on the recent observation that the search for a homogeneous polynomial Lyapunov function for some LTV systems is easily encoded as the search for a quadratic Lyapunov function for a related LTV system, and our main contribution is an intuitive algorithm for generating upper and lower bounds on the system's stability margin. We show also how the worst-case switching scheme - which draws an LTV system closest to a periodic orbit - is generated. Three numerical examples are provided to aid the reader and demonstrate the theoretical contributions of the work.

eess.SY↗

arXiv2020

Performance Analysis and Non-Quadratic Lyapunov Functions for Linear Time-Varying Systems

Performance analysis for linear time-invariant (LTI) systems has been closely tied to quadratic Lyapunov functions ever since it was shown that LTI system stability is equivalent to the existence of such a Lyapunov function. Some metrics for LTI systems, however, have resisted treatment via means of quadratic Lyapunov functions. Among these, point-wise-in-time metrics, such as peak norms, are not captured accurately using these techniques, and this shortcoming has prevented the development of tools to analyze system behavior by means other than e.g. time-domain simulations. This work demonstrates how the more general class of homogeneous polynomial Lyapunov functions can be used to approximate point-wise-in-time behavior for LTI systems with greater accuracy, and we extend this to the case of linear time-varying (LTV) systems as well. Our findings rely on the recent observation that the search for homogeneous polynomial Lyapunov functions for LTV systems can be recast as a search for quadratic Lyapunov functions for a related hierarchy of time-varying Lyapunov differential equations; thus, performance guarantees for LTV systems are attainable without heavy computation. Numerous examples are provided to demonstrate the findings of this work.

eess.SY↗

arXiv2020

Computing Robustly Forward Invariant Sets for Mixed-Monotone Systems

This work presents new tools for studying reachability and set invariance for continuous-time mixed-monotone dynamical systems subject to a disturbance input. The vector field of a mixed-monotone system is decomposable via a decomposition function into increasing and decreasing components, and this decomposition enables embedding the original dynamics in a higher-dimensional embedding system. While the original system is subject to an unknown disturbance input, the embedding system has no disturbances and its trajectories provide bounds for finite-time reachable sets of the original dynamics. Our main contribution is to show how one can efficiently identify robustly forward invariant and attractive sets for mixed-monotone systems by studying certain equilibria of this embedding system. We show also how this approach, when applied to the backward-time dynamics, establishes different robustly forward invariant sets for the original dynamics. Lastly, we present an independent result for computing decomposition functions for systems with polynomial dynamics. These tools and results are demonstrated through several examples and a case study.

eess.SY↗

arXiv2020

Enforcing Safety at Runtime for Systems with Disturbances

Safety for control systems is often posed as an invariance constraint; the system is said to be safe if state trajectories avoid some unsafe region of the statespace for all time. An assured controller is one that enforces safety online by filtering a desired control input at runtime, and control barrier functions (CBFs) provide an assured controller that renders a safe subset of the state-space forward invariant. Recent extensions propose CBF-based assured controllers that allow the system to leave a known safe set so long as a given backup control strategy eventually returns to the safe set, however, these methods have yet to be extended to consider systems subjected to unknown disturbance inputs. In this work, we present a problem formulation for CBF-based runtime assurance for systems with disturbances, and controllers which solve this problem must, in some way, incorporate the online computation of reachable sets. In general, computing reachable sets in the presence of disturbances is computationally costly and cannot be directly incorporated in a CBF framework. To that end, we present a particular solution to the problem, whereby reachable sets are approximated via the mixed-monotonicity property. Efficient algorithms exist for overapproximating reachable sets for mixed-monotone systems with hyperrectangles, and we show that such approximations are suitable for incorporating into a CBF-based runtime assurance framework.

eess.SY↗

arXiv2020

Tight Decomposition Functions for Continuous-Time Mixed-Monotone Systems with Disturbances

The vector field of a mixed-monotone system is decomposable via a decomposition function into increasing (cooperative) and decreasing (competitive) components, and this decomposition allows for, e.g., efficient computation of reachable sets and forward invariant sets. A main challenge in this approach, however, is identifying an appropriate decomposition function. In this work, we show that any continuous-time dynamical system with a Lipschitz continuous vector field is mixed-monotone, and we provide a construction for the decomposition function that yields the tightest approximation of reachable sets when used with the standard tools for mixed-monotone systems. Our construction is similar to that recently proposed by Yang and Ozay for computing decomposition functions of discrete-time systems [1] where we make appropriate modifications for the continuous-time setting and also extend to the case with unknown disturbance inputs. As in [1], our decomposition function construction requires solving an optimization problem for each point in the state-space; however, we demonstrate through example how tight decomposition functions can sometimes be calculated in closed form. As a second contribution, we show how under-approximations of reachable sets can be efficiently computed via the mixed-monotonicity property by considering the backward-time dynamics.

eess.SY↗

arXiv2020

Lyapunov Differential Equation Hierarchy and Polynomial Lyapunov Functions for Switched Linear Systems

This work studies the problem of searching for homogeneous polynomial Lyapunov functions for stable switched linear systems. Specifically, we show an equivalence between polynomial Lyapunov functions for systems of this class and quadratic Lyapunov functions for a related hierarchy of Lyapunov differential equations. This creates an intuitive procedure for checking the stability properties of switched linear systems and a computationally competitive algorithm is presented for generating high-order homogeneous polynomial Lyapunov functions in this manner. Additionally, we provide a comparison between polynomial Lyapunov functions generated with our proposed approach and polynomial Lyapunov functions generated with a more traditional sum-of-squares based approach.

eess.SY↗

arXiv2020

Extent-Compatible Control Barrier Functions

Safety requirements in dynamical systems are commonly enforced with set invariance constraints over a safe region of the state space. Control barrier functions, which are Lyapunov-like functions for guaranteeing set invariance, are an effective tool to enforce such constraints and guarantee safety when the system is represented as a point in the state space. In this paper, we introduce extent-compatible control barrier functions as a tool to enforce safety for the system including its volume (extent) in the physical world. In order to implement the extent-compatible control barrier functions framework, a sum-of-squares based optimization program is proposed. Since sum-of-squares programs can be computationally prohibitive, we additionally introduce a sampling based method in order to retain the computational advantage of a traditional barrier function based quadratic program controller. We prove that the proposed sampling based controller retains the guarantee for safety. Simulation and robotic implementation results are also provided.

eess.SY↗

arXiv2019

Monitor-Based Runtime Assurance for Temporal Logic Specifications

This paper introduces the safety controller architecture as a runtime assurance mechanism for system specifications expressed as safety properties in Linear Temporal Logic (LTL). The safety controller has three fundamental components: a performance controller, a backup controller, and an assurance mechanism. The assurance mechanism uses a monitor, constructed as a finite state machine (FSM), to analyze a suggested performance control input and search for system trajectories that are bad prefixes of the system specification. A fault flag from the assurance mechanism denotes a potentially dangerous future system state and triggers a sequence of inputs that is guaranteed to keep the system safe for all time. A case study is presented which details the construction and implementation of a safety controller on a non-deterministic cyber-physical system.

eess.SY↗