arXiv subjects

Ali Baheri

Publications and source records attributed to Ali Baheri.

48 records · Page 3Linked to original sources

arXiv2023

Joint Falsification and Fidelity Settings Optimization for Validation of Safety-Critical Systems: A Theoretical Analysis

Safety validation is a crucial component in the development and deployment of autonomous systems, such as self-driving vehicles and robotic systems. Ensuring safe operation necessitates extensive testing and verification of control policies, typically conducted in simulation environments. High-fidelity simulators accurately model real-world dynamics but entail high computational costs, limiting their scalability for exhaustive testing. Conversely, low-fidelity simulators offer efficiency but may not capture the intricacies of high-fidelity simulators, potentially yielding false conclusions. We propose a joint falsification and fidelity optimization framework for safety validation of autonomous systems. Our mathematical formulation combines counterexample searches with simulator fidelity improvement, facilitating more efficient exploration of the critical environmental configurations challenging the control system. Our contributions encompass a set of theorems addressing counterexample sensitivity analysis, sample complexity, convergence, the interplay between the outer and inner optimization loops, and regret bound analysis. The proposed joint optimization approach enables a more targeted and efficient testing process, optimizes the use of available computational resources, and enhances confidence in autonomous system safety validation.

eess.SY↗

arXiv2023

Joint Learning of Policy with Unknown Temporal Constraints for Safe Reinforcement Learning

In many real-world applications, safety constraints for reinforcement learning (RL) algorithms are either unknown or not explicitly defined. We propose a framework that concurrently learns safety constraints and optimal RL policies in such environments, supported by theoretical guarantees. Our approach merges a logically-constrained RL algorithm with an evolutionary algorithm to synthesize signal temporal logic (STL) specifications. The framework is underpinned by theorems that establish the convergence of our joint learning process and provide error bounds between the discovered policy and the true optimal policy. We showcased our framework in grid-world environments, successfully identifying both acceptable safety constraints and RL policies while demonstrating the effectiveness of our theorems in practice.

eess.SY↗

arXiv2023

Falsification of Learning-Based Controllers through Multi-Fidelity Bayesian Optimization

Simulation-based falsification is a practical testing method to increase confidence that the system will meet safety requirements. Because full-fidelity simulations can be computationally demanding, we investigate the use of simulators with different levels of fidelity. As a first step, we express the overall safety specification in terms of environmental parameters and structure this safety specification as an optimization problem. We propose a multi-fidelity falsification framework using Bayesian optimization, which is able to determine at which level of fidelity we should conduct a safety evaluation in addition to finding possible instances from the environment that cause the system to fail. This method allows us to automatically switch between inexpensive, inaccurate information from a low-fidelity simulator and expensive, accurate information from a high-fidelity simulator in a cost-effective way. Our experiments on various environments in simulation demonstrate that multi-fidelity Bayesian optimization has falsification performance comparable to single-fidelity Bayesian optimization but with much lower cost.

eess.SY↗

arXiv2023

Black-Box Safety Validation of Autonomous Systems: A Multi-Fidelity Reinforcement Learning Approach

The increasing use of autonomous and semi-autonomous agents in society has made it crucial to validate their safety. However, the complex scenarios in which they are used may make formal verification impossible. To address this challenge, simulation-based safety validation is employed to test the complex system. Recent approaches using reinforcement learning are prone to excessive exploitation of known failures and a lack of coverage in the space of failures. To address this limitation, a type of Markov decision process called the "knowledge MDP" has been defined. This approach takes into account both the learned model and its metadata, such as sample counts, in estimating the system's knowledge through the "knows what it knows" framework. A novel algorithm that extends bidirectional learning to multiple fidelities of simulators has been developed to solve the safety validation problem. The effectiveness of this approach is demonstrated through a case study in which an adversary is trained to intercept a test model in a grid-world environment. Monte Carlo trials compare the sample efficiency of the proposed algorithm to learning with a single-fidelity simulator and show the importance of incorporating knowledge about learned models into the decision-making process.

eess.SY↗

arXiv2023

Forward variable selection enables fast and accurate dynamic system identification with Karhunen-Loève decomposed Gaussian processes

A promising approach for scalable Gaussian processes (GPs) is the Karhunen-Loève (KL) decomposition, in which the GP kernel is represented by a set of basis functions which are the eigenfunctions of the kernel operator. Such decomposed kernels have the potential to be very fast, and do not depend on the selection of a reduced set of inducing points. However KL decompositions lead to high dimensionality, and variable selection becomes paramount. This paper reports a new method of forward variable selection, enabled by the ordered nature of the basis functions in the KL expansion of the Bayesian Smoothing Spline ANOVA kernel (BSS-ANOVA), coupled with fast Gibbs sampling in a fully Bayesian approach. It quickly and effectively limits the number of terms, yielding a method with competitive accuracies, training and inference times for tabular datasets of low feature set dimensionality. The inference speed and accuracy makes the method especially useful for dynamic systems identification, by modeling the dynamics in the tangent space as a static problem, then integrating the learned dynamics using a high-order scheme. The methods are demonstrated on two dynamic datasets: a `Susceptible, Infected, Recovered' (SIR) toy problem, with the transmissibility used as forcing function, along with the experimental `Cascaded Tanks' benchmark dataset. Comparisons on the static prediction of time derivatives are made with a random forest (RF), a residual neural network (ResNet), and the Orthogonal Additive Kernel (OAK) inducing points scalable GP, while for the timeseries prediction comparisons are made with LSTM and GRU recurrent neural networks (RNNs) along with the SINDy package.

cs.LG↗

arXiv2022

A Verification Framework for Certifying Learning-Based Safety-Critical Aviation Systems

We present a safety verification framework for design-time and run-time assurance of learning-based components in aviation systems. Our proposed framework integrates two novel methodologies. From the design-time assurance perspective, we propose offline mixed-fidelity verification tools that incorporate knowledge from different levels of granularity in simulated environments. From the run-time assurance perspective, we propose reachability- and statistics-based online monitoring and safety guards for a learning-based decision-making model to complement the offline verification methods. This framework is designed to be loosely coupled among modules, allowing the individual modules to be developed using independent methodologies and techniques, under varying circumstances and with different tool access. The proposed framework offers feasible solutions for meeting system safety requirements at different stages throughout the system development and deployment cycle, enabling the continuous learning and assessment of the system product.

eess.SY↗

arXiv2022

A Framework for Controlling Multi-Robot Systems Using Bayesian Optimization and Linear Combination of Vectors

We propose a general framework for creating parameterized control schemes for decentralized multi-robot systems. A variety of tasks can be seen in the decentralized multi-robot literature, each with many possible control schemes. For several of them, the agents choose control velocities using algorithms that extract information from the environment and combine that information in meaningful ways. From this basic formation, a framework is proposed that classifies each robots' measurement information as sets of relevant scalars and vectors and creates a linear combination of the measured vector sets. Along with an optimizable parameter set, the scalar measurements are used to generate the coefficients for the linear combination. With this framework and Bayesian optimization, we can create effective control systems for several multi-robot tasks, including cohesion and segregation, pattern formation, and searching/foraging.

cs.RO↗

arXiv2020

Waypoint Optimization Using Bayesian Optimization: A Case Study in Airborne Wind Energy Systems

We present a data-driven optimization framework that aims to address online adaptation of the flight path shape for an airborne wind energy system (AWE) that follows a repetitive path to generate power. Specifically, Bayesian optimization, which is a data-driven algorithm for finding the optimum of an unknown objective function, is utilized to solve the waypoint adaptation. To form a computationally efficient optimization framework, we describe each figure-$8$ flight via a compact set of parameters, termed as basis parameters. We model the underlying objective function by a Gaussian Process (GP). Bayesian optimization utilizes the predictive uncertainty information from the GP to determine the best subsequent basis parameters. Once a path is generated using Bayesian optimization, a path following mechanism is used to track the generated figure-$8$ flight. The proposed framework is validated on a simplified $2$-dimensional model that mimics the key behaviors of a $3$-dimensional AWE system. We demonstrate the capability of the proposed framework in a simulation environment for a simplified $2$-dimensional AWE system model.

eess.SY↗

arXiv2020

Safe Reinforcement Learning with Mixture Density Network: A Case Study in Autonomous Highway Driving

This paper presents a safe reinforcement learning system for automated driving that benefits from multimodal future trajectory predictions. We propose a safety system that consists of two safety components: a heuristic safety and a learning-based safety. The heuristic safety module is based on common driving rules. On the other hand, the learning-based safety module is a data-driven safety rule that learns safety patterns from driving data. Specifically, it utilizes mixture density recurrent neural networks (MD-RNN) for multimodal future trajectory predictions to accelerate the learning progress. Our simulation results demonstrate that the proposed safety system outperforms previously reported results in terms of average reward and number of collisions.

eess.SY↗

arXiv2020

Deep Reinforcement Learning with Enhanced Safety for Autonomous Highway Driving

In this paper, we present a safe deep reinforcement learning system for automated driving. The proposed framework leverages merits of both rule-based and learning-based approaches for safety assurance. Our safety system consists of two modules namely handcrafted safety and dynamically-learned safety. The handcrafted safety module is a heuristic safety rule based on common driving practice that ensure a minimum relative gap to a traffic vehicle. On the other hand, the dynamically-learned safety module is a data-driven safety rule that learns safety patterns from driving data. Specifically, the dynamically-leaned safety module incorporates a model lookahead beyond the immediate reward of reinforcement learning to predict safety longer into the future. If one of the future states leads to a near-miss or collision, then a negative reward will be assigned to the reward function to avoid collision and accelerate the learning process. We demonstrate the capability of the proposed framework in a simulation environment with varying traffic density. Our results show the superior capabilities of the policy enhanced with dynamically-learned safety module.

eess.SY↗

arXiv2020

Vision-Based Autonomous Driving: A Model Learning Approach

We present an integrated approach for perception and control for an autonomous vehicle and demonstrate this approach in a high-fidelity urban driving simulator. Our approach first builds a model for the environment, then trains a policy exploiting the learned model to identify the action to take at each time-step. To build a model for the environment, we leverage several deep learning algorithms. To that end, first we train a variational autoencoder to encode the input image into an abstract latent representation. We then utilize a recurrent neural network to predict the latent representation of the next frame and handle temporal information. Finally, we utilize an evolutionary-based reinforcement learning algorithm to train a controller based on these latent representations to identify the action to take. We evaluate our approach in CARLA, a high-fidelity urban driving simulator, and conduct an extensive generalization study. Our results demonstrate that our approach outperforms several previously reported approaches in terms of the percentage of successfully completed episodes for a lane keeping task.

eess.SY↗

arXiv2019

Economically Efficient Combined Plant and Controller Design Using Batch Bayesian Optimization: Mathematical Framework and Airborne Wind Energy Case Study

We present a novel data-driven nested optimization framework that addresses the problem of coupling between plant and controller optimization. This optimization strategy is tailored towards instances where a closed-form expression for the system dynamic response is unobtainable and simulations or experiments are necessary. Specifically, Bayesian Optimization, which is a data-driven technique for finding the optimum of an unknown and expensive-to-evaluate objective function, is employed to solve a nested optimization problem. The underlying objective function is modeled by a Gaussian Process (GP); then, Bayesian Optimization utilizes the predictive uncertainty information from the GP to determine the best subsequent control or plant parameters. The proposed framework differs from the majority of co-design literature where there exists a closed-form model of the system dynamics. Furthermore, we utilize the idea of Batch Bayesian Optimization at the plant optimization level to generate a set of plant designs at each iteration of the overall optimization process, recognizing that there will exist economies of scale in running multiple experiments in each iteration of the plant design process. We validate the proposed framework for a Buoyant Airborne Turbine (BAT). We choose the horizontal stabilizer area, longitudinal center of mass relative to center of buoyancy (plant parameters), and the pitch angle set-point (controller parameter) as our decision variables. Our results demonstrate that these plant and control parameters converge to their respective optimal values within only a few iterations.

eess.SY↗