arXiv subjects

Zhaojian Li

Publications and source records attributed to Zhaojian Li.

72 records · Page 4Linked to original sources

arXiv2022

Event-Triggered Model Predictive Control with Deep Reinforcement Learning for Autonomous Driving

Event-triggered model predictive control (eMPC) is a popular optimal control method with an aim to alleviate the computation and/or communication burden of MPC. However, it generally requires priori knowledge of the closed-loop system behavior along with the communication characteristics for designing the event-trigger policy. This paper attempts to solve this challenge by proposing an efficient eMPC framework and demonstrate successful implementation of this framework on the autonomous vehicle path following. First of all, a model-free reinforcement learning (RL) agent is used to learn the optimal event-trigger policy without the need for a complete dynamical system and communication knowledge in this framework. Furthermore, techniques including prioritized experience replay (PER) buffer and long-short term memory (LSTM) are employed to foster exploration and improve training efficiency. In this paper, we use the proposed framework with three deep RL algorithms, i.e., Double Q-learning (DDQN), Proximal Policy Optimization (PPO), and Soft Actor-Critic (SAC), to solve this problem. Experimental results show that all three deep RL-based eMPC (deep-RL-eMPC) can achieve better evaluation performance than the conventional threshold-based and previous linear Q-based approach in the autonomous path following. In particular, PPO-eMPC with LSTM and DDQN-eMPC with PER and LSTM obtains a superior balance between the closed-loop control performance and event-trigger frequency. The associated code is open-sourced and available at: https://github.com/DangFengying/RL-based-event-triggered-MPC.

cs.RO↗

arXiv2022

Cloud-Assisted Collaborative Road Information Discovery with Gaussian Process: Application to Road Profile Estimation

There is an increasing popularity in exploiting modern vehicles as mobile sensors to obtain important road information such as potholes, black ice and road profile. Availability of such information has been identified as a key enabler for next-generation vehicles with enhanced safety, efficiency, and comfort. However, existing road information discovery approaches have been predominately performed in a single-vehicle setting, which is inevitably susceptible to vehicle model uncertainty and measurement errors. To overcome these limitations, this paper presents a novel cloud-assisted collaborative estimation framework that can utilize multiple heterogeneous vehicles to iteratively enhance estimation performance. Specifically, each vehicle combines its onboard measurements with a cloud-based Gaussian process (GP), crowdsourced from prior participating vehicles as "pseudo-measurements", into a local estimator to refine the estimation. The resultant local onboard estimation is then sent back to the cloud to update the GP, where we utilize a noisy input GP (NIGP) method to explicitly handle uncertain GPS measurements. We employ the proposed framework to the application of collaborative road profile estimation. Promising results on extensive simulations and hardware-in-the-loop experiments show that the proposed collaborative estimation can significantly enhance estimation and iteratively improve the performance from vehicle to vehicle, despite vehicle heterogeneity, model uncertainty, and measurement noises.

eess.SY↗

arXiv2022

Simultaneous Suspension Control and Energy Harvesting through Novel Design and Control of a New Nonlinear Energy Harvesting Shock Absorber

Simultaneous vibration control and energy harvesting of vehicle suspensions have attracted significant research attention over the past decades. However, existing energy harvesting shock absorbers (EHSAs) are mainly designed based on the principle of linear resonance, thereby compromising suspension performance for high-efficiency energy harvesting and being only responsive to narrow bandwidth vibrations. In this paper, we propose a new EHSA design -- inerter pendulum vibration absorber (IPVA) -- that integrates an electromagnetic rotary EHSA with a nonlinear pendulum vibration absorber. We show that this design simultaneously improves ride comfort and energy harvesting efficiency by exploiting the nonlinear effects of pendulum inertia. To further improve the performance, we develop a novel stochastic linearization model predictive control (SL-MPC) approach in which we employ stochastic linearization to approximate the nonlinear dynamics of EHSA that has superior accuracy compared to standard linearization. In particular, we develop a new stochastic linearization method with guaranteed stabilizability, which is a prerequisite for control designs. This leads to an MPC problem that is much more computationally efficient than the nonlinear MPC counterpart with no major performance degradation. Extensive simulations are performed to show the superiority of the proposed new nonlinear EHSA and to demonstrate the efficacy of the proposed SL-MPC.

eess.SY↗

arXiv2022

A Nonlinear Lateral Controller Design for Vehicle Path-following with an Arbitrary Sensor Location

This paper investigates the lateral control problem in vehicular path-following when the feedback sensor(s) are mounted at an arbitrary location in the longitudinal symmetric axis. We point out that some existing literature has abused the kinematic bicycle model describing the motion of rear axle center for other locations, which may lead to poor performance in practical implementations. A new nonlinear controller with low-complexity and high-maneuverability is then designed that takes into account senor mounting location, driving comfort and transient response with large initial errors. Design insights and intuitions are also provided in detail. Furthermore, analysis on stability and tracking performance for the closed-loop system are studied, and conditions and guidelines are provided on the selection of control parameters. Comprehensive simulations are performed to demonstrate the efficacy of the proposed nonlinear controller for arbitrary sensor locations. Meanwhile, we also show that designing controllers ignoring the sensor location may lead to unexpected vehicular sway motion in non-straight paths.

eess.SY↗

arXiv2021

Offline Reinforcement Learning for Autonomous Driving with Safety and Exploration Enhancement

Reinforcement learning (RL) is a powerful data-driven control method that has been largely explored in autonomous driving tasks. However, conventional RL approaches learn control policies through trial-and-error interactions with the environment and therefore may cause disastrous consequences such as collisions when testing in real-world traffic. Offline RL has recently emerged as a promising framework to learn effective policies from previously-collected, static datasets without the requirement of active interactions, making it especially appealing for autonomous driving applications. Despite promising, existing offline RL algorithms such as Batch-Constrained deep Q-learning (BCQ) generally lead to rather conservative policies with limited exploration efficiency. To address such issues, this paper presents an enhanced BCQ algorithm by employing a learnable parameter noise scheme in the perturbation model to increase the diversity of observed actions. In addition, a Lyapunov-based safety enhancement strategy is incorporated to constrain the explorable state space within a safe region. Experimental results in highway and parking traffic scenarios show that our approach outperforms the conventional RL method, as well as state-of-the-art offline RL algorithms.

cs.RO↗

arXiv2021

Cloud-Assisted Nonlinear Model Predictive Control for Finite-Duration Tasks

Cloud computing creates new possibilities for control applications by offering powerful computation and storage capabilities. In this paper, we propose a novel cloud-assisted model predictive control (MPC) framework in which we systematically fuse a cloud MPC that uses a high-fidelity nonlinear model but is subject to communication delays with a local MPC that exploits simplified dynamics (due to limited computation) but has timely feedback. Unlike traditional cloud-based control that treats the cloud as powerful, remote, and sole controller in a networked-system control setting, the proposed framework aims at seamlessly integrating the two controllers for enhanced performance. In particular, we formalize the fusion problem for finite-duration tasks by explicitly considering model mismatches and errors due to request-response communication delays. We analyze stability-like properties of the proposed cloud-assisted MPC framework and establish approaches to robustly handling constraints within this framework in spite of plant-model mismatch and disturbances. A fusion scheme is then developed to enhance control performance while satisfying stability-like conditions, the efficacy of which is demonstrated with multiple simulation examples, including an automotive control example to show its industrial application potentials.

eess.SY↗

arXiv2020

A Mechanical System Inspired Microscopic Traffic Model: Modeling, Analysis, and Validation

In this paper, we develop a mechanical system inspired microscopic traffic model to characterize the longitudinal interaction dynamics among a chain of vehicles. In particular, we extend our prior work on mass-spring-damper-clutch based car-following model between two vehicles to multi-vehicle scenario. This model can naturally capture the driver's tendency to maintain the same speed as the vehicle ahead while keeping a (speed-dependent) desired spacing. It is also capable of characterizing the impact of the following vehicle on the preceding vehicle, which is generally neglected in existing models. A new string stability criterion is defined for the considered multi-vehicle dynamics, and stability analysis is performed on the system parameters and time delays. An efficient online parameter identification algorithm, sequential recursive least squares with inverse QR decomposition (SRLS-IQR), is developed to estimate the driving-related model parameters. These real-time estimated parameters can be employed in advanced longitudinal control systems to enable accurate prediction of vehicle trajectories for improved safety and fuel efficiency. The proposed model and the parameter identification algorithm are validated on NGSIM, a naturalistic driving dataset, as well as our own connected vehicle driving data. Promising performance is demonstrated.

eess.SY↗

arXiv2020

System Design and Control of an Apple Harvesting Robot

There is a growing need for robotic apple harvesting due to decreasing availability and rising cost in labor. Towards the goal of developing a viable robotic system for apple harvesting, this paper presents synergistic mechatronic design and motion control of a robotic apple harvesting prototype, which lays a critical foundation for future advancements. Specifically, we develop a deep learning-based fruit detection and localization system using an RGB-D camera. A three degree-of-freedom manipulator is then designed with a hybrid pneumatic/motor actuation mechanism to achieve fast and dexterous movements. A vacuum-based end-effector is used for apple detaching. These three components are integrated into a robotic apple harvesting prototype with simplicity, compactness, and robustness. Moreover, a nonlinear velocity-based control scheme is developed for the manipulator to achieve accurate and agile motion control. Test experiments are conducted to demonstrate the performance of the developed apple harvesting robot.

cs.RO↗

arXiv2020

DeepApple: Deep Learning-based Apple Detection using a Suppression Mask R-CNN

Robotic apple harvesting has received much research attention in the past few years due to growing shortage and rising cost in labor. One key enabling technology towards automated harvesting is accurate and robust apple detection, which poses great challenges as a result of the complex orchard environment that involves varying lighting conditions and foliage/branch occlusions. This letter reports on the development of a novel deep learning-based apple detection framework named DeepApple. Specifically, we first collect a comprehensive apple orchard dataset for 'Gala' and 'Blondee' apples, using a color camera, under different lighting conditions (sunny vs. overcast and front lighting vs. back lighting). We then develop a novel suppression Mask R-CNN for apple detection, in which a suppression branch is added to the standard Mask R-CNN to suppress non-apple features generated by the original network. Comprehensive evaluations are performed, which show that the developed suppression Mask R-CNN network outperforms state-of-the-art models with a higher F1-score of 0.905 and a detection time of 0.25 second per frame on a standard desktop computer.

cs.CV↗

arXiv2020

Privacy-Preserving Dynamic Average Consensus via State Decomposition: Case Study on Multi-Robot Formation Control

Dynamic average consensus is a decentralized control/estimation framework where a group of agents cooperatively track the average of local time-varying reference signals. In this paper, we develop a novel state decomposition-based privacy preservation scheme to protect the privacy of agents when sharing information with neighboring agents. Specifically, we first show that an external eavesdropper can successfully wiretap the reference signals of all agents in a conventional dynamic average consensus algorithm. To protect privacy against the eavesdropper, a state decomposition scheme is developed where the original state of each agent is decomposed into two sub-states: one succeeds the role of the original state in inter-node interactions, while the other sub-state only communicates with the first one and is invisible to other neighboring agents. Rigorous analyses are performed to show that 1) the proposed privacy scheme preserves the convergence of the average consensus; and 2) the privacy of the agents is protected such that an eavesdropper cannot discover the private reference signals with any guaranteed accuracy. The developed privacy-preserving dynamic average consensus framework is then applied to the formation control of multiple non-holonomic mobile robots, in which the efficacy of the scheme is demonstrated. Numerical simulation is provided to illustrate the effectiveness of the proposed approach.

eess.SY↗

arXiv2020

Privacy-Preserved Collaborative Estimation for Networked Vehicles with Application to Road Anomaly Detection

Road information such as road profile and traffic density have been widely used in intelligent vehicle systems to improve road safety, ride comfort, and fuel economy. However, vehicle heterogeneity and parameter uncertainty make it extremely difficult for a single vehicle to accurately and reliably measure such information. In this work, we propose a unified framework for learning-based collaborative estimation to fuse local road estimation from a fleet of connected heterogeneous vehicles. The collaborative estimation scheme exploits the sequential measurements made by multiple vehicles traversing the same road segment and let these vehicles relay a learning signal to iteratively refine local estimations. Given that the privacy of individual vehicles' identity must be protected in collaborative estimation, we directly incorporate privacy-protection design into the collaborative estimation design and establish a unified framework for privacy-preserving collaborative estimation. Different from patching conventional privacy mechanisms like differential privacy which will compromise algorithmic accuracy or homomorphic encryption which will incur heavy communication/computational overhead, we leverage the dynamical properties of collective estimation to enable inherent privacy protection without sacrificing accuracy or significantly increasing communication/computation overhead. Numerical simulations confirm the effectiveness and efficiency of our proposed framework.

eess.SY↗

arXiv2020

Robust Platoon Control in Mixed Traffic Flow Based on Tube Model Predictive Control

The design of cooperative adaptive cruise control is critical in mixed traffic flow, where connected and automated vehicles (CAVs) and human-driven vehicles (HDVs) coexist. Compared with pure CAVs, the major challenge is how to handle the prediction uncertainty of HDVs, which can cause significant state deviation of CAVs from planned trajectories. In most existing studies, model predictive control (MPC) is utilized to replan CAVs' trajectories to mitigate the deviation at each time step. However, as the replan process is usually conducted by solving an optimization problem with information through inter-vehicular communication, MPC methods suffer from heavy computational and communicational burdens. To address this limitation, a robust platoon control framework is proposed based on tube MPC in this paper. The prediction uncertainty is dynamically mitigated by the feedback control and restricted inside a set with a high probability. When the uncertainty exceeds the set or additional external disturbance emerges, the feedforward control is triggered to plan a ``tube'' (a sequence of the set), which can bound CAVs' actual trajectories. As the replan process is usually not required, the proposed method is much more efficient regarding computation and communication, compared with the MPC method. Comprehensive simulations are provided to validate the effectiveness of the proposed framework.

eess.SY↗

arXiv2020

MPC-based Vibration Control and Energy Harvesting Using an Electromagnetic Vibration Absorber With Inertia Nonlinearity

Simultaneous vibration control and energy harvesting of vehicle suspensions has attracted great research interests over the past decades. However, existing frameworks tradeoff suspension performance for energy recovery and are only responsive to narrow-bandwidth vibrations. In this paper, a new energy-regenerative vibration absorber (ERVA) using a ball-screw mechanism is investigated. The ERVA system is based on a rotary electromagnetic generator with adjustable nonlinear rotational inertia which passively increases the moment of inertia as the vibration amplitude increases. This structure is effective for energy harvesting and vibration control without increasing the suspension size. Furthermore, a nonlinear model predictive controller (NMPC) is applied to the system for further performance enhancement where we exploit road profile information as a preview. The performance of NMPC-based ERVA is demonstrated in a number of simulations and superior performance is demonstrated.

eess.SY↗

arXiv2019

Autonomous Driving using Safe Reinforcement Learning by Incorporating a Regret-based Human Lane-Changing Decision Model

It is expected that many human drivers will still prefer to drive themselves even if the self-driving technologies are ready. Therefore, human-driven vehicles and autonomous vehicles (AVs) will coexist in a mixed traffic for a long time. To enable AVs to safely and efficiently maneuver in this mixed traffic, it is critical that the AVs can understand how humans cope with risks and make driving-related decisions. On the other hand, the driving environment is highly dynamic and ever-changing, and it is thus difficult to enumerate all the scenarios and hard-code the controllers. To face up these challenges, in this work, we incorporate a human decision-making model in reinforcement learning to control AVs for safe and efficient operations. Specifically, we adapt regret theory to describe a human driver's lane-changing behavior, and fit the personalized models to individual drivers for predicting their lane-changing decisions. The predicted decisions are incorporated in the safety constraints for reinforcement learning in training and in implementation. We then use an extended version of double deep Q-network (DDQN) to train our AV controller within the safety set. By doing so, the amount of collisions in training is reduced to zero, while the training accuracy is not impinged.

cs.RO↗

arXiv2019

A New Microscopic Traffic Model Using a Spring-Mass-Damper-Clutch System

Microscopic traffic models describe how cars interact with their neighbors in an uninterrupted traffic flow and are frequently used for reference in advanced vehicle control design. In this paper, we propose a novel mechanical system inspired microscopic traffic model using a mass-spring-damper-clutch system. This model naturally captures the ego vehicle's resistance to large relative speed and deviation from a (driver and speed dependent) desired relative distance when following the lead vehicle. Comparing to existing car following (CF) models, this model offers physically interpretable insights on the underlying CF dynamics, and is able to characterize the impact of the ego vehicle on the lead vehicle, which is neglected in existing CF models. Thanks to the nonlinear wave propagation analysis techniques for mechanical systems, the proposed model therefore has great scalability so that multiple mass-spring-damper-clutch system can be chained to study the macroscopic traffic flow. We investigate the stability of the proposed model on the system parameters and the time delay using spectral element method. We also develop a parallel recursive least square with inverse QR decomposition (PRLS-IQR) algorithm to identify the model parameters online. These real-time estimated parameters can be used to predict the driving trajectory that can be incorporated in advanced vehicle longitudinal control systems for improved safety and fuel efficiency. The PRLS-IQR is computationally efficient and numerically stable so it is suitable for online implementation. The traffic model and the parameter identification algorithm are validated on both simulations and naturalistic driving data from multiple drivers. Promising performance is demonstrated.

eess.SY↗

arXiv2019

Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control

Reinforcement learning (RL) is a promising data-driven approach for adaptive traffic signal control (ATSC) in complex urban traffic networks, and deep neural networks further enhance its learning power. However, centralized RL is infeasible for large-scale ATSC due to the extremely high dimension of the joint action space. Multi-agent RL (MARL) overcomes the scalability issue by distributing the global control to each local RL agent, but it introduces new challenges: now the environment becomes partially observable from the viewpoint of each local agent due to limited communication among agents. Most existing studies in MARL focus on designing efficient communication and coordination among traditional Q-learning agents. This paper presents, for the first time, a fully scalable and decentralized MARL algorithm for the state-of-the-art deep RL agent: advantage actor critic (A2C), within the context of ATSC. In particular, two methods are proposed to stabilize the learning procedure, by improving the observability and reducing the learning difficulty of each local agent. The proposed multi-agent A2C is compared against independent A2C and independent Q-learning algorithms, in both a large synthetic traffic grid and a large real-world traffic network of Monaco city, under simulated peak-hour traffic dynamics. Results demonstrate its optimality, robustness, and sample efficiency over other state-of-the-art decentralized MARL algorithms.

cs.LG↗

arXiv2017

Cloud Resource Allocation for Cloud-Based Automotive Applications

There is a rapidly growing interest in the use of cloud computing for automotive vehicles to facilitate computation and data intensive tasks. Efficient utilization of on-demand cloud resources holds a significant potential to improve future vehicle safety, comfort, and fuel economy. In the meanwhile, issues like cyber security and resource allocation pose great challenges. In this paper, we treat the resource allocation problem for cloud-based automotive systems. Both private and public cloud paradigms are considered where a private cloud provides an internal, company-owned internet service dedicated to its own vehicles while a public cloud serves all subscribed vehicles. This paper establishes comprehensive models of cloud resource provisioning for both private and public cloud- based automotive systems. Complications such as stochastic communication delays and task deadlines are explicitly considered. In particular, a centralized resource provisioning model is developed for private cloud and chance constrained optimization is exploited to utilize the cloud resources for best Quality of Services. On the other hand, a decentralized auction-based model is developed for public cloud and reinforcement learning is employed to obtain an optimal bidding policy for a "selfish" agent. Numerical examples are presented to illustrate the effectiveness of the developed techniques.

eess.SY↗

arXiv2017

H-infinity Filtering for Cloud-Aided Semi-active Suspension with Delayed Information

This chapter presents an H-infinity filtering framework for cloud-aided semiactive suspension system with time-varying delays. In this system, road profile information is downloaded from a cloud database to facilitate onboard estimation of suspension states. Time-varying data transmission delays are considered and assumed to be bounded. A quarter-car linear suspension model is used and an H-infinity filter is designed with both onboard sensor measurements and delayed road profile information from the cloud. The filter design procedure is designed based on linear matrix inequalities (LMIs). Numerical simulation results are reported that illustrates the fusion of cloud-based and on-board information that can be achieved in Vehicleto- Cloud-to-Vehicle (V2C2V) implementation.

eess.SY↗