arXiv subjects

H. Eric Tseng

Publications and source records attributed to H. Eric Tseng.

At least 37 records · Page 2Linked to original sources

arXiv2023

Decision-Making for Autonomous Vehicles with Interaction-Aware Behavioral Prediction and Social-Attention Neural Network

Autonomous vehicles need to accomplish their tasks while interacting with human drivers in traffic. It is thus crucial to equip autonomous vehicles with artificial reasoning to better comprehend the intentions of the surrounding traffic, thereby facilitating the accomplishments of the tasks. In this work, we propose a behavioral model that encodes drivers' interacting intentions into latent social-psychological parameters. Leveraging a Bayesian filter, we develop a receding-horizon optimization-based controller for autonomous vehicle decision-making which accounts for the uncertainties in the interacting drivers' intentions. For online deployment, we design a neural network architecture based on the attention mechanism which imitates the behavioral model with online estimated parameter priors. We also propose a decision tree search algorithm to solve the decision-making problem online. The proposed behavioral model is then evaluated in terms of its capabilities for real-world trajectory prediction. We further conduct extensive evaluations of the proposed decision-making module, in forced highway merging scenarios, using both simulated environments and real-world traffic datasets. The results demonstrate that our algorithms can complete the forced merging tasks in various traffic conditions while ensuring driving safety.

cs.AI↗

arXiv2023

Predictive Control for Autonomous Driving with Uncertain, Multi-modal Predictions

We propose a Stochastic MPC (SMPC) formulation for path planning with autonomous vehicles in scenarios involving multiple agents with multi-modal predictions. The multi-modal predictions capture the uncertainty of urban driving in distinct modes/maneuvers (e.g., yield, keep speed) and driving trajectories (e.g., speed, turning radius), which are incorporated for multi-modal collision avoidance chance constraints for path planning. In the presence of multi-modal uncertainties, it is challenging to reliably compute feasible path planning solutions at real-time frequencies ($\geq$ 10 Hz). Our main technological contribution is a convex SMPC formulation that simultaneously (1) optimizes over parameterized feedback policies and (2) allocates risk levels for each mode of the prediction. The use of feedback policies and risk allocation enhances the feasibility and performance of the SMPC formulation against multi-modal predictions with large uncertainty. We evaluate our approach via simulations and road experiments with a full-scale vehicle interacting in closed-loop with virtual vehicles. We consider distinct, multi-modal driving scenarios: 1) Negotiating a traffic light and a fast, tailgating agent, 2) Executing an unprotected left turn at a traffic intersection, and 3) Changing lanes in the presence of multiple agents. For all of these scenarios, our approach reliably computes multi-modal solutions to the path-planning problem at real-time frequencies.

cs.RO↗

arXiv2023

Nash or Stackelberg? -- A comparative study for game-theoretic AV decision-making

This paper studies game-theoretic decision-making for autonomous vehicles (AVs). A receding horizon multi-player game is formulated to model the AV decision-making problem. Two classes of games, including Nash game and Stackelber games, are developed respectively. For each of the two games, two solution settings, including pairwise games and multi-player games, are introduced, respectively, to solve the game in multi-agent scenarios. Comparative studies are conducted via statistical simulations to gain understandings of the performance of the two classes of games and of the two solution settings, respectively. The simulations are conducted in intersection-crossing scenarios, and the game performance is quantified by three metrics: safety, travel efficiency, and computational time.

cs.GT↗

arXiv2023

Interaction-Aware Decision-Making for Autonomous Vehicles in Forced Merging Scenario Leveraging Social Psychology Factors

Understanding the intention of vehicles in the surrounding traffic is crucial for an autonomous vehicle to successfully accomplish its driving tasks in complex traffic scenarios such as highway forced merging. In this paper, we consider a behavioral model that incorporates both social behaviors and personal objectives of the interacting drivers. Leveraging this model, we develop a receding-horizon control-based decision-making strategy, that estimates online the other drivers' intentions using Bayesian filtering and incorporates predictions of nearby vehicles' behaviors under uncertain intentions. The effectiveness of the proposed decision-making strategy is demonstrated and evaluated based on simulation studies in comparison with a game theoretic controller and a real-world traffic dataset.

cs.AI↗

arXiv2023

Improved Robustness and Safety for Pre-Adaptation of Meta Reinforcement Learning with Prior Regularization

Meta Reinforcement Learning (Meta-RL) has seen substantial advancements recently. In particular, off-policy methods were developed to improve the data efficiency of Meta-RL techniques. \textit{Probabilistic embeddings for actor-critic RL} (PEARL) is a leading approach for multi-MDP adaptation problems. A major drawback of many existing Meta-RL methods, including PEARL, is that they do not explicitly consider the safety of the prior policy when it is exposed to a new task for the first time. Safety is essential for many real-world applications, including field robots and Autonomous Vehicles (AVs). In this paper, we develop the PEARL PLUS (PEARL$^+$) algorithm, which optimizes the policy for both prior (pre-adaptation) safety and posterior (after-adaptation) performance. Building on top of PEARL, our proposed PEARL$^+$ algorithm introduces a prior regularization term in the reward function and a new Q-network for recovering the state-action value under prior context assumptions, to improve the robustness to task distribution shift and safety of the trained network exposed to a new task for the first time. The performance of PEARL$^+$ is validated by solving three safety-critical problems related to robots and AVs, including two MuJoCo benchmark problems. From the simulation experiments, we show that safety of the prior policy is significantly improved and more robust to task distribution shift compared to PEARL.

cs.LG↗

arXiv2022

REFINE: Reachability-based Trajectory Design using Robust Feedback Linearization and Zonotopes

Performing real-time receding horizon motion planning for autonomous vehicles while providing safety guarantees remains difficult. This is because existing methods to accurately predict ego vehicle behavior under a chosen controller use online numerical integration that requires a fine time discretization and thereby adversely affects real-time performance. To address this limitation, several recent papers have proposed to apply offline reachability analysis to conservatively predict the behavior of the ego vehicle. This reachable set can be constructed by utilizing a simplified model whose behavior is assumed a priori to conservatively bound the dynamics of a full-order model. However, guaranteeing that one satisfies this assumption is challenging. This paper proposes a framework named REFINE to overcome the limitations of these existing approaches. REFINE utilizes a parameterized robust controller that partially linearizes the vehicle dynamics even in the presence of modeling error. Zonotope-based reachability analysis is then performed on the closed-loop, full-order vehicle dynamics to compute the corresponding control-parameterized, over-approximate Forward Reachable Sets (FRS). Because reachability analysis is applied to the full-order model, the potential conservativeness introduced by using a simplified model is avoided. The pre-computed, control-parameterized FRS is then used online in an optimization framework to ensure safety. The proposed method is compared to several state of the art methods during a simulation-based evaluation on a full-size vehicle model and is evaluated on a 1/10th race car robot in real hardware testing. In contrast to existing methods, REFINE is shown to enable the vehicle to safely navigate itself through complex environments.

cs.RO↗

arXiv2022

Safe, Learning-Based MPC for Highway Driving under Lane-Change Uncertainty: A Distributionally Robust Approach

We present a case study applying learning-based distributionally robust model predictive control to highway motion planning under stochastic uncertainty of the lane change behavior of surrounding road users. The dynamics of road users are modelled using Markov jump systems, in which the switching variable describes the desired lane of the vehicle under consideration and the continuous state describes the pose and velocity of the vehicles. We assume the switching probabilities of the underlying Markov chain to be unknown. As the vehicle is observed and thus, samples from the Markov chain are drawn, the transition probabilities are estimated along with an ambiguity set which accounts for misestimations of these probabilities. Correspondingly, a distributionally robust optimal control problem is formulated over a scenario tree, and solved in receding horizon. As a result, a motion planning procedure is obtained which through observation of the target vehicle gradually becomes less conservative while avoiding overconfidence in estimates obtained from small sample sizes. We present an extensive numerical case study, comparing the effects of several different design aspects on the controller performance and safety.

eess.SY↗

arXiv2022

Robust Action Governor for Uncertain Piecewise Affine Systems with Non-convex Constraints and Safe Reinforcement Learning

The action governor is an add-on scheme to a nominal control loop that monitors and adjusts the control actions to enforce safety specifications expressed as pointwise-in-time state and control constraints. In this paper, we introduce the Robust Action Governor (RAG) for systems the dynamics of which can be represented using discrete-time Piecewise Affine (PWA) models with both parametric and additive uncertainties and subject to non-convex constraints. We develop the theoretical properties and computational approaches for the RAG. After that, we introduce the use of the RAG for realizing safe Reinforcement Learning (RL), i.e., ensuring all-time constraint satisfaction during online RL exploration-and-exploitation process. This development enables safe real-time evolution of the control policy and adaptation to changes in the operating environment and system parameters (due to aging, damage, etc.). We illustrate the effectiveness of the RAG in constraint enforcement and safe RL using the RAG by considering their applications to a soft-landing problem of a mass-spring-damper system.

eess.SY↗

arXiv2022

Vehicle Models and Optimal Control on a Nonplanar Surface

We present a 10 DoF dynamic vehicle model for model-based control on nonplanar road surfaces. A parametric surface is used to describe the road surface, allowing the surface parameterization to describe the pose of the vehicle. We use the proposed approach to compute minimum-time vehicle trajectories on nonplanar surfaces and compare planar and nonplanar models.

eess.SY↗

arXiv2022

Overtaking Maneuvers on a Nonplanar Racetrack

We leverage game theory and a new vehicle modeling approach to compute overtaking maneuvers for racecars on a nonplanar surface. We solve for equilibria between noncooperative racing agents and demonstrate that by leveraging the novel nonplanar vehicle dynamics, overtaking can be achieved in situations where simpler models can do not provide a winning strategy.

eess.SY↗

arXiv2022

Models and Predictive Control for Nonplanar Vehicle Navigation

We present a simplified model of a vehicle driving on a nonplanar road. A parametric surface is used to describe the nonplanar road which can describe any combination of curvature, bank and slope. We show that the proposed modeling approach generalizes planar vehicle models that reference a centerline, such as the Frenet model. We use the proposed approach for vehicle path planning and following using model predictive control. We also model and control vehicle contact with the road surface. We demonstrate that the proposed controller improves speed and lane following on complex roads compared to planar vehicle controllers, and mitigates loss of control on complex road surfaces including off-camber turns.

eess.SY↗

arXiv2022

Stochastic MPC with Multi-modal Predictions for Traffic Intersections

We propose a Stochastic MPC (SMPC) formulation for autonomous driving at traffic intersections which incorporates multi-modal predictions of surrounding vehicles for collision avoidance constraints. The multi-modal predictions are obtained with Gaussian Mixture Models (GMM) and constraints are formulated as chance-constraints. Our main theoretical contribution is a SMPC formulation that optimizes over a novel feedback policy class designed to exploit additional structure in the GMM predictions, and that is amenable to convex programming. The use of feedback policies for prediction is motivated by the need for reduced conservatism in handling multi-modal predictions of the surrounding vehicles, especially prevalent in traffic intersection scenarios. We evaluate our algorithm along axes of mobility, comfort, conservatism and computational efficiency at a simulated intersection in CARLA. Our simulations use a kinematic bicycle model and multimodal predictions trained on a subset of the Lyft Level 5 prediction dataset. To demonstrate the impact of optimizing over feedback policies, we compare our algorithm with two SMPC baselines that handle multi-modal collision avoidance chance constraints by optimizing over open-loop sequences.

eess.SY↗

arXiv2021

Interaction-Aware Trajectory Prediction and Planning for Autonomous Vehicles in Forced Merge Scenarios

Merging is, in general, a challenging task for both human drivers and autonomous vehicles, especially in dense traffic, because the merging vehicle typically needs to interact with other vehicles to identify or create a gap and safely merge into. In this paper, we consider the problem of autonomous vehicle control for forced merge scenarios. We propose a novel game-theoretic controller, called the Leader-Follower Game Controller (LFGC), in which the interactions between the autonomous ego vehicle and other vehicles with a priori uncertain driving intentions is modeled as a partially observable leader-follower game. The LFGC estimates the other vehicles' intentions online based on observed trajectories, and then predicts their future trajectories and plans the ego vehicle's own trajectory using Model Predictive Control (MPC) to simultaneously achieve probabilistically guaranteed safety and merging objectives. To verify the performance of LFGC, we test it in simulations and with the NGSIM data, where the LFGC demonstrates a high success rate of 97.5% in merging.

eess.SY↗

arXiv2021

Safe Reinforcement Learning Using Robust Action Governor

Reinforcement Learning (RL) is essentially a trial-and-error learning procedure which may cause unsafe behavior during the exploration-and-exploitation process. This hinders the application of RL to real-world control problems, especially to those for safety-critical systems. In this paper, we introduce a framework for safe RL that is based on integration of a RL algorithm with an add-on safety supervision module, called the Robust Action Governor (RAG), which exploits set-theoretic techniques and online optimization to manage safety-related requirements during learning. We illustrate this proposed safe RL framework through an application to automotive adaptive cruise control.

cs.LG↗

arXiv2021

Sensitivity Analysis of Passenger Behavioral Model for Dynamic Pricing of Shared Mobility on Demand

This paper provides a framework to quantify the sensitivity associated with behavioral models based on Cumulative Prospect Theory (CPT). These are used to design dynamic pricing strategies aimed at maximizing performance metrics of the Shared Mobility On-Demand Service (SMoDS), as solutions to a constrained nonlinear optimization problem. We analyze the sensitivity of both the optimal tariff as well as the optimal objective function with respect to CPT model parameters. In addition to deriving analytical solutions under certain assumptions, more general numerical results are obtained via computational experiments and simulations to analyze the sensitivity. We find that the model is relatively robust for small to moderate parameter perturbations. Although some of the trends in sensitivity are fairly general, the exact nature of variations in many cases depends heavily on the specific travel scenarios and modes being considered. This is primarily due to the complex nonlinearities in the problem, as well as the significant heterogeneity in passenger preferences across different types of trips.

math.OC↗

arXiv2021

Quick Learner Automated Vehicle Adapting its Roadmanship to Varying Traffic Cultures with Meta Reinforcement Learning

It is essential for an automated vehicle in the field to perform discretionary lane changes with appropriate roadmanship - driving safely and efficiently without annoying or endangering other road users - under a wide range of traffic cultures and driving conditions. While deep reinforcement learning methods have excelled in recent years and been applied to automated vehicle driving policy, there are concerns about their capability to quickly adapt to unseen traffic with new environment dynamics. We formulate this challenge as a multi-Markov Decision Processes (MDPs) adaptation problem and developed Meta Reinforcement Learning (MRL) driving policies to showcase their quick learning capability. Two types of distribution variation in environments were designed and simulated to validate the fast adaptation capability of resulting MRL driving policies which significantly outperform a baseline RL.

cs.LG↗

arXiv2020

Towards Dynamic Pricing for Shared Mobility on Demand using Markov Decision Processes and Dynamic Programming

In a Shared Mobility on Demand Service (SMoDS), dynamic pricing plays an important role in the form of an incentive for the decision of the empowered passenger on the ride offer. Strategies for determining the dynamic tariff should be suitably designed so that the incurred demand and supply are balanced and therefore economic efficiency is achieved. In this manuscript, we formulate a discrete time Markov Decision Process (MDP) to determine the probability desired by the SMoDS platform corresponding to the acceptance rate of each empowered passenger at each state of the system. We use Estimated Waiting Time (EWT) as the metric for the balance between demand and supply, with the goal that EWT be regulated around a target value. We then develop a Dynamic Programming (DP) algorithm to derive the optimal policy of the MDP that regulates EWT around the target value. Computational experiments are conducted that demonstrate the regulation of EWT is effective, through various scenarios. The overall demonstration is carried out offline. The MDP formulation together with the DP algorithm can be utilized to an online determination of the dynamic tariff by integrating with our earlier works on Cumulative Prospect Theory based passenger behavioral modeling and the AltMin dynamic routing algorithm, and form the subject of future works.

math.OC↗

arXiv2020

Action Governor for Discrete-Time Linear Systems with Non-Convex Constraints

This paper introduces an add-on, supervisory scheme, referred to as Action Governor (AG), for discrete-time linear systems to enforce exclusion-zone avoidance requirements. It does so by monitoring, and minimally modifying when necessary, the nominal control signal to a constraint-admissible one. The AG operates based on set-theoretic techniques and online optimization. This paper establishes its theoretical foundation, discusses its computational realization, and uses two simulation examples to illustrate its effectiveness.

eess.SY↗