arXiv subjects

Sangjae Bae

Publications and source records attributed to Sangjae Bae.

44 records · Page 3Linked to original sources

arXiv2022

Driving in Dense Traffic with Model-Free Reinforcement Learning

Traditional planning and control methods could fail to find a feasible trajectory for an autonomous vehicle to execute amongst dense traffic on roads. This is because the obstacle-free volume in spacetime is very small in these scenarios for the vehicle to drive through. However, that does not mean the task is infeasible since human drivers are known to be able to drive amongst dense traffic by leveraging the cooperativeness of other drivers to open a gap. The traditional methods fail to take into account the fact that the actions taken by an agent affect the behaviour of other vehicles on the road. In this work, we rely on the ability of deep reinforcement learning to implicitly model such interactions and learn a continuous control policy over the action space of an autonomous vehicle. The application we consider requires our agent to negotiate and open a gap in the road in order to successfully merge or change lanes. Our policy learns to repeatedly probe into the target road lane while trying to find a safe spot to move in to. We compare against two model-predictive control-based algorithms and show that our policy outperforms them in simulation.

cs.RO↗

arXiv2022

Spatiotemporal Costmap Inference for MPC via Deep Inverse Reinforcement Learning

It can be difficult to autonomously produce driver behavior so that it appears natural to other traffic participants. Through Inverse Reinforcement Learning (IRL), we can automate this process by learning the underlying reward function from human demonstrations. We propose a new IRL algorithm that learns a goal-conditioned spatiotemporal reward function. The resulting costmap is used by Model Predictive Controllers (MPCs) to perform a task without any hand-designing or hand-tuning of the cost function. We evaluate our proposed Goal-conditioned SpatioTemporal Zeroing Maximum Entropy Deep IRL (GSTZ)-MEDIRL framework together with MPC in the CARLA simulator for autonomous driving, lane keeping, and lane changing tasks in a challenging dense traffic highway scenario. Our proposed methods show higher success rates compared to other baseline methods including behavior cloning, state-of-the-art RL policies, and MPC with a learning-based behavior prediction model.

cs.RO↗

arXiv2021

Risk-Aware Lane Selection on Highway with Dynamic Obstacles

This paper proposes a discretionary lane selection algorithm. In particular, highway driving is considered as a targeted scenario, where each lane has a different level of traffic flow. When lane-changing is discretionary, it is advised not to change lanes unless highly beneficial, e.g., reducing travel time significantly or securing higher safety. Evaluating such "benefit" is a challenge, along with multiple surrounding vehicles in dynamic speed and heading with uncertainty. We propose a real-time lane-selection algorithm with careful cost considerations and with modularity in design. The algorithm is a search-based optimization method that evaluates uncertain dynamic positions of other vehicles under a continuous time and space domain. For demonstration, we incorporate a state-of-the-art motion planner framework (Neural Networks integrated Model Predictive Control) under a CARLA simulation environment.

cs.RO↗

arXiv2020

Optimal Dispatch of Electrified Autonomous Mobility on Demand Vehicles during Power Outages

The era of fully autonomous, electrified taxi fleets is rapidly approaching, and with it the opportunity to innovate myriad on-demand services that extend beyond the realm of human mobility. This project envisions a future where autonomous plug-in electric vehicle (PEV) fleets can be dispatched as both a taxi service and a source of on-demand power serving customers during power outages. We develop a PDE-based scheme to manage the optimal dispatch of an autonomous fleet to serve passengers and electric power demand during outages as an additional stream of revenue. We use real world power outage and taxi data from San Francisco for our case study, modeling the optimal dispatch of several fleet sizes over the course of one day; we examine both moderate and extreme outage scenarios. In the moderate scenario, the revenue earned serving power demand is negligible compared with revenue earned serving passenger trips. In the extreme scenario, supplying power accounts for between $1 and $2 million, amounting to between 32\% and 40\% more revenue than is earned serving mobility only, depending on fleet size. While the overall value of providing on-demand power depends on the frequency and severity of power outages, our results show that serving power demand during large-scale outages can provide a substantial value stream, comparable to the value to be earned providing grid services.

eess.SY↗

arXiv2019

Inducing Human Behavior to Alleviate Overstay at PEV Charging Station

As the plug-in electric vehicle (PEV) market expands worldwide, PEV penetration has out-paced public PEV charging accessibility. In addition to charging infrastructure deployment, charging station operation is another key factor for improving charging service accessibility. In this paper, we propose a mathematical framework to optimally operate a PEV charging station, whose service capability is constrained by the number of available chargers. This mathematical framework specifically exploits human behavioral modeling to alleviate the "overstaying" issue that occurs when a vehicle is fully charged. Our behavioral model effectively captures human decision-making when humans are exposed to multiple charging product options, which differ in both price and quality-of-service. We reformulate the associated non-convex problem to a multi-convex problem via the Young-Fenchel transform. We then apply the Block Coordinate Descent algorithm to efficiently solve the optimization problem. Numerical experiments illustrate the performance of the proposed method. Simulation results show that a station operator who leverages optimally priced charging options could realize benefits in three ways: (i) net profits gains, (ii) overstay reduction, and (iii) increased quality-of-service.

eess.SY↗

arXiv2019

Real-time Ecological Velocity Planning for Plug-in Hybrid Vehicles with Partial Communication to Traffic Lights

This paper presents the design of an ecological adaptive cruise controller (ECO-ACC) for a plug-in hybrid vehicle (PHEV) which exploits automated driving and connectivity. Most existing papers for ECO-ACC focus on a short-sighted control scheme. A two-level control framework for long-sighted ECO-ACC was only recently introduced. However, that work is based on a deterministic traffic signal phase and timing (SPaT) over the entire route. In practice, connectivity with traffic lights may be limited by communication range, e.g. just one upcoming traffic light. We propose a two-level receding-horizon control framework for long-sighted ECO-ACC that exploits deterministic SPaT for the upcoming traffic light, and utilizes historical SPaT for other traffic lights within a receding control horizon. We also incorporate a powertrain control mechanism to enhance PHEV energy prediction accuracy. Hardware-in-the-loop simulation results validate the energy savings of the receding-horizon control framework in various traffic scenarios.

eess.SY↗

arXiv2019

Cooperation-Aware Lane Change Maneuver in Dense Traffic based on Model Predictive Control with Recurrent Neural Network

This paper presents a real-time lane change control framework of autonomous driving in dense traffic, which exploits cooperative behaviors of other drivers. This paper focuses on heavy traffic where vehicles cannot change lanes without cooperating with other drivers. In this case, classical robust controls may not apply since there is no safe area to merge to without interacting with the other drivers. That said, modeling complex and interactive human behaviors is highly non-trivial from the perspective of control engineers. We propose a mathematical control framework based on Model Predictive Control (MPC) encompassing a state-of-the-art Recurrent Neural network (RNN) architecture. In particular, RNN predicts interactive motions of other drivers in response to potential actions of the autonomous vehicle, which are then systematically evaluated in safety constraints. We also propose a real-time heuristic algorithm to find locally optimal control inputs. Finally, quantitative and qualitative analysis on simulation studies are presented to illustrate the benefits of the proposed framework.

cs.RO↗

arXiv2018

Design and Implementation of Ecological Adaptive Cruise Control for Autonomous Driving with Communication to Traffic Lights

This paper presents the design and implementation results of an ecological adaptive cruise controller (ECO-ACC) which exploits driving automation and connectivity. The controller avoids front collisions and traffic light violations, and is designed to reduce the energy consumption of connected automated vehicles by utilizing historical and real-time signal phase and timing data of traffic lights that adapt to the current traffic conditions. We propose an optimization-based generation of a reference velocity, and a velocity-tracking model predictive controller that avoids front collisions and violations. We present an experimental setup encompassing the real vehicle and controller in the loop, and an environment simulator in which the traffic flow and the traffic light patterns are calibrated on real-world data. We present and analyze simulation and experimental results, finding a significant potential for energy consumption reduction, even in the presence of traffic.

eess.SY↗