arXiv subjects

Alexandre Bayen

Publications and source records attributed to Alexandre Bayen.

At least 37 records · Page 2Linked to original sources

arXiv2022

A rigorous multi-population multi-lane hybrid traffic model and its mean-field limit for dissipation of waves via autonomous vehicles

In this paper, a multi-lane multi-population microscopic model, which presents stop and go waves, is proposed to simulate traffic on a ring-road. Vehicles are divided between human-driven and autonomous vehicles (AV). Control strategies are designed with the ultimate goal of using a small number of AVs (less than 5\% penetration rate) to represent Lagrangian control actuators that can smooth the multilane traffic flow and dissipate the stop-and-go waves. This in turn may reduce fuel consumption and emissions. The lane-changing mechanism is based on three components that we treat as parameters in the model: safety, incentive and cool-down time. The choice of these parameters in the lane-change mechanism is critical to modeling traffic accurately, because different parameter values can lead to drastically different traffic behaviors. In particular, the number of lane-changes and the speed variance are highly affected by the choice of parameters. Despite this modeling issue, when using sufficiently simple and robust controllers for AVs, the stabilization of uniform flow steady-state is effective for any realistic value of the parameters, and ultimately bypasses the observed modeling issue. Our approach is based on accurate and rigorous mathematical models, which allows a limit procedure that is termed, in gas dynamic terminology, mean-field. In simple words, from increasing the human-driven population to infinity, a system of coupled ordinary and partial differential equations are obtained. Moreover, control problems also pass to the limit, allowing the design to be tackled at different scales.

eess.SY↗

arXiv2022

Limitations and Improvements of the Intelligent Driver Model (IDM)

This contribution analyzes the widely used and well-known "intelligent driver model (briefly IDM), which is a second order car-following model governed by a system of ordinary differential equations. Although this model was intensively studied in recent years for properly capturing traffic phenomena and driver braking behavior, a rigorous study of the well-posedness has, to our knowledge, never been performed. First it is shown that, for a specific class of initial data, the vehicles' velocities become negative or even diverge to $-\infty$ in finite time, both undesirable properties for a car-following model. Various modifications of the IDM are then proposed in order to avoid such ill-posedness. The theoretical remediation of the model, rather than post facto by ad-hoc modification of code implementations, allows a more sound numerical implementation and preservation of the model features. Indeed, to avoid inconsistencies and ensure dynamics close to the one of the original model, one may need to inspect and clean large input data, which may result in practically impossible scenarios for large-scale simulations. Although well-posedness issues occur only for specific initial data, this may happen frequently when different traffic scenarios are analyzed, and especially in presence of lane-changing, on ramps and other network components as it is the case for most commonly used micro-simulators. On the other side, it is shown that well-posedness can be guaranteed by straight-forward improvements, such as those obtained by slightly changing the acceleration to prevent the velocity from becoming negative.

eess.SY↗

arXiv2021

Learning Generalizable Multi-Lane Mixed-Autonomy Behaviors in Single Lane Representations of Traffic

Reinforcement learning techniques can provide substantial insights into the desired behaviors of future autonomous driving systems. By optimizing for societal metrics of traffic such as increased throughput and reduced energy consumption, such methods can derive maneuvers that, if adopted by even a small portion of vehicles, may significantly improve the state of traffic for all vehicles involved. These methods, however, are hindered in practice by the difficulty of designing efficient and accurate models of traffic, as well as the challenges associated with optimizing for the behaviors of dozens of interacting agents. In response to these challenges, this paper tackles the problem of learning generalizable traffic control strategies in simple representations of vehicle driving dynamics. In particular, we look to mixed-autonomy ring roads as depictions of instabilities that result in the formation of congestion. Within this problem, we design a curriculum learning paradigm that exploits the natural extendability of the network to effectively learn behaviors that reduce congestion over long horizons. Next, we study the implications of modeling lane changing on the transferability of policies. Our findings suggest that introducing lane change behaviors that even approximately match trends in more complex systems can significantly improve the generalizability of subsequent learned models to more accurate multi-lane models of traffic.

cs.MA↗

arXiv2021

Parallel Network Flow Allocation in Repeated Routing Games via LQR Optimal Control

In this article, we study the repeated routing game problem on a parallel network with affine latency functions on each edge. We cast the game setup in a LQR control theoretic framework, leveraging the Rosenthal potential formulation. We use control techniques to analyze the convergence of the game dynamics with specific cases that lend themselves to optimal control. We design proper dynamics parameters so that the conservation of flow is guaranteed. We provide an algorithmic solution for the general optimal control setup using a multiparametric quadratic programming approach (explicit MPC). Finally we illustrate with numerics the impact of varying system parameters on the solutions.

math.OC↗

arXiv2021

Multi-Adversarial Safety Analysis for Autonomous Vehicles

This work in progress considers reachability-based safety analysis in the domain of autonomous driving in multi-agent systems. We formulate the safety problem for a car following scenario as a differential game and study how different modelling strategies yield very different behaviors regardless of the validity of the strategies in other scenarios. Given the nature of real-life driving scenarios, we propose a modeling strategy in our formulation that accounts for subtle interactions between agents, and compare its Hamiltonian results to other baselines. Our formulation encourages reduction of conservativeness in Hamilton-Jacobi safety analysis to provide better safety guarantees during navigation.

eess.SY↗

arXiv2021

Quasi-Dynamic Traffic Assignment using High Performance Computing

Traffic assignment methods are some of the key approaches used to model flow patterns that arise in transportation networks. Since static traffic assignment does not have a notion of time, it is not designed to represent temporal dynamics that arise as vehicles flow through the network and demand varies through the day. Dynamic traffic assignment methods attempt to resolve these issues, but require significant computational resources if modeling urban-scale regions (on the order of millions of links and vehicles) and often take days of compute time to complete. The focus of this work is two-fold: 1) to introduce a new traffic assignment approach - a quasi-dynamic traffic assignment (QDTA) model and 2) to describe how we parallelized the QDTA algorithms to leverage High-Performance Computing (HPC) and scale to large metropolitan areas while dramatically reducing compute time. We examine and compare different scenarios, including a baseline static traffic assignment (STA) and a quasi-dynamic scenario inspired by the user-equilibrium (UET). Results are presented for the San Francisco Bay Area which accounts for 19M trips/day and an urban road network of 1M links. We utilize an iterative gradient descent method, where the step size is selected using a Quasi-Newton method with parallelized cost function evaluations and compare it to using pre-defined step sizes (MSA). Using the parallelized line search provides a 16 percent reduction in total execution time due to a reduction in the number of gradient descent iterations required for convergence. The full day QDTA comprising 96 optimization steps over 15 minute intervals runs in about 4 minutes on 1,024 cores of the NERSC Cori computer, which represents a speedup of over 36x versus serial execution. To our knowledge, this compute time is significantly lower than other traffic assignment solutions for a problem of this scale.

cs.DC↗

arXiv2021

Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design

A wide range of reinforcement learning (RL) problems - including robustness, transfer learning, unsupervised RL, and emergent complexity - require specifying a distribution of tasks or environments in which a policy will be trained. However, creating a useful distribution of environments is error prone, and takes a significant amount of developer time and effort. We propose Unsupervised Environment Design (UED) as an alternative paradigm, where developers provide environments with unknown parameters, and these parameters are used to automatically produce a distribution over valid, solvable environments. Existing approaches to automatically generating environments suffer from common failure modes: domain randomization cannot generate structure or adapt the difficulty of the environment to the agent's learning progress, and minimax adversarial training leads to worst-case environments that are often unsolvable. To generate structured, solvable environments for our protagonist agent, we introduce a second, antagonist agent that is allied with the environment-generating adversary. The adversary is motivated to generate environments which maximize regret, defined as the difference between the protagonist and antagonist agent's return. We call our technique Protagonist Antagonist Induced Regret Environment Design (PAIRED). Our experiments demonstrate that PAIRED produces a natural curriculum of increasingly complex environments, and PAIRED agents achieve higher zero-shot transfer performance when tested in highly novel environments.

cs.LG↗

arXiv2021

Reinforcement Learning versus PDE Backstepping and PI Control for Congested Freeway Traffic

We develop reinforcement learning (RL) boundary controllers to mitigate stop-and-go traffic congestion on a freeway segment. The traffic dynamics of the freeway segment are governed by a macroscopic Aw-Rascle-Zhang (ARZ) model, consisting of $2\times 2$ quasi-linear partial differential equations (PDEs) for traffic density and velocity. Boundary stabilization of the linearized ARZ PDE model has been solved by PDE backstepping, guaranteeing spatial $L^2$ norm regulation of the traffic state to uniform density and velocity and ensuring that traffic oscillations are suppressed. Collocated Proportional (P) and Proportional-Integral (PI) controllers also provide stability guarantees under certain restricted conditions, and are always applicable as model-free control options through gain tuning by trail and error, or by model-free optimization. Although these approaches are mathematically elegant, the stabilization result only holds locally and is usually affected by the change of model parameters. Therefore, we reformulate the PDE boundary control problem as a RL problem that pursues stabilization without knowing the system dynamics, simply by observing the state values. The proximal policy optimization, a neural network-based policy gradient algorithm, is employed to obtain RL controllers by interacting with a numerical simulator of the ARZ PDE. Being stabilization-inspired, the RL state-feedback boundary controllers are compared and evaluated against the rigorously stabilizing controllers in two cases: (i) in a system with perfect knowledge of the traffic flow dynamics, and then (ii) in one with only partial knowledge. We obtain RL controllers that nearly recover the performance of the backstepping, P, and PI controllers with perfect knowledge and outperform them in some cases with partial knowledge.

math.OC↗

arXiv2020

A Graph Convolutional Network with Signal Phasing Information for Arterial Traffic Prediction

Accurate and reliable prediction of traffic measurements plays a crucial role in the development of modern intelligent transportation systems. Due to more complex road geometries and the presence of signal control, arterial traffic prediction is a level above freeway traffic prediction. Many existing studies on arterial traffic prediction only consider temporal measurements of flow and occupancy from loop sensors and neglect the rich spatial relationships between upstream and downstream detectors. As a result, they often suffer large prediction errors, especially for long horizons. We fill this gap by enhancing a deep learning approach, Diffusion Convolutional Recurrent Neural Network, with spatial information generated from signal timing plans at targeted intersections. Traffic at signalized intersections is modeled as a diffusion process with a transition matrix constructed from the phase splits of the signal phase timing plan. We apply this novel method to predict traffic flow from loop sensor measurements and signal timing plans at an arterial intersection in Arcadia, CA. We demonstrate that our proposed method yields superior forecasts; for a prediction horizon of 30 minutes, we cut the MAPE down to 16% for morning peaks, 10% for off peaks, and even 8% for afternoon peaks. In addition, we exemplify the robustness of our model through a number of experiments with various settings in detector coverage, detector type, and data quality.

cs.LG↗

arXiv2020

Optimizing Mixed Autonomy Traffic Flow With Decentralized Autonomous Vehicles and Multi-Agent RL

We study the ability of autonomous vehicles to improve the throughput of a bottleneck using a fully decentralized control scheme in a mixed autonomy setting. We consider the problem of improving the throughput of a scaled model of the San Francisco-Oakland Bay Bridge: a two-stage bottleneck where four lanes reduce to two and then reduce to one. Although there is extensive work examining variants of bottleneck control in a centralized setting, there is less study of the challenging multi-agent setting where the large number of interacting AVs leads to significant optimization difficulties for reinforcement learning methods. We apply multi-agent reinforcement algorithms to this problem and demonstrate that significant improvements in bottleneck throughput, from 20\% at a 5\% penetration rate to 33\% at a 40\% penetration rate, can be achieved. We compare our results to a hand-designed feedback controller and demonstrate that our results sharply outperform the feedback controller despite extensive tuning. Additionally, we demonstrate that the RL-based controllers adopt a robust strategy that works across penetration rates whereas the feedback controllers degrade immediately upon penetration rate variation. We investigate the feasibility of both action and observation decentralization and demonstrate that effective strategies are possible using purely local sensing. Finally, we open-source our code at https://github.com/eugenevinitsky/decentralized_bottlenecks.

eess.SY↗

arXiv2020

Robust Reinforcement Learning using Adversarial Populations

Reinforcement Learning (RL) is an effective tool for controller design but can struggle with issues of robustness, failing catastrophically when the underlying system dynamics are perturbed. The Robust RL formulation tackles this by adding worst-case adversarial noise to the dynamics and constructing the noise distribution as the solution to a zero-sum minimax game. However, existing work on learning solutions to the Robust RL formulation has primarily focused on training a single RL agent against a single adversary. In this work, we demonstrate that using a single adversary does not consistently yield robustness to dynamics variations under standard parametrizations of the adversary; the resulting policy is highly exploitable by new adversaries. We propose a population-based augmentation to the Robust RL formulation in which we randomly initialize a population of adversaries and sample from the population uniformly during training. We empirically validate across robotics benchmarks that the use of an adversarial population results in a more robust policy that also improves out-of-distribution generalization. Finally, we demonstrate that this approach provides comparable robustness and generalization as domain randomization on these benchmarks while avoiding a ubiquitous domain randomization failure mode.

cs.LG↗

arXiv2020

On the Approximability of Time Disjoint Walks

We introduce the combinatorial optimization problem Time Disjoint Walks (TDW), which has applications in collision-free routing of discrete objects (e.g., autonomous vehicles) over a network. This problem takes as input a digraph $G$ with positive integer arc lengths, and $k$ pairs of vertices that each represent a trip demand from a source to a destination. The goal is to find a walk and delay for each demand so that no two trips occupy the same vertex at the same time, and so that a min-max or min-sum objective over the trip durations is realized. We focus here on the min-sum variant of Time Disjoint Walks, although most of our results carry over to the min-max case. We restrict our study to various subclasses of DAGs, and observe that there is a sharp complexity boundary between Time Disjoint Walks on oriented stars and on oriented stars with the central vertex replaced by a path. In particular, we present a poly-time algorithm for min-sum and min-max TDW on the former, but show that min-sum TDW on the latter is NP-hard. Our main hardness result is that for DAGs with max degree $Δ\leq3$, min-sum Time Disjoint Walks is APX-hard. We present a natural approximation algorithm for the same class, and provide a tight analysis. In particular, we prove that it achieves an approximation ratio of $Θ(k/\log k)$ on bounded-degree DAGs, and $Θ(k)$ on DAGs and bounded-degree digraphs.

cs.DS↗

arXiv2019

Simulation to Scaled City: Zero-Shot Policy Transfer for Traffic Control via Autonomous Vehicles

Using deep reinforcement learning, we train control policies for autonomous vehicles leading a platoon of vehicles onto a roundabout. Using Flow, a library for deep reinforcement learning in micro-simulators, we train two policies, one policy with noise injected into the state and action space and one without any injected noise. In simulation, the autonomous vehicle learns an emergent metering behavior for both policies in which it slows to allow for smoother merging. We then directly transfer this policy without any tuning to the University of Delaware Scaled Smart City (UDSSC), a 1:25 scale testbed for connected and automated vehicles. We characterize the performance of both policies on the scaled city. We show that the noise-free policy winds up crashing and only occasionally metering. However, the noise-injected policy consistently performs the metering behavior and remains collision-free, suggesting that the noise helps with the zero-shot policy transfer. Additionally, the transferred, noise-injected policy leads to a 5% reduction of average travel time and a reduction of 22% in maximum travel time in the UDSSC. Videos of the controllers can be found at https://sites.google.com/view/iccps-policy-transfer.

eess.SY↗

arXiv2018

A unified software framework for solving traffic assignment problems

We describe a software framework for solving user equilibrium traffic assignment problems. The design is based on the formulation of the problem as a variational inequality. The software implements these as well as several numerical methods for find equilirbria. We compare the solutions obtained under several models: static, Merchant-Nemhauser, `CTM with instantaneous travel time', and `CTM with actual travel time'. Some important differences are demonstrated.

cs.DC↗

arXiv2017

MOBILITY21: Strategic Investments for Transportation Infrastructure & Technology

America's transportation infrastructure is the backbone of our economy. A strong infrastructure means a strong America - an America that competes globally, supports local and regional economic development, and creates jobs. Strategic investments in our transportation infrastructure are vital to our national security, economic growth, transportation safety and our technology leadership. This document outlines critical needs for our transportation infrastructure, identifies new technology drivers and proposes strategic investments for safe and efficient air, ground, rail and marine mobility of people and goods.

cs.CY↗

arXiv2016

Minimizing Regret on Reflexive Banach Spaces and Learning Nash Equilibria in Continuous Zero-Sum Games

We study a general version of the adversarial online learning problem. We are given a decision set $\mathcal{X}$ in a reflexive Banach space $X$ and a sequence of reward vectors in the dual space of $X$. At each iteration, we choose an action from $\mathcal{X}$, based on the observed sequence of previous rewards. Our goal is to minimize regret, defined as the gap between the realized reward and the reward of the best fixed action in hindsight. Using results from infinite dimensional convex analysis, we generalize the method of Dual Averaging (or Follow the Regularized Leader) to our setting and obtain general upper bounds on the worst-case regret that subsume a wide range of results from the literature. Under the assumption of uniformly continuous rewards, we obtain explicit anytime regret bounds in a setting where the decision set is the set of probability distributions on a compact metric space $S$ whose Radon-Nikodym derivatives are elements of $L^p(S)$ for some $p > 1$. Importantly, we make no convexity assumptions on either the set $S$ or the reward functions. We also prove a general lower bound on the worst-case regret for any online algorithm. We then apply these results to the problem of learning in repeated continuous two-player zero-sum games, in which players' strategy sets are compact metric spaces. In doing so, we first prove that if both players play a Hannan-consistent strategy, then with probability 1 the empirical distributions of play weakly converge to the set of Nash equilibria of the game. We then show that, under mild assumptions, Dual Averaging on the (infinite-dimensional) space of probability distributions indeed achieves Hannan-consistency. Finally, we illustrate our results through numerical examples.

cs.LG↗

arXiv2014

Anatomy of a Crash

Transportation networks constitute a critical infrastructure enabling the transfers of passengers and goods, with a significant impact on the economy at different scales. Transportation modes, whether air, road or rail, are coupled and interdependent. The frequent occurrence of perturbations on one or several modes disrupts passengers' entire journeys, directly and through ripple effects. The present paper provides a case report of the Asiana Crash in San Francisco International Airport on July 6th 2013 and its repercussions on the multimodal transportation network. It studies the resulting propagation of disturbances on the transportation infrastructure in the United States. The perturbation takes different forms and varies in scale and time frame : cancellations and delays snowball in the airspace, highway traffic near the airport is impacted by congestion in previously never congested locations, and transit passenger demand exhibit unusual traffic peaks in between airports in the Bay Area. This paper, through a case study, aims at stressing the importance of further data-driven research on interdependent infrastructure networks for increased resilience. The end goal is to form the basis for optimization models behind providing more reliable passenger door-to-door journeys.

eess.SY↗

arXiv2014

Building-in-Briefcase (BiB)

A building's environment has profound influence on occupant comfort and health. Continuous monitoring of building occupancy and environment is essential to fault detection, intelligent control, and building commissioning. Though many solutions for environmental measuring based on wireless sensor networks exist, they are not easily accessible to households and building owners who may lack time or technical expertise needed to set up a system and get quick and detailed overview of environmental conditions. Building-in-Briefcase (BiB) is a portable sensor network platform that is trivially easy to deploy in any building environment. Once the sensors are distributed, the environmental data is collected and communicated to the BiB router via TCP/IP protocol and WiFi technology which then forwards the data to the central database securely over the internet through a 3G radio. The user, with minimal effort, can access the aggregated data and visualize the trends in real time on the BiB web portal. Paramount to the adoption and continued operation of an indoor sensing platform is battery lifetime. This design has achieved a multi-year lifespan by careful selection of components, an efficient binary communications protocol and data compression. Our BiB sensor is capable of collecting a rich set of environmental parameters, and is expandable to measure others, such as CO2. This paper describes the power characteristics of BiB sensors and their occupancy estimation and activity recognition functionality. Our vision is large-scale deployment of BiB in thousands of buildings, which would provide ample research opportunities and opportunities to identify ways to improve the building environment and energy efficiency.

cs.CY↗