arXiv subjects

H. Michael Zhang

Publications and source records attributed to H. Michael Zhang.

4 recordsLinked to original sources

arXiv2025

Combat Urban Congestion via Collaboration: Heterogeneous GNN-based MARL for Coordinated Platooning and Traffic Signal Control

Over the years, reinforcement learning has emerged as a popular approach to develop signal control and vehicle platooning strategies either independently or in a hierarchical way. However, jointly controlling both in real-time to alleviate traffic congestion presents new challenges, such as the inherent physical and behavioral heterogeneity between signal control and platooning, as well as coordination between them. This paper proposes an innovative solution to tackle these challenges based on heterogeneous graph multi-agent reinforcement learning and traffic theories. Our approach involves: 1) designing platoon and signal control as distinct reinforcement learning agents with their own set of observations, actions, and reward functions to optimize traffic flow; 2) designing coordination by incorporating graph neural networks within multi-agent reinforcement learning to facilitate seamless information exchange among agents on a regional scale; 3) applying alternating optimization for training, allowing agents to update their own policies and adapt to other agents' policies. We evaluate our approach through SUMO simulations, which show convergent results in terms of both travel time and fuel consumption, and superior performance compared to other adaptive signal control methods.

cs.LG↗

arXiv2025

A Multi-scale Perimeter Control and Route Guidance System for Large-scale Road Networks

Perimeter control and route guidance are effective ways to reduce traffic congestion and improve traffic efficiency by controlling the spatial and temporal traffic distribution on the network. This paper presents a multi-scale joint perimeter control and route guidance (MSJC) framework for controlling traffic in large-scale networks. The network is first partitioned into several subnetworks (regions) with traffic in each region governed by its macroscopic fundamental diagram (MFD), which forms the macroscale network (upper level). Each subnetwork, comprised of actual road links and signalized intersections, forms the microscale network (lower level). At the upper level, a joint perimeter control and route guidance model solves the region-based inflow rate and hyper-path flows to control the accumulation of each region and thus maximize the throughput of each region. At the lower level, a perimeter control strategy integrated with a backpressure policy determines the optimal signal phases of the intersections at the regional boundary. At the same time, a route choice model for vehicles is constructed to meet hyper-path flows and ensure the intra-region homogeneity of traffic density. The case study results demonstrate that the proposed MSJC outperforms other benchmarks in regulating regional accumulation, thereby improving throughput.

math.OC↗

arXiv2025

A Hierarchical Signal Coordination and Control System Using a Hybrid Model-based and Reinforcement Learning Approach

Signal control in urban corridors faces the dual challenge of maintaining arterial traffic progression while adapting to demand variations at local intersections. We propose a hierarchical traffic signal coordination and control scheme that integrates model-based optimization with reinforcement learning. The system consists of: (i) a High-Level Coordinator (HLC) that selects coordination strategies based on observed and predicted demand; (ii) a Corridor Coordinator that derives phase constraints from the selected strategy-either Max-Flow Coordination (MFC) or Green-Wave Coordination (GWC); and (iii) Hybrid Signal Agents (HSAs) that determine signal phases via reinforcement learning with action masking to enforce feasibility. Hierarchical reinforcement learning with Proximal Policy Optimization (PPO) is used to train HSA and HLC policies. At the lower level, three HSA policies-MFC-aware, GWC-aware, and pure agent control (PAC) are trained in conjunction with their respective coordination strategies. At the higher level, the HLC is trained to dynamically switch strategies using a multi-objective reward balancing corridor-level and network-wide performance. The proposed scheme was developed and evaluated on a SUMO-RLlib platform. Case results show that hybrid MFC maximizes throughput under heavy demand; hybrid GWC consistently minimizes arterial stops and maintains progression across diverse traffic conditions but can reduce network-wide efficiency; and PAC improves network-wide travel time in moderate demand but is less effective under heavy demand. The hierarchical design enables adaptive strategy selection, achieving robust performance across all demand levels.

eess.SY↗

arXiv2025

Evidence and quantification of cooperation of driving agents in mixed traffic flow

Cooperation is a ubiquitous phenomenon in many natural, social, and engineered systems with multiple agents. Understanding the formation of cooperation in mixed traffic is of theoretical interest in its own right, and could also benefit the design and operations of future automated and mixed-autonomy transportation systems. However, how cooperativeness of driving agents can be defined and identified from empirical data seems ambiguous and this hinders further empirical characterizations of the phenomenon and revealing its behavior mechanisms. Towards mitigating this gap, in this paper, we propose a unified conceptual framework to identify collective cooperativeness of driving agents. This framework expands the concept of collective rationality from our recent model (Li et al. 2022a), making it empirically identifiable and behaviorally interpretable in realistic (microscopic and dynamic) settings. This framework integrates mixed traffic observations at both microscopic and macroscopic scales to estimate critical behavioral parameters that describe the collective cooperativeness of driving agents. Applying this framework to NGSIM I-80 trajectory data, we empirically confirm the existence of collective cooperation and quantify the condition and likelihood of its emergence. This study provides the first empirical understanding of collective cooperativeness in human-driven mixed traffic and points to new possibilities to manage mixed autonomy traffic systems.

physics.soc-ph↗