arXiv subjects

Sean Qian

Publications and source records attributed to Sean Qian.

At least 19 recordsLinked to original sources

arXiv2026

Ultra-fast Traffic Nowcasting and Control via Differentiable Agent-based Simulation

Traffic digital twins, which inform policymakers of effective interventions based on large-scale, high-fidelity computational models calibrated to real-world traffic, hold promise for addressing societal challenges in our rapidly urbanizing world. However, conventional fine-grained traffic simulations are non-differentiable and typically rely on inefficient gradient-free optimization, making calibration for real-world applications computationally infeasible. Here we present a differentiable agent-based traffic simulator that enables ultra-fast model calibration, traffic nowcasting, and control on large-scale networks. We develop several differentiable computing techniques for simulating individual vehicle movements, including stochastic decision-making and inter-agent interactions, while ensuring that entire simulation trajectories remain end-to-end differentiable for efficient gradient-based optimization. On the large-scale Chicago road network, with over 10,000 calibration parameters, our model simulates more than one million vehicles at 173 times real-time speed. This ultra-fast simulation, together with efficient gradient-based optimization, enables us to complete model calibration using the previous 30 minutes of traffic data in 455 s, provide a one-hour-ahead traffic nowcast in 21 s, and solve the resulting traffic control problem in 728 s. This yields a full calibration--nowcast--control loop in under 20 minutes, leaving about 40 minutes of lead time for implementing interventions. Our work thus provides a practical computational basis for realizing traffic digital twins.

cs.MA↗

arXiv2026

Joint Estimation of Dynamic O-D Demand and Choice Models for Dynamic Multi-modal Networks: Computational Graph-Based Learning and Hypothesis Tests

Understanding travel demand and behavior, particularly route and mode choices, is critical for effective transportation planning and policy design in multi-modal systems with emerging mobility options. Multi-modal system-level data, such as traffic counts, probe speeds, and transit ridership, offer scalable, cost-effective, and privacy-preserving advantages for inferring and analyzing travel behavior. This research uses such system-level data to infer travel demand and choices that vary by time of day, origin/destination location, and mode. Existing studies focus on a single transportation mode, consider limited behavioral factors in disutility functions, rely on static travel time functions, and face computational challenges when applied to large-scale networks. This research addresses these gaps by proposing a joint estimation framework for dynamic origin-destination demand and disutility functions within a multi-modal transportation system that includes both private driving and public transit, using multi-source system-level data. A multi-modal dynamic traffic assignment model that accounts for both route and mode choices is integrated into the framework, with detailed travel time modeling for multiple modes. Alternative-specific and zone-specific factors are incorporated into generic disutility functions to capture heterogeneous traveler perceptions. The estimation problem is formulated and solved using a computational graph-based approach, enabling dynamic network modeling and scalable inference across large-scale networks and diverse data sets. Furthermore, we propose a hypothesis testing framework tailored to this complex estimation setting to assess the statistical significance of behavioral parameters, thereby enabling model selection and statistically rigorous insights for real-world applications.

eess.SY↗

arXiv2026

On Path-based Marginal Cost of Heterogeneous Traffic Flow for General Networks

Path marginal cost (PMC) is a crucial component in solving path-based system-optimal dynamic traffic assignment (SO-DTA), dynamic origin-destination demand estimation (DODE), and network resilience analysis. However, accurately evaluating PMC in heterogeneous traffic conditions poses significant challenges. Previous studies often focus on homogeneous traffic flow of single vehicle class and do not well address the interactive effect of heterogeneous traffic flows and the resultant computational issues. This study proposes a novel but simple method for approximately evaluating PMC in complex heterogeneous traffic condition. The method decomposes PMC into intra-class and inter-class terms and uses conversion factor derived from heterogeneous link dynamics to explicitly model the intricate relationships between vehicle classes. Additionally, the method considers the non-differentiable issue that arises when mixed traffic flow approaches system optimum conditions. The proposed method is tested on a small corridor network with synthetic demand and a large-scale network with calibrated demand from real-world data. Results demonstrated that our method exhibits superior performance in solving bi-class SO-DTA problems, yielding lower total travel cost and capturing the multi-class flow competition at the system optimum state.

cs.CE↗

arXiv2025

Scalable Dynamic Origin-Destination Demand Estimation Enhanced by High-Resolution Satellite Imagery Data

This study presents a novel integrated framework for dynamic origin-destination demand estimation (DODE) in multi-class mesoscopic network models, incorporating high-resolution satellite imagery together with conventional traffic data from local sensors. Unlike sparse local detectors, satellite imagery offers consistent, city-wide road and traffic information of both parking and moving vehicles, overcoming data availability limitations. To extract information from imagery data, we design a computer vision pipeline for class-specific vehicle detection and map matching, generating link-level traffic density observations by vehicle class. Building upon this information, we formulate a computational graph-based DODE framework that calibrates dynamic network states by jointly matching observed traffic counts/speeds from local sensors with density measurements derived from satellite imagery. To assess the accuracy and robustness of the proposed framework, we conduct a series of numerical experiments using both synthetic and real-world data. The results demonstrate that supplementing traditional data with satellite-derived density significantly improves estimation performance, especially for links without local sensors. Real-world experiments also show the framework's potential for practical deployment on large-scale networks. Sensitivity analysis further evaluates the impact of data quality related to satellite imagery data.

cs.CV↗

arXiv2025

Modeling the Curbside Congestion Effects of Ride-hailing Services for Morning Commute using Bi-modal Two-Tandem Bottlenecks

With the proliferation of ride-hailing services, curb space in urban areas has become highly congested due to the massive passenger pick-ups and drop-offs. Particularly during peak hours, the massive ride-hailing vehicles waiting to drop off obstruct curb spaces and even disrupt the flow of mainline traffic. However, there is a lack of an analytical model that formulates and mitigates the congestion effects of ride-hailing drop-offs in curb spaces. To address this issue, this paper proposes a novel bi-modal two-tandem bottleneck model to depict the commuting behaviors of private vehicles (PVs) and ride-hailing vehicles (RVs) during the morning peak in a linear city. In the model, the upstream bottleneck models the congestion on highways, and the downstream curbside bottlenecks depict the congestion caused by RV drop-offs in curb spaces, PV queue on main roads, and the spillover effects between them in the urban area. The proposed model can be solved in a closed form under eight different scenarios. A time-varying optimal congestion pricing scheme, combined curbside pricing and parking pricing, is proposed to achieve the social optimum. It is found that potential waste of road capacity could occur when there is a mismatch between the highway and curbside bottlenecks, and hence the optimal pricing should be determined in a coordinated manner. A real-world case from Hong Kong shows that the limited curb space and main road in the urban area could be the major congestion bottleneck. Expanding the capacity of the curb space or the main road in the urban area, rather than the highway bottleneck, can effectively reduce social costs. This paper highlights the critical role of curbside management and provides policy implications for the coordinated management of highways and curb spaces.

math.OC↗

arXiv2024

Know Unreported Roadway Incidents in Real-time: Early Traffic Anomaly Detection

This research aims to know traffic anomalies as early as possible. A traffic anomaly refers to a generic incident on the road that influences traffic flow and calls for urgent traffic management measures. `Knowing'' the occurrence of a traffic anomaly is twofold: the ability to detect this anomaly before it is reported anywhere, or it may be such that an anomaly can be predicted before it actually occurs on the road (e.g., non-recurrent traffic breakdown). In either way, the objective is to inform traffic operators of unreported incidents in real time and as early as possible. The key is to stay ahead of the curve. Time is of the essence. Conventional automatic incident detection (AID) methods often struggle with early detection due to their limited consideration of spatial effects and early-stage characteristics. Therefore, we propose a deep learning framework utilizing prior domain knowledge and model-designing strategies. This allows the model to detect a broader range of anomalies, not only incidents that significantly influence traffic flow but also early characteristics of incidents along with historically unreported anomalies. We specially design the model to target the early-stage detection/prediction of an incident. Additionally, unlike most conventional AID studies, our method is highly scalable and generalizable, as it is fully automated with no manual selection of historical reports required, relies solely on widely available low-cost data, and requires no additional detectors. The experimental results across numerous road segments on different maps demonstrate that our model leads to more effective and early anomaly detection.

cs.LG↗

arXiv2024

Interpretable mixture of experts for time series prediction under recurrent and non-recurrent conditions

Non-recurrent conditions caused by incidents are different from recurrent conditions that follow periodic patterns. Existing traffic speed prediction studies are incident-agnostic and use one single model to learn all possible patterns from these drastically diverse conditions. This study proposes a novel Mixture of Experts (MoE) model to improve traffic speed prediction under two separate conditions, recurrent and non-recurrent (i.e., with and without incidents). The MoE leverages separate recurrent and non-recurrent expert models (Temporal Fusion Transformers) to capture the distinct patterns of each traffic condition. Additionally, we propose a training pipeline for non-recurrent models to remedy the limited data issues. To train our model, multi-source datasets, including traffic speed, incident reports, and weather data, are integrated and processed to be informative features. Evaluations on a real road network demonstrate that the MoE achieves lower errors compared to other benchmark algorithms. The model predictions are interpreted in terms of temporal dependencies and variable importance in each condition separately to shed light on the differences between recurrent and non-recurrent conditions.

cs.LG↗

arXiv2024

Real-time system optimal traffic routing under uncertainties -- Can physics models boost reinforcement learning?

System optimal traffic routing can mitigate congestion by assigning routes for a portion of vehicles so that the total travel time of all vehicles in the transportation system can be reduced. However, achieving real-time optimal routing poses challenges due to uncertain demands and unknown system dynamics, particularly in expansive transportation networks. While physics model-based methods are sensitive to uncertainties and model mismatches, model-free reinforcement learning struggles with learning inefficiencies and interpretability issues. Our paper presents TransRL, a novel algorithm that integrates reinforcement learning with physics models for enhanced performance, reliability, and interpretability. TransRL begins by establishing a deterministic policy grounded in physics models, from which it learns from and is guided by a differentiable and stochastic teacher policy. During training, TransRL aims to maximize cumulative rewards while minimizing the Kullback Leibler (KL) divergence between the current policy and the teacher policy. This approach enables TransRL to simultaneously leverage interactions with the environment and insights from physics models. We conduct experiments on three transportation networks with up to hundreds of links. The results demonstrate TransRL's superiority over traffic model-based methods for being adaptive and learning from the actual network data. By leveraging the information from physics models, TransRL consistently outperforms state-of-the-art reinforcement learning algorithms such as proximal policy optimization (PPO) and soft actor critic (SAC). Moreover, TransRL's actions exhibit higher reliability and interpretability compared to baseline reinforcement learning approaches like PPO and SAC.

cs.LG↗

arXiv2024

Traffic estimation in unobserved network locations using data-driven macroscopic models

This paper leverages macroscopic models and multi-source spatiotemporal data collected from automatic traffic counters and probe vehicles to accurately estimate traffic flow and travel time in links where these measurements are unavailable. This problem is critical in transportation planning applications where the sensor coverage is low and the planned interventions have network-wide impacts. The proposed model, named the Macroscopic Traffic Estimator (MaTE), can perform network-wide estimations of traffic flow and travel time only using the set of observed measurements of these quantities. Because MaTE is grounded in macroscopic flow theory, all parameters and variables are interpretable. The estimated traffic flow satisfies fundamental flow conservation constraints and exhibits an increasing monotonic relationship with the estimated travel time. Using logit-based stochastic traffic assignment as the principle for routing flow behavior makes the model fully differentiable with respect to the model parameters. This property facilitates the application of computational graphs to learn parameters from vast amounts of spatiotemporal data. We also integrate neural networks and polynomial kernel functions to capture link flow interactions and enrich the mapping of traffic flows into travel times. MaTE also adds a destination choice model and a trip generation model that uses historical data on the number of trips generated by location. Experiments on synthetic data show that the model can accurately estimate travel time and traffic flow in out-of-sample links. Results obtained using real-world multi-source data from a large-scale transportation network suggest that MaTE outperforms data-driven benchmarks, especially in travel time estimation. The estimated parameters of MaTE are also informative about the hourly change in travel demand and supply characteristics of the transportation network.

cs.LG↗

arXiv2022

Estimating and Mitigating the Congestion Effect of Curbside Pick-ups and Drop-offs: A Causal Inference Approach

Curb space is one of the busiest areas in urban road networks. Especially in recent years, the rapid increase of ride-hailing trips and commercial deliveries has induced massive pick-ups/drop-offs (PUDOs), which occupy the limited curb space that was designed and built decades ago. These PUDOs could jam curbside utilization and disturb the mainline traffic flow, evidently leading to significant negative societal externalities. However, there is a lack of an analytical framework that rigorously quantifies and mitigates the congestion effect of PUDOs in the system view, particularly with little data support and involvement of confounding effects. To bridge this research gap, this paper develops a rigorous causal inference approach to estimate the congestion effect of PUDOs on general regional networks. A causal graph is set to represent the spatio-temporal relationship between PUDOs and traffic speed, and a double and separated machine learning (DSML) method is proposed to quantify how PUDOs affect traffic congestion. Additionally, a re-routing formulation is developed and solved to encourage passenger walking and traffic flow re-routing to achieve system optimization. Numerical experiments are conducted using real-world data in the Manhattan area. On average, 100 additional units of PUDOs in a region could reduce the traffic speed by 3.70 and 4.54 mph on weekdays and weekends, respectively. Re-routing trips with PUDOs on curb space could respectively reduce the system-wide total travel time by 2.44% and 2.12% in Midtown and Central Park on weekdays. Sensitivity analysis is also conducted to demonstrate the effectiveness and robustness of the proposed framework.

cs.LG↗

arXiv2022

Statistical inference of travelers' route choice preferences with system-level data

Traditional network models encapsulate travel behavior among all origin-destination pairs based on a simplified and generic utility function. Typically, the utility function consists of travel time solely and its coefficients are equated to estimates obtained from stated preference data. While this modeling strategy is reasonable, the inherent sampling bias in individual-level data may be further amplified over network flow aggregation, leading to inaccurate flow estimates. This data must be collected from surveys or travel diaries, which may be labor intensive, costly and limited to a small time period. To address these limitations, this study extends classical bi-level formulations to estimate travelers' utility functions with multiple attributes using system-level data. We formulate a methodology grounded on non-linear least squares to statistically infer travelers' utility function in the network context using traffic counts, traffic speeds, traffic incidents and sociodemographic information, among other attributes. The analysis of the mathematical properties of the optimization problem and of its pseudo-convexity motivate the use of normalized gradient descent. We also develop a hypothesis test framework to examine statistical properties of the utility function coefficients and to perform attributes selection. Experiments on synthetic data show that the coefficients are consistently recovered and that hypothesis tests are a reliable statistic to identify which attributes are determinants of travelers' route choices. Besides, a series of Monte-Carlo experiments suggest that statistical inference is robust to noise in the Origin-Destination matrix and in the traffic counts, and to various levels of sensor coverage. The methodology is also deployed at a large scale using real-world multi-source data in Fresno, CA collected before and during the COVID-19 outbreak.

stat.AP↗

arXiv2022

Estimating probabilistic dynamic origin-destination demands using multi-day traffic data on computational graphs

System-level decision making in transportation needs to understand day-to-day variation of network flows, which calls for accurate modeling and estimation of probabilistic dynamic travel demand on networks. Most existing studies estimate deterministic dynamic origin-destination (OD) demand, while the day-to-day variation of demand and flow is overlooked. Estimating probabilistic distributions of dynamic OD demand is challenging due to the complexity of the spatio-temporal networks and the computational intensity of the high-dimensional problems. With the availability of massive traffic data and the emergence of advanced computational methods, this paper develops a data-driven framework that solves the probabilistic dynamic origin-destination demand estimation (PDODE) problem using multi-day data. Different statistical distances (e.g., lp-norm, Wasserstein distance, KL divergence, Bhattacharyya distance) are used and compared to measure the gap between the estimated and the observed traffic conditions, and it is found that 2-Wasserstein distance achieves a balanced accuracy in estimating both mean and standard deviation. The proposed framework is cast into the computational graph and a reparametrization trick is developed to estimate the mean and standard deviation of the probabilistic dynamic OD demand simultaneously. We demonstrate the effectiveness and efficiency of the proposed PDODE framework on both small and real-world networks. In particular, it is demonstrated that the proposed PDODE framework can mitigate the overfitting issues by considering the demand variation. Overall, the developed PDODE framework provides a practical tool for public agencies to understand the sources of demand stochasticity, evaluate day-to-day variation of network flow, and make reliable decisions for intelligent transportation systems.

eess.SY↗

arXiv2021

From Twitter to Traffic Predictor: Next-Day Morning Traffic Prediction Using Social Media Data

The effectiveness of traditional traffic prediction methods is often extremely limited when forecasting traffic dynamics in early morning. The reason is that traffic can break down drastically during the early morning commute, and the time and duration of this break-down vary substantially from day to day. Early morning traffic forecast is crucial to inform morning-commute traffic management, but they are generally challenging to predict in advance, particularly by midnight. In this paper, we propose to mine Twitter messages as a probing method to understand the impacts of people's work and rest patterns in the evening/midnight of the previous day to the next-day morning traffic. The model is tested on freeway networks in Pittsburgh as experiments. The resulting relationship is surprisingly simple and powerful. We find that, in general, the earlier people rest as indicated from Tweets, the more congested roads will be in the next morning. The occurrence of big events in the evening before, represented by higher or lower tweet sentiment than normal, often implies lower travel demand in the next morning than normal days. Besides, people's tweeting activities in the night before and early morning are statistically associated with congestion in morning peak hours. We make use of such relationships to build a predictive framework which forecasts morning commute congestion using people's tweeting profiles extracted by 5 am or as late as the midnight prior to the morning. The Pittsburgh study supports that our framework can precisely predict morning congestion, particularly for some road segments upstream of roadway bottlenecks with large day-to-day congestion variation. Our approach considerably outperforms those existing methods without Twitter message features, and it can learn meaningful representation of demand from tweeting profiles that offer managerial insights.

cs.SI↗

arXiv2020

Learning to Recommend Signal Plans under Incidents with Real-Time Traffic Prediction

The main question to address in this paper is to recommend optimal signal timing plans in real time under incidents by incorporating domain knowledge developed with the traffic signal timing plans tuned for possible incidents, and learning from historical data of both traffic and implemented signals timing. The effectiveness of traffic incident management is often limited by the late response time and excessive workload of traffic operators. This paper proposes a novel decision-making framework that learns from both data and domain knowledge to real-time recommend contingency signal plans that accommodate non-recurrent traffic, with the outputs from real-time traffic prediction at least 30 minutes in advance. Specifically, considering the rare occurrences of engagement of contingency signal plans for incidents, we propose to decompose the end-to-end recommendation task into two hierarchical models: real-time traffic prediction and plan association. We learn the connections between the two models through metric learning, which reinforces partial-order preferences observed from historical signal engagement records. We demonstrate the effectiveness of our approach by testing this framework on the traffic network in Cranberry Township in 2019. Results show that our recommendation system has a precision score of 96.75% and recall of 87.5% on the testing plan, and make recommendation of an average of 22.5 minutes lead time ahead of Waze alerts. The results suggest that our framework is capable of giving traffic operators a significant time window to access the conditions and respond appropriately.

eess.SP↗

arXiv2019

Understanding and predicting travel time with spatio-temporal features of network traffic flow, weather and incidents

Travel time on a route varies substantially by time of day and from day to day. It is critical to understand to what extent this variation is correlated with various factors, such as weather, incidents, events or travel demand level in the context of dynamic networks. This helps a better decision making for infrastructure planning and real-time traffic operation. We propose a data-driven approach to understand and predict highway travel time using spatio-temporal features of those factors, all of which are acquired from multiple data sources. The prediction model holistically selects the most related features from a high-dimensional feature space by correlation analysis, principle component analysis and LASSO. We test and compare the performance of several regression models in predicting travel time 30 min in advance via two case studies: (1) a 6-mile highway corridor of I-270N in D.C. region, and (2) a 2.3-mile corridor of I-376E in Pittsburgh region. We found that some bottlenecks scattered in the network can imply congestion on those corridors at least 30 minutes in advance, including those on the alternative route to the corridors of study. In addition, real-time travel time is statistically related to incidents on some specific locations, morning/afternoon travel demand, visibility, precipitation, wind speed/gust and the weather type. All those spatio-temporal information together help improve prediction accuracy, comparing to using only speed data. In both case studies, random forest shows the most promise, reaching a root-mean-squared error of 16.6\% and 17.0\% respectively in afternoon peak hours for the entire year of 2014.

stat.AP↗

arXiv2019

High-Resolution Traffic Sensing with Autonomous Vehicles

The last decades have witnessed the breakthrough of autonomous vehicles (AVs), and the perception capabilities of AVs have been dramatically improved. Various sensors installed on AVs, including, but are not limited to, LiDAR, radar, camera and stereovision, will be collecting massive data and perceiving the surrounding traffic states continuously. In fact, a fleet of AVs can serve as floating (or probe) sensors, which can be utilized to infer traffic information while cruising around the roadway networks. In contrast, conventional traffic sensing methods rely on fixed traffic sensors such as loop detectors, cameras and microwave vehicle detectors. Due to the high cost of conventional traffic sensors, traffic state data are usually obtained in a low-frequency and sparse manner. In view of this, this paper leverages rich data collected through AVs to propose the high-resolution traffic sensing framework. The proposed framework estimates the fundamental traffic state variables, namely, flow, density and speed in high spatio-temporal resolution, and it is developed under different levels of AV perception capabilities and low AV market penetration rate. The Next Generation Simulation (NGSIM) data is adopted to examine the accuracy and robustness of the proposed framework. Experimental results show that the proposed estimation framework achieves high accuracy even with low AV market penetration rate. Sensitivity analysis regarding AV penetration rate, sensor configuration, and perception accuracy will also be studied. This study will help policymakers and private sectors (e.g Uber, Waymo) to understand the values of AVs, especially the values of massive data collected by AVs, in traffic operation and management.

eess.SP↗

arXiv2019

Measuring and reducing the disequilibrium levels of dynamic networks through ride-sourcing vehicle data

Transportation systems are being reshaped by ride-sourcing and shared mobility services in recent years. The transportation network companies (TNCs) have been collecting high-granular ride-sourcing vehicle (RV) trajectory data over the past decade, while it is still unclear how the RV data can improve current dynamic network modeling for network traffic management. This paper proposes to statistically estimate network disequilibrium level (NDL), namely to what extent the dynamic user equilibrium (DUE) conditions are deviated in real-world networks. Using the data based on RV trajectories, we present a novel method to estimate the real-world NDL measure. More importantly, we present a method to compute zone-to-zone travel time data from trajectory-level RV data. This would become a data-sharing scheme for TNCs such that, while being used to effectively estimate and reduce NDL, the zone-to-zone data reveals neither personally identifiable information nor trip-level business information if shared with the public. In addition, we present an NDL based traffic management method to perform user optimal routing on a small fraction of vehicles in the network. The NDL measures and NDL-based routing are examined on two real-world large-scale networks: the City of Chengdu with trajectory-level RV data and the City of Pittsburgh with zone-to-zone travel time data. We found that, on weekdays in each city, NDLs are likely high when travel demand is high (thus when congestion is mild or heavy).

physics.soc-ph↗

arXiv2019

A deep learning approach to real-time parking occupancy prediction in spatio-temporal networks incorporating multiple spatio-temporal data sources

A deep learning model is applied for predicting block-level parking occupancy in real time. The model leverages Graph-Convolutional Neural Networks (GCNN) to extract the spatial relations of traffic flow in large-scale networks, and utilizes Recurrent Neural Networks (RNN) with Long-Short Term Memory (LSTM) to capture the temporal features. In addition, the model is capable of taking multiple heterogeneously structured traffic data sources as input, such as parking meter transactions, traffic speed, and weather conditions. The model performance is evaluated through a case study in Pittsburgh downtown area. The proposed model outperforms other baseline methods including multi-layer LSTM and Lasso with an average testing MAPE of 10.6\% when predicting block-level parking occupancies 30 minutes in advance. The case study also shows that, in generally, the prediction model works better for business areas than for recreational locations. We found that incorporating traffic speed and weather information can significantly improve the prediction performance. Weather data is particularly useful for improving predicting accuracy in recreational areas.

cs.LG↗