arXiv subjects

Ehsan Hashemi

Publications and source records attributed to Ehsan Hashemi.

At least 19 recordsLinked to original sources

arXiv2026

Learning High-Level Decision Making with an Interaction-Aware Attention-Based Network in Autonomous Driving

Reliable learning-based high-level decision making for lane changes and speed control in automated driving must accommodate dynamically sized inputs due to varying scene traffic flow. DeepSet and its variants represent the state of the art among shared-encoder approaches; however, they neglect explicit traffic interaction modeling, limiting performance in negotiation-intensive scenarios such as intersections. Attention-based methods capture interactions among static and dynamic agents, but incur quadratic memory and computational complexity and provide limited control over representation granularity. Inspired by Perceiver IO, an attention-based architecture, DecisionPerceiver, is proposed to project dynamic agent features into a fixed-size latent space, where feature granularity is regulated by the number of latent queries, improving scalability for larger networks. A finer discretization of the action set is further proposed to increase the performance gain due to interaction awareness. Extensive evaluations across three driving scenarios that require different levels of interaction awareness demonstrate consistent performance gains and generalization across various navigation objectives. In addition, the proposed architecture is assessed in scenarios with an increasing number of vehicles to demonstrate scalability.

cs.RO↗

arXiv2025

An Adaptive Transition Framework for Game-Theoretic Based Takeover

The transition of control from autonomous systems to human drivers is critical in automated driving systems, particularly due to the out-of-the-loop (OOTL) circumstances that reduce driver readiness and increase reaction times. Existing takeover strategies are based on fixed time-based transitions, which fail to account for real-time driver performance variations. This paper proposes an adaptive transition strategy that dynamically adjusts the control authority based on both the time and tracking ability of the driver trajectory. Shared control is modeled as a cooperative differential game, where control authority is modulated through time-varying objective functions instead of blending control torques directly. To ensure a more natural takeover, a driver-specific state-tracking matrix is introduced, allowing the transition to align with individual control preferences. Multiple transition strategies are evaluated using a cumulative trajectory error metric. Human-in-the-loop control scenarios of the standardized ISO lane change maneuvers demonstrate that adaptive transitions reduce trajectory deviations and driver control effort compared to conventional strategies. Experiments also confirm that continuously adjusting control authority based on real-time deviations enhances vehicle stability while reducing driver effort during takeover.

cs.RO↗

arXiv2025

Lidar-based Tracking of Traffic Participants with Sensor Nodes in Existing Urban Infrastructure

This paper presents a lidar-only state estimation and tracking framework, along with a roadside sensing unit for integration with existing urban infrastructure. Urban deployments demand scalable, real-time tracking solutions, yet traditional remote sensing remains costly and computationally intensive, especially under perceptually degraded conditions. Our sensor node couples a single lidar with an edge computing unit and runs a computationally efficient, GPU-free observer that simultaneously estimates object state, class, dimensions, and existence probability. The pipeline performs: (i) state updates via an extended Kalman filter, (ii) dimension estimation using a 1D grid-map/Bayesian update, (iii) class updates via a lookup table driven by the most probable footprint, and (iv) existence estimation from track age and bounding-box consistency. Experiments in dynamic urban-like scenes with diverse traffic participants demonstrate real-time performance and high precision: The complete end-to-end pipeline finishes within \SI{100}{\milli\second} for \SI{99.88}{\%} of messages, with an excellent detection rate. Robustness is further confirmed under simulated wind and sensor vibration. These results indicate that reliable, real-time roadside tracking is feasible on CPU-only edge hardware, enabling scalable, privacy-friendly deployments within existing city infrastructure. The framework integrates with existing poles, traffic lights, and buildings, reducing deployment costs and simplifying large-scale urban rollouts and maintenance efforts.

cs.RO↗

arXiv2025

CoInfra: A Large-Scale Cooperative Infrastructure Perception System and Dataset for Vehicle-Infrastructure Cooperation in Adverse Weather

Vehicle-infrastructure (V2I) cooperative perception can substantially extend the range, coverage, and robustness of autonomous driving systems beyond the limits of onboard-only sensing, particularly in occluded and adverse-weather environments. However, its practical value is still difficult to quantify because existing benchmarks do not adequately capture large-scale multi-node deployments, realistic communication conditions, and adverse-weather operation. This paper presents CoInfra, a deployable cooperative infrastructure perception platform comprising 14 roadside sensor nodes connected through a commercial 5G network, together with a large-scale dataset and an open-source system stack for V2I cooperation research. The system supports synchronized multi-node sensing and delay-aware fusion under real 5G communication constraints. The released dataset covers an eight-node urban roundabout under four weather conditions (sunny, rainy, heavy snow, and freezing rain) and contains 294k LiDAR frames, 589k camera images, and 332k globally consistent 3D bounding boxes. It also includes a synchronized V2I subset collected with an autonomous vehicle. Beyond standard perception benchmarks, we further evaluate whether infrastructure sensing improves awareness of safety-critical traffic participants during roundabout interactions. In structured conflict scenarios, V2I cooperation increases critical-frame completeness from 33%-46% with vehicle-only sensing to 86%-100%. These results show that multi-node infrastructure perception can significantly improve situational awareness in conflict-rich traffic scenarios where vehicle-only sensing is most limited.

cs.RO↗

arXiv2025

Real-World Deployment of Cloud-based Autonomous Mobility Systems for Outdoor and Indoor Environments

Autonomous mobility systems increasingly operate in dense and dynamic environments where perception occlusions, limited sensing coverage, and multi-agent interactions pose major challenges. While onboard sensors provide essential local perception, they often struggle to maintain reliable situational awareness in crowded urban or indoor settings. This article presents the Cloud-based Autonomous Mobility (CAM) framework, a generalized architecture that integrates infrastructure-based intelligent sensing with cloud-level coordination to enhance autonomous operations. The system deploys distributed Intelligent Sensor Nodes (ISNs) equipped with cameras, LiDAR, and edge computing to perform multi-modal perception and transmit structured information to a cloud platform via high-speed wireless communication. The cloud aggregates observations from multiple nodes to generate a global scene representation for other autonomous modules, such as decision making, motion planning, etc. Real-world deployments in an urban roundabout and a hospital-like indoor environment demonstrate improved perception robustness, safety, and coordination for future intelligent mobility systems.

cs.RO↗

arXiv2025

SAP-CoPE: Social-Aware Planning using Cooperative Pose Estimation with Infrastructure Sensor Nodes

Autonomous driving systems must operate smoothly in human-populated indoor environments, where challenges arise including limited perception and occlusions when relying only on onboard sensors, as well as the need for socially compliant motion planning that accounts for human psychological comfort zones. These factors complicate accurate recognition of human intentions and the generation of comfortable, socially aware trajectories. To address these challenges, we propose SAP-CoPE, an indoor navigation system that integrates cooperative infrastructure with a novel 3D human pose estimation method and a socially-aware model predictive control (MPC)-based motion planner. In the perception module, an optimization problem is formulated to account for uncertainty propagation in the camera projection matrix while enforcing human joint coherence. The proposed method is adaptable to both single- and multi-camera configurations and can incorporate sparse LiDAR point-cloud data. For motion planning, we integrate a psychology inspired personal-space field using the information from estimated human poses into an MPC framework to enhance socially comfort in human-populated environments. Extensive real-world evaluations demonstrate the effectiveness of the proposed approach in generating socially aware trajectories for autonomous systems.

cs.RO↗

arXiv2025

Interaction-Aware Model Predictive Decision-Making for Socially-Compliant Autonomous Driving in Mixed Urban Traffic Scenarios

Autonomous vehicles must negotiate with pedestrians in ways that are both safe and socially compliant. We present an interaction-aware model predictive decision-making (IAMPDM) framework that integrates a gap-acceptance-inspired intention model with MPC to jointly reason about human intent and vehicle control in real time. The pedestrian module produces a continuous crossing-propensity signal - driven by time-to-collision (TTC) with an intention discounting mechanism - that modulates MPC safety terms and minimum-distance constraints. We implement IAMPDM in a projection-based, motion-tracked simulator and compare it against a rule-based intention-aware controller (RBDM) and a conservative non-interactive baseline (NIA). In a human-in-the-decision-loop study with 25 participants, intention-aware methods shortened negotiation and completion time relative to NIA across scenarios, at the expense of tighter TTC/DST margins, with no significant difference between IAMPDM and RBDM except for TTC in one scenario. Results indicate that intention-aware decision-making algorithms reduce pedestrian crossing time and improve subjective ratings of comfort, safety, and trust relative to a non-cooperative decision-making algorithm. We discuss implications for real-world deployment of interaction-aware autonomous vehicles. We detail decision-making calibration and real-time implementation (CasADi/IPOPT) and propose deployment guardrails - minimum surrogate-safety margins, deadlock prevention - to balance efficiency with safety.

eess.SY↗

arXiv2024

Enhancing Indoor Mobility with Connected Sensor Nodes: A Real-Time, Delay-Aware Cooperative Perception Approach

This paper presents a novel real-time, delay-aware cooperative perception system designed for intelligent mobility platforms operating in dynamic indoor environments. The system contains a network of multi-modal sensor nodes and a central node that collectively provide perception services to mobility platforms. The proposed Hierarchical Clustering Considering the Scanning Pattern and Ground Contacting Feature based Lidar Camera Fusion improve intra-node perception for crowded environment. The system also features delay-aware global perception to synchronize and aggregate data across nodes. To validate our approach, we introduced the Indoor Pedestrian Tracking dataset, compiled from data captured by two indoor sensor nodes. Our experiments, compared to baselines, demonstrate significant improvements in detection accuracy and robustness against delays. The dataset is available in the repository: https://github.com/NingMingHao/MVSLab-IndoorCooperativePerception

cs.CV↗

arXiv2024

An Efficient Approach to Generate Safe Drivable Space by LiDAR-Camera-HDmap Fusion

In this paper, we propose an accurate and robust perception module for Autonomous Vehicles (AVs) for drivable space extraction. Perception is crucial in autonomous driving, where many deep learning-based methods, while accurate on benchmark datasets, fail to generalize effectively, especially in diverse and unpredictable environments. Our work introduces a robust easy-to-generalize perception module that leverages LiDAR, camera, and HD map data fusion to deliver a safe and reliable drivable space in all weather conditions. We present an adaptive ground removal and curb detection method integrated with HD map data for enhanced obstacle detection reliability. Additionally, we propose an adaptive DBSCAN clustering algorithm optimized for precipitation noise, and a cost-effective LiDAR-camera frustum association that is resilient to calibration discrepancies. Our comprehensive drivable space representation incorporates all perception data, ensuring compatibility with vehicle dimensions and road regulations. This approach not only improves generalization and efficiency, but also significantly enhances safety in autonomous vehicle operations. Our approach is tested on a real dataset and its reliability is verified during the daily (including harsh snowy weather) operation of our autonomous shuttle, WATonoBus

cs.RO↗

arXiv2024

Intelligent Mobility System with Integrated Motion Planning and Control Utilizing Infrastructure Sensor Nodes

This paper introduces a framework for an indoor autonomous mobility system that can perform patient transfers and materials handling. Unlike traditional systems that rely on onboard perception sensors, the proposed approach leverages a global perception and localization (PL) through Infrastructure Sensor Nodes (ISNs) and cloud computing technology. Using the global PL, an integrated Model Predictive Control (MPC)-based local planning and tracking controller augmented with Artificial Potential Field (APF) is developed, enabling reliable and efficient motion planning and obstacle avoidance ability while tracking predefined reference motions. Simulation results demonstrate the effectiveness of the proposed MPC controller in smoothly navigating around both static and dynamic obstacles. The proposed system has the potential to extend to intelligent connected autonomous vehicles, such as electric or cargo transport vehicles with four-wheel independent drive/steering (4WID-4WIS) configurations.

cs.RO↗

arXiv2024

WATonoBus: Field-Tested All-Weather Autonomous Shuttle Technology

All-weather autonomous vehicle operation poses significant challenges, encompassing modules from perception and decision-making to path planning and control. The complexity arises from the need to address adverse weather conditions such as rain, snow, and fog across the autonomy stack. Conventional model-based single-module approaches often lack holistic integration with upstream or downstream tasks. We tackle this problem by proposing a multi-module and modular system architecture with considerations for adverse weather across the perception level, through features such as snow covered curb detection, to decision-making and safety monitoring. Through daily weekday service on the WATonoBus platform for almost two years, we demonstrate that our proposed approach is capable of addressing adverse weather conditions and provide valuable insights from edge cases observed during operation.

cs.RO↗

arXiv2022

A Holistic Robust Motion Controller Framework for Autonomous Platooning

Safety is the foremost concern for autonomous platooning. The vehicle-to-vehicle (V2V) communication delay and the sudden appearance of obstacles will trigger the safety of the intended functionality (SOTIF) issues for autonomous platooning. This research proposes a holistic robust motion controller framework (MCF) for an intelligent and connected vehicle platoon system. The MCF utilizes a hierarchical structure to resolve the longitudinal string stability and the lateral control problem under the complex driving environment and time-varying communication delay. Firstly, the H-infinity feedback controller is developed to ensure the robustness of the platoon under time-varying communication delay in the upper-level coordination layer (UCL). The output from UCL will be delivered to the lower-level motion-planning layer (LML) as reference signals. Secondly, the model predictive control (MPC) algorithm is implemented in the LML to achieve multi-objective control, which comprehensively considers the reference signals, the artificial potential field, and multiple vehicle dynamics constraints. Furthermore, three critical scenarios are co-simulated for case studies, including platooning under time-varying communication delay, merging, and obstacle avoidance scenarios. The simulation results indicate that, compared with single-structure MPC, the proposed MCF can offer a better suppression on position error propagation, and get improvements on maximum position error in the three scenarios by $19.2\%$, $59.8\%$, and $15.3\%$, respectively. Last, the practicability and effectiveness of the proposed MCF are verified via hardware-in-the-loop experiment. The average conducting time of the proposed method on Speedgoat real-time target machine is 1.1 milliseconds, which meets the real-time requirements.

eess.SY↗

arXiv2021

Navigating A Mobile Robot Using Switching Distributed Sensor Networks

This paper proposes a method to navigate a mobile robot by estimating its state over a number of distributed sensor networks (DSNs) such that it can successively accomplish a sequence of tasks, i.e., its state enters each targeted set and stays inside no less than the desired time, under a resource-aware, time-efficient, and computation- and communication-constrained setting.We propose a new robot state estimation and navigation architecture, which integrates an event-triggered task-switching feedback controller for the robot and a two-time-scale distributed state estimator for each sensor. The architecture has three major advantages over existing approaches: First, in each task only one DSN is active for sensing and estimating the robot state, and for different tasks the robot can switch the active DSN by taking resource saving and system performance into account; Second, the robot only needs to communicate with one active sensor at each time to obtain its state information from the active DSN; Third, no online optimization is required. With the controller, the robot is able to accomplish a task by following a reference trajectory and switch to the next task when an event-triggered condition is fulfilled. With the estimator, each active sensor is able to estimate the robot state. Under proper conditions, we prove that the state estimation error and the trajectory tracking deviation are upper bounded by two time-varying sequences respectively, which play an essential role in the event-triggered condition. Furthermore, we find a sufficient condition for accomplishing a task and provide an upper bound of running time for the task. Numerical simulations of an indoor robot's localization and navigation are provided to validate the proposed architecture.

cs.RO↗

arXiv2021

Soft Constrained Autonomous Vehicle Navigation using Gaussian Processes and Instance Segmentation

This paper presents a generic feature-based navigation framework for autonomous vehicles using a soft constrained Particle Filter. Selected map features, such as road and landmark locations, and vehicle states are used for designing soft constraints. After obtaining features of mapped landmarks in instance-based segmented images acquired from a monocular camera, vehicle-to-landmark distances are predicted using Gaussian Process Regression (GPR) models in a mixture of experts approach. Both mean and variance outputs of GPR models are used for implementing adaptive constraints. Experimental results confirm that the use of image segmentation features improves the vehicle-to-landmark distance prediction notably, and that the proposed soft constrained approach reliably localizes the vehicle even with reduced number of landmarks and noisy observations.

cs.RO↗

arXiv2020

Distributed control under compromised measurements:Resilient estimation, attack detection, and vehicle platooning

We study how to design a secure observer-based distributed controller such that a group of vehicles can achieve accurate state estimates and formation control even if the measurements of a subset of vehicle sensors are compromised by a malicious attacker. We propose an architecture consisting of a resilient observer, an attack detector, and an observer-based distributed controller. The distributed detector is able to update three sets of vehicle sensors: the ones surely under attack, surely attack-free, and suspected to be under attack. The adaptive observer saturates the measurement innovation through a preset static or time-varying threshold, such that the potentially compromised measurements have limited influence on the estimation. Essential properties of the proposed architecture include: 1) The detector is fault-free, and the attacked and attack-free vehicle sensors can be identified in finite time; 2) The observer guarantees both real-time error bounds and asymptotic error bounds, with tighter bounds when more attacked or attack-free vehicle sensors are identified by the detector; 3) The distributed controller ensures closed-loop stability. The effectiveness of the proposed methods is evaluated through simulations by an application to vehicle platooning.

eess.SY↗

arXiv2020

Secure Platooning of Autonomous Vehicles Under Attacked GPS Data

In this paper, we study how to secure the platooning of autonomous vehicles when an unknown vehicle is under attack and bounded system uncertainties exist. For the attacked vehicle, its position and speed measurements from GPS can be manipulated arbitrarily by a malicious attacker. First, to find out which vehicle is under attack, two detectors are proposed by using the relative measurements (by camera or radar) and the local innovation obtained through measurements from neighboring vehicles. Then, based on the results of the detectors, we design a local state observer for each vehicle by applying a saturation method to the measurement innovation. Moreover, based on the neighbor state estimates provided by the observer, a distributed controller is proposed to achieve the consensus in vehicle speed and keep fixed desired distance between two neighboring vehicles. The estimation error by the observer and the platooning error by the controller are shown to be asymptotically upper bounded under certain conditions. The effectiveness of the proposed methods is also evaluated in numerical simulations.

eess.SY↗

arXiv2019

Event-triggered Add-on Safety for Connected and Automated Vehicles Using Road-side Network Infrastructure

This paper proposes an event-triggered add-on safety mechanism to adjust the control parameters for timely braking in a networked vehicular system while maintaining maneuverability. Passenger vehicle maneuverability is significantly affected by the combined-slip friction effect, in which larger longitudinal tire slips result in considerable drop in lateral tire forces. This is of higher importance when unexpected dangerous situations occur on the road and immediate actions, such as braking, need to be taken to avoid collision. Harsh braking can lead to high-slip and loss of maneuverability, hence, timely braking is essential to reduce high-slip scenarios. In addition to the vehicles own active safety systems, the proposed event-triggered add-on safety is activated upon being informed about dangers by the road-side infrastructure. The aim is to incorporate the add-on safety feature to adjust the automatic control parameters for smooth and timely braking such that a collision is avoided while vehicle's maneuverability is maintained. We study two different wireless technologies for communication between the infrastructure and the vehicles, the Long-Term Evolution (LTE) and the fifth generation (5G) schemes. The framework is validated through high-fidelity software simulations and the advantages of including the add-on feature to augment the safety margins for each communication technology is evaluated.

eess.SY↗

arXiv2018

Resilient Estimation and Control on $k$-Nearest Neighbor Platoons: A Network-Theoretic Approach

This paper is concerned with the network-theoretic properties of so-called $k$-nearest neighbor intelligent vehicular platoons, where each vehicle communicates with $k$ vehicles, both in front and behind. The network-theoretic properties analyzed in this paper play major roles in quantifying the resilience and robustness of three generic distributed estimation and control algorithms against communication failures and disturbances, namely resilient distributed estimation, resilient distributed consensus, and robust network formation. Based on the results for the connectivity measures of the $k$-nearest neighbor platoon, we show that extending the traditional platooning topologies (which were based on interacting with nearest neighbors) to $k$-nearest neighbor platoons increases the resilience of distributed estimation and control algorithms to both communication failures and disturbances. Finally, we discuss how the performance of each algorithm scales with the size of the vehicle platoon.

math.OC↗