Untangling Dense Crowds with mmWave Radar: From Crowd Semantics to Individual Spatial Behaviors

Abstract

In this paper, we present a novel framework for recovering individual footprints and pedestrian spatial behaviors in dense crowds. This is a highly challenging problem, as pedestrians can merge or block one another often and for prolonged periods in crowded areas, leading to strongly entangled observations. Our approach jointly leverages learned crowd semantics and principled reasoning over persistent radar observability loss to systematically infer individual spatial footprints. At the macroscopic level, we learn spatial usage patterns of the crowd from radar point clouds, while at the microscopic level we develop a physics-informed model of radar observability loss caused by sustained merging and occlusion. Building on this macro-micro structure, we introduce a semantic-guided multi-hypothesis reasoning foundation that evaluates competing hypotheses via principled reasoning over many-to-one and one-to-none observation mappings, enabling untangling of individual spatial footprints. We evaluate the framework through 18 real-world experiments using an off-the-shelf mmWave radar, involving crowds of up to (and including) 10 individuals across six categories of diverse pedestrian behaviors, and four environments. Our methodology robustly recovers individual spatial footprints, even during prolonged and persistent observability loss, demonstrating strong structural consistency with the ground-truth. The recovered footprints further enable accurate zonal analysis of pedestrian space usage. Finally, we present an ablation study and further compare against state-of-the-art. The proposed foundation substantially outperforms the state-of-the-art across all experiments, achieving an average median DTW of 30.5 cm compared to 95.94 cm, highlighting the framework's robustness to frequent and prolonged merging and occlusion.