Passive None-line-of-sight imaging with arbitrary scene condition and detection pattern in small amount of prior data

Abstract

Passive Non-Line-of-Sight (NLOS) imaging requires to reconstruct objects which cannot be seen in line without using external controllable light sources. It can be widely applied in areas like counter-terrorism, urban-Warfare, autonomous-driving and robot-vision. Existing methods for passive NLOS typically required extensive prior information and significant computational resources to establish light transport matrices or train neural networks. These constraints pose significant challenges for transitioning models to different NLOS scenarios. Thus, the pressing issue in passive NLOS imaging currently lies in whether it is possible to estimate the light transport matrices which corresponding to relay surfaces and scenes, as well as the specific distribution of targets, with a small amount of prior knowledge. In this work, we hypothesized a high-dimensional manifold and mathematically proved its existence. Within this high-dimensional manifold, the structural information of obscured targets is minimally disrupted. Therefore, we proposed a universal framework named High-Dimensional Projection Selection (HDPS) which can establish this high-dimensional manifold and output its projection onto corresponding surfaces on low-dimensional. HDPS can be applied to most mature network architectures and estimate the distribution of target and light spot obtained by camera with only minimal prior data. Certainly, with the help of the estimated information, it can establish a high-dimensional manifold consisting of target and input. As demonstrated in experiment, our framework, even when applied to the most basic network structures, can achieve higher accuracy results with significantly smaller amounts of prior data. Thereby, our approach enables passive NLOS scenarios to reconstruct target by limited prior data and computational resources.