Multimodal learning enables instant ionizing radiation alerts on unmodified mobile phones for real-world emergency response

Abstract

In a radiation emergency, every second counts, yet the public rarely has immediate access to dedicated monitoring devices when they are needed most. Here, the first practical mobile phone-based emergency ionizing radiation detection method is presented that operates entirely without requiring camera coverage or additional hardware modifications. Utilizing a multimodal deep learning approach that integrates sparse radiation-induced signal distributions with the brightness patterns, the proposed framework effectively isolates subtle radiation signals from overwhelming visual interference. A hybrid 3D-2D convolutional neural network (CNN) identifies radiation-induced spots from raw mobile phone video, while a multi-layer perceptron (MLP) fuses the radiation signal and brightness maps for the dose rate estimation. The method detects hazardous dose rates (25-280 mRem/h) rapidly within six seconds (accuracy 86-96%), and low-level radiation (-0.6 mRem/h) with extended measurement durations achieves 87% accuracy. The developed method greatly enhances mobile phone radiation detection practicality and shows substantial potential as an accessible radiation emergency detection tool.