Neuromorphic cameras for Atmospheric Cherenkov Telescopes and fast optical astronomy: new paradigm, challenges and opportunities

Abstract

The astronomy community has witnessed an explosive growth in the use of deep-learning techniques based on neural networks since the mid-2010s. The widespread adoption of these nature-inspired technologies has helped astronomers tackle previously insurmountable problems and provided an unprecedented opportunity for new discoveries. However, one of the primary tools of today's optical astronomy is neither natural nor efficient: their photo-sensing devices. Specifically, the modern CCD camera - like that of the cutting-edge Rubin Observatory - requires an internal clock to regularly expose the sensor to light, consumes a large amount of energy and information bandwidth, and has a limited dynamic range. On the contrary, biological eyes lack an internal clock and a shutter, have much higher pixel density but consume significantly less energy and bandwidth, and can adapt to bright and low light conditions. Inspired by the nature of the eyes, M. Mahowald and C. Mead introduced the revolutionary concept of a silicon retina sensor in 1991. Also known as event-based cameras (EBCs), these types of devices operate in a vastly different way compared to conventional CCD-based imaging sensors. EBCs mimic the operating principles of optic nerves and continuously produce a stream of events, with each event generated only when a pixel detects a change in light intensity. EBCs do not have fixed exposure times, have high dynamic range, require low power for operation, and can capture high-speed phenomena. These properties are important requirements for Cherenkov telescopes as well as other high-speed optical astronomy. This work presents the opportunities and challenges of using EBCs in those cases, and proposes a low-cost approach to experimentally assess the feasibility of this innovative technique.