Parallel nonlinear neuromorphic computing with temporal encoding

Abstract

The proliferation of deep learning applications has intensified the demand for electronic hardware with low energy consumption and fast computing speed. Neuromorphic photonics have emerged as a viable alternative to directly process high-throughput information at the physical space. However, the simultaneous attainment of high linear and nonlinear expressivity posse a considerable challenge due to the power efficiency and impaired manipulability in conventional nonlinear materials and optoelectronic conversion. Here we introduce a parallel nonlinear neuromorphic processor that enables arbitrary superposition of information states in multi-dimensional channels, only by leveraging the temporal encoding of spatiotemporal metasurfaces to map the input data and trainable weights. The proposed temporal encoding nonlinearity is theoretically proved to flexibly customize the nonlinearity, while preserving quasi-static linear transformation capability within each time partition. We experimentally demonstrated the concept based on distributed spatiotemporal metasurfaces, showcasing robust performance in multi-label recognition and multi-task parallelism with asynchronous modulation. Remarkably, our nonlinear processor demonstrates dynamic memory capability in autonomous planning tasks and real-time responsiveness to canonical maze-solving problem. Our work opens up a flexible avenue for a variety of temporally-modulated neuromorphic processors tailored for complex scenarios.