arXiv subjects

Laurent Larger

Publications and source records attributed to Laurent Larger.

At least 19 recordsLinked to original sources

arXiv2023

Deterministic and stochastic coarsening control in optically-addressed spatial light modulators subject to optical feedback

Phase separation accompanied by further domain growth and coarsening is a phenomenon common to a broad variety of dynamical systems. In this connection, controlling such processes represents a relevant interdisciplinary problem. Using methods of numerical modelling, we demonstrate two approaches for the coarsening control in bistable systems based on the example of a spatially-extended model describing an optically-addressed spatial light modulator with two color illumination subject to optical feedback. The first method implies varying system parameters such that the system evolves as the pitchfork or saddle-node normal forms. The second method leverages noise whose intensity is used as an additional system parameter. Both, deterministic and stochastic schemes allow to control the direction and speed of the fronts separating spatial domains. The considered stochastic control represents a particular case of the noise-sustained front propagation in bistable systems and involves the properties of the optical system under study. In contrast, the proposed deterministic control technique can be applied to bistable systems of different nature.

physics.optics↗

arXiv2021

Understanding and mitigating noise in trained deep neural networks

Deep neural networks unlocked a vast range of new applications by solving tasks of which many were previously deemed as reserved to higher human intelligence. One of the developments enabling this success was a boost in computing power provided by special purpose hardware, such as graphic or tensor processing units. However, these do not leverage fundamental features of neural networks like parallelism and analog state variables. Instead, they emulate neural networks relying on binary computing, which results in unsustainable energy consumption and comparatively low speed. Fully parallel and analogue hardware promises to overcome these challenges, yet the impact of analogue neuron noise and its propagation, i.e. accumulation, threatens rendering such approaches inept. Here, we determine for the first time the propagation of noise in deep neural networks comprising noisy nonlinear neurons in trained fully connected layers. We study additive and multiplicative as well as correlated and uncorrelated noise, and develop analytical methods that predict the noise level in any layer of symmetric deep neural networks or deep neural networks trained with back propagation. We find that noise accumulation is generally bound, and adding additional network layers does not worsen the signal to noise ratio beyond a limit. Most importantly, noise accumulation can be suppressed entirely when neuron activation functions have a slope smaller than unity. We therefore developed the framework for noise in fully connected deep neural networks implemented in analog systems, and identify criteria allowing engineers to design noise-resilient novel neural network hardware.

cs.NE↗

arXiv2021

Two-color optically-addressed spatial light modulator as generic spatio-temporal systems

Nonlinear spatio-temporal systems are the basis for countless physical phenomena in such diverse fields as ecology, optics, electronics and neuroscience. The canonical approach to unify models originating from different fields is the normal form description, which determines the generic dynamical aspects and different bifurcation scenarios. Realizing different types of dynamical systems via one experimental platform that enables continuous transition between normal forms through tuning accessible system parameters is therefore highly relevant. Here, we show that a transmissive, optically-addressed spatial light modulator under coherent optical illumination and optical feedback coupling allows tuning between pitchfork, transcritical and saddle-node bifurcations of steady states. We demonstrate this by analytically deriving the system's normal form equations and confirm these results via extensive numerical simulations. Our model describes a nematic liquid crystal device using nano-dimensional dichalcogenide (a-As$_2$S$_3$) glassy thin-films as photo sensors and alignment layers, and we use device parameters obtained from experimental characterization. Optical coupling, for example using diffraction, holography or integrated unitary maps allow implementing a variety of system topologies of technological relevance for neural networks and potentially XY-Hamiltonian models with ultra low energy consumption.

physics.optics↗

arXiv2021

Nonlinear photonic dynamical systems for unconventional computing

Driven by the remarkable breakthroughs during the past decade, photonics neural networks have experienced a revival. Here, we provide a general overview of progress over the past decade, and sketch a roadmap of important future developments. We focus on photonic implementations of the reservoir computing machine learning paradigm, which offers a conceptually simple approach that is amenable to hardware implementations. In particular, we provide an overview of photonic reservoir computing implemented via either spatio temporal or delay dynamical systems. Going beyond reservoir computing, we discuss recent advances and future challenges of photonic implementations of deep neural networks, of the quest for learning methods that are hardware-friendly as well as realizing autonomous photonic neural networks, i.e. with minimal digital electronic auxiliary hardware.

cs.ET↗

arXiv2021

Boolean learning under noise-perturbations in hardware neural networks

A high efficiency hardware integration of neural networks benefits from realizing nonlinearity, network connectivity and learning fully in a physical substrate. Multiple systems have recently implemented some or all of these operations, yet the focus was placed on addressing technological challenges. Fundamental questions regarding learning in hardware neural networks remain largely unexplored. Noise in particular is unavoidable in such architectures, and here we investigate its interaction with a learning algorithm using an opto-electronic recurrent neural network. We find that noise strongly modifies the system's path during convergence, and surprisingly fully decorrelates the final readout weight matrices. This highlights the importance of understanding architecture, noise and learning algorithm as interacting players, and therefore identifies the need for mathematical tools for noisy, analogue system optimization.

cs.NE↗

arXiv2020

Three dimensional waveguide-interconnects for scalable integration of photonic neural networks

Photonic waveguides are prime candidates for integrated and parallel photonic interconnects. Such interconnects correspond to large-scale vector matrix products, which are at the heart of neural network computation. However, parallel interconnect circuits realized in two dimensions, for example by lithography, are strongly limited in size due to disadvantageous scaling. We use three dimensional (3D) printed photonic waveguides to overcome this limitation. 3D optical-couplers with fractal topology efficiently connect large numbers of input and output channels, and we show that the substrate's footprint area scales linearly. Going beyond simple couplers, we introduce functional circuits for discrete spatial filters identical to those used in deep convolutional neural networks.

cs.ET↗

arXiv2019

Advancing Fourier: space-time concepts in ultrafast optics, imaging and photonic neural networks

The concepts of Fourier optics were established in France in the 1940s by Pierre-Michel Duffieux, and laid the foundations of an extensive series of activities in the French research community that have touched on nearly every aspect of contemporary optics and photonics. In this paper, we review a selection of results where applications of the Fourier transform and transfer functions in optics have been applied to yield significant advances in unexpected areas of optics, including the spatial shaping of complex laser beams in amplitude and in phase, real-time ultrafast measurements, novel ghost imaging techniques, and the development of parallel processing methodologies for photonic artificial intelligence.

physics.optics↗

arXiv2019

Reservoir-size dependent learning in analogue neural networks

The implementation of artificial neural networks in hardware substrates is a major interdisciplinary enterprise. Well suited candidates for physical implementations must combine nonlinear neurons with dedicated and efficient hardware solutions for both connectivity and training. Reservoir computing addresses the problems related with the network connectivity and training in an elegant and efficient way. However, important questions regarding impact of reservoir size and learning routines on the convergence-speed during learning remain unaddressed. Here, we study in detail the learning process of a recently demonstrated photonic neural network based on a reservoir. We use a greedy algorithm to train our neural network for the task of chaotic signals prediction and analyze the learning-error landscape. Our results unveil fundamental properties of the system's optimization hyperspace. Particularly, we determine the convergence speed of learning as a function of reservoir size and find exceptional, close to linear scaling. This linear dependence, together with our parallel diffractive coupling, represent optimal scaling conditions for our photonic neural network scheme.

cs.NE↗

arXiv2019

Fundamental aspects of noise in analog-hardware neural networks

We study and analyze the fundamental aspects of noise propagation in recurrent as well as deep, multi-layer networks. The main focus of our study are neural networks in analogue hardware, yet the methodology provides insight for networks in general. The system under study consists of noisy linear nodes, and we investigate the signal-to-noise ratio at the network's outputs which is the upper limit to such a system's computing accuracy. We consider additive and multiplicative noise which can be purely local as well as correlated across populations of neurons. This covers the chief internal-perturbations of hardware networks and noise amplitudes were obtained from a physically implemented recurrent neural network and therefore correspond to a real-world system. Analytic solutions agree exceptionally well with numerical data, enabling clear identification of the most critical components and aspects for noise management. Focusing on linear nodes isolates the impact of network connections and allows us to derive strategies for mitigating noise. Our work is the starting point in addressing this aspect of analogue neural networks, and our results identify notoriously sensitive points while simultaneously highlighting the robustness of such computational systems.

cs.ET↗

arXiv2019

Diffractive coupling for photonic networks: how big can we go?

Photonic networks are considered a promising substrate for high-performance future computing systems. Compared to electronics, photonics has significant advantages for a fully parallel implementation of networks. A promising approach for parallel large-scale photonic networks is realizing the connections using diffraction. Here, we characterize the scalability of such diffractive coupling in great detail. Based on experiments, analytically obtained bounds and numerical simulations considering real-world optical imaging setups, we find that the concept in principle enables networks hosting over a million optical emitters. This would be a breakthrough in multiple areas, illustrating a clear path toward future, large scale photonic networks.

physics.optics↗

arXiv2019

Coupled nonlinear delay systems as deep convolutional neural networks

Neural networks are currently transforming the field of computer algorithms, yet their emulation on current computing substrates is highly inefficient. Reservoir computing was successfully implemented on a large variety of substrates and gave new insight in overcoming this implementation bottleneck. Despite its success, the approach lags behind the state of the art in deep learning. We therefore extend time-delay reservoirs to deep networks and demonstrate that these conceptually correspond to deep convolutional neural networks. Convolution is intrinsically realized on a substrate level by generic drive-response properties of dynamical systems. The resulting novelty is avoiding vector-matrix products between layers, which cause low efficiency in today's substrates. Compared to singleton time-delay reservoirs, our deep network achieves accuracy improvements by at least an order of magnitude in Mackey-Glass and Lorenz timeseries prediction.

cs.ET↗

arXiv2018

Efficient Design of Hardware-Enabled Reservoir Computing in FPGAs

In this work, we propose a new approach towards the efficient optimization and implementation of reservoir computing hardware reducing the required domain expert knowledge and optimization effort. First, we adapt the reservoir input mask to the structure of the data via linear autoencoders. We therefore incorporate the advantages of dimensionality reduction and dimensionality expansion achieved by conventional algorithmically efficient linear algebra procedures of principal component analysis. Second, we employ evolutionary-inspired genetic algorithm techniques resulting in a highly efficient optimization of reservoir dynamics with dramatically reduced number of evaluations comparing to exhaustive search. We illustrate the method on the so-called single-node reservoir computing architecture, especially suitable for implementation in ultrahigh-speed hardware. The combination of both methods and the resulting reduction of time required for performance optimization of a hardware system establish a strategy towards machine learning hardware capable of self-adaption to optimally solve specific problems. We confirm the validity of those principles building reservoir computing hardware based on a field-programmable gate array.

cs.ET↗

arXiv2018

Delayed Dynamical Systems: Networks, Chimeras and Reservoir Computing

We present a systematic approach to reveal the correspondence between time delay dynamics and networks of coupled oscillators. After early demonstrations of the usefulness of spatio-temporal representations of time-delay system dynamics, extensive research on optoelectronic feedback loops has revealed their immense potential for realizing complex system dynamics such as chimeras in rings of coupled oscillators and applications to reservoir computing. Delayed dynamical systems have been enriched in recent years through the application of digital signal processing techniques. Very recently, we have showed that one can significantly extend the capabilities and implement networks with arbitrary topologies through the use of field programmable gate arrays (FPGAs). This architecture allows the design of appropriate filters and multiple time delays which greatly extend the possibilities for exploring synchronization patterns in arbitrary topological networks. This has enabled us to explore complex dynamics on networks with nodes that can be perfectly identical, introduce parameter heterogeneities and multiple time delays, as well as change network topologies to control the formation and evolution of patterns of synchrony.

nlin.AO↗

arXiv2017

Spatio-temporal complexity in dual delay nonlinear laser dynamics: chimeras and dissipative solitons

We demonstrate for a nonlinear photonic system that two highly asymmetric feedback delays can induce a variety of emergent patterns which are highly robust during the system's global evolution. Explicitly, two-dimensional chimeras and dissipative solitons become visible upon a space-time transformation. Switching between chimeras and dissipative solitons requires only adjusting two system parameters, demonstrating self-organization exclusively based on the system's dynamical properties. Experiments were performed using a tunable semiconductor laser's transmission through a Fabry-Perot resonator resulting in an Airy function as nonlinearity. Resulting dynamics were band-pass filtered and propagated along two feedback paths whose time delays differ by two orders of magnitude. An excellent agreement between experimental results and theoretical model given by modified Ikeda equations was achieved.

physics.optics↗

arXiv2017

Reinforcement Learning in a large scale photonic Recurrent Neural Network

Photonic Neural Network implementations have been gaining considerable attention as a potentially disruptive future technology. Demonstrating learning in large scale neural networks is essential to establish photonic machine learning substrates as viable information processing systems. Realizing photonic Neural Networks with numerous nonlinear nodes in a fully parallel and efficient learning hardware was lacking so far. We demonstrate a network of up to 2500 diffractively coupled photonic nodes, forming a large scale Recurrent Neural Network. Using a Digital Micro Mirror Device, we realize reinforcement learning. Our scheme is fully parallel, and the passive weights maximize energy efficiency and bandwidth. The computational output efficiently converges and we achieve very good performance.

cs.NE↗

arXiv2016

FPGA Design for Pseudorandom Number Generator Based on Chaotic Iteration used in Information Hiding Application

Lots of researches indicate that the inefficient generation of random numbers is a significant bottleneck for information communication applications. Therefore, Field Programmable Gate Array (FPGA) is developed to process a scalable fixed-point method for random streams generation. In our previous researches, we have proposed a technique by applying some well-defined discrete chaotic iterations that satisfy the reputed Devaney's definition of chaos, namely chaotic iterations (CI). We have formerly proven that the generator with CI can provide qualified chaotic random numbers. In this paper, this generator based on chaotic iterations is optimally redesigned for FPGA device. By doing so, the generation rate can be largely improved. Analyses show that these hardware generators can also provide good statistical chaotic random bits and can be cryptographically secure too. An application in the information hiding security field is finally given as an illustrative example.

cs.CR↗

arXiv2015

Nonlinear memory capacity of parallel time-delay reservoir computers in the processing of multidimensional signals

This paper addresses the reservoir design problem in the context of delay-based reservoir computers for multidimensional input signals, parallel architectures, and real-time multitasking. First, an approximating reservoir model is presented in those frameworks that provides an explicit functional link between the reservoir parameters and architecture and its performance in the execution of a specific task. Second, the inference properties of the ridge regression estimator in the multivariate context is used to assess the impact of finite sample training on the decrease of the reservoir capacity. Finally, an empirical study is conducted that shows the adequacy of the theoretical results with the empirical performances exhibited by various reservoir architectures in the execution of several nonlinear tasks with multidimensional inputs. Our results confirm the robustness properties of the parallel reservoir architecture with respect to task misspecification and parameter choice that had already been documented in the literature.

cs.NE↗

arXiv2014

Laser Chimeras as a paradigm for multi-stable patterns in complex systems

Chimera is a rich and fascinating class of self-organized solutions developed in high dimensional networks having non-local and symmetry breaking coupling features. Its accurate understanding is expected to bring important insight in many phenomena observed in complex spatio-temporal dynamics, from living systems, brain operation principles, and even turbulence in hydrodynamics. In this article we report on a powerful and highly controllable experiment based on optoelectronic delayed feedback applied to a wavelength tunable semiconductor laser, with which a wide variety of Chimera patterns can be accurately investigated and interpreted. We uncover a cascade of higher order Chimeras as a pattern transition from N to N - 1 clusters of chaoticity. Finally, we follow visually, as the gain increases, how Chimera is gradually destroyed on the way to apparent turbulence-like system behaviour.

physics.optics↗