arXiv subjects

Daniel Brunner

Publications and source records attributed to Daniel Brunner.

54 records · Page 3Linked to original sources

arXiv2021

Boolean learning under noise-perturbations in hardware neural networks

A high efficiency hardware integration of neural networks benefits from realizing nonlinearity, network connectivity and learning fully in a physical substrate. Multiple systems have recently implemented some or all of these operations, yet the focus was placed on addressing technological challenges. Fundamental questions regarding learning in hardware neural networks remain largely unexplored. Noise in particular is unavoidable in such architectures, and here we investigate its interaction with a learning algorithm using an opto-electronic recurrent neural network. We find that noise strongly modifies the system's path during convergence, and surprisingly fully decorrelates the final readout weight matrices. This highlights the importance of understanding architecture, noise and learning algorithm as interacting players, and therefore identifies the need for mathematical tools for noisy, analogue system optimization.

cs.NE↗

arXiv2021

Prospects and applications of photonic neural networks

Neural networks have enabled applications in artificial intelligence through machine learning, and neuromorphic computing. Software implementations of neural networks on conventional computers that have separate memory and processor (and that operate sequentially) are limited in speed and energy efficiency. Neuromorphic engineering aims to build processors in which hardware mimics neurons and synapses in the brain for distributed and parallel processing. Neuromorphic engineering enabled by photonics (optical physics) can offer sub-nanosecond latencies and high bandwidth with low energies to extend the domain of artificial intelligence and neuromorphic computing applications to machine learning acceleration, nonlinear programming, intelligent signal processing, etc. Photonic neural networks have been demonstrated on integrated platforms and free-space optics depending on the class of applications being targeted. Here, we discuss the prospects and demonstrated applications of these photonic neural networks.

cs.ET↗

arXiv2020

A complete, parallel and autonomous photonic neural network in a semiconductor multimode laser

Neural networks are one of the disruptive computing concepts of our time. However, they fundamentally differ from classical, algorithmic computing in a number of fundamental aspects. These differences result in equally fundamental, severe and relevant challenges for neural network computing using current computing substrates. Neural networks urge for parallelism across the entire processor and for a co-location of memory and arithmetic, i.e. beyond von Neumann architectures. Parallelism in particular made photonics a highly promising platform, yet until now scalable and integratable concepts are scarce. Here, we demonstrate for the first time how a fully parallel and fully implemented photonic neural network can be realized using spatially distributed modes of an efficient and fast semiconductor laser. Importantly, all neural network connections are realized in hardware, and our processor produces results without pre- or post-processing. 130+ nodes are implemented in a large-area vertical cavity surface emitting laser, input and output weights are realized via the complex transmission matrix of a multimode fiber and a digital micro-mirror array, respectively. We train the readout weights to perform 2-bit header recognition, a 2-bit XOR and 2-bit digital analog conversion, and obtain < 0.9 10^-3 and 2.9 10^-2 error rates for digit recognition and XOR, respectively. Finally, the digital analog conversion can be realized with a standard deviation of only 5.4 10^-2. Our system is scalable to much larger sizes and to bandwidths in excess of 20 GHz.

cs.NE↗

arXiv2020

Optical Neural Networks: The 3D connection

We motivate a new canonical strategy for integrating photonic neural networks (NNs) by leveraging 3D printing. Our believe is that a NN's parallel and dense connectivity is not scalable without 3D integration. 3D additive fabrication complemented with photonic signal transduction can dramatically augment the current capabilities of 2D CMOS and integrated photonics. Here we review some of our recent advances made towards such a breakthrough architecture.

cs.ET↗

arXiv2020

Bayesian optimisation of large-scale photonic reservoir computers

Introduction. Reservoir computing is a growing paradigm for simplified training of recurrent neural networks, with a high potential for hardware implementations. Numerous experiments in optics and electronics yield comparable performance to digital state-of-the-art algorithms. Many of the most recent works in the field focus on large-scale photonic systems, with tens of thousands of physical nodes and arbitrary interconnections. While this trend significantly expands the potential applications of photonic reservoir computing, it also complicates the optimisation of the high number of hyper-parameters of the system. Methods. In this work, we propose the use of Bayesian optimisation for efficient exploration of the hyper-parameter space in a minimum number of iteration. Results. We test this approach on a previously reported large-scale experimental system, compare it to the commonly used grid search, and report notable improvements in performance and the number of experimental iterations required to optimise the hyper-parameters. Conclusion. Bayesian optimisation thus has the potential to become the standard method for tuning the hyper-parameters in photonic reservoir computing.

cs.NE↗

arXiv2020

Human action recognition with a large-scale brain-inspired photonic computer

The recognition of human actions in video streams is a challenging task in computer vision, with cardinal applications in e.g. brain-computer interface and surveillance. Deep learning has shown remarkable results recently, but can be found hard to use in practice, as its training requires large datasets and special purpose, energy-consuming hardware. In this work, we propose a scalable photonic neuro-inspired architecture based on the reservoir computing paradigm, capable of recognising video-based human actions with state-of-the-art accuracy. Our experimental optical setup comprises off-the-shelf components, and implements a large parallel recurrent neural network that is easy to train and can be scaled up to hundreds of thousands of nodes. This work paves the way towards simply reconfigurable and energy-efficient photonic information processing systems for real-time video processing.

cs.NE↗

arXiv2020

Noise-enhanced spatial-photonic Ising machine

Ising machines are novel computing devices for the energy minimization of Ising models. These combinatorial optimization problems are of paramount importance for science and technology, but remain difficult to tackle on large scale by conventional electronics. Recently, various photonics-based Ising machines demonstrated ultra-fast computing of Ising ground state by data processing through multiple temporal or spatial optical channels. Experimental noise acts as a detrimental effect in many of these devices. On the contrary, we here demonstrate that an optimal noise level enhances the performance of spatial-photonic Ising machines on frustrated spin problems. By controlling the error rate at the detection, we introduce a noisy-feedback mechanism in an Ising machine based on spatial light modulation. We investigate the device performance on systems with hundreds of individually-addressable spins with all-to-all couplings and we found an increased success probability at a specific noise level. The optimal noise amplitude depends on graph properties and size, thus indicating an additional tunable parameter helpful in exploring complex energy landscapes and in avoiding trapping into local minima. The result points out noise as a resource for optical computing. This concept, which also holds in different nanophotonic neural networks, may be crucial in developing novel hardware with optics-enabled parallel architecture for large-scale optimizations.

physics.optics↗

arXiv2020

Three dimensional waveguide-interconnects for scalable integration of photonic neural networks

Photonic waveguides are prime candidates for integrated and parallel photonic interconnects. Such interconnects correspond to large-scale vector matrix products, which are at the heart of neural network computation. However, parallel interconnect circuits realized in two dimensions, for example by lithography, are strongly limited in size due to disadvantageous scaling. We use three dimensional (3D) printed photonic waveguides to overcome this limitation. 3D optical-couplers with fractal topology efficiently connect large numbers of input and output channels, and we show that the substrate's footprint area scales linearly. Going beyond simple couplers, we introduce functional circuits for discrete spatial filters identical to those used in deep convolutional neural networks.

cs.ET↗

arXiv2019

Advancing Fourier: space-time concepts in ultrafast optics, imaging and photonic neural networks

The concepts of Fourier optics were established in France in the 1940s by Pierre-Michel Duffieux, and laid the foundations of an extensive series of activities in the French research community that have touched on nearly every aspect of contemporary optics and photonics. In this paper, we review a selection of results where applications of the Fourier transform and transfer functions in optics have been applied to yield significant advances in unexpected areas of optics, including the spatial shaping of complex laser beams in amplitude and in phase, real-time ultrafast measurements, novel ghost imaging techniques, and the development of parallel processing methodologies for photonic artificial intelligence.

physics.optics↗

arXiv2019

Reservoir-size dependent learning in analogue neural networks

The implementation of artificial neural networks in hardware substrates is a major interdisciplinary enterprise. Well suited candidates for physical implementations must combine nonlinear neurons with dedicated and efficient hardware solutions for both connectivity and training. Reservoir computing addresses the problems related with the network connectivity and training in an elegant and efficient way. However, important questions regarding impact of reservoir size and learning routines on the convergence-speed during learning remain unaddressed. Here, we study in detail the learning process of a recently demonstrated photonic neural network based on a reservoir. We use a greedy algorithm to train our neural network for the task of chaotic signals prediction and analyze the learning-error landscape. Our results unveil fundamental properties of the system's optimization hyperspace. Particularly, we determine the convergence speed of learning as a function of reservoir size and find exceptional, close to linear scaling. This linear dependence, together with our parallel diffractive coupling, represent optimal scaling conditions for our photonic neural network scheme.

cs.NE↗

arXiv2019

Fundamental aspects of noise in analog-hardware neural networks

We study and analyze the fundamental aspects of noise propagation in recurrent as well as deep, multi-layer networks. The main focus of our study are neural networks in analogue hardware, yet the methodology provides insight for networks in general. The system under study consists of noisy linear nodes, and we investigate the signal-to-noise ratio at the network's outputs which is the upper limit to such a system's computing accuracy. We consider additive and multiplicative noise which can be purely local as well as correlated across populations of neurons. This covers the chief internal-perturbations of hardware networks and noise amplitudes were obtained from a physically implemented recurrent neural network and therefore correspond to a real-world system. Analytic solutions agree exceptionally well with numerical data, enabling clear identification of the most critical components and aspects for noise management. Focusing on linear nodes isolates the impact of network connections and allows us to derive strategies for mitigating noise. Our work is the starting point in addressing this aspect of analogue neural networks, and our results identify notoriously sensitive points while simultaneously highlighting the robustness of such computational systems.

cs.ET↗

arXiv2019

Diffractive coupling for photonic networks: how big can we go?

Photonic networks are considered a promising substrate for high-performance future computing systems. Compared to electronics, photonics has significant advantages for a fully parallel implementation of networks. A promising approach for parallel large-scale photonic networks is realizing the connections using diffraction. Here, we characterize the scalability of such diffractive coupling in great detail. Based on experiments, analytically obtained bounds and numerical simulations considering real-world optical imaging setups, we find that the concept in principle enables networks hosting over a million optical emitters. This would be a breakthrough in multiple areas, illustrating a clear path toward future, large scale photonic networks.

physics.optics↗

arXiv2019

Coupled nonlinear delay systems as deep convolutional neural networks

Neural networks are currently transforming the field of computer algorithms, yet their emulation on current computing substrates is highly inefficient. Reservoir computing was successfully implemented on a large variety of substrates and gave new insight in overcoming this implementation bottleneck. Despite its success, the approach lags behind the state of the art in deep learning. We therefore extend time-delay reservoirs to deep networks and demonstrate that these conceptually correspond to deep convolutional neural networks. Convolution is intrinsically realized on a substrate level by generic drive-response properties of dynamical systems. The resulting novelty is avoiding vector-matrix products between layers, which cause low efficiency in today's substrates. Compared to singleton time-delay reservoirs, our deep network achieves accuracy improvements by at least an order of magnitude in Mackey-Glass and Lorenz timeseries prediction.

cs.ET↗

arXiv2018

Efficient Design of Hardware-Enabled Reservoir Computing in FPGAs

In this work, we propose a new approach towards the efficient optimization and implementation of reservoir computing hardware reducing the required domain expert knowledge and optimization effort. First, we adapt the reservoir input mask to the structure of the data via linear autoencoders. We therefore incorporate the advantages of dimensionality reduction and dimensionality expansion achieved by conventional algorithmically efficient linear algebra procedures of principal component analysis. Second, we employ evolutionary-inspired genetic algorithm techniques resulting in a highly efficient optimization of reservoir dynamics with dramatically reduced number of evaluations comparing to exhaustive search. We illustrate the method on the so-called single-node reservoir computing architecture, especially suitable for implementation in ultrahigh-speed hardware. The combination of both methods and the resulting reduction of time required for performance optimization of a hardware system establish a strategy towards machine learning hardware capable of self-adaption to optimally solve specific problems. We confirm the validity of those principles building reservoir computing hardware based on a field-programmable gate array.

cs.ET↗

arXiv2017

Spatio-temporal complexity in dual delay nonlinear laser dynamics: chimeras and dissipative solitons

We demonstrate for a nonlinear photonic system that two highly asymmetric feedback delays can induce a variety of emergent patterns which are highly robust during the system's global evolution. Explicitly, two-dimensional chimeras and dissipative solitons become visible upon a space-time transformation. Switching between chimeras and dissipative solitons requires only adjusting two system parameters, demonstrating self-organization exclusively based on the system's dynamical properties. Experiments were performed using a tunable semiconductor laser's transmission through a Fabry-Perot resonator resulting in an Airy function as nonlinearity. Resulting dynamics were band-pass filtered and propagated along two feedback paths whose time delays differ by two orders of magnitude. An excellent agreement between experimental results and theoretical model given by modified Ikeda equations was achieved.

physics.optics↗

arXiv2017

Reinforcement Learning in a large scale photonic Recurrent Neural Network

Photonic Neural Network implementations have been gaining considerable attention as a potentially disruptive future technology. Demonstrating learning in large scale neural networks is essential to establish photonic machine learning substrates as viable information processing systems. Realizing photonic Neural Networks with numerous nonlinear nodes in a fully parallel and efficient learning hardware was lacking so far. We demonstrate a network of up to 2500 diffractively coupled photonic nodes, forming a large scale Recurrent Neural Network. Using a Digital Micro Mirror Device, we realize reinforcement learning. Our scheme is fully parallel, and the passive weights maximize energy efficiency and bandwidth. The computational output efficiently converges and we achieve very good performance.

cs.NE↗

arXiv2013

High resolution coherent population trapping on a single hole spin in a semiconductor

We report high resolution coherent population trapping on a single hole spin in a semiconductor quantum dot. The absorption dip signifying the formation of a dark state exhibits an atomic physics-like dip width of just 10 MHz. We observe fluctuations in the absolute frequency of the absorption dip, evidence of very slow spin dephasing. We identify this process as charge noise by, first, demonstrating that the hole spin g-factor in this configuration (in-plane magnetic field) is strongly dependent on the vertical electric field, and second, by characterizing the charge noise through its effects on the optical transition frequency. An important conclusion is that charge noise is an important hole spin dephasing process.

cond-mat.mes-hall↗

arXiv2013

A dark-field microscope for background-free detection of resonance fluorescence from single semiconductor quantum dots operating in a set-and-forget mode

Optically active quantum dots, for instance self-assembled InGaAs quantum dots, are potentially excellent single photon sources. The fidelity of the single photons is much improved using resonant rather than non-resonant excitation. With resonant excitation, the challenge is to distinguish between resonance fluorescence and scattered laser light. We have met this challenge by creating a polarization-based dark-field microscope to measure the resonance fluorescence from a single quantum dot at low temperature. We achieve a suppression of the scattered laser exceeding a factor of 10^7 and background-free detection of resonance fluorescence. The same optical setup operates over the entire quantum dot emission range 920-980 nm and also in high magnetic fields. The major development is the outstanding long-term stability: once the dark-field point has been established, the microscope operates for days without alignment. The mechanical and optical design of the microscope is presented, as well as exemplary resonance fluorescence spectroscopy results on individual quantum dots to underline the microscope's excellent performance.

cond-mat.mes-hall↗