arXiv subjects

Atsushi Uchida

Publications and source records attributed to Atsushi Uchida.

33 records · Page 2Linked to original sources

arXiv2022

Parallel bandit architecture based on laser chaos for reinforcement learning

Accelerating artificial intelligence by photonics is an active field of study aiming to exploit the unique properties of photons. Reinforcement learning is an important branch of machine learning, and photonic decision-making principles have been demonstrated with respect to the multi-armed bandit problems. However, reinforcement learning could involve a massive number of states, unlike previously demonstrated bandit problems where the number of states is only one. Q-learning is a well-known approach in reinforcement learning that can deal with many states. The architecture of Q-learning, however, does not fit well photonic implementations due to its separation of update rule and the action selection. In this study, we organize a new architecture for multi-state reinforcement learning as a parallel array of bandit problems in order to benefit from photonic decision-makers, which we call parallel bandit architecture for reinforcement learning or PBRL in short. Taking a cart-pole balancing problem as an instance, we demonstrate that PBRL adapts to the environment in fewer time steps than Q-learning. Furthermore, PBRL yields faster adaptation when operated with a chaotic laser time series than the case with uniformly distributed pseudorandom numbers where the autocorrelation inherent in the laser chaos provides a positive effect. We also find that the variety of states that the system undergoes during the learning phase exhibits completely different properties between PBRL and Q-learning. The insights obtained through the present study are also beneficial for existing computing platforms, not just photonic realizations, in accelerating performances by the PBRL algorithms and correlated random sequences.

cs.ET↗

arXiv2022

Controlling chaotic itinerancy in laser dynamics for reinforcement learning

Photonic artificial intelligence has attracted considerable interest in accelerating machine learning; however, the unique optical properties have not been fully utilized for achieving higher-order functionalities. Chaotic itinerancy, with its spontaneous transient dynamics among multiple quasi-attractors, can be employed to realize brain-like functionalities. In this paper, we propose a method for controlling the chaotic itinerancy in a multi-mode semiconductor laser to solve a machine learning task, known as the multi-armed bandit problem, which is fundamental to reinforcement learning. The proposed method utilizes ultrafast chaotic itinerant motion in mode competition dynamics controlled via optical injection. We found that the exploration mechanism is completely different from a conventional searching algorithm and is highly scalable, outperforming the conventional approaches for large-scale bandit problems. This study paves the way to utilize chaotic itinerancy for effectively solving complex machine learning tasks as photonic hardware accelerators.

physics.optics↗

arXiv2022

Photonic reinforcement learning based on optoelectronic reservoir computing

Reinforcement learning has been intensively investigated and developed in artificial intelligence in the absence of training data, such as autonomous driving vehicles, robot control, internet advertising, and elastic optical networks. However, the computational cost of reinforcement learning with deep neural networks is extremely high and reducing the learning cost is a challenging issue. We propose a photonic on-line implementation of reinforcement learning using optoelectronic delay-based reservoir computing, both experimentally and numerically. In the proposed scheme, we accelerate reinforcement learning at a rate of several megahertz because there is no required learning process for the internal connection weights in reservoir computing. We perform two benchmark tasks, CartPole-v0 and MountanCar-v0 tasks, to evaluate the proposed scheme. Our results represent the first hardware implementation of reinforcement learning based on photonic reservoir computing and pave the way for fast and efficient reinforcement learning as a novel photonic accelerator.

cs.ET↗

arXiv2021

Photonic neural field on a silicon chip: large-scale, high-speed neuro-inspired computing and sensing

Photonic neural networks have significant potential for high-speed neural processing with low latency and ultralow energy consumption. However, the on-chip implementation of a large-scale neural network is still challenging owing to its low scalability. Herein, we propose the concept of a photonic neural field and implement it experimentally on a silicon chip to realize highly scalable neuro-inspired computing. In contrast to existing photonic neural networks, the photonic neural field is a spatially continuous field that nonlinearly responds to optical inputs, and its high spatial degrees of freedom allow for large-scale and high-density neural processing on a millimeter-scale chip. In this study, we use the on-chip photonic neural field as a reservoir of information and demonstrate a high-speed chaotic time-series prediction with low errors using a training approach similar to reservoir computing. We discuss that the photonic neural field is potentially capable of executing more than one peta multiply-accumulate operations per second for a single input wavelength on a footprint as small as a few square millimeters. In addition to processing, the photonic neural field can be used for rapidly sensing the temporal variation of an optical phase, facilitated by its high sensitivity to optical inputs. The merging of optical processing with optical sensing paves the way for an end-to-end data-driven optical sensing scheme.

cs.NE↗

arXiv2021

Using multidimensional speckle dynamics for high-speed, large-scale, parallel photonic computing

The recent rapid increase in demand for data processing has resulted in the need for novel machine learning concepts and hardware. Physical reservoir computing and an extreme learning machine are novel computing paradigms based on physical systems themselves, where the high dimensionality and nonlinearity play a crucial role in the information processing. Herein, we propose the use of multidimensional speckle dynamics in multimode fibers for information processing, where input information is mapped into the space, frequency, and time domains by an optical phase modulation technique. The speckle-based mapping of the input information is high-dimensional and nonlinear and can be realized at the speed of light; thus, nonlinear time-dependent information processing can successfully be achieved at fast rates when applying a reservoir-computing-like-approach. As a proof-of-concept, we experimentally demonstrate chaotic time-series prediction at input rates of 12.5 Gigasamples per second. Moreover, we show that owing to the passivity of multimode fibers, multiple tasks can be simultaneously processed within a single system, i.e., multitasking. These results offer a novel approach toward realizing parallel, high-speed, and large-scale photonic computing.

physics.optics↗

arXiv2020

Adaptive model selection in photonic reservoir computing by reinforcement learning

Photonic reservoir computing is an emergent technology toward beyond-Neumann computing. Although photonic reservoir computing provides superior performance in environments whose characteristics are coincident with the training datasets for the reservoir, the performance is significantly degraded if these characteristics deviate from the original knowledge used in the training phase. Here, we propose a scheme of adaptive model selection in photonic reservoir computing using reinforcement learning. In this scheme, a temporal waveform is generated by different dynamic source models that change over time. The system autonomously identifies the best source model for the task of time series prediction using photonic reservoir computing and reinforcement learning. We prepare two types of output weights for the source models, and the system adaptively selected the correct model using reinforcement learning, where the prediction errors are associated with rewards. We succeed in adaptive model selection when the source signal is temporally mixed, having originally been generated by two different dynamic system models, as well as when the signal is a mixture from the same model but with different parameter values. This study paves the way for autonomous behavior in photonic artificial intelligence and could lead to new applications in load forecasting and multi-objective control, where frequent environment changes are expected.

cs.ET↗

arXiv2020

Lotka-Volterra competition mechanism embedded in a decision-making method

Decision making is a fundamental capability of living organisms, and has recently been gaining increasing importance in many engineering applications. Here, we consider a simple decision-making principle to identify an optimal choice in multi-armed bandit (MAB) problems, which is fundamental in the context of reinforcement learning. We demonstrate that the identification mechanism of the method is well described by using a competitive ecosystem model, i.e., the competitive Lotka--Volterra (LV) model. Based on the "winner-take-all" mechanism in the competitive LV model, we demonstrate that non-best choices are eliminated and only the best choice survives; the failure of the non-best choices exponentially decreases while repeating the choice trials. Furthermore, we apply a mean-field approximation to the proposed decision-making method and show that the method has an excellent scalability of $O(\log N)$ with respect to the number of choices $N$. These results allow for a new perspective on optimal search capabilities in competitive systems.

physics.app-ph↗

arXiv2020

Dynamic channel selection in wireless communications via a multi-armed bandit algorithm using laser chaos time series

Dynamic channel selection is among the most important wireless communication elements in dynamically changing electromagnetic environments wherein a user can experience improved communication quality by choosing a better channel. Multi-armed bandit (MAB) algorithms are a promising approach by which the difficult tradeoff between exploration to search for better a channel and exploitation to experience enhanced communication quality is resolved. Ultrafast solution of MAB problems has been demonstrated by utilizing chaotically oscillating time series generated by semiconductor lasers. In this study, we experimentally demonstrate a MAB algorithm incorporating laser chaos time series in a wireless local area network (WLAN). Autonomous and adaptive dynamic channel selection is successfully demonstrated in an IEEE802.11a-based, four-channel WLAN. Although the laser chaos time series is arranged prior to the WLAN experiments, the results confirm the usefulness of ultrafast chaotic sequences for real wireless applications. In addition, we numerically examine the underlining adaptation mechanism of the significantly simplified MAB algorithm implemented in the present study compared with the previously reported chaos-based decision makers. This study provides a first step toward the application of ultrafast chaotic lasers for future high-performance wireless communication networks.

eess.SP↗

arXiv2019

Photonic reservoir computing based on nonlinear wave dynamics at a microscale

High-dimensional nonlinear dynamical systems including neural networks can be utilized as a computational resource for information processing. In this sense, nonlinear wave systems are good candidate for such a computational resource. Here, we propose and numerically demonstrate information processing based on nonlinear wave dynamics in microcavity lasers, i.e., optical spatiotemporal systems at a microscale. One of the remarkable features is the ability of high-dimensional and nonlinear mapping of input information into the wave states, enabling efficient and fast information processing at a microscale. We show that the computational capability for nonlinear/memory tasks is maximized at the edge of the dynamical stability. Moreover, we also show that the computational capability can be enhanced by applying a time division multiplexing technique to the wave dynamics; thus, the computational potential of the wave dynamics can sufficiently be extracted even when the number of detectors to monitor the wave states is limited. In addition, we discuss the merging of optical information processing and optical sensing, opening a novel method of model-free sensing by using a microcavity reservoir itself as a sensing element. These results open a way to on-chip photonic computing with high-dimensional dynamics and a novel model-free sensing scheme.

physics.app-ph↗

arXiv2019

Generative adversarial network based on chaotic time series

Generative adversarial network (GAN) is gaining increased importance in artificially constructing natural images and related functionalities wherein two networks called generator and discriminator are evolving through adversarial mechanisms. Using deep convolutional neural networks and related techniques, high-resolution, highly realistic scenes, human faces, among others have been generated. While GAN in general needs a large amount of genuine training data sets, it is noteworthy that vast amounts of pseudorandom numbers are required. Here we utilize chaotic time series generated experimentally by semiconductor lasers for the latent variables of GAN whereby the inherent nature of chaos can be reflected or transformed into the generated output data. We show that the similarity in proximity, which is a degree of robustness of the generated images with respects to a minute change in the input latent variables, is enhanced while the versatility as a whole is not severely degraded. Furthermore, we demonstrate that the surrogate chaos time series eliminates the signature of generated images that is originally observed corresponding to the negative autocorrelation inherent in the chaos sequence. We also discuss the impact of utilizing chaotic time series in retrieving images from the trained generator.

cs.LG↗

arXiv2018

Scalable photonic reinforcement learning by time-division multiplexing of laser chaos

Reinforcement learning involves decision making in dynamic and uncertain environments and constitutes a crucial element of artificial intelligence. In our previous work, we experimentally demonstrated that the ultrafast chaotic oscillatory dynamics of lasers can be used to solve the two-armed bandit problem efficiently, which requires decision making concerning a class of difficult trade-offs called the exploration-exploitation dilemma. However, only two selections were employed in that research; thus, the scalability of the laser-chaos-based reinforcement learning should be clarified. In this study, we demonstrated a scalable, pipelined principle of resolving the multi-armed bandit problem by introducing time-division multiplexing of chaotically oscillated ultrafast time-series. The experimental demonstrations in which bandit problems with up to 64 arms were successfully solved are presented in this report. Detailed analyses are also provided that include performance comparisons among laser chaos signals generated in different physical conditions, which coincide with the diffusivity inherent in the time series. This study paves the way for ultrafast reinforcement learning by taking advantage of the ultrahigh bandwidths of light wave and practical enabling technologies.

cs.ET↗

arXiv2017

Recommendations and illustrations for the evaluation of photonic random number generators

The never-ending quest to improve the security of digital information combined with recent improvements in hardware technology has caused the field of random number generation to undergo a fundamental shift from relying solely on pseudo-random algorithms to employing optical entropy sources. Despite these significant advances on the hardware side, commonly used statistical measures and evaluation practices remain ill-suited to understand or quantify the optical entropy that underlies physical random number generation. We review the state of the art in the evaluation of optical random number generation and recommend a new paradigm: quantifying entropy generation and understanding the physical limits of the optical sources of randomness. In order to do this, we advocate for the separation of the physical entropy source from deterministic post-processing in the evaluation of random number generators and for the explicit consideration of the impact of the measurement and digitization process on the rate of entropy production. We present the Cohen-Procaccia estimate of the entropy rate $h(ε,τ)$ as one way to do this. In order to provide an illustration of our recommendations, we apply the Cohen-Procaccia estimate as well as the entropy estimates from the new NIST draft standards for physical random number generators to evaluate and compare three common optical entropy sources: single photon time-of-arrival detection, chaotic lasers, and amplified spontaneous emission.

physics.optics↗

arXiv2017

Ultrafast photonic reinforcement learning based on laser chaos

Reinforcement learning involves decision making in dynamic and uncertain environments, and constitutes one important element of artificial intelligence (AI). In this paper, we experimentally demonstrate that the ultrafast chaotic oscillatory dynamics of lasers efficiently solve the multi-armed bandit problem (MAB), which requires decision making concerning a class of difficult trade-offs called the exploration-exploitation dilemma. To solve the MAB, a certain degree of randomness is required for exploration purposes. However, pseudo-random numbers generated using conventional electronic circuitry encounter severe limitations in terms of their data rate and the quality of randomness due to their algorithmic foundations. We generate laser chaos signals using a semiconductor laser sampled at a maximum rate of 100 GSample/s, and combine it with a simple decision-making principle called tug-of-war with a variable threshold, to ensure ultrafast, adaptive and accurate decision making at a maximum adaptation speed of 1 GHz. We found that decision-making performance was maximized with an optimal sampling interval, and we highlight the exact coincidence between the negative autocorrelation inherent in laser chaos and decision-making performance. This study paves the way for a new realm of ultrafast photonics in the age of AI, where the ultrahigh bandwidth of photons can provide new value.

physics.optics↗

arXiv2007

Nonresonant entrainment of detuned oscillators induced by common external noise

We have found that a novel type of entrainment occurs in two nonidentical limit cycle oscillators subjected to a common external white Gaussian noise. This entrainment is anomalous in the sense that the two oscillators have different mean frequencies, where the difference is constant as the noise intensity increases, but their phases come to be locked for almost all the time. We present a theory and numerical evidence for this phenomenon.

nlin.CD↗

arXiv2007

Consistency in the driven butterfly

We study the consistency of the responses in the chaotic Lorenz model driven by chaotic and colored-noise signals. A measure is introduced for the consistency of the response in the system for repeated applications of the same drive signal, starting at different initial conditions of the response system. The convergence of responses is analyzed in dynamical phase space in terms of the local conditional Lyapunov exponent. The dependence of the local conditional Lyapunov exponent on drive signal, and the existence of multiple basins of consistency depending on strength of drive signal, are shown.

nlin.CD↗