Searcharxiv⌕ Search

arXiv subjects

Kazuyuki Aihara

Publications and source records attributed to Kazuyuki Aihara.

At least 37 records · Page 2Linked to original sources

Spatial prisoner's dilemma optimally played in small-world networks

Cooperation is commonly found in ecological and social systems even when it apparently seems that individuals can benefit from selfish behavior. We investigate how cooperation emerges with the spatial prisoner's dilemma played in a class of networks ranging from regular lattices to random networks. We find that, among these networks, small-world topology is the optimal structure when we take into account the speed at which cooperative behavior propagates. Our results may explain why the small-world properties are self-organized in real networks.

physics.soc-ph↗

Traffic signal optimization in large-scale urban road networks: an adaptive-predictive controller using Ising models

Realizing smooth traffic flow is important for achieving carbon neutrality. Adaptive traffic signal control, which considers traffic conditions, has thus attracted attention. However, it is difficult to ensure optimal vehicle flow throughout a large city using existing control methods because of their heavy computational load. Here, we propose a control method called AMPIC (Adaptive Model Predictive Ising Controller) that guarantees both scalability and optimality. The proposed method employs model predictive control to solve an optimal control problem at each control interval with explicit consideration of a predictive model of vehicle flow. This optimal control problem is transformed into a combinatorial optimization problem with binary variables that is equivalent to the so-called Ising problem. This transformation allows us to use an Ising solver, which has been widely studied and is expected to have fast and efficient optimization performance. We performed numerical experiments using a microscopic traffic simulator for a realistic city road network. The results show that AMPIC enables faster vehicle cruising speed with less waiting time than that achieved by classical control methods, resulting in lower CO2 emissions. The model predictive approach with a long prediction horizon thus effectively improves control performance. Systematic parametric studies on model cities indicate that the proposed method realizes smoother traffic flows for large city road networks. Among Ising solvers, D-Wave's quantum annealing is shown to find near-optimal solutions at a reasonable computational cost.

math.OC↗

Designing Chaotic Attractors: A Semi-supervised Approach

Chaotic dynamics are ubiquitous in nature and useful in engineering, but their geometric design can be challenging. Here, we propose a method using reservoir computing to generate chaos with a desired shape by providing a periodic orbit as a template, called a skeleton. We exploit a bifurcation of the reservoir to intentionally induce unsuccessful training of the skeleton, revealing inherent chaos. The emergence of this untrained attractor, resulting from the interaction between the skeleton and the reservoir's intrinsic dynamics, offers a novel semi-supervised framework for designing chaos.

cs.NE↗

Predicting unobserved climate time series data at distant areas via spatial correlation using reservoir computing

Collecting time series data spatially distributed in many locations is often important for analyzing climate change and its impacts on ecosystems. However, comprehensive spatial data collection is not always feasible, requiring us to predict climate variables at some locations. This study focuses on a prediction of climatic elements, specifically near-surface temperature and pressure, at a target location apart from a data observation point. Our approach uses two prediction methods: reservoir computing (RC), known as a machine learning framework with low computational requirements, and vector autoregression models (VAR), recognized as a statistical method for analyzing time series data. Our results show that the accuracy of the predictions degrades with the distance between the observation and target locations. We quantitatively estimate the distance in which effective predictions are possible. We also find that in the context of climate data, a geographical distance is associated with data correlation, and a strong data correlation significantly improves the prediction accuracy with RC. In particular, RC outperforms VAR in predicting highly correlated data within the predictive range. These findings suggest that machine learning-based methods can be used more effectively to predict climatic elements in remote locations by assessing the distance to them from the data observation point in advance. Our study on low-cost and accurate prediction of climate variables has significant value for climate change strategies.

cs.LG↗

Trained Latent Space Navigation to Prevent Lack of Photorealism in Generated Images on Style-based Models

Recent studies on StyleGAN variants show promising performances for various generation tasks. In these models, latent codes have traditionally been manipulated and searched for the desired images. However, this approach sometimes suffers from a lack of photorealism in generated images due to a lack of knowledge about the geometry of the trained latent space. In this paper, we show a simple unsupervised method that provides well-trained local latent subspace, enabling latent code navigation while preserving the photorealism of the generated images. Specifically, the method identifies densely mapped latent spaces and restricts latent manipulations within the local latent subspace. Experimental results demonstrate that images generated within the local latent subspace maintain photorealism even when the latent codes are significantly and repeatedly manipulated. Moreover, experiments show that the method can be applied to latent code optimization for various types of style-based models. Our empirical evidence of the method will benefit applications in style-based models.

cs.CV↗

Sparse-firing regularization methods for spiking neural networks with time-to-first spike coding

The training of multilayer spiking neural networks (SNNs) using the error backpropagation algorithm has made significant progress in recent years. Among the various training schemes, the error backpropagation method that directly uses the firing time of neurons has attracted considerable attention because it can realize ideal temporal coding. This method uses time-to-first spike (TTFS) coding, in which each neuron fires at most once, and this restriction on the number of firings enables information to be processed at a very low firing frequency. This low firing frequency increases the energy efficiency of information processing in SNNs, which is important not only because of its similarity with information processing in the brain, but also from an engineering point of view. However, only an upper limit has been provided for TTFS-coded SNNs, and the information-processing capability of SNNs at lower firing frequencies has not been fully investigated. In this paper, we propose two spike timing-based sparse-firing (SSR) regularization methods to further reduce the firing frequency of TTFS-coded SNNs. The first is the membrane potential-aware SSR (M-SSR) method, which has been derived as an extreme form of the loss function of the membrane potential value. The second is the firing condition-aware SSR (F-SSR) method, which is a regularization function obtained from the firing conditions. Both methods are characterized by the fact that they only require information about the firing timing and associated weights. The effects of these regularization methods were investigated on the MNIST, Fashion-MNIST, and CIFAR-10 datasets using multilayer perceptron networks and convolutional neural network structures.

cs.NE↗

Entropic Herding

Herding is a deterministic algorithm used to generate data points that can be regarded as random samples satisfying input moment conditions. The algorithm is based on the complex behavior of a high-dimensional dynamical system and is inspired by the maximum entropy principle of statistical inference. In this paper, we propose an extension of the herding algorithm, called entropic herding, which generates a sequence of distributions instead of points. Entropic herding is derived as the optimization of the target function obtained from the maximum entropy principle. Using the proposed entropic herding algorithm as a framework, we discuss a closer connection between herding and the maximum entropy principle. Specifically, we interpret the original herding algorithm as a tractable version of entropic herding, the ideal output distribution of which is mathematically represented. We further discuss how the complex behavior of the herding algorithm contributes to optimization. We argue that the proposed entropic herding algorithm extends the application of herding to probabilistic modeling. In contrast to original herding, entropic herding can generate a smooth distribution such that both efficient probability density calculation and sample generation become possible. To demonstrate the viability of these arguments in this study, numerical experiments were conducted, including a comparison with other conventional methods, on both synthetic and real data.

stat.ML↗

Learning Reservoir Dynamics with Temporal Self-Modulation

Reservoir computing (RC) can efficiently process time-series data by transferring the input signal to randomly connected recurrent neural networks (RNNs), which are referred to as a reservoir. The high-dimensional representation of time-series data in the reservoir significantly simplifies subsequent learning tasks. Although this simple architecture allows fast learning and facile physical implementation, the learning performance is inferior to that of other state-of-the-art RNN models. In this paper, to improve the learning ability of RC, we propose self-modulated RC (SM-RC), which extends RC by adding a self-modulation mechanism. The self-modulation mechanism is realized with two gating variables: an input gate and a reservoir gate. The input gate modulates the input signal, and the reservoir gate modulates the dynamical properties of the reservoir. We demonstrated that SM-RC can perform attention tasks where input information is retained or discarded depending on the input signal. We also found that a chaotic state emerged as a result of learning in SM-RC. This indicates that self-modulation mechanisms provide RC with qualitatively different information-processing capabilities. Furthermore, SM-RC outperformed RC in NARMA and Lorentz model tasks. In particular, SM-RC achieved a higher prediction accuracy than RC with a reservoir 10 times larger in the Lorentz model tasks. Because the SM-RC architecture only requires two additional gates, it is physically implementable as RC, providing a new direction for realizing edge AI.

cs.LG↗

High-Speed Resource Allocation Algorithm Using a Coherent Ising Machine for NOMA Systems

Non-orthogonal multiple access (NOMA) technique is important for achieving a high data rate in next-generation wireless communications. A key challenge to fully utilizing the effectiveness of the NOMA technique is the optimization of the resource allocation (RA), e.g., channel and power. However, this RA optimization problem is NP-hard, and obtaining a good approximation of a solution with a low computational complexity algorithm is not easy. To overcome this problem, we propose the coherent Ising machine (CIM) based optimization method for channel allocation in NOMA systems. The CIM is an Ising system that can deliver fair approximate solutions to combinatorial optimization problems at high speed (millisecond order) by operating optimization algorithms based on mutually connected photonic neural networks. The performance of our proposed method was evaluated using a simulation model of the CIM. We compared the performance of our proposed method to simulated annealing, a conventional-NOMA pairing scheme, deep Q learning based scheme, and an exhaustive search scheme. Simulation results indicate that our proposed method is superior in terms of speed and the attained optimal solutions.

cs.IT↗

Timing-Based Backpropagation in Spiking Neural Networks Without Single-Spike Restrictions

We propose a novel backpropagation algorithm for training spiking neural networks (SNNs) that encodes information in the relative multiple spike timing of individual neurons without single-spike restrictions. The proposed algorithm inherits the advantages of conventional timing-based methods in that it computes accurate gradients with respect to spike timing, which promotes ideal temporal coding. Unlike conventional methods where each neuron fires at most once, the proposed algorithm allows each neuron to fire multiple times. This extension naturally improves the computational capacity of SNNs. Our SNN model outperformed comparable SNN models and achieved as high accuracy as non-convolutional artificial neural networks. The spike count property of our networks was altered depending on the time constant of the postsynaptic current and the membrane potential. Moreover, we found that there existed the optimal time constant with the maximum test accuracy. That was not seen in conventional SNNs with single-spike restrictions on time-to-fast-spike (TTFS) coding. This result demonstrates the computational properties of SNNs that biologically encode information into the multi-spike timing of individual neurons. Our code would be publicly available.

cs.NE↗

Experimental observation of chimera states in spiking neural networks based on degenerate optical parametric oscillators

We experimentally demonstrate that networks of identical photonic spiking neurons based on coupled degenerate parametric oscillators can show various chimera states, in which, depending on their local synchronization and desynchronization, different kinds of spiking dynamics can develop in a self-organized manner. Even when only a static interaction is implemented, through synchronized inputs from connected neurons, the spiking mode of photonic neurons can be spontaneously and adaptively changed between the Class-I and Class-II modes classified by A. L. Hodgkin. This spontaneous spiking-mode shift induces a significant change in the spiking frequency despite the all neurons having the same natural spiking frequency, which encourages the generation of chimera states. Controllability and self-organized flexibility of the spiking modes in the present system allow us to create an experimental platform to explore the nature of chimera states in neuromorphic spiking neural networks.

physics.optics↗

Reservoir Computing with Diverse Timescales for Prediction of Multiscale Dynamics

Machine learning approaches have recently been leveraged as a substitute or an aid for physical/mathematical modeling approaches to dynamical systems. To develop an efficient machine learning method dedicated to modeling and prediction of multiscale dynamics, we propose a reservoir computing (RC) model with diverse timescales by using a recurrent network of heterogeneous leaky integrator (LI) neurons. We evaluate computational performance of the proposed model in two time series prediction tasks related to four chaotic fast-slow dynamical systems. In a one-step-ahead prediction task where input data are provided only from the fast subsystem, we show that the proposed model yields better performance than the standard RC model with identical LI neurons. Our analysis reveals that the timescale required for producing each component of target multiscale dynamics is appropriately and flexibly selected from the reservoir dynamics by model training. In a long-term prediction task, we demonstrate that a closed-loop version of the proposed model can achieve longer-term predictions compared to the counterpart with identical LI neurons depending on the hyperparameter setting.

cs.LG↗

Effects of VLSI Circuit Constraints on Temporal-Coding Multilayer Spiking Neural Networks

The spiking neural network (SNN) has been attracting considerable attention not only as a mathematical model for the brain, but also as an energy-efficient information processing model for real-world applications. In particular, SNNs based on temporal coding are expected to be much more efficient than those based on rate coding, because the former requires substantially fewer spikes to carry out tasks. As SNNs are continuous-state and continuous-time models, it is favorable to implement them with analog VLSI circuits. However, the construction of the entire system with continuous-time analog circuits would be infeasible when the system size is very large. Therefore, mixed-signal circuits must be employed, and the time discretization and quantization of the synaptic weights are necessary. Moreover, the analog VLSI implementation of SNNs exhibits non-idealities, such as the effects of noise and device mismatches, as well as other constraints arising from the analog circuit operation. In this study, we investigated the effects of the time discretization and/or weight quantization on the performance of SNNs. Furthermore, we elucidated the effects the lower bound of the membrane potentials and the temporal fluctuation of the firing threshold. Finally, we propose an optimal approach for the mapping of mathematical SNN models to analog circuits with discretized time.

cs.AR↗

Scaling advantage of nonrelaxational dynamics for high-performance combinatorial optimization

The development of physical simulators, called Ising machines, that sample from low energy states of the Ising Hamiltonian has the potential to drastically transform our ability to understand and control complex systems. However, most of the physical implementations of such machines have been based on a similar concept that is closely related to relaxational dynamics such as in simulated, mean-field, chaotic, and quantum annealing. We show that nonrelaxational dynamics that is associated with broken detailed balance and positive entropy production rate can accelerate the sampling of low energy states compared to that of conventional methods. By implementing such dynamics on field programmable gate array, we show that the nonrelaxational dynamics that we propose, called chaotic amplitude control, exhibits a scaling with problem size of the time to finding optimal solutions and its variance that is significantly smaller than that of relaxational schemes recently implemented on Ising machines.

physics.comp-ph↗

Analytical estimation of maximum fraction of infected individuals with one-shot non-pharmaceutical intervention in a hybrid epidemic model

Facing a global epidemic of new infectious diseases such as COVID-19, non-pharmaceutical interventions (NPIs), which reduce transmission rates without medical actions, are being implemented around the world to mitigate spreads. One of the problems in assessing the effects of NPIs is that different NPIs have been implemented at different times based on the situation of each country; therefore, few assumptions can be shared about how the introduction of policies affects the patient population. Mathematical models can contribute to further understanding these phenomena by obtaining analytical solutions as well as numerical simulations. In this study, an NPI was introduced into the SIR model for a conceptual study of infectious diseases under the condition that the transmission rate was reduced to a fixed value only once within a finite time duration, and its effect was analyzed numerically and theoretically. It was analytically shown that the maximum fraction of infected individuals and the final size could be larger if the intervention starts too early. The analytical results also suggested that more individuals may be infected at the peak of the second wave with a stronger intervention. This study provides quantitative relationship between the strength of a one-shot intervention and the reduction in the number of patients with no approximation. This suggests the importance of the strength and time of NPIs, although detailed studies are necessary for the implementation of NPIs in complicated real-world environments as the model used in this study is based on various simplifications.

q-bio.PE↗

Traffic Signal Optimization on a Square Lattice with Quantum Annealing

The spread of intelligent transportation systems in urban cities has caused heavy computational loads, requiring a novel architecture for managing large-scale traffic. In this study, we develop a method for globally controlling traffic signals arranged on a square lattice by means of a quantum annealing machine, namely the D-Wave quantum annealer. We first formulate a signal optimization problem that minimizes the imbalance of traffic flows in two orthogonal directions. Then we reformulate this problem as an Ising Hamiltonian, which is fully compatible with quantum annealers. The new control method is compared with a conventional local control method for a large 50-by-50 city, and the results exhibit the superiority of our global control method in suppressing traffic imbalance over wide parameter ranges. Furthermore, the solutions to the global control method obtained with the quantum annealing machine are better than those obtained with conventional simulated annealing. In addition, we prove analytically that the local and the global control methods converge at the limit where cars have equal probabilities for turning and going straight. These results are verified with numerical experiments.

math.OC↗

Collective and synchronous dynamics of photonic spiking neurons

Nonlinear dynamics of spiking neural networks has recently attracted much interest as an approach to understand possible information processing in the brain and apply it to artificial intelligence. Since information can be processed by collective spiking dynamics of neurons, the fine control of spiking dynamics is desirable for neuromorphic devices. Here we show that photonic spiking neurons implemented with paired nonlinear optical oscillators can be controlled to generate two modes of bio-realistic spiking dynamics by changing the optical pump amplitude. When they are coupled in a network, we found that the interaction between the photonic neurons induces an effective change in the pump amplitude depending on the order parameter that characterizes synchronization. The experimental results show that the effective change causes spontaneous modification of the spiking modes and firing rates of clustered neurons, and such collective dynamics can be utilized to realize efficient heuristics for solving NP-hard combinatorial optimization problems.

physics.optics↗

Model-Size Reduction for Reservoir Computing by Concatenating Internal States Through Time

Reservoir computing (RC) is a machine learning algorithm that can learn complex time series from data very rapidly based on the use of high-dimensional dynamical systems, such as random networks of neurons, called "reservoirs." To implement RC in edge computing, it is highly important to reduce the amount of computational resources that RC requires. In this study, we propose methods that reduce the size of the reservoir by inputting the past or drifting states of the reservoir to the output layer at the current time step. These proposed methods are analyzed based on information processing capacity, which is a performance measure of RC proposed by Dambre et al. (2012). In addition, we evaluate the effectiveness of the proposed methods on time-series prediction tasks: the generalized Henon-map and NARMA. On these tasks, we found that the proposed methods were able to reduce the size of the reservoir up to one tenth without a substantial increase in regression error. Because the applications of the proposed methods are not limited to a specific network structure of the reservoir, the proposed methods could further improve the energy efficiency of RC-based systems, such as FPGAs and photonic systems.

cs.LG↗