arXiv subjects

Masato Okada

Publications and source records attributed to Masato Okada.

At least 55 records · Page 3Linked to original sources

arXiv2016

Simultaneous Estimation of Noise Variance and Number of Peaks in Bayesian Spectral Deconvolution

The heuristic identification of peaks from noisy complex spectra often leads to misunderstanding of the physical and chemical properties of matter. In this paper, we propose a framework based on Bayesian inference, which enables us to separate multipeak spectra into single peaks statistically and consists of two steps. The first step is estimating both the noise variance and the number of peaks as hyperparameters based on Bayes free energy, which generally is not analytically tractable. The second step is fitting the parameters of each peak function to the given spectrum by calculating the posterior density, which has a problem of local minima and saddles since multipeak models are nonlinear and hierarchical. Our framework enables the escape from local minima or saddles by using the exchange Monte Carlo method and calculates Bayes free energy via the multiple histogram method. We discuss a simulation demonstrating how efficient our framework is and show that estimating both the noise variance and the number of peaks prevents overfitting, overpenalizing, and misunderstanding the precision of parameter estimation.

physics.data-an↗

arXiv2016

Compressed sensing in scanning tunneling microscopy/spectroscopy for observation of quasi-particle interference

We applied a method of compressed sensing to the observation of quasi-particle interference (QPI) by scanning tunneling microscopy/spectroscopy to improve efficiency and save measurement time. To solve an ill-posed problem owing to the scarcity of data, the compressed sensing utilizes the sparseness of QPI patterns in momentum space. We examined the performance of a sparsity-inducing algorithm called least absolute shrinkage and selection operator (LASSO), and demonstrated that LASSO enables us to recover a double-circle QPI pattern of the Ag(111) surface from a dataset whose size is less than that necessary for the conventional Fourier transformation method. In addition, the smallest number of data required for the recovery is discussed on the basis of cross validation.

cond-mat.mes-hall↗

arXiv2016

Sparse approximation based on a random overcomplete basis

We discuss a strategy of sparse approximation that is based on the use of an overcomplete basis, and evaluate its performance when a random matrix is used as this basis. A small combination of basis vectors is chosen from a given overcomplete basis, according to a given compression rate, such that they compactly represent the target data with as small a distortion as possible. As a selection method, we study the $\ell_0$- and $\ell_1$-based methods, which employ the exhaustive search and $\ell_1$-norm regularization techniques, respectively. The performance is assessed in terms of the trade-off relation between the representation distortion and the compression rate. First, we evaluate the performance analytically in the case that the methods are carried out ideally, using methods of statistical mechanics. Our result clarifies the fact that the $\ell_0$-based method greatly outperforms the $\ell_1$-based one. Second, we examine the practical performances of two well-known algorithms, orthogonal matching pursuit and approximate message passing, when they are used to execute the $\ell_0$- and $\ell_1$-based methods, respectively. Our examination shows that orthogonal matching pursuit achieves a much better performance than the exact execution of the $\ell_1$-based method, as well as approximate message passing. However, regarding the $\ell_0$-based method, there is still room to design more effective greedy algorithms than orthogonal matching pursuit. Finally, we evaluate the performances of the algorithms when they are applied to image data compression.

cs.IT↗

arXiv2014

Oscillations in Spurious States of the Associative Memory Model with Synaptic Depression

The associative memory model is a typical neural network model, which can store discretely distributed fixed-point attractors as memory patterns. When the network stores the memory patterns extensively, however, the model has other attractors besides the memory patterns. These attractors are called spurious memories. Both spurious states and memory states are equilibrium, so there is little difference between their dynamics. Recent physiological experiments have shown that short-term dynamic synapse called synaptic depression decreases its transmission efficacy to postsynaptic neurons according to the activities of presynaptic neurons. Previous studies have shown that synaptic depression induces oscillation in the network and decreases the storage capacity at finite temperature. How synaptic depression affects spurious states, however, is still unclear. We investigate the effect of synaptic depression on spurious states through Monte Carlo simulation. The results demonstrate that synaptic depression does not affect the memory states but mainly destabilizes the spurious states and induces the periodic oscillations.

q-bio.NC↗

arXiv2013

Generating Functional Analysis for Iterative CDMA Multiuser Detectors

We investigate the detection dynamics of a soft parallel interference canceller (soft-PIC), which includes a hard-PIC as a special case, for code-division multiple-access (CDMA) multiuser detection, applied to a randomly spread, fully synchronous base-band uncoded CDMA channel model with additive white Gaussian noise under perfect power control in the large-system limit. We analyze the detection dynamics of some iterative detectors, namely soft-PIC, the Onsager-reaction-cancelling parallel interference canceller (ORC-PIC) and the belief-propagation-based detector (BP-based detector), by the generating functional analysis (GFA). The GFA allows us to study the asymptotic behavior of the dynamics in the infinitely large system without assuming the independence of messages. We study the detection dynamics and the stationary estimates of an iterative algorithm. We also show the decoupling principle in iterative multiuser detection algorithms in the large-system limit. For a generic iterative multiuser detection algorithm with binary input, it is shown that the multiuser channel is equivalent to a bank of independent single-user additive non-Gaussian channels, whose signal-to-noise ratio degrades due to both the multiple-access interference and the Onsager reaction, at each stage of the algorithm. If an algorithm cancels the Onsager reaction, the equivalent single-user channels coincide with an additive white Gaussian noise channel. We also discuss ORC-PIC and the BP-based detector.

cs.IT↗

arXiv2012

Solvable model of a phase oscillator network on a circle with infinite-range Mexican-hat-type interaction

We describe a solvable model of a phase oscillator network on a circle with infinite-range Mexican-hat-type interaction. We derive self-consistent equations of the order parameters and obtain three non-trivial solutions characterized by the rotation number. We also derive relevant characteristics such as the location-dependent distributions of the resultant frequencies of desynchronized oscillators. Simulation results closely agree with the theoretical ones.

cond-mat.dis-nn↗

arXiv2011

Influence of synaptic depression on memory storage capacity

Synaptic efficacy between neurons is known to change within a short time scale dynamically. Neurophysiological experiments show that high-frequency presynaptic inputs decrease synaptic efficacy between neurons. This phenomenon is called synaptic depression, a short term synaptic plasticity. Many researchers have investigated how the synaptic depression affects the memory storage capacity. However, the noise has not been taken into consideration in their analysis. By introducing "temperature", which controls the level of the noise, into an update rule of neurons, we investigate the effects of synaptic depression on the memory storage capacity in the presence of the noise. We analytically compute the storage capacity by using a statistical mechanics technique called Self Consistent Signal to Noise Analysis (SCSNA). We find that the synaptic depression decreases the storage capacity in the case of finite temperature in contrast to the case of the low temperature limit, where the storage capacity does not change.

cond-mat.dis-nn↗

arXiv2011

Belief Propagation for Error Correcting Codes and Lossy Compression Using Multilayer Perceptrons

The belief propagation (BP) based algorithm is investigated as a potential decoder for both of error correcting codes and lossy compression, which are based on non-monotonic tree-like multilayer perceptron encoders. We discuss that whether the BP can give practical algorithms or not in these schemes. The BP implementations in those kind of fully connected networks unfortunately shows strong limitation, while the theoretical results seems a bit promising. Instead, it reveals it might have a rich and complex structure of the solution space via the BP-based algorithms.

cs.IT↗

arXiv2010

Complex sequencing rules of birdsong can be explained by simple hidden Markov processes

Complex sequencing rules observed in birdsongs provide an opportunity to investigate the neural mechanism for generating complex sequential behaviors. To relate the findings from studying birdsongs to other sequential behaviors, it is crucial to characterize the statistical properties of the sequencing rules in birdsongs. However, the properties of the sequencing rules in birdsongs have not yet been fully addressed. In this study, we investigate the statistical propertiesof the complex birdsong of the Bengalese finch (Lonchura striata var. domestica). Based on manual-annotated syllable sequences, we first show that there are significant higher-order context dependencies in Bengalese finch songs, that is, which syllable appears next depends on more than one previous syllable. This property is shared with other complex sequential behaviors. We then analyze acoustic features of the song and show that higher-order context dependencies can be explained using first-order hidden state transition dynamics with redundant hidden states. This model corresponds to hidden Markov models (HMMs), well known statistical models with a large range of application for time series modeling. The song annotation with these models with first-order hidden state dynamics agreed well with manual annotation, the score was comparable to that of a second-order HMM, and surpassed the zeroth-order model (the Gaussian mixture model (GMM)), which does not use context information. Our results imply that the hierarchical representation with hidden state dynamics may underlie the neural implementation for generating complex sequences with higher-order dependencies.

q-bio.NC↗

arXiv2010

Statistical mechanics of digital halftoning

We consider the problem of digital halftoning from the view point of statistical mechanics. The digital halftoning is a sort of image processing, namely, representing each grayscale in terms of black and white binary dots. The digital halftoning is achieved by making use of the threshold mask, namely, for each pixel, the halftoned binary pixel is determined as black if the original grayscale pixel is greater than or equal to the mask value and is determined as white vice versa. To determine the optimal value of the mask on each pixel for a given original grayscale image, we first assume that the human-eyes might recognize the black and white binary halftoned image as the corresponding grayscale one by linear filters. The Hamiltonian is constructed as a distance between the original and the recognized images which is written in terms of the threshold mask. We are confirmed that the system described by the Hamiltonian is regarded as a kind of antiferromagnetic Ising model with quenched disorders. By searching the ground state of the Hamiltonian, we obtain the optimal threshold mask and the resulting halftoned binary dots simultaneously. From the power-spectrum analysis, we find that the binary dots image is physiologically plausible from the view point of human-eyes modulation properties. We also propose a theoretical framework to investigate statistical performance of inverse digital halftoning, that is, the inverse process of halftoning. From the Bayesian inference view point, we rigorously show that the Bayes-optimal inverse-halftoning is achieved on a specific condition which is very similar to the so-called Nishimori line in the research field of spin glasses.

cond-mat.dis-nn↗

arXiv2010

Instabilities in associative memory model with synaptic depression and switching phenomena among attractors

We investigated how the stability of macroscopic states in the associative memory model is affected by synaptic depression. To this model, we applied the dynamical mean-field theory, which has recently been developed in stochastic neural network models with synaptic depression. By introducing a sublattice method, we derived macroscopic equations for firing state variables and depression variables. By using the macroscopic equations, we obtained the phase diagram when the strength of synaptic depression and the correlation level among stored patterns were changed. We found that there is an unstable region in which both the memory state and mixed state cannot be stable and that various switching phenomena can occur in this region.

cond-mat.dis-nn↗

arXiv2010

Mean Field Analysis of Stochastic Neural Network Models with Synaptic Depression

We investigated the effects of synaptic depression on the macroscopic behavior of stochastic neural networks. Dynamical mean field equations were derived for such networks by taking the average of two stochastic variables: a firing state varialbe and a synaptic variable. In these equations, their average product is decoupled as the product of averaged them because the two stochastic variables are independent. We proved the independence of these two stochastic variables assuming that the synaptic weight is of the order of 1/N with respect to the number of neurons N. Using these equations, we derived macroscopic steady state equations for a network with uniform connections and a ring attractor network with Mexican hat type connectivity and investigated the stability of the steady state solutions. An oscillatory uniform state was observed in the network with uniform connections due to a Hopf instability. With the ring network, high-frequency perturbations were shown not to affect system stability. Two mechanisms destabilize the inhomogeneous steady state, leading two oscillatory states. A Turing instability leads to a rotating bump state, while a Hopf instability leads to an oscillatory bump state, which was previous unreported. Various oscillatory states take place in a network with synaptic depression depending on the strength of the interneuron connections.

cond-mat.dis-nn↗

arXiv2010

Error correcting code using tree-like multilayer perceptron

An error correcting code using a tree-like multilayer perceptron is proposed. An original message $\mbi{s}^0$ is encoded into a codeword $\boldmath{y}_0$ using a tree-like committee machine (committee tree) or a tree-like parity machine (parity tree). Based on these architectures, several schemes featuring monotonic or non-monotonic units are introduced. The codeword $\mbi{y}_0$ is then transmitted via a Binary Asymmetric Channel (BAC) where it is corrupted by noise. The analytical performance of these schemes is investigated using the replica method of statistical mechanics. Under some specific conditions, some of the proposed schemes are shown to saturate the Shannon bound at the infinite codeword length limit. The influence of the monotonicity of the units on the performance is also discussed.

cond-mat.stat-mech↗

arXiv2009

Neural network model with discrete and continuous information representation

An associative memory model and a neural network model with a Mexican-hat type interaction are the two most typical attractor networks used in the artificial neural network models. The associative memory model has discretely distributed fixed-point attractors, and achieves a discrete information representation. On the other hand, a neural network model with a Mexican-hat type interaction uses a line attractor to achieves a continuous information representation, which can be seen in the working memory in the prefrontal cortex and columnar activity in the visual cortex. In the present study, we propose a neural network model that achieves discrete and continuous information representation. We use a statistical-mechanical analysis to find that a localized retrieval phase exists in the proposed model, where the memory pattern is retrieved in the localized subpopulation of the network. In the localized retrieval phase, the discrete and continuous information representation is achieved by using the orthogonality of the memory patterns and the neutral stability of fixed points along the positions of the localized retrieval. The obtained phase diagram suggests that the antiferromagnetic interaction and the external field are important for generating the localized retrieval phase.

q-bio.NC↗

arXiv2009

Effect of Slow Switching in On-line Learning for Ensemble Teachers

We have analyzed the generalization performance of a student which slowly switches ensemble teachers. By calculating the generalization error analytically using statistical mechanics in the framework of on-line learning, we show that the dynamical behaviors of generalization error have the periodicity that is synchronized with the switching period and the behaviors differ with the number of ensemble teachers. Furthermore, we show that the smaller the switching period is, the larger the difference is.

physics.soc-ph↗

arXiv2008

Statistical mechanics of lossy compression for non-monotonic multilayer perceptrons

A lossy data compression scheme for uniformly biased Boolean messages is investigated via statistical mechanics techniques. We utilize tree-like committee machine (committee tree) and tree-like parity machine (parity tree) whose transfer functions are non-monotonic. The scheme performance at the infinite code length limit is analyzed using the replica method. Both committee and parity treelike networks are shown to saturate the Shannon bound. The AT stability of the Replica Symmetric solution is analyzed, and the tuning of the non-monotonic transfer function is also discussed.

cond-mat.stat-mech↗

arXiv2008

Bayes-optimal inverse halftoning and statistical mechanics of the Q-Ising model

On the basis of statistical mechanics of the Q-Ising model, we formulate the Bayesian inference to the problem of inverse halftoning, which is the inverse process of representing gray-scales in images by means of black and white dots. Using Monte Carlo simulations, we investigate statistical properties of the inverse process, especially, we reveal the condition of the Bayes-optimal solution for which the mean-square error takes its minimum. The numerical result is qualitatively confirmed by analysis of the infinite-range model. As demonstrations of our approach, we apply the method to retrieve a grayscale image, such as standard image `Lenna', from the halftoned version. We find that the Bayes-optimal solution gives a fine restored grayscale image which is very close to the original.

cond-mat.dis-nn↗

arXiv2007

Sparse and Dense Encoding in Layered Associative Network of Spiking Neurons

A synfire chain is a simple neural network model which can propagate stable synchronous spikes called a pulse packet and widely researched. However how synfire chains coexist in one network remains to be elucidated. We have studied the activity of a layered associative network of Leaky Integrate-and-Fire neurons in which connection we embed memory patterns by the Hebbian Learning. We analyzed their activity by the Fokker-Planck method. In our previous report, when a half of neurons belongs to each memory pattern (memory pattern rate $F=0.5$), the temporal profiles of the network activity is split into temporally clustered groups called sublattices under certain input conditions. In this study, we show that when the network is sparsely connected ($F<0.5$), synchronous firings of the memory pattern are promoted. On the contrary, the densely connected network ($F>0.5$) inhibit synchronous firings. The sparseness and denseness also effect the basin of attraction and the storage capacity of the embedded memory patterns. We show that the sparsely(densely) connected networks enlarge(shrink) the basion of attraction and increase(decrease) the storage capacity.

q-bio.NC↗