arXiv subjects

Ayaka Sakata

Publications and source records attributed to Ayaka Sakata.

31 records · Page 2Linked to original sources

arXiv2016

Evaluation of Generalized Degrees of Freedom for Sparse Estimation by Replica Method

We develop a method to evaluate the generalized degrees of freedom (GDF), which is a key quantity of a model selection criterion, for linear regression with sparse regularization. Using the replica method, GDF is expressed by the variables that characterize the saddle point of the free energy without depending on the form of the regularization. Within the framework of replica symmetric (RS) analysis, GDF is provided with a physical meaning as the effective density of non-zero components. The validity of our method in the RS phase is supported by the consistency of our results with previous mathematical results. The analytical results in the RS phase are numerically achieved by the belief propagation algorithm.

cs.IT↗

arXiv2016

Phase transitions and sample complexity in Bayes-optimal matrix factorization

We analyse the matrix factorization problem. Given a noisy measurement of a product of two matrices, the problem is to estimate back the original matrices. It arises in many applications such as dictionary learning, blind matrix calibration, sparse principal component analysis, blind source separation, low rank matrix completion, robust principal component analysis or factor analysis. It is also important in machine learning: unsupervised representation learning can often be studied through matrix factorization. We use the tools of statistical mechanics - the cavity and replica methods - to analyze the achievability and computational tractability of the inference problems in the setting of Bayes-optimal inference, which amounts to assuming that the two matrices have random independent elements generated from some known distribution, and this information is available to the inference algorithm. In this setting, we compute the minimal mean-squared-error achievable in principle in any computational time, and the error that can be achieved by an efficient approximate message passing algorithm. The computation is based on the asymptotic state-evolution analysis of the algorithm. The performance that our analysis predicts, both in terms of the achieved mean-squared-error, and in terms of sample complexity, is extremely promising and motivating for a further development of the algorithm.

math.NA↗

arXiv2015

Replica Symmetric Bound for Restricted Isometry Constant

We develop a method for evaluating restricted isometry constants (RICs). This evaluation is reduced to the identification of the zero-points of entropy, which is defined for submatrices that are composed of columns selected from a given measurement matrix. Using the replica method developed in statistical mechanics, we assess RICs for Gaussian random matrices under the replica symmetric (RS) assumption. In order to numerically validate the adequacy of our analysis, we employ the exchange Monte Carlo (EMC) method, which has been empirically demonstrated to achieve much higher numerical accuracy than naive Monte Carlo methods. The EMC method suggests that our theoretical estimation of an RIC corresponds to an upper bound that is tighter than in preceding studies. Physical consideration indicates that our assessment of the RIC could be improved by taking into account the replica symmetry breaking.

cs.IT↗

arXiv2014

Error correcting codes and spatial coupling

These are notes from the lecture of Rüdiger Urbanke given at the autumn school "Statistical Physics, Optimization, Inference, and Message-Passing Algorithms", that took place in Les Houches, France from Monday September 30th, 2013, till Friday October 11th, 2013. The school was organized by Florent Krzakala from UPMC and ENS Paris, Federico Ricci-Tersenghi from La Sapienza Roma, Lenka Zdeborovà from CEA Saclay and CNRS, and Riccardo Zecchina from Politecnico Torino. The first three sections cover the basics of polar codes and low density parity check codes. In the last three sections, we see how the spatial coupling helps belief propagation decoding.

cs.IT↗

arXiv2014

Sample Complexity of Bayesian Optimal Dictionary Learning

We consider a learning problem of identifying a dictionary matrix D (M times N dimension) from a sample set of M dimensional vectors Y = N^{-1/2} DX, where X is a sparse matrix (N times P dimension) in which the density of non-zero entries is 0 rho is satisfied in the limit of N to infinity. Our analysis also implies that the posterior distribution given Y is condensed only at the correct dictionary D when the compression rate alpha is greater than a certain critical value alpha_M(rho). This suggests that belief propagation may allow us to learn D with a low computational complexity using O(N) samples.

cs.LG↗

arXiv2013

Statistical Mechanics of Dictionary Learning

Finding a basis matrix (dictionary) by which objective signals are represented sparsely is of major relevance in various scientific and technological fields. We consider a problem to learn a dictionary from a set of training signals. We employ techniques of statistical mechanics of disordered systems to evaluate the size of the training set necessary to typically succeed in the dictionary learning. The results indicate that the necessary size is much smaller than previously estimated, which theoretically supports and/or encourages the use of dictionary learning in practical situations.

cond-mat.dis-nn↗

arXiv2012

Time evolution of autocorrelation function in dynamical replica theory

Asynchronous dynamics given by the master equation in the Sherrington--Kirkpatrick (SK) spin-glass model is studied based on dynamical replica theory (DRT) with an extension to take into account the autocorrelation function. The dynamical behaviour of the system is approximately described by dynamical equations of the macroscopic quantities: magnetization, energy contributed by randomness, and the autocorrelation function. The dynamical equations under the replica symmetry assumption are derived by introducing the subshell equipartitioning assumption and exploiting the replica method. The obtained dynamical equations are compared with Monte Carlo (MC) simulations, and it is demonstrated that the proposed formula describes well the time evolution of the autocorrelation function in some parameter regions. The study offers a reasonable description of the autocorrelation function in the SK spin-glass system.

cond-mat.dis-nn↗

arXiv2012

A Mean-field Approach for an Intercarrier Interference Canceller for OFDM

The similarity of the mathematical description of random-field spin systems to orthogonal frequency-division multiplexing (OFDM) scheme for wireless communication is exploited in an intercarrier-interference (ICI) canceller used in the demodulation of OFDM. The translational symmetry in the Fourier domain generically concentrates the major contribution of ICI from each subcarrier in the subcarrier's neighborhood. This observation in conjunction with mean field approach leads to a development of an ICI canceller whose necessary cost of computation scales linearly with respect to the number of subcarriers. It is also shown that the dynamics of the mean-field canceller are well captured by a discrete map of a single macroscopic variable, without taking the spatial and time correlations of estimated variables into account.

cond-mat.dis-nn↗

arXiv2011

Replica symmetry breaking in an adiabatic spin-glass model of adaptive evolution

We study evolutionary canalization using a spin-glass model with replica theory, where spins and their interactions are dynamic variables whose configurations correspond to phenotypes and genotypes, respectively. The spins are updated under temperature T_S, and the genotypes evolve under temperature T_J, according to the evolutionary fitness. It is found that adaptation occurs at T_S < T_S^{RS}, and a replica symmetric phase emerges at T_S^{RSB} < T_S < T_S^{RS}. The replica symmetric phase implies canalization, and replica symmetry breaking at lower temperatures indicates loss of robustness.

cond-mat.dis-nn↗

arXiv2010

Partial annealing of a coupled mean-field spin-glass model with an embedded pattern

A partially annealed mean-field spin-glass model with a locally embedded pattern is studied. The model consists of two dynamical variables, spins and interactions, that are in contact with thermal baths at temperatures T_S and T_J, respectively. Unlike the quenched system, characteristic correlations among the interactions are induced by the partial annealing. The model exhibits three phases, which are paramagnetic, ferromagnetic and spin-glass phases. In the ferromagnetic phase, the embedded pattern is stably realized. The phase diagram depends significantly on the ratio of two temperatures n=T_J/T_S. In particular, a reentrant transition from the embedded ferromagnetic to the spin-glass phases with T_S decreasing is found only below at a certain value of n. This indicates that above the critical value n_c the embedded pattern is supported by local field from a non-embedded region. Some equilibrium properties of the interactions in the partial annealing are also discussed in terms of frustration.

cond-mat.dis-nn↗

arXiv2009

A statistical-mechanical study of evolution of robustness in noisy environment

In biological systems, expression dynamics that can provide fitted phenotype patterns with respect to a specific function have evolved through mutations. This has been observed in the evolution of proteins for realizing folding dynamics through which a target structure is shaped. We study this evolutionary process by introducing a statistical-mechanical model of interacting spins, where a configuration of spins and their interactions $\bm{J}$ represent a phenotype and genotype, respectively. The phenotype dynamics are given by a stochastic process with temperature $T_{S}$ under a Hamiltonian with $\bm{J}$. The evolution of $\bm{J}$ is also stochastic with temperature $T_{J}$ and follows mutations introduced into $\bm{J}$ and selection based on a fitness defined for a configuration of a given set of target spins. Below a certain temperature $T_{S}^{c2}$, the interactions $\bm{J}$ that achieve the target pattern evolve, whereas another phase transition is observed at $T_{S}^{c1}<T_{S}^{c2}$. At low temperatures $T_{S}<T_{S}^{c1}$, the Hamiltonian exhibits a spin-glass like phase, where the dynamics toward the target pattern require long time steps, and the fitness often decreases drastically as a result of a single mutation to $\bm{J}$. In the intermediate-temperature region, the dynamics to shape the target pattern proceed rapidly and are robust to mutations of $\bm{J}$. The interactions in this region have no frustration around the target pattern and results in funnel-type dynamics. We propose that the ubiquity of funnel-type dynamics, as observed in protein folding, is a consequence of evolution subjected to thermal noise beyond a certain level; this also leads to mutational robustness of the fitness.

cond-mat.dis-nn↗

arXiv2009

Funnel landscape and mutational robustness as a result of evolution under thermal noise

In biological systems, expression dynamics to shape a fitted phenotype for function has evolved through mutations to genes, as observed in the evolution of funnel landscape in protein. We study this evolutionary process with a statistical-mechanical model of interacting spins, where the fitted phenotype is represented by a configuration of a given set of "target spins" and interaction matrix J among spins is genotype evolving over generations. The expression dynamics is given by stochastic process with temperature T_S to decrease energy for a given set of J. The evolution of J is also stochastic with temperature T_J, following mutation in J and selection based on a fitness given by configurations of the target spins. Below a certain temperature T_S^{c2}, the highly adapted J evolves, whereasanother phase transition characterised by frustration occurs at T_S^{c1}<T_S^{c2}. At temperature lower than T_S^{c1}, the Hamiltonian exhibits a spin-glass like phase, where the dynamics requires long time steps to produce the fitted phenotype, and the fitness often decreases drastically by single mutation. In contrast, in the intermediate temperature phase between T_S^{c1} and T_S^{c2}, the evolved genotypes, that have no frustration around the target spins (we call "local Mattis state"), give a funnel-like rapid expression dynamics and are robust to mutation. These results imply that evolution under thermal noise beyond a certain level leads to funnel dynamics and mutational robustness. We will explain its mechanism with the statistical-mechanical method.

cond-mat.dis-nn↗

arXiv2008

Infectious Default Model with Recovery and Continuous Limit

We introduce an infectious default and recovery model for N obligors. Obligors are assumed to be exchangeable and their states are described by N Bernoulli random variables S_{i} (i=1,...,N). They are expressed by multiplying independent Bernoulli variables X_{i},Y_{ij},Y'_{ij}, and default and recovery infections are described by Y_{ij} and Y'_{ij}. We obtain the default probability function P(k) for k defaults. Taking its continuous limit, we find two nontrivial probability distributions with the reflection symmetry of S_{i} \leftrightarrow 1-S_{i}. Their profiles are singular and oscillating and we understand it theoretically. We also compare P(k) with an implied default distribution function inferred from the quotes of iTraxx-CJ. In order to explain the behavior of the implied distribution, the recovery effect may be necessary.

physics.data-an↗