arXiv subjects

Yoshiyuki Kabashima

Publications and source records attributed to Yoshiyuki Kabashima.

At least 91 records · Page 5Linked to original sources

arXiv2014

Cavity-based robustness analysis of interdependent networks: Influences of intranetwork and internetwork degree-degree correlations

We develop a methodology for analyzing the percolation phenomena of two mutually coupled (interdependent) networks based on the cavity method of statistical mechanics. In particular, we take into account the influence of degree-degree correlations inside and between the networks on the network robustness against targeted attacks and random failures. We show that the developed methodology is reduced to the well-known generating function formalism in the absence of degree-degree correlations. The validity of the developed methodology is confirmed by a comparison with the results of numerical experiments. Our analytical results imply that the robustness of the interdependent networks depends considerably on both the intra- and internetwork degree-degree correlations in the case of targeted attacks, whereas the significance of the degree-degree correlations is relatively low for random failures.

cond-mat.dis-nn↗

arXiv2014

Sample Complexity of Bayesian Optimal Dictionary Learning

We consider a learning problem of identifying a dictionary matrix D (M times N dimension) from a sample set of M dimensional vectors Y = N^{-1/2} DX, where X is a sparse matrix (N times P dimension) in which the density of non-zero entries is 0 rho is satisfied in the limit of N to infinity. Our analysis also implies that the posterior distribution given Y is condensed only at the correct dictionary D when the compression rate alpha is greater than a certain critical value alpha_M(rho). This suggests that belief propagation may allow us to learn D with a low computational complexity using O(N) samples.

cs.LG↗

arXiv2013

Reconstruction algorithm in compressed sensing based on maximum a posteriori estimation

We propose a systematic method for constructing a sparse data reconstruction algorithm in compressed sensing at a relatively low computational cost for general observation matrix. It is known that the cost of l1-norm minimization using a standard linear programming algorithm is O(N^3). We show that this cost can be reduced to O(N^2) by applying the approach of posterior maximization. Furthermore, in principle, the algorithm from our approach is expected to achieve the widest successful reconstruction region, which is evaluated from theoretical argument. We also discuss the relation between the belief propagation-based reconstruction algorithm introduced in preceding works and our approach.

cs.IT↗

arXiv2013

Statistical Mechanics Approach to Sparse Noise Denoising

Reconstruction fidelity of sparse signals contaminated by sparse noise is considered. Statistical mechanics inspired tools are used to show that the l1-norm based convex optimization algorithm exhibits a phase transition between the possibility of perfect and imperfect reconstruction. Conditions characterizing this threshold are derived and the mean square error of the estimate is obtained for the case when perfect reconstruction is not possible. Detailed calculations are provided to expose the mathematical tools to a wide audience.

cs.IT↗

arXiv2013

Entropy landscape of solutions in the binary perceptron problem

The statistical picture of the solution space for a binary perceptron is studied. The binary perceptron learns a random classification of input random patterns by a set of binary synaptic weights. The learning of this network is difficult especially when the pattern (constraint) density is close to the capacity, which is supposed to be intimately related to the structure of the solution space. The geometrical organization is elucidated by the entropy landscape from a reference configuration and of solution-pairs separated by a given Hamming distance in the solution space. We evaluate the entropy at the annealed level as well as replica symmetric level and the mean field result is confirmed by the numerical simulations on single instances using the proposed message passing algorithms. From the first landscape (a random configuration as a reference), we see clearly how the solution space shrinks as more constraints are added. From the second landscape of solution-pairs, we deduce the coexistence of clustering and freezing in the solution space.

cond-mat.dis-nn↗

arXiv2013

Adaptive Thouless-Anderson-Palmer approach to inverse Ising problems with quenched random fields

The adaptive Thouless-Anderson-Palmer equation is derived for inverse Ising problems in the presence of quenched random fields. We test the proposed scheme on Sherrington-Kirkpatrick, Hopfield, and random orthogonal models and find that the adaptive Thouless-Anderson-Palmer approach allows surprisingly accurate inference of quenched random fields whose distribution can be either Gaussian or bimodal, compared with other existing mean-field methods.

cond-mat.dis-nn↗

arXiv2013

Compressed sensing with sparse, structured matrices

In the context of the compressed sensing problem, we propose a new ensemble of sparse random matrices which allow one (i) to acquire and compress a ρ0-sparse signal of length N in a time linear in N and (ii) to perfectly recover the original signal, compressed at a rate α, by using a message passing algorithm (Expectation Maximization Belief Propagation) that runs in a time linear in N. In the large N limit, the scheme proposed here closely approaches the theoretical bound ρ0 = α, and so it is both optimal and efficient (linear time complexity). More generally, we show that several ensembles of dense random matrices can be converted into ensembles of sparse random matrices, having the same thresholds, but much lower computational complexity.

cs.IT↗

arXiv2013

Statistical Mechanics of Dictionary Learning

Finding a basis matrix (dictionary) by which objective signals are represented sparsely is of major relevance in various scientific and technological fields. We consider a problem to learn a dictionary from a set of training signals. We employ techniques of statistical mechanics of disordered systems to evaluate the size of the training set necessary to typically succeed in the dictionary learning. The results indicate that the necessary size is much smaller than previously estimated, which theoretically supports and/or encourages the use of dictionary learning in practical situations.

cond-mat.dis-nn↗

arXiv2012

Typical $l_1$-recovery limit of sparse vectors represented by concatenations of random orthogonal matrices

We consider the problem of recovering an $N$-dimensional sparse vector $\vm{x}$ from its linear transformation $\vm{y}=\vm{D} \vm{x}$ of $M(< N)$ dimension. Minimizing the $l_{1}$-norm of $\vm{x}$ under the constraint $\vm{y} = \vm{D} \vm{x}$ is a standard approach for the recovery problem, and earlier studies report that the critical condition for typically successful $l_1$-recovery is universal over a variety of randomly constructed matrices $\vm{D}$. For examining the extent of the universality, we focus on the case in which $\vm{D}$ is provided by concatenating $\nb=N/M$ matrices $\vm{O}_{1}, \vm{O}_{2},..., \vm{O}_\nb$ drawn uniformly according to the Haar measure on the $M \times M$ orthogonal matrices. By using the replica method in conjunction with the development of an integral formula for handling the random orthogonal matrices, we show that the concatenated matrices can result in better recovery performance than what the universality predicts when the density of non-zero signals is not uniform among the $\nb$ matrix modules. The universal condition is reproduced for the special case of uniform non-zero signal densities. Extensive numerical experiments support the theoretical predictions.

cs.IT↗

arXiv2012

First eigenvalue/eigenvector in sparse random symmetric matrices: influences of degree fluctuation

The properties of the first (largest) eigenvalue and its eigenvector (first eigenvector) are investigated for large sparse random symmetric matrices that are characterized by bimodal degree distributions. In principle, one should be able to accurately calculate them by solving a functional equation concerning auxiliary fields which come out in an analysis based on replica/cavity methods. However, the difficulty in analytically solving this equation makes an accurate calculation infeasible in practice. To overcome this problem, we develop approximation schemes on the basis of two exceptionally solvable examples. The schemes are reasonably consistent with numerical experiments when the statistical bias of positive matrix entries is sufficiently large, and they qualitatively explain why considerably large finite size effects of the first eigenvalue can be observed when the bias is relatively small.

cond-mat.dis-nn↗

arXiv2012

Analysis of Sparse Representations Using Bi-Orthogonal Dictionaries

The sparse representation problem of recovering an N dimensional sparse vector x from M < N linear observations y = Dx given dictionary D is considered. The standard approach is to let the elements of the dictionary be independent and identically distributed (IID) zero-mean Gaussian and minimize the l1-norm of x under the constraint y = Dx. In this paper, the performance of l1-reconstruction is analyzed, when the dictionary is bi-orthogonal D = [O1 O2], where O1,O2 are independent and drawn uniformly according to the Haar measure on the group of orthogonal M x M matrices. By an application of the replica method, we obtain the critical conditions under which perfect l1-recovery is possible with bi-orthogonal dictionaries.

cs.IT↗

arXiv2012

A Mean-field Approach for an Intercarrier Interference Canceller for OFDM

The similarity of the mathematical description of random-field spin systems to orthogonal frequency-division multiplexing (OFDM) scheme for wireless communication is exploited in an intercarrier-interference (ICI) canceller used in the demodulation of OFDM. The translational symmetry in the Fourier domain generically concentrates the major contribution of ICI from each subcarrier in the subcarrier's neighborhood. This observation in conjunction with mean field approach leads to a development of an ICI canceller whose necessary cost of computation scales linearly with respect to the number of subcarriers. It is also shown that the dynamics of the mean-field canceller are well captured by a discrete map of a single macroscopic variable, without taking the spatial and time correlations of estimated variables into account.

cond-mat.dis-nn↗

arXiv2012

A study of the universal threshold in the L1 recovery by statistical mechanics

We discuss the universality of the L1 recovery threshold in compressed sensing. Previous studies in the fields of statistical mechanics and random matrix integration have shown that L1 recovery under a random matrix with orthogonal symmetry has a universal threshold. This indicates that the threshold of L1 recovery under a non-orthogonal random matrix differs from the universal one. Taking this into account, we use a simple random matrix without orthogonal symmetry, where the random entries are not independent, and show analytically that the threshold of L1 recovery for such a matrix does not coincide with the universal one. The results of an extensive numerical experiment are in good agreement with the analytical results, which validates our methodology. Though our analysis is based on replica heuristics in statistical mechanics and is not rigorous, the findings nevertheless support the fact that the universality of the threshold is strongly related to the symmetry of the random matrix.

cs.IT↗

arXiv2011

Statistical mechanical assessment of a reconstruction limit of compressed sensing: Toward theoretical analysis of correlated signals

We provide a scheme for exploring the reconstruction limit of compressed sensing by minimizing the general cost function under the random measurement constraints for generic correlated signal sources. Our scheme is based on the statistical mechanical replica method for dealing with random systems. As a simple but non-trivial example, we apply the scheme to a sparse autoregressive model, where the first differences in the input signals of the correlated time series are sparse, and evaluate the critical compression rate for a perfect reconstruction. The results are in good agreement with a numerical experiment for a signal reconstruction.

cs.IT↗

arXiv2011

A signal recovery algorithm for sparse matrix based compressed sensing

We have developed an approximate signal recovery algorithm with low computational cost for compressed sensing on the basis of randomly constructed sparse measurement matrices. The law of large numbers and the central limit theorem suggest that the developed algorithm saturates the Donoho-Tanner weak threshold for the perfect recovery when the matrix becomes as dense as the signal size $N$ and the number of measurements $M$ tends to infinity keep $α=M/N \sim O(1)$, which is supported by extensive numerical experiments. Even when the numbers of non-zero entries per column/row in the measurement matrices are limited to $O(1)$, numerical experiments indicate that the algorithm can still typically recover the original signal perfectly with an $O(N)$ computational cost per update as well if the density $ρ$ of non-zero entries of the signal is lower than a certain critical value $ρ_{\rm th}(α)$ as $N,M \to \infty$.

cs.IT↗

arXiv2010

Cavity analysis on the robustness of random networks against targeted attacks: Influences of degree-degree correlations

We developed a scheme for evaluating the size of the largest connected subnetwork (giant component) in random networks and the percolation threshold when sites (nodes) and/or bonds (edges) are removed from the networks based on the cavity method of statistical mechanics of disordered systems. An advantage of our scheme is the capability of handling targeted attacks on sites/bonds in the presence of degree correlations beyond naive analyses on random failures (crashes) in networks of no degree correlations. We apply our scheme particularly to random networks of bimodal degree distribution (two-peak networks), which have been proposed in earlier studies as robust networks against random failures of site and/or targeted attacks on sites, and show that the correlations among degrees affect a network's robustness against targeted attacks on sites or bonds non-trivially depending on details of network configurations.

cond-mat.dis-nn↗

arXiv2010

Statistical Mechanical Analysis of Compressed Sensing Utilizing Correlated Compression Matrix

We investigate a reconstruction limit of compressed sensing for a reconstruction scheme based on the L1-norm minimization utilizing a correlated compression matrix with a statistical mechanics method. We focus on the compression matrix modeled as the Kronecker-type random matrix studied in research on multi-input multi-output wireless communication systems. We found that strong one-dimensional correlations between expansion bases of original information slightly degrade reconstruction performance.

cs.IT↗

arXiv2010

Statistical Mechanical Analysis of a Typical Reconstruction Limit of Compressed Sensing

We use the replica method of statistical mechanics to examine a typical performance of correctly reconstructing $N$-dimensional sparse vector $bx=(x_i)$ from its linear transformation $by=bF bx$ of $P$ dimensions on the basis of minimization of the $L_p$-norm $||bx||_p= lim_{epsilon to +0} sum_{i=1}^N |x_i|^{p+epsilon}$. We characterize the reconstruction performance by the critical relation of the successful reconstruction between the ratio $alpha=P/N$ and the density $rho$ of non-zero elements in $bx$ in the limit $P,,N to infty$ while keeping $alpha sim O(1)$ and allowing asymptotically negligible reconstruction errors. We show that the critical relation $alpha_c(rho)$ holds universally as long as $bF^{rm T}bF$ can be characterized asymptotically by a rotationally invariant random matrix ensemble and $bF bF^{rm T}$ is typically of full rank. This supports the universality of the critical relation observed by Donoho and Tanner ({em Phil. Trans. R. Soc. A}, vol.~367, pp.~4273--4293, 2009; arXiv: 0807.3590) for various ensembles of compression matrices.

cs.IT↗