Searcharxiv⌕ Search

arXiv subjects

Kazushi Mimura

Publications and source records attributed to Kazushi Mimura.

At least 19 recordsLinked to original sources
arXiv2025

Dynamical mean field approach to associative memory model with non-monotonic transfer functions

The Hopfield associative memory model stores random patterns in synaptic couplings according to Hebb's rule and retrieves them through gradient descent on an energy function. This conventional setting, where neurons are assumed to have monotonic transfer functions, has been central to understanding associative memory. Morita (1993, Neural Netw. 6 115), however, showed that introducing non-monotonic transfer functions can dramatically enhance retrieval performance. While this phenomenon has been qualitatively examined, a full quantitative theory remains elusive due to the difficulty of analysis in the absence of an underlying energy function. In this work, we apply dynamical mean-field theory to the discrete-time synchronous retrieval dynamics of the non-monotonic model, which succeeds in accurately characterizing its macroscopic dynamical properties. We also derive conditions for retrieval states, and clarify their relation to previous studies. Our results provide new insights into the non-equilibrium retrieval dynamics of associative memory models.

cond-mat.dis-nn↗
arXiv2025

Neural Tangent Kernels and Fisher Information Matrices for Simple ReLU Networks with Random Hidden Weights

Fisher information matrices and neural tangent kernels (NTK) for 2-layer ReLU networks with random hidden weight are argued. We discuss the relation between both notions as a linear transformation and show that spectral decomposition of NTK with concrete forms of eigenfunctions with major eigenvalues. We also obtain an approximation formula of the functions presented by the 2-layer neural networks.

cs.LG↗
arXiv2025

Dynamical Properties of Dense Associative Memory

Dense associative memory, a fundamental instance of modern Hopfield networks, can store a large number of memory patterns as equilibrium states of recurrent networks. While the stationary-state storage capacity has been investigated, its dynamical properties have not yet been discussed. In this paper, we analyze the dynamics using an exact approach based on generating functional analysis. We show results on convergence properties of memory retrieval, such as the convergence time and the size of the attraction basins. Our analysis enables a quantitative evaluation of the convergence time and the storage capacity of dense associative memory, which is useful for model design. Unlike the traditional Hopfield model, the retrieval of a pattern does not act as additional noise to itself, suggesting that the structure of modern networks makes recall more robust. Furthermore, the methodology addressed here can be applied to other energy-based models, and thus has the potential to contribute to the design of future architectures.

cond-mat.dis-nn↗
arXiv2024

Highly Versatile FPGA-Implemented Cyber Coherent Ising Machine

In recent years, quantum Ising machines have drawn a lot of attention, but due to physical implementation constraints, it has been difficult to achieve dense coupling, such as full coupling with sufficient spins to handle practical large-scale applications. Consequently, classically computable equations have been derived from quantum master equations for these quantum Ising machines. Parallel implementations of these algorithms using FPGAs have been used to rapidly find solutions to these problems on a scale that is difficult to achieve in physical systems. We have developed an FPGA implemented cyber coherent Ising machine (cyber CIM) that is much more versatile than previous implementations using FPGAs. Our architecture is versatile since it can be applied to the open-loop CIM, which was proposed when CIM research began, to the closed-loop CIM, which has been used recently, as well as to Jacobi successive over-relaxation method. By modifying the sequence control code for the calculation control module, other algorithms such as Simulated Bifurcation (SB) can also be implemented. Earlier research on large-scale FPGA implementations of SB and CIM used binary or ternary discrete values for connections, whereas the cyber CIM used FP32 values. Also, the cyber CIM utilized Zeeman terms that were represented as FP32, which were not present in other large-scale FPGA systems. Our implementation with continuous interaction realizes N=4096 on a single FPGA, comparable to the single-FPGA implementation of SB with binary interactions, with N=4096. The cyber CIM enables applications such as CDMA multi-user detector and L0 compressed sensing which were not possible with earlier FPGA systems, while enabling superior calculation speeds, more than ten times faster than a GPU implementation. The calculation speed can be further improved by increasing parallelism, such as through clustering.

cs.AR↗
arXiv2024

L0-regularized compressed sensing with Mean-field Coherent Ising Machines

Coherent Ising Machine (CIM) is a network of optical parametric oscillators that solves combinatorial optimization problems by finding the ground state of an Ising Hamiltonian. As a practical application of CIM, Aonishi et al. proposed a quantum-classical hybrid system to solve optimization problems of L0-regularization-based compressed sensing (L0RBCS). Gunathilaka et al. has further enhanced the accuracy of the system. However, the computationally expensive CIM's stochastic differential equations (SDEs) limit the use of digital hardware implementations. As an alternative to Gunathilaka et al.'s CIM SDEs used previously, we propose using the mean-field CIM (MF-CIM) model, which is a physics-inspired heuristic solver without quantum noise. MF-CIM surmounts the high computational cost due to the simple nature of the differential equations (DEs). Furthermore, our results indicate that the proposed model has similar performance to physically accurate SDEs in both artificial and magnetic resonance imaging data, paving the way for implementing CIM-based L0RBCS on digital hardware such as Field Programmable Gate Arrays (FPGAs).

cs.ET↗
arXiv2023

Photoemission Orbital Tomography Using Robust Sparse PhaseLift

Photoemission orbital tomography (POT) from photoelectron momentum maps (PMMs) has enabled detailed analysis of the shape and energy of molecular orbitals in the adsorbed state. This study proposes a new POT method based on the PhaseLift. Molecular orbitals, including three-dimensional phases, can be identified from a single PMM by actively providing atomic positions and basis. Moreover, our method is robust to noise and can perfectly discriminate adsorption-induced molecular deformations with an accuracy of 0.05 [angstrom]. Our new method enables simultaneous analysis of the three-dimensional shapes of molecules and molecular orbitals and thus paves the way for advanced quantum-mechanical interpretation of adsorption-induced electronic state changes and photo-excited inter-molecular interactions.

cond-mat.mtrl-sci↗
arXiv2023

Effective implementation of $l_0$-Regularised Compressed Sensing with Chaotic-Amplitude-Controlled Coherent Ising Machines

Coherent Ising Machine (CIM) is a network of optical parametric oscillators that can solve large-scale combinatorial optimisation problems by finding the ground state of an Ising Hamiltonian. As a practical application of CIM, Aonishi et al., proposed a quantum-classical hybrid system to solve optimisation problems of $l_0$-regularisation-based compressed sensing. In the hybrid system, the CIM was an open-loop system without an amplitude control feedback loop. In this case, the hybrid system is enhanced by using a closed-loop CIM to achieve chaotic behaviour around the target amplitude, which would enable escaping from local minima in the energy landscape. Both artificial and magnetic resonance image data were used for the testing of our proposed closed-loop system. Compared with the open-loop system, the results of this study demonstrate an improved degree of accuracy and a wider range of effectiveness.

quant-ph↗
arXiv2022

L0 regularization-based compressed sensing with quantum-classical hybrid approach

L0-regularization-based compressed sensing (L0-RBCS) has the potential to outperform L1-regularization-based compressed sensing (L1-RBCS), but the optimization in L0-RBCS is difficult because it is a combinatorial optimization problem. To perform optimization in L0-RBCS, we propose a quantum-classical hybrid system consisting of a quantum machine and a classical digital processor. The coherent Ising machine (CIM) is a suitable quantum machine for this system because this optimization problem can only be solved with a densely connected network. To evaluate the performance of the CIM-classical hybrid system theoretically, a truncated Wigner stochastic differential equation (W-SDE) is introduced as a model for the network of degenerate optical parametric oscillators, and macroscopic equations are derived by applying statistical mechanics to the W-SDE. We show that the system performance in principle approaches the theoretical limit of compressed sensing and this hybrid system may exceed the estimation accuracy of L1-RBCS in actual situations, such as in magnetic resonance imaging data analysis.

quant-ph↗
arXiv2018

Generating Functional Analysis of Iterative Sparse Signal Recovery Algorithms with Divergence-Free Estimators

Approximate message passing (AMP) is an effective iterative sparse recovery algorithm for linear system models. Its performance is characterized by the state evolution (SE) which is a simple scalar recursion. However, depending on a measurement matrix ensemble, AMP may face a convergence problem. To avoid this problem, orthogonal AMP (OAMP), which uses de-correlation linear estimation and divergence-free non-linear estimation, was proposed by Ma and Ping. They also provide the SE analysis for OAMP. In their SE analysis, the following two assumptions were made: (i) The estimated vector of the de-correlation linear estimator consists of i.i.d. zero-mean Gaussian entries independent of the vector to be estimated and (ii) the estimated vector of the divergence-free non-linear estimator consists of i.i.d. entries independent of the measurement matrix and the noise vector. In this paper, we derive a simple scalar recursion to characterize iterative sparse recovery algorithms with divergence-free estimators without such assumptions of independence of messages by using the generating functional analysis (GFA), which allows us to study the dynamics by an exact way in the large system limit.

eess.SP↗
arXiv2013

Generating Functional Analysis for Iterative CDMA Multiuser Detectors

We investigate the detection dynamics of a soft parallel interference canceller (soft-PIC), which includes a hard-PIC as a special case, for code-division multiple-access (CDMA) multiuser detection, applied to a randomly spread, fully synchronous base-band uncoded CDMA channel model with additive white Gaussian noise under perfect power control in the large-system limit. We analyze the detection dynamics of some iterative detectors, namely soft-PIC, the Onsager-reaction-cancelling parallel interference canceller (ORC-PIC) and the belief-propagation-based detector (BP-based detector), by the generating functional analysis (GFA). The GFA allows us to study the asymptotic behavior of the dynamics in the infinitely large system without assuming the independence of messages. We study the detection dynamics and the stationary estimates of an iterative algorithm. We also show the decoupling principle in iterative multiuser detection algorithms in the large-system limit. For a generic iterative multiuser detection algorithm with binary input, it is shown that the multiuser channel is equivalent to a bank of independent single-user additive non-Gaussian channels, whose signal-to-noise ratio degrades due to both the multiple-access interference and the Onsager reaction, at each stage of the algorithm. If an algorithm cancels the Onsager reaction, the equivalent single-user channels coincide with an additive white Gaussian noise channel. We also discuss ORC-PIC and the BP-based detector.

cs.IT↗
arXiv2011

Linear Complexity Lossy Compressor for Binary Redundant Memoryless Sources

A lossy compression algorithm for binary redundant memoryless sources is presented. The proposed scheme is based on sparse graph codes. By introducing a nonlinear function, redundant memoryless sequences can be compressed. We propose a linear complexity compressor based on the extended belief propagation, into which an inertia term is heuristically introduced, and show that it has near-optimal performance for moderate block lengths.

cs.IT↗
arXiv2011

Generating Functional Analysis of Iterative Algorithms for Compressed Sensing

It has been shown that approximate message passing algorithm is effective in reconstruction problems for compressed sensing. To evaluate dynamics of such an algorithm, the state evolution (SE) has been proposed. If an algorithm can cancel the correlation between the present messages and their past values, SE can accurately tract its dynamics via a simple one-dimensional map. In this paper, we focus on dynamics of algorithms which cannot cancel the correlation and evaluate it by the generating functional analysis (GFA), which allows us to study the dynamics by an exact way in the large system limit.

cs.IT↗
arXiv2011

Belief Propagation for Error Correcting Codes and Lossy Compression Using Multilayer Perceptrons

The belief propagation (BP) based algorithm is investigated as a potential decoder for both of error correcting codes and lossy compression, which are based on non-monotonic tree-like multilayer perceptron encoders. We discuss that whether the BP can give practical algorithms or not in these schemes. The BP implementations in those kind of fully connected networks unfortunately shows strong limitation, while the theoretical results seems a bit promising. Instead, it reveals it might have a rich and complex structure of the solution space via the BP-based algorithms.

cs.IT↗
arXiv2010

Error correcting code using tree-like multilayer perceptron

An error correcting code using a tree-like multilayer perceptron is proposed. An original message $\mbi{s}^0$ is encoded into a codeword $\boldmath{y}_0$ using a tree-like committee machine (committee tree) or a tree-like parity machine (parity tree). Based on these architectures, several schemes featuring monotonic or non-monotonic units are introduced. The codeword $\mbi{y}_0$ is then transmitted via a Binary Asymmetric Channel (BAC) where it is corrupted by noise. The analytical performance of these schemes is investigated using the replica method of statistical mechanics. Under some specific conditions, some of the proposed schemes are shown to saturate the Shannon bound at the infinite codeword length limit. The influence of the monotonicity of the units on the performance is also discussed.

cond-mat.stat-mech↗
arXiv2009

Parallel dynamics of disordered Ising spin systems on finitely connected directed random graphs with arbitrary degree distributions

We study the stochastic parallel dynamics of Ising spin systems defined on finitely connected directed random graphs with arbitrary degree distributions, using generating functional analysis. For fully asymmetric graphs the dynamics of the system can be completely solved, due to the asymptotic absence of loops. For arbitrary graph symmetry, we solve the dynamics exactly for the first few time steps, and we construct approximate stationary solutions.

cond-mat.dis-nn↗
arXiv2009

Parallel dynamics of continuous Hopfield model revisited

We have applied the generating functional analysis (GFA) to the continuous Hopfield model. We have also confirmed that the GFA predictions in some typical cases exhibit good consistency with computer simulation results. When a retarded self-interaction term is omitted, the GFA result becomes identical to that obtained using the statistical neurodynamics as well as the case of the sequential binary Hopfield model.

cond-mat.stat-mech↗