SEARCH · Searcharxiv

Results for “cond-mat.dis-nn”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32Linked to original sources

arXiv2019

Chaos in a quantum rotor model

We study scrambling in a model consisting of a number $N$ of $M$-component quantum rotors coupled by random infinite-range interactions. This model is known to have both a paramagnetic phase and a spin glass phase separated by second order phase transition. We calculate in perturbation theory the squared commutator of rotor fields at different sites in the paramagnetic phase, to leading non-trivial order at large $N$ and large $M$. This quantity diagnoses the onset of quantum chaos in this system, and we show that the squared commutator grows exponentially with time, with a Lyapunov exponent proportional to $\frac{1}{M}$. At high temperature, the Lyapunov exponent limits to a value set by the microscopic couplings, while at low temperature, the exponent exhibits a $T^4$ dependence on temperature $T$.

cond-mat.dis-nn↗

arXiv2019

Ensembles, Dynamics, and Cell Types: Revisiting the Statistical Mechanics Perspective on Cellular Regulation

Genetic regulatory networks control ontogeny. For fifty years Boolean networks have served as models of such systems, ranging from ensembles of random Boolean networks as models for generic properties of gene regulation to working dynamical models of a growing number of sub-networks of real cells. At the same time, their statistical mechanics has been thoroughly studied. Here we recapitulate their original motivation in the context of current theoretical and empirical research. We discuss ensembles of random Boolean networks whose dynamical attractors model cell types. A sub-ensemble is the critical ensemble. There is now strong evidence that genetic regulatory networks are dynamically critical, and that evolution is exploring the critical sub-ensemble. The generic properties of this sub-ensemble predict essential features of cell differentiation. In particular, the number of attractors in such networks scales as the DNA content raised to the 0.63 power. Data on the number of cell types as a function of the DNA content per cell shows a scaling relationship of 0.88. Thus, the theory correctly predicts a power law relationship between the number of cell types and the DNA contents per cell, and a comparable slope. We discuss these new scaling values and show prospects for new research lines for Boolean networks as a base model for systems biology.

q-bio.MN↗

arXiv2019

From Parisi to Boltzmann

We sketch a new framework for the analysis of disordered systems, in particular mean field spin glasses, which is variational in nature and within the formalism of classical thermodynamics. For concreteness, only the Sherrington-Kirkpatrick model is considered here. For this we show how the Parisi solution (replica symmetric, or when replica symmetry is broken) emerges, in large but finite volumes, from a high temperature expansion to second order of the Gibbs potential with respect to order parameters encoding the law of the effective fields. In contrast with classical systems where convexity in the order parameters is the default situation, the functionals employed here are, at infinite temperature, concave: this feature is eventually due to the Gaussian nature of the interaction and implies, in particular, that the canonical Boltzmann-Gibbs variational principles must be reversed. The considerations suggest that thermodynamical phase transitions are intimately related to the divergence of the infinite expansions.

math.PR↗

arXiv2019

Mean Field Limit of the Learning Dynamics of Multilayer Neural Networks

Can multilayer neural networks -- typically constructed as highly complex structures with many nonlinearly activated neurons across layers -- behave in a non-trivial way that yet simplifies away a major part of their complexities? In this work, we uncover a phenomenon in which the behavior of these complex networks -- under suitable scalings and stochastic gradient descent dynamics -- becomes independent of the number of neurons as this number grows sufficiently large. We develop a formalism in which this many-neurons limiting behavior is captured by a set of equations, thereby exposing a previously unknown operating regime of these networks. While the current pursuit is mathematically non-rigorous, it is complemented with several experiments that validate the existence of this behavior.

cs.LG↗

arXiv2019

Machine Learning Forcefield for Silicate Glasses

Developing accurate, transferable, and computationally-efficient interatomic forcefields is key to facilitate the modeling of silicate glasses. However, the high number of forcefield parameters that need to be optimized render traditional parameterization methods poorly efficient or potentially subject to bias. Here, we present a new forcefield parameterization methodology based on ab initio molecular dynamics simulations, Gaussian process regression, and Bayesian optimization. By taking the example of glassy silica, we show that our methodology yields a new interatomic forcefield that offers an unprecedented description of the atomic structure of silica. This methodology offers a new route to efficiently parameterize new empirical interatomic forcefields for silicate glasses with very limited need for intuition.

cond-mat.mtrl-sci↗

arXiv2019

From Sharma-Mittal to von-Neumann Entropy of a Graph

In this article, we introduce the Sharma-Mittal entropy of a graph, which is a generalization of the existing idea of the von-Neumann entropy. The well-known R{é}nyi, Thallis, and von-Neumann entropies can be expressed as limiting cases of Sharma-Mittal entropy. We have explicitly calculated them for cycle, path, and complete graphs. Also, we have proposed a number of bounds for these entropies. In addition, we have also discussed the entropy of product graphs, such as Cartesian, Kronecker, Lexicographic, Strong, and Corona products. The change in entropy can also be utilized in the analysis of growing network models (Corona graphs), useful in generating complex networks.

cs.DM↗

arXiv2019

On the complexity of logistic regression models

We investigate the complexity of logistic regression models which is defined by counting the number of indistinguishable distributions that the model can represent (Balasubramanian, 1997). We find that the complexity of logistic models with binary inputs does not only depend on the number of parameters but also on the distribution of inputs in a non-trivial way which standard treatments of complexity do not address. In particular, we observe that correlations among inputs induce effective dependencies among parameters thus constraining the model and, consequently, reducing its complexity. We derive simple relations for the upper and lower bounds of the complexity. Furthermore, we show analytically that, defining the model parameters on a finite support rather than the entire axis, decreases the complexity in a manner that critically depends on the size of the domain. Based on our findings, we propose a novel model selection criterion which takes into account the entropy of the input distribution. We test our proposal on the problem of selecting the input variables of a logistic regression model in a Bayesian Model Selection framework. In our numerical tests, we find that, while the reconstruction errors of standard model selection approaches (AIC, BIC, $\ell_1$ regularization) strongly depend on the sparsity of the ground truth, the reconstruction error of our method is always close to the minimum in all conditions of sparsity, data size and strength of input correlations. Finally, we observe that, when considering categorical instead of binary inputs, in a simple and mathematically tractable case, the contribution of the alphabet size to the complexity is very small compared to that of parameter space dimension. We further explore the issue by analysing the dataset of the "13 keys to the White House" which is a method for forecasting the outcomes of US presidential elections.

stat.ML↗

arXiv2019

Mean-field Analysis of Batch Normalization

Batch Normalization (BatchNorm) is an extremely useful component of modern neural network architectures, enabling optimization using higher learning rates and achieving faster convergence. In this paper, we use mean-field theory to analytically quantify the impact of BatchNorm on the geometry of the loss landscape for multi-layer networks consisting of fully-connected and convolutional layers. We show that it has a flattening effect on the loss landscape, as quantified by the maximum eigenvalue of the Fisher Information Matrix. These findings are then used to justify the use of larger learning rates for networks that use BatchNorm, and we provide quantitative characterization of the maximal allowable learning rate to ensure convergence. Experiments support our theoretically predicted maximum learning rate, and furthermore suggest that networks with smaller values of the BatchNorm parameter achieve lower loss after the same number of epochs of training.

cs.LG↗

arXiv2019

Topologically Protected Quantum Entanglement

Quantum entanglement, as the strictly non-classical phenomena, is the kernel of quantum computing and quantum simulation, and has been widely applied ranging from fundamental tests of quantum physics to quantum information processing. The decoherence of quantum states restricts the capability of building quantum simulators and quantum computers in a scalable fashion. Meanwhile, the topological phase is found inherently capable of protecting physical fields from unavoidable fabrication-induced disorder, which inspires the potential application of topological protection on quantum states. Here, we present the first experimental demonstration of topologically protected quantum polarization entangled states on a photonic chip. The process tomography shows that quantum entanglement can be well preserved by the boundary states even when the chip material substantially introduces relative polarization rotation in phase space. Our work links topology, material and quantum physics, opening the door to wide applications of topological enhancement in genuine quantum regime.

quant-ph↗

arXiv2019

Dynamical quantum phase transitions in many-body localized systems

We investigate dynamical quantum phase transitions in disordered quantum many-body models that can support many-body localized phases. Employing $l$-bits formalism, we lay out the conditions for which singularities indicative of the transitions appear in the context of many-body localization. Using the combination of the mapping onto $l$-bits and exact diagonalization results, we explicitly demonstrate the presence of these singularities for a candidate model that features many-body localization. Our work paves the way for understanding dynamical quantum phase transitions in the context of many-body localization, and elucidating whether different phases of the latter can be detected from analyzing the former. The results presented are experimentally accessible with state-of-the-art ultracold-atom and ion-trap setups.

cond-mat.stat-mech↗

arXiv2019

On-line learning dynamics of ReLU neural networks using statistical physics techniques

We introduce exact macroscopic on-line learning dynamics of two-layer neural networks with ReLU units in the form of a system of differential equations, using techniques borrowed from statistical physics. For the first experiments, numerical solutions reveal similar behavior compared to sigmoidal activation researched in earlier work. In these experiments the theoretical results show good correspondence with simulations. In ove-rrealizable and unrealizable learning scenarios, the learning behavior of ReLU networks shows distinctive characteristics compared to sigmoidal networks.

cs.LG↗

arXiv2019

Do tensor renormalization group methods work for frustrated spin systems?

No. To illustrate that tensor renormalization group methods are poorly suited for frustrated magnetic systems, we study the thermodynamic properties of the two-dimensional Edwards-Anderson Ising spin-glass model on a square lattice. We show that the limited precision of standard 64-bit data types and not a small cut-off parameter is the main reason for unphysical negative partition function values in spin glasses.

cond-mat.dis-nn↗

arXiv2019

Fast learning synapses with molecular spin valves via selective magnetic potentiation

We studied LSMO/Alq3/AlOx/Co molecular spin valves in view of their use as synapses in neuromorphic computing. In neuromorphic computing, the learning ability is embodied in specific changes of the synaptic weight. In this perspective, the relevant parameter is the conductance of the molecular spin valve, which plays the role of the synaptic weight. In this work we demonstrated that the conductance can be changes by the repeated application of voltage pulses. We studied the parameter space of the pulses in order to determine the most effective voltage and duration of the pulses. The conductance could also be modified by aligning the magnetizations of the ferromagnetic electrodes parallel or anti parallel to each other. This phenomenon, known as magnetoresistance, affects high conductance devices while leaving low conductance devices unaffected. We studied how this weight update rule affected the speed of reward-based learning in an actor-critic framework, compared to a linear update rule. This nonlinear update performed significantly better (50 learning trials; Epochs to reach a performance goal of 0.975 was 896+/-301 in the nonlinear case and 1076+/-484 in the nonlinear case; Welch t-test: p<0.05). The linear update resulted in more learning trails with very long convergence times, which was largely absent in the nonlinear update.

cs.ET↗

arXiv2019

Spike-based primitives for graph algorithms

In this paper we consider graph algorithms and graphical analysis as a new application for neuromorphic computing platforms. We demonstrate how the nonlinear dynamics of spiking neurons can be used to implement low-level graph operations. Our results are hardware agnostic, and we present multiple versions of routines that can utilize static synapses or require synapse plasticity.

cs.NE↗

arXiv2019

Avalanches in an excitable network

We study propagation of avalanches in a certain excitable network. The model is a particular case of the one introduced in [23], and is mathematically equivalent to an endemic variation of the Reed-Frost epidemic model introduced in [27]. Two types of heuristic approximation are frequently used for models of this type in applications, a branching process for avalanches of a small size at the beginning of the process and a deterministic dynamical system once the avalanche spreads to a significant fraction of a large network. In this paper we prove several results concerning the exact relation between the avalanche model and these limits, including rates of convergence and rigorous bounds for common characteristics of the model.

math.PR↗

arXiv2019

Large deviations for the largest eigenvalues and eigenvectors of spiked random matrices

We consider matrices formed by a random $N\times N$ matrix drawn from the Gaussian Orthogonal Ensemble (or Gaussian Unitary Ensemble) plus a rank-one perturbation of strength $θ$, and focus on the largest eigenvalue, $x$, and the component, $u$, of the corresponding eigenvector in the direction associated to the rank-one perturbation. We obtain the large deviation principle governing the atypical joint fluctuations of $x$ and $u$. Interestingly, for $θ>1$, in large deviations characterized by a small value of $u$, i.e. $u<1-1/θ$, the second-largest eigenvalue pops out from the Wigner semi-circle and the associated eigenvector orients in the direction corresponding to the rank-one perturbation. We generalize these results to the Wishart Ensemble, and we extend them to the first $n$ eigenvalues and the associated eigenvectors.

math.PR↗

arXiv2019

Functional Geometry of Human Connectome and Robustness of Gender Differences

Mapping the brain imaging data to networks, where each node represents a specific area of the brain, has enabled an objective graph-theoretic analysis of human connectome. However, the latent structure on higher-order connections remains unexplored, where many brain regions acting in synergy perform complex functions. Here we analyse this hidden structure using the simplicial complexes parametrisation where the shared faces of simplexes encode higher-order relationships between groups of nodes and emerging hyperbolic geometry. Based on data collected within the Human Connectome Project, we perform a systematic analysis of consensus networks of 100 female (F-connectome) and 100 male (M-connectome) subjects by varying the number of fibres launched. Our analysis reveals that the functional geometry of the common F\&M-connectome coincides with the M-connectome and is characterized by a complex architecture of simplexes to the 14th order, which is built in six anatomical communities, and short cycles among them. Furthermore, the F-connectome has additional connections that involve different brain regions, thereby increasing the size of simplexes and introducing new cycles. By providing new insights into the internal organisation of anatomical brain modules as well as into the links between them that are essential to dynamics, these results also highlight the functional gender-related differences

q-bio.NC↗

arXiv2019

Bayesian Neural Networks at Finite Temperature

We recapitulate the Bayesian formulation of neural network based classifiers and show that, while sampling from the posterior does indeed lead to better generalisation than is obtained by standard optimisation of the cost function, even better performance can in general be achieved by sampling finite temperature ($T$) distributions derived from the posterior. Taking the example of two different deep (3 hidden layers) classifiers for MNIST data, we find quite different $T$ values to be appropriate in each case. In particular, for a typical neural network classifier a clear minimum of the test error is observed at $T>0$. This suggests an early stopping criterion for full batch simulated annealing: cool until the average validation error starts to increase, then revert to the parameters with the lowest validation error. As $T$ is increased classifiers transition from accurate classifiers to classifiers that have higher training error than assigning equal probability to each class. Efficient studies of these temperature-induced effects are enabled using a replica-exchange Hamiltonian Monte Carlo simulation technique. Finally, we show how thermodynamic integration can be used to perform model selection for deep neural networks. Similar to the Laplace approximation, this approach assumes that the posterior is dominated by a single mode. Crucially, however, no assumption is made about the shape of that mode and it is not required to precisely compute and invert the Hessian.

stat.ML↗