Searcharxiv⌕ Search

SEARCH · Searcharxiv

Results for “cond-mat.dis-nn”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33Linked to original sources
arXiv2019

Predicting synchronous firing of large neural populations from sequential recordings

A major goal in neuroscience is to understand how populations of neurons code for stimuli or actions. While the number of neurons that can be recorded simultaneously is increasing at a fast pace, in most cases these recordings cannot access a complete population. In particular, it is hard to simultaneously record all the neurons of the same type in a given area. Recent progress have made possible to profile each recorded neuron in a given area thanks to genetic and physiological tools, and to pool together recordings from neurons of the same type across different experimental sessions. However, it is unclear how to infer the activity of a full population of neurons of the same type from these sequential recordings. Neural networks exhibit collective behaviour, e.g. noise correlations and synchronous activity, that are not directly captured by a conditionally-independent model that would just put together the spike trains from sequential recordings. Here we show that we can infer the activity of a full population of retina ganglion cells from sequential recordings, using a novel method based on copula distributions and maximum entropy modeling. From just the spiking response of each ganglion cell to a repeated stimulus, and a few pairwise recordings, we could predict the noise correlations using copulas, and then the full activity of a large population of ganglion cells of the same type using maximum entropy modeling. Remarkably, we could generalize to predict the population responses to different stimuli and even to different experiments. We could therefore use our method to construct a very large population merging cells' responses from different experiments. We predicted synchronous activity accurately and showed it grew substantially with the number of neurons. This approach is a promising way to infer population activity from sequential recordings in sensory areas.

q-bio.NC↗
arXiv2019

Hamilton-Jacobi equations for finite-rank matrix inference

We compute the large-scale limit of the free energy associated with the problem of inference of a finite-rank matrix. The method follows the principle put forward in arXiv:1811.01432 which consists in identifying a suitable Hamilton-Jacobi equation satisfied by the limit free energy. We simplify the approach of arXiv:1811.01432 using a notion of weak solution of the Hamilton-Jacobi equation which is more convenient to work with and is applicable whenever the non-linearity in the equation is convex.

math.PR↗
arXiv2019

Room temperature spin-ice physics in cadmium cyanide

Spin-ices are frustrated magnets that support a particularly rich variety of emergent physics. Typically, it is the interplay of magnetic dipole interactions, spin anisotropy, and geometric frustration on the pyrochlore lattice that drives spin-ice formation. The relevant physics occurs at temperatures commensurate with the magnetic interaction strength, which for most systems is 1--5\,K. This low energy scale poses severe challenges for experimental studies of spin-ices and the practical exploitation of their unusual properties. Here, we show that non-magnetic cadmium cyanide (Cd(CN)$_2$) exhibits analogous behaviour to magnetic spin-ices, but does so on a temperature scale that is nearly two orders of magnitude greater. The electric dipole moments of cyanide ions in Cd(CN)$_2$ assume the role of magnetic pseudospins, with the difference in energy scale reflecting the increased strength of electric \emph{vs} magnetic dipolar interactions. As a result, spin-ice physics influences the structural behaviour of Cd(CN)$_2$ even at room temperature.

cond-mat.mtrl-sci↗
arXiv2019

High Entropy Alloys Mined From Phase Diagrams

High entropy alloys (HEA) show promise as a new type of high-performance structural material. Their vast degrees of freedom provide for extensive opportunities to design alloys with tailored properties. However, the compositional complexities of HEAs present great challenges for alloy design. Current approaches have shown limited reliability in accounting for the compositional regions of single solid solution and composite phases. We present a phenomenological method, analyzing binary phase diagrams to predict HEA phase formation on the hypothesis that the structural stability of HEAs is encoded within. Accordingly, we introduce a small number of phase-diagram inspired parameters and employ machine learning to partition the formation region of 500+ reported HEA compositions. The model achieved a single phase HEA prediction rate >80 %. To validate our method, we demonstrated the capability of this method in predicting HEA solid solution phases with and without intermetallics in 30 randomly selected complex compositions, with a success rate of 77 %. The presented efficient search approach with high predictive capability can be exploited to complement computation-intense methods in providing a powerful platform for the design of high entropy alloys.

cond-mat.dis-nn↗
arXiv2019

Breaking of 1RSB in random MAX-NAE-SAT

For several models of random constraint satisfaction problems, it was conjectured by physicists and later proved that a sharp satisfiability transition occurs. For random $k$-SAT and related models it happens at clause density $α$ around $2^k$. Just below the threshold, further results suggest that the solution space has a "1RSB" structure of a large bounded number of near-orthogonal clusters inside the space of variable assignments $\{0,1\}^N$. In the unsatisfiable regime, it is natural to consider max-satisfiability: violating the least number of constraints. For a simplified variant, the strong refutation problem, there is strong evidence that an algorithmic transition occurs around $α= N^{k/2-1}$. For $α$ bounded in $N$, a very precise estimate of the max-sat value was obtained by Achlioptas, Naor, and Peres (2007), but it is not sharp enough to indicate the nature of the energy landscape. Later work (Sen, 2016; Panchenko, 2016) shows that for $α$ very large (roughly, above $64^k$) the max-sat value approaches the mean-field (complete graph) limit: this is conjectured to have an "FRSB" structure where near-optimal configurations form clusters within clusters, in an ultrametric hierarchy of infinite depth inside $\{0,1\}^N$. A stronger form of FRSB was shown in several recent works to have algorithmic implications (again, in complete graphs). Consequently we find it of interest to understand how the model transitions from 1RSB near the satisfiability threshold, to (conjecturally) FRSB for large $α$. In this paper we show that in the random regular $k$-NAE-SAT model, the 1RSB description breaks down already above $α\asymp 4^k/k^3$. This is proved by an explicit perturbation in the 2RSB parameter space, inspired by the "bug proliferation" mechanism proposed by physicists (Montanari and Ricci-Tersenghi, 2003; Krzakala, Pagnani, and Weigt, 2004).

math.PR↗
arXiv2019

On the Structure of Liquids: More order than expected

Disordered systems like liquids, gels, glasses, or granular materials are not only ubiquitous in daily life and in industrial applications but they are also crucial for the mechanical stability of cells or the transport of chemical and biological agents in living organisms [1-3]. Despite the importance of these systems, their microscopic structure is understood only on a rudimentary level, thus in stark contrast to the case of gases and crystals [1,4]. Experimental and theoretical investigations indicate that disordered systems have a structural order on the length scale of a few particle diameters but which then quickly vanishes at larger distances [5]. This conclusion is, however, based mainly on the behavior of two-point correlation functions such as the static structure factor or the radial distribution function [5.6]. Whether or not disordered systems have an order that extents to larger length scales is therefore a highly important question that is still open [7-10]. Here we use computer simulations to show that liquids have an intricate structural order given by alternating layers with icosahedral and dodecahedral symmetries and which extends to surprisingly large distances. We show that the temperature dependence of the corresponding length scale can be detected in the static structure factor, making it directly accessible to scattering experiments. These results, obtained by a novel type of four point correlation function that probes directly the particle density in three dimensions, show that liquids are far more ordered than envisaged so far but that this structural information is encoded in non-standard correlation functions.

cond-mat.dis-nn↗
arXiv2019

Winner-Take-All Computation in Spiking Neural Networks

In this work we study biological neural networks from an algorithmic perspective, focusing on understanding tradeoffs between computation time and network complexity. Our goal is to abstract real neural networks in a way that, while not capturing all interesting features, preserves high-level behavior and allows us to make biologically relevant conclusions. Towards this goal, we consider the implementation of algorithmic primitives in a simple yet biologically plausible model of $stochastic\ spiking\ neural\ networks$. In particular, we show how the stochastic behavior of neurons in this model can be leveraged to solve a basic $symmetry-breaking\ task$ in which we are given neurons with identical firing rates and want to select a distinguished one. In computational neuroscience, this is known as the winner-take-all (WTA) problem, and it is believed to serve as a basic building block in many tasks, e.g., learning, pattern recognition, and clustering. We provide efficient constructions of WTA circuits in our stochastic spiking neural network model, as well as lower bounds in terms of the number of auxiliary neurons required to drive convergence to WTA in a given number of steps. These lower bounds demonstrate that our constructions are near-optimal in some cases. This work covers and gives more in-depth proofs of a subset of results originally published in [LMP17a]. It is adapted from the last chapter of C. Musco's Ph.D. thesis [Mus18].

cs.DC↗
arXiv2019

Long-wavelength fluctuations and static correlations in quasi-2D colloidal suspensions

Dimensionality strongly affects thermal fluctuations and critical dynamics of equilibrium systems. These influences persist in amorphous systems going through the nonequilibrium glass transition. Here, we experimentally study the glass transition of quasi-2D suspensions of spherical and ellipsoidal particles under different degrees of circular confinement. We show that the strength of the long-wavelength fluctuations increases logarithmically with system sizes and displays the signature of the Mermin-Wagner fluctuations. Moreover, using confinement as a tool, we also measure static structural correlations and extract a growing static correlation length in 2D supercooled liquids. Finally, we explore the influence of the Mermin-Wagner fluctuations on the translational and orientational relaxations of 2D ellipsoidal suspensions, which leads to a new interpretation of the two-step glass transition and the orientational glass phase of anisotropic particles. Our study reveals the importance of long-wavelength fluctuations in 2D supercooled liquids and provides new insights into the role of dimensionality in the glass transition.

cond-mat.soft↗
arXiv2019

Physicist's Journeys Through the AI World - A Topical Review. There is no royal road to unsupervised learning

Artificial Intelligence (AI), defined in its most simple form, is a technological tool that makes machines intelligent. Since learning is at the core of intelligence, machine learning poses itself as a core sub-field of AI. Then there comes a subclass of machine learning, known as deep learning, to address the limitations of their predecessors. AI has generally acquired its prominence over the past few years due to its considerable progress in various fields. AI has vastly invaded the realm of research. This has led physicists to attentively direct their research towards implementing AI tools. Their central aim has been to gain better understanding and enrich their intuition. This review article is meant to supplement the previously presented efforts to bridge the gap between AI and physics, and take a serious step forward to filter out the "Babelian" clashes brought about from such gabs. This necessitates first to have fundamental knowledge about common AI tools. To this end, the review's primary focus shall be on deep learning models called artificial neural networks. They are deep learning models which train themselves through different learning processes. It discusses also the concept of Markov decision processes. Finally, shortcut to the main goal, the review thoroughly examines how these neural networks are capable to construct a physical theory describing some observations without applying any previous physical knowledge.

cs.LG↗
arXiv2019

Finite-size effects in exponential random graphs and cluster evaporation

In this Letter we find numerically the strong finite-size effects in the critical behavior of Erdős-Rényi (ER) networks supplemented with chemical potentials for some motifs, in particular 2-stars and triangles. For the 2-star model above the critical value of the chemical potential a ground state looks as star-like graph with the finite set of hubs at ER parameter $p<0.5$ or as the single cluster at $p>0.5$. It is found that there exists the critical value of number of nodes $N^{*}(p)$ when the ground state undergoes clear-cut crossover and at $N>N^{*}(p)$ the network flows via a cluster evaporation to the state involving the small star in the ER environment. The similar evaporation of the cluster takes place at $N>N^{*}(p)$ in the Strauss model. We suggest that the entropic trap mechanism is relevant for microscopic mechanism behind the crossover regime. The possible analogies concerning the strong entropic finite-size effects in the holographic description of matrix black hole (BH) formation and evaporation are mentioned.

cond-mat.dis-nn↗
arXiv2019

Mutual Information Scaling and Expressive Power of Sequence Models

Sequence models assign probabilities to variable-length sequences such as natural language texts. The ability of sequence models to capture temporal dependence can be characterized by the temporal scaling of correlation and mutual information. In this paper, we study the mutual information of recurrent neural networks (RNNs) including long short-term memories and self-attention networks such as Transformers. Through a combination of theoretical study of linear RNNs and empirical study of nonlinear RNNs, we find their mutual information decays exponentially in temporal distance. On the other hand, Transformers can capture long-range mutual information more efficiently, making them preferable in modeling sequences with slow power-law mutual information, such as natural languages and stock prices. We discuss the connection of these results with statistical mechanics. We also point out the non-uniformity problem in many natural language datasets. We hope this work provides a new perspective in understanding the expressive power of sequence models and shed new light on improving the architecture of them.

cs.LG↗
arXiv2019

Theoretical model for the structural relaxation time in co-amorphous drugs

We propose a simple approach to investigate the structural relaxation time and glass transition of amorphous drugs. Amorphous materials are modeled as a set of equal sized hard spheres. The structural relaxation time over many decades in hard sphere fluids is theoretically calculated using the Elastically Collective Nonlinear Langevin Equation theory associated with Kramer's theory. Then, a new thermal mapping from a real material to an effective hard sphere fluid provides temperature-dependent relaxation time, which can compare to experiments. Numerical results quantitatively agree with previous experiments for pharmaceutical binary mixtures having different weight ratios. We carry out experiments to test our calculations for an ezetimibe-simvastatin-Kollidon VA64 mixture. Our approach would provide a simple but comprehensive description of glassy dynamics in amorphous composites.

cond-mat.soft↗
arXiv2019

On the visible continuum and lines in the Interstellar Extinction Curve

This work purports to help understand the InterStellar Extinction Curve in and near the visible range. In this range, crystalline materials are known to be transparent, so amorphous dust is needed. Molecular modeling experiments are used to compute the electronic spectra of various, relatively large, carbon and silicate structures. Hardly any transition shows up beyond 0.4 mum when the structure is in its ground state (the lowest, most stable state, usually crystalline). This is no longer the case as soon as the structure is distorted in any way. Examples of simulated distortions (or ``defects'') are: angular or linear bond alteration, insertion of free radicals near the main structure, dangling bonds; their cumulative effects lead to the amorphous state. It is shown that, in this state, a structure bears a majority of weak transitions and a minority of strong ones. As the structure grows in size, the former ultimately form a weak continuum already detected experimentally, in the visible, on amorphous carbons and silicates. The stronger transitions will manage to emerge above the continuum, especially when they bunch together by chance near the same wavelength. Parallels are drawn between several properties of the computed continua and transitions and the observed continuum and Diffuse Interstellar Bands.

astro-ph.GA↗
arXiv2019

Persistent Threshold Dynamics with Recovery in Complex Networks

Threshold rules of spreading in binary-state networks lead to cascades. We study persistent cascade-recovery dynamics on quasi-robust networks, i.e., networks which are robust against small trigger but may collapse for larger one. It is observed that depending on the relative rate of triggering and recovery, the network falls into one of the two dynamical phases: collapsing or active phase. We devise an analytical framework which characterizes not only the critical behavior but also the temporal evolution of network activity in both phases. Agent-based simulation results show good agreement with the analytical calculations, indicating strong predicative power of our method for persistent cascade dynamics in complex networks.

physics.soc-ph↗
arXiv2019

Replicated Vector Approximate Message Passing For Resampling Problem

Resampling techniques are widely used in statistical inference and ensemble learning, in which estimators' statistical properties are essential. However, existing methods are computationally demanding, because repetitions of estimation/learning via numerical optimization/integral for each resampled data are required. In this study, we introduce a computationally efficient method to resolve such problem: replicated vector approximate message passing. This is based on a combination of the replica method of statistical physics and an accurate approximate inference algorithm, namely the vector approximate message passing of information theory. The method provides tractable densities without repeating estimation/learning, and the densities approximately offer an arbitrary degree of the estimators' moment in practical time. In the experiment, we apply the proposed method to the stability selection method, which is commonly used in variable selection problems. The numerical results show its fast convergence and high approximation accuracy for problems involving both synthetic and real-world datasets.

stat.ML↗
arXiv2019

Controlling a complex system near its critical point via temporal correlations

A wide variety of complex systems exhibit large fluctuations both in space and time that often can be attributed to the presence of some kind of critical phenomena. Under such critical scenario it is well known that the properties of the correlation functions in space and time are two sides of the same coin. Here we test wether systems exhibiting a phase transition could self-tune to its critical point taking advantage of such correlation properties. We describe results in three models: the 2D Ising ferromagnetic model, the 3D Vicsek flocking model and a small-world neuronal network model. We illustrate how the feedback of the autocorrelation function of the order parameter fluctuations is able to shift the system towards its critical point. Since the results rely on universal properties they are expected to be relevant to a variety of other settings.

cond-mat.dis-nn↗
arXiv2019

A mean-field limit for certain deep neural networks

Understanding deep neural networks (DNNs) is a key challenge in the theory of machine learning, with potential applications to the many fields where DNNs have been successfully used. This article presents a scaling limit for a DNN being trained by stochastic gradient descent. Our networks have a fixed (but arbitrary) number $L\geq 2$ of inner layers; $N\gg 1$ neurons per layer; full connections between layers; and fixed weights (or "random features" that are not trained) near the input and output. Our results describe the evolution of the DNN during training in the limit when $N\to +\infty$, which we relate to a mean field model of McKean-Vlasov type. Specifically, we show that network weights are approximated by certain "ideal particles" whose distribution and dependencies are described by the mean-field model. A key part of the proof is to show existence and uniqueness for our McKean-Vlasov problem, which does not seem to be amenable to existing theory. Our paper extends previous work on the $L=1$ case by Mei, Montanari and Nguyen; Rotskoff and Vanden-Eijnden; and Sirignano and Spiliopoulos. We also complement recent independent work on $L>1$ by Sirignano and Spiliopoulos (who consider a less natural scaling limit) and Nguyen (who nonrigorously derives similar results).

math.ST↗