arXiv subjects

Giulio Biroli

Publications and source records attributed to Giulio Biroli.

At least 109 records · Page 6Linked to original sources

arXiv2019

Classical Glasses, Black Holes, and Strange Quantum Liquids

From the dynamics of a broad class of classical mean-field glass models one may obtain a quantum model with finite zero-temperature entropy, a quantum transition at zero temperature, and a time-reparametrization (quasi-)invariance in the dynamical equations for correlations. The low eigenvalue spectrum of the resulting quantum model is directly related to the structure and exploration of metastable states in the landscape of the original classical glass model. This mapping reveals deep connections between classical glasses and the properties of SYK-like models.

hep-th↗

arXiv2019

Scaling description of generalization with number of parameters in deep learning

Supervised deep learning involves the training of neural networks with a large number $N$ of parameters. For large enough $N$, in the so-called over-parametrized regime, one can essentially fit the training data points. Sparsity-based arguments would suggest that the generalization error increases as $N$ grows past a certain threshold $N^{*}$. Instead, empirical studies have shown that in the over-parametrized regime, generalization error keeps decreasing with $N$. We resolve this paradox through a new framework. We rely on the so-called Neural Tangent Kernel, which connects large neural nets to kernel methods, to show that the initialization causes finite-size random fluctuations $\|f_{N}-\bar{f}_{N}\|\sim N^{-1/4}$ of the neural net output function $f_{N}$ around its expectation $\bar{f}_{N}$. These affect the generalization error $ε_{N}$ for classification: under natural assumptions, it decays to a plateau value $ε_{\infty}$ in a power-law fashion $\sim N^{-1/2}$. This description breaks down at a so-called jamming transition $N=N^{*}$. At this threshold, we argue that $\|f_{N}\|$ diverges. This result leads to a plausible explanation for the cusp in test error known to occur at $N^{*}$. Our results are confirmed by extensive empirical observations on the MNIST and CIFAR image datasets. Our analysis finally suggests that, given a computational envelope, the smallest generalization error is obtained using several networks of intermediate sizes, just beyond $N^{*}$, and averaging their outputs.

cond-mat.dis-nn↗

arXiv2019

Can endogenous fluctuations persist in high-diversity ecosystems?

When can complex ecological interactions drive an entire ecosystem into a persistent non-equilibrium state, where species abundances keep fluctuating without going to extinction? We show that high-diversity spatially-extended systems, in which conditions vary somewhat between spatial locations, can exhibit chaotic dynamics which persist for extremely long times. We develop a theoretical framework, based on dynamical mean-field theory, to quantify the conditions under which these fluctuating states exist, and predict their properties. We uncover parallels with the persistence of externally-perturbed ecosystems, such as the role of perturbation strength, synchrony and correlation time. But uniquely to endogenous fluctuations, these properties arise from the species dynamics themselves, creating feedback loops between perturbation and response. A key result is that the fluctuation amplitude and species diversity are tightly linked, in particular fluctuations enable dramatically more species to coexist than at equilibrium in the very same system. Our findings highlight crucial differences between well-mixed and spatially-extended systems, with implications for experiments and their ability to reproduce natural dynamics. They shed light on the maintenance of biodiversity, and the strength and synchrony of fluctuations observed in natural systems.

q-bio.PE↗

arXiv2019

A jamming transition from under- to over-parametrization affects loss landscape and generalization

We argue that in fully-connected networks a phase transition delimits the over- and under-parametrized regimes where fitting can or cannot be achieved. Under some general conditions, we show that this transition is sharp for the hinge loss. In the whole over-parametrized regime, poor minima of the loss are not encountered during training since the number of constraints to satisfy is too small to hamper minimization. Our findings support a link between this transition and the generalization properties of the network: as we increase the number of parameters of a given model, starting from an under-parametrized network, we observe that the generalization error displays three phases: (i) initial decay, (ii) increase until the transition point --- where it displays a cusp --- and (iii) slow decay toward a constant for the rest of the over-parametrized regime. Thereby we identify the region where the classical phenomenon of over-fitting takes place, and the region where the model keeps improving, in line with previous empirical observations for modern neural networks.

cs.LG↗

arXiv2019

The jamming transition as a paradigm to understand the loss landscape of deep neural networks

Deep learning has been immensely successful at a variety of tasks, ranging from classification to AI. Learning corresponds to fitting training data, which is implemented by descending a very high-dimensional loss function. Understanding under which conditions neural networks do not get stuck in poor minima of the loss, and how the landscape of that loss evolves as depth is increased remains a challenge. Here we predict, and test empirically, an analogy between this landscape and the energy landscape of repulsive ellipses. We argue that in FC networks a phase transition delimits the over- and under-parametrized regimes where fitting can or cannot be achieved. In the vicinity of this transition, properties of the curvature of the minima of the loss are critical. This transition shares direct similarities with the jamming transition by which particles form a disordered solid as the density is increased, which also occurs in certain classes of computational optimization and learning problems such as the perceptron. Our analysis gives a simple explanation as to why poor minima of the loss cannot be encountered in the overparametrized regime, and puts forward the surprising result that the ability of fully connected networks to fit random data is independent of their depth. Our observations suggests that this independence also holds for real data. We also study a quantity $Δ$ which characterizes how well ($Δ<0$) or badly ($Δ>0$) a datum is learned. At the critical point it is power-law distributed, $P_+(Δ)\simΔ^θ$ for $Δ>0$ and $P_-(Δ)\sim(-Δ)^{-γ}$ for $Δ<0$, with $θ\approx0.3$ and $γ\approx0.2$. This observation suggests that near the transition the loss landscape has a hierarchical structure and that the learning dynamics is prone to avalanche-like dynamics, with abrupt changes in the set of patterns that are learned.

cond-mat.dis-nn↗

arXiv2019

Maximum-energy records in glassy energy landscapes

We study the evolution of the maximum energy $E_\max(t)$ reached between time $0$ and time $t$ in the dynamics of simple models with glassy energy landscapes, in instant quenches from infinite temperature to a target temperature $T$. Through a detailed description of the activated dynamics, we are able to describe the evolution of $E_\max(t)$ from short times, through the aging regime, until after equilibrium is reached, thus providing a detailed description of the long-time dynamics. Finally, we compare our findings with numerical simulations of the $p$-spin glass and show how the maximum energy record can be used to identify the threshold energy in this model.

cond-mat.dis-nn↗

arXiv2019

Perspective: Gardner Physics in Amorphous Solids and Beyond

One of the most remarkable predictions to emerge out of the exact infinite-dimensional solution of the glass problem is the Gardner transition. Although this transition was first theoretically proposed a generation ago for certain mean-field spin glass models, its materials relevance was only realized when a systematic effort to relate glass formation and jamming was undertaken. A number of nontrivial physical signatures associated to the Gardner transition have since been considered in various areas, from models of structural glasses to constraint satisfaction problems. This Perspective surveys these recent advances and discusses the novel research opportunities that arise from them.

cond-mat.stat-mech↗

arXiv2019

Large deviations for the largest eigenvalues and eigenvectors of spiked random matrices

We consider matrices formed by a random $N\times N$ matrix drawn from the Gaussian Orthogonal Ensemble (or Gaussian Unitary Ensemble) plus a rank-one perturbation of strength $θ$, and focus on the largest eigenvalue, $x$, and the component, $u$, of the corresponding eigenvector in the direction associated to the rank-one perturbation. We obtain the large deviation principle governing the atypical joint fluctuations of $x$ and $u$. Interestingly, for $θ>1$, in large deviations characterized by a small value of $u$, i.e. $u<1-1/θ$, the second-largest eigenvalue pops out from the Wigner semi-circle and the associated eigenvector orients in the direction corresponding to the rank-one perturbation. We generalize these results to the Wishart Ensemble, and we extend them to the first $n$ eigenvalues and the associated eigenvectors.

math.PR↗

arXiv2019

Numerical implementation of dynamical mean field theory for disordered systems: application to the Lotka-Volterra model of ecosystems

Dynamical mean field theory (DMFT) is a tool that allows to analyze the stochastic dynamics of $N$ interacting degrees of freedom in terms of a self-consistent $1$-body problem. In this work, focusing on models of ecosystems, we present the derivation of DMFT through the dynamical cavity method, and we develop a method for solving it numerically. Our numerical procedure can be applied to a large variety of systems for which DMFT holds. We implement and test it for the generalized random Lotka-Volterra model, and show that complex dynamical regimes characterized by chaos and aging can be captured and studied by this framework.

cond-mat.dis-nn↗

arXiv2018

Out-of-equilibrium dynamical mean-field equations for the perceptron model

Perceptrons are the building blocks of many theoretical approaches to a wide range of complex systems, ranging from neural networks and deep learning machines, to constraint satisfaction problems, glasses and ecosystems. Despite their applicability and importance, a detailed study of their Langevin dynamics has never been performed yet. Here we derive the mean-field dynamical equations that describe the continuous random perceptron in the thermodynamic limit, in a very general setting with arbitrary noise and friction kernels, not necessarily related by equilibrium relations. We derive the equations in two ways: via a dynamical cavity method, and via a path-integral approach in its supersymmetric formulation. The end point of both approaches is the reduction of the dynamics of the system to an effective stochastic process for a representative dynamical variable. Because the perceptron is formally very close to a system of interacting particles in a high dimensional space, the methods we develop here can be transferred to the study of liquid and glasses in high dimensions. Potentially interesting applications are thus the study of the glass transition in active matter, the study of the dynamics around the jamming transition, and the calculation of rheological properties in driven systems.

cond-mat.dis-nn↗

arXiv2018

Delocalization and ergodicity of the Anderson model on Bethe lattices

We review the state of the art on the delocalized non-ergodic regime of the Anderson model on Bethe lattices. We also present new results using Belief Propagation, which consists in solving the self-consistent recursion relations for the Green's functions directly on a given sample. This allows us to numerically study very large system sizes and to directly access observables related to the eigenfunctions and energy level statistics. In agreement with recent works, we establish the existence of a delocalized non-ergodic phase on Cayley trees. On random regular graphs instead our results indicate that ergodicity is recovered when the system size is larger than a cross-over scale $N_c (W)$, which diverges exponentially fast approaching the localization transition. This scale corresponds to the size at which the mean-level spacing becomes smaller than the Thouless energy $E_{Th} (W)$. Such energy scale, which vanishes exponentially fast approaching the localization transition, is the one below which ergodicity in the level statistics is restored in the thermodynamic limit. Remarkably, the behavior of random regular graphs below $N_c (W)$ coincides with the one found close to the root of loop-less infinite Cayley trees, {\it i.e.} only above $N_c (W)$ the effects of loops emerge and random regular graphs behave differently from Cayley trees. Our results indicate that ergodicity is recovered in the thermodynamic limit on random regular graph. However, all observables probing volumes smaller than $N_c(W)$ and times smaller than $\hbar/E_{Th} (W)$ are expected to behave as if there were an intermediate phase. Given the very fast divergence of $N_c(W)$ and $\hbar/E_{Th} (W)$ these non-ergodic effects are very pronounced in a large region preceding the localization transition, and they can be related to the intermediate phase present on Cayley trees.

cond-mat.dis-nn↗

arXiv2018

Dynamics around the Site Percolation Threshold on High-Dimensional Hypercubic Lattices

Recent advances on the glass problem motivate reexamining classical models of percolation. Here, we consider the displacement of an ant in a labyrinth near the percolation threshold on cubic lattices both below and above the upper critical dimension of simple percolation, d_u=6. Using theory and simulations, we consider the scaling regime part, and obtain that both caging and subdiffusion scale logarithmically for d >= d_u. The theoretical derivation considers Bethe lattices with generalized connectivity and a random graph model, and employs a scaling analysis to confirm that logarithmic scalings should persist in the infinite dimension limit. The computational validation employs accelerated random walk simulations with a transfer-matrix description of diffusion to evaluate directly the dynamical critical exponents below d_u as well as their logarithmic scaling above d_u. Our numerical results improve various earlier estimates and are fully consistent with our theoretical predictions.

cond-mat.stat-mech↗

arXiv2018

Complexity of energy barriers in mean-field glassy systems

We analyze the energy barriers that allow escapes from a given local minimum in a mean-field model of glasses. We perform this study by using the Kac-Rice method and computing the typical number of critical points of the energy function at a given distance from the minimum. We analyze their Hessian in terms of random matrix theory and show that for a certain regime of energies and distances critical points are index-one saddles and are associated to barriers. We find that the lowest barrier, important for activated dynamics at low temperature, is strictly lower than the "threshold" level above which saddles proliferate. We characterize how the quenched complexity of barriers, important for activated process at finite temperature, depends on the energy of the barrier, the energy of the initial minimum, and the distance between them. The overall picture gained from this study is expected to hold generically for mean-field models of the glass transition.

cond-mat.dis-nn↗

arXiv2018

Can the glass transition be explained without a growing static length scale?

It was recently discovered that SWAP, a Monte Carlo algorithm that involves the exchange of pairs of particles of differing diameters, can dramatically accelerate the equilibration of simulated supercooled liquids in regimes where the normal dynamics is glassy. This spectacular effect was subsequently interpreted as direct evidence against a static, cooperative explanation of the glass transition such as the one offered by the random first-order transition (RFOT) theory. We review several empirical facts that support the opposite view, namely, that a local mechanism cannot explain the glass transition phenomenology. We explain the speedup induced by SWAP within the framework of the RFOT theory. We suggest that the efficiency of SWAP stems from a postponed onset of glassy dynamics, which allows the efficient exploration of configuration space even in the regime where the physical dynamics is dominated by activated events across free-energy barriers. We describe this effect in terms of `crumbling metastability' and use the example of nucleation to illustrate the possibility of circumventing free-energy barriers of thermodynamic origin by a change of the local dynamical rules.

cond-mat.stat-mech↗

arXiv2018

A random critical point separates brittle and ductile yielding transitions in amorphous materials

We combine an analytically solvable mean-field elasto-plastic model with molecular dynamics simulations of a generic glass-former to demonstrate that, depending on their preparation protocol, amorphous materials can yield in two qualitatively distinct ways. We show that well-annealed systems yield in a discontinuous brittle way, as metallic and molecular glasses do. Yielding corresponds in this case to a first-order nonequilibrium phase transition. As the degree of annealing decreases, the first-order character becomes weaker and the transition terminates in a second-order critical point in the universality class of an Ising model in a random field. For even more poorly annealed systems, yielding becomes a smooth crossover, representative of the ductile rheological behavior generically observed in foams, emulsions, and colloidal glasses. Our results show that the variety of yielding behavior found in amorphous materials does not result from the diversity of particle interactions or microscopic dynamics {\it per se}, but is instead unified by carefully considering the role of the initial stability of the system.

cond-mat.dis-nn↗

arXiv2018

Activated dynamics: an intermediate model between REM and p-spin

In order to study the activated dynamics of mean-field glasses, which takes place on times of order exp(N), where N is the system size, we introduce a new model, the Correlated Random Energy Model (CREM), that allows for a smooth interpolation between the REM and the p-spin models. We study numerically and analytically the CREM in the intermediate regime between REM and p-spin. We fully characterize its energy landscape, which is like a golf-course but, at variance with the REM, has metabasins (or holes) containing several configurations. We find that an effective trap-like description for the dynamics emerges, provided that one identifies metabasins in the CREM with configurations in the trap model.

cond-mat.dis-nn↗

arXiv2018

Complex energy landscapes in spiked-tensor and simple glassy models: ruggedness, arrangements of local minima and phase transitions

We study rough high-dimensional landscapes in which an increasingly stronger preference for a given configuration emerges. Such energy landscapes arise in glass physics and inference. In particular we focus on random Gaussian functions, and on the spiked-tensor model and generalizations. We thoroughly analyze the statistical properties of the corresponding landscapes and characterize the associated geometrical phase transitions. In order to perform our study, we develop a framework based on the Kac-Rice method that allows to compute the complexity of the landscape, i.e. the logarithm of the typical number of stationary points and their Hessian. This approach generalizes the one used to compute rigorously the annealed complexity of mean-field glass models. We discuss its advantages with respect to previous frameworks, in particular the thermodynamical replica method which is shown to lead to partially incorrect predictions.

cond-mat.dis-nn↗

arXiv2018

Experimental Determination of Configurational Entropy in a Two-Dimensional Liquid under Random Pinning

A quasi two-dimensional colloidal suspension is studied under the influence of immobilisation (pinning) of a random fraction of its particles. We introduce a novel experimental method to perform random pinning and, with the support of numerical simulation, we find that increasing the pinning concentration smoothly arrests the system, with a cross-over from a regime of high mobility and high entropy to a regime of low mobility and low entropy. At the local level, we study fluctuations in area fraction and concentration of pins and map them to entropic structural signatures and local mobility, obtaining a measure for the local entropic fluctuations of the experimental system.

cond-mat.soft↗