Dynamical Heterogeneity in Glass-Forming Liquids
We review the phenomena of dynamical heterogeneity in glass-forming systems and its description within replica and mean-field theories of the glass transition.
cond-mat.dis-nn↗
arXiv subjects
Giulio Biroli
Publications and source records attributed to Giulio Biroli.
The RFOT Theory of Glasses: Recent Progress and Open Issues
The Random First Order Transition (RFOT) theory started with the pioneering work of Kirkpatrick, Thirumalai and Wolynes. It leverages the methods and advances of the theory of disordered systems. It fares remarkably well at reproducing the salient experimental facts of super-cooled liquids. Yet, direct and indisputable experimental validations are missing. In this short survey, we will review recent investigations that broadly support all static aspects of RFOT, but also those for which the standard dynamical extension of the theory appears to be struggling, in particular in relation with facilitation effects. We discuss possible solutions and open issues.
cond-mat.dis-nn↗
Rare events and disorder control the brittle yielding of well-annealed amorphous solids
We use atomistic computer simulations to provide a microscopic description of the brittle failure of amorphous materials, and we assess the role of rare events and quenched disorder. We argue that brittle yielding originates at rare soft regions, similarly to Griffiths effects in disordered systems. We numerically demonstrate how localized plastic events in such soft regions trigger macroscopic failure via the propagation of a shear band. This physical picture, which no longer holds in poorly annealed ductile materials, allows us to discuss the role of finite size effects in brittle yielding and reinforces the similarities between yielding and other disorder-controlled nonequilibrium phase transitions.
cond-mat.soft↗
Wavelet Conditional Renormalization Group
We develop a multiscale approach to estimate high-dimensional probability distributions from a dataset of physical fields or configurations observed in experiments or simulations. In this way we can estimate energy functions (or Hamiltonians) and efficiently generate new samples of many-body systems in various domains, from statistical physics to cosmology. Our method -- the Wavelet Conditional Renormalization Group (WC-RG) -- proceeds scale by scale, estimating models for the conditional probabilities of "fast degrees of freedom" conditioned by coarse-grained fields. These probability distributions are modeled by energy functions associated with scale interactions, and are represented in an orthogonal wavelet basis. WC-RG decomposes the microscopic energy function as a sum of interaction energies at all scales and can efficiently generate new samples by going from coarse to fine scales. Near phase transitions, it avoids the "critical slowing down" of direct estimation and sampling algorithms. This is explained theoretically by combining results from RG and wavelet theories, and verified numerically for the Gaussian and $φ^4$ field theories. We show that multiscale WC-RG energy-based models are more general than local potential models and can capture the physics of complex many-body interacting systems at all length scales. This is demonstrated for weak-gravitational-lensing fields reflecting dark matter distributions in cosmology, which include long-range interactions with long-tail probability distributions. WC-RG has a large number of potential applications in non-equilibrium systems, where the underlying distribution is not known {\it a priori}. Finally, we discuss the connection between WC-RG and deep network architectures.
cond-mat.stat-mech↗
Equilibrium Fluctuations in Mean-field Disordered Models
Mean-field models of glasses that present a random first order transition exhibit highly non-trivial fluctuations. Building on previous studies that focused on the critical scaling regime, we here obtain a fully quantitative framework for all equilibrium conditions. By means of the replica method we evaluate Gaussian fluctuations of the overlaps around the thermodynamic limit, decomposing them in thermal fluctuations inside each state and heterogeneous fluctuations between different states. We first test and compare our analytical results with numerical simulation results for the p-spin spherical model and the random orthogonal model, and then analyze the random Lorentz gas. In all cases, a strong quantitative agreement is obtained. Our analysis thus provides a robust scheme for identifying the key finite-size (or finite-dimensional) corrections to the mean-field treatment of these paradigmatic glass models.
cond-mat.dis-nn↗
Local dynamical heterogeneity in glass formers
We study the local dynamical fluctuations in glass-forming models of particles embedded in $d$-dimensional space, in the mean-field limit of $d\to\infty$. Our analytical calculation reveals that single-particle observables, such as squared particle displacements, display divergent fluctuations around the dynamical (or mode-coupling) transition, due to the emergence of nontrivial correlations between displacements along different directions. This effect notably gives rise to a divergent non-Gaussian parameter, $α_2$. The $d\to\infty$ local dynamics therefore becomes quite rich upon approaching the glass transition. The finite-$d$ remnant of this phenomenon further provides a long sought-after, first-principle explanation for the growth of $α_2$ around the glass transition that is \emph{not based on multi-particle correlations}.
cond-mat.dis-nn↗
Optimal learning rate schedules in high-dimensional non-convex optimization problems
Learning rate schedules are ubiquitously used to speed up and improve optimisation. Many different policies have been introduced on an empirical basis, and theoretical analyses have been developed for convex settings. However, in many realistic problems the loss-landscape is high-dimensional and non convex -- a case for which results are scarce. In this paper we present a first analytical study of the role of learning rate scheduling in this setting, focusing on Langevin optimization with a learning rate decaying as $η(t)=t^{-β}$. We begin by considering models where the loss is a Gaussian random function on the $N$-dimensional sphere ($N\rightarrow \infty$), featuring an extensive number of critical points. We find that to speed up optimization without getting stuck in saddles, one must choose a decay rate $β<1$, contrary to convex setups where $β=1$ is generally optimal. We then add to the problem a signal to be recovered. In this setting, the dynamics decompose into two phases: an \emph{exploration} phase where the dynamics navigates through rough parts of the landscape, followed by a \emph{convergence} phase where the signal is detected and the dynamics enter a convex basin. In this case, it is optimal to keep a large learning rate during the exploration phase to escape the non-convex region as quickly as possible, then use the convex criterion $β=1$ to converge rapidly to the solution. Finally, we demonstrate that our conclusions hold in a common regression task involving neural networks.
cs.LG↗
On the interplay between data structure and loss function in classification problems
One of the central puzzles in modern machine learning is the ability of heavily overparametrized models to generalize well. Although the low-dimensional structure of typical datasets is key to this behavior, most theoretical studies of overparametrization focus on isotropic inputs. In this work, we instead consider an analytically tractable model of structured data, where the input covariance is built from independent blocks allowing us to tune the saliency of low-dimensional structures and their alignment with respect to the target function. Using methods from statistical physics, we derive a precise asymptotic expression for the train and test error achieved by random feature models trained to classify such data, which is valid for any convex loss function. We study in detail how the data structure affects the double descent curve, and show that in the over-parametrized regime, its impact is greater for logistic loss than for mean-squared loss: the easier the task, the wider the gap in performance at the advantage of the logistic loss. Our insights are confirmed by numerical experiments on MNIST and CIFAR10.
cs.LG↗
Critical behavior of the Anderson model on the Bethe lattice via a large-deviation approach
We present a new large-deviation approach to investigate the critical properties of the Anderson model on the Bethe lattice close to the localization transition in the thermodynamic limit. Our method allows us to study accurately the distribution of the local density of states (LDoS) down to very small probability tails as small as $10^{-50}$ which are completely out of reach for standard numerical techniques. We perform a thorough analysis of the functional form and of the tails of the probability distribution of the LDoS which yields for the first time a direct, transparent, and precise estimation of the correlation volume close to the Anderson transition. Such correlation volume is found to diverge exponentially when the localization is approached from the delocalized regime, in a singular way that is in agreement with the analytic predictions of the supersymmetric treatment.
cond-mat.dis-nn↗
On the Dynamics of Liquids in the Large-Dimensional Limit
In this work, we analytically derive the exact closed dynamical equations for a liquid with short-ranged interactions in large spatial dimensions using the same statistical mechanics tools employed to analyze Brownian motion. Our derivation greatly simplifies the original path-integral-based route to these equations and provides new insight into the physical features associated with high-dimensional liquids and glass formation. Most importantly, our construction provides a facile route to the exact dynamical analysis of important related dynamical problems, as well as a means to devise cluster generalizations of the exact solution in infinite dimensions. This latter fact opens the door to the construction of increasingly accurate theories of vitrification in three-dimensional liquids.
cond-mat.stat-mech↗
Transformed CNNs: recasting pre-trained convolutional layers with self-attention
Vision Transformers (ViT) have recently emerged as a powerful alternative to convolutional networks (CNNs). Although hybrid models attempt to bridge the gap between these two architectures, the self-attention layers they rely on induce a strong computational bottleneck, especially at large spatial resolutions. In this work, we explore the idea of reducing the time spent training these layers by initializing them as convolutional layers. This enables us to transition smoothly from any pre-trained CNN to its functionally identical hybrid model, called Transformed CNN (T-CNN). With only 50 epochs of fine-tuning, the resulting T-CNNs demonstrate significant performance gains over the CNN (+2.2% top-1 on ImageNet-1k for a ResNet50-RS) as well as substantially improved robustness (+11% top-1 on ImageNet-C). We analyze the representations learnt by the T-CNN, providing deeper insights into the fruitful interplay between convolutions and self-attention. Finally, we experiment initializing the T-CNN from a partially trained CNN, and find that it reaches better performance than the corresponding hybrid model trained from scratch, while reducing training time.
cs.LG↗
ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases
Convolutional architectures have proven extremely successful for vision tasks. Their hard inductive biases enable sample-efficient learning, but come at the cost of a potentially lower performance ceiling. Vision Transformers (ViTs) rely on more flexible self-attention layers, and have recently outperformed CNNs for image classification. However, they require costly pre-training on large external datasets or distillation from pre-trained convolutional networks. In this paper, we ask the following question: is it possible to combine the strengths of these two architectures while avoiding their respective limitations? To this end, we introduce gated positional self-attention (GPSA), a form of positional self-attention which can be equipped with a ``soft" convolutional inductive bias. We initialise the GPSA layers to mimic the locality of convolutional layers, then give each attention head the freedom to escape locality by adjusting a gating parameter regulating the attention paid to position versus content information. The resulting convolutional-like ViT architecture, ConViT, outperforms the DeiT on ImageNet, while offering a much improved sample efficiency. We further investigate the role of locality in learning by first quantifying how it is encouraged in vanilla self-attention layers, then analysing how it is escaped in GPSA layers. We conclude by presenting various ablations to better understand the success of the ConViT. Our code and models are released publicly at https://github.com/facebookresearch/convit.
cs.CV↗
Revisiting the Concept of Activation in Supercooled Liquids
In this work we revisit the description of dynamics based on the concepts of metabasins and activation in mildly supercooled liquids via the analysis of the dynamics of a paradigmatic glass former between its onset temperature $T_{o}$ and mode-coupling temperature $T_{c}$. First, we provide measures that demonstrate that the onset of glassiness is indeed connected to the landscape, and that metabasin waiting time distributions are so broad that the system can remain stuck in a metabasin for times that exceed $τ_α$ by orders of magnitude. We then reanalyze the transitions between metabasins, providing several indications that the standard picture of activated dynamics in terms of traps does not hold in this regime. Instead, we propose that here activation is principally driven by entropic instead of energetic barriers. In particular, we illustrate that activation is not controlled by the hopping of high energetic barriers, and should more properly be interpreted as the entropic selection of nearly barrierless but rare pathways connecting metabasins on the landscape.
cond-mat.soft↗
Effects of intraspecific cooperative interactions in large ecosystems
We analyze the role of the Allee effect, a positive correlation between population density and mean individual fitness, for ecological communities formed by a large number of species. Our study is performed using the generalized Lotka-Volterra model with random interactions between species. We obtain the phase diagram and analyze the nature of the multiple equilibria phase. Remarkable differences emerge with respect to the case of the logistic growth case, thus revealing the major role played by the functional response in determining aggregate behaviours of ecosystems.
cond-mat.dis-nn↗
Sifting out the features by pruning: Are convolutional networks the winning lottery ticket of fully connected ones?
Pruning methods can considerably reduce the size of artificial neural networks without harming their performance. In some cases, they can even uncover sub-networks that, when trained in isolation, match or surpass the test accuracy of their dense counterparts. Here we study the inductive bias that pruning imprints in such "winning lottery tickets". Focusing on visual tasks, we analyze the architecture resulting from iterative magnitude pruning of a simple fully connected network (FCN). We show that the surviving node connectivity is local in input space, and organized in patterns reminiscent of the ones found in convolutional networks (CNN). We investigate the role played by data and tasks in shaping the architecture of pruned sub-networks. Our results show that the winning lottery tickets of FCNs display the key features of CNNs. The ability of such automatic network-simplifying procedure to recover the key features "hand-crafted" in the design of CNNs suggests interesting applications to other datasets and tasks, in order to discover new and efficient architectural inductive biases.
cs.LG↗
Out of equilibrium Phase Diagram of the Quantum Random Energy Model
In this paper we study the out-of-equilibrium phase diagram of the quantum version of Derrida's Random Energy Model, which is the simplest model of mean-field spin glasses. We interpret its corresponding quantum dynamics in Fock space as a one-particle problem in very high dimension to which we apply different theoretical methods tailored for high-dimensional lattices: the Forward-Scattering Approximation, a mapping to the Rosenzweig-Porter model, and the cavity method. Our results indicate the existence of two transition lines and three distinct dynamical phases: a completely many-body localized phase at low energy, a fully ergodic phase at high energy, and a multifractal "bad metal" phase at intermediate energy. In the latter, eigenfunctions occupy a diverging volume, yet an exponentially vanishing fraction of the total Hilbert space. We discuss the limitations of our approximations and the relationship with previous studies.
cond-mat.dis-nn↗
Elastoplasticity Mediates Dynamical Heterogeneity Below the Mode-Coupling Temperature
As liquids approach the glass transition temperature, dynamical heterogeneity emerges as a crucial universal feature of their behavior. Dynamic facilitation, where local motion triggers further motion nearby, plays a major role in this phenomenon. Here we show that long-range, elastically-mediated facilitation appears below the mode-coupling temperature, adding to the short-range component present at all temperatures. Our results suggest deep connections between the supercooled liquid and glass states, and pave the way for a deeper understanding of dynamical heterogeneity in glassy systems.
cond-mat.soft↗
Interplay between percolation and glassiness in the random Lorentz gas
The random Lorentz gas (RLG) is a minimal model of transport in heterogeneous media. It also models the dynamics of a tracer in a glassy system. These two perspectives, however, are fundamentally inconsistent. Arrest in the former is related to percolation, and hence continuous, while glass-like arrest is discontinuous. In order to clarify the interplay between percolation and glassiness in the RLG, we consider its exact solution in the infinite-dimensional $d\rightarrow\infty$ limit, as well as numerics in $d=2\ldots 20$. We find that the mean field solutions of the RLG and glasses fall in the same universality class, and that instantonic corrections related to rare cage escapes destroy the glass transition in finite dimensions. This advance suggests that the RLG can be used as a toy model to develop a first-principle description of hopping in structural glasses.
cond-mat.stat-mech↗