arXiv subjects

Giulio Biroli

Publications and source records attributed to Giulio Biroli.

At least 37 records · Page 2Linked to original sources

arXiv2025

Eigenvalue spectral tails and localization properties of asymmetric networks

In contrast to the neatly bounded spectra of densely populated large random matrices, sparse random matrices often exhibit unbounded eigenvalue tails on the real and imaginary axis, called Lifshitz tails. In the case of asymmetric matrices, concise mathematical results have proved elusive. In this work, we present an analytical approach to characterising these tails. We exploit the fact that eigenvalues in the tail region have corresponding eigenvectors that are exponentially localised on highly-connected hubs of the network associated to the random matrix. We approximate these eigenvectors using a series expansion in the inverse connectivity of the hub, where successive terms in the series take into account further sets of next-nearest neighbours. By considering the ensemble of such hubs, we are able to characterise the eigenvalue density and the extent of localisation in the tails of the spectrum in a general fashion. As such, we classify a number of different asymptotic behaviours in the Lifshitz tails, as well as the leading eigenvalue and the inverse participation ratio. We demonstrate how an interplay between matrix asymmetry, network structure, and the edge-weight distribution leads to the variety of observed behaviours.

cond-mat.dis-nn↗

arXiv2025

Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Training

Diffusion models have achieved remarkable success across a wide range of generative tasks. A key challenge is understanding the mechanisms that prevent their memorization of training data and allow generalization. In this work, we investigate the role of the training dynamics in the transition from generalization to memorization. Through extensive experiments and theoretical analysis, we identify two distinct timescales: an early time $τ_\mathrm{gen}$ at which models begin to generate high-quality samples, and a later time $τ_\mathrm{mem}$ beyond which memorization emerges. Crucially, we find that $τ_\mathrm{mem}$ increases linearly with the training set size $n$, while $τ_\mathrm{gen}$ remains constant. This creates a growing window of training times with $n$ where models generalize effectively, despite showing strong memorization if training continues beyond it. It is only when $n$ becomes larger than a model-dependent threshold that overfitting disappears at infinite training times. These findings reveal a form of implicit dynamical regularization in the training dynamics, which allow to avoid memorization even in highly overparameterized settings. Our results are supported by numerical experiments with standard U-Net architectures on realistic and synthetic datasets, and by a theoretical analysis using a tractable random features model studied in the high-dimensional limit.

cs.LG↗

arXiv2025

The Role of the Time-Dependent Hessian in High-Dimensional Optimization

Gradient descent is commonly used to find minima in rough landscapes, particularly in recent machine learning applications. However, a theoretical understanding of why good solutions are found remains elusive, especially in strongly non-convex and high-dimensional settings. Here, we focus on the phase retrieval problem as a typical example, which has received a lot of attention recently in theoretical machine learning. We analyze the Hessian during gradient descent, identify a dynamical transition in its spectral properties, and relate it to the ability of escaping rough regions in the loss landscape. When the signal-to-noise ratio (SNR) is large enough, an informative negative direction exists in the Hessian at the beginning of the descent, i.e in the initial condition. While descending, a BBP transition in the spectrum takes place in finite time: the direction is lost, and the dynamics is trapped in a rugged region filled with marginally stable bad minima. Surprisingly, for finite system sizes, this window of negative curvature allows the system to recover the signal well before the theoretical SNR found for infinite sizes, emphasizing the central role of initialization and early-time dynamics for efficiently navigating rough landscapes.

cs.LG↗

arXiv2025

Entropy Production from Density Field Theories for interacting particles systems

Entropy production quantifies the breaking of time-reversal symmetry in non-equilibrium systems. Here, we develop a direct method to obtain closed, tractable expressions for entropy production in a broad class of dynamical density functional theories, from Dean's exact stochastic equation for microscopic densities to coarse-grained fluctuating-hydrodynamics models with density-dependent mobility. The method employs an Onsager-Machlup path-integral formulation. Our results reproduce particle-level calculations and matches recent Doi-Peliti treatments, confirming that the irregular noise structure of Dean's equation poses no obstacle when handled consistently. We further extend the framework to active mixtures with non-reciprocal interactions and to run-and-tumble or active-Brownian suspensions, generalizations that require a careful treatment of the spurious-drift. Our method furnishes a practical route to quantify irreversibility in density functional field theories and paves the way for systematic studies of entropy production in multi-field active fluids that couple density, momentum and orientation.

cond-mat.stat-mech↗

arXiv2025

Numerical investigation of the equilibrium Kauzmann transition in a two-dimensional atomistic glass

Dense liquids gradually transform into non-equilibrium amorphous solids as they pass through the experimental glass transition. Experimentally, ergodicity is lost because measurements are conducted within a finite time window. More than seventy years ago, Kauzmann posed a fundamental question: If experiments could run indefinitely, would there exist a critical temperature at which an ergodicity-breaking phase transition occurs? Random first-order transitions represent the modern theoretical framework for this idea, rigorously established in the mean-field limit of high-dimensional atomistic systems and several idealized physical models. However, achieving theoretical understanding in finite dimensions is challenging, while experimental and numerical limitations on accessible timescales hinder direct observation of the putative Kauzmann transition. Here, we overcome this longstanding barrier by developing a computational strategy that combines three advanced Monte Carlo methods to access the equilibrium thermodynamic properties of a two-dimensional atomistic glass-former down to zero temperature across a range of system sizes. This enables us to directly measure thermodynamic and structural observables that provide unambiguous evidence that the system undergoes a Kauzmann transition at a temperature that vanishes in the thermodynamic limit. This transition is towards an ideal glass state characterized by a complex energy landscape with a hierarchical organization of low-lying states. Our results are the first demonstration that computer simulations can fully probe the statistical mechanics of the bulk transition to a non-ergodic glass state. We anticipate that our study will serve as a foundation for future simulation work on larger systems, three-dimensional materials, and more complex glass-forming models to fully elucidate the nature of the glass state of matter.

cond-mat.soft↗

arXiv2025

Classifier-Free Guidance: From High-Dimensional Analysis to Generalized Guidance Forms

Classifier-Free Guidance (CFG) is a widely adopted technique in diffusion and flow-based generative models, enabling high-quality conditional generation. A key theoretical challenge is characterizing the distribution induced by CFG, particularly in high-dimensional settings relevant to real-world data. Previous works have shown that CFG modifies the target distribution, steering it towards a distribution sharper than the target one, more shifted towards the boundary of the class. In this work, we provide a high-dimensional analysis of CFG, showing that these distortions vanish as the data dimension grows. We present a blessing-of-dimensionality result demonstrating that in sufficiently high and infinite dimensions, CFG accurately reproduces the target distribution. Using our high-dimensional theory, we show that there is a large family of guidances enjoying this property, in particular non-linear CFG generalizations. We study a simple non-linear power-law version, for which we demonstrate improved robustness, sample fidelity and diversity. Our findings are validated with experiments on class-conditional and text-to-image generation using state-of-the-art diffusion and flow-matching models.

cs.LG↗

arXiv2025

A Differentiable Rank-Based Objective For Better Feature Learning

In this paper, we leverage existing statistical methods to better understand feature learning from data. We tackle this by modifying the model-free variable selection method, Feature Ordering by Conditional Independence (FOCI), which is introduced in \cite{azadkia2021simple}. While FOCI is based on a non-parametric coefficient of conditional dependence, we introduce its parametric, differentiable approximation. With this approximate coefficient of correlation, we present a new algorithm called difFOCI, which is applicable to a wider range of machine learning problems thanks to its differentiable nature and learnable parameters. We present difFOCI in three contexts: (1) as a variable selection method with baseline comparisons to FOCI, (2) as a trainable model parametrized with a neural network, and (3) as a generic, widely applicable neural network regularizer, one that improves feature learning with better management of spurious correlations. We evaluate difFOCI on increasingly complex problems ranging from basic variable selection in toy examples to saliency map comparisons in convolutional networks. We then show how difFOCI can be incorporated in the context of fairness to facilitate classifications without relying on sensitive data.

stat.ML↗

arXiv2025

Cascade of phase transitions in the training of Energy-based models

In this paper, we investigate the feature encoding process in a prototypical energy-based generative model, the Restricted Boltzmann Machine (RBM). We start with an analytical investigation using simplified architectures and data structures, and end with numerical analysis of real trainings on real datasets. Our study tracks the evolution of the model's weight matrix through its singular value decomposition, revealing a series of phase transitions associated to a progressive learning of the principal modes of the empirical probability distribution. The model first learns the center of mass of the modes and then progressively resolve all modes through a cascade of phase transitions. We first describe this process analytically in a controlled setup that allows us to study analytically the training dynamics. We then validate our theoretical results by training the Bernoulli-Bernoulli RBM on real data sets. By using data sets of increasing dimension, we show that learning indeed leads to sharp phase transitions in the high-dimensional limit. Moreover, we propose and test a mean-field finite-size scaling hypothesis. This shows that the first phase transition is in the same universality class of the one we studied analytically, and which is reminiscent of the mean-field paramagnetic-to-ferromagnetic phase transition.

cs.LG↗

arXiv2025

Optimizing Noise Schedules of Generative Models in High Dimensionss

Recent works have shown that diffusion models can undergo phase transitions, the resolution of which is needed for accurately generating samples. This has motivated the use of different noise schedules, the two most common choices being referred to as variance preserving (VP) and variance exploding (VE). Here we revisit these schedules within the framework of stochastic interpolants. Using the Gaussian Mixture (GM) and Curie-Weiss (CW) data distributions as test case models, we first investigate the effect of the variance of the initial noise distribution and show that VP recovers the low-level feature (the distribution of each mode) but misses the high-level feature (the asymmetry between modes), whereas VE performs oppositely. We also show that this dichotomy, which happens when denoising by a constant amount in each step, can be avoided by using noise schedules specific to VP and VE that allow for the recovery of both high- and low-level features. Finally we show that these schedules yield generative models for the GM and CW model whose probability flow ODE can be discretized using $Θ_d(1)$ steps in dimension $d$ instead of the $Θ_d(\sqrt{d})$ steps required by constant denoising.

cs.LG↗

arXiv2024

Effective Energy, Interactions And Out Of Equilibrium Nature Of Scalar Active Matter

Estimating the effective energy, $E_\text{eff}$ of a stationary probability distribution is a challenge for non-equilibrium steady states. Its solution could offer a novel framework for describing and analyzing non-equilibrium systems. In this work, we address this issue within the context of scalar active matter, focusing on the continuum field theory of Active Model B+. We show that the Wavelet Conditional Renormalization Group method allows us to estimate the effective energy of active model B+ from samples obtained by numerical simulations. We investigate the qualitative changes of $E_\text{eff}$ as the activity level increases. Our key finding is that in the regimes corresponding to low activity and to standard phase separation the interactions in $E_\text{eff}$ are short-ranged, whereas for strong activity the interactions become long-ranged and lead to micro-phase separation. By analyzing the violation of Fluctuation-Dissipation theorem and entropy production patterns, which are directly accessible within the WCRG framework, we connect the emergence of these long-range interactions to the non-equilibrium nature of the steady state. This connection highlights the interplay between activity, range of the interactions and the fundamental properties of non-equilibrium systems.

cond-mat.stat-mech↗

arXiv2024

Quantum Thermalization via Travelling Waves

Isolated quantum many-body systems which thermalize under their own dynamics are expected to act as their own thermal baths, thereby bringing their local subsystems to thermal equilibrium. Here we show that the infinite-dimensional limit of a quantum lattice model, as described by Dynamical Mean-Field theory (DMFT), provides a natural framework to understand this self-consistent thermalization process. Using the Fermi-Hubbard model as working example, we demonstrate that the emergence of a self-consistent bath thermalising the system is characterized by a sharp thermalization front, moving balistically and separating the initial condition from the long-time thermal fixed point. We characterize the full DMFT dynamics through an effective temperature for which we derive a travelling-wave equation of the Fisher-Kolmogorov-Petrovsky-Piskunov (FKPP) type. This equation allows to predict the asymptotic shape of the front and its velocity, which match perfectly the full DMFT numerics. Our results provide a new angle to understand the onset of quantum thermalisation in closed isolated systems.

cond-mat.str-el↗

arXiv2024

Kernel Density Estimators in Large Dimensions

This paper studies Kernel Density Estimation for a high-dimensional distribution $ρ(x)$. Traditional approaches have focused on the limit of large number of data points $n$ and fixed dimension $d$. We analyze instead the regime where both the number $n$ of data points $y_i$ and their dimensionality $d$ grow with a fixed ratio $α=(\log n)/d$. Our study reveals three distinct statistical regimes for the kernel-based estimate of the density $\hat ρ_h^{\mathcal {D}}(x)=\frac{1}{n h^d}\sum_{i=1}^n K\left(\frac{x-y_i}{h}\right)$, depending on the bandwidth $h$: a classical regime for large bandwidth where the Central Limit Theorem (CLT) holds, which is akin to the one found in traditional approaches. Below a certain value of the bandwidth, $h_{CLT}(α)$, we find that the CLT breaks down. The statistics of $\hatρ_h^{\mathcal {D}}(x)$ for a fixed $x$ drawn from $ρ(x)$ is given by a heavy-tailed distribution (an alpha-stable distribution). In particular below a value $h_G(α)$, we find that $\hatρ_h^{\mathcal {D}}(x)$ is governed by extreme value statistics: only a few points in the database matter and give the dominant contribution to the density estimator. We provide a detailed analysis for high-dimensional multivariate Gaussian data. We show that the optimal bandwidth threshold based on Kullback-Leibler divergence lies in the new statistical regime identified in this paper. As known by practitioners, when decreasing the bandwidth a Kernel-estimated estimated changes from a smooth curve to a collections of peaks centred on the data points. Our findings reveal that this general phenomenon is related to sharp transitions between phases characterized by different statistical properties, and offer new insights for Kernel density estimation in high-dimensional settings.

cs.LG↗

arXiv2024

Roadmap on machine learning glassy dynamics

Unraveling the connections between microscopic structure, emergent physical properties, and slow dynamics has long been a challenge when studying the glass transition. The absence of clear visible structural order in amorphous configurations complicates the identification of the key physical mechanisms underpinning slow dynamics. The difficulty in sampling equilibrated configurations at low temperatures hampers thorough numerical and theoretical investigations. This perspective article explores the potential of machine learning (ML) techniques to face these challenges, building on the algorithms that have revolutionized computer vision and image recognition. We present recent successful ML applications, as well as many open problems for the future, such as transferability and interpretability of ML approaches. We highlight new ideas and directions in which ML could provide breakthroughs to better understand the fundamental mechanisms at play in glass-forming liquids. To foster a collaborative community effort, this article also introduces the ``GlassBench" dataset, providing simulation data and benchmarks for both two-dimensional and three-dimensional glass-formers. We propose critical metrics to compare the performance of emerging ML methodologies, in line with benchmarking practices in image and text recognition. The goal of this roadmap is to provide guidelines for the development of ML techniques in systems displaying slow dynamics, while inspiring new directions to improve our theoretical understanding of glassy liquids.

cond-mat.soft↗

arXiv2024

Normalizing flows as an enhanced sampling method for atomistic supercooled liquids

Normalizing flows can transform a simple prior probability distribution into a more complex target distribution. Here, we evaluate the ability and efficiency of generative machine learning methods to sample the Boltzmann distribution of an atomistic model for glass-forming liquids. This is a notoriously difficult task, as it amounts to ergodically exploring the complex free energy landscape of a disordered and frustrated many-body system. We optimize a normalizing flow model to successfully transform high-temperature configurations of a dense liquid into low-temperature ones, near the glass transition. We perform a detailed comparative analysis with established enhanced sampling techniques developed in the physics literature to assess and rank the performance of normalizing flows against state-of-the-art algorithms. We demonstrate that machine learning methods are very promising, showing a large speedup over conventional molecular dynamics. Normalizing flows show performances comparable to parallel tempering and population annealing, while still falling far behind the swap Monte Carlo algorithm. Our study highlights the potential of generative machine learning models in scientific computing for complex systems, but also points to some of its current limitations and the need for further improvement.

cond-mat.soft↗

arXiv2024

Dynamical Facilitation Governs the Equilibration Dynamics of Glasses

Convincing evidence of domain growth in the heating of ultrastable glasses suggests that the equilibration dynamics of super-cooled liquids could be driven by a nucleation and growth mechanism. We investigate this possibility by simulating the equilibration dynamics of a model glass during both heating and cooling between poorly and well-annealed states. Though we do observe the growth of domains during heating, we find that domains are absent during cooling. This absence is inconsistent with classical nucleation theory. By comparing the equilibration dynamics of our glass with that of two models with kinetic constraints, we demonstrate that dynamical facilitation generically leads to heating driven by domain growth and cooling without domains. Our results provide strong evidence that dynamical facilitation, not nucleation and interfacial-tension-driven domain growth, is the driving mechanism for the equilibration dynamics of glass-formers.

cond-mat.soft↗

arXiv2024

Dynamical Regimes of Diffusion Models

Using statistical physics methods, we study generative diffusion models in the regime where the dimension of space and the number of data are large, and the score function has been trained optimally. Our analysis reveals three distinct dynamical regimes during the backward generative diffusion process. The generative dynamics, starting from pure noise, encounters first a 'speciation' transition where the gross structure of data is unraveled, through a mechanism similar to symmetry breaking in phase transitions. It is followed at later time by a 'collapse' transition where the trajectories of the dynamics become attracted to one of the memorized data points, through a mechanism which is similar to the condensation in a glass phase. For any dataset, the speciation time can be found from a spectral analysis of the correlation matrix, and the collapse time can be found from the estimation of an 'excess entropy' in the data. The dependence of the collapse time on the dimension and number of data provides a thorough characterization of the curse of dimensionality for diffusion models. Analytical solutions for simple models like high-dimensional Gaussian mixtures substantiate these findings and provide a theoretical framework, while extensions to more complex scenarios and numerical validations with real datasets confirm the theoretical predictions.

cs.LG↗

arXiv2024

Interactions and migration rescuing ecological diversity

How diversity is maintained in natural ecosystems is a long-standing question in Theoretical Ecology. By studying a system that combines ecological dynamics, heterogeneous interactions and spatial structure, we uncover a new mechanism for the survival of diversity-rich ecosystems in the presence of demographic fluctuations. For a single species, one finds a continuous phase transition between an extinction and a survival state, that falls into the universality class of Directed Percolation. Here we show that the case of many species with heterogeneous interactions is different and richer. By merging theory and simulations, we demonstrate that with sufficiently strong demographic noise, the system exhibits behavior akin to the single-species case, undergoing a continuous transition. Conversely, at low demographic noise, we observe unique features indicative of the ecosystem's complexity. The combined effects of the heterogeneity in the interaction network and migration enable the community to thrive, even in situations where demographic noise would lead to the extinction of isolated species. The emergence of mutualism induces the development of global bistability, accompanied by sudden tipping points. We present a way to predict the catastrophic shift from high diversity to extinction by probing responses to perturbations as an early warning signal.

q-bio.PE↗

arXiv2024

Yielding and plasticity in amorphous solids

The physics of disordered media, from metallic glasses to colloidal suspensions, granular matter and biological tissues, offers difficult challenges because it often occurs far from equilibrium, in materials lacking symmetries and evolving through complex energy landscapes. Here, we review recent theoretical efforts to provide microscopic insights into the mechanical properties of amorphous media using approaches from statistical mechanics as unifying frameworks. We cover both the initial regime corresponding to small deformations, and the yielding transition marking a change between elastic response and plastic flow. We discuss the specific features arising for systems evolving near a jamming transition, and extend our discussion to recent studies of the rheology of dense biological and active materials.

cond-mat.stat-mech↗