arXiv subjects

Giulio Biroli

Publications and source records attributed to Giulio Biroli.

At least 19 recordsLinked to original sources

arXiv2026

Generative diffusion in long-range correlated landscapes: Speciation cascade and the foam structure of the score

While state of the art for image and video generation, the fundamental reasons for the effectiveness of generative diffusion remain unclear. In particular, the observed advantage of following an entire reverse process with respect to single-shot or autoregressive methods, and for what type of data it may hold, is not well understood. In this work, we aim to better understand the role of the reverse process by considering a complex yet analytically tractable data model, the Gibbs measure of a high-dimensional Gaussian random potential with long-range correlations. Focusing on the exact score, we show that the speciation threshold, in which a trajectory commits to one class of the data, becomes a speciation window, in which the reverse process progressively resolves the Parisi tree of pure states of the full replica-symmetry breaking measure, committing to ever finer branches down to a single pure state. Importantly, these commitments take place at noise levels where the noisy sample distinguishes the pure states only through an energy of order one, beyond the reach of standard replica saddle points. Constructing a Larkin argument suited to such $O(1)$ energy gaps, we describe the score as a foam, with slab-like cells separated by shock-like boundaries. In the variance-exploding process we consider, this window is followed by a shrinkage of the samples towards the data support, and finally by reconstruction within the selected pure state. The denoising error of the exact score follows these stages, dropping during the speciation window with a profile set by the Parisi function alone, and then within the selected state. We believe that the generative foam we describe, and in particular the boundaries between cells, could play an important role in determining where the errors of trained diffusion models redistribute the mass of the generated samples when the data are long-range correlated.

cond-mat.dis-nn↗

arXiv2026

WaiT for the Signal: Simple Frequency-Aware Flow-Matching

As image generation models scale to ever higher resolutions, global coherence, local detail, and texture fidelity become critical axes for generation quality. However, standard flow matching treats all spatial frequencies uniformly, ignoring the natural frequency hierarchy where high-frequency bands become indistinguishable from pure noise far earlier than coarse structures. We introduce WaiT, a Wavelet-aware image Transformer that decomposes generation into coarse and fine bands via lossless wavelets. True to its name, the high-frequency bands wait for the signal: staying pure noise until coarse structure has emerged, then joining the flow for joint refinement. Since standard FID discards fine-grained detail through aggressive downsampling, we introduce a more stringent three-axis evaluation protocol to assess quality at native resolution. On ImageNet 512x512, WaiT achieves a pixel-space FID of 1.43 and is Pareto-optimal across all three axes, reducing sampling compute by up to 50%. With our largest 2B model, we set a new state-of-the-art FID of 1.3 for pixel-space models on ImageNet 512 resolution. Our formulation outperforms even the strongest latent-space models on texture fidelity, and scales seamlessly to high-resolution OpenImages and to video generation, achieving a state-of-the-art FVD of 0.84 on Kinetics-600 with no algorithmic modifications.

cs.CV↗

arXiv2026

Silent Alignment in Two-Layer Linear Networks: A High-Dimensional Perspective

We study gradient flow in a two-layer linear network in the high-dimensional regime where the input, inner and output dimensions diverge proportionally. Under whitened inputs and isotropic imbalance, we combine a closed-form solution of the dynamics with prior random matrix theory results to derive a high-dimensional limit for the dynamics. Our method yields exact asymptotic spectral distributions for the weight matrices at all times and characterizes the evolution of the learned subspace. In the small-initialization limit, we identify a silent alignment phenomenon: the singular directions of the network align on an order-one timescale, while the singular values and empirical loss remain at their initial values. Visible learning occurs later, on a logarithmic timescale, as singular values progressively escape zero and the loss decreases. This phase is characterized by a saddle-to-saddle dynamics, which becomes a continuous process of spectral learning in the high-dimensional limit.

stat.ML↗

arXiv2026

Some remarks on the connection between generative diffusion and optimal transport

We discuss three simple connections between generative diffusion and optimal transport. First, the joint law of the data and of their Ornstein-Uhlenbeck noising is, at every time, the solution of an entropic optimal-transport problem, at a temperature set by the noise level. Backward diffusion can consequently be read as an annealing which gradually decreases the entropic regularization. Second, we study the speciation transition--how the modes of the data distribution emerge during generation--and compare how it happens in diffusion and flow matching with respect to optimal transport. Using Gaussian mixtures as high-dimensional example, we show that the dynamical instability through which diffusion and flow matching separate the modes becomes, for optimal transport, an exponentially thin boundary layer at the beginning of generation, which is rounded by entropic regularization. Third, considering a generic forward dynamics with a general confining potential, we show that the backward process driven by the score also satisfies a drift-based variational principle whose minimal cost is the entropy difference between initial and final distributions.

stat.ML↗

arXiv2026

Boltzmann generators for amorphous particle systems

Sampling configurations in thermodynamic equilibrium is a long-standing challenge in statistical physics. Boltzmann generators address this problem by employing generative models to propose independent configurations, which are then reweighted via importance sampling using exact likelihood evaluations. Recent Boltzmann Generators based on continuous normalizing flows and flow matching have achieved significant success for particle systems and biomolecules. However, these approaches have not been extended to amorphous materials (glasses), for which equilibrium sampling is notoriously slow. Because of their disordered structure, the invariances and geometrical constraints of amorphous materials differ from those of crystals and biomolecules, preventing the direct use of existing generative models. Here, we develop Boltzmann Generators tailored to amorphous materials by building the required equivariances directly into Riemannian stochastic interpolants. Our framework incorporates periodic boundary conditions and particle symmetries using equivariant graph neural networks. Numerical experiments demonstrate that enforcing physical symmetries significantly improves the accuracy of Boltzmann Generators, but also reveal an intrinsic limitation of the continuous-flow formulation: accumulated numerical errors during likelihood integration break time-reversibility, compromising exact thermodynamic reweighting. These results reveal a fundamental challenge for continuous-flow generative models in statistical mechanics and call for alternative approaches that preserve exact thermodynamic consistency.

stat.ML↗

arXiv2026

Double Descent and Malign Overfitting in Diffusion Models

Conventional wisdom in deep learning holds that overparameterization---having more parameters $p$ than training samples $n$---is benign: larger models generalize better and, even without regularization, interpolating models generalize well, the test error following a double-descent curve. One might expect the same benign overfitting for diffusion models, whose training reduces to regression, i.e. to minimizing a quadratic score-matching loss. Yet the opposite is observed: overfitting here is catastrophic, driving the model into a memorization regime. We resolve this paradox by combining experiments on U-Nets trained on CelebA with a random-features model for which we derive closed-form learning curves. We show that with a fixed number $m$ of noise realizations per training sample, an interpolation peak does occur, but at $p\sim nm$ rather than at $p\sim n$ as in standard regression. The rise of the test loss, however, sets in much earlier, at $p\sim n$, independently of $m$. This overfitting is malign because, although the implicit regularization of training is fully at work, it drives the model toward the empirical score, which memorizes the training set, rather than toward the true score. A bias-variance decomposition pinpoints the mechanism: the bias of the score estimator starts to grow at $p\sim n$; past the peak the variance decays, as in regression, whereas the bias keeps growing and both saturate at a large value. Since diffusion models are trained with $m\gg1$, the peak is pushed to very large model sizes, and therefore sit on the rising branch that precedes it, where malign overfitting is already in play. Nevertheless, overparameterization remains beneficial when paired with regularization: in the random-features theory and in U-Net experiments, optimally regularized large models---via a ridge penalty or early stopping, respectively---outperform any unregularized models.

cs.LG↗

arXiv2026

Weighting Schedules Govern What and When Score-Based Generative Models Learn from Multimodal Data

Score-based generative models generate new samples by integrating a time-dependent drift that carries Gaussian noise onto the target distribution. In practice this drift is modeled by a neural network, trained on a loss integrated over time $t$ with a weighting schedule $w(t)$. Along the backward dynamics, and for multi-modal distributions, trajectories commit to modes of the target within a narrow time window, the \textit{speciation time}. In this work, focusing on high-dimensional data, we decompose the integrated loss into its single-time contributions and analyze each at fixed signal-to-noise ratio $Λ(t)$: we show that $Λ(t)$ sets the rate at which each feature of a multimodal target - the mode directions and their relative weights - is acquired during training. Crucially, at high $Λ(t)$ all mode directions are acquired together, on a single timescale insensitive to their amplitudes, while the relative weights are not learned at all. Only near the speciation time, where $Λ(t)$ becomes of order one, do all features become learnable, each on its own timescale: the weights are acquired jointly with the directions, and the directions at rates set by their relative amplitudes. For models trained on time-integrated objectives, the learning dynamics is then governed by how much of the weighting effectively sits near the speciation time, which provides insights on $w(t)$ design choices. These results follow from an exact high-dimensional analysis of the training dynamics of unbalanced and hierarchical Gaussian mixtures. Numerical experiments on image and human genome haplotype generation recover the predicted hierarchy of learning timescales in more complex settings.

cs.LG↗

arXiv2026

First Learn, Then Memorize: The Spectral Bias of Diffusion Models

Diffusion models trained on a finite dataset first learn to generate novel, high-quality samples and only much later collapse onto their training set. We identify the mechanism behind this separation of timescales and the object that probes it. The training dynamics of the score function are governed---exactly, and at any width---by the Gram matrix of the Neural Tangent Kernel (NTK) evaluated on the noisy training data, so the timescales of generalization and of memorization must be encoded in its spectrum. We show that they are, and that the structure responsible has no analogue in standard kernel settings. The use of multiple noise realizations per sample ($m$ noised copies at a fixed noise level) in the score-matching loss is what restructures the Gram matrix spectrum into two distinct parts. The first, of large eigenvalues, carries the global features of the target distribution and is present already for $m=1$. The second, which the repeated noising creates, consists of the smallest eigenvalues and is supported on eigenvectors aligned with the sample-specific noise directions; it sets a memorization timescale parametrically larger in the training set size $n$. We establish this picture on two fronts. Analytically, we solve the spectrum in the lazy high-dimensional limit for both linear ($n \asymp d$) and polynomial ($n \asymp d^k$) sample complexities, and prove through a bias--variance decomposition that the first bulk minimizes the approximation error while the second drives the error associated with memorization. Empirically, we show the same two-bulk structure in Convolutional NTKs on CelebA and in finite-width U-Nets trained well beyond the lazy regime, and we make the link causal: truncating the Gram matrix at rank $r$ tunes the generalization--memorization transition, and an $L_2$ penalty targeting the second bulk suppresses memorization in feature-learning U-Nets.

cs.LG↗

arXiv2026

The critical slowing down in training diffusion models

Computational sampling has been central to the sciences since the mid-20th century. While machine-learning-based approaches have recently enabled major advances, their behavior remains poorly understood, with limited theoretical control over when and why they succeed. Here we provide such insight for diffusion models---a class of generative schemes highly effective in practice---by analyzing their application to the $O(n)$ model of statistical field theory in the Gaussian limit $n \to \infty$. In this analytically tractable setting, we show that training a score model with a one-layer network architecture matching the exact solution exhibits a form of critical slowing down in parameter learning. This slowing down also impacts the generation process, indicating that the well-known difficulties of sampling near criticality persist even for learned generative models. To overcome this bottleneck, we consider the power of architectural depth. We find that using a two-layer architecture drastically reduces the critical slowing down, with the training time scaling logarithmically rather than quadratically with system size. Using a Fourier implementation of the architecture, we further show that this acceleration in training time can be achieved without drastically increasing operational complexity. Taken together, these results demonstrate that diffusion models can overcome the critical slowing down through appropriate architectural design, and establish a controlled framework for understanding and improving learned sampling methods in statistical physics and beyond.

cond-mat.dis-nn↗

arXiv2026

Overcoming critical slowing down in frustrated spin systems by learned multiscale sampling

Cluster algorithms, such as the Swendsen--Wang and Wolff methods, are among the most successful MCMC methods for mitigating critical slowing down in statistical systems. These constructive cluster algorithms, however, fail in the presence of even extremely weak frustration. Here, we sidestep this fundamental limitation by learning rather than constructing the relevant clusters. Specifically, we use the wavelet conditional renormalization group (WCRG) sampling method to learn the probability distribution of collective fluctuations of a frustrated two-dimensional soft-spin model. Configurations are then generated recursively from coarse to fine scales by sampling conditional wavelet distributions. The WCRG method reproduces the main statistical properties of the system across different phases, including the local-field distribution and the structure factor. At an Ising-like critical point, the conditional dynamics remains decorrelated within $\mathcal{O}(1)$ sweeps at each scale, yielding an overall sampling complexity of $\mathcal{O}(\log_2 L)$, thus making WCRG much more efficient than standard local MCMC methods. These results show that learned multiscale sampling can overcome critical slowing down in frustrated systems for which conventional cluster algorithms fail. By assessing the sampling accuracy of different observables, we also clarify the main tradeoff of the WCRG method: the accuracy of the fast sampling scheme depends on the expressiveness of the energy-based model used to estimate the wavelet conditional distributions.

cond-mat.stat-mech↗

arXiv2026

Nucleation beyond Equilibrium: Fronts Control Invasion in Bistable Ecosystems

Bistability, the existence of two alternative stable states with distinct basins of attraction, is common across ecology and many other biological, chemical, and physical systems. In spatially extended systems, the invasion of one state by the other proceeds through nucleation, the fluctuation-driven growth of a droplet beyond a critical size. Classical Nucleation Theory (CNT) quantifies this process using the energy landscape, but this description relies on detailed balance, the condition of microscopic reversibility that holds at equilibrium. Ecological dynamics generally violate detailed balance and are described not by a single scalar field but by several coupled, non-conserved abundances--a vector order parameter--for which no general nucleation theory exists. Here we derive such a theory for reaction-diffusion systems with a non-conserved vector order parameter, valid both close to the binodal, where invasion proceeds through propagating fronts, and close to the spinodal, where the metastable state loses stability. Extensive numerical computations of the quasipotential confirm the theory in both regimes. We show that the mathematical structure of CNT survives out of equilibrium, once energetic quantities are replaced by dynamical properties of fronts: the front speed plays the role of the bulk free-energy difference between phases, and the diffusivity that of the surface tension. Applied to the two-species Lotka-Volterra model, an archetypal system of bistable ecological antagonism, our theory shows that strong interspecific competition generates a pronounced depletion region within fronts, where the total abundance falls well below carrying capacity. This vectorial structure, invisible to a scalar description based on species frequency alone, makes invasion exponentially harder as competition strengthens at fixed competitive advantage--a prediction testable in microbial systems.

cond-mat.stat-mech↗

arXiv2026

Large deviations in the many-body localization transition: The case of the random-field XXZ chain

The effect of rare system-wide resonances in the many-body localization (MBL) transition has recently attracted significant attention. They are expected to play a prominent role in the stability of the MBL phase, prompting the development of new theoretical frameworks to properly account for their statistical weight. We employ a method based on an analogy with mean-field disordered glassy systems to characterize the statistics of transmission amplitudes between distant many-body configurations in Hilbert space, and apply it to the random-field XXZ spin chain. By introducing a Lagrange multiplier, which formally plays the role of an effective temperature controlling the influence of extreme outliers in the heavy-tailed distribution of propagators, we identify three distinct regimes: (i) an ergodic phase with uniform spreading in Hilbert space, (ii) an intermediate regime where delocalization is driven by rare, disorder-dependent long-range resonances, and (iii) a robust MBL phase where such resonances cannot destabilize localization. We derive a finite-size phase diagram in the disorder--interaction plane both in the spin and in the Anderson basis that quantitatively agrees with recent numerical results based on real-space spin-spin correlation functions. We further demonstrate that even infinitesimal interactions can destroy the Anderson insulator at finite disorder, with the critical disorder remaining finite down to small interaction strengths. By visualizing resonant transmission pathways on the Hilbert space graph, we provide a complementary perspective to real-space and spectral probes, revealing how the destabilization of the MBL phase at finite sizes stems from the emergence of resonant paths that become progressively rarer and shorter-ranged deep in the localized phase.

cond-mat.dis-nn↗

arXiv2026

Inertial Asynchronous Computation

Computation is the controlled evolution of a state. Asynchronous evolutions, where all parts of the state change in their own time without stopping each other, put this control in jeopardy. It is in fact a mystery how natural processes perform asynchronous computations using many units with no global orchestration. Here we demonstrate how collective computational abilities can emerge in asynchronous many-body systems. The key insight is to split the physical "hardware" underlying the computation into two asymmetrically coupled parts, analogous to position and momentum in a harmonic oscillator. The resulting inertia nudges the evolution of the state so that the asynchronous computation proceeds in the right order. By treating our inertial asynchronous computer as a nonequilibrium material, we map out its phase diagram numerically and analytically using a framework we dub loop dynamical mean-field theory. We experimentally demonstrate our approach using analog spiking neuromorphic chips designed to mimic actual neurons in the brain. In addition, we construct software that can run on asynchronous hardware: we denoise movies whose clean versions were never seen during training, an instantiation of the generalization transition underlying modern machine learning. Our results point to a general strategy for reliable, decentralized computation in energy-constrained settings from dynamics self-assembly to cell differentiation.

cond-mat.stat-mech↗

arXiv2026

When is nonreciprocity relevant?

Nonreciprocal interactions are widely observed in nonequilibrium systems, from biological or sociological dynamics to open quantum systems. Despite the ubiquity of nonreciprocity, its impact on phase transitions is not fully understood. In this work, we derive criteria to perturbatively assess whether nonreciprocity changes the universality class of two-species systems undergoing a phase transition. These criteria, stated in terms of the unperturbed critical exponents in the spirit of the Harris criterion for disordered systems, assess whether static critical exponents change at first order under a given perturbation. For example, in the case of a nonreciprocal version of model A with two species, a homogeneous nonreciprocal perturbation is relevant whenever the two parts are initially identical and uncoupled, and irrelevant otherwise. Our results agree with existing renormalization group calculations and with numerical simulations.

cond-mat.stat-mech↗

arXiv2026

High-Dimensional Analysis of Gradient Flow for Extensive-Width Quadratic Neural Networks

We study the high-dimensional training dynamics of a shallow neural network with quadratic activation in a teacher-student setup. We focus on the extensive-width regime, where the teacher and student network widths scale proportionally with the input dimension, and the sample size grows quadratically. This scaling aims to describe overparameterized neural networks in which feature learning still plays a central role. In the high-dimensional limit, we derive a dynamical characterization of the gradient flow, in the spirit of dynamical mean-field theory (DMFT). Under l2-regularization, we analyze these equations at long times and characterize the performance and spectral properties of the resulting estimator. This result provides a quantitative understanding of the effect of overparameterization on learning and generalization, and reveals a double descent phenomenon in the presence of label noise, where generalization improves beyond interpolation. In the small regularization limit, we obtain an exact expression for the perfect recovery threshold as a function of the network widths, providing a precise characterization of how overparameterization influences recovery.

math.OC↗

arXiv2026

Discontinuous BBP transitions

The Baik-Ben Arous-Peche (BBP) transition sets fundamental limits for detecting low-rank structure in noisy high-dimensional data and underlies a wide range of spectral methods in many fields from physics to statistics and data sciences. In standard settings, this transition is continuous, implying that signal recovery emerges gradually above a sharp threshold. We show that BBP transitions can instead be discontinuous in very general settings and provide a full theory of this phenomenon. When the eigenvalue density vanishes faster than linearly at the spectral edge, the overlap between the leading eigenvector and the signal jumps discontinuously at the critical point. We study this mechanism in deformed Gaussian and reweighted Wishart ensembles. We analyze in detail the finite-size effects, which play a central and qualitatively new role in the discontinuous BBP transition. Unlike the continuous BBP transition, we establish the existence of an extended pre-critical region where informative eigenvectors emerge well before the asymptotic threshold. The main consequence-and difference from the continuous BBP transition-is that signal recovery can occur at significantly lower signal-to-noise ratio and it is accompanied by strong sample-to-sample variability. Our results show the relevance and the novelty of the discontinuous BBP transition, and highlight the practical implications for signal detection.

cond-mat.dis-nn↗

arXiv2026

On the anomalous elasticity in the mechanical response of amorphous solids

The response of amorphous solids to a mechanical perturbation consists in an elastic and a plastic deformation. The latter is mediated by localized irreversible rearrangements associated with Eshelby-like quadrupolar singularities in the displacement field. It has recently been argued that a density of such singularities leads to an anomalous elastic behavior taking the form of screening effects, which goes beyond classical elastic predictions. Here, we reexamine this scenario using general theoretical arguments and a description in terms of an elasto-plastic model, which we compare with atomistic simulations of the canonical Eshelby inclusion geometry. We discuss the conditions under which a finite, i.e., nonvanishing, density of quadrupolar events is created by an imposed perturbation. We argue that, except when the perturbation is macroscopic, there are many situations in which the density of quadrupolar defects is zero in the thermodynamic limit. In these cases, we find that plastically active quadrupoles emerge in a region whose size generically scales as the spatial extent $\ell$ of the mechanical perturbation. This mechanism leads to anomalous elasticity on a scale $\ell$ close to the perturbation and to conventional elasticity beyond. The simulations of the elasto-plastic model reproduce the emergence of plastic quadrupoles in a region set by $\ell$ and the associated renormalization of the effective shear modulus, but they do not exhibit the dipole-screening signatures reported in atomistic and experimental studies. Our analysis delineates the scale-dependent breakdown of long-wavelength elasticity in amorphous materials and suggests directions for incorporating anomalous screening into mesoscopic modeling frameworks.

cond-mat.soft↗

arXiv2026

Topological Exploration of High-Dimensional Empirical Risk Landscapes: general approach, and applications to phase retrieval

We consider the landscape of empirical risk minimization for high-dimensional Gaussian single-index models (generalized linear models). The objective is to recover an unknown signal $\boldsymbolθ^\star \in \mathbb{R}^d$ (where $d \gg 1$) from a loss function $\hat{R}(\boldsymbolθ)$ that depends on pairs of labels $(\mathbf{x}_i \cdot \boldsymbolθ, \mathbf{x}_i \cdot \boldsymbolθ^\star)_{i=1}^n$, with $\mathbf{x}_i \sim \mathcal{N}(0, I_d)$, in the proportional asymptotic regime $n \asymp d$. Using the Kac-Rice formula, we analyze different complexities of the landscape -- defined as the expected number of critical points -- corresponding to various types of critical points, including local minima. We first show that some variational formulas previously established in the literature for these complexities can be drastically simplified, reducing to explicit variational problems over a finite number of scalar parameters that we can efficiently solve numerically. Our framework also provides detailed predictions for properties of the critical points, including the spectral properties of the Hessian and the joint distribution of labels. We apply our analysis to the real phase retrieval problem for which we derive complete topological phase diagrams of the loss landscape, characterizing notably BBP-type transitions where the Hessian at local minima (as predicted by the Kac-Rice formula) becomes unstable in the direction of the signal. We test the predictive power of our analysis to characterize gradient flow dynamics, finding excellent agreement with finite-size simulations of local optimization algorithms, and capturing fine-grained details such as the empirical distribution of labels. Overall, our results open new avenues for the asymptotic study of loss landscapes and topological trivialization phenomena in high-dimensional statistical models.

stat.ML↗