arXiv subjects

Damien Barbier

Publications and source records attributed to Damien Barbier.

11 recordsLinked to original sources

arXiv2026

When low-loss paths make a binary neuron trainable: detecting algorithmic transitions with the connected ensemble

We study the connected ensemble, a statistical-mechanics framework that characterizes the formation of low-loss paths in rugged landscapes. First introduced in a previous paper, this ensemble allows one to identify when a network can be trained on a simple task and which minima should be targeted during training. We apply this new framework to the symmetric binary perceptron model (SBP), and study how its typical {connected} minima behave. We show that {connected} minima exist only above a critical threshold $\kappa_{\rm connected}$, or equivalently below a critical constraint density $\alpha_{\rm connected}$. This defines a parameter range in which training the network is easy, as local algorithms can efficiently access this connected manifold. We also highlight that these minima become increasingly robust and closer to one another as the task on which the network is trained becomes more difficult.

cond-mat.dis-nn↗

arXiv2025

Finding the right path: statistical mechanics of connected solutions in constraint satisfaction problems

We define and study a statistical mechanics ensemble that characterizes connected solutions in constraint satisfaction problems (CSPs). Built around a well-known local entropy bias, it allows us to better identify hardness transitions in problems where the energy landscape is dominated by isolated solutions. We apply this new device to the symmetric binary perceptron model (SBP), and study how its manifold of connected solutions behaves. We choose this particular problem because, while its typical solutions are isolated, it can be solved using local algorithms for a certain range of constraint density $\alpha$ and threshold $\kappa$. With this new ensemble, we unveil the presence of a cluster composed of delocalized connected solutions. In particular, we demonstrate its stability until a critical threshold $\kappa^{\rm no-mem}_{\rm loc.\, stab.}$ (dependent on $\alpha$). This transition appears as paths of solutions shatter, a phenomenon that more conventional statistical mechanics approaches fail to grasp. Finally, we compared our predictions to simulations. For this, we used a modified Monte-Carlo algorithm, designed specifically to target these delocalized solutions. We obtained, as predicted, that the algorithm finds solutions until $\kappa\approx\kappa^{\rm no-mem}_{\rm loc.\, stab.}$.

cond-mat.dis-nn↗

arXiv2024

How to escape atypical regions in the symmetric binary perceptron: a journey through connected-solutions states

We study the binary symmetric perceptron model, and in particular its atypical solutions. While the solution-space of this problem is dominated by isolated configurations, it is also solvable for a certain range of constraint density $\alpha$ and threshold $\kappa$. We provide in this paper a statistical measure probing sequences of solutions, where two consecutive elements shares a strong overlap. After simplifications, we test its predictions by comparing it to Monte-Carlo simulations. We obtain good agreement and show that connected states with a Markovian correlation profile can fully decorrelate from their initialization only for $\kappa>\kappa_{\rm no-mem.\, state}$ ($\kappa_{\rm no-mem.\, state}\sim \sqrt{0.91\log(N)}$ for $\alpha=0.5$ and $N$ being the dimension of the problem). For $\kappa<\kappa_{\rm no-mem.\, state}$, we show that decorrelated sequences still exist but have a non-trivial correlations profile. To study this regime we introduce an $Ansatz$ for the correlations that we label as the nested Markov chain.

cond-mat.dis-nn↗

arXiv2024

The phase diagram of compressed sensing with $\ell_0$-norm regularization

Noiseless compressive sensing is a two-steps setting that allows for undersampling a sparse signal and then reconstructing it without loss of information. The LASSO algorithm, based on $\lone$ regularization, provides an efficient and robust to address this problem, but it fails in the regime of very high compression rate. Here we present two algorithms based on $\lzero$-norm regularization instead that outperform the LASSO in terms of compression rate in the Gaussian design setting for measurement matrix. These algorithms are based on the Approximate Survey Propagation, an algorithmic family within the Approximate Message Passing class. In the large system limit, they can be rigorously tracked through State Evolution equations and it is possible to exactly predict the range compression rates for which perfect signal reconstruction is possible. We also provide a statistical physics analysis of the $\lzero$-norm noiseless compressive sensing model. We show the existence of both a replica symmetric state and a 1-step replica symmmetry broken (1RSB) state for sufficiently low $\lzero$-norm regularization. The recovery limits of our algorithms are linked to the behavior of the 1RSB solution.

cs.IT↗

arXiv2024

On the Atypical Solutions of the Symmetric Binary Perceptron

We study the random binary symmetric perceptron problem, focusing on the behavior of rare high-margin solutions. While most solutions are isolated, we demonstrate that these rare solutions are part of clusters of extensive entropy, heuristically corresponding to non-trivial fixed points of an approximate message-passing algorithm. We enumerate these clusters via a local entropy, defined as a Franz-Parisi potential, which we rigorously evaluate using the first and second moment methods in the limit of a small constraint density $α$ (corresponding to vanishing margin $κ$) under a certain assumption on the concentration of the entropy. This examination unveils several intriguing phenomena: i) We demonstrate that these clusters have an entropic barrier in the sense that the entropy as a function of the distance from the reference high-margin solution is non-monotone when $κ\le 1.429 \sqrt{-α/\logα}$, while it is monotone otherwise, and that they have an energetic barrier in the sense that there are no solutions at an intermediate distance from the reference solution when $κ\le 1.239 \sqrt{-α/ \logα}$. The critical scaling of the margin $κ$ in $\sqrt{-α/\logα}$ corresponds to the one obtained from the earlier work of Gamarnik et al. (2022) for the overlap-gap property, a phenomenon known to present a barrier to certain efficient algorithms. ii) We establish using the replica method that the complexity (the logarithm of the number of clusters of such solutions) versus entropy (the logarithm of the number of solutions in the clusters) curves are partly non-concave and correspond to very large values of the Parisi parameter, with the equilibrium being reached when the Parisi parameter diverges.

math.PR↗

arXiv2022

Generalised Gibbs Ensemble for spherically constrained harmonic models

We build and analytically calculate the Generalised Gibbs Ensemble partition function of the integrable Soft Neumann Model. This is the model of a classical particle which is constrained to move, on average over the initial conditions, on an $N$ dimensional sphere, and feels the effect of anisotropic harmonic potentials. We derive all relevant averaged static observables in the (thermodynamic) $N\rightarrow\infty$ limit. We compare them to their long-term dynamic averages finding excellent agreement in all phases of a non-trivial phase diagram determined by the characteristics of the initial conditions and the amount of energy injected or extracted in an instantaneous quench. We discuss the implications of our results for the proper Neumann model in which the spherical constraint is imposed strictly.

cond-mat.stat-mech↗

arXiv2022

Destruction of Localization by Thermal Inclusions: Anomalous Transport and Griffiths Effects in the Anderson and André-Aubry-Harper Models

We discuss and compare two recently proposed toy models for anomalous transport and Griffiths effects in random systems near the Many-Body Localization transitions: the random dephasing model, which adds thermal inclusions in an Anderson Insulator as local Markovian dephasing channels that heat up the system, and the random Gaussian Orthogonal Ensemble (GOE) approach which models them in terms of ensembles of random regular graphs. For these two settings we discuss and compare transport and dissipative properties and their statistics. We show that both types of dissipation lead to similar Griffiths-like phenomenology, with the GOE bath being less effective in thermalising the system due to its finite bandwidth. We then extend these models to the case of a quasi-periodic potential as described by the André-Aubry-Harper model coupled to random thermal inclusions, that we show to display, for large strength of the quasiperiodic potential, a similar phenomenology to the one of the purely random case. In particular, we show the emergence of subdiffusive transport and broad statistics of the local density of states, suggestive of Griffiths like effects arising from the interplay between quasiperiodic localization and random coupling to the baths.

cond-mat.dis-nn↗

arXiv2021

Finite size effects and loss of self-averageness in the relaxational dynamics of the spherical Sherrington-Kirkpatrick model

We revisit the gradient descent dynamics of the spherical Sherrington-Kirkpatrick ($p=2$) model with finite number of degrees of freedom. For fully random initial conditions we confirm that the relaxation takes place in three time regimes: a first algebraic one controlled by the decay of the eigenvalue distribution of the random exchange interaction matrix at its edge in the infinite size limit; a faster algebraic one determined by the distribution of the gap between the two extreme eigenvalues; and a final exponential one determined by the minimal gap sampled in the disorder average. We also analyse the finite size effects on the relaxation from initial states which are almost projected on the saddles of the potential energy landscape, and we show that for deviations scaling as $N^{-ν}$ from perfect alignment the system escapes the initial configuration in a time-scale scaling as $\ln N$ after which the dynamics no longer "self-averages" with respect to the initial conditions. We prove these statements with a combination of analytic and numerical methods.

cond-mat.stat-mech↗

arXiv2020

(Non equilibrium) Thermodynamics of Integrable models: The Generalized Gibbs Ensemble description of the classical Neumann Model

We study a classical integrable (Neumann) model describing the motion of a particle on the sphere, subject to harmonic forces. We tackle the problem in the infinite dimensional limit by introducing a soft version in which the spherical constraint is imposed only on average over initial conditions. We show that the Generalized Gibbs Ensemble captures the long-time averages of the soft model. We reveal the full dynamic phase diagram with extended, quasi-condensed, coordinate-, and coordinate and momentum-condensed phases. The scaling properties of the fluctuations allow us to establish in which cases the strict and soft spherical constraints are equivalent, confirming the validity of the GGE hypothesis for the Neumann model on a large portion of the dynamic phase diagram.

cond-mat.stat-mech↗

arXiv2020

A constrained TAP approach for disordered spin models: application to the mixed spherical case

We revisit the metastability properties of the mixed p-spin spherical disordered models. Firstly, using known methods, we show that there is temperature chaos in a broad range of temperatures. Secondly, we modify the definition of the Thouless-Anderson-Palmer free energy density by including constraints that enforce a chosen overlap between the searched metastable states and another reference state, that could be a characteristic one of a different temperature. We argue that this refined analysis provides clues to understand the weird behaviour of the low temperature relaxation dynamics of these models, and suggests ways to improve the treatment of the initial conditions to overcome the difficulties encountered so far.

cond-mat.dis-nn↗

arXiv2019

Pre-asymptotic dynamics of the infinite size Neumann (p=2 spherical) model

In this contribution we further study the classical disordered p=2 spherical model with Hamiltonian dynamics, or in integrable systems terms, the Neumann model, in the infinite size limit. We summarise the asymptotic results that some of us presented in a recent publication, and we deepen the analysis of the pre-asymptotic dynamics. We also discuss the possible description of the asymptotic steady state with a Generalised Gibbs Ensemble.

cond-mat.stat-mech↗