arXiv subjects

Sebastiano Ariosto

Publications and source records attributed to Sebastiano Ariosto.

3 recordsLinked to original sources

arXiv2025

Replication and Information Extraction in a Minimal Agent-Environment Model

How can information be extracted from data without explicit guidance or rewards? We investigate this question in a minimal setting where a classifying agent is exposed to a stream of structured data produced by a generative environment, and evolves by seeking consistency of its own labels in time. We find that imposing a simplicity bias on the classification rule can drive the dynamics toward label-coherent steady states, which we coin functional replicators. Remarkably, these persistent labeling rules align with the latent structure of the data. Using analytical tools from statistical mechanics, we characterize this spontaneous learning phase transition. Extending the analysis to a population of agents that pool labels from one another, we show that interaction reshapes the learning phase boundary, in some regimes enabling spontaneous learning that no isolated agent can achieve, while suppressing it in others. Our minimal framework thus opens a route to decentralized learning through label exchange alone, requiring no access to the internal weights of other agents.

cond-mat.dis-nn↗

arXiv2025

Statistical Physics of Deep Neural Networks: Generalization Capability, Beyond the Infinite Width, and Feature Learning

Deep Neural Networks (DNNs) excel at many tasks, often rivaling or surpassing human performance. Yet their internal processes remain elusive, frequently described as "black boxes." While performance can be refined experimentally, achieving a fundamental grasp of their inner workings is still a challenge. Statistical Mechanics has long tackled computational problems, and this thesis applies physics-based insights to understand DNNs via three complementary approaches. First, by averaging over data, we derive an asymptotic bound on generalization that depends solely on the size of the last layer, rather than on the total number of parameters -- revealing how deep architectures process information differently across layers. Second, adopting a data-dependent viewpoint, we explore a finite-width thermodynamic limit beyond the infinite-width regime. This leads to: (i) a closed-form expression for the generalization error in a finite-width one-hidden-layer network (regression task); (ii) an approximate partition function for deeper architectures; and (iii) a link between deep networks in this thermodynamic limit and Student's t-processes. Finally, from a task-explicit perspective, we present a preliminary analysis of how DNNs interact with a controlled dataset, investigating whether they truly internalize its structure -- collapsing to the teacher -- or merely memorize it. By understanding when a network must learn data structure rather than just memorize, it sheds light on fostering meaningful internal representations. In essence, this thesis leverages the synergy between Statistical Physics and Machine Learning to illuminate the inner behavior of DNNs.

cond-mat.dis-nn↗

arXiv2020

Random geometric graphs in high dimension

Many machine learning algorithms used for dimensional reduction and manifold learning leverage on the computation of the nearest neighbours to each point of a dataset to perform their tasks. These proximity relations define a so-called geometric graph, where two nodes are linked if they are sufficiently close to each other. Random geometric graphs, where the positions of nodes are randomly generated in a subset of $\mathbb{R}^{d}$, offer a null model to study typical properties of datasets and of machine learning algorithms. Up to now, most of the literature focused on the characterization of low-dimensional random geometric graphs whereas typical datasets of interest in machine learning live in high-dimensional spaces ($d \gg 10^{2}$). In this work, we consider the infinite dimensions limit of hard and soft random geometric graphs and we show how to compute the average number of subgraphs of given finite size $k$, e.g. the average number of $k$-cliques. This analysis highlights that local observables display different behaviors depending on the chosen ensemble: soft random geometric graphs with continuous activation functions converge to the naive infinite dimensional limit provided by Erdös-Rényi graphs, whereas hard random geometric graphs can show systematic deviations from it. We present numerical evidence that our analytical insights, exact in infinite dimensions, provide a good approximation also for dimension $d\gtrsim10$.

cond-mat.stat-mech↗