Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.23783

Sampling the XY model with an attention-based neural network

Piotr Białas·Christian D. Morales·Tomasz Stebel·Dawid Zapolski
View original source ↗Read source document ↗Export metadata ↓

Abstract

We present a transformer-based variational autoregressive network for sampling lattice systems with continuous degrees of freedom and apply it to the two-dimensional XY model near the Berezinskii-Kosterlitz-Thouless transition. The architecture is combined with physics-informed ingredients: locality-informed attention and approximate probabilities. The resulting sampler significantly outperforms existing neural samplers. We train the model on lattices up to $96\times96$, verify with a mode-dropping estimator that no parts of the configuration space are missed, and reproduce the energy, magnetization and helicity modulus obtained with the traditional Markov Chain Monte Carlo algorithm.

Explore related subjects

cond-mat.dis-nncond-mat.stat-mechcs.LGhep-lat

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Piotr Białas, Christian D. Morales, Tomasz Stebel, Dawid Zapolski. 2026-10-16. Sampling the XY model with an attention-based neural network. https://arxiv.org/abs/2610.23783

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Finite-size geometric crossover toward fragmentation of the potential energy shell in the quantum random energy model

The quantum random energy model (QREM) is a paradigmatic model for studying delocalization-localization transitions at finite energy density in Fock space. In the computational basis, its diagonal energies are independent Gaussian random variables, in contrast to the correlated energies of local interacting models such as the disordered XXZ chain. In this work, we construct a potential energy shell associated with a basis state, defined through its energy variance, and investigate its connectivity on the Fock-space hypercube. As the transverse field is varied, this energy-selected subgraph undergoes a finite-size geometric crossover between fully connected and fragmented regimes. Furthermore, we observe a correlation between the shell connectivity and the finite-size fractal dimension of the eigenstates, as extracted from the participation entropy. For the QREM, the statistical independence of the diagonal energies allows the shell connectivity to be analyzed as an effective independent site percolation problem on the hypercube, with its crossover scale estimated within a one-hop approximation. At finite energy density, this estimate gives a nonmonotonic dependence of the crossover scale on system size $L$: increasing $L$ initially favors connectivity, while beyond an energy-dependent scale $L^*(\varepsilon)$ the trend reverses towards fragmentation.

cond-mat.dis-nn↗
arXiv2026

Attention over noisy examples: generalization and memorization in the exponential Hopfield network

In Hebbian learning from examples a Hopfield network of $N$ neurons is not supplied with the $K$ archetypes it has to retrieve but with $M$ blurred examples of each, of quality $r$, and whether it can infer them depends on the load $α=K/N$ (interference among them) and on the dataset entropy $ρ=(1-r^2)/(Mr^2)$ (how little the examples say about them). We transfer this protocol to the exponential Hopfield model, whose zero-temperature update is a step of softmax attention with the unlabelled examples as keys. A signal-to-noise analysis gives in closed form the overlap between the output and an unseen archetype, $m^{(1)}={\rm erf}(1/\sqrt{2ρ_{\rm eff}})$: the majority-rule law of the pairwise model at small load, with $ρ$ replaced by an attention entropy $ρ_{\rm eff}$ built from the weight the softmax gives to the correct class and the number of examples it spreads it over. Two consequences follow. First, in a window of inverse temperatures $β$ that widens as $\sqrt N$ the interference between archetypes is exponentially small in $Nβ$: the load drops out and the number of archetypes that can be learnt grows as $\exp(c\sqrt N)$. Second, learning fails differently: in the pairwise model through a spin-glass phase, in the exponential model through memorization: the softmax concentrates its weight on the training example closest to the input and returns it in place of the archetype. Seen from the input the softmax weights are those of the random energy model, and memorization is its freezing transition, at $β\sqrt{N(1-r^2)}\simeq\sqrt{2\ln M}$; it is measurable on a head whose keys are examples as the number of examples carrying weight. On MNIST the diagnostic follows the random-energy prediction; on fashion-MNIST it detects sub-clusters in the classes, and the best accuracy lies beyond the memorization threshold, where the softmax reads out a local sub-prototype.

cond-mat.dis-nn↗
arXiv2026

Spontaneous and Collective Learning in a Minimal Agent-Environment Model

How can information be extracted from data without explicit guidance or rewards? We investigate this question in a minimal setting where a classifying agent is exposed to a stream of structured data produced by a generative environment, and evolves by seeking consistency of its own labels in time. We find that imposing a simplicity bias on the classification rule can drive the dynamics toward label-coherent steady states, which we coin functional replicators. Remarkably, these persistent labeling rules align with the latent structure of the data. Using analytical tools from statistical mechanics, we characterize this spontaneous learning phase transition. Extending the analysis to a collective of agents that pool labels from one another, we show that interaction reshapes the learning phase boundary, in some regimes enabling spontaneous learning that no isolated agent can achieve, while suppressing it in others. Our minimal framework thus opens a route to decentralized learning through label exchange alone, requiring no access to the internal weights of other agents.

cond-mat.dis-nn↗