Transformer Neural-Network Quantum States for lattice models of spins and fermions: Application to the Ancilla Layer Model

Abstract

We introduce a variational wave function based on Neural-Network Quantum States (NQS) to study lattice systems whose local Hilbert space contains both spin and fermionic degrees of freedom. Our approach is based on the use of the Transformer architecture, which can naturally handle composite local Hilbert spaces through a tokenization procedure closely inspired by techniques from natural language processing. The neural network predicts a set of fermionic orbitals that depend on the spin configuration in a backflow-inspired manner. We apply the method to the one-dimensional Ancilla Layer Model, consisting of a chain of mobile spin-$1/2$ fermions coupled to a two-leg spin-$1/2$ ladder. For open boundary conditions, we achieve excellent quantitative agreement with Density Matrix Renormalization Group (DMRG) results across the full range of parameters considered. We find a phase in which the chain forms an effectively decoupled Luttinger liquid (LL), and a LL* phase with a distinct Fermi wavevector in which the mobile fermions are Kondo screened by one leg of the ladder, while the other leg forms the critical Bethe spin liquid. The LL* is the analog of the phase describing the pseudogap in two dimensions. We also find a Luther-Emery (LE) phase, where the LL* state becomes unstable toward the formation of a spin gap. The Transformer Ansatz maintains comparable accuracy for periodic boundary conditions, where tensor-network methods are computationally more demanding. Together, these findings establish Transformer-based NQS as an accurate and scalable variational framework for correlated lattice systems with composite local Hilbert spaces and highlight their potential for studying higher-dimensional models where boundary effects and heterogeneous local structures pose significant challenges.