Searcharxiv⌕ Search

arXiv subjects

Vijay Balasubramanian

Publications and source records attributed to Vijay Balasubramanian.

At least 19 recordsLinked to original sources

Stochastic Thermodynamics of Associative Memory

Dense Associative Memory networks (DenseAMs) unify several popular paradigms in Artificial Intelligence (AI), such as Hopfield Networks, transformers, and diffusion models, while casting their computational properties into the language of dynamical systems and energy landscapes. This formulation provides a natural setting for studying thermodynamics and computation in neural systems, because DenseAMs are simultaneously simple enough to admit analytic treatment and rich enough to implement nontrivial computational function. Aspects of these networks have been studied at equilibrium and at zero temperature, but the thermodynamic costs associated with their operation out of equilibrium are largely unexplored. Here, we define the thermodynamic entropy production associated with the operation of such networks, and study polynomial DenseAMs at intermediate memory load. At large system sizes and intermediate and low load, we use dynamical mean field theory to characterize out-of-equilibrium properties, work requirements, and memory transition times when driving the system with corrupted memories. We characterize a failure mode of higher order networks not observed at zero temperature. Further, we develop a method for calculating work and power costs in the mean field limit. Finally, we find tradeoffs between entropy production, memory retrieval accuracy, and operation speed.

cond-mat.stat-mech↗

Superstars, quantum wormholes and von Neumann algebras

We entangle heavy 1/2-BPS states in two copies of $\mathcal{N}=4$ SU($N$) Super-Yang-Mills (SYM) theory. Each copy is separately dual to an extremal, supersymmetric, singular black hole geometry, in which the horizon coincides with a singularity. General ideas in the AdS/CFT correspondence suggest that the entanglement between the two copies should lead to some form of connectivity between the two geometries---a kind of wormhole. We use von Neumann algebra methods in the SYM theory to diagnose the possible existence of such wormholes.

hep-th↗

Time Evolution of Multi-Party Entanglement Signals

We study the real-time dynamics of multi-party entanglement signals in chaotic quantum many-body systems including but not necessarily restricted to holographic conformal field theories. We find that scrambling dynamics generates multiparty entanglement with rich structure including: (a) qualitatively different dynamical behaviours for different signals, likely reflecting different dynamics for different kinds of entanglement patterns, (b) discontinuities indicating dynamical phase transitions in the entanglement structure, (c) transient and non-monotonic multiparty entanglement, and (d) periods during which the extensive entanglement of some regions is entirely multipartite. Our main technical tool is the membrane theory of entanglement dynamics.

hep-th↗

The Dichotomy Between Pattern Recognition and Step-by-Step Reasoning

We argue that pattern recognition and step-by-step reasoning are two ends of a spectrum. A large language model (LLM) learns to reason step-by-step when data is structured such that the next token depends on a small amount of preceding context. Inference in LLMs resembles pattern recognition when the next token depends on a large amount of preceding context. If the next token depends on only the $c$ most recent tokens, reasoning traces are paths on a De Bruijn graph whose nodes are $c$-length contexts and edges are next-token transitions between contexts. The set of reasoning traces of a task forms a directed acyclic subgraph of the De Bruijn graph. An LLM that has learned all edges of this subgraph can compose them to solve longer, unseen tasks, i.e., it reasons step-by-step. We prove that the number of edges is vanishingly small compared to the number of reasoning traces. Empirically, the number of training samples a transformer needs is a power law in the number of edges, so learning to reason step-by-step is sample efficient. We can induce De Bruijn structure in any task by maintaining a ``state'' that makes future reasoning independent of the past. The frequency of states in the reasoning trace determines $c$. We show, by fine-tuning Qwen2.5-1.5B-Instruct to solve equations and answer questions about stories, that frequent states (small $c$) result in higher accuracy but greater fragility to perturbations at test time. LLMs trained with a large $c$ are only as good as models that perform pattern recognition without reasoning. A moderate density of states balances accuracy and robustness. We show that real-world data has De Bruijn structure: Qwen3-14B and Qwen3-32B retain over 75% of their accuracy on GSM8K, MATH-500 and GPQA-Diamond when attention is restricted to a sliding window less than 15% as long as the full reasoning trace.

cs.LG↗

The entropy of finite gravitating regions

We develop a formalism for calculating the entanglement entropy of an arbitrary spatial region of a gravitating spacetime at a moment of time symmetry. The crucial ingredient is a path integral over embeddings of the region into the overall spacetime, interpretable as a sum over the edge modes associated with the region. We argue that this entanglement entropy can be interpreted, to leading order in Newton's constant, as the minimal generalized entropy among all regions that enclose it. This suggests a notion of ``terrestrial holography'' where regions of space can encode larger ones, in contrast to the standard form of holography, in which degrees of freedom on the celestial sphere at the boundary of the universe encode the interior.

hep-th↗

Topological entanglement entropy in 3D gravity

We derive an entropy formula for a recently proposed tensor network model of states in 3D gravity which satisfy the constraint equations of diffeomorphism invariance, and including the effects of matter. After accounting for the invertibility of the spacetime metric, we argue that this formula matches the quantum extremal surface formula (for small but finite $G_N$), and is not restricted to time symmetric spacetimes. We derive the result via canonical quantization without relying on path integrals, and discuss an extension to compact bulk subregions.

hep-th↗

Entanglement entropy in topological tensor networks

We derive an entropy formula for recently proposed tensor network models which prepare diffeomorphism invariant states of topological quantum field theories with non-compact and/or continuous gauge groups. We show that our entropy formula generalizes the notion of ``topological entanglement entropy'' to incorporate the infinite number of particle-like excitations in such theories. When our networks are endowed with gauge group $\mathrm{SL}(2,\mathbb{R})$, we can interpret them as models of three-dimensional gravity with small Newton's constant and possibly non-invertible metrics.

hep-th↗

Observing Spacetime

Complex states of quantum gravity in flat and AdS gravity can have features that are inaccessible to classical asymptotic observers. The missing information appears to such observers to be hidden behind a horizon or in a baby universe. Here we use the gravitational path integral to ask whether quantum observables can access the hidden data. We show that generic probes give a universal result and contain no information about the state. However, a probe appropriately fine-tuned to the state can give a large signal because of novel wormhole saddles in the path integral. Thus, in these settings, asymptotic observers cannot easily determine the state of the universe, but can check a proposal for it. Using these fine-tuned probes we show that an asymptotic observer can detect information hidden in a disconnected baby universe. Furthermore we show that the state of a two-boundary black hole can be detected using Lorentzian operators localised on just one of the boundaries.

hep-th↗

A Nonperturbative Toolkit for Quantum Gravity

We propose a method for demonstrating equivalences beyond the saddlepoint approximation between quantities in quantum gravity that are defined by the Euclidean path integral, without assumptions about holographic duality. The method involves three ingredients: (1) a way of resolving the identity with an overcomplete basis of microstates that is under semiclassical control, (2) a drastic simplification of the sum over topologies in the limit where the basis is infinitely overcomplete, and (3) a way of cutting and splicing geometries to demonstrate equality between two different gravitational path integrals even if neither can be explicitly computed. We illustrate our methods by giving a general argument that the thermal partition function of quantum gravity with two boundaries factorises. One implication of our results is that universes containing a horizon can sometimes be understood as superpositions of horizonless geometries entangled with a closed universe.

hep-th↗

How to Count States in Gravity

Gibbons and Hawking proposed that the Euclidean gravity path integral with periodic boundary conditions in time computes the thermal partition sum of gravity. As a corollary, they argued that a derivative of the associated free energy with respect to the Euclidean time period computes gravitational entropy. Why is this interpretation correct? That is, why does this path integral compute a trace over the Hilbert space? Here, we show that the quantity computed by the Gibbons-Hawking path integral is equal to an {\it a priori} different object -- an explicit thermal trace over the Hilbert space spanned by states produced by the Euclidean gravity path integral. This follows in two ways: (a) if the Hilbert space with two boundaries factorizes into a product of two single boundary Hilbert spaces, as we have previously shown; and (b) via explicit resolution of the trace by a spanning basis of states. We similarly show how a replicated Euclidean gravity path integral with a single periodic boundary computes a Hilbert space trace of powers of the density matrix, explaining why this approach computes the entropy of states entangled between two universes.

hep-th↗

Wigner negativity in Krylov space and emergent semiclassicality

We propose that the Krylov basis gives a semiclassical representation of dynamics in general large-$N$, complex, many-body systems. As a probe of this semiclassicality, we study the growth of Wigner negativity -- a measure of the complexity of classical simulation -- under time evolution in Krylov space in several solvable models. We begin with 2d CFTs, initially in either the vacuum or the thermofield double state on a line excited by a primary operator. In both cases, Wigner negativity remains an $O(1)$ constant and does not grow at late times, indicating approximately classical dynamics in the Krylov basis. We then study random matrix theory with the maximally entangled state between two copies as the initial state. For general one-cut matrix models, we argue that Wigner negativity in the Krylov basis grows as $t^{1/2}$ at large $O(1)$ times but does not scale with the Hilbert space dimension, thus indicating semiclassical dynamics in Krylov space. Finally, in the double-scaled SYK model, we find an approximately classical phase (constant negativity) at early times and a semiclassical phase ($t^{1/2}$ growth) at late times. In all these examples, Wigner negativity either remains constant or grows slowly, demonstrating emergent semiclassicality of dynamics in Krylov space.

hep-th↗

Constraints on four-party entanglement in holography

We show that in pure time-reflection-symmetric holographic states several known four-party entanglement signals vanish unless the triple information $I_3$ is non-zero. In this sense, our results show that $I_3$ is the strongest known signal of the presence of quadripartite entanglement. Additionally, $I_3$ quantitatively bounds all four-party entanglement signals built from the multi-entropy. However, the residual entropy $Q_4$, also a measure of four-party entanglement, is not bounded by $I_3$, although $I_3=0$ does imply $Q_4=0$ for holographic states (except on a set of measure zero for which $Q_4$ is ill-defined).

hep-th↗

How does Chain of Thought decompose complex tasks?

Many language tasks can be modeled as classification problems where a large language model (LLM) is given a prompt and selects one among many possible answers. We show that the classification error in such problems scales as a power law in the number of classes. This has a dramatic consequence: the prediction error can be reduced substantially by splitting the overall task into a sequence of smaller classification problems, each with the same number of classes ("degree"). This tree-structured decomposition models chain-of-thought (CoT). It has been observed that CoT-based predictors perform better when they "think", i.e., when they develop a deeper tree, thus decomposing the problem into a larger number of steps. We identify a critical threshold for the degree, below which thinking is detrimental, and above which there exists an optimal depth that minimizes the error. It is impossible to surpass this minimal error by increasing the depth of thinking.

cs.LG↗

A simple model of co-emergence of grid and place fields

Grid cells in the medial entorhinal cortex and place cells in the hippocampus together support spatial navigation. The two regions are reciprocally connected, and there is a chicken-and-egg problem for how both arise and reinforce each other during development. Current computational accounts either derive one type from the other or use network dynamics to model the emergence of one type in isolation. We introduce a unified recurrent network model that instantiates Dale's Law (every neuron is either excitatory or inhibitory), and is trained to predict the next sensory observation from masked previous sensory observations and egocentric motion. To our knowledge, this is the first single-objective model in which grid and place cells co-emerge without supervision of either type, or reliance on pre-existing spatial-cell representations. The two kinds of spatial codes coexist across 1,000 different training configurations, with their balance set by the amount of sensory noise and masking. Without retraining, the network qualitatively reproduces experimentally observed grid fragmentation in hairpin mazes, grid merging after wall removal, lattice alignment across connected rooms, locally ordered 3D fields observed in freely flying bats, as well as the developmental order in which place cells precede grid cells. We interpret these results in terms of two complementary encoding pressures within a single sensory-prediction objective: (1) correcting errors or reconstructing missing components of sensory observations, and (2) prediction of the next sensory state during navigation. Our results suggest a circuit-level account of the co-emergence of grid and place cells, and experimentally testable predictions for the two kinds of spatial codes.

q-bio.NC↗

Detector-based measurement-induced state updates in AdS/CFT

Conventional understandings of quantum theory hold that measurements change the state of an observed system following the Lüders update rule. Textbooks describe the application of this idea to non-relativistic systems, but extensions to relativistic and gravitating systems encounter subtleties. One consistent approach is via detector-based measurements. We study the effects of such measurements in a CFT with a holographic dual. We work out the boundary space-time regions associated to a Lüders update and how the outcome extends to modifications of the bulk gravity state. We explore information-theoretic consequences of this picture, and relate the information extracted by a measurement to updates of the semiclassical parameters of the bulk state.

hep-th↗

The Nonperturbative Hilbert Space of Quantum Gravity With One Boundary

We discuss a basis for the nonperturbative Hilbert space of quantum gravity with one asymptotic boundary. We use this basis to show that the Hilbert space for gravity with two disconnected boundaries factorizes into a product of two copies of the single boundary Hilbert space.

hep-th↗

Contrastive learning in tunable dynamical systems

We generalize the theory of supervised contrastive learning, previously applied to physical systems at equilibrium or steady state, to systems following any dynamics described by coupled ordinary differential equations. We show that if physical dynamics break time reversal symmetry, gradient descent on a cost function embodying the desired behavior cannot be achieved with a scalable process, even in principle. We therefore introduce Probably Approximately Right (PAR) learning processes, composed of a local contrastive learning rule and a scalable supervision protocol. We show that approximate, local supervision with forward propagation of the error signal can be used to successfully train several tunable models of physical dynamics inspired by examples in biological and machine learning.

cond-mat.dis-nn↗

When and Where: A Model Hippocampal Network Unifies Formation of Time Cells and Place Cells

Hippocampal place and time cells encode spatial and temporal aspects of experience. Both have the same neural substrate, but have been modeled as having different functions and mechanistic origins, place cells as continuous attractors, and time cells as leaky integrators. Here, we show that both types emerge from two dynamical regimes of a single recurrent network (RNN) modeling hippocampal CA3 as a predictive autoencoder. The network receives simulated, partially occluded ``experience vectors" containing spatial patterns (location-specific activity sampled during environmental traversal) and/or temporal patterns (correlated activity pairs separated by ``void" intervals), and is trained to reconstruct missing input. During spatial navigation, the network generates stable attractor-like place fields. But trained on temporally structured inputs, the network produces sequentially broadened fields, recapitulating time cells. By varying spatio-temporal input patterning, we observe hidden units transition smoothly between time cell-like and place cell-like representations. These results suggest a shared origin, but task-driven difference, between place and time cells.

q-bio.NC↗