arXiv subjects

Elliot Nelson

Publications and source records attributed to Elliot Nelson.

15 recordsLinked to original sources

arXiv2025

Q-function Decomposition with Intervention Semantics with Factored Action Spaces

Many practical reinforcement learning environments have a discrete factored action space that induces a large combinatorial set of actions, thereby posing significant challenges. Existing approaches leverage the regular structure of the action space and resort to a linear decomposition of Q-functions, which avoids enumerating all combinations of factored actions. In this paper, we consider Q-functions defined over a lower dimensional projected subspace of the original action space, and study the condition for the unbiasedness of decomposed Q-functions using causal effect estimation from the no unobserved confounder setting in causal statistics. This leads to a general scheme which we call action decomposed reinforcement learning that uses the projected Q-functions to approximate the Q-function in standard model-free reinforcement learning algorithms. The proposed approach is shown to improve sample complexity in a model-based reinforcement learning setting. We demonstrate improvements in sample efficiency compared to state-of-the-art baselines in online continuous control environments and a real-world offline sepsis treatment environment.

cs.LG↗

arXiv2025

EpMAN: Episodic Memory AttentioN for Generalizing to Longer Contexts

Recent advances in Large Language Models (LLMs) have yielded impressive successes on many language tasks. However, efficient processing of long contexts using LLMs remains a significant challenge. We introduce \textbf{EpMAN} -- a method for processing long contexts in an \textit{episodic memory} module while \textit{holistically attending to} semantically relevant context chunks. The output of \textit{episodic attention} is then used to reweigh the decoder's self-attention to the stored KV cache of the context during training and generation. When an LLM decoder is trained using \textbf{EpMAN}, its performance on multiple challenging single-hop long-context recall and question-answering benchmarks is found to be stronger and more robust across the range from 16k to 256k tokens than baseline decoders trained with self-attention, and popular retrieval-augmented generation frameworks.

cs.CL↗

arXiv2024

Needle in the Haystack for Memory Based Large Language Models

Current large language models (LLMs) often perform poorly on simple fact retrieval tasks. Here we investigate if coupling a dynamically adaptable external memory to a LLM can alleviate this problem. For this purpose, we test Larimar, a recently proposed language model architecture which uses an external associative memory, on long-context recall tasks including passkey and needle-in-the-haystack tests. We demonstrate that the external memory of Larimar, which allows fast write and read of an episode of text samples, can be used at test time to handle contexts much longer than those seen during training. We further show that the latent readouts from the memory (to which long contexts are written) control the decoder towards generating correct outputs, with the memory stored off of the GPU. Compared to existing transformer-based LLM architectures for long-context recall tasks that use larger parameter counts or modified attention mechanisms, a relatively smaller size Larimar is able to maintain strong performance without any task-specific training or training on longer contexts.

cs.CL↗

arXiv2024

Larimar: Large Language Models with Episodic Memory Control

Efficient and accurate updating of knowledge stored in Large Language Models (LLMs) is one of the most pressing research challenges today. This paper presents Larimar - a novel, brain-inspired architecture for enhancing LLMs with a distributed episodic memory. Larimar's memory allows for dynamic, one-shot updates of knowledge without the need for computationally expensive re-training or fine-tuning. Experimental results on multiple fact editing benchmarks demonstrate that Larimar attains accuracy comparable to most competitive baselines, even in the challenging sequential editing setup, but also excels in speed - yielding speed-ups of 8-10x depending on the base LLM - as well as flexibility due to the proposed architecture being simple, LLM-agnostic, and hence general. We further provide mechanisms for selective fact forgetting, information leakage prevention, and input context length generalization with Larimar and show their effectiveness. Our code is available at https://github.com/IBM/larimar

cs.LG↗

arXiv2017

Classical branches and entanglement structure in the wavefunction of cosmological fluctuations

The emergence of preferred classical variables within a many-body wavefunction is encoded in its entanglement structure in the form of redundant classical information shared between many spatially local subsystems. We show how such structure can be generated via cosmological dynamics from the vacuum state of a massless field, causing the wavefunction to branch into classical field configurations on large scales. An accelerating epoch first excites the vacuum into a superposition of classical fields as well as a highly sensitive bath of squeezed super-horizon modes. During a subsequent decelerating epoch, gravitational interactions allow these modes to collect information about longer-wavelength fluctuations. This information disperses spatially, creating long-range redundant correlations in the wavefunction. The resulting classical observables, preferred basis for decoherence, and system/environment decomposition emerge from the translation invariant wavefunction and Hamiltonian, rather than being fixed by hand. We discuss the role of squeezing, the cosmological horizon scale, and phase information in the wavefunction, as well as aspects of wavefunction branching for continuous variables and in field theories.

quant-ph↗

arXiv2017

Classical Entanglement Structure in the Wavefunction of Inflationary Fluctuations

We argue that the preferred classical variables that emerge from a pure quantum state are determined by its entanglement structure in the form of redundant records: information shared between many subsystems. Focusing on the early universe, we ask how classical metric perturbations emerge from vacuum fluctuations in an inflationary background. We show that the squeezing of the quantum state for super-horizon modes, along with minimal gravitational interactions, leads to decoherence and to an exponential number of records of metric fluctuations on very large scales, $\lambda/\lambda_{\rm Hubble}>\Delta_\zeta^{-2/3}$, where $\Delta_\zeta\lesssim 10^{-5}$ is the amplitude of scalar metric fluctuations. This determines a preferred decomposition of the inflationary wavefunction into orthogonal "branches" corresponding to classical metric perturbations, which defines an inflationary entropy production rate and accounts for the emergence of stochastic, inhomogeneous spacetime geometry.

gr-qc↗

arXiv2017

Pulsar Timing Constraints on Physics Beyond the Standard Model

We argue that massive quantum fields source low-frequency long-wavelength metric fluctuations through the quantum fluctuations of their stress-energy, given reasonable assumptions about the analytic structure of its correlators. This can be traced back to the non-local nature of the gauge symmetry in General Relativity, which prevents an efficient screening of UV scales (what we call the cosmological non-constant problem). We define a covariant and gauge-invariant observable which probes line-of-sight spacetime curvature fluctuations on an observer's past lightcone, and show that current pulsar timing data constrains any massive particle to $m\lesssim 600$ GeV. This astrophysical bound severely limits the possibilities for physics beyond the standard model below the scale of quantum gravity.

hep-th↗

arXiv2016

Quantum Decoherence During Inflation from Gravitational Nonlinearities

We study the inflationary quantum-to-classical transition for the adiabatic curvature perturbation $\zeta$ due to quantum decoherence, focusing on the role played by squeezed-limit mode couplings. We evolve the quantum state $\Psi$ in the Schr\"odinger picture, for a generic cubic coupling to additional environment degrees of freedom. Focusing on the case of minimal gravitational interactions, we find the evolution of the reduced density matrix for a given long-wavelength fluctuation by tracing out the other (mostly shorterwavelength) modes of $\zeta$ as an environment. We show that inflation produces phase oscillations in the wave functional $\Psi[\zeta(\mathbf{x})]$, which suppress off-diagonal components of the reduced density matrix, leaving a diagonal mixture of different classical configurations. Gravitational nonlinearities thus provide a minimal mechanism for generating classical stochastic perturbations from inflation. We identify the time when decoherence occurs, which is delayed after horizon crossing due to the weak coupling, and find that Hubble-scale modes act as the decohering environment. We also comment on the observational relevance of decoherence and its relation to the squeezing of the quantum state.

gr-qc↗

arXiv2015

Cosmological bounds on TeV-scale physics and beyond

We study the influence of the fluctuations of a Lorentz invariant and conserved vacuum on cosmological metric perturbations, and show that they generically blow up in the IR. We compute this effect using the K\"all\'en-Lehmann spectral representation of stress correlators in generic quantum field theories, as well as the holographic bound on their entanglement entropy, both leading to an IR cut-off that scales as the fifth power of the highest UV scale (in Planck units). One may view this as analogous to the Heisenberg uncertainty principle, which is imposed on the phase space of gravitational theories by the Einstein constraint equations. The leading effect on cosmological observables come from anisotropic vacuum stresses which imply: i) any extension of the standard model of particle physics can only have masses (or resonances) $\lesssim$ 24 TeV, and ii) perturbative quantum field theory or quantum gravity becomes strongly coupled beyond a cut-off scale of $\Lambda\lesssim1$ PeV. Such a low cut-off is independently motivated by the Higgs hierarchy problem. This result, which we dub the cosmological non-constant problem, can be viewed as an extension of the cosmological constant (CC) problem, demonstrating the non-trivial UV-IR coupling and (yet another) limitation of effective field theory in gravity. However, it is more severe than the old CC problem, as vacuum fluctuations cannot be tuned to cancel due to the positivity of spectral densities or entropy. We thus predict that future advances in cosmological observations and collider technology will sandwich from above and below, and eventually discover, new (non-perturbative) physics beyond the Standard Model within the TeV-PeV energy range.

hep-th↗

arXiv2015

Non-local bispectra from super cosmic variance

We present examples of non-Gaussian statistics that can induce bispectra matching local and non-local (including equilateral) templates in biased sub-volumes. We find cases where the biasing from coupling to long wavelength modes affects only the power spectrum, only the bispectrum or both. Our results suggest that ruling out multi-field scenarios is quite difficult: some measurements of the scalar correlations, including the shape of the bispectrum, can be consistent with single-clock predictions even when cosmic variance from super-horizon modes is at work. Furthermore, if observations of the density perturbations rule out single-clock inflation, we will face a serious cosmic variance obstacle in drawing any further conclusions about the particle physics origin of the scalar fluctuations.

astro-ph.CO↗

arXiv2014

Fossilized Gravitational Wave Relic and Primordial Clocks

If long wavelength primordial tensor modes are coupled to short wavelength scalar modes, the scalar curvature two-point function will have an off-diagonal component. This `fossil' remnant is a signature of a mode coupling that cannot be achieved in single clock inflation. Any constraint on its presence allows a cross check of the relationship between the dynamical generation of the fluctuations and the evolution of the inflationary background. We use the example of non-Bunch Davies initial states for the tensor and scalar modes to demonstrate that physically reasonable fossils, consistent with current data, can be observable in the near future. We illustrate how the fossil off-diagonal power spectrum is a complementary probe to the squeezed limit bispectra of the scalar and tensor sectors individually. We also quantify the relation between the observable signal and the squeezed limit bispectrum for a general scalar-scalar-fossil coupling, and note the effect of superhorizon tensor modes on the anisotropy in scalar modes.

astro-ph.CO↗

arXiv2013

Statistical Naturalness and non-Gaussianity in a Finite Universe

We study the behavior of n-point functions of the primordial curvature perturbations, assuming our observed Universe is only a subset of a larger space with statistically homogeneous and isotropic perturbations. If the larger space has arbitrary n-point functions in a family of local type non-Gaussian statistics, sufficiently biased smaller volumes will have statistics from a `natural' version of that family with moments that are weakly non-Gaussian and ordered, regardless of the statistics of the original field. We also describe the effect of this bias on the shape of the bispectrum.

astro-ph.CO↗

arXiv2013

Cosmic Variance of the Spectral Index from Mode Coupling

We demonstrate that local, scale-dependent non-Gaussianity can generate cosmic variance uncertainty in the observed spectral index of primordial curvature perturbations. In a universe much larger than our current Hubble volume, locally unobservable long wavelength modes can induce a scale-dependence in the power spectrum of typical subvolumes, so that the observed spectral index varies at a cosmologically significant level (|\Delta ns | ~ O(0.04)). Similarly, we show that the observed bispectrum can have an induced scale dependence that varies about the global shape. If tensor modes are coupled to long wavelength modes of a second field, the locally observed tensor power and spectral index can also vary. All of these effects, which can be introduced in models where the observed non-Gaussianity is consistent with bounds from the Planck satellite, loosen the constraints that observations place on the parameters of theories of inflation with mode coupling. We suggest observational constraints that future measurements could aim for to close this window of cosmic variance uncertainty.

astro-ph.CO↗

arXiv2013

Non-Gaussian Mode Coupling and the Statistical Cosmological Principle

Local-type primordial non-Gaussianity couples statistics of the curvature perturbation \zeta on vastly different physical scales. Because of this coupling, statistics (i.e. the polyspectra) of \zeta in our Hubble volume may not be representative of those in the larger universe -- that is, they may be biased. The bias depends on the local background value of \zeta, which includes contributions from all modes with wavelength k ~< H_0 and is therefore enhanced if the entire post-inflationary patch is large compared with our Hubble volume. We study the bias to locally-measured statistics for general local-type non-Gaussianity. We consider three examples in detail: (i) the usual fNL, gNL model, (ii) a strongly non-Gaussian model with \zeta ~ \zeta_G^p, and (iii) two-field non-Gaussian initial conditions. In each scenario one may generate statistics in a Hubble-size patch that are weakly Gaussian and consistent with observations despite the fact that the statistics in the larger, post-inflationary patch look very different.

astro-ph.CO↗

arXiv2012

Higher time derivatives in effective equations of canonical quantum systems

Quantum-corrected equations of motion generically contain higher time derivatives, computed here in the setting of canonically quantized systems. The main example in which detailed derivations are presented is a general anharmonic oscillator, but conclusions can be drawn also for systems in quantum gravity and cosmology.

quant-ph↗