arXiv subjects

Marcel Binz

Publications and source records attributed to Marcel Binz.

At least 37 records · Page 2Linked to original sources

arXiv2023

Turning large language models into cognitive models

Large language models are powerful systems that excel at many tasks, ranging from translation to mathematical reasoning. Yet, at the same time, these models often show unhuman-like characteristics. In the present paper, we address this gap and ask whether large language models can be turned into cognitive models. We find that -- after finetuning them on data from psychological experiments -- these models offer accurate representations of human behavior, even outperforming traditional cognitive models in two decision-making domains. In addition, we show that their representations contain the information necessary to model behavior on the level of individual subjects. Finally, we demonstrate that finetuning on multiple tasks enables large language models to predict human behavior in a previously unseen task. Taken together, these results suggest that large, pre-trained models can be adapted to become generalist cognitive models, thereby opening up new research directions that could transform cognitive psychology and the behavioral sciences as a whole.

cs.CL↗

arXiv2023

Reinforcement Learning with Simple Sequence Priors

Everything else being equal, simpler models should be preferred over more complex ones. In reinforcement learning (RL), simplicity is typically quantified on an action-by-action basis -- but this timescale ignores temporal regularities, like repetitions, often present in sequential strategies. We therefore propose an RL algorithm that learns to solve tasks with sequences of actions that are compressible. We explore two possible sources of simple action sequences: Sequences that can be learned by autoregressive models, and sequences that are compressible with off-the-shelf data compression algorithms. Distilling these preferences into sequence priors, we derive a novel information-theoretic objective that incentivizes agents to learn policies that maximize rewards while conforming to these priors. We show that the resulting RL algorithm leads to faster learning, and attains higher returns than state-of-the-art model-free approaches in a series of continuous control tasks from the DeepMind Control Suite. These priors also produce a powerful information-regularized agent that is robust to noisy observations and can perform open-loop control.

cs.LG↗

arXiv2023

Meta-in-context learning in large language models

Large language models have shown tremendous performance in a variety of tasks. In-context learning -- the ability to improve at a task after being provided with a number of demonstrations -- is seen as one of the main contributors to their success. In the present paper, we demonstrate that the in-context learning abilities of large language models can be recursively improved via in-context learning itself. We coin this phenomenon meta-in-context learning. Looking at two idealized domains, a one-dimensional regression task and a two-armed bandit task, we show that meta-in-context learning adaptively reshapes a large language model's priors over expected tasks. Furthermore, we find that meta-in-context learning modifies the in-context learning strategies of such models. Finally, we extend our approach to a benchmark of real-world regression problems where we observe competitive performance to traditional learning algorithms. Taken together, our work improves our understanding of in-context learning and paves the way toward adapting large language models to the environment they are applied purely through meta-in-context learning rather than traditional finetuning.

cs.CL↗

arXiv2023

Meta-Learned Models of Cognition

Meta-learning is a framework for learning learning algorithms through repeated interactions with an environment as opposed to designing them by hand. In recent years, this framework has established itself as a promising tool for building models of human cognition. Yet, a coherent research program around meta-learned models of cognition is still missing. The purpose of this article is to synthesize previous work in this field and establish such a research program. We rely on three key pillars to accomplish this goal. We first point out that meta-learning can be used to construct Bayes-optimal learning algorithms. This result not only implies that any behavioral phenomenon that can be explained by a Bayesian model can also be explained by a meta-learned model but also allows us to draw strong connections to the rational analysis of cognition. We then discuss several advantages of the meta-learning framework over traditional Bayesian methods. In particular, we argue that meta-learning can be applied to situations where Bayesian inference is impossible and that it enables us to make rational models of cognition more realistic, either by incorporating limited computational resources or neuroscientific knowledge. Finally, we reexamine prior studies from psychology and neuroscience that have applied meta-learning and put them into the context of these new insights. In summary, our work highlights that meta-learning considerably extends the scope of rational analysis and thereby of cognitive theories more generally.

cs.AI↗

arXiv2022

Modeling Human Exploration Through Resource-Rational Reinforcement Learning

Equipping artificial agents with useful exploration mechanisms remains a challenge to this day. Humans, on the other hand, seem to manage the trade-off between exploration and exploitation effortlessly. In the present article, we put forward the hypothesis that they accomplish this by making optimal use of limited computational resources. We study this hypothesis by meta-learning reinforcement learning algorithms that sacrifice performance for a shorter description length (defined as the number of bits required to implement the given algorithm). The emerging class of models captures human exploration behavior better than previously considered approaches, such as Boltzmann exploration, upper confidence bound algorithms, and Thompson sampling. We additionally demonstrate that changing the description length in our class of models produces the intended effects: reducing description length captures the behavior of brain-lesioned patients while increasing it mirrors cognitive development during adolescence.

cs.LG↗

arXiv2022

Stochastic Gradient Descent Captures How Children Learn About Physics

As children grow older, they develop an intuitive understanding of the physical processes around them. They move along developmental trajectories, which have been mapped out extensively in previous empirical research. We investigate how children's developmental trajectories compare to the learning trajectories of artificial systems. Specifically, we examine the idea that cognitive development results from some form of stochastic optimization procedure. For this purpose, we train a modern generative neural network model using stochastic gradient descent. We then use methods from the developmental psychology literature to probe the physical understanding of this model at different degrees of optimization. We find that the model's learning trajectory captures the developmental trajectories of children, thereby providing support to the idea of development as stochastic optimization.

cs.LG↗

arXiv2022

Using cognitive psychology to understand GPT-3

We study GPT-3, a recent large language model, using tools from cognitive psychology. More specifically, we assess GPT-3's decision-making, information search, deliberation, and causal reasoning abilities on a battery of canonical experiments from the literature. We find that much of GPT-3's behavior is impressive: it solves vignette-based tasks similarly or better than human subjects, is able to make decent decisions from descriptions, outperforms humans in a multi-armed bandit task, and shows signatures of model-based reinforcement learning. Yet we also find that small perturbations to vignette-based tasks can lead GPT-3 vastly astray, that it shows no signatures of directed exploration, and that it fails miserably in a causal reasoning task. These results enrich our understanding of current large language models and pave the way for future investigations using tools from cognitive psychology to study increasingly capable and opaque artificial agents.

cs.CL↗

arXiv2022

Time-resolved Ultrafast Interatomic Coulombic Decay in Superexcited Sodium-doped Helium Nanodroplets

The autoionization dynamics of superexcited superfluid He nanodroplets doped with Na atoms is studied by extreme-ultraviolet (XUV) time-resolved electron spectroscopy. Following excitation into the higher-lying droplet absorption band, the droplet relaxes into the lowest metastable atomic $1s2s$ $^{1,\,3}$S states from which Interatomic Coulombic Decay (ICD) takes places either between two excited He atoms or between an excited He atom and a Na atom attached to the droplet surface. Four main ICD channels are identified and their time constants are determined by varying the delay between the XUV pulse and a UV pulse that ionizes the initial excited state and thereby quenches ICD. The time constants for the different channels all fall in the range $\sim$1~ps indicating that the ICD dynamics are mainly determined by the droplet environment. A periodic modulation of the transient ICD signals is tentatively attributed to the oscillation of the bubble forming around the localized He excitation. The ICD efficiency depends on the total number of excited states in a droplet rather than the density of excited states pointing to a collective enhancement of ICD.

physics.atm-clus↗

arXiv2021

Improved stabilization scheme for extreme ultraviolet quantum interference experiments

Interferometric pump-probe experiments in the extreme ultraviolet (XUV) domain are experimentally very challenging due to the high phase stability required between the XUV pulses. Recently, an efficient phase stabilization scheme was introduced for seeded XUV free electron lasers (FELs) combining shot-to-shot phase modulation with lock-in detection. This method stabilized the seed laser beampath on the fundamental ultraviolet wavelength to a high degree. Here, we extend this scheme including the stabilization of the XUV beampath, incorporating phase fluctuations from the FEL high gain harmonic generation process. Our analysis reveals a clear signal improvement with the new method compared to the previous stabilization scheme.

physics.atom-ph↗

arXiv2021

Effects of high pulse intensity and chirp in two-dimensional electronic spectroscopy of an atomic vapor

The effects of high pulse intensity and chirp on two-dimensional electronic spectroscopy signals are experimentally investigated in the highly non-perturbative regime using atomic rubidium vapor as clean model system. Data analysis is performed based on higher-order Feynman diagrams and non-perturbative numerical simulations of the system response. It is shown that higher-order contributions may lead to a fundamental change of the static appearance and beating-maps of the 2D spectra and that chirped pulses enhance or suppress distinct higher-order pathways. We further give an estimate of the threshold intensity beyond which the high-intensity effects become visible for the system under consideration.

physics.optics↗

arXiv2021

Unravelling the Full Relaxation Dynamics of Superexcited Helium Nanodroplets

The relaxation dynamics of superexcited superfluid He nanodroplets is thoroughly investigated by means of extreme-ultraviolet (XUV) femtosecond electron and ion spectroscopy complemented by time-dependent density functional theory (TDDFT). Three main paths leading to the emission of electrons and ions are identified: Droplet autoionization, pump-probe photoionization, and autoionization induced by re-excitation of droplets relaxing into levels below the droplet ionization threshold. The most abundant product of both droplet autoionization and photoionization is He$_2^+$, whereas the delayed appearance of He$^+$ is indicative of the ejection of excited He atoms from the droplets. The state-resolved time-dependent photoelectron spectra reveal that intermediate excited states of the droplets are populated in the course of the relaxation, terminating in the lowest-lying metastable singlet and triplet He atomic states. The slightly faster relaxation of the triplet state compared to the singlet state is in agreement with the simulation showing faster formation of a bubble around a He atom in the triplet state.

physics.atm-clus↗

arXiv2020

Tracking Attosecond Electronic Coherences Using Phase-Manipulated Extreme Ultraviolet Pulses

The recent development of ultrafast extreme ultraviolet (XUV) coherent light sources bears great potential for a better understanding of the structure and dynamics of matter. Promising routes are advanced coherent control and nonlinear spectroscopy schemes in the XUV energy range, yielding unprecedented spatial and temporal resolution. However, their implementation has been hampered by the experimental challenge of generating XUV pulse sequences with precisely controlled timing and phase properties. In particular, direct control and manipulation of the phase of individual pulses within a XUV pulse sequence opens exciting possibilities for coherent control and multidimensional spectroscopy, but has not been accomplished. Here, we overcome these constraints in a highly time-stabilized and phase-modulated XUV-pump, XUV-probe experiment, which directly probes the evolution and dephasing of an inner subshell electronic coherence. This approach, avoiding any XUV optics for direct pulse manipulation, opens up extensive applications of advanced nonlinear optics and spectroscopy at XUV wavelengths.

physics.atom-ph↗

arXiv2020

High-Gain Harmonic Generation with temporally overlapping seed pulses and application to ultrafast spectroscopy

Collinear double-pulse seeding of the High-Gain Harmonic Generation (HGHG) process in a free-electron laser (FEL) is a promising approach to facilitate various coherent nonlinear spectroscopy schemes in the extreme ultraviolet (XUV) spectral range. However, in collinear arrangements using a single nonlinear medium, temporally overlapping seed pulses may introduce nonlinear mixing signals that compromise the experiment at short time delays. Here, we investigate these effects in detail by extending the analysis described in a recent publication (Wituschek et al., Nat. Commun., 11, 883, 2020). High-order fringe-resolved autocorrelation and wave-packet interferometry experiments at photon energies > $23\,$eV are performed, accompanied by numerical simulations. It turns out that both the autocorrelation and the wave-packet interferometry data are very sensitive to saturation effects and can thus be used to characterize saturation in the HGHG process. Our results further imply that time-resolved spectroscopy experiments are feasible even for time delays smaller than the seed pulse duration.

physics.atom-ph↗

arXiv2019

Coherent multidimensional spectroscopy in the gas phase

Recent work applying multidimentional coherent electronic spectroscopy at dilute samples in the gas phase is reviewed. The development of refined phase-cycling approaches with improved sensitivity has opened-up new opportunities to probe even dilute gas-phase samples. In this context, first results of 2-dimensional spectroscopy performed at doped helium droplets reveal the femtosecond dynamics upon electronic excitation of cold, weakly-bound molecules, and even the induced dynamics from the interaction with the helium environment. Such experiments, offering well-defined conditions at low temperatures, are potentially enabling the isolation of fundamental processes in the excitation and charge transfer dynamics of molecular structures which so far have been masked in complex bulk environments.

physics.atm-clus↗

arXiv2019

Where Do Human Heuristics Come From?

Human decision-making deviates from the optimal solution, that maximizes cumulative rewards, in many situations. Here we approach this discrepancy from the perspective of bounded rationality and our goal is to provide a justification for such seemingly sub-optimal strategies. More specifically we investigate the hypothesis, that humans do not know optimal decision-making algorithms in advance, but instead employ a learned, resource-bounded approximation. The idea is formalized through combining a recently proposed meta-learning model based on Recurrent Neural Networks with a resource-bounded objective. The resulting approach is closely connected to variational inference and the Minimum Description Length principle. Empirical evidence is obtained from a two-armed bandit task. Here we observe patterns in our family of models that resemble differences between individual human participants.

cs.LG↗

arXiv2019

Emulating Human Developmental Stages with Bayesian Neural Networks

We compare the acquisition of knowledge in humans and machines. Research from the field of developmental psychology indicates, that human-employed hypothesis are initially guided by simple rules, before evolving into more complex theories. This observation is shared across many tasks and domains. We investigate whether stages of development in artificial learning systems are based on the same characteristics. We operationalize developmental stages as the size of the data-set, on which the artificial system is trained. For our analysis we look at the developmental progress of Bayesian Neural Networks on three different data-sets, including occlusion, support and quantity comparison tasks. We compare the results with prior research from developmental psychology and find agreement between the family of optimized models and pattern of development observed in infants and children on all three tasks, indicating common principles for the acquisition of knowledge.

cs.LG↗

arXiv2018

Coherent multidimensional spectroscopy of dilute gas-phase nanosystems

Two-dimensional electronic spectroscopy (2DES) is one of the most powerful spectroscopic techniques, capable of attaining a nearly complete picture of a quantum system including its couplings, quantum coherence properties and its real-time dynamics. While successfully applied to a variety of condensed phase samples, high precision experiments on isolated quantum systems in the gas phase have been so far precluded by insufficient sensitivity. However, such experiments are essential for a precise understanding of fundamental mechanisms and to avoid misinterpretations, e.g. as for the nature of quantum coherences in energy trans-port. Here, we solve this issue by extending 2DES to isolated nanosystems in the gas phase prepared by helium nanodroplet isolation in a molecular beam-type experiment. This approach uniquely provides high flexibility in synthesizing tailored, quantum state-selected model systems of single and many-body properties. For demonstration, we deduce a precise and conclusive picture of the ultrafast coherent dynamics in isolated high-spin Rb2 molecules and present for the first time a dynamics study of the system-bath interaction between a single molecule (here Rb3) and a superfluid helium environment. The results demonstrate the unique capacity to elucidate prototypical interactions and dynamics in tailored quantum systems and bridges the gap to experiments in ultracold quantum science.

physics.chem-ph↗

arXiv2018

Phase-synchronous undersampling in nonlinear spectroscopy

We introduce the concept of phase-synchronous undersampling in nonlinear spectroscopy. The respective theory is presented and validated experimentally in a phase-modulated quantum beat experiment by sampling high phase modulation frequencies with low laser repetition rates. The advantage of undersampling in terms of signal quality and reduced acquisition time is demonstrated and breakdown conditions are identified. The presented method is particularly beneficial for experimental setups with limited signal/detection rates.

physics.chem-ph↗