arXiv · 2611.10501
Sampling Is Not Curiosity: Why LLM Agents Should Investigate
Abstract
LLM agents are increasingly used for open-ended tasks, very common in research scenarios where they are required to explore, investigate, or act curiously. At inference time, an LLM is a fixed map from context to next-token distribution, the pretrained weights determine the function, and only its conditioning changes as context accumulates. Mechanisms like temperature, best-of-N, tree search, or RAG vary how the distribution is sampled or what it is conditioned on, but do not update the agent's model of the environment. We argue that current LLM agents do not explore curiously; they explore stochastically. From how curiosity is understood in the RL literature, it requires a predictive structure over how the environment responds to the agent's actions, which improves online from current interaction, drives action selection through its predicted improvement, and persists beyond the context window. Additionally, in its open-ended form, the agent should intrinsically originate what to investigate. Thus, we propose a framework defining curiosity in LLM agents, reserving the term for systems that satisfy it. In doing so, we identify the research directions its implementation opens up.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Alfonso Amayuelas, Piotr Piękos, Xin Eric Wang, William Yang Wang, Jürgen Schmidhuber, Alex Pentland. 2026-11-07. Sampling Is Not Curiosity: Why LLM Agents Should Investigate. https://searcharxiv.com/abs/2611.10501
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.