Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.34638

Intrinsic Goals for Autonomous Agents: Hierarchical Exploration Captures Future Volitional Action in Drosophila

Declan Kutscher·Reece Keller·Alyn Kirsch·Felix Pei·Aran Nayebi
View original source ↗Read source document ↗Export metadata ↓

Abstract

Animals have a remarkable capacity for autonomy---driven by intrinsic motivation---without external sensory cues or extrinsic objectives, yet the computational mechanisms behind this ability largely remain a mystery. We combine whole-brain recordings in the fruit fly (Drosophila melanogaster), which reveal neural representations of spontaneous exploratory decisions more than 10 seconds in advance, with a neuromechanical digital twin to test whether optimizing intrinsic goals yields realistically embodied artificial agents that quantitatively match fly behavior and internal representations of future volitional action during exploration. Our hierarchical policy selects from a discrete library of low-level motor primitives, consistent with ethological theories of behavioral syllables. We train agents without external reward using 20 intrinsic motivation algorithms spanning curiosity, entropy, and competency methods. The agents are evaluated in a virtual tethered-ball assay modeled on the fly experiment, comparing spontaneous locomotor statistics and internal representations and decoding future turn direction from kinematics and neural activity on the same timescales observed in flies. DIAYN, a hierarchical competency algorithm learning discrete policies that maximize behavioral entropy and the separability of induced sensory observations, achieves the strongest joint behavioral and neural prospective coding performance relative to flies. Its learned policies closely capture individual variability in fly exploration strategies, suggesting that this diversity serves to maximize state-space coverage at the population level. DIAYN's latent activity also closely captures the sum-difference rule for interhemispheric encoding of kinematics in biological flies. These results establish hierarchical, intrinsically motivated agents as a powerful framework for reverse-engineering animal autonomy.

Explore related subjects

q-bio.NCcs.AIcs.LGcs.RO

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Declan Kutscher, Reece Keller, Alyn Kirsch, Felix Pei, Aran Nayebi. 2026-10-25. Intrinsic Goals for Autonomous Agents: Hierarchical Exploration Captures Future Volitional Action in Drosophila. https://arxiv.org/abs/2610.34638

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Retrieval-Augmented Parkinson's Disease Classification from Resting-State Functional Connectivity

Altered resting-state functional connectivity (FC) is a candidate marker for the diagnosis of Parkinson's disease (PD). Publicly available PD datasets, however, are small and fragmented across sites, scanners and clinical criteria. Transfer learning mitigates data scarcity by reusing a model trained on larger cohorts, but represents their knowledge primarily through a single set of weights shared by all individuals. Motivated by precision neuroimaging, we propose a retrieval-augmented classification (RAC) framework that additionally places each target individual in the context of similar individuals from external cohorts. A graph convolutional autoencoder (GAE), trained on external PD and control cohorts, encodes these cohorts into a neuro-knowledge base. Each target individual is encoded using the pretrained GAE, and its nearest neighbors are retrieved from the external knowledge base. Their embeddings are combined with the target embedding using mean aggregation or attention for classification. A systematic comparison on the TaoWu cohort separated the contributions of encoder pretraining, retrieval, attention and partial fine-tuning. Pretraining improved mean classification accuracy from 60.5% to 66.0%. However, retrieval and attention did not consistently improve performance over the pretrained-only variants. Several retrieval-based configurations nevertheless showed modestly higher ROC-AUC than the randomly initialized baseline. Varying the number of retrieved neighbors and partially fine-tuning the encoder provided no consistent additional gains. Retrieval-neighborhood analysis further showed diagnostically mixed neighborhoods. These findings provide initial evidence that population-learned representations can be complemented by individual-specific retrieval in neuroimaging, although larger datasets are needed to determine when such contextual information improves classification.

q-bio.NC↗
arXiv2026

Phase response approach to networked neural activity models with delays

Networks of delay-coupled neural mass models alternate between synchrony and anti-phase locking as the delay between brain regions increases. The origin of these delay-dependent bands has remained unclear. Here, we show that a single function, the phase interaction function $H$ of an isolated node, predicts where the bands occur and explains why they repeat. For a Wilson--Cowan model with intra-nodal delay $τ$ and inter-nodal delay $ρ$, we compute the limit cycle and phase response, and hence $H$, by harmonic balance. Each band boundary is a zero of $H'$ shifted by the phase lag $Ωρ$, so the bands repeat in $ρ$ with the oscillation period. Simulations of the full delay-differential networks confirm these predictions, from two nodes, where predicted frequencies agree to second order in the coupling strength, to rings, where the theory also identifies coexisting stable twisted states that random initial conditions rarely reach. At weak coupling, the delays at which synchrony gains or loses stability are set by the node dynamics alone, whereas stable anti-phase states require a bipartite network. For a macaque connectome, metastability peaks at the edges of the synchrony bands, where the stability margin of phase-locking vanishes. The reduction is accurate for coupling strengths well below the slowest Floquet exponent of the node, beyond which a phase--amplitude description is needed. More broadly, vanishing phase-locking stability provides a simple route to metastability in delay-coupled oscillator networks and a candidate mechanism for the flexible coordination of brain dynamics.

q-bio.NC↗
arXiv2026

On the stability of learning

Learning algorithms are usually analyzed under the assumption that informative gradients are continuously available. But this is not always true---learning signals in biological neural networks, for example, may be sparse, local, and only intermittently available at any given synapse. In this work, we show that if gradient information is noisy and only intermittently available, learning may be unstable. In particular, even if average learning dynamics converge to the correct optimum, variance in learning may be asymptotically infinite. We explore this point in a number of analytically tractable settings, including single-layer and multi-layer linear networks, and derive explicit conditions under which learning is stable. These conditions complement, but are conceptually distinct from, existing conditions related to the stability of stochastic gradient descent. Our work demonstrates that unbiased credit assignment is not enough for reliable learning; sufficiently frequent and informative feedback is also required, which yields an additional constraint on models of biologically plausible learning.

q-bio.NC↗