arXiv subjects

Robin A. Heinonen

Publications and source records attributed to Robin A. Heinonen.

11 recordsLinked to original sources

arXiv2026

Smart strategies to navigate turbulent odor plumes reorienting to local wind

Olfactory search in turbulent environments is a sensorimotor problem that many animals solve with remarkable efficiency, yet replicating this ability in artificial systems is an enduring challenge because detections are intermittent and wind direction fluctuates strongly, rendering standard search strategies unreliable. We introduce a wind-relative reinforcement-learning framework in which an agent navigates a turbulent plume with a single internal variable --- the elapsed time since the last odor detection --- and selects actions relative to a locally estimated wind direction filtered through an exponential memory kernel. Policies are trained and evaluated in direct numerical simulations of turbulence, capturing the multi-scale characteristics of velocity and odor fields in natural environments, both in the presence and absence of a mean wind. In a mild mean wind, the learned policy outperforms well-known biomimetic policies such as cast-and-surge regardless of the wind memory time, yet adapts its movement pattern to wind-estimation quality. In isotropic turbulence, performance peaks at an intermediate wind memory time, identifying temporal wind integration as a regime-dependent resource. Our results highlight the importance of developing and validating olfactory-navigation strategies under realistic turbulent conditions, and offer a compact design principle for minimal robotic olfactory navigation and testable predictions for biological search behavior.

physics.flu-dyn↗

arXiv2026

Clock-state olfactory search in turbulent flows using Q-learning: The geometry of plume recovery

Finding an odor source in a turbulent flow requires effectively leveraging the history of olfactory observations into a robust navigation strategy. In this work, we use tabular Q-learning to train an olfactory search agent with a minimal memory of past observations: only a running clock since the last whiff. This agent learns an interpretable strategy to recover the plume which combines well-known behaviors observed in insects: surging, casting, and a return downwind. While achieving good performance on data from direct numerical simulations of turbulence, the agent is limited by an inability to adapt its strategy to the local intermittency level; we show that providing more flexibility improves robustness.

physics.bio-ph↗

arXiv2026

Olfactory pursuit: catching a moving odor source in complex flows

Locating and intercepting a moving target from possibly delayed, intermittent sensory signals is a paradigmatic problem in decision-making under uncertainty, and a fundamental challenge for, e.g., animals seeking prey or mates and autonomous robotic systems. Odor signals are intermittent, strongly mixed by turbulent-like transport, and typically lag behind the true target position, thereby complicating localization. Here, we formulate olfactory pursuit as a partially observable Markov decision process in which an agent maintains a joint belief over the target's position and velocity. Using a discrete run-and-tumble model, we compute quasi-optimal policies by numerically solving the Bellman equation and benchmark them against well-established information-theoretic strategies such as Infotaxis. We show that purely exploratory policies are near-optimal when the target frequently reorients, but fail dramatically when the target exhibits persistent motion. We thus introduce a computationally efficient hybrid policy that combines the information-gain drive of Infotaxis with a "greedy" value function derived from an associated fully observable control problem. Our heuristic achieves near-optimal performance across all persistence times and substantially outperforms purely exploratory approaches. Moreover, our proposal demonstrates strong robustness even in more complex search scenarios, including continuous run-and-tumble prey motion with moderate persistence time, model mismatch, and more accurate plume dynamics representation. Our results identify predictive inference of target motion as the key ingredient for effective olfactory pursuit and provide a general framework for search in information-poor, dynamically evolving environments.

cs.RO↗

arXiv2026

Asymptotics of solutions to the linear search problem

The exact leading asymptotics of solutions to the symmetric linear search problem are obtained for any positive probability density on the real line with a monotonic, sufficiently regular tail. A similar result holds for densities on a compact interval.

math.OC↗

arXiv2025

TURB-Smoke. A database of Lagrangian pollutants emitted from point-sources and dispersed in turbulent flows

Identifying the location and characteristics of pollution sources in turbulent flows is challenging, especially for environmental monitoring and emergency response, due to sparse, stochastic, and infrequent cue detection. Even in idealized settings, accurately modeling these phenomena remains highly complex, with realistic representations typically achievable only through experimental or simulation-based data. We introduce TURB-Smoke, a cutting-edge numerical dataset designed for investigating odor and contaminant dispersion in turbulent environments with and without mean wind. Generated via direct numerical simulations of the fully resolved three-dimensional Navier-Stokes equations, TURB-Smoke tracks hundreds of millions of Lagrangian particles released from five distinct point sources in fully developed turbulence, thus providing a reliable ground-truth framework for developing and evaluating source-tracking strategies using stationary sensors or mobile agents in realistic flows. Each particle's trajectory is continuously tracked on many characteristic turbulence timescales, recording both the position and the local flow velocity. Additionally, we provide coarse-grained concentration fields in 3D and in quasi-2D slabs containing the source, ideal for quickly testing and optimizing search algorithms under varying flow conditions.

physics.flu-dyn↗

arXiv2025

Policy heterogeneity improves collective olfactory search in 3-D turbulence

We investigate the role of policy heterogeneity in enhancing the olfactory search capabilities of cooperative agent swarms operating in complex, real-world turbulent environments. Using odor fields from direct numerical simulations of the Navier-Stokes equations, we demonstrate that heterogeneous groups, with exploratory and exploitative agents, consistently outperform homogeneous swarms where the exploration-exploitation tradeoff is managed at the individual level. Our results reveal that policy diversity enables the group to reach the odor source more efficiently by mitigating the detrimental effects of spatial correlations in the signal. These findings provide new insights into collective search behavior in biological systems and offer promising strategies for the design of robust, bioinspired search algorithms in engineered systems.

physics.flu-dyn↗

arXiv2025

Many wrong models approach to localize an odor source in turbulence with static sensors

The problem of locating an odor source in turbulent flows is central to key applications such as environmental monitoring and disaster response. We address this challenge by designing an algorithm based on Bayesian inference, which uses odor measurements from an ensemble of static sensors to estimate the source position through a stochastic model of the environment. The problem is difficult because of the multiscale and out-of-equilibrium properties of turbulent transport, which lack accurate analytical and phenomenological modeling, thus preventing a guaranteed convergence for Bayesian approaches. To overcome the risk of relying on a single unavoidably wrong model approximation, we propose a method to rank ``many wrong models'' and to blend their predictions. We evaluated our \emph{weighted Bayesian update} algorithm by its ability to estimate the source location with predefined accuracy and/or within a specified time frame and compare it to standard Monte Carlo sampling methods. To demonstrate the robustness and potential applications of both approaches under realistic environmental conditions, we use high-quality direct numerical simulations of the Navier-Stokes equations to mimic the turbulent transport of odors in presence of a strong mean wind. Despite minimal prior information on the source and environmental conditions, our proposed approach consistently proves to be more accurate, reliable, and robust than Monte Carlo methods, thus showing promise as a new tool for addressing the odor source localization problem in real-world scenarios.

physics.flu-dyn↗

arXiv2025

Optimal trajectories for Bayesian olfactory search in turbulent flows: the low information limit and beyond

In turbulent flows, tracking the source of a passive scalar cue requires exploiting the limited information that can be gleaned from rare, stochastic encounters with the cue. When crafting a search policy, the most challenging and important decision is what to do in the absence of an encounter. In this work, we perform high-fidelity direct numerical simulations of a turbulent flow with a stationary source of tracer particles, and obtain quasi-optimal policies (in the sense of minimal average search time) with respect to the empirical encounter statistics. We study the trajectories under such policies and compare the results to those of the infotaxis heuristic. In the presence of a strong mean wind, the optimal motion in the absence of an encounter is zigzagging (akin to the well-known insect behavior "casting") followed by a return to the starting location. The zigzag motion generates characteristic $t^{1/2}$ scaling of the rms displacement envelope. By passing to the limit where the probability of detection vanishes, we connect these results to the classical linear search problem and derive an estimate of the tail of the arrival time pdf as a stretched exponential, which agrees with Monte Carlo results. We also discuss what happens as the wind speed decreases.

physics.flu-dyn↗

arXiv2025

Exploring Bayesian olfactory search in realistic turbulent flows

The problem of tracking the source of a passive scalar in a turbulent flow is relevant to flying insect behavior and several other applications. Extensive previous work has shown that certain Bayesian strategies, such as "infotaxis," can be very effective for this difficult "olfactory search" problem. More recently, a quasi-optimal Bayesian strategy was computed under the assumption that encounters with the scalar are independent. However, the Bayesian approach has not been adequately studied in realistic flows which exhibit spatiotemporal correlations. In this work, we perform direct numerical simulations (DNS) of an incompressible flow at $\mathrm{Re}_λ\simeq150,$ while tracking Lagrangian particles which are emitted by a point source and imposing a uniform mean flow with several magnitudes (including zero). We extract the spatially-dependent statistics of encounters with the particles, which we use to build Bayesian policies, including generalized ("space-aware") infotactic heuristics and quasi-optimal policies. We then assess the relative performance of these policies when they are used to search using scalar cue data from the DNS, and in particular study how this performance depends on correlations between encounters. Among other results, we find that quasi-optimal strategies continue to outperform heuristics in the presence of strong mean flow but fail to do so in the absence of a mean flow. We also explore how to choose optimal search parameters, including the frequency and threshold concentration of observation.

physics.flu-dyn↗

arXiv2023

Optimal policies for Bayesian olfactory search in turbulent flows

In many practical scenarios, a flying insect must search for the source of an emitted cue which is advected by the atmospheric wind. On the macroscopic scales of interest, turbulence tends to mix the cue into patches of relatively high concentration over a background of very low concentration, so that the insect will only detect the cue intermittently and cannot rely on chemotactic strategies which simply climb the concentration gradient. In this work, we cast this search problem in the language of a partially observable Markov decision process (POMDP) and use the Perseus algorithm to compute strategies that are near-optimal with respect to the arrival time. We test the computed strategies on a large two-dimensional grid, present the resulting trajectories and arrival time statistics, and compare these to the corresponding results for several heuristic strategies, including (space-aware) infotaxis, Thompson sampling, and QMDP. We find that the near-optimal policy found by our implementation of Perseus outperforms all heuristics we test by several measures. We use the near-optimal policy to study how the search difficulty depends on the starting location. We discuss additionally the choice of initial belief and the robustness of the policies to changes in the environment. Finally, we present a detailed and pedagogical discussion about the implementation of the Perseus algorithm, including the benefits -- and pitfalls -- of employing a reward shaping function.

physics.flu-dyn↗

arXiv2023

Deep reinforcement learning for the olfactory search POMDP: a quantitative benchmark

The olfactory search POMDP (partially observable Markov decision process) is a sequential decision-making problem designed to mimic the task faced by insects searching for a source of odor in turbulence, and its solutions have applications to sniffer robots. As exact solutions are out of reach, the challenge consists in finding the best possible approximate solutions while keeping the computational cost reasonable. We provide a quantitative benchmarking of a solver based on deep reinforcement learning against traditional POMDP approximate solvers. We show that deep reinforcement learning is a competitive alternative to standard methods, in particular to generate lightweight policies suitable for robots.

cs.RO↗