arXiv subjects

Harry J. Stroud

Publications and source records attributed to Harry J. Stroud.

3 recordsLinked to original sources

arXiv2026

Sensing the properties of virtual objects without physical feedback

People who have interacted with simulated worlds and simulated objects in extended reality (XR) often have a sense that they can 'feel' the objects being simulated despite them not being physical. Our sense of touch is essential for how we 'feel' the physical world, however, there is an open question as to what it means to 'feel' virtual objects when interacting with them in immersive digital environments. In prior research, we have reported that participants often describe a subjective experience of 'feeling' the properties of simulated molecular objects while using interactive molecular dynamics in extended reality (iMD-XR), a field-based interaction paradigm for manipulating real-time simulations of molecular objects without haptic feedback. To better understand these subjective reports of 'feeling', we used a psychophysics approach to quantify the threshold at which participants perceive differences in the rigidity of simulated molecular objects (C$_{60}$ molecules) in iMD-XR. To evaluate this, we carried out experiments to compare the just-noticeable differences (JNDs) in two conditions: (1) via direct interaction with a real-time C$_{60}$ simulation, and (2) via observation-only$\unicode{x2013}$i.e. watching another person interacting with the simulations. Our findings show that direct interaction enabled participants to perceive more subtle rigidity differences of 11.5%, compared to 18.5% for observation-only. Furthermore, participants who undertook interaction first were better able to distinguish rigidity differences in the subsequent observation-only condition, suggesting that interaction trained participants to better perceive differences in molecular properties. These findings demonstrate a novel and flexible approach for sensing the properties of virtual objects in XR, and offer new insights into iMD-XR's potential in molecular research and education.

cs.HC↗

arXiv2026

NanoVer: An open-source framework for interactive molecular dynamics in extended reality (iMD-XR) on commodity hardware

This article outlines 'NanoVer', an open-source software framework which enables groups of people to co-habit the same virtual space and manipulate real-time MD (Molecular Dynamics) simulations of flexible 3D molecular structures with atomic-level precision as if they were tangible objects, an approach that we call 'interactive Molecular Dynamics in eXtended Reality' (iMD-XR). Distinct from our earlier iMD work that relied on tethered PC-VR systems with large graphics cards, NanoVer represents a change in philosophy, emphasizing compatibility with standalone mobile consumer XR hardware and corresponding software APIs. The NanoVer architecture enables multiple XR clients and/or Python clients to simultaneously communicate with a flexible server architecture that can carry out a range of tasks, including for example: recording iMD-XR sessions, static structure visualization, and MD trajectory visualization. NanoVer allows researchers, educators, and students to fluidly move between AR and VR environments, to explore creative new approaches to molecular research and education, including for example: molecular conformational sampling, protein-ligand binding, molecular psychophysics, training AI agents to sample molecular transitions, and a new interface which allows iMD-XR participants to sketch 3D conformational paths which automated agents can then follow. As an immersive platform that offers new ways to understand, engineer, communicate, and interact with dynamical behaviour at the nanoscale, NanoVer invites us to imagine new ways for combining human intelligence (e.g., spatial cognition and design reasoning) with machine intelligence. To expand NanoVer's accessibility, we have published a version to the Meta Horizon Store, for easy download by those with a Meta Quest 3/3S headset, to explore pre-recorded iMD-XR trajectory visualizations and set up their own multi-user system.

physics.chem-ph↗

arXiv2025

AI-Guided Molecular Simulations in VR: Exploring Strategies for Imitation Learning in Hyperdimensional Molecular Systems

Molecular dynamics (MD) simulations are a crucial computational tool for researchers to understand and engineer molecular structure and function in areas such as drug discovery, protein engineering, and material design. Despite their utility, MD simulations are expensive, owing to the high dimensionality of molecular systems. Interactive molecular dynamics in virtual reality (iMD-VR) has recently emerged as a "human-in-the-loop" strategy for efficiently navigating hyper-dimensional molecular systems. By providing an immersive 3D environment that enables visualization and manipulation of real-time molecular simulations running on high-performance computing architectures, iMD-VR enables researchers to reach out and guide molecular conformational dynamics, in order to efficiently explore complex, high-dimensional molecular systems. Moreover, iMD-VR simulations generate rich datasets that capture human experts' spatial insight regarding molecular structure and function. This paper explores the use of researcher-generated iMD-VR datasets to train AI agents via imitation learning (IL). IL enables agents to mimic complex behaviours from expert demonstrations, circumventing the need for explicit programming or intricate reward design. In this article, we review IL across robotics and Multi-agents systems domains which are comparable to iMD-VR, and discuss how iMD-VR recordings could be used to train IL models to interact with MD simulations. We then illustrate the applications of these ideas through a proof-of-principle study where iMD-VR data was used to train a CNN network on a simple molecular manipulation task; namely, threading a small molecule through a nanotube pore. Finally, we outline future research directions and potential challenges of using AI agents to augment human expertise in navigating vast molecular conformational spaces.

cs.LG↗