arXiv subjects

Rhoslyn Roebuck Williams

Publications and source records attributed to Rhoslyn Roebuck Williams.

7 recordsLinked to original sources

arXiv2026

Sensing the properties of virtual objects without physical feedback

People who have interacted with simulated worlds and simulated objects in extended reality (XR) often have a sense that they can 'feel' the objects being simulated despite them not being physical. Our sense of touch is essential for how we 'feel' the physical world, however, there is an open question as to what it means to 'feel' virtual objects when interacting with them in immersive digital environments. In prior research, we have reported that participants often describe a subjective experience of 'feeling' the properties of simulated molecular objects while using interactive molecular dynamics in extended reality (iMD-XR), a field-based interaction paradigm for manipulating real-time simulations of molecular objects without haptic feedback. To better understand these subjective reports of 'feeling', we used a psychophysics approach to quantify the threshold at which participants perceive differences in the rigidity of simulated molecular objects (C$_{60}$ molecules) in iMD-XR. To evaluate this, we carried out experiments to compare the just-noticeable differences (JNDs) in two conditions: (1) via direct interaction with a real-time C$_{60}$ simulation, and (2) via observation-only$\unicode{x2013}$i.e. watching another person interacting with the simulations. Our findings show that direct interaction enabled participants to perceive more subtle rigidity differences of 11.5%, compared to 18.5% for observation-only. Furthermore, participants who undertook interaction first were better able to distinguish rigidity differences in the subsequent observation-only condition, suggesting that interaction trained participants to better perceive differences in molecular properties. These findings demonstrate a novel and flexible approach for sensing the properties of virtual objects in XR, and offer new insights into iMD-XR's potential in molecular research and education.

cs.HC↗

arXiv2026

NanoVer: An open-source framework for interactive molecular dynamics in extended reality (iMD-XR) on commodity hardware

This article outlines 'NanoVer', an open-source software framework which enables groups of people to co-habit the same virtual space and manipulate real-time MD (Molecular Dynamics) simulations of flexible 3D molecular structures with atomic-level precision as if they were tangible objects, an approach that we call 'interactive Molecular Dynamics in eXtended Reality' (iMD-XR). Distinct from our earlier iMD work that relied on tethered PC-VR systems with large graphics cards, NanoVer represents a change in philosophy, emphasizing compatibility with standalone mobile consumer XR hardware and corresponding software APIs. The NanoVer architecture enables multiple XR clients and/or Python clients to simultaneously communicate with a flexible server architecture that can carry out a range of tasks, including for example: recording iMD-XR sessions, static structure visualization, and MD trajectory visualization. NanoVer allows researchers, educators, and students to fluidly move between AR and VR environments, to explore creative new approaches to molecular research and education, including for example: molecular conformational sampling, protein-ligand binding, molecular psychophysics, training AI agents to sample molecular transitions, and a new interface which allows iMD-XR participants to sketch 3D conformational paths which automated agents can then follow. As an immersive platform that offers new ways to understand, engineer, communicate, and interact with dynamical behaviour at the nanoscale, NanoVer invites us to imagine new ways for combining human intelligence (e.g., spatial cognition and design reasoning) with machine intelligence. To expand NanoVer's accessibility, we have published a version to the Meta Horizon Store, for easy download by those with a Meta Quest 3/3S headset, to explore pre-recorded iMD-XR trajectory visualizations and set up their own multi-user system.

physics.chem-ph↗

arXiv2025

AI-Guided Molecular Simulations in VR: Exploring Strategies for Imitation Learning in Hyperdimensional Molecular Systems

Molecular dynamics (MD) simulations are a crucial computational tool for researchers to understand and engineer molecular structure and function in areas such as drug discovery, protein engineering, and material design. Despite their utility, MD simulations are expensive, owing to the high dimensionality of molecular systems. Interactive molecular dynamics in virtual reality (iMD-VR) has recently emerged as a "human-in-the-loop" strategy for efficiently navigating hyper-dimensional molecular systems. By providing an immersive 3D environment that enables visualization and manipulation of real-time molecular simulations running on high-performance computing architectures, iMD-VR enables researchers to reach out and guide molecular conformational dynamics, in order to efficiently explore complex, high-dimensional molecular systems. Moreover, iMD-VR simulations generate rich datasets that capture human experts' spatial insight regarding molecular structure and function. This paper explores the use of researcher-generated iMD-VR datasets to train AI agents via imitation learning (IL). IL enables agents to mimic complex behaviours from expert demonstrations, circumventing the need for explicit programming or intricate reward design. In this article, we review IL across robotics and Multi-agents systems domains which are comparable to iMD-VR, and discuss how iMD-VR recordings could be used to train IL models to interact with MD simulations. We then illustrate the applications of these ideas through a proof-of-principle study where iMD-VR data was used to train a CNN network on a simple molecular manipulation task; namely, threading a small molecule through a nanotube pore. Finally, we outline future research directions and potential challenges of using AI agents to augment human expertise in navigating vast molecular conformational spaces.

cs.LG↗

arXiv2024

Measuring the limit of perception of bond stiffness of interactive molecules in VR via a gamified psychophysics experiment

Molecular dynamics (MD) simulations provide crucial insight into molecular interactions and biomolecular function. With interactive MD simulations in VR (iMD-VR), chemists can now interact with these molecular simulations in real-time. Our sense of touch is essential for exploring the properties of physical objects, but recreating this sensory experience for virtual objects poses challenges. Furthermore, employing haptics in the context of molecular simulation is especially difficult since \textit{we do not know what molecules actually feel like}. In this paper, we build upon previous work that demonstrated how VR-users can distinguish properties of molecules without haptic feedback. We present the results of a gamified two-alternative forced choice (2AFC) psychophysics user study in which we quantify the threshold at which iMD-VR users can differentiate the stiffness of molecular bonds. Our preliminary analysis suggests that participants can sense differences between buckminsterfullerene molecules with different bond stiffness parameters and that this limit may fall within the chemically relevant range. Our results highlight how iMD-VR may facilitate a more embodied way of exploring complex and dynamic molecular systems, enabling chemists to sense the properties of molecules purely by interacting with them in VR.

cs.HC↗

arXiv2021

Dissolving yourself in connection to others: shared experiences of ego attenuation and connectedness during group VR experiences can be comparable to psychedelics

With a growing body of research highlighting the therapeutic potential of experiential phenomenology which diminishes egoic identity and increases one's sense of connectedness, there is significant interest in how to elicit such 'self-transcendent experiences' (STEs) in laboratory contexts. Psychedelic drugs (YDs) have proven particularly effective in this respect, producing subjective phenomenology which reliably elicits intense STEs. With virtual reality (VR) emerging as a powerful tool for constructing new perceptual environments, we describe a VR framework called 'Isness-distributed' (Isness-D) which harnesses the unique affordances of distributed multi-person VR to blur conventional self-other boundaries. Within Isness-D, groups of participants co-habit a shared virtual space, collectively experiencing their bodies as luminous energetic essences with diffuse spatial boundaries. It enables moments of 'energetic coalescence', a new class of embodied phenomenological intersubjective experience where bodies can fluidly merge, enabling participants to have an experience of including multiple others within their self-representation. To evaluate Isness-D, we adopted a citizen science approach, coordinating an international network of Isness-D 'nodes'. We analyzed the results (N = 58) using 4 different self-report scales previously applied to analyze subjective YD phenomenology (the inclusion of community in self scale, ego-dissolution inventory, communitas scale, and the MEQ30 mystical experience questionnaire). Despite the complexities associated with a distributed experiment like this, the Isness-D scores on all 4 scales were statistically indistinguishable from recently published YD studies, demonstrating that distributed VR can be used to design intersubjective STEs where people dissolve their sense of self in the connection to others.

cs.HC↗

arXiv2020

Subtle Sensing: Detecting Differences in the Flexibility of Virtually Simulated Molecular Objects

During VR demos we have performed over last few years, many participants (in the absence of any haptic feedback) have commented on their perceived ability to 'feel' differences between simulated molecular objects. The mechanisms for such 'feeling' are not entirely clear: observing from outside VR, one can see that there is nothing physical for participants to 'feel'. Here we outline exploratory user studies designed to evaluate the extent to which participants can distinguish quantitative differences in the flexibility of VR-simulated molecular objects. The results suggest that an individual's capacity to detect differences in molecular flexibility is enhanced when they can interact with and manipulate the molecules, as opposed to merely observing the same interaction. Building on these results, we intend to carry out further studies investigating humans' ability to sense quantitative properties of VR simulations without haptic technology.

cs.HC↗

arXiv2020

OMG-VR: Open-source Mudra Gloves for Manipulating Molecular Simulations in VR

As VR finds increasing application in scientific research domains like nanotechnology and biochemistry, we are beginning to better understand the domains in which it brings the most benefit, as well as the gestures and form factors that are most useful for specific applications. Here we describe Open-source Mudra Gloves for Virtual Reality (OMG-VR): etextile gloves designed to facilitate research scientists and students carrying out detailed and complex manipulation of simulated 3d molecular objects in VR. The OMG-VR is designed to sense when a user pinches together their thumb and index finger, or thumb and middle finger, forming a "mudra" position. Tests show that they provide good positional tracking of the point at which a pinch takes place, require no calibration, and are sufficiently accurate and robust to enable scientists to accomplish a range of tasks that involve complex spatial manipulation of molecules. The open source design offers a promising alternative to existing controllers and more costly commercial VR data gloves.

cs.HC↗