Sensing the properties of virtual objects without physical feedback

Abstract

People who have interacted with simulated worlds and simulated objects in extended reality (XR) often have a sense that they can 'feel' the objects being simulated despite them not being physical. Our sense of touch is essential for how we 'feel' the physical world, however, there is an open question as to what it means to 'feel' virtual objects when interacting with them in immersive digital environments. In prior research, we have reported that participants often describe a subjective experience of 'feeling' the properties of simulated molecular objects while using interactive molecular dynamics in extended reality (iMD-XR), a field-based interaction paradigm for manipulating real-time simulations of molecular objects without haptic feedback. To better understand these subjective reports of 'feeling', we used a psychophysics approach to quantify the threshold at which participants perceive differences in the rigidity of simulated molecular objects (C$_{60}$ molecules) in iMD-XR. To evaluate this, we carried out experiments to compare the just-noticeable differences (JNDs) in two conditions: (1) via direct interaction with a real-time C$_{60}$ simulation, and (2) via observation-only$\unicode{x2013}$i.e. watching another person interacting with the simulations. Our findings show that direct interaction enabled participants to perceive more subtle rigidity differences of 11.5%, compared to 18.5% for observation-only. Furthermore, participants who undertook interaction first were better able to distinguish rigidity differences in the subsequent observation-only condition, suggesting that interaction trained participants to better perceive differences in molecular properties. These findings demonstrate a novel and flexible approach for sensing the properties of virtual objects in XR, and offer new insights into iMD-XR's potential in molecular research and education.