arXiv subjects

Jeremy N. Bailenson

Publications and source records attributed to Jeremy N. Bailenson.

6 recordsLinked to original sources

arXiv2026

FIDAC: An Easy-to-use Pipeline to Extract and Interpret Interpersonal Distance From Video

The distance between persons reveals significant information about their perception of each other. However, such information is not easily extractable and interpretable from video input. We developed an open-sourced library, Facial Interpersonal Distance Analysis and Coding (FIDAC) that transforms facial detection results into actionable data about location and interpersonal distance. This tool merges data from multiple open-source facial detection models, strategically compensating for gaps in any individual model. In addition, we include methods for more accurate tracking, such as a pipeline for human coding of the selection of faces and a benchmarking tool to reduce depth distortion. For next steps, we plan on building upon FIDAC by evaluating its effectiveness at measuring interpersonal distance at various depths and orientations while further integrating features of proxemic analysis such as synchrony into its software.

cs.CV↗

arXiv2025

Audio Personas: Augmenting Social Perception via Body-Anchored Audio Cues

We introduce Audio Personas, enabling users to "decorate" themselves with body-anchored sounds in audio augmented reality. Like outfits, makeup, and fragrances, audio personas offer an alternative yet dynamic channel to augment face-to-face interactions. For instance, one can set their audio persona as rain sounds to reflect a bad mood, bee sounds to establish personal boundaries, or a playful "woosh" sound to mimic passing by someone like a breeze. To instantiate the concept, we implemented a headphone-based prototype with multi-user tracking and audio streaming. Our preregistered in-lab study with 64 participants showed that audio personas influenced how participants formed impressions. Individuals with positive audio personas were rated as more socially attractive, more likable, and less threatening than those with negative audio personas. Our study with audio designers revealed that audio personas were preferred in public and semi-public-private spaces for managing social impressions (e.g., personality) and signaling current states (e.g., emotions).

cs.HC↗

arXiv2025

Predicting and Understanding Turn-Taking Behavior in Open-Ended Group Activities in Virtual Reality

In networked virtual reality (VR), user behaviors, individual differences, and group dynamics can serve as important signals into future speech behaviors, such as who the next speaker will be and the timing of turn-taking behaviors. The ability to predict and understand these behaviors offers opportunities to provide adaptive and personalized assistance, for example helping users with varying sensory abilities navigate complex social scenes and instantiating virtual moderators with natural behaviors. In this work, we predict turn-taking behaviors using features extracted based on social dynamics literature. We discuss results from a large-scale VR classroom dataset consisting of 77 sessions and 1660 minutes of small-group social interactions collected over four weeks. In our evaluation, gradient boosting classifiers achieved the best performance, with accuracies of 0.71--0.78 AUC (area under the ROC curve) across three tasks concerning the "what", "who", and "when" of turn-taking behaviors. In interpreting these models, we found that group size, listener personality, speech-related behavior (e.g., time elapsed since the listener's last speech event), group gaze (e.g., how much the group looks at the speaker), as well as the listener's and previous speaker's head pitch, head y-axis position, and left hand y-axis position more saliently influenced predictions. Results suggested that these features remain reliable indicators in novel social VR settings, as prediction performance is robust over time and with groups and activities not used in the training dataset. We discuss theoretical and practical implications of the work.

cs.HC↗

arXiv2024

Effect of Duration and Delay on the Identifiability of VR Motion

Social virtual reality is an emerging medium of communication. In this medium, a user's avatar (virtual representation) is controlled by the tracked motion of the user's headset and hand controllers. This tracked motion is a rich data stream that can leak characteristics of the user or can be effectively matched to previously-identified data to identify a user. To better understand the boundaries of motion data identifiability, we investigate how varying training data duration and train-test delay affects the accuracy at which a machine learning model can correctly classify user motion in a supervised learning task simulating re-identification. The dataset we use has a unique combination of a large number of participants, long duration per session, large number of sessions, and a long time span over which sessions were conducted. We find that training data duration and train-test delay affect identifiability; that minimal train-test delay leads to very high accuracy; and that train-test delay should be controlled in future experiments.

cs.CR↗

arXiv2024

How Video Passthrough Headsets Influence Perception of Self and Others

With the increasing adoption of mixed reality headsets with video passthrough functionality, concerns over perceptual and social effects have surfaced. Building on prior qualitative findings, this study quantitatively investigates the impact of video passthrough on users. Forty participants completed a body transfer task twice, once while wearing a headset in video passthrough and once without a headset. Results indicate that using video passthrough induces simulator sickness, creates social absence, (another person in the physical room feels less present), alters self-reported body schema, and distorts distance perception. On the other hand, compared to past research which showed perceptual aftereffects from video passthrough, the current study found none. We discuss the broader implications for the widespread adoption of mixed reality headsets and their impact on theories surrounding presence and body transfer.

cs.HC↗

arXiv2023

A Large-Scale Study of Personal Identifiability of Virtual Reality Motion Over Time

In recent years, social virtual reality (VR), sometimes described as the "metaverse," has become widely available. With its potential comes risks, including risks to privacy. To understand these risks, we study the identifiability of participants' motion in VR in a dataset of 232 VR users with eight weekly sessions of about thirty minutes each, totaling 764 hours of social interaction. The sample is unique as we are able to study the effect of user, session, and time independently. We find that the number of sessions recorded greatly increases identifiability, and duration per session increases identifiability as well, but to a lesser degree. We also find that greater delay between training and testing sessions reduces identifiability. Ultimately, understanding the identifiability of VR activities will help designers, security professionals, and consumer advocates make VR safer.

cs.CR↗