arXiv subjects

Benjamin Yang

Publications and source records attributed to Benjamin Yang.

5 recordsLinked to original sources

arXiv2026

Augmenting Human Performance with an XR Agent Learning from Online Behavior and BCI Evidence

We present OLIVE, a framework for adapting a foundation model to provide real-time assistance in temporally demanding, high-stakes, and dynamic tasks. We show that passive EEG, fused online with behavioral evidence, can meaningfully extend the number of targets users detect and engage beyond their unaided action bandwidth. OLIVE learns from both explicit behavioral signals (the targets the user shoots down in an XR first-person shooter game) and implicit physiological signals (fixation-locked EEG) to provide timely guidance, continuously adapting a frozen vision-language model's inference on which items are task-relevant by jointly estimating per-source reliability without manual labels or offline training. Through three user studies, including two live deployments of an assistive agent driven by OLIVE in XR, we show that OLIVE Pareto-dominates prior test-time adaptation frameworks, achieving the highest convergence rate at comparable convergence speed. Combining implicit physiological and explicit behavioral signals, the OLIVE agent produces the largest and most reliable within-session improvement to a user's ability to detect and engage targets, largely independent of the individual's skill. When the target switches silently, the agent that uses both behavioral and physiological signals reconverges significantly faster than the behavior-only agent (1.27 times faster on average, p = .008), restoring trustworthy guidance at the moment the task changes, precisely when reliable assistance matters most.

cs.AI↗

arXiv2026

Agents of ViTAL: Ethics Missions -- A Narrative-Centered Learning Environment with a Co-Designed Conversational Agent for Middle School AI Ethics

Agents of ViTAL: Ethics Missions is a browser-based, narrative-centered learning environment in which middle school students collaboratively evaluate whether a fictional school should adopt an AI-powered classroom feedback tool. Students investigate stakeholder perspectives, weigh tradeoffs across three AI ethics dimensions (privacy, bias, and environmental impact), and negotiate group consensus through a shared Ranking Challenge interface. The environment embeds EthicsBot, a conversational agent designed as a peer-like thought partner that scaffolds ethical reasoning and collaborative discussion. The project is being iteratively co-designed with high school students, whose feedback shapes EthicsBot's role, behavior, and guardrails. Initial classroom implementations with four ninth-grade classes demonstrated strong engagement and substantive ethical reasoning grounded in students' lived experiences with AI. The demo invites attendees to explore the learning environment, participate in the collaborative Ranking Challenge, and interact with EthicsBot.

cs.CY↗

arXiv2026

SwEYEpinch: Exploring Intuitive, Efficient Text Entry for Extended Reality via Eye and Hand Tracking

Despite steady progress, text entry in Extended Reality (XR) often remains slower and more effortful than typing on a physical keyboard or touchscreen. We explore a simple idea: use gaze to swipe through a virtual keyboard for the fast, low-effort where and a manual pinch held throughout the swipe for the when, extending and validating it through a series of user studies. We first show that a basic version including a low-latency decoder with spatiotemporal Dynamic Time Warping and fixation filtering outperforms selecting individual keys sequentially, either by finger tapping each or gazing at each while pinching. We then add mid-swipe prediction and in-gesture cancellation, improving words per minute (WPM) without hurting accuracy. We show that this approach is faster and more preferred than previous gaze-swipe approaches, finger tapping with prediction, or hand swiping with the same additions. Furthermore, a seven-day, 30-session study demonstrates sustained learning, with peak performance reaching 64.7 WPM.

cs.HC↗

arXiv2026

Bimanual XR Specification of Relative and Absolute Assembly Hierarchies for Teleoperation

We present a bimanual XR interaction approach for specifying remote assembly tasks as hierarchies of relative and absolute object constraints that specify high-level teleoperation goals for robots. Grabbing one object in each hand creates a constraint group (visualized as a hull) and groups can be nested into hierarchies. Each group can be relative (with a robot-specifiable 6DoF pose) or absolute (with an author-specified fixed 6DoF pose) in relation to its parent. A relative group specifies a subassembly that can be constructed at a location chosen by the robot software for efficiency rather than mandated by the user.

cs.HC↗

arXiv2019

Prediction of Silicate Glasses' Stiffness by High-Throughput Molecular Dynamics Simulations and Machine Learning

The development by machine learning of models predicting materials' properties usually requires the use of a large number of consistent data for training. However, quality experimental datasets are not always available or self-consistent. Here, as an alternative route, we combine machine learning with high-throughput molecular dynamics simulations to predict the Young's modulus of silicate glasses. We demonstrate that this combined approach offers excellent predictions over the entire compositional domain. By comparing the performance of select machine learning algorithms, we discuss the nature of the balance between accuracy, simplicity, and interpretability in machine learning.

cond-mat.mtrl-sci↗