Searcharxiv⌕ Search

arXiv subjects

Junhan Kong

Publications and source records attributed to Junhan Kong.

3 recordsLinked to original sources

AdaptType: Accessible and Customizable Multimodal Text Entry in Virtual Reality for People with Upper-Body Motor Impairments

Text entry in virtual reality (VR) is cumbersome, especially for people with upper-body motor impairments. Existing techniques often demand both fine and gross motor control, yet users' motor abilities vary widely. To address this range of needs, we present AdaptType, a multimodal VR text entry system that enables users to configure input techniques that best suit their abilities, and scaffolds customization through ability-based recommendations. Through a study with 10 participants, we found that customized multimodal techniques improved text entry speed compared to gaze- or hand-only techniques, and complemented voice for error correction, decreasing failure rate from 15.1% to 2.0%. We found that AdaptType yielded high text entry speed (36.26 WPM) and accuracy (0.36% error rate), with low task load and high usability ratings. We also found ability-based recommendations effectively scaffolded customization. Our work contributes more accessible and personalized VR text entry solutions for people with upper-body motor impairments.

cs.HC↗

\$M Recognizer for User-Defined Mid-Air Gestures

User-defined mid-air gestures hold promise to customize device interactions to a user's preferences, cultural background, and physical abilities. This is especially true for individuals with upper-body motor disabilities, who may find standardized mid-air gestures inaccessible. However, the biosignals that enable such mid-air gesture recognition are multi-modal, multi-dimensional, and noisy, requiring domain expertise to design input features for gesture recognizers. The \$M-recognizer enables mid-air gesture recognition without needing expertise in biosignals or algorithms. \$M resamples, normalizes, and performs dimensionality reduction to reduce noise and enhance signals relevant to the recognition. We evaluated \$M on 26 participants with and 8 participants without upper-body motor disabilities performing user-defined ability-based gestures. For user-dependent and gesture articulation variability conditions, \$M outperformed our comparison algorithms (traditional machine learning with expert features and deep learning), with >95% recognition rate. We tested \$M's ease-of-implementation with six coders with minimal machine learning experience. Participants implemented $M 53% faster and with 23% lower workload compared to deep learning. \$M highlights the potential and feasibility of user-defined and accessible mid-air gesture interfaces.

cs.HC↗

StateLens: A Reverse Engineering Solution for Making Existing Dynamic Touchscreens Accessible

Blind people frequently encounter inaccessible dynamic touchscreens in their everyday lives that are difficult, frustrating, and often impossible to use independently. Touchscreens are often the only way to control everything from coffee machines and payment terminals, to subway ticket machines and in-flight entertainment systems. Interacting with dynamic touchscreens is difficult non-visually because the visual user interfaces change, interactions often occur over multiple different screens, and it is easy to accidentally trigger interface actions while exploring the screen. To solve these problems, we introduce StateLens - a three-part reverse engineering solution that makes existing dynamic touchscreens accessible. First, StateLens reverse engineers the underlying state diagrams of existing interfaces using point-of-view videos found online or taken by users using a hybrid crowd-computer vision pipeline. Second, using the state diagrams, StateLens automatically generates conversational agents to guide blind users through specifying the tasks that the interface can perform, allowing the StateLens iOS application to provide interactive guidance and feedback so that blind users can access the interface. Finally, a set of 3D-printed accessories enable blind people to explore capacitive touchscreens without the risk of triggering accidental touches on the interface. Our technical evaluation shows that StateLens can accurately reconstruct interfaces from stationary, hand-held, and web videos; and, a user study of the complete system demonstrates that StateLens successfully enables blind users to access otherwise inaccessible dynamic touchscreens.

cs.HC↗