arXiv subjects

Brian A. Smith

Publications and source records attributed to Brian A. Smith.

17 recordsLinked to original sources

arXiv2026

Cyber-Physical Systems for Accessibility and Ability Augmentation: Bridging Diverse Communities

The powerful convergence of wearables, robotics, extended reality, and smart environments is expanding the design space for cyber-physical systems (CPS) that support and augment human abilities in daily life. By sensing real-world contexts, modeling user needs, and providing situated assistance, these systems can improve accessibility for people with disabilities while enhancing broader human abilities such as perception, memory, learning, and mobility. However, realizing this potential requires addressing key challenges in context sensing, user modeling, adaptive interaction, privacy, and evaluation to ensure that CPS are reliable and effective in real-world contexts. This workshop will bring together researchers and practitioners across HCI, AI, wearables, robotics, XR, smart environments, accessibility, and ability augmentation to examine shared strategies and challenges for designing accessibility- and ability-centered CPS. Through panel discussions, interactive demos, and mixed-group design activities, participants will identify recurring design principles, technical challenges, and future directions for CPS that support and augment human abilities in real-world settings. For details, please visit: https://cps4all.github.io.

cs.HC↗

arXiv2026

Sonic Stage: Auto-Generating Interactive Spatial Soundscapes to Facilitate Dialogue Video Comprehension for Blind Viewers

Audio description (AD) makes film and television accessible to blind and low-vision (BLV) audiences by narrating characters' actions. However, in scenes with lots of dialogue, AD often omits important actions because it is constrained not to overlap with speech. It is not yet known how to convey characters' actions during dialogue. We present Sonic Stage, a system that transforms dialogue videos into interactive spatial soundscapes, enabling BLV audiences to intuitively understand characters' actions and movements through immersive auditory cues. Sonic Stage conveys essential visual information during dialogue through three auditory techniques: (1) spatialized dialogue to represent spatial layout, (2) diegetic sound to convey character actions, and (3) interactive descriptions to provide context-specific visual details. Evaluation with 12 BLV viewers showed that Sonic Stage significantly improved video comprehension, spatial presence, and narrative engagement. We highlight opportunities for enhancing video accessibility across diverse genres through immersive, interactive audio representations.

cs.HC↗

arXiv2026

Challenges in Synchronous & Remote Collaboration Around Visualization

We characterize 16 challenges faced by those investigating and developing remote and synchronous collaborative experiences around visualization. Our work reflects the perspectives and prior research efforts of an international group of 29 experts from across human-computer interaction and visualization sub-communities. The challenges are anchored around five collaborative activities that exhibit a centrality of visualization and multimodal communication. These activities include exploratory data analysis, creative ideation, visualization-rich presentations, joint decision making grounded in data, and real-time data monitoring. The challenges also reflect the changing dynamics of these activities in the face of recent advances in extended reality (XR) and artificial intelligence (AI). As an organizing scheme for future research at the intersection of visualization and computer-supported cooperative work, we align the challenges with a sequence of four sets of research and development activities: technological choices, social factors, AI assistance, and evaluation.

cs.HC↗

arXiv2024

StreetNav: Leveraging Street Cameras to Support Precise Outdoor Navigation for Blind Pedestrians

Blind and low-vision (BLV) people rely on GPS-based systems for outdoor navigation. GPS's inaccuracy, however, causes them to veer off track, run into obstacles, and struggle to reach precise destinations. While prior work has made precise navigation possible indoors via hardware installations, enabling this outdoors remains a challenge. Interestingly, many outdoor environments are already instrumented with hardware such as street cameras. In this work, we explore the idea of repurposing existing street cameras for outdoor navigation. Our community-driven approach considers both technical and sociotechnical concerns through engagements with various stakeholders: BLV users, residents, business owners, and Community Board leadership. The resulting system, StreetNav, processes a camera's video feed using computer vision and gives BLV pedestrians real-time navigation assistance. Our evaluations show that StreetNav guides users more precisely than GPS, but its technical performance is sensitive to environmental occlusions and distance from the camera. We discuss future implications for deploying such systems at scale.

cs.HC↗

arXiv2024

Surveyor: Facilitating Discovery Within Video Games for Blind and Low Vision Players

Video games are increasingly accessible to blind and low vision (BLV) players, yet many aspects remain inaccessible. One aspect is the joy players feel when they explore environments and make new discoveries, which is integral to many games. Sighted players experience discovery by surveying environments and identifying unexplored areas. Current accessibility tools, however, guide BLV players directly to items and places, robbing them of that experience. Thus, a crucial challenge is to develop navigation assistance tools that also foster exploration and discovery. To address this challenge, we propose the concept of exploration assistance in games and design Surveyor, an in-game exploration assistance tool that enhances discovery by tracking where BLV players look and highlighting unexplored areas. We designed Surveyor using insights from a formative study and compared Surveyor's effectiveness to approaches found in existing accessible games. Our findings reveal implications for facilitating richer play experiences for BLV users within games.

cs.HC↗

arXiv2024

Help Supporters: Exploring the Design Space of Assistive Technologies to Support Face-to-Face Help Between Blind and Sighted Strangers

Blind and low-vision (BLV) people face many challenges when venturing into public environments, often wishing it were easier to get help from people nearby. Ironically, while many sighted individuals are willing to help, such interactions are infrequent. Asking for help is socially awkward for BLV people, and sighted people lack experience in helping BLV people. Through a mixed-ability research-through-design process, we explore four diverse approaches toward how assistive technology can serve as help supporters that collaborate with both BLV and sighted parties throughout the help process. These approaches span two phases: the connection phase (finding someone to help) and the collaboration phase (facilitating help after finding someone). Our findings from a 20-participant mixed-ability study reveal how help supporters can best facilitate connection, which types of information they should present during both phases, and more. We discuss design implications for future approaches to support face-to-face help.

cs.HC↗

arXiv2023

Social Wormholes: Exploring Preferences and Opportunities for Distributed and Physically-Grounded Social Connections

Ubiquitous computing encapsulates the idea for technology to be interwoven into the fabric of everyday life. As computing blends into everyday physical artifacts, powerful opportunities open up for social connection. Prior connected media objects span a broad spectrum of design combinations. Such diversity suggests that people have varying needs and preferences for staying connected to one another. However, since these designs have largely been studied in isolation, we do not have a holistic understanding around how people would configure and behave within a ubiquitous social ecosystem of physically-grounded artifacts. In this paper, we create a technology probe called Social Wormholes, that lets people configure their own home ecosystem of connected artifacts. Through a field study with 24 participants, we report on patterns of behaviors that emerged naturally in the context of their daily lives and shine a light on how ubiquitous computing could be leveraged for social computing.

cs.HC↗

arXiv2023

Supporting Piggybacked Co-Located Leisure Activities via Augmented Reality

Technology, especially the smartphone, is villainized for taking meaning and time away from in-person interactions and secluding people into "digital bubbles". We believe this is not an intrinsic property of digital gadgets, but evidence of a lack of imagination in technology design. Leveraging augmented reality (AR) toward this end allows us to create experiences for multiple people, their pets, and their environments. In this work, we explore the design of AR technology that "piggybacks" on everyday leisure to foster co-located interactions among close ties (with other people and pets. We designed, developed, and deployed three such AR applications, and evaluated them through a 41-participant and 19-pet user study. We gained key insights about the ability of AR to spur and enrich interaction in new channels, the importance of customization, and the challenges of designing for the physical aspects of AR devices (e.g., holding smartphones). These insights guide design implications for the novel research space of co-located AR.

cs.HC↗

arXiv2023

ImageAssist: Tools for Enhancing Touchscreen-Based Image Exploration Systems for Blind and Low Vision Users

Blind and low vision (BLV) users often rely on alt text to understand what a digital image is showing. However, recent research has investigated how touch-based image exploration on touchscreens can supplement alt text. Touchscreen-based image exploration systems allow BLV users to deeply understand images while granting a strong sense of agency. Yet, prior work has found that these systems require a lot of effort to use, and little work has been done to explore these systems' bottlenecks on a deeper level and propose solutions to these issues. To address this, we present ImageAssist, a set of three tools that assist BLV users through the process of exploring images by touch -- scaffolding the exploration process. We perform a series of studies with BLV users to design and evaluate ImageAssist, and our findings reveal several implications for image exploration tools for BLV users.

cs.HC↗

arXiv2023

Towards Inclusive Avatars: Disability Representation in Avatar Platforms

Digital avatars are an important part of identity representation, but there is little work on understanding how to represent disability. We interviewed 18 people with disabilities and related identities about their experiences and preferences in representing their identities with avatars. Participants generally preferred to represent their disability identity if the context felt safe and platforms supported their expression, as it was important for feeling authentically represented. They also utilized avatars in strategic ways: as a means to signal and disclose current abilities, access needs, and to raise awareness. Some participants even found avatars to be a more accessible way to communicate than alternatives. We discuss how avatars can support disability identity representation because of their easily customizable format that is not strictly tied to reality. We conclude with design recommendations for creating platforms that better support people in representing their disability and other minoritized identities.

cs.HC↗

arXiv2022

Exploring Immersive Interpersonal Communication via AR

A central challenge of social computing research is to enable people to communicate expressively with each other remotely. Augmented reality has great promise for expressive communication since it enables communication beyond texts and photos and towards immersive experiences rendered in recipients' physical environments. Little research, however, has explored AR's potential for everyday interpersonal communication. In this work, we prototype an AR messaging system, ARwand, to understand people's behaviors and perceptions around communicating with friends via AR messaging. We present our findings under four themes observed from a user study with 24 participants, including the types of immersive messages people choose to send to each other, which factors contribute to a sense of immersiveness, and what concerns arise over this new form of messaging. We discuss important implications of our findings on the design of future immersive communication systems.

cs.HC↗

arXiv2022

"I Want to Figure Things Out": Supporting Exploration in Navigation for People with Visual Impairments

Navigation assistance systems (NASs) aim to help visually impaired people (VIPs) navigate unfamiliar environments. Most of today's NASs support VIPs via turn-by-turn navigation, but a growing body of work highlights the importance of exploration as well. It is unclear, however, how NASs should be designed to help VIPs explore unfamiliar environments. In this paper, we perform a qualitative study to understand VIPs' information needs and challenges with respect to exploring unfamiliar environments, with the aim of informing the design of NASs that support exploration. Our findings reveal the types of spatial information that VIPs need as well as factors that affect VIPs' information preferences. We also discover specific challenges that VIPs face that future NASs can address such as orientation and mobility education and collaborating effectively with others. We present design implications for NASs that support exploration, and we identify specific research opportunities and discuss open socio-technical challenges for making such NASs possible. We conclude by reflecting on our study procedure to inform future approaches in research on ethical considerations that may be adopted while interacting with the broader VIP community.

cs.HC↗

arXiv2022

Uncovering Visually Impaired Gamers' Preferences for Spatial Awareness Tools Within Video Games

Sighted players gain spatial awareness within video games through sight and spatial awareness tools (SATs) such as minimaps. Visually impaired players (VIPs), however, must often rely heavily on SATs to gain spatial awareness, especially in complex environments where using rich ambient sound design alone may be insufficient. Researchers have developed many SATs for facilitating spatial awareness within VIPs. Yet this abundance disguises a gap in our understanding about how exactly these approaches assist VIPs in gaining spatial awareness and what their relative merits and limitations are. To address this, we investigate four leading approaches to facilitating spatial awareness for VIPs within a 3D video game context. Our findings uncover new insights into SATs for VIPs within video games, including that VIPs value position and orientation information the most from an SAT; that none of the approaches we investigated convey position and orientation effectively; and that VIPs highly value the ability to customize SATs.

cs.HC↗

arXiv2022

ARcall: Real-Time AR Communication using Smartphones and Smartglasses

Augmented Reality (AR) smartglasses are increasingly regarded as the next generation personal computing platform. However, there is a lack of understanding about how to design communication systems using them. We present ARcall, a novel Augmented Reality-based real-time communication system that enables an immersive, delightful, and privacy-preserving experience between a smartphone user and a smartglasses wearer. ARcall allows a remote friend (Friend) to send and project AR content to a smartglasses wearer (Wearer). The ARcall system was designed with the practical limits of existing AR glasses in mind, including shorter battery life and a reduced field of view. We conduct a qualitative evaluation of the three main components of ARcall: Drop-In, ARaction, and Micro-Chat. Our results provide novel insights for building future AR-based communication methods, including, the importance of context priming, user control over AR content placement, and the feeling of co-presence while conversing.

cs.HC↗

arXiv2022

Understanding the Role of Context in Creating Enjoyable Co-Located Interactions

In recent years, public discourse has blamed digital technologies for making people feel "alone together," distracting us from engaging with one another, even when we are interacting in-person. We argue that in order to design technologies that foster and augment co-located interactions, we need to first understand the context in which enjoyable co-located socialization takes place. We address this gap by surveying and interviewing over 1,000 U.S.-based participants to understand what, where, with whom, how, and why people enjoy spending time in-person. Our findings suggest that people enjoy engaging in everyday activities with individuals with whom they have strong social ties because it helps enable nonverbal cues, facilitate spontaneity, support authenticity, encourage undivided attention, and leverage the physicality of their bodies and the environment. We conclude by providing a set of recommendations for designers interested in creating co-located technologies that encourage social engagement and relationship building.

cs.HC↗

arXiv2021

Friendscope: Exploring In-the-Moment Experience Sharing on Camera Glasses via a Shared Camera

We introduce Friendscope, an instant, in-the-moment experience sharing system for lightweight commercial camera glasses. Friendscope explores a new concept called a shared camera. This concept allows a wearer to share control of their camera with a remote friend, making it possible for both people to capture photos/videos from the camera in the moment. Through a user study with 48 participants, we found that users felt connected to each other, describing the shared camera as a more intimate form of livestreaming. Moreover, even privacy-sensitive users were able to retain their sense of privacy and control with the shared camera. Friendscope's different shared camera configurations give wearers ultimate control over who they share the camera with and what photos/videos they share. We conclude with design implications for future experience sharing systems.

cs.HC↗

arXiv2021

NavStick: Making Video Games Blind-Accessible via the Ability to Look Around

Video games remain largely inaccessible to visually impaired people (VIPs). Today's blind-accessible games are highly simplified renditions of what sighted players enjoy, and they do not give VIPs the same freedom to look around and explore game worlds on their own terms. In this work, we introduce NavStick, an audio-based tool for looking around within virtual environments, with the aim of making 3D adventure video games more blind-accessible. NavStick repurposes a game controller's thumbstick to allow VIPs to survey what is around them via line-of-sight. In a user study, we compare NavStick with traditional menu-based surveying for different navigation tasks and find that VIPs were able to form more accurate mental maps of their environment with NavStick than with menu-based surveying. In an additional exploratory study, we investigate NavStick in the context of a representative 3D adventure game. Our findings reveal several implications for blind-accessible games, and we close by discussing these.

cs.HC↗