arXiv subjects

JooYoung Seo

Publications and source records attributed to JooYoung Seo.

17 recordsLinked to original sources

arXiv2026

QUARTZ: Qualitative Understanding via Accessible Representation and Visualization

Qualitative data visualizations -- concept maps, network graphs, Sankey diagrams, and coding stripes -- are integral to research practice, yet remain entirely inaccessible to blind and low-vision (BLV) researchers. While visualization has seen advanced multimodal solutions for quantitative charts, qualitative visualizations, and their non-linear, semantically rich structures have received no attention. We present QUARTZ, a web-based system that provides screen-reader-accessible, multimodal representations of qualitative data visualizations. Using the Rapid Iterative Testing and Evaluation (RITE) method, we conducted a user study with 8 BLV participants who completed 12 tasks across four visualization types. Our findings expose accessibility barriers unique to qualitative visualizations -- non-linear navigation breakdowns and semantic comprehension gaps absent from quantitative chart research---and document how iterative co-design with BLV users resolved them. We contribute empirical evidence and design guidelines for an underexplored visualization domain, advancing the infrastructure BLV researchers need to participate independently in qualitative inquiry.

cs.HC↗

arXiv2026

"I Don't Want My Mental Health App To Give Me Mental Health Barriers": Unpacking The Need For Digital Mental Health Tracking Services With And For The Blind Community

Digital mental health (DMH) tracking services promise continuous, personalized support for well-being, but their design often assumes sighted users. For the blind community, this assumption produces a distinct pattern of exclusion: services whose accessibility cannot be evaluated without first paying for them, community features that exclude the users they purport to support, and interfaces that leave users digitally literate but functionally blocked. We report on an explanatory sequential mixed-methods study of blind users' experiences with DMH tracking services in the United States. In the first phase, 93 legally blind adults completed a survey about their usage patterns, adoption decisions, and data-agency preferences; in the second, 10 survey respondents participated in semi-structured interviews. We analyzed closed-ended responses using descriptive statistics and the Kruskal-Wallis test, and open-ended and interview data using inductive thematic analysis, interpreting findings through Norman and Skinner's eHealth Literacy framework. Participants identified mindfulness, sleep, and goal-tracking services as their most-used categories, but also described recurring exclusion from the community-support features that other users value most. We argue that the framework's "computer literacy" dimension is insufficient on its own: many of our participants possessed the literacy but were blocked from applying it by design choices that predate the user. We contribute design recommendations for transparent pre-purchase accessibility evaluation, accessibility-native rather than retrofitted interfaces, and user-controlled data agency -- recommendations intended not to accommodate blind users but to design DMH tracking services with them from the start.

cs.HC↗

arXiv2026

Three Modalities, Two Design Probes, One Prototype, and No Vision: Experience-Based Co-Design of a Multi-modal 3D Data Visualization Tool

Three-dimensional (3D) data visualizations, such as surface plots, are vital in STEM fields from biomedical imaging to spectroscopy, yet remain largely inaccessible to blind and low-vision (BLV) people. To address this gap, we conducted an Experience-Based Co-Design with BLV co-designers with expertise in non-visual data representations to create an accessible, multi-modal, web-native visualization tool. Using a multi-phase methodology, our team of five BLV and one non-BLV researcher(s) participated in two iterative sessions, comparing a low-fidelity tactile probe with a high-fidelity digital prototype. This process produced a prototype with empirically grounded features, including reference sonification, stereo and volumetric audio, and configurable buffer aggregation, which our co-designers validated as improving analytic accuracy and learnability. In this study, we target core analytic tasks essential for non-visual 3D data exploration: orientation, landmark and peak finding, comparing local maxima versus global trends, gradient tracing, and identifying occluded or partially hidden features. Our work offers accessibility researchers and developers a co-design protocol for translating tactile knowledge to digital interfaces, concrete design guidance for future systems, and opportunities to extend accessible 3D visualization into embodied data environments.

cs.HC↗

arXiv2026

"I Don't Trust Any Professional Research Tool": A Re-Imagination of Knowledge Production Workflows by, with, and for Blind and Low-Vision Researchers

Research touts universal participation through accessibility initiatives, yet blind and low-vision (BLV) researchers face systematic exclusion as visual representations dominate modern research workflows. To materialize inclusive processes, we, as BLV researchers, examined how our peers combat inaccessible infrastructures. Through an explanatory sequential mixed-methods approach, we conducted a cross-sectional, observational survey (n=57) and follow-up semi-structured interviews (n=15), analyzing open-ended data using reflexive thematic analysis and framing findings through activity theory to highlight research's systemic shortcomings. We expose how BLV researchers sacrifice autonomy and shoulder physical burdens, with nearly one-fifth unable to independently perform literature review or evaluate visual outputs, delegating tasks to sighted colleagues or relying on AI-driven retrieval to circumvent fatigue. Researchers also voiced frustration with specialized tools, citing developers' performative responses and losing deserved professional accolades. We seek follow-through on research's promises through design recommendations that reconceptualize accessibility as fundamental to successful research and supporting BLV scholars' workflows.

cs.HC↗

arXiv2026

What Lies Beneath: A Call for Distribution-based Visual Question & Answer Datasets

Visual Question Answering (VQA) has become an important benchmark for assessing how large multimodal models (LMMs) interpret images. However, most VQA datasets focus on real-world images or simple diagrammatic analysis, with few focused on interpreting complex scientific charts. Indeed, many VQA datasets that analyze charts do not contain the underlying data behind those charts or assume a 1-to-1 correspondence between chart marks and underlying data. In reality, charts are transformations (i.e. analysis, simplification, modification) of data. This distinction introduces a reasoning challenge in VQA that the current datasets do not capture. In this paper, we argue for a dedicated VQA benchmark for scientific charts where there is no 1-to-1 correspondence between chart marks and underlying data. To do so, we survey existing VQA datasets and highlight limitations of the current field. We then generate synthetic histogram charts based on ground truth data, and ask both humans and a large reasoning model questions where precise answers depend on access to the underlying data. We release the open-source dataset, including figures, underlying data, distribution parameters used to generate the data, and bounding boxes for all figure marks and text for future research.

cs.CV↗

arXiv2025

vashTimer: A Multi-Purpose, Multimodal Mobile App For Maintaining Passage of Time by means of Visual, Auditory, Speech, and Haptic Alerts

Effective time management during presentations is challenging, particularly for Blind and Low-Vision (BLV) individuals, as existing tools often lack accessibility and multimodal feedback. To address this gap, we developed vashTimer: a free, open-source, and accessible iOS application. This paper demonstrates the design and functionality of vashTimer, which provides presenters with a robust tool for temporal awareness. The application delivers highly customizable alerts across four distinct modalities: visual, auditory, speech, and haptic; and supports multiple configurable intervals within a single session. By offering a flexible and non-intrusive time management solution, vashTimer empowers presenters of all visual abilities. The implications of this work extend beyond public speaking to any professional, such as a clinical therapist, who requires discreet temporal cues, fostering greater independence and focus for a wide range of users. This demonstration serves as the foundation for a planned formal user evaluation.

cs.HC↗

arXiv2025

Sensing the Shape of Data: Non-Visual Exploration of Statistical Concepts in Histograms with Blind and Low-Vision Learners

Statistical concepts often rely heavily on visual cues for comprehension, presenting challenges for individuals who face difficulties using visual information, such as the blind and low-vision (BLV) community. While prior work has explored making data visualizations accessible, limited research examines how BLV individuals conceptualize and learn the underlying statistical concepts these visualizations represent. To better understand BLV individuals' learning strategies for potentially unfamiliar statistical concepts, we conducted a within-subjects experiment with 7 BLV individuals, controlling for vision condition using blindfolds. Each participant leveraged three different non-visual representations (Swell Touch tactile graph (STGs), shaped data patterns on a refreshable display (BDPs), sonification) to understand three different statistical concepts in histograms (skewness, modality, kurtosis). We collected quantitative metrics (accuracy, completion time, self-reported confidence levels) and qualitative insights (gesture analysis) to identify participants' unique meaning-making strategies. Results revealed that the braille condition led to the most accurate results, with sonification tasks being completed the fastest. Participants demonstrated various adaptive techniques when exploring each histogram, often developing alternative mental models that helped them non-visually encode statistical visualization concepts. Our findings reveal important implications for statistics educators and assistive technology designers, suggesting that effective learning tools must go beyond simple translation of visual information to support the unique cognitive strategies employed by BLV learners.

cs.HC↗

arXiv2025

Py maidr: Bridging Visual and Non-Visual Data Experiences Through a Unified Python Framework

Although recent efforts have developed accessible data visualization tools for blind and low-vision (BLV) users, most follow a "design for them" approach that creates an unintentional divide between sighted creators and BLV consumers. This unidirectional paradigm perpetuates a power dynamic where sighted creators produce non-visual content boundaries for BLV consumers to access. This paper proposes a bidirectional approach, "design for us," where both sighted and BLV collaborators can employ the same tool to create, interpret, and communicate data visualizations for each other. We introduce Py maidr, a Python package that seamlessly encodes multimodal (e.g., tactile, auditory, conversational) data representations into visual plots generated by Matplotlib and Seaborn. By simply importing the maidr package and invoking the maidr.show() method, users can generate accessible plots with minimal changes to their existing codebase regardless of their visual dis/abilities. Our technical case studies demonstrate how this tool is scalable and can be integrated into interactive computing (e.g., Jupyter Notebook, Google Colab), reproducible and literate programming (e.g., Quarto), and reactive dashboards (e.g., Shiny, Streamlit). Our performance benchmarks demonstrate that Py maidr introduces minimal and consistent overhead during the rendering and export of plots against Matplotlib and Seaborn baselines. This work significantly contributes to narrowing the accessibility gap in data visualization by providing a unified framework that fosters collaboration and communication between sighted and BLV individuals.

cs.HC↗

arXiv2025

Explore, Listen, Inspect: Supporting Multimodal Interaction with 3D Surface and Point Data Visualizations

Blind and low-vision (BLV) users remain largely excluded from three-dimensional (3D) surface and point data visualizations due to the reliance on visual interaction. Existing approaches inadequately support non-visual access, especially in browser-based environments. This study introduces DIXTRAL, a hosted web-native system, co-designed with BLV researchers to address these gaps through multimodal interaction. Conducted with two blind and one sighted researcher, this study took place over sustained design sessions. Data were gathered through iterative testing of the prototype, collecting feedback on spatial navigation, sonification, and usability. Co-design observations demonstrate that synchronized auditory, visual, and textual feedback, combined with keyboard and gamepad navigation, enhances both structure discovery and orientation. DIXTRAL aims to improve access to 3D continuous scalar fields for BLV users and inform best practices for creating inclusive 3D visualizations.

cs.HC↗

arXiv2025

Phoenix: A Novel Context-Aware Voice-Powered Math Equation Workspace and Editor

Writing mathematical notation requires substantial effort, diverting cognitive resources from conceptual understanding to documentation mechanics, significantly impacting individuals with fine motor disabilities (FMDs). Current limits of speech-based math technologies rely on precise dictation of math symbols and unintuitive command-based interfaces. We present a novel voice-powered math workspace, applying neuroscience insights to create an intuitive problem-solving environment. To minimize cognitive load, we leverage large language models with our novel context engine to support natural language interaction. Ultimately, we enable fluid mathematical engagement for individuals with FMDs -- freed from mechanical constraints.

cs.HC↗

arXiv2025

PunchPulse: A Physically Demanding Virtual Reality Boxing Game Designed with, for and by Blind and Low-Vision Players

Blind and low-vision (BLV) individuals experience lower levels of physical activity (PA) compared to sighted peers due to a lack of accessible, engaging exercise options. Existing solutions often rely on auditory cues but do not fully integrate rich sensory feedback or support spatial navigation, limiting their effectiveness. This study introduces PunchPulse, a virtual reality (VR) boxing exergame designed to motivate BLV users to reach and sustain moderate to vigorous physical activity (MVPA) levels. Over a seven-month, multi-phased study, PunchPulse was iteratively refined with three BLV co-designers, informed by two early pilot testers, and evaluated by six additional BLV user-study participants. Data collection included both qualitative (researcher observations, SOPI) and quantitative (MVPA zones, aid usage, completion times) measures of physical exertion and gameplay performance. The user study revealed that all participants reached moderate MVPA thresholds, with high levels of immersion and engagement observed. This work demonstrates the potential of VR as an inclusive medium for promoting meaningful PA in the BLV community and addresses a critical gap in accessible, intensity-driven exercise interventions.

cs.HC↗

arXiv2025

AROMA: Mixed-Initiative AI Assistance for Non-Visual Cooking by Grounding Multi-modal Information Between Reality and Videos

Videos offer rich audiovisual information that can support people in performing activities of daily living (ADLs), but they remain largely inaccessible to blind or low-vision (BLV) individuals. In cooking, BLV people often rely on non-visual cues, such as touch, taste, and smell, to navigate their environment, making it difficult to follow the predominantly audiovisual instructions found in video recipes. To address this problem, we introduce AROMA, an AI system that provides timely responses to the user based on real-time, context-aware assistance by integrating non-visual cues perceived by the user, a wearable camera feed, and video recipe content. AROMA uses a mixed-initiative approach: it responds to user requests while also proactively monitoring the video stream to offer timely alerts and guidance. This collaborative design leverages the complementary strengths of the user and AI system to align the physical environment with the video recipe, helping the user interpret their current cooking state and make sense of the steps. We evaluated AROMA through a study with eight BLV participants and offered insights for designing interactive AI systems to support BLV individuals in performing ADLs.

cs.HC↗

arXiv2025

"Sighted People Have Their Pick Of The Litter": Unpacking The Need For Digital Mental Health (DMH) Tracking Services With And For The Blind Community

The proliferation of digital mental health (DMH) tracking services promises personalized support, yet accessibility barriers limit equal access. This study investigates blind community experiences with DMH tracking services across the United States as a step toward inclusive health technology design. Working with blind advocacy organizations, we distributed a cross-sectional observational survey (n = 93) and analyzed open-ended responses using Norman and Skinner's eHealth Literacy framework. Our findings reveal significant challenges in navigation, content interpretation, and overall user experience, which impede the blind community's effective engagement with DMH tools. Results highlight the need for adaptive interfaces, accessible tracking strategies, and voice-guided interactions. These insights inform design recommendations for developers and policymakers, promoting more inclusive mental health technologies. By prioritizing accessibility, we make forward progress in ensuring that DMH tracking services fulfill their potential to support mental well-being across diverse user groups, fostering digital equality in mental health care.

cs.HC↗

arXiv2025

StereoMath: An Accessible and Musical Equation Editor

For blind and low-vision (BLV) individuals, digital math communication is uniquely difficult due to the lack of accessible tools. Currently, the state of the art is either code-based, like LaTeX, or WYSIWYG, like visual editors. However, both paradigms view math communication as primarily a visual typesetting problem, and may be accessible but difficult to use. In this paper, we present an equation editor that is built from the ground up with BLV accessibility in mind. Specifically, we notice that two of the biggest barriers with current technology are the high cognitive load and the lack of spatial relationships. Thus, we build an editor that uses spatial audio cues, muscle memory, tones, and more intuitive navigation to properly contextualize math equations. We discuss how this new paradigm can enable new levels of math communication, engagement, and literacy. Finally, we discuss natural next steps.

cs.HC↗

arXiv2024

Designing Born-Accessible Courses in Data Science and Visualization: Challenges and Opportunities of a Remote Curriculum Taught by Blind Instructors to Blind Students

While recent years have seen a growing interest in accessible visualization tools and techniques for blind people, little attention is paid to the learning opportunities and teaching strategies of data science and visualization tailored for blind individuals. Whereas the former focuses on the accessibility issues of data visualization tools, the latter is concerned with the learnability of concepts and skills for data science and visualization. In this paper, we present novel approaches to teaching data science and visualization to blind students in an online setting. Taught by blind instructors, nine blind learners having a wide range of professional backgrounds participated in a two-week summer course. We describe the course design, teaching strategies, and learning outcomes. We also discuss the challenges and opportunities of teaching data science and visualization to blind students. Our work contributes to the growing body of knowledge on accessible data science and visualization education, and provides insights into the design of online courses for blind students.

cs.HC↗

arXiv2024

MAIDR: Making Statistical Visualizations Accessible with Multimodal Data Representation

This paper investigates new data exploration experiences that enable blind users to interact with statistical data visualizations$-$bar plots, heat maps, box plots, and scatter plots$-$leveraging multimodal data representations. In addition to sonification and textual descriptions that are commonly employed by existing accessible visualizations, our MAIDR (multimodal access and interactive data representation) system incorporates two additional modalities (braille and review) that offer complementary benefits. It also provides blind users with the autonomy and control to interactively access and understand data visualizations. In a user study involving 11 blind participants, we found the MAIDR system facilitated the accurate interpretation of statistical visualizations. Participants exhibited a range of strategies in combining multiple modalities, influenced by their past interactions and experiences with data visualizations. This work accentuates the overlooked potential of combining refreshable tactile representation with other modalities and elevates the discussion on the importance of user autonomy when designing accessible data visualizations.

cs.HC↗

arXiv2022

Teaching Visual Accessibility in Introductory Data Science Classes with Multi-Modal Data Representations

Although there are various ways to represent data patterns and models, visualization has been primarily taught in many data science courses for its efficiency. Such vision-dependent output may cause critical barriers against those who are blind and visually impaired and people with learning disabilities. We argue that instructors need to teach multiple data representation methods so that all students can produce data products that are more accessible. In this paper, we argue that accessibility should be taught as early as the introductory course as part of the data science curriculum so that regardless of whether learners major in data science or not, they can have foundational exposure to accessibility. As data science educators who teach accessibility as part of our lower-division courses in two different institutions, we share specific examples that can be utilized by other data science instructors.

cs.HC↗