arXiv subjects

Tiffany Tseng

Publications and source records attributed to Tiffany Tseng.

7 recordsLinked to original sources

arXiv2026

Beyond Text: Probing K-12 Educators' Perspectives and Ideas for Learning Opportunities Leveraging Multimodal Large Language Models

Multimodal Large Language Models (MLLMs) are beginning to enable new user experiences from generated content across a range of media, including images, text, speech, and video. These capabilities have the potential to enrich learning by enabling users to interact with information using a variety of modalities, but little is known about how \textit{educators} envision how MLLMs might shape the future of learning, what challenges they encounter when using these models, and what practical needs should be considered for future implementation in educational contexts. We investigated educator perspectives through workshops with 12 K-12 educators, where participants brainstormed learning opportunities, discussed practical concerns, and prototyped MLLM learning applications using Claude 3.5 and its Artifacts feature. Through this work, we uncover how educators imagined MLLMs as a way for themselves and their students to author multimedia content, and how this could provide a pathway to support learning through iterative design. At the same time, educators anticipated challenges with younger students' ability to evaluate and refine model output to better meet their design goals. We end with implications for designing with and for MLLMs in future learning experiences.

cs.HC↗

arXiv2026

Flowcode: An AI-Powered Programming Environment for Scaffolding Iteration in Creative Computing Education

Building upon found examples is a popular way people learn to code, especially in creative coding communities where sharing projects and remixing are common practices. But effectively doing so requires being able to 1) understand how existing code works, and 2) extend it by writing code that implements your own ideas, practices that can be challenging for new creative coders. We explored how to support these two processes through the design of Flowcode, a creative coding programming environment that integrates a flowchart for visualizing code structure and a chat interface tailored to support learning to code over vibe coding. We share how we iterated on the design of Flowcode over two studies with new creative coders, reflecting on the roles visualization and friction may play in enabling productive AI-use in computing education.

cs.HC↗

arXiv2025

Facilitating Longitudinal Interaction Studies of AI Systems

UIST researchers develop tools to address user challenges. However, user interactions with AI evolve over time through learning, adaptation, and repurposing, making one time evaluations insufficient. Capturing these dynamics requires longer-term studies, but challenges in deployment, evaluation design, and data collection have made such longitudinal research difficult to implement. Our workshop aims to tackle these challenges and prepare researchers with practical strategies for longitudinal studies. The workshop includes a keynote, panel discussions, and interactive breakout groups for discussion and hands-on protocol design and tool prototyping sessions. We seek to foster a community around longitudinal system research and promote it as a more embraced method for designing, building, and evaluating UIST tools.

cs.HC↗

arXiv2025

Keyframer: Empowering Animation Design using Large Language Models

Creating 2D animations is a complex, iterative process requiring continuous adjustments to movement, timing, and coordination of multiple elements within a scene. To support designers of varying levels of experience with animation design and implementation, we developed Keyframer, a design tool that generates animation code in response to natural language prompts, enabling users to preview rendered animations inline and edit them directly through provided editors. Through a user study with 13 novices and experts in animation design and programming, we contribute 1) a categorization of semantic prompt types for describing motion and identification of a 'decomposed' prompting style where users continually adapt their goals in response to generated output; and 2) design insights on supporting iterative refinement of animations through the combination of direct editing and natural language interfaces.

cs.HC↗

arXiv2025

Crafting a Personal Journaling Practice: Negotiating Ecosystems of Materials, Personal Context, and Community in Analog Journaling

Analog journaling has grown in popularity, with journaling on paper encompassing a range of motivations, styles, and practices including planning, habit-tracking, and reflecting. Journalers develop strong personal preferences around the tools they use, the ideas they capture, and the layout in which they represent their ideas and memories. Understanding how analog journaling practices are individually shaped and crafted over time is critical to supporting the varied benefits associated with journaling, including improved mental health and positive support for identity development. To understand this development, we qualitatively analyzed publicly-shared journaling content from YouTube and Instagram and interviewed 11 journalers. We report on our identification of the journaling ecosystem in which journaling practices are shaped by materials, personal context, and communities, sharing how this ecosystem plays a role in the practices and identities of journalers as they customize their journaling routine to best suit their personal goals. Using these insights, we discuss design opportunities for how future tools can better align with and reflect the rich affordances and practices of journaling on paper.

cs.HC↗

arXiv2024

Co-ML: Collaborative Machine Learning Model Building for Developing Dataset Design Practices

Machine learning (ML) models are fundamentally shaped by data, and building inclusive ML systems requires significant considerations around how to design representative datasets. Yet, few novice-oriented ML modeling tools are designed to foster hands-on learning of dataset design practices, including how to design for data diversity and inspect for data quality. To this end, we outline a set of four data design practices (DDPs) for designing inclusive ML models and share how we designed a tablet-based application called Co-ML to foster learning of DDPs through a collaborative ML model building experience. With Co-ML, beginners can build image classifiers through a distributed experience where data is synchronized across multiple devices, enabling multiple users to iteratively refine ML datasets in discussion and coordination with their peers. We deployed Co-ML in a 2-week-long educational AIML Summer Camp, where youth ages 13-18 worked in groups to build custom ML-powered mobile applications. Our analysis reveals how multi-user model building with Co-ML, in the context of student-driven projects created during the summer camp, supported development of DDPs including incorporating data diversity, evaluating model performance, and inspecting for data quality. Additionally, we found that students' attempts to improve model performance often prioritized learnability over class balance. Through this work, we highlight how the combination of collaboration, model testing interfaces, and student-driven projects can empower learners to actively engage in exploring the role of data in ML systems.

cs.HC↗

arXiv2023

Collaborative Machine Learning Model Building with Families Using Co-ML

Existing novice-friendly machine learning (ML) modeling tools center around a solo user experience, where a single user collects only their own data to build a model. However, solo modeling experiences limit valuable opportunities for encountering alternative ideas and approaches that can arise when learners work together; consequently, it often precludes encountering critical issues in ML around data representation and diversity that can surface when different perspectives are manifested in a group-constructed data set. To address this issue, we created Co-ML -- a tablet-based app for learners to collaboratively build ML image classifiers through an end-to-end, iterative model-building process. In this paper, we illustrate the feasibility and potential richness of collaborative modeling by presenting an in-depth case study of a family (two children 11 and 14-years-old working with their parents) using Co-ML in a facilitated introductory ML activity at home. We share the Co-ML system design and contribute a discussion of how using Co-ML in a collaborative activity enabled beginners to collectively engage with dataset design considerations underrepresented in prior work such as data diversity, class imbalance, and data quality. We discuss how a distributed collaborative process, in which individuals can take on different model-building responsibilities, provides a rich context for children and adults to learn ML dataset design.

cs.HC↗