arXiv subjects

Pengcheng An

Publications and source records attributed to Pengcheng An.

At least 37 records · Page 2Linked to original sources

arXiv2024

MultiSurf-GPT: Facilitating Context-Aware Reasoning with Large-Scale Language Models for Multimodal Surface Sensing

Surface sensing is widely employed in health diagnostics, manufacturing and safety monitoring. Advances in mobile sensing affords this potential for context awareness in mobile computing, typically with a single sensing modality. Emerging multimodal large-scale language models offer new opportunities. We propose MultiSurf-GPT, which utilizes the advanced capabilities of GPT-4o to process and interpret diverse modalities (radar, microscope and multispectral data) uniformly based on prompting strategies (zero-shot and few-shot prompting). We preliminarily validated our framework by using MultiSurf-GPT to identify low-level information, and to infer high-level context-aware analytics, demonstrating the capability of augmenting context-aware insights. This framework shows promise as a tool to expedite the development of more complex context-aware applications in the future, providing a faster, more cost-effective, and integrated solution.

cs.HC↗

arXiv2024

FlowGPT: Exploring Domains, Output Modalities, and Goals of Community-Generated AI Chatbots

The advent of Generative AI and Large Language Models has not only enhanced the intelligence of interactive applications but also catalyzed the formation of communities passionate about customizing these AI capabilities. FlowGPT, an emerging platform for sharing AI prompts and use cases, exemplifies this trend, attracting many creators who develop and share chatbots with a broader community. Despite its growing popularity, there remains a significant gap in understanding the types and purposes of the AI tools created and shared by community members. In this study, we delve into FlowGPT and present our preliminary findings on the domain, output modality, and goals of chatbots. We aim to highlight common types of AI applications and identify future directions for research in AI-sharing communities.

cs.HC↗

arXiv2024

Influencer: Empowering Everyday Users in Creating Promotional Posts via AI-infused Exploration and Customization

Creating promotional posts on social platforms enables everyday users to disseminate their creative outcomes, engage in community exchanges, or generate additional income from micro-businesses. However, creating eye-catching posts combining both original, appealing images and articulate, effective captions can be rather challenging and time-consuming for everyday users who are mostly design novices. We propose Influen, an interactive tool to assist novice creators in crafting high-quality promotional post designs, achieving quick design ideation and unencumbered content creation through AI. Within Influencer, we contribute a multi-dimensional recommendation framework that allows users to intuitively generate new ideas through example-based image and caption recommendation. Further, Influencer implements a holistic promotional post design system that supports context-aware image and caption exploration considering brand messages and user-specified design constraints, flexible fusion of various images and captions, and a mind-map-like layout for thinking tracking and post-recording. We evaluated Influencer with 12 design enthusiasts through an in-lab user study by comparing it to a baseline combining Google Search + Figma. Quantitative and qualitative results demonstrate that \sysname{} is effective in assisting design novices to generate ideas as well as creative and diverse promotional posts with user-friendly interaction.

cs.SE↗

arXiv2024

Designing AI-Enabled Games to Support Social-Emotional Learning for Children with Autism Spectrum Disorders

Children with autism spectrum disorder (ASD) experience challenges in grasping social-emotional cues, which can result in difficulties in recognizing emotions and understanding and responding to social interactions. Social-emotional intervention is an effective method to improve emotional understanding and facial expression recognition among individuals with ASD. Existing work emphasizes the importance of personalizing interventions to meet individual needs and motivate engagement for optimal outcomes in daily settings. We design a social-emotional game for ASD children, which generates personalized stories by leveraging the current advancement of artificial intelligence. Via a co-design process with five domain experts, this work offers several design insights into developing future AI-enabled gamified systems for families with autistic children. We also propose a fine-tuned AI model and a dataset of social stories for different basic emotions.

cs.HC↗

arXiv2024

PeerGPT: Probing the Roles of LLM-based Peer Agents as Team Moderators and Participants in Children's Collaborative Learning

In children's collaborative learning, effective peer conversations can significantly enhance the quality of children's collaborative interactions. The integration of Large Language Model (LLM) agents into this setting explores their novel role as peers, assessing impacts as team moderators and participants. We invited two groups of participants to engage in a collaborative learning workshop, where they discussed and proposed conceptual solutions to a design problem. The peer conversation transcripts were analyzed using thematic analysis. We discovered that peer agents, while managing discussions effectively as team moderators, sometimes have their instructions disregarded. As participants, they foster children's creative thinking but may not consistently provide timely feedback. These findings highlight potential design improvements and considerations for peer agents in both roles.

cs.HC↗

arXiv2024

"My lollipop dropped..."-Probing Design Opportunities for SEL Agents through Children's Peer Co-Creation of Social-Emotional Stories

This Late-Breaking Work explores the significance of socio-emotional learning (SEL) and the challenges inherent in designing child-appropriate technologies, namely storytelling agents, to support SEL. We aim to probe their needs and preferences regarding agents for SEL by conducting co-design which involves children co-creating characters and social-emotional stories. We conducted collaborative story-making activities with children aged four to six years old. Our findings could inform the design of both verbal and nonverbal interactions of agents, which are to be aligned with children's understanding and interest. Based on the child-led peer co-design, our work enhances the understanding of SEL agent designs and behaviors tailored to children's socio-emotional needs, thereby offering practical implications for more effective SEL tools in future HCI research and practice.

cs.HC↗

arXiv2024

EmoWear: Exploring Emotional Teasers for Voice Message Interaction on Smartwatches

Voice messages, by nature, prevent users from gauging the emotional tone without fully diving into the audio content. This hinders the shared emotional experience at the pre-retrieval stage. Research scarcely explored "Emotional Teasers"-pre-retrieval cues offering a glimpse into an awaiting message's emotional tone without disclosing its content. We introduce EmoWear, a smartwatch voice messaging system enabling users to apply 30 animation teasers on message bubbles to reflect emotions. EmoWear eases senders' choice by prioritizing emotions based on semantic and acoustic processing. EmoWear was evaluated in comparison with a mirroring system using color-coded message bubbles as emotional cues (N=24). Results showed EmoWear significantly enhanced emotional communication experience in both receiving and sending messages. The animated teasers were considered intuitive and valued for diverse expressions. Desirable interaction qualities and practical implications are distilled for future design. We thereby contribute both a novel system and empirical knowledge concerning emotional teasers for voice messaging.

cs.HC↗

arXiv2024

"When He Feels Cold, He Goes to the Seahorse"-Blending Generative AI into Multimaterial Storymaking for Family Expressive Arts Therapy

Storymaking, as an integrative form of expressive arts therapy, is an effective means to foster family communication. Yet, the integration of generative AI as expressive materials in therapeutic storymaking remains underexplored. And there is a lack of HCI implications on how to support families and therapists in this context. Addressing this, our study involved five weeks of storymaking sessions with seven families guided by a professional therapist. In these sessions, the families used both traditional art-making materials and image-based generative AI to create and evolve their family stories. Via the rich empirical data and commentaries from four expert therapists, we contextualize how families creatively melded AI and traditional expressive materials to externalize their ideas and feelings. Through the lens of Expressive Therapies Continuum (ETC), we characterize the therapeutic implications of AI as expressive materials. Desirable interaction qualities to support children, parents, and therapists are distilled for future HCI research.

cs.HC↗

arXiv2023

Affective Affordance of Message Balloon Animations: An Early Exploration of AniBalloons

We introduce the preliminary exploration of AniBalloons, a novel form of chat balloon animations aimed at enriching nonverbal affective expression in text-based communications. AniBalloons were designed using extracted motion patterns from affective animations and mapped to six commonly communicated emotions. An evaluation study with 40 participants assessed their effectiveness in conveying intended emotions and their perceived emotional properties. The results showed that 80% of the animations effectively conveyed the intended emotions. AniBalloons covered a broad range of emotional parameters, comparable to frequently used emojis, offering potential for a wide array of affective expressions in daily communication. The findings suggest AniBalloons' promise for enhancing emotional expressiveness in text-based communication and provide early insights for future affective design.

cs.HC↗

arXiv2023

NaMemo2: Facilitating Teacher-Student Interaction with Theory-Based Design and Student Autonomy Consideration

Teacher-student interaction (TSI) is essential for learning efficiency and harmonious teacher-student interpersonal relationships. However, studies on TSI support tools often focus on teacher needs while neglecting student needs and autonomy. To enhance both lecturer competence in delivering interpersonal interaction and student autonomy in TSI, we developed NaMemo2, a novel augmented-reality system that allows students to express their willingness to TSI and displays student information to teachers during lectures. The design and evaluation process follows a new framework, STUDIER, which can facilitate the development of theory-based ethnics-aware TSI support tools in general. The quantitative results of our four-week field study with four classes in a university suggested that NaMemo2 can improve 1) TSI in the classroom from both teacher and student perspectives, 2) student attitudes and willingness to TSI, and 3) student attitudes to the deployment of NaMemo2. The qualitative feedback from students and teachers indicated that improving TSI may be responsible for improved attention in students and a better classroom atmosphere during lectures.

cs.HC↗

arXiv2022

Puffy: A Step-by-step Guide to Craft Bio-inspired Artifacts with Interactive Materiality

A rising number of HCI scholars have begun to use materiality as a starting point for exploring the design's potential and restrictions. Despite the theoretical flourishing, the practical design process and instruction for beginner practitioners are still in scarcity. We leveraged the pictorial format to illustrate our crafting process of Puffy, a bio-inspired artifact that features a cilia-mimetic surface expressing anthropomorphic qualities through shape changes. Our approach consists of three key activities (i.e., analysis, synthesis, and detailing) interlaced recursively throughout the journey. Using this approach, we analyzed different input sources, synthesized peers' critiques and self-reflection, and detailed the designed experience with iterative prototypes. Building on a reflective analysis of our approach, we concluded with a set of practical implications and design recommendations to inform other practitioners to initiate their investigations in interactive materiality.

cs.HC↗

arXiv2022

Dancing with the Unexpected and Beyond: The Use of AI Assistance in Design Fiction Creation

The creation process of design fiction is going participatory and inclusive with non experts. Recognizing the potential of artificial intelligence in creativity support, we explore the use of AI assistance in creating design fiction. This investigation is based on a workshop on future work in 2040 with Chinese youth. We look into fiction quality, participants experiences with the AI agent, and their ways of incorporating those texts into writing. Our findings show that human writers while responding to messy and unexpected AI-generated texts, can elevate the richness and creativity in writing and initiate joyful and inspirational interactions. Furthermore, for the design of AI assistance in creativity support, we suggest two implications of enhancing interactional quality between human and AI and prompt programming. Our study indicates the potential of applying design fiction outside the design context using a more inclusive approach for future speculation with critical reflection on technology.

cs.HC↗

arXiv2021

VibEmoji: Exploring User-authoring Multi-modal Emoticons in Social Communication

Emoticons are indispensable in online communications. With users' growing needs for more customized and expressive emoticons, recent messaging applications begin to support (limited) multi-modal emoticons: e.g., enhancing emoticons with animations or vibrotactile feedback. However, little empirical knowledge has been accumulated concerning how people create, share and experience multi-modal emoticons in everyday communication, and how to better support them through design. To tackle this, we developed VibEmoji, a user-authoring multi-modal emoticon interface for mobile messaging. Extending existing designs, VibEmoji grants users greater flexibility to combine various emoticons, vibrations, and animations on-the-fly, and offers non-aggressive recommendations based on these components' emotional relevance. Using VibEmoji as a probe, we conducted a four-week field study with 20 participants, to gain new understandings from in-the-wild usage and experience, and extract implications for design. We thereby contribute both a novel system and various insights for supporting users' creation and communication of multi-modal emoticons.

cs.HC↗

arXiv2021

Designing Mobile EEG Neurofeedback Games for Children with Autism: Implications from Industry Practice

Neurofeedback games are an effective and playful approach to enhance certain social and attentional capabilities in children with autism, which are promising to become widely accessible along with the commercialization of mobile EEG modules. However, little industry-based experiences are shared, regarding how to better design neurofeedback games to fine-tune their playability and user experiences for autistic children. In this paper, we review the experiences we gained from industry practice, in which a series of mobile EEG neurofeedback games have been developed for preschool autistic children. We briefly describe our design and development in a one-year collaboration with a special education center involving a group of stakeholders: children with autism and their caregivers and parents. We then summarize four concrete implications we learnt concerning the design of game characters, game narratives, as well as gameplay elements, which aim to support future work in creating better neurofeedback games for preschool children with autism.

cs.HC↗

arXiv2021

Explainability via Interactivity? Supporting Nonexperts' Sensemaking of Pretrained CNN by Interacting with Their Daily Surroundings

Current research on Explainable AI (XAI) heavily targets on expert users (data scientists or AI developers). However, increasing importance has been argued for making AI more understandable to nonexperts, who are expected to leverage AI techniques, but have limited knowledge about AI. We present a mobile application to support nonexperts to interactively make sense of Convolutional Neural Networks (CNN); it allows users to play with a pretrained CNN by taking pictures of their surrounding objects. We use an up-to-date XAI technique (Class Activation Map) to intuitively visualize the model's decision (the most important image regions that lead to a certain result). Deployed in a university course, this playful learning tool was found to support design students to gain vivid understandings about the capabilities and limitations of pretrained CNNs in real-world environments. Concrete examples of students' playful explorations are reported to characterize their sensemaking processes reflecting different depths of thought.

cs.HC↗

arXiv2020

How Peripheral Interactive Systems Can Support Teachers with Differentiated Instruction: Using FireFlies as a Probe

Teachers' response to the real-time needs of diverse learners in the classroom is important for each learner's success. Teachers who give differentiated instruction (DI) provide pertinent support to each student and acknowledge their differences in learning style and pace. However, due to the already complex and intensive routines in classrooms, it is demanding and time-consuming for teachers to implement DI on-the-spot. This study aims to explore how to ease teachers' classroom differentiation by enabling effortless, low-threshold student-teacher communications through a peripheral interactive system. Namely, we present a six-week study, in which we iteratively co-designed and field-tested interaction solutions with eight school teachers, using a set of distributed, interactive LED-objects (the 'FireFlies' platform). By connecting our findings to the theories of DI, we contribute empirical knowledge about the advantages and limitations of a peripheral interactive system in supporting DI. Taken together, we summarize concrete opportunities and recommendations for future design.

physics.ed-ph↗

arXiv2020

NaMemo: Enhancing Lecturers' Interpersonal Competence of Remembering Students' Names

Addressing students by their names helps a teacher to start building rapport with students and thus facilitates their classroom participation. However, this basic yet effective skill has become rather challenging for university lecturers, who have to handle large-sized (sometimes exceeding 100) groups in their daily teaching. To enhance lecturers' competence in delivering interpersonal interaction, we developed NaMemo, a real-time name-indicating system based on a dedicated face-recognition pipeline. This paper presents the system design, the pilot feasibility test, and our plan for the following study, which aims to evaluate NaMemo's impacts on learning and teaching, as well as to probe design implications including privacy considerations.

cs.HC↗

arXiv2020

Dandelion Diagram: Aggregating Positioning and Orientation Data in the Visualization of Classroom Proxemics

In the past two years, an emerging body of HCI work has been focused on classroom proxemics - how teachers divide time and attention over students in the different regions of the classroom. Tracking and visualizing this implicit yet relevant dimension of teaching can benefit both research and teacher professionalization. Prior work has proved the value of depicting teachers' whereabouts. Yet a major opportunity remains in the design of new, synthesized visualizations that help researchers and practitioners to gain more insights in the vast tracking data. We present Dandelion Diagram, a synthesized heatmap technique that combines both teachers' positioning and orientation (heading) data, and affords richer representations in addition to whereabouts - For example, teachers' attention pattern (which directions they were attending to), and their mobility pattern (i.e., trajectories in the classroom). Utilizing various classroom data from a field study, this paper illustrates the design and utility of Dandelion Diagram.

cs.HC↗