arXiv subjects

Brian Magerko

Publications and source records attributed to Brian Magerko.

5 recordsLinked to original sources

arXiv2025

Rethinking AI Evaluation in Education: The TEACH-AI Framework and Benchmark for Generative AI Assistants

As generative artificial intelligence (AI) continues to transform education, most existing AI evaluations rely primarily on technical performance metrics such as accuracy or task efficiency while overlooking human identity, learner agency, contextual learning processes, and ethical considerations. In this paper, we present TEACH-AI (Trustworthy and Effective AI Classroom Heuristics), a domain-independent, pedagogically grounded, and stakeholder-aligned framework with measurable indicators and a practical toolkit for guiding the design, development, and evaluation of generative AI systems in educational contexts. Built on an extensive literature review and synthesis, the ten-component assessment framework and toolkit checklist provide a foundation for scalable, value-aligned AI evaluation in education. TEACH-AI rethinks "evaluation" through sociotechnical, educational, theoretical, and applied lenses, engaging designers, developers, researchers, and policymakers across AI and education. Our work invites the community to reconsider what constructs "effective" AI in education and to design model evaluation approaches that promote co-creation, inclusivity, and long-term human, social, and educational impact.

cs.CY↗

arXiv2025

Sound Clouds: Exploring ambient intelligence in public spaces to elicit deep human experience of awe, wonder, and beauty

While the ambient intelligence (AmI) systems we encounter in our daily lives, including security monitoring and energy-saving systems, typically serve pragmatic purposes, we wonder how we can design and implement ambient artificial intelligence experiences in public spaces that elicit deep human feelings of awe, wonder, and beauty. As a manifestation, we introduce Sound Clouds, an immersive art installation that generates live music based on participants' interaction with several human-height spheres. Our installation serves as a provocation into future ambient intelligence that provokes, not limits, the future possibilities of AmI.

cs.HC↗

arXiv2025

Generative AI Literacy: A Comprehensive Framework for Literacy and Responsible Use

After the release of several widely adopted artificial intelligence (AI) literacy guidelines by 2021, the unprecedented rise of generative AI since 2023 has transformed the way we work and acquire information worldwide. Unlike traditional AI algorithms, generative AI exhibits distinct and more nuanced characteristics. However, a lack of robust understanding of generative AI hinders individuals' ability to use generative AI effectively, critically, and responsibly, which we can call generative AI literacy. To address this gap, we reviewed and synthesized existing literature and proposed generative AI literacy guidelines with 12 items organized into four aspects: (1) generative AI tool selection and prompting, (2) understanding interaction with generative AI, (3) understanding generative AI outputs, and (4) high-level understanding of generative AI technologies. These guidelines aim to support schools, companies, and organizations in developing frameworks that support their members to use generative AI in an efficient, ethical, and informed way.

cs.HC↗

arXiv2023

Making Sense of Machine Learning: Integrating Youth's Conceptual, Creative, and Critical Understandings of AI

Understanding how youth make sense of machine learning and how learning about machine learning can be supported in and out of school is more relevant than ever before as young people interact with machine learning powered applications everyday; while connecting with friends, listening to music, playing games, or attending school. In this symposium, we present different perspectives on understanding how learners make sense of machine learning in their everyday lives, how sensemaking of machine learning can be supported in and out of school through the construction of applications, and how youth critically evaluate machine learning powered systems. We discuss how sensemaking of machine learning applications involves the development and integration of conceptual, creative, and critical understandings that are increasingly important to prepare youth to participate in the world.

cs.CY↗

arXiv2021

Empirically Evaluating Creative Arc Negotiation for Improvisational Decision-making

Action selection from many options with few constraints is crucial for improvisation and co-creativity. Our previous work proposed creative arc negotiation to solve this problem, i.e., selecting actions to follow an author-defined `creative arc' or trajectory over estimates of novelty, unexpectedness, and quality for potential actions. The CARNIVAL agent architecture demonstrated this approach for playing the Props game from improv theatre in the Robot Improv Circus installation. This article evaluates the creative arc negotiation experience with CARNIVAL through two crowdsourced observer studies and one improviser laboratory study. The studies focus on subjects' ability to identify creative arcs in performance and their preference for creative arc negotiation compared to a random selection baseline. Our results show empirically that observers successfully identified creative arcs in performances. Both groups also preferred creative arc negotiation in agent creativity and logical coherence, while observers enjoyed it more too.

cs.AI↗