arXiv subjects

Ronald Metoyer

Publications and source records attributed to Ronald Metoyer.

At least 19 recordsLinked to original sources

arXiv2026

TeachingCoach: A Fine-Tuned Scaffolding Chatbot for Instructional Guidance to Instructors

Higher education instructors often lack timely and pedagogically grounded support, as scalable instructional guidance remains limited and existing tools rely on generic chatbot advice or non-scalable teaching center human-human consultations. We present TeachingCoach, a pedagogically grounded chatbot designed to support instructor professional development through real-time, conversational guidance. TeachingCoach is built on a data-centric pipeline that extracts pedagogical rules from educational resources and uses synthetic dialogue generation to fine-tune a specialized language model that guides instructors through problem identification, diagnosis, and strategy development. Expert evaluations show TeachingCoach produces clearer, more reflective, and more responsive guidance than a GPT-4o mini baseline, while a user study with higher education instructors highlights trade-offs between conversational depth and interaction efficiency. Together, these results demonstrate that pedagogically grounded, synthetic data driven chatbots can improve instructional support and offer a scalable design approach for future instructional chatbot systems.

cs.AI↗

arXiv2026

AI4CAREER: Responsible AI for STEM Career Development at Scale in K-16 Education

Rapid advances in artificial intelligence (AI) are reshaping how students imagine, explore, and prepare for STEM careers across K-16 education. As AI systems increasingly influence feedback, advising, and access to information about opportunities, they are becoming part of the developmental infrastructure that shapes career identity formation and readiness. Yet uncertainty remains about how AI-supported career exploration tools should be designed, governed, and evaluated at scale, particularly across developmental stages and institutional contexts. This half-day workshop convenes researchers, educators, practitioners, and policymakers to examine responsible AI for STEM career development. We focus on four themes: (1) how AI reshapes definitions and assessment of STEM career readiness; (2) appropriate roles and boundaries for AI in career decision-making; (3) developmental alignment of AI supports across the K-16 continuum; and (4) equity-related design considerations that prevent the reproduction of structural disparities. Through lightning talks, structured group activities, and cross-sector dialogue, participants will surface design tensions, articulate governance principles, and identify research gaps. The workshop aims to advance shared language and actionable frameworks for responsible, developmentally grounded AI use in STEM career learning at scale.

cs.CY↗

arXiv2026

Automated Benchmark Generation from Domain Guidelines Informed by Bloom's Taxonomy

Open-ended question answering (QA) evaluates a model's ability to perform contextualized reasoning beyond factual recall. This challenge is especially acute in practice-based domains, where knowledge is procedural and grounded in professional judgment, while most existing LLM benchmarks depend on pre-existing human exam datasets that are often unavailable in such settings. We introduce a framework for automated benchmark generation from expert-authored guidelines informed by Bloom's Taxonomy. It converts expert practices into implicit violation-based scenarios and expands them into auto-graded multiple-choice questions (MCQs) and multi-turn dialogues across four cognitive levels, enabling deterministic, reproducible, and scalable evaluation. Applied to three applied domains: teaching, dietetics, and caregiving, we find differences between model and human-like reasoning: LLMs sometimes perform relatively better on higher-order reasoning (Analyze) but fail more frequently on lower-level items (Remember). We produce large-scale, psychometrically informed benchmarks that surface these non-intuitive model behaviors and enable evaluation of contextualized reasoning in real-world settings.

cs.CL↗

arXiv2026

Nonvisual Support for Understanding and Reasoning about Data Structures

Blind and visually impaired (BVI) computer science students face systematic barriers when learning data structures: current accessibility approaches typically translate diagrams into alternative text, focusing on visual appearance rather than preserving the underlying structure essential for conceptual understanding. More accessible alternatives often do not scale in complexity, cost to produce, or both. Motivated by a recent shift to tools for creating visual diagrams from code, we propose a solution that automatically creates accessible representations from structural information about diagrams. Based on a Wizard-of-Oz study, we derive design requirements for an automated system, Arboretum, that compiles text-based diagram specifications into three synchronized nonvisual formats$\unicode{x2013}$tabular, navigable, and tactile. Our evaluation with BVI users highlights the strength of tactile graphics for complex tasks such as binary search; the benefits of offering multiple, complementary nonvisual representations; and limitations of existing digital navigation patterns for structural reasoning. This work reframes access to data structures by preserving their structural properties. The solution is a practical system to advance accessible CS education.

cs.HC↗

arXiv2025

Building AI Literacy at Home: How Families Navigate Children's Self-Directed Learning with AI

As generative AI becomes embedded in children's learning spaces, families face new challenges in guiding its use. Middle childhood (ages 7-13) is a critical stage where children seek autonomy even as parental influence remains strong. Using self-directed learning (SDL) as a lens, we examine how parents perceive and support children's developing AI literacy through focus groups with 13 parent-child pairs. Parents described evolving phases of engagement driven by screen time, self-motivation, and growing knowledge. While many framed AI primarily as a study tool, few considered its non-educational roles or risks, such as privacy and infrastructural embedding. Parents also noted gaps in their own AI understanding, often turning to joint exploration and engagement as a form of co-learning. Our findings reveal how families co-construct children's AI literacy, exposing tensions between practical expectations and critical literacies, and provide design implications that foster SDL while balancing autonomy and oversight.

cs.HC↗

arXiv2025

Adaptive Testing for LLM Evaluation: A Psychometric Alternative to Static Benchmarks

Evaluating large language models (LLMs) typically requires thousands of benchmark items, making the process expensive, slow, and increasingly impractical at scale. Existing evaluation protocols rely on average accuracy over fixed item sets, treating all items as equally informative despite substantial variation in difficulty and discrimination. We introduce ATLAS, an adaptive testing framework based on Item Response Theory (IRT) that estimates model ability using Fisher information-guided item selection. ATLAS reduces the number of required items by up to 90% while maintaining measurement precision. For instance, it matches whole-bank ability estimates using only 41 items (0.157 MAE) on HellaSwag (5,600 items). We further reconstruct accuracy from ATLAS's ability estimates and find that reconstructed accuracies closely match raw accuracies across all five benchmarks, indicating that ability ${\theta}$ preserves the global performance structure. At the same time, ${\theta}$ provides finer discrimination within accuracy-equivalent models: among more than 3,000 evaluated models, 23-31% shift by more than 10 rank positions, and models with identical accuracies receive meaningfully different ability estimates. Code and calibrated item banks are available at https://github.com/Peiyu-Georgia-Li/ATLAS.git.

cs.CL↗

arXiv2025

AI Academy: Building Generative AI Literacy in Higher Ed Instructors

Generative AI is reshaping higher education, yet research has focused largely on students, while instructors remain understudied despite their central role in mediating adoption and modeling responsible use. We present the \textit{AI Academy}, a faculty development program that combined AI exploration with pedagogical reflection and peer learning. Rather than a course evaluated for outcomes, the Academy provided a setting to study how instructors build AI literacies in relation to tools, policies, peer practices, and institutional supports. We studied 25 instructors through pre/post surveys, learning logs, and facilitator interviews. Findings show AI literacy gains alongside new insights. We position instructors as designers of responsible AI practices and contribute a replicable program model, a co-constructed survey instrument, and design insights for professional development that adapts to evolving tools and fosters ethical discussion.

cs.HC↗

arXiv2025

Exploring Conversational Design Choices in LLMs for Pedagogical Purposes: Socratic and Narrative Approaches for Improving Instructor's Teaching Practice

Large language models (LLMs) typically generate direct answers, yet they are increasingly used as learning tools. Studying instructors' usage is critical, given their role in teaching and guiding AI adoption in education. We designed and evaluated TeaPT, an LLM for pedagogical purposes that supports instructors' professional development through two conversational approaches: a Socratic approach that uses guided questioning to foster reflection, and a Narrative approach that offers elaborated suggestions to extend externalized cognition. In a mixed-method study with 41 higher-education instructors, the Socratic version elicited greater engagement, while the Narrative version was preferred for actionable guidance. Subgroup analyses further revealed that less-experienced, AI-optimistic instructors favored the Socratic version, whereas more-experienced, AI-cautious instructors preferred the Narrative version. We contribute design implications for LLMs for pedagogical purposes, showing how adaptive conversational approaches can support instructors with varied profiles while highlighting how AI attitudes and experience shape interaction and learning.

cs.HC↗

arXiv2025

Building Scaffolding Dialogue Data with LLM-Simulated Novices

High-quality, multi-turn instructional dialogues between novices and experts are essential for developing AI systems that support teaching, learning, and decision-making. These dialogues often involve scaffolding -- the process by which an expert supports a novice's thinking through questions, feedback, and step-by-step guidance. However, such data are scarce due to privacy concerns in recording and the vulnerability inherent in help-seeking. We present SimInstruct, a scalable, expert-in-the-loop tool for collecting scaffolding dialogues. Using teaching development coaching as an example domain, SimInstruct simulates novice instructors via LLMs, varying their teaching challenges and LLM's persona traits, while human experts provide multi-turn feedback, reasoning, and instructional support. This design enables the creation of realistic, pedagogically rich dialogues without requiring real novice participants. Our results reveal that persona traits, such as extroversion and introversion, meaningfully influence how experts engage. Compared to real mentoring recordings, SimInstruct dialogues demonstrate comparable pedagogical relevance and cognitive depth. Experts also reported the process as engaging and reflective, improving both data quality and their own professional insight. We further fine-tuned a LLaMA model to be an expert model using the augmented dataset, which outperformed GPT-4o in instructional quality. Our analysis highlights GPT-4o's limitations in weak reflective questioning, overuse of generic praise, a condescending tone, and a tendency to overwhelm novices with excessive suggestions.

cs.AI↗

arXiv2025

Understanding Student Attitudes and Acceptability of GenAI Tools in Higher Ed: Scale Development and Evaluation

As generative AI (GenAI) tools like ChatGPT become more common in higher education, understanding student attitudes is essential for evaluating their educational impact and supporting responsible AI integration. This study introduces a validated survey instrument designed to assess students' perceptions of GenAI, including its acceptability for academic tasks, perceived influence on learning and careers, and broader societal concerns. We administered the survey to 297 undergraduates at a U.S. university. The instrument includes six thematic domains: institutional understanding, fairness and trust, academic and career influence, societal concerns, and GenAI use in writing and coursework. Exploratory factor analysis revealed four attitudinal dimensions: societal concern, policy clarity, fairness and trust, and career impact. Subgroup analyses identified statistically significant differences across student backgrounds. Male students and those speaking a language other than English at home rated GenAI use in writing tasks as more acceptable. First-year students expressed greater societal concern than upper-year peers. Students from multilingual households perceived greater clarity in institutional policy, while first-generation students reported a stronger belief in GenAI's impact on future careers. This work contributes a practical scale for evaluating the student impact of GenAI tools, informing the design of educational AI systems.

cs.CY↗

arXiv2025

Bridging the AI Adoption Gap: Designing an Interactive Pedagogical Agent for Higher Education Instructors

Instructors play a pivotal role in integrating AI into education, yet their adoption of AI-powered tools remains inconsistent. Despite this, limited research explores how to design AI tools that support broader instructor adoption. This study applies a human-centered design approach, incorporating qualitative methods, to investigate the design of interactive pedagogical agents that provide instructional suggestions in response to instructors' questions. We conducted a formative study involving interviews with five pedagogy experts to examine existing strategies for supporting instructors' pedagogical needs. Building on these insights, we facilitated a participatory design session with ten pedagogy experts, where participants reviewed a storyboard depicting a chatbot designed for instructors with varying levels of AI literacy and differing attitudes toward AI. Experts also evaluated the quality of LLM-generated suggestions based on common teaching challenges. Our findings highlight the need for chatbot interactions that foster trust, especially for AI-conservative instructors. Experts emphasized the importance of social transparency (for example, showing how peers use the tool) and allowing instructors to flexibly control how much or how little they engage with the system. We also propose design recommendations to enhance the quality of AI-generated teaching suggestions, such as adapting them to reflect instructors' prior teaching experience. This work underscores the urgent need to support AI-conservative instructors, as AI literacy and attitudes are closely intertwined. Without thoughtful design, there is a risk of widening pedagogical divides and reducing students' learning opportunities.

cs.HC↗

arXiv2022

Recipe2Vec: Multi-modal Recipe Representation Learning with Graph Neural Networks

Learning effective recipe representations is essential in food studies. Unlike what has been developed for image-based recipe retrieval or learning structural text embeddings, the combined effect of multi-modal information (i.e., recipe images, text, and relation data) receives less attention. In this paper, we formalize the problem of multi-modal recipe representation learning to integrate the visual, textual, and relational information into recipe embeddings. In particular, we first present Large-RG, a new recipe graph data with over half a million nodes, making it the largest recipe graph to date. We then propose Recipe2Vec, a novel graph neural network based recipe embedding model to capture multi-modal information. Additionally, we introduce an adversarial attack strategy to ensure stable learning and improve performance. Finally, we design a joint objective function of node classification and adversarial learning to optimize the model. Extensive experiments demonstrate that Recipe2Vec outperforms state-of-the-art baselines on two classic food study tasks, i.e., cuisine category classification and region prediction. Dataset and codes are available at https://github.com/meettyj/Recipe2Vec.

cs.LG↗

arXiv2022

RecipeRec: A Heterogeneous Graph Learning Model for Recipe Recommendation

Recipe recommendation systems play an essential role in helping people decide what to eat. Existing recipe recommendation systems typically focused on content-based or collaborative filtering approaches, ignoring the higher-order collaborative signal such as relational structure information among users, recipes and food items. In this paper, we formalize the problem of recipe recommendation with graphs to incorporate the collaborative signal into recipe recommendation through graph modeling. In particular, we first present URI-Graph, a new and large-scale user-recipe-ingredient graph. We then propose RecipeRec, a novel heterogeneous graph learning model for recipe recommendation. The proposed model can capture recipe content and collaborative signal through a heterogeneous graph neural network with hierarchical attention and an ingredient set transformer. We also introduce a graph contrastive augmentation strategy to extract informative graph knowledge in a self-supervised manner. Finally, we design a joint objective function of recommendation and contrastive learning to optimize the model. Extensive experiments demonstrate that RecipeRec outperforms state-of-the-art methods for recipe recommendation. Dataset and codes are available at https://github.com/meettyj/RecipeRec.

cs.IR↗

arXiv2022

Generating and Visualizing Trace Link Explanations

Recent breakthroughs in deep-learning (DL) approaches have resulted in the dynamic generation of trace links that are far more accurate than was previously possible. However, DL-generated links lack clear explanations, and therefore non-experts in the domain can find it difficult to understand the underlying semantics of the link, making it hard for them to evaluate the link's correctness or suitability for a specific software engineering task. In this paper we present a novel NLP pipeline for generating and visualizing trace link explanations. Our approach identifies domain-specific concepts, retrieves a corpus of concept-related sentences, mines concept definitions and usage examples, and identifies relations between cross-artifact concepts in order to explain the links. It applies a post-processing step to prioritize the most likely acronyms and definitions and to eliminate non-relevant ones. We evaluate our approach using project artifacts from three different domains of interstellar telescopes, positive train control, and electronic health-care systems, and then report coverage, correctness, and potential utility of the generated definitions. We design and utilize an explanation interface which leverages concept definitions and relations to visualize and explain trace link rationales, and we report results from a user study that was conducted to evaluate the effectiveness of the explanation interface. Results show that the explanations presented in the interface helped non-experts to understand the underlying semantics of a trace link and improved their ability to vet the correctness of the link.

cs.SE↗

arXiv2018

Documenting API Input/Output Examples

When learning to use an Application Programming Interface (API), programmers need to understand the inputs and outputs (I/O) of the API functions. Current documentation tools automatically document the static information of I/O, such as parameter types and names. What is missing from these tools is dynamic information, such as I/O examples---actual valid values of inputs that produce certain outputs. In this paper, we demonstrate a prototype toolset we built to generate I/O examples. Our tool logs I/O values when API functions are executed, for example in running test suites. Then, the tool puts I/O values into API documents as I/O examples. Our tool has three programs: 1) funcWatch, which collects I/O values when API developers run test suites, 2) ioSelect, which selects one I/O example from a set of I/O values, and 3) ioPresent, which embeds the I/O examples into documents. In a preliminary evaluation, we used our tool to generate four hundred I/O examples for three C libraries: ffmpeg, libssh, and protobuf-c.

cs.SE↗

arXiv2018

How Do Practitioners Perceive Assurance Cases in Safety-Critical Software Systems?

Safety-critical software systems are those whose failure or malfunction could result in casualty and/or serious financial loss. In such systems, safety assurance cases (SACs) are an emerging approach that adopts a proactive strategy to produce structuralized safety justifications and arguments. While SACs are recommended in many software-intensive safety-critical domains, the lack of knowledge regarding the practitioners' perspectives on using SACs hinders effective adoption of this approach. To gain such knowledge, we interviewed nine practitioners and safety experts who focused on safety-critical software systems. In general, our participants found the SAC approach beneficial for communication of safety arguments and management of safety issues in a multidisciplinary setting. The challenges they faced when using SACs were primarily associated with (1) a lack of tool support, (2) insufficient process integration, and (3) scarcity of experienced personnel. To overcome those challenges, our participants suggested tactics that focused on creating direct safety arguments. Process and organizational adjustments are also needed to streamline SAC analysis and creation. Finally, our participants emphasized the importance of knowledge sharing about SACs across software-intensive safety-critical domains.

cs.SE↗

arXiv2018

Coupling Story to Visualization: Using Textual Analysis as a Bridge Between Data and Interpretation

Online writers and journalism media are increasingly combining visualization (and other multimedia content) with narrative text to create narrative visualizations. Often, however, the two elements are presented independently of one another. We propose an approach to automatically integrate text and visualization elements. We begin with a writer's narrative that presumably can be supported with visual data evidence. We leverage natural language processing, quantitative narrative analysis, and information visualization to (1) automatically extract narrative components (who, what, when, where) from data-rich stories, and (2) integrate the supporting data evidence with the text to develop a narrative visualization. We also employ bidirectional interaction from text to visualization and visualization to text to support reader exploration in both directions. We demonstrate the approach with a case study in the data-rich field of sports journalism.

cs.HC↗

arXiv2017

Factoring Exogenous State for Model-Free Monte Carlo

Policy analysts wish to visualize a range of policies for large simulator-defined Markov Decision Processes (MDPs). One visualization approach is to invoke the simulator to generate on-policy trajectories and then visualize those trajectories. When the simulator is expensive, this is not practical, and some method is required for generating trajectories for new policies without invoking the simulator. The method of Model-Free Monte Carlo (MFMC) can do this by stitching together state transitions for a new policy based on previously-sampled trajectories from other policies. This "off-policy Monte Carlo simulation" method works well when the state space has low dimension but fails as the dimension grows. This paper describes a method for factoring out some of the state and action variables so that MFMC can work in high-dimensional MDPs. The new method, MFMCi, is evaluated on a very challenging wildfire management MDP.

cs.LG↗