arXiv subjects

Eason Chen

Publications and source records attributed to Eason Chen.

33 records · Page 2Linked to original sources

arXiv2024

Exploring the Roles of NLP-based Dialog Indicators in Predicting User Experience in interacting with Large Language Model System

The use of Large Language Models for dialogue systems is rising, presenting a new challenge: how do we assess users' chat experience in these systems? Leveraging Natural Language Processing (NLP)-powered dialog analyzers to create dialog indicators like Coherence and Emotion has the potential to predict the chat experience. In this paper, we proposed a conceptual model to explain the relationship between the dialog indicators and various factors related to the chat experience, such as users' intentions, affinity toward dialog agents, and prompts of the agents' characters. We evaluated the conceptual model using PLS-SEM with 120 participants and found it well fit. Our results suggest that dialog indicators can predict the chat experience and fully mediate the impact of prompts and user intentions. Additionally, users' affinity toward agents can partially explain these predictions. Our findings demonstrate the potential of using dialog indicators in predicting the chat experience. Through the conceptual model we propose, researchers can apply the dialog analyzers to generate dialog indicators to constantly monitor the dialog process and improve the user's chat experience accordingly.

cs.HC↗

arXiv2024

Bringing Generative AI to Adaptive Learning in Education

The recent surge in generative AI technologies, such as large language models and diffusion models, has boosted the development of AI applications in various domains, including science, finance, and education. Concurrently, adaptive learning, a concept that has gained substantial interest in the educational sphere, has proven its efficacy in enhancing students' learning efficiency. In this position paper, we aim to shed light on the intersectional studies of these two methods, which combine generative AI with adaptive learning concepts. By presenting discussions about the benefits, challenges, and potentials in this field, we argue that this union will contribute significantly to the development of the next-stage learning format in education.

cs.CY↗

arXiv2024

GPTutor: Great Personalized Tutor with Large Language Models for Personalized Learning Content Generation

We developed GPTutor, a pioneering web application designed to revolutionize personalized learning by leveraging the capabilities of Generative AI at scale. GPTutor adapts educational content and practice exercises to align with individual students' interests and career goals, enhancing their engagement and understanding of critical academic concepts. The system uses a serverless architecture to deliver personalized and scalable learning experiences. By integrating advanced Chain-of-Thoughts prompting methods, GPTutor provides a personalized educational journey that not only addresses the unique interests of each student but also prepares them for future professional success. This demo paper presents the design, functionality, and potential of GPTutor to foster a more engaging and effective educational environment.

cs.HC↗

arXiv2024

How Can I Improve? Using GPT to Highlight the Desired and Undesired Parts of Open-ended Responses

Automated explanatory feedback systems play a crucial role in facilitating learning for a large cohort of learners by offering feedback that incorporates explanations, significantly enhancing the learning process. However, delivering such explanatory feedback in real-time poses challenges, particularly when high classification accuracy for domain-specific, nuanced responses is essential. Our study leverages the capabilities of large language models, specifically Generative Pre-Trained Transformers (GPT), to explore a sequence labeling approach focused on identifying components of desired and less desired praise for providing explanatory feedback within a tutor training dataset. Our aim is to equip tutors with actionable, explanatory feedback during online training lessons. To investigate the potential of GPT models for providing the explanatory feedback, we employed two commonly-used approaches: prompting and fine-tuning. To quantify the quality of highlighted praise components identified by GPT models, we introduced a Modified Intersection over Union (M-IoU) score. Our findings demonstrate that: (1) the M-IoU score effectively correlates with human judgment in evaluating sequence quality; (2) using two-shot prompting on GPT-3.5 resulted in decent performance in recognizing effort-based (M-IoU of 0.46) and outcome-based praise (M-IoU of 0.68); and (3) our optimally fine-tuned GPT-3.5 model achieved M-IoU scores of 0.64 for effort-based praise and 0.84 for outcome-based praise, aligning with the satisfaction levels evaluated by human coders. Our results show promise for using GPT models to provide feedback that focuses on specific elements in their open-ended responses that are desirable or could use improvement.

cs.CL↗

arXiv2024

Buck You: Designing Easy-to-Onboard Blockchain Applications with Zero-Knowledge Login and Sponsored Transactions on Sui

In this paper, we developed a blockchain application to demonstrate the functionality of Sui's recent innovations: Zero Knowledge Login and Sponsored Transactions. Zero Knowledge Login allows users to create and access their blockchain wallets just with their OAuth accounts (e.g., Google, Facebook, Twitch), while Sponsored Transactions eliminate the need for users to prepare transaction fees, as they can delegate fees to sponsors' accounts. Additionally, thanks to Sui's Storage Rebate feature, sponsors in Sponsored Transactions can profit from the sponsorship, achieving a win-win and sustainable service model. Zero Knowledge Login and Sponsored Transactions are pivotal in overcoming key challenges novice blockchain users face, particularly in managing private keys and depositing initial transaction fees. By addressing these challenges in the user experience of blockchain, Sui makes the blockchain more accessible and engaging for novice users and paves the way for the broader adoption of blockchain applications in everyday life.

cs.CR↗

arXiv2024

Improving Assessment of Tutoring Practices using Retrieval-Augmented Generation

One-on-one tutoring is an effective instructional method for enhancing learning, yet its efficacy hinges on tutor competencies. Novice math tutors often prioritize content-specific guidance, neglecting aspects such as social-emotional learning. Social-emotional learning promotes equity and inclusion and nurturing relationships with students, which is crucial for holistic student development. Assessing the competencies of tutors accurately and efficiently can drive the development of tailored tutor training programs. However, evaluating novice tutor ability during real-time tutoring remains challenging as it typically requires experts-in-the-loop. To address this challenge, this preliminary study aims to harness Generative Pre-trained Transformers (GPT), such as GPT-3.5 and GPT-4 models, to automatically assess tutors' ability of using social-emotional tutoring strategies. Moreover, this study also reports on the financial dimensions and considerations of employing these models in real-time and at scale for automated assessment. The current study examined four prompting strategies: two basic Zero-shot prompt strategies, Tree of Thought prompt, and Retrieval-Augmented Generator (RAG) based prompt. The results indicate that the RAG prompt demonstrated more accurate performance (assessed by the level of hallucination and correctness in the generated assessment texts) and lower financial costs than the other strategies evaluated. These findings inform the development of personalized tutor training interventions to enhance the the educational effectiveness of tutored learning.

cs.CY↗

arXiv2023

Building Random, Fair, and Verifiable Games on Blockchain. Raffle smart contract designs on Sui Network

Randomness plays a pivotal role in modern online gaming, but disputes have arisen over the accuracy of stated winning chances, resulting in legal issues and financial setbacks for gaming companies. Fortunately, blockchain-based games offer a solution to the transparency and fairness issue regarding randomness. Furthermore, emerging blockchain technology like Sui Network enhances the efficiency of smart contracts by eliminating traditional web3 barriers, such as inefficiencies and expensive transaction fees. This unlocks the potential for extensive decentralized gaming applications. This paper aims to provide insights into designing a fair, verifiable, and efficient smart contract game on blockchain by the example of building raffles on the Sui Network. We explore efficient methods for implementing randomness on smart contracts, including DRAND committee-based decentralized random beacons and single private-key-based verifiable random functions (VRF). Then, progress from basic to comprehensive smart contract design. We addressed limitations in developing blockchain games in general, such as data input and storage space constraints. We propose corresponding solutions, encompassing the utilization of Object Tables, Delegate Object Creation, and Zero-Knowledge Proofs (ZKP) to optimize storage and input efficiency. After testing our designs, we found that the transaction fees for DRAND beacons and private-key-based VRFs are similar. Moreover, Object Tables incur higher overall transaction fees, while the ZKP setup fee is cheap but becomes very expensive during the verification process. Moreover, we identified suitable designs for different application scenarios by comparing the pros and cons of different smart contract implementations. Our findings provide valuable guidance for future researchers and developers in building random, fair, and verifiable games with smart contracts.

cs.CR↗

arXiv2023

GPTutor: an open-source AI pair programming tool alternative to Copilot

This paper presents the latest progress of GPTutor: a ChatGPT-powered programming tool extension in Visual Studio Code. The emergence of Large Language Models (LLMs) has improved software development efficiency, but their performance can be hindered by training data limitations and prompt design issues. Existing LLM development tools often operate as black boxes, with users unable to view the prompts used and unable to improve performance by correcting prompts when errors occur. To address the aforementioned issues, GPTutor was introduced as an open-source AI pair programming tool, offering an alternative to Copilot. GPTutor empowers users to customize prompts for various programming languages and scenarios, with support for 120+ human languages and 50+ programming languages. Users can fine-tune prompts to correct the errors from LLM for precision and efficient code generation. At the end of the paper, we underscore GPTutor's potential through examples, including demonstrating its proficiency in interpreting and generating Sui-Move, a newly introduced smart contract language, using prompt engineering.

cs.HC↗

arXiv2023

GPTutor: a ChatGPT-powered programming tool for code explanation

Learning new programming skills requires tailored guidance. With the emergence of advanced Natural Language Generation models like the ChatGPT API, there is now a possibility of creating a convenient and personalized tutoring system with AI for computer science education. This paper presents GPTutor, a ChatGPT-powered programming tool, which is a Visual Studio Code extension using the ChatGPT API to provide programming code explanations. By integrating Visual Studio Code API, GPTutor can comprehensively analyze the provided code by referencing the relevant source codes. As a result, GPTutor can use designed prompts to explain the selected code with a pop-up message. GPTutor is now published at the Visual Studio Code Extension Marketplace, and its source code is openly accessible on GitHub. Preliminary evaluation indicates that GPTutor delivers the most concise and accurate explanations compared to vanilla ChatGPT and GitHub Copilot. Moreover, the feedback from students and teachers indicated that GPTutor is user-friendly and can explain given codes satisfactorily. Finally, we discuss possible future research directions for GPTutor. This includes enhancing its performance and personalization via further prompt programming, as well as evaluating the effectiveness of GPTutor with real users.

cs.HC↗

arXiv2023

Which Factors Predict the Chat Experience of a Natural Language Generation Dialogue Service?

In this paper, we proposed a conceptual model to predict the chat experience in a natural language generation dialog system. We evaluated the model with 120 participants with Partial Least Squares Structural Equation Modeling (PLS-SEM) and obtained an R-square (R2) with 0.541. The model considers various factors, including the prompts used for generation; coherence, sentiment, and similarity in the conversation; and users' perceived dialog agents' favorability. We then further explore the effectiveness of the subset of our proposed model. The results showed that users' favorability and coherence, sentiment, and similarity in the dialogue are positive predictors of users' chat experience. Moreover, we found users may prefer dialog agents with characteristics of Extroversion, Openness, Conscientiousness, Agreeableness, and Non-Neuroticism. Through our research, an adaptive dialog system might use collected data to infer factors in our model, predict the chat experience for users through these factors, and optimize it by adjusting prompts.

cs.CL↗

arXiv2023

Conversion of Legal Agreements into Smart Legal Contracts using NLP

A Smart Legal Contract (SLC) is a specialized digital agreement comprising natural language and computable components. The Accord Project provides an open-source SLC framework containing three main modules: Cicero, Concerto, and Ergo. Currently, we need lawyers, programmers, and clients to work together with great effort to create a usable SLC using the Accord Project. This paper proposes a pipeline to automate the SLC creation process with several Natural Language Processing (NLP) models to convert law contracts to the Accord Project's Concerto model. After evaluating the proposed pipeline, we discovered that our NER pipeline accurately detects CiceroMark from Accord Project template text with an accuracy of 0.8. Additionally, our Question Answering method can extract one-third of the Concerto variables from the template text. We also delve into some limitations and possible future research for the proposed pipeline. Finally, we describe a web interface enabling users to build SLCs. This interface leverages the proposed pipeline to convert text documents to Smart Legal Contracts by using NLP models.

cs.CY↗

arXiv2023

Generate labeled training data using Prompt Programming and GPT-3. An example of Big Five Personality Classification

We generated 25000 conversations labeled with Big Five Personality traits using prompt programming at GPT-3. Then we train Big Five classification models with these data and evaluate them with 2500 data from generated dialogues and real conversational datasets labeled in Big Five by human annotators. The results indicated that this approach is promising for creating effective training data. We then compare the performance by different training approaches and models. Our results suggest that using Adapter-Transformers and transfer learning from pre-trained RoBERTa sentiment analysis model will perform best with the generated data. Our best model obtained an accuracy of 0.71 in generated data and 0.65 in real datasets. Finally, we discuss this approach's potential limitations and confidence metric.

cs.HC↗

arXiv2022

The Effect of Multiple Replies for Natural Language Generation Chatbots

In this research, by responding to users' utterances with multiple replies to create a group chat atmosphere, we alleviate the problem that Natural Language Generation chatbots might reply with inappropriate content, thus causing a bad user experience. Because according to our findings, users tend to pay attention to appropriate replies and ignore inappropriate replies. We conducted a 2 (single reply vs. five replies) x 2 (anonymous avatar vs. anime avatar) repeated measures experiment to compare the chatting experience in different conditions. The result shows that users will have a better chatting experience when receiving multiple replies at once from the NLG model compared to the single reply. Furthermore, according to the effect size of our result, to improve the chatting experience for NLG chatbots which is single reply and anonymous avatar, providing five replies will have more benefits than setting an anime avatar.

cs.HC↗

arXiv2022

Focus Plus: Detect Learner's Distraction by Web Camera in Distance Teaching

Distance teaching has become popular these years because of the COVID-19 epidemic. However, both students and teachers face several challenges in distance teaching, like being easy to distract. We proposed Focus+, a system designed to detect learners' status with the latest AI technology from their web camera to solve such challenges. By doing so, teachers can know students' status, and students can regulate their learning experience. In this research, we will discuss the expected model's design for training and evaluating the AI detection model of Focus+.

cs.HC↗

arXiv2021

Ergo -- a programming language for Smart Legal Contracts

We present a smart legal contract platform to support a wide range of smart legal contract use cases. We see this as a step towards improving existing approaches to representing the complexity of legal agreements and executing aspects of these agreements.

cs.CY↗