arXiv subjects

Aravind Sesagiri Raamkumar

Publications and source records attributed to Aravind Sesagiri Raamkumar.

8 recordsLinked to original sources

arXiv2024

Towards a Multidimensional Evaluation Framework for Empathetic Conversational Systems

Empathetic Conversational Systems (ECS) are built to respond empathetically to the user's emotions and sentiments, regardless of the application domain. Current ECS studies evaluation approaches are restricted to offline evaluation experiments primarily for gold standard comparison & benchmarking, and user evaluation studies for collecting human ratings on specific constructs. These methods are inadequate in measuring the actual quality of empathy in conversations. In this paper, we propose a multidimensional empathy evaluation framework with three new methods for measuring empathy at (i) structural level using three empathy-related dimensions, (ii) behavioral level using empathy behavioral types, and (iii) overall level using an empathy lexicon, thereby fortifying the evaluation process. Experiments were conducted with the state-of-the-art ECS models and large language models (LLMs) to show the framework's usefulness.

cs.CL↗

arXiv2023

Harnessing Large Language Models' Empathetic Response Generation Capabilities for Online Mental Health Counselling Support

Large Language Models (LLMs) have demonstrated remarkable performance across various information-seeking and reasoning tasks. These computational systems drive state-of-the-art dialogue systems, such as ChatGPT and Bard. They also carry substantial promise in meeting the growing demands of mental health care, albeit relatively unexplored. As such, this study sought to examine LLMs' capability to generate empathetic responses in conversations that emulate those in a mental health counselling setting. We selected five LLMs: version 3.5 and version 4 of the Generative Pre-training (GPT), Vicuna FastChat-T5, Pathways Language Model (PaLM) version 2, and Falcon-7B-Instruct. Based on a simple instructional prompt, these models responded to utterances derived from the EmpatheticDialogues (ED) dataset. Using three empathy-related metrics, we compared their responses to those from traditional response generation dialogue systems, which were fine-tuned on the ED dataset, along with human-generated responses. Notably, we discovered that responses from the LLMs were remarkably more empathetic in most scenarios. We position our findings in light of catapulting advancements in creating empathetic conversational systems.

cs.CL↗

arXiv2022

Empathetic Conversational Systems: A Review of Current Advances, Gaps, and Opportunities

Empathy is a vital factor that contributes to mutual understanding, and joint problem-solving. In recent years, a growing number of studies have recognized the benefits of empathy and started to incorporate empathy in conversational systems. We refer to this topic as empathetic conversational systems. To identify the critical gaps and future opportunities in this topic, this paper examines this rapidly growing field using five review dimensions: (i) conceptual empathy models and frameworks, (ii) adopted empathy-related concepts, (iii) datasets and algorithmic techniques developed, (iv) evaluation strategies, and (v) state-of-the-art approaches. The findings show that most studies have centered on the use of the EMPATHETICDIALOGUES dataset, and the text-based modality dominates research in this field. Studies mainly focused on extracting features from the messages of the users and the conversational systems, with minimal emphasis on user modeling and profiling. Notably, studies that have incorporated emotion causes, external knowledge, and affect matching in the response generation models, have obtained significantly better results. For implementation in diverse real-world settings, we recommend that future studies should address key gaps in areas of detecting and authenticating emotions at the entity level, handling multimodal inputs, displaying more nuanced empathetic behaviors, and encompassing additional dialogue system features.

cs.CL↗

arXiv2019

Understanding the Twitter Usage of Science Citation Index (SCI) Journals

This paper investigates the Twitter interaction patterns of journals from the Science Citation Index (SCI) of Master Journal List (MJL). A total of 953,253 tweets extracted from 857 journal accounts, were analyzed in this study. Findings indicate that SCI journals interacted more with each other but much less with journals from other citation indices. The network structure of the communication graph resembled a tight crowd network, with Nature journals playing a major part. Information sources such as news portals and scientific organizations were mentioned more in tweets, than academic journal Twitter accounts. Journals with high journal impact factors (JIFs) were found to be prominent hubs in the communication graph. Differences were found between the Twitter usage of SCI journals with Humanities and Social Sciences (HSS) journals.

cs.DL↗

arXiv2018

Understanding the Twitter Usage of Humanities and Social Sciences Academic Journals

Scholarly communication has the scope to transcend the limitations of the physical world through social media extended coverage and shortened information paths. Accordingly, publishers have created profiles for their journals in Twitter to promote their publications and to initiate discussions with public. This paper investigates the Twitter presence of humanities and social sciences (HSS) journal titles obtained from mainstream citation indices, by analysing the interaction and communication patterns. This study utilizes webometric data collection, descriptive analysis, and social network analysis. Findings indicate that the presence of HSS journals in Twitter across disciplines is not yet substantial. Sharing of general websites appears to be the key activity performed by HSS journals in Twitter. Among them, web content from news portals and magazines are highly disseminated. Sharing of research articles and retweeting was not majorly observed. Inter-journal communication is apparent within the same citation index, but it is very minimal with journals from the other index. However, there seems to be an effort to broaden communication beyond the research community, reaching out to connect with the public.

cs.SI↗

arXiv2016

A Framework for Scientific Paper Retrieval and Recommender Systems

Information retrieval (IR) and recommender systems (RS) have been employed for addressing search tasks executed during literature review and the overall scholarly communication lifecycle. Majority of the studies have concentrated on algorithm design for improving the accuracy and usefulness of these systems. Contextual elements related to the scholarly tasks have been largely ignored. In this paper, we propose a framework called the Scientific Paper Recommender and Retrieval Framework (SPRRF) that combines aspects of user role modeling and user-interface features with IR/RS components. The framework is based on eight emergent themes identified from participants feedback in a user evaluation study conducted with a prototype assistive system. 119 researchers participated in the study for evaluating the prototype system that provides recommendations for two literature review and one manuscript writing tasks. This holistic framework is meant to guide future studies in this area.

cs.IR↗

arXiv2016

Whats in a Country Name - Twitter Hashtag Analysis of #singapore

Twitter as a micro-blogging platform rose to instant fame mainly due to its minimalist features that allow seamless communication between users. As the conversations grew thick and faster, a placeholder feature called as Hashtags became important as it captured the themes behind the tweets. Prior studies have investigated the conversation dynamics, inter-play with other media platforms and communication patterns between users for specific event-based hashtags such as the #Occupy movement. Commonplace hashtags which are used on a daily basis have been largely ignored due to their seemingly innocuous presence in tweets and also due to the lack of connection with real-world events. However, it can be postulated that utility of these hashtags is the main reason behind their continued usage. This study is aimed at understanding the rationale behind the usage of a particular type of commonplace hashtags - location hashtags such as country and city name hashtags. Tweets with the hashtag #singapore were extracted for one week duration. Manual and automatic tweet classification was performed along with social network analysis, to identify the underlying themes. Seven themes were identified. Findings indicate that the hashtag is prominent in tweets about local events, local news, users current location and landmark related information sharing. Users who share content from social media sites such as Instagram make use of the hashtag in a more prominent way when compared to users who post textual content. News agencies, commercial bodies and celebrities make use of the hashtag more than common individuals. Overall, the results show the non-conversational nature of the hashtag. The findings are to be validated with other country names and cross-validated with hashtag data from other social media platforms.

cs.SI↗

arXiv2015

Designing a Linked Data Migrational Framework for Singapore Government Datasets

The subject area of this report is Linked Data and its application to the Government domain. Linked Data is an alternative method of data representation that aims to interlink data from varied sources through relationships. Governments around the world have started publishing their data in this format to assist citizens in making better use of public services. This report provides an eight step migrational framework for converting Singapore Government data from legacy systems to Linked Data format. The framework formulation is based on a study of the Singapore data ecosystem with help from Infocomm Development Authority (iDA) of Singapore. Each step in the migrational framework has been constructed with objectives, recommendations, best practices and issues with entry and exit points. This work builds on the existing Linked Data literature, implementations in other countries and cookbooks provided by Linked Data researchers. iDA can use this report to gain an understanding of the effort and work involved in the implementation of Linked Data system on top of their legacy systems. The framework can be evaluated by building a Proof of Concept (POC) application.

cs.CY↗