arXiv subjects

Bastin Tony Roy Savarimuthu

Publications and source records attributed to Bastin Tony Roy Savarimuthu.

At least 19 recordsLinked to original sources

arXiv2026

Taming the Online Software Engineering Tongue: Detecting and Reducing Offensive Language in Software Communities

Online Software Engineering (SE) communities such as Stack Overflow have become unwelcoming and hostile spaces, as evidenced by the presence of rude and offensive comments that are detrimental to the emotional wellbeing of participants. The main objective of this work is to investigate the prevalence and nature of offensive language in online SE communities, and develop approaches to classify them with the ultimate goal of reducing offensive language by proposing an Offensive Language Reduction System (OLRSSE) that has the ability to detect, classify, and explain to the poster why a comment is offensive and also propose non-offensive alternative suggestions. Using data from 130,000 user comments on discussions held in four prominent SE platforms: Stack Overflow, GitHub, Gitter and Slack, we developed six sets of twelve (72 in total) different machine learning models (both traditional and deep learning). The F1-Score of the best performing models in the system ranges from 0.56 to 0.94, with an average F1-Score of 0.85, predominantly using BERT as the primary model for various classification tasks. Also, two language-based models were developed for paraphrasing offensive to non-offensive comments. The overall utility of the OLRS-SE was evaluated using user studies with 30 participants, which showed that the developed system is easy to use and also effective in reducing offensive language. Investigating the capabilities of different large language models for offensive language reduction is a promising avenue for future research.

cs.SE↗

arXiv2026

A Pilot Study on Detecting Software Design Patterns with Large Language Models: An Empirical Evaluation

Design patterns provide reusable solutions to recurring software design problems. Automatically detecting these patterns in source code can help bootstrap new developers' understanding of unfamiliar software system architectures, and can help experienced developers to quickly identify and rectify potential quality issues. While many prior research works have explored graph based and machine-learning based detection techniques, this work evaluates the design pattern recognition capabilities of four Large Language Models and two ensemble approaches consisting three out of the four models. We also compare their performance when prompted with a) Source code, b) PlantUML representation of source code, and c) Text-based descriptions of the source code. We investigate the detection of five design patterns: singleton, adapter, bridge, composite and decorator. Our preliminary results indicate that LLMs show promise for automatically detecting design patterns, with NextCoder and Gemma 3 demonstrating comparatively higher accuracy than other models evaluated, and the ensemble approaches enhancing the overall efficiency of design pattern detection. We identify several directions for future work.

cs.SE↗

arXiv2026

Evaluating LLM Alignment With Human Trust Models

Trust plays a pivotal role in enabling effective cooperation, reducing uncertainty, and guiding decision-making in both human interactions and multi-agent systems. Although it is significant, there is limited understanding of how large language models (LLMs) internally conceptualize and reason about trust. This work presents a white-box analysis of trust representation in EleutherAI/gpt-j-6B, using contrastive prompting to generate embedding vectors within the activation space of the LLM for diadic trust and related interpersonal relationship attributes. We first identified trust-related concepts from five established human trust models. We then determined a threshold for significant conceptual alignment by computing pairwise cosine similarities across 60 general emotional concepts. Then we measured the cosine similarities between the LLM's internal representation of trust and the derived trust-related concepts. Our results show that the internal trust representation of EleutherAI/gpt-j-6B aligns most closely with the Castelfranchi socio-cognitive model, followed by the Marsh Model. These findings indicate that LLMs encode socio-cognitive constructs in their activation space in ways that support meaningful comparative analyses, inform theories of social cognition, and support the design of human-AI collaborative systems.

cs.MA↗

arXiv2026

Social Norm Reasoning in Multimodal Language Models: An Evaluation

In Multi-Agent Systems (MAS), agents are designed with social capabilities, allowing them to understand and reason about social concepts such as norms when interacting with others (e.g., inter-robot interactions). In Normative MAS (NorMAS), researchers study how norms develop, and how violations are detected and sanctioned. However, existing research in NorMAS use symbolic approaches (e.g., formal logic) for norm representation and reasoning whose application is limited to simplified environments. In contrast, Multimodal Large Language Models (MLLMs) present promising possibilities to develop software used by robots to identify and reason about norms in a wide variety of complex social situations embodied in text and images. However, prior work on norm reasoning have been limited to text-based scenarios. This paper investigates the norm reasoning competence of five MLLMs by evaluating their ability to answer norm-related questions based on thirty text-based and thirty image-based stories, and comparing their responses against humans. Our results show that MLLMs demonstrate superior performance in norm reasoning in text than in images. GPT-4o performs the best in both modalities offering the most promise for integration with MAS, followed by the free model Qwen-2.5VL. Additionally, all models find reasoning about complex norms challenging.

cs.MA↗

arXiv2025

Evolution of Cooperation in LLM-Agent Societies: A Preliminary Study Using Different Punishment Strategies

The evolution of cooperation has been extensively studied using abstract mathematical models and simulations. Recent advances in Large Language Models (LLMs) and the rise of LLM agents have demonstrated their ability to perform social reasoning, thus providing an opportunity to test the emergence of norms in more realistic agent-based simulations with human-like reasoning using natural language. In this research, we investigate whether the cooperation dynamics presented in Boyd and Richerson's model persist in a more realistic simulation of the Diner's Dilemma using LLM agents compared to the abstract mathematical nature in the work of Boyd and Richerson. Our findings indicate that agents follow the strategies defined in the Boyd and Richerson model, and explicit punishment mechanisms drive norm emergence, reinforcing cooperative behaviour even when the agent strategy configuration varies. Our results suggest that LLM-based Multi-Agent System simulations, in fact, can replicate the evolution of cooperation predicted by the traditional mathematical models. Moreover, our simulations extend beyond the mathematical models by integrating natural language-driven reasoning and a pairwise imitation method for strategy adoption, making them a more realistic testbed for cooperative behaviour in MASs.

cs.MA↗

arXiv2025

Can LLMs Reason About Trust?: A Pilot Study

In human society, trust is an essential component of social attitude that helps build and maintain long-term, healthy relationships which creates a strong foundation for cooperation, enabling individuals to work together effectively and achieve shared goals. As many human interactions occur through electronic means such as using mobile apps, the potential arises for AI systems to assist users in understanding the social state of their relationships. In this paper we investigate the ability of Large Language Models (LLMs) to reason about trust between two individuals in an environment which requires fostering trust relationships. We also assess whether LLMs are capable of inducing trust by role-playing one party in a trust based interaction and planning actions which can instil trust.

cs.HC↗

arXiv2024

Norm Violation Detection in Multi-Agent Systems using Large Language Models: A Pilot Study

Norms are an important component of the social fabric of society by prescribing expected behaviour. In Multi-Agent Systems (MAS), agents interacting within a society are equipped to possess social capabilities such as reasoning about norms and trust. Norms have long been of interest within the Normative Multi-Agent Systems community with researchers studying topics such as norm emergence, norm violation detection and sanctioning. However, these studies have some limitations: they are often limited to simple domains, norms have been represented using a variety of representations with no standard approach emerging, and the symbolic reasoning mechanisms generally used may suffer from a lack of extensibility and robustness. In contrast, Large Language Models (LLMs) offer opportunities to discover and reason about norms across a large range of social situations. This paper evaluates the capability of LLMs to detecting norm violations. Based on simulated data from 80 stories in a household context, with varying complexities, we investigated whether 10 norms are violated. For our evaluations we first obtained the ground truth from three human evaluators for each story. Then, the majority result was compared against the results from three well-known LLM models (Llama 2 7B, Mixtral 7B and ChatGPT-4). Our results show the promise of ChatGPT-4 for detecting norm violations, with Mixtral some distance behind. Also, we identify areas where these models perform poorly and discuss implications for future work.

cs.MA↗

arXiv2024

Harnessing the power of LLMs for normative reasoning in MASs

Software agents, both human and computational, do not exist in isolation and often need to collaborate or coordinate with others to achieve their goals. In human society, social mechanisms such as norms ensure efficient functioning, and these techniques have been adopted by researchers in multi-agent systems (MAS) to create socially aware agents. However, traditional techniques have limitations, such as operating in limited environments often using brittle symbolic reasoning. The advent of Large Language Models (LLMs) offers a promising solution, providing a rich and expressive vocabulary for norms and enabling norm-capable agents that can perform a range of tasks such as norm discovery, normative reasoning and decision-making. This paper examines the potential of LLM-based agents to acquire normative capabilities, drawing on recent Natural Language Processing (NLP) and LLM research. We present our vision for creating normative LLM agents. In particular, we discuss how the recently proposed "LLM agent" approaches can be extended to implement such normative LLM agents. We also highlight challenges in this emerging field. This paper thus aims to foster collaboration between MAS, NLP and LLM researchers in order to advance the field of normative agents.

cs.AI↗

arXiv2023

Barriers for Social Inclusion in Online Software Engineering Communities -- A Study of Offensive Language Use in Gitter Projects

Social inclusion is a fundamental feature of thriving societies. This paper first investigates barriers for social inclusion in online Software Engineering (SE) communities, by identifying a set of 11 attributes and organising them as a taxonomy. Second, by applying the taxonomy and analysing language used in the comments posted by members in 189 Gitter projects (with > 3 million comments), it presents the evidence for the social exclusion problem. It employs a keyword-based search approach for this purpose. Third, it presents a framework for improving social inclusion in SE communities.

cs.SE↗

arXiv2022

Investigating Expectation Violations in Mobile Apps

Information technology and software services are pervasive, occupying the centre of most aspects of contemporary societies. This has given rise to commonly expected norms and expectations around how such systems should work, appropriate penalties for violating these expectations, and more importantly, indicators of how to reduce the consequences of violations and sanctions. Evidence for expectation violations and ensuing sanctions exists in a range of portals used by individuals and groups to start new friendships, explore new ideas, and provide feedback for products and services. Therein lies insights that could lead to functional socio-technical systems, and general awareness and anticipations of human actions (and interactions) when using information technology and software services. However, limited previous work has examined such artifacts to provide these understandings. To contribute to such understandings and theoretical advancement we study expectation violations in mobile apps, considered among the most engaging socio-technical systems. We used content analysis and expectancy violation theory (EVT) and expectation confirmation theory (ECT) to explore the evidence and nature of sanctions in app reviews for a specific domain of apps. Our outcomes show that users respond to expectation violation with sanctions when their app does not work as anticipated, developers seem to target specific market niches when providing services in an app domain, and users within an app domain respond with similar sanctions. We contribute to the advancement of expectation violation theories, and we provide practical insights for the mobile app community.

cs.SE↗

arXiv2021

Towards offensive language detection and reduction in four Software Engineering communities

Software Engineering (SE) communities such as Stack Overflow have become unwelcoming, particularly through members' use of offensive language. Research has shown that offensive language drives users away from active engagement within these platforms. This work aims to explore this issue more broadly by investigating the nature of offensive language in comments posted by users in four prominent SE platforms - GitHub, Gitter, Slack and Stack Overflow (SO). It proposes an approach to detect and classify offensive language in SE communities by adopting natural language processing and deep learning techniques. Further, a Conflict Reduction System (CRS), which identifies offence and then suggests what changes could be made to minimize offence has been proposed. Beyond showing the prevalence of offensive language in over 1 million comments from four different communities which ranges from 0.07% to 0.43%, our results show promise in successful detection and classification of such language. The CRS system has the potential to drastically reduce manual moderation efforts to detect and reduce offence in SE communities.

cs.SE↗

arXiv2021

Influence of Roles in Decision-Making during OSS Development -- A Study of Python

Governance has been highlighted as a key factor in the success of an Open Source Software (OSS) project. It is generally seen that in a mixed meritocracy and autocracy governance model, the decision-making (DM) responsibility regarding what features are included in the OSS is shared among members from select roles; prominently the project leader. However, less examination has been made whether members from these roles are also prominent in DM discussions and how decisions are made, to show they play an integral role in the success of the project. We believe that to establish their influence, it is necessary to examine not only discussions of proposals in which the project leader makes the decisions, but also those where others make the decisions. Therefore, in this study, we examine the prominence of members performing different roles in: (i) making decisions, (ii) performing certain social roles in DM discussions (e.g., discussion starters), (iii) contributing to the OSS development social network through DM discussions, and (iv) how decisions are made under both scenarios. We examine these aspects in the evolution of the well-known Python project. We carried out a data-driven longitudinal study of their email communication spanning 20 years, comprising about 1.5 million emails. These emails contain decisions for 466 Python Enhancement Proposals (PEPs) that document the language's evolution. Our findings make the influence of different roles transparent to future (new) members, other stakeholders, and more broadly, to the OSS research community.

cs.SE↗

arXiv2021

Augmenting Text Mining Approaches with Social Network Analysis to Understand the Complex Relationships among Users' Requests: a Case Study of the Android Operating System

Text mining approaches are being used increasingly for business analytics. In particular, such approaches are now central to understanding users' feedback regarding systems delivered via online application distribution platforms such as Google Play. In such settings, large volumes of reviews of potentially numerous apps and systems means that it is infeasible to use manual mechanisms to extract insights and knowledge that could inform product improvement. In this context of identifying software system improvement options, text mining techniques are used to reveal the features that are mentioned most often as being in need of correction (e.g., GPS), and topics that are associated with features perceived as being defective (e.g., inaccuracy of GPS). Other approaches may supplement such techniques to provide further insights for online communities and solution providers. In this work we augment text mining approaches with social network analysis to demonstrate the utility of using multiple techniques. Our outcomes suggest that text mining approaches may indeed be supplemented with other methods to deliver a broader range of insights.

cs.SE↗

arXiv2021

They'll Know It When They See It: Analyzing Post-Release Feedback from the Android Community

It is known that user involvement and user-centered design enhance system acceptance, particularly when end-users' views are considered early in the process. However, the increasingly common method of system deployment, through frequent releases via an online application distribution platform, relies more on post-release feedback from a virtual community. Such feedback may be received from large and diverse communities of users, posing challenges to developers in terms of extracting and identifying the most pressing requests to address. In seeking to tackle these challenges we have used natural language processing techniques to study enhancement requests logged by the Android community. We observe that features associated with a specific subset of topics were most frequently requested for improvement, and that end-users expressed particular discontent with the Jellybean release. End-users also tended to request improvements to specific issues together, potentially posing a prioritization challenge to Google.

cs.SE↗

arXiv2021

Extracting Rationale for Open Source Software Development Decisions -- A Study of Python Email Archives

A sound Decision-Making (DM) process is key to the successful governance of software projects. In many Open Source Software Development (OSSD) communities, DM processes lie buried amongst vast amounts of publicly available data. Hidden within this data lie the rationale for decisions that led to the evolution and maintenance of software products. While there have been some efforts to extract DM processes from publicly available data, the rationale behind how the decisions are made have seldom been explored. Extracting the rationale for these decisions can facilitate transparency (by making them known), and also promote accountability on the part of decision-makers. This work bridges this gap by means of a large-scale study that unearths the rationale behind decisions from Python development email archives comprising about 1.5 million emails. This paper makes two main contributions. First, it makes a knowledge contribution by unearthing and presenting the rationale behind decisions made. Second, it makes a methodological contribution by presenting a heuristics-based rationale extraction system called Rationale Miner that employs multiple heuristics, and follows a data-driven, bottom-up approach to infer the rationale behind specific decisions (e.g., whether a new module is implemented based on core developer consensus or benevolent dictator's pronouncement). Our approach can be applied to extract rationale in other OSSD communities that have similar governance structures.

cs.SE↗

arXiv2020

Exploring Research Interest in Stack Overflow -- A Systematic Mapping Study and Quality Evaluation

Platforms such as Stack Overflow are available for software practitioners to solicit solutions to their challenges and knowledge needs. The practices therein have in recent times however triggered quality related concerns. This is a noteworthy issue when considering that the Stack Overflow platform is used by numerous software developers. Academic research tends to provide validation for the practices and processes employed by Stack Overflow and other such forums. However, previous work did not review the scale of scientific attention that is given to this cause. Continuing from our preliminary work, we conducted a Systematic Mapping study involving 265 papers from six relevant databases to address this gap. In this work, we explored the level of academic interest Stack Overflow has generated, the publication venues that are targeted, the topics that are studied, approaches used, types of contributions and the quality of the publications that are written about Stack Overflow. Outcomes show that Stack Overflow has attracted increasing research interest over the years, with topics relating to both community dynamics and human factors, and technical issues. In addition, research studies have been largely evaluative or proposed solutions; however, the latter approach tends to lack validation. The contributions of these studies are often techniques or answers to a specific problem. Evaluating the quality of all studies that were dedicated to software programming (58 papers), our outcomes show that on average only 58% of the developed quality criteria were met. Notwithstanding that research is continually aiming to understand Stack Overflow and other similar communities, further investigations are required to validate such studies and the solutions they propose.

cs.SE↗

arXiv2020

Impact of different belief facets on agents' decision -- a refined cognitive architecture to model the interaction between organisations' institutional characteristics and agents' behaviour

This paper presents a conceptual refinement of agent cognitive architecture inspired from the beliefs-desires-intentions (BDI) and the theory of planned behaviour (TPB) models, with an emphasis on different belief facets. This enables us to investigate the impact of personality and the way that an agent weights its internal beliefs and social sanctions on an agent's actions. The study also uses the concept of cognitive dissonance associated with the fairness of institutions to investigate the agents' behaviour. To showcase our model, we simulate two historical long-distance trading societies, namely Armenian merchants of New-Julfa and the English East India Company. The results demonstrate the importance of internal beliefs of agents as a pivotal aspect for following institutional rules.

cs.AI↗

arXiv2020

Impact of meta-roles on the evolution of organisational institutions

This paper investigates the impact of changes in agents' beliefs coupled with dynamics in agents' meta-roles on the evolution of institutions. The study embeds agents' meta-roles in the BDI architecture. In this context, the study scrutinises the impact of cognitive dissonance in agents due to unfairness of institutions. To showcase our model, two historical long-distance trading societies, namely Armenian merchants of New-Julfa and the English East India Company are simulated. Results show how change in roles of agents coupled with specific institutional characteristics leads to changes of the rules in the system.

cs.AI↗