arXiv subjects

Mojtaba Shahin

Publications and source records attributed to Mojtaba Shahin.

At least 37 records · Page 2Linked to original sources

arXiv2025

ArchISMiner: A Framework for Automatic Mining of Architectural Issue-Solution Pairs from Online Developer Communities

Stack Overflow (SO), a leading online community forum, is a rich source of software development knowledge. However, locating architectural knowledge, such as architectural solutions remains challenging due to the overwhelming volume of unstructured content and fragmented discussions. Developers must manually sift through posts to find relevant architectural insights, which is time-consuming and error-prone. This study introduces ArchISMiner, a framework for mining architectural knowledge from SO. The framework comprises two complementary components: ArchPI and ArchISPE. ArchPI trains and evaluates multiple models, including conventional ML/DL models, Pre-trained Language Models (PLMs), and Large Language Models (LLMs), and selects the best-performing model to automatically identify Architecture-Related Posts (ARPs) among programming-related discussions. ArchISPE employs an indirect supervised approach that leverages diverse features, including BERT embeddings and local TextCNN features, to extract architectural issue-solution pairs. Our evaluation shows that the best model in ArchPI achieves an F1-score of 0.960 in ARP detection, and ArchISPE outperforms baselines in both SE and NLP fields, achieving F1-scores of 0.883 for architectural issues and 0.894 for solutions. A user study further validated the quality (e.g., relevance and usefulness) of the identified ARPs and the extracted issue-solution pairs. Moreover, we applied ArchISMiner to three additional forums, releasing a dataset of over 18K architectural issue-solution pairs. Overall, ArchISMiner can help architects and developers identify ARPs and extract succinct, relevant, and useful architectural knowledge from developer communities more accurately and efficiently. The replication package of this study has been provided at https://github.com/JeanMusenga/ArchISPE

cs.SE↗

arXiv2025

How Do Users Revise Architectural Related Questions on Stack Overflow: An Empirical Study

Technical Questions and Answers (Q&A) sites, such as Stack Overflow (SO), accumulate a significant variety of information related to software development in posts from users. To ensure the quality of this information, SO encourages its users to review posts through various mechanisms (e.g., question and answer revision processes). Although Architecture Related Posts (ARPs) communicate architectural information that has a system-wide impact on development, little is known about how SO users revise information shared in ARPs. To fill this gap, we conducted an empirical study to understand how users revise Architecture Related Questions (ARQs) on SO. We manually checked 13,205 ARPs and finally identified 4,114 ARQs that contain revision information. Our main findings are that: (1) The revision of ARQs is not prevalent in SO, and an ARQ revision starts soon after this question is posted (i.e., from 1 minute onward). Moreover, the revision of an ARQ occurs before and after this question receives its first answer/architecture solution, with most revisions beginning before the first architecture solution is posted. Both Question Creators (QCs) and non-QCs actively participate in ARQ revisions, with most revisions being made by QCs. (2) A variety of information (14 categories) is missing and further provided in ARQs after being posted, among which design context, component dependency, and architecture concern are dominant information. (3) Clarify the understanding of architecture under design and improve the readability of architecture problem are the two major purposes of the further provided information in ARQs. (4) The further provided information in ARQs has several impacts on the quality of answers/architecture solutions, including making architecture solution useful, making architecture solution informative, making architecture solution relevant, among others.

cs.SE↗

arXiv2025

Toward Inclusive Low-Code Development: Detecting Accessibility Issues in User Reviews

Low-code applications are gaining popularity across various fields, enabling non-developers to participate in the software development process. However, due to the strong reliance on graphical user interfaces, they may unintentionally exclude users with visual impairments, such as color blindness and low vision. This paper investigates the accessibility issues users report when using low-code applications. We construct a comprehensive dataset of low-code application reviews, consisting of accessibility-related reviews and non-accessibility-related reviews. We then design and implement a complex model to identify whether a review contains an accessibility-related issue, combining two state-of-the-art Transformers-based models and a traditional keyword-based system. Our proposed hybrid model achieves an accuracy and F1-score of 78% in detecting accessibility-related issues.

cs.SE↗

arXiv2025

Managing Technical Debt in a Multidisciplinary Data Intensive Software Team: an Observational Case Study

Context: There is an increase in the investment and development of data-intensive (DI) solutions, systems that manage large amounts of data. Without careful management, this growing investment will also grow associated technical debt (TD). Delivery of DI solutions requires a multidisciplinary skill set, but there is limited knowledge about how multidisciplinary teams develop DI systems and manage TD. Objective: This research contributes empirical, practice based insights about multidisciplinary DI team TD management practices. Method: This research was conducted as an exploratory observation case study. We used socio-technical grounded theory (STGT) for data analysis to develop concepts and categories that articulate TD and TDs debt management practices. Results: We identify TD that the DI team deals with, in particular technical data components debt and pipeline debt. We explain how the team manages the TD, assesses TD, what TD treatments they consider and how they implement TD treatments to fit sprint capacity constraints. Conclusion: We align our findings to existing TD and TDM taxonomies, discuss their implications and highlight the need for new implementation patterns and tool support for multidisciplinary DI teams.

cs.SE↗

arXiv2025

How Do OSS Developers Reuse Architectural Solutions from Q&A Sites: An Empirical Study

Developers reuse programming-related knowledge on Q&A sites that functionally matches the programming problems they encounter in their development. Despite extensive research on Q&A sites, being a high-level and important type of development-related knowledge, architectural solutions and their reuse are rarely explored. To fill this gap, we conducted a mixed-methods study that includes a mining study and a survey study. For the mining study, we mined 984 commits and issues from 893 OSS projects on GitHub that explicitly referenced architectural solutions from SO and SWESE. For the survey study, we identified practitioners involved in the reuse of these architectural solutions and surveyed 227 of them to further understand how practitioners reuse architectural solutions from Q&A sites in their OSS development. Our findings: (1) OSS practitioners use architectural solutions from Q&A sites to solve a large variety of architectural problems, wherein Component design issue, Architectural anti-pattern, and Security issue are dominant; (2) Seven categories of architectural solutions from Q&A sites have been reused to solve those problems, among which Architectural refactoring, Use of frameworks, and Architectural tactic are the three most reused architectural solutions; (3) OSS developers often rely on ad hoc ways (e.g., informal, improvised, or unstructured approaches) to incorporate architectural solutions from SO, drawing on personal experience and intuition rather than standardized or systematic practices; (4) Using architectural solutions from SO comes with a variety of challenges, e.g., OSS practitioners complain that they need to spend significant time to adapt such architectural solutions to address design concerns raised in their OSS development, and it is challenging to use architectural solutions that are not tailored to the design context of their OSS projects.

cs.SE↗

arXiv2025

Demystifying Issues, Causes and Solutions in LLM Open-Source Projects

With the advancements of Large Language Models (LLMs), an increasing number of open-source software projects are using LLMs as their core functional component. Although research and practice on LLMs are capturing considerable interest, no dedicated studies explored the challenges faced by practitioners of LLM open-source projects, the causes of these challenges, and potential solutions. To fill this research gap, we conducted an empirical study to understand the issues that practitioners encounter when developing and using LLM open-source software, the possible causes of these issues, and potential solutions. We collected all closed issues from 15 LLM open-source projects and labelled issues that met our requirements. We then randomly selected 994 issues from the labelled issues as the sample for data extraction and analysis to understand the prevalent issues, their underlying causes, and potential solutions. Our study results show that (1) Model Issue is the most common issue faced by practitioners, (2) Model Problem, Configuration and Connection Problem, and Feature and Method Problem are identified as the most frequent causes of the issues, and (3) Optimize Model is the predominant solution to the issues. Based on the study results, we provide implications for practitioners and researchers of LLM open-source projects.

cs.SE↗

arXiv2025

A Systematic Literature Review on Task Recommendation Systems for Crowdsourced Software Engineering

Crowdsourced Software Engineering (CSE) offers outsourcing work to software practitioners by leveraging a global online workforce. However, these software practitioners struggle to identify suitable tasks due to the variety of options available. Hence, there have been a growing number of studies on introducing recommendation systems to recommend CSE tasks to software practitioners. The goal of this study is to analyze the existing CSE task recommendation systems, investigating their extracted data, recommendation methods, key advantages and limitations, recommended task types, the use of human factors in recommendations, popular platforms, and features used to make recommendations. This SLR was conducted according to the Kitchenham and Charters' guidelines. We used both manual and automatic search strategies without putting any time limitation for searching the relevant papers. We selected 65 primary studies for data extraction, analysis, and synthesis based on our predefined inclusion and exclusion criteria. From the results of the data analysis, we classified the extracted data into four categories based on the data extraction source, categorized the proposed recommendation systems to fit into a taxonomy, and identified the key advantages and limitations of these systems. Our results revealed that human factors play a major role in CSE task recommendation. Further, we identified five popular task types recommended, popular platforms, and their features used in task recommendation. We also provided recommendations for future research directions. This SLR provides insights into current trends, gaps, and future research directions in CSE task recommendation systems

cs.SE↗

arXiv2025

Security Weaknesses of Copilot-Generated Code in GitHub Projects: An Empirical Study

Modern code generation tools utilizing AI models like Large Language Models (LLMs) have gained increased popularity due to their ability to produce functional code. However, their usage presents security challenges, often resulting in insecure code merging into the code base. Thus, evaluating the quality of generated code, especially its security, is crucial. While prior research explored various aspects of code generation, the focus on security has been limited, mostly examining code produced in controlled environments rather than open source development scenarios. To address this gap, we conducted an empirical study, analyzing code snippets generated by GitHub Copilot and two other AI code generation tools (i.e., CodeWhisperer and Codeium) from GitHub projects. Our analysis identified 733 snippets, revealing a high likelihood of security weaknesses, with 29.5% of Python and 24.2% of JavaScript snippets affected. These issues span 43 Common Weakness Enumeration (CWE) categories, including significant ones like CWE-330: Use of Insufficiently Random Values, CWE-94: Improper Control of Generation of Code, and CWE-79: Cross-site Scripting. Notably, eight of those CWEs are among the 2023 CWE Top-25, highlighting their severity. We further examined using Copilot Chat to fix security issues in Copilot-generated code by providing Copilot Chat with warning messages from the static analysis tools, and up to 55.5% of the security issues can be fixed. We finally provide the suggestions for mitigating security issues in generated code.

cs.SE↗

arXiv2024

Exploring the Problems, their Causes and Solutions of AI Pair Programming: A Study on GitHub and Stack Overflow

With the recent advancement of Artificial Intelligence (AI) and Large Language Models (LLMs), AI-based code generation tools become a practical solution for software development. GitHub Copilot, the AI pair programmer, utilizes machine learning models trained on a large corpus of code snippets to generate code suggestions using natural language processing. Despite its popularity in software development, there is limited empirical evidence on the actual experiences of practitioners who work with Copilot. To this end, we conducted an empirical study to understand the problems that practitioners face when using Copilot, as well as their underlying causes and potential solutions. We collected data from 473 GitHub issues, 706 GitHub discussions, and 142 Stack Overflow posts. Our results reveal that (1) Operation Issue and Compatibility Issue are the most common problems faced by Copilot users, (2) Copilot Internal Error, Network Connection Error, and Editor/IDE Compatibility Issue are identified as the most frequent causes, and (3) Bug Fixed by Copilot, Modify Configuration/Setting, and Use Suitable Version are the predominant solutions. Based on the results, we discuss the potential areas of Copilot for enhancement, and provide the implications for the Copilot users, the Copilot team, and researchers.

cs.SE↗

arXiv2024

Fairness Concerns in App Reviews: A Study on AI-based Mobile Apps

Fairness is one of the socio-technical concerns that must be addressed in software systems. Considering the popularity of mobile software applications (apps) among a wide range of individuals worldwide, mobile apps with unfair behaviors and outcomes can affect a significant proportion of the global population, potentially more than any other type of software system. Users express a wide range of socio-technical concerns in mobile app reviews. This research aims to investigate fairness concerns raised in mobile app reviews. Our research focuses on AI-based mobile app reviews as the chance of unfair behaviors and outcomes in AI-based mobile apps may be higher than in non-AI-based apps. To this end, we first manually constructed a ground-truth dataset, including 1,132 fairness and 1,473 non-fairness reviews. Leveraging the ground-truth dataset, we developed and evaluated a set of machine learning and deep learning models that distinguish fairness reviews from non-fairness reviews. Our experiments show that our best-performing model can detect fairness reviews with a precision of 94%. We then applied the best-performing model on approximately 9.5M reviews collected from 108 AI-based apps and identified around 92K fairness reviews. Next, applying the K-means clustering technique to the 92K fairness reviews, followed by manual analysis, led to the identification of six distinct types of fairness concerns (e.g., 'receiving different quality of features and services in different platforms and devices' and 'lack of transparency and fairness in dealing with user-generated content'). Finally, the manual analysis of 2,248 app owners' responses to the fairness reviews identified six root causes (e.g., 'copyright issues') that app owners report to justify fairness concerns.

cs.SE↗

arXiv2024

Mining Architectural Information: A Systematic Mapping Study

Mining Software Repositories (MSR) has become an essential activity in software development. Mining architectural information to support architecting activities, such as architecture understanding, has received significant attention in recent years. However, there is a lack of clarity on what literature on mining architectural information is available. Consequently, this may create difficulty for practitioners to understand and adopt the state-of-the-art research results, such as what approaches should be adopted to mine what architectural information in order to support architecting activities. It also hinders researchers from being aware of the challenges and remedies for the identified research gaps. We aim to identify, analyze, and synthesize the literature on mining architectural information in terms of architectural information and sources mined, architecting activities supported, approaches and tools used, and challenges faced. An SMS has been conducted on the literature published between January 2006 and December 2022. Of the 104 primary studies selected, 7 categories of architectural information have been mined, among which architectural description is the most mined architectural information; 11 categories of sources have been leveraged for mining architectural information, among which version control system is the most popular source; 11 architecting activities can be supported by the mined architectural information, among which architecture understanding is the most supported activity; 95 approaches and 56 tools were proposed and employed in mining architectural information; and 4 types of challenges in mining architectural information were identified. This SMS provides researchers with future directions and help practitioners be aware of what approaches and tools can be used to mine what architectural information from what sources to support various architecting activities.

cs.SE↗

arXiv2024

What Can Self-Admitted Technical Debt Tell Us About Security? A Mixed-Methods Study

Self-Admitted Technical Debt (SATD) encompasses a wide array of sub-optimal design and implementation choices reported in software artefacts (e.g., code comments and commit messages) by developers themselves. Such reports have been central to the study of software maintenance and evolution over the last decades. However, they can also be deemed as dreadful sources of information on potentially exploitable vulnerabilities and security flaws. This work investigates the security implications of SATD from a technical and developer-centred perspective. On the one hand, it analyses whether security pointers disclosed inside SATD sources can be used to characterise vulnerabilities in Open-Source Software (OSS) projects and repositories. On the other hand, it delves into developers' perspectives regarding the motivations behind this practice, its prevalence, and its potential negative consequences. We followed a mixed-methods approach consisting of (i) the analysis of a preexisting dataset containing 8,812 SATD instances and (ii) an online survey with 222 OSS practitioners. We gathered 201 SATD instances through the dataset analysis and mapped them to different Common Weakness Enumeration (CWE) identifiers. Overall, 25 different types of CWEs were spotted across commit messages, pull requests, code comments, and issue sections, from which 8 appear among MITRE's Top-25 most dangerous ones. The survey shows that software practitioners often place security pointers across SATD artefacts to promote a security culture among their peers and help them spot flaky code sections, among other motives. However, they also consider such a practice risky as it may facilitate vulnerability exploits. Our findings suggest that preserving the contextual integrity of security pointers disseminated across SATD artefacts is critical to safeguard both commercial and OSS solutions against zero-day attacks.

cs.SE↗

arXiv2023

Towards an Understanding of Developers' Perceptions of Transparency in Software Development: A Preliminary Study

Software applications play an increasingly critical role in various aspects of our lives, from communication and entertainment to business and healthcare. As these applications become more pervasive, the importance of considering human values in software development has gained significant attention. In this preliminary study, we investigate developers's perceptions and experiences related to human values, with a focus on the human value of transparency. We interviewed five experienced developers and conducted thematic analysis to explore how developers perceive transparency, violations of transparency, and the process of fixing reported violations of transparency. Our findings reveal the significance of transparency as a fundamental value in software development, with developers recognising its importance for building trust, promoting accountability, and fostering ethical practices. Developers recognise the negative consequences of the violation of the human value of transparency and follow a systematic process to fix reported violations. This includes investigation, root cause analysis, corrective action planning, collaborative problem-solving, and testing and verification. These preliminary findings contribute to the understanding of transparency in software development and provide insights for promoting ethical practices.

cs.SE↗

arXiv2023

Security Defect Detection via Code Review: A Study of the OpenStack and Qt Communities

Background: Despite the widespread use of automated security defect detection tools, software projects still contain many security defects that could result in serious damage. Such tools are largely context-insensitive and may not cover all possible scenarios in testing potential issues, which makes them susceptible to missing complex security defects. Hence, thorough detection entails a synergistic cooperation between these tools and human-intensive detection techniques, including code review. Code review is widely recognized as a crucial and effective practice for identifying security defects. Aim: This work aims to empirically investigate security defect detection through code review. Method: To this end, we conducted an empirical study by analyzing code review comments derived from four projects in the OpenStack and Qt communities. Through manually checking 20,995 review comments obtained by keyword-based search, we identified 614 comments as security-related. Results: Our results show that (1) security defects are not prevalently discussed in code review, (2) more than half of the reviewers provided explicit fixing strategies/solutions to help developers fix security defects, (3) developers tend to follow reviewers' suggestions and action the changes, (4) Not worth fixing the defect now and Disagreement between the developer and the reviewer are the main causes for not resolving security defects. Conclusions: Our research results demonstrate that (1) software security practices should combine manual code review with automated detection tools, achieving a more comprehensive coverage to identifying and addressing security defects, and (2) promoting appropriate standardization of practitioners' behaviors during code review remains necessary for enhancing software security.

cs.SE↗

arXiv2023

Dealing with Data Challenges when Delivering Data-Intensive Software Solutions

The predicted increase in demand for data-intensive solution development is driving the need for software, data, and domain experts to effectively collaborate in multi-disciplinary data-intensive software teams (MDSTs). We conducted a socio-technical grounded theory study through interviews with 24 practitioners in MDSTs to better understand the challenges these teams face when delivering data-intensive software solutions. The interviews provided perspectives across different types of roles including domain, data and software experts, and covered different organisational levels from team members, team managers to executive leaders. We found that the key concern for these teams is dealing with data-related challenges. In this paper, we present the theory of dealing with data challenges that explains the challenges faced by MDSTs including gaining access to data, aligning data, understanding data, and resolving data quality issues; the context in and condition under which these challenges occur, the causes that lead to the challenges, and the related consequences such as having to conduct remediation activities, inability to achieve expected outcomes and lack of trust in the delivered solutions. We also identified contingencies or strategies applied to address the challenges including high-level strategic approaches such as implementing data governance, implementing new tools and techniques such as data quality visualisation and monitoring tools, as well as building stronger teams by focusing on people dynamics, communication skill development and cross-skilling. Our findings have direct implications for practitioners and researchers to better understand the landscape of data challenges and how to deal with them.

cs.SE↗

arXiv2023

A Study of Gender Discussions in Mobile Apps

Mobile software apps ("apps") are one of the prevailing digital technologies that our modern life heavily depends on. A key issue in the development of apps is how to design gender-inclusive apps. Apps that do not consider gender inclusion, diversity, and equality in their design can create barriers (e.g., excluding some of the users because of their gender) for their diverse users. While there have been some efforts to develop gender-inclusive apps, a lack of deep understanding regarding user perspectives on gender may prevent app developers and owners from identifying issues related to gender and proposing solutions for improvement. Users express many different opinions about apps in their reviews, from sharing their experiences, and reporting bugs, to requesting new features. In this study, we aim at unpacking gender discussions about apps from the user perspective by analysing app reviews. We first develop and evaluate several Machine Learning (ML) and Deep Learning (DL) classifiers that automatically detect gender reviews (i.e., reviews that contain discussions about gender). We apply our ML and DL classifiers on a manually constructed dataset of 1,440 app reviews from the Google App Store, composing 620 gender reviews and 820 non-gender reviews. Our best classifier achieves an F1-score of 90.77%. Second, our qualitative analysis of a randomly selected 388 out of 620 gender reviews shows that gender discussions in app reviews revolve around six topics: App Features, Appearance, Content, Company Policy and Censorship, Advertisement, and Community. Finally, we provide some practical implications and recommendations for developing gender-inclusive apps.

cs.SE↗

arXiv2023

Characterizing Architecture Related Posts and Their Usefulness in Stack Overflow

Context: Stack Overflow (SO) has won the intention from software engineers (e.g., architects) to learn, practice, and utilize development knowledge, such as Architectural Knowledge (AK). But little is known about AK communicated in SO, which is a type of high-level but important knowledge in development. Objective: This study aims to investigate the AK in SO posts in terms of their categories and characteristics as well as their usefulness from the point of view of SO users. Method: We conducted an exploratory study by qualitatively analyzing a statistically representative sample of 968 Architecture Related Posts (ARPs) from SO. Results: The main findings are: (1) architecture related questions can be classified into 9 core categories, in which "architecture configuration" is the most common category, followed by the "architecture decision" category, and (2) architecture related questions that provide clear descriptions together with architectural diagrams increase their likelihood of getting more than one answer, while poorly structured architecture questions tend to only get one answer. Conclusions: Our findings suggest that future research can focus on enabling automated approaches and tools that could facilitate the search and (re)use of AK in SO. SO users can refer to our proposed guidelines to compose architecture related questions with the likelihood of getting more responses in SO.

cs.SE↗

arXiv2022

Architecture Decisions in AI-based Systems Development: An Empirical Study

Artificial Intelligence (AI) technologies have been developed rapidly, and AI-based systems have been widely used in various application domains with opportunities and challenges. However, little is known about the architecture decisions made in AI-based systems development, which has a substantial impact on the success and sustainability of these systems. To this end, we conducted an empirical study by collecting and analyzing the data from Stack Overflow (SO) and GitHub. More specifically, we searched on SO with six sets of keywords and explored 32 AI-based projects on GitHub, and finally we collected 174 posts and 128 GitHub issues related to architecture decisions. The results show that in AI-based systems development (1) architecture decisions are expressed in six linguistic patterns, among which Solution Proposal and Information Giving are most frequently used, (2) Technology Decision, Component Decision, and Data Decision are the main types of architecture decisions made, (3) Game is the most common application domain among the eighteen application domains identified, (4) the dominant quality attribute considered in architecture decision-making is Performance, and (5) the main limitations and challenges encountered by practitioners in making architecture decisions are Design Issues and Data Issues. Our results suggest that the limitations and challenges when making architecture decisions in AI-based systems development are highly specific to the characteristics of AI-based systems and are mainly of technical nature, which need to be properly confronted.

cs.SE↗