arXiv subjects

Mojtaba Shahin

Publications and source records attributed to Mojtaba Shahin.

At least 19 recordsLinked to original sources

arXiv2026

What Motivates Whom? A Survey of Newcomers to OSS and Experienced OSS Practitioners

Open source software (OSS) development continues to expand, yet software practitioners often struggle to select suitable projects, leading to inefficient onboarding and disengagement. Understanding how contributors select OSS projects is important for supporting contributors onboarding, engagement, and long-term participation within OSS communities. This study investigates contributors' project-selection preferences in OSS projects and examines how these preferences correlate with contributors' motivations and demographic backgrounds. Through an online survey of 208 practitioners, we found that demographic factors, such as age, gender, and the OSS role they held, significantly correlate with their motivations. Additionally, preferences for project characteristics such as project age, development stage, and documentation quality vary based on specific motivations. Importantly, our findings are presented through a comparative lens, analyzing the responses of newcomers to OSS and experienced OSS practitioners separately to uncover their distinct preferences. Lastly, we explore software practitioners' perspectives on how existing recommendation systems could better support project selection and align with their motivations. By disentangling the unique needs of newcomers to OSS and OSS practitioners, our findings provide insights for researchers, OSS project owners, and software practitioners to improve contributor onboarding, engagement, and retention, while also informing future project recommendation systems and improving the OSS ecosystem.

cs.SE↗

arXiv2026

CodeTeam: An LLM-Powered Multi-Agent Framework for Repository-Level Code Generation

Natural language to repository generation (NL2Repo) requires a system to construct an entire software repository from a natural-language requirements document. Compared with function-level code generation, this task demands longer planning horizons, stable interfaces across files, and iterative debugging of cross-file inconsistencies. To address these challenges, we propose CodeTeam, an LLM-based multi-agent framework that separates planning, decision making, and implementation into distinct, coordinated stages. In the planning stage, multiple Architect agents draft competing software design sketches (SDS), optionally grounded by retrieved design references. A CTO agent then evaluates, selects, and normalizes the most promising SDS into a machine-checkable contract that specifies file ownership, public interfaces, and dependency constraints. In the implementation stage, Developer agents generate code under a dependency-aware scheduler with bounded context and lightweight Git-based coordination, while a QA agent runs tests and drives iterative repairs. On the synthesis-based SketchEval benchmark, we explicitly compare CodeTeam's prompt-engineering (PE) and supervised fine-tuning (SFT) variants with the corresponding CodeS variants, where CodeTeam improves the overall SketchBLEU by 4.1 and 2.9 absolute points, respectively. On the execution-based NL2Repo-Bench benchmark, used as an external validation protocol, CodeTeam achieves the highest average test pass rate in both settings (34.6% PE, 42.3% SFT), confirming that the sketch-improvements extend to functional correctness under upstream test suites. Ablation results show that project-specific developer allocation and retrieval-augmented planning each contribute substantially to the SketchBLEU improvement (9.9% and 8.1% relative, respectively). CodeTeam and the experimental results are available at https://github.com/WhitenWhiten/CodeTeam

cs.SE↗

arXiv2026

Beyond Models: Reflections on Engineering AI-enabled Systems in a Project-Based Course

Teaching Software Engineering for AI-enabled systems entails addressing the integration of AI components within full-scale software architectures under realistic constraints. While machine learning courses emphasize model development, students often lack experience in architectural design, deployment, and monitoring of AI-enabled systems. Empirical evaluations of such system-oriented AI courses remain limited. This paper reflects on the design and implementation of a project-based master's-level course titled AI Algorithms: Theory and Engineering, at the University of Bremen, in which students developed a movie recommendation system while making architectural design decisions to address challenges related to scalability, deployment, and evolving requirements. We conducted a mixed-methods study combining analyses of student submissions and questionnaire responses to investigate integration challenges, learning outcomes, and opportunities for improvement. Our results indicate persistent difficulties in early architectural decisions, heterogeneous ML integration, evolving requirements, and data management, largely due to uneven ML and software engineering expertise. From the educator's perspective, the course fostered system-level reasoning and strengthened awareness of data-centric ML practices in AI-enabled systems.

cs.SE↗

arXiv2026

Rule Taxonomy and Evolution in AI IDEs: A Mining and Survey Study

The adoption of AI-powered Integrated Development Environments (AI IDEs) has introduced "Rules" as a novel software artifact, allowing developers to persistently inject project-specific constraints and architectural guidelines into the context of Large Language Models (LLMs). Despite their role in aligning AI behavior with developer intent, the taxonomy, evolution, and practical impact of these rules remain largely unexplored. To bridge this gap, we conducted a mixed-methods empirical study on AI IDE rules. By mining 83 open-source projects and extracting 7,310 rules, we established a comprehensive taxonomy comprising 5 primary and 25 secondary categories. We then triangulated these artifacts with survey responses from 99 practitioners. Our analysis identified a contrast between developer priorities and actual configurations: while practitioners rate architectural constraints as highly important, rule files in repositories primarily consist of low-level workflow and code formatting constraints. Furthermore, our analysis of 1,540 rule evolution events revealed that rules are updated frequently. Repository data further indicate that rule evolution is primarily driven by constructive context expansions (29.17%) and enrichments (26.59%). In contrast, surveyed developers reported modifying rules primarily to correct AI errors (77.78%), typically by adding new negative constraints rather than editing existing ones. Finally, an artifact compliance assessment of 160 rule evolution events revealed that updating rules significantly improves the adherence of software artifacts, with the average artifact compliance rate increasing by 22.99% (from 49.14% to 72.13%) following an update. Our study provides empirical insights that can help developers optimize prompting strategies and guide tool builders in designing automated conflict-detection and context-management mechanisms for AI IDEs.

cs.SE↗

arXiv2026

Using LLMs in Software Design: An Empirical Study of GitHub and A Practitioner Survey

Recent advancements in Large Language Models (LLMs) have demonstrated significant potential across a wide range of software engineering tasks, including software design, an area traditionally regarded as highly dependent on human expertise and judgment. However, there has been little research focusing on how LLMs are used in software design, nor on the associated benefits and drawbacks. This paper aims to bridge this gap by empirically investigating how software developers utilize LLMs in the context of software design. We conduct a mixed-methods study, combining a mining study of 291 developer-ChatGPT conversations shared on GitHub with a survey of 65 software practitioners. Our findings reveal nine distinct categories of design tasks supported by ChatGPT, including architecture design, data model design, and the use of design patterns. We further characterize developer-ChatGPT interactions, showing that developers primarily use ChatGPT for knowledge acquisition and design-related code generation, with most tasks situated at the detailed design level. The study identifies seven key benefits of utilizing LLMs in software design as perceived by developers, such as better technology selection and the early detection of design flaws. We also uncover six limitations, including the generation of overly lengthy and difficult-to-read outputs, the creation of inexecutable or incorrect code, and a heavy reliance on context that can lead to hallucinated results. These findings provide an evidence-based characterization of current LLM use in software design from both open-source and practitioner perspectives, highlighting a tension between perceived benefits and limitations, which lays a foundation for future research and the development of effective techniques and tools to integrate LLMs into software design practices.

cs.SE↗

arXiv2026

Beyond Functional Correctness: Design Issues in AI IDE-Generated Large-Scale Projects

New generation of AI coding tools, including AI-powered IDEs equipped with agentic capabilities, can generate code within the context of the project. These AI IDEs are increasingly perceived as capable of producing project-level code at scale. However, there is limited empirical evidence on the extent to which they can generate large-scale software systems and what design issues such systems may exhibit. To address this gap, we conducted a study to explore the capability of Cursor in generating large-scale projects and to evaluate the design quality of projects generated by Cursor. First, we propose a Feature-Driven Human-In-The-Loop (FD-HITL) framework that systematically guides project generation from curated project descriptions. We generated 10 projects using Cursor with the FD-HITL framework across three application domains and multiple technologies. We assessed the functional correctness of these projects through manual evaluation, obtaining an average functional correctness score of 91%. Next, we analyzed the generated projects using two static analysis tools, CodeScene and SonarQube, to detect design issues. We identified 1,305 design issues categorized into 9 categories by CodeScene and 3,193 issues in 11 categories by SonarQube. Our findings show that (1) when used with the FD-HITL framework, Cursor can generate functional large-scale projects averaging 16,965 LoC and 114 files; (2) the generated projects nevertheless contain design issues that may pose long-term maintainability and evolvability risks, requiring careful review by experienced developers; (3) the most prevalent issues include Code Duplication, high Code Complexity, Large Methods, Framework Best-Practice Violations, Exception-Handling Issues and Accessibility Issues; (4) these design issues violate design principles such as SRP, SoC, and DRY. The replication package is at https://github.com/Kashifraz/DIinAGP

cs.SE↗

arXiv2026

One-Year Internship Program on Software Engineering: Students' Perceptions and Educators' Lessons Learned

The inclusion of internship courses in Software Engineering (SE) programs is essential for closing knowledge gaps and improving graduates' readiness for the software industry. Our study focuses on year-long internships at RMIT University (Melbourne, Australia), which offers in-depth industry engagement. We analysed how the course evolved over the last 10 years to incorporate students' needs and summarised the lessons learned that can be helpful for other educators supporting internship courses. Our qualitative analysis of internship data based on 91 reports during 2023-2024 identified three challenge themes the students faced, and which courses were found by students to be particularly beneficial during their internships. On this basis, we proposed recommendations for educators and companies to help interns overcome challenges and maximise their learning experience.

cs.SE↗

arXiv2026

Behaviour Driven Development Scenario Generation with Large Language Models

This paper presents an evaluation of three LLMs, GPT-4, Claude 3, and Gemini, for automated Behaviour-Driven Development (BDD) scenarios generation. To support this evaluation, we constructed a dataset of 500 user stories, requirement descriptions, and their corresponding BDD scenarios, drawn from four proprietary software products. We assessed the quality of BDD scenarios generated by LLMs using a multidimensional evaluation framework encompassing text and semantic similarity metrics, LLM-based evaluation, and human expert assessment. Our findings reveal that although GPT-4 achieves higher scores in text and semantic similarity metrics, Claude 3 produces scenarios rated highest by both human experts and LLM-based evaluators. LLM-based evaluators, particularly DeepSeek, show a stronger correlation with human judgment than with text similarity and semantic similarity metrics. The effectiveness of prompting techniques is model-specific: GPT-4 performs best with zero-shot, Claude 3 benefits from chain-of-thought reasoning, and Gemini achieves optimal results with few-shot examples. Input quality determines the effectiveness of BDD scenario generation: detailed requirement descriptions alone yield high-quality scenarios, whereas user stories alone yield low-quality scenarios. Our experiments indicate that setting temperature to 0 and top_p to 1.0 produced the highest-quality BDD scenarios across all models.

cs.SE↗

arXiv2026

Beyond Code: Empirical Insights into How Team Dynamics Influence OSS Project Selection

Open-source software (OSS) development relies on effective collaboration among distributed contributors. Yet, current OSS project recommendation systems primarily emphasize technical attributes, overlooking the collaboration and community aspects that influence contributors' decisions to join and remain in projects. This study investigates how team dynamics within OSS communities influence project selection and how these preferences vary across contributors' motivations. We conducted an online survey with 198 OSS practitioners, combining quantitative and qualitative analyses to capture contributors' perceptions of team dynamics. The results reveal that communication-related team dynamics such as responsiveness, tone, and clarity of replies are consistently prioritized across practitioners. However, the relative importance of these team dynamics differs according to contributors' motivations. For instance, practitioners motivated by gaining reputation or networking preferred inclusive project communities that encouraged diverse participation. These findings highlight that understanding how team dynamics align with contributors' motivations provides valuable insights into practitioners' project selection behaviour. Those insights can inform the design of future human-aware project recommendation systems that better account for social collaboration quality and motivational fit.

cs.SE↗

arXiv2026

Reading Between the Code Lines: On the Use of Self-Admitted Technical Debt for Security Analysis

Static Analysis Tools (SATs) are central to security engineering activities, as they enable early identification of code weaknesses without requiring execution. However, their effectiveness is often limited by high false-positive rates and incomplete coverage of vulnerability classes. At the same time, developers frequently document security-related shortcuts and compromises as Self-Admitted Technical Debt (SATD) in software artifacts, such as code comments. While prior work has recognized SATD as a rich source of security information, it remains unclear whether -and in what ways- it is utilized during SAT-aided security analysis. OBJECTIVE: This work explores whether and how the security-related information encoded in SATD provides complementary security insights to SATs. METHOD: We followed a mixed-methods approach comprising (i) the analysis of a manually curated, SATD-annotated vulnerability dataset using three SATs and (ii) an online survey involving 72 security-aware software practitioners. RESULTS: The selected SATs flagged 114 of the 135 validated Security-related SATD instances (SSATD), yet the overlap between SAT-derived and manually mapped Common Weakness Enumeration (CWE) identifiers was only 6.42%, indicating that both sources often expose different kinds of security information. In particular, SSATD captured several dynamic and context-dependent weakness types that SATs commonly overlook or struggle to detect. Survey responses further indicate that practitioners rely on SSATD to contextualize SAT findings by understanding their impact, root causes, and potential fixes. IMPLICATIONS: Our findings suggest that SSATD constitutes a valuable and cost-effective source of complementary security knowledge that can support the interpretation, prioritization, and further assessment of SAT findings.

cs.CR↗

arXiv2026

Age Matters: Analyzing Age-Related Discussions in App Reviews

In recent years, mobile applications have become indispensable tools for managing various aspects of life. From enhancing productivity to providing personalized entertainment, mobile apps have revolutionized people's daily routines. Despite this rapid growth and popularity, gaps remain in how these apps address the needs of users from different age groups. Users of varying ages face distinct challenges when interacting with mobile apps, from younger users dealing with inappropriate content to older users having difficulty with usability due to age-related vision and cognition impairments. Although there have been initiatives to create age-inclusive apps, a limited understanding of user perspectives on age-related issues may hinder developers from recognizing specific challenges and implementing effective solutions. In this study, we explore age discussions in app reviews to gain insights into how mobile apps should cater to users across different age groups.We manually curated a dataset of 4,163 app reviews from the Google Play Store and identified 1,429 age-related reviews and 2,734 non-age-related reviews. We employed eight machine learning, deep learning, and large language models to automatically detect age discussions, with RoBERTa performing the best, achieving a precision of 92.46%. Additionally, a qualitative analysis of the 1,429 age-related reviews uncovers six dominant themes reflecting user concerns.

cs.SE↗

arXiv2025

FasterPy: An LLM-based Code Execution Efficiency Optimization Framework

Code often suffers from performance bugs. These bugs necessitate the research and practice of code optimization. Traditional rule-based methods rely on manually designing and maintaining rules for specific performance bugs (e.g., redundant loops, repeated computations), making them labor-intensive and limited in applicability. In recent years, machine learning and deep learning-based methods have emerged as promising alternatives by learning optimization heuristics from annotated code corpora and performance measurements. However, these approaches usually depend on specific program representations and meticulously crafted training datasets, making them costly to develop and difficult to scale. With the booming of Large Language Models (LLMs), their remarkable capabilities in code generation have opened new avenues for automated code optimization. In this work, we proposed FasterPy, a low-cost and efficient framework that adapts LLMs to optimize the execution efficiency of Python code. FasterPy combines Retrieval-Augmented Generation (RAG), supported by a knowledge base constructed from existing performance-improving code pairs and corresponding performance measurements, with Low-Rank Adaptation (LoRA) to enhance code optimization performance. Our experimental results on the Performance Improving Code Edits (PIE) benchmark demonstrate that our method outperforms existing models on multiple metrics. The FasterPy tool and the experimental results are available at https://github.com/WuYue22/fasterpy.

cs.SE↗

arXiv2025

Using LLMs in Generating Design Rationale for Software Architecture Decisions

Design Rationale (DR) for software architecture decisions refers to the reasoning underlying architectural choices, which provides valuable insights into the different phases of the architecting process throughout software development. However, in practice, DR is often inadequately documented due to a lack of motivation and effort from developers. With the recent advancements in Large Language Models (LLMs), their capabilities in text comprehension, reasoning, and generation may enable the generation and recovery of DR for architecture decisions. In this study, we evaluated the performance of LLMs in generating DR for architecture decisions. First, we collected 50 Stack Overflow (SO) posts, 25 GitHub issues, and 25 GitHub discussions related to architecture decisions to construct a dataset of 100 architecture-related problems. Then, we selected five LLMs to generate DR for the architecture decisions with three prompting strategies, including zero-shot, chain of thought (CoT), and LLM-based agents. With the DR provided by human experts as ground truth, the Precision of LLM-generated DR with the three prompting strategies ranges from 0.267 to 0.278, Recall from 0.627 to 0.715, and F1-score from 0.351 to 0.389. Additionally, 64.45% to 69.42% of the arguments of DR not mentioned by human experts are also helpful, 4.12% to 4.87% of the arguments have uncertain correctness, and 1.59% to 3.24% of the arguments are potentially misleading. To further understand the trustworthiness and applicability of LLM-generated DR in practice, we conducted semi-structured interviews with six practitioners. Based on the experimental and interview results, we discussed the pros and cons of the three prompting strategies, the strengths and limitations of LLM-generated DR, and the implications for the practical use of LLM-generated DR.

cs.SE↗

arXiv2025

Designing LLM-based Multi-Agent Systems for Software Engineering Tasks: Quality Attributes, Design Patterns and Rationale

As the complexity of Software Engineering (SE) tasks continues to escalate, Multi-Agent Systems (MASs) have emerged as a focal point of research and practice due to their autonomy and scalability. Furthermore, through leveraging the reasoning and planning capabilities of Large Language Models (LLMs), the application of LLM-based MASs in the field of SE is garnering increasing attention. However, there is no dedicated study that systematically explores the design of LLM-based MASs, including the Quality Attributes (QAs) on which designers mainly focus, the design patterns used by designers, and the rationale guiding the design of LLM-based MASs for SE tasks. To this end, we conducted a study to identify the QAs that LLM-based MASs for SE tasks focus on, the design patterns used in the MASs, and the design rationale for the MASs. We collected 94 papers on LLM-based MASs for SE tasks as the source. Our study shows that: (1) Code Generation is the most common SE task solved by LLM-based MASs among ten identified SE tasks, (2) Functional Suitability is the QA on which designers of LLM-based MASs pay the most attention, (3) Role-Based Cooperation is the design pattern most frequently employed among 16 patterns used to construct LLM-based MASs, and (4) Improving the Quality of Generated Code is the most common rationale behind the design of LLM-based MASs. Based on the study results, we presented the implications for the design of LLM-based MASs to support SE tasks.

cs.SE↗

arXiv2025

ArchISMiner: A Framework for Automatic Mining of Architectural Issue-Solution Pairs from Online Developer Communities

Stack Overflow (SO), a leading online community forum, is a rich source of software development knowledge. However, locating architectural knowledge, such as architectural solutions remains challenging due to the overwhelming volume of unstructured content and fragmented discussions. Developers must manually sift through posts to find relevant architectural insights, which is time-consuming and error-prone. This study introduces ArchISMiner, a framework for mining architectural knowledge from SO. The framework comprises two complementary components: ArchPI and ArchISPE. ArchPI trains and evaluates multiple models, including conventional ML/DL models, Pre-trained Language Models (PLMs), and Large Language Models (LLMs), and selects the best-performing model to automatically identify Architecture-Related Posts (ARPs) among programming-related discussions. ArchISPE employs an indirect supervised approach that leverages diverse features, including BERT embeddings and local TextCNN features, to extract architectural issue-solution pairs. Our evaluation shows that the best model in ArchPI achieves an F1-score of 0.960 in ARP detection, and ArchISPE outperforms baselines in both SE and NLP fields, achieving F1-scores of 0.883 for architectural issues and 0.894 for solutions. A user study further validated the quality (e.g., relevance and usefulness) of the identified ARPs and the extracted issue-solution pairs. Moreover, we applied ArchISMiner to three additional forums, releasing a dataset of over 18K architectural issue-solution pairs. Overall, ArchISMiner can help architects and developers identify ARPs and extract succinct, relevant, and useful architectural knowledge from developer communities more accurately and efficiently. The replication package of this study has been provided at https://github.com/JeanMusenga/ArchISPE

cs.SE↗

arXiv2025

How Do Users Revise Architectural Related Questions on Stack Overflow: An Empirical Study

Technical Questions and Answers (Q&A) sites, such as Stack Overflow (SO), accumulate a significant variety of information related to software development in posts from users. To ensure the quality of this information, SO encourages its users to review posts through various mechanisms (e.g., question and answer revision processes). Although Architecture Related Posts (ARPs) communicate architectural information that has a system-wide impact on development, little is known about how SO users revise information shared in ARPs. To fill this gap, we conducted an empirical study to understand how users revise Architecture Related Questions (ARQs) on SO. We manually checked 13,205 ARPs and finally identified 4,114 ARQs that contain revision information. Our main findings are that: (1) The revision of ARQs is not prevalent in SO, and an ARQ revision starts soon after this question is posted (i.e., from 1 minute onward). Moreover, the revision of an ARQ occurs before and after this question receives its first answer/architecture solution, with most revisions beginning before the first architecture solution is posted. Both Question Creators (QCs) and non-QCs actively participate in ARQ revisions, with most revisions being made by QCs. (2) A variety of information (14 categories) is missing and further provided in ARQs after being posted, among which design context, component dependency, and architecture concern are dominant information. (3) Clarify the understanding of architecture under design and improve the readability of architecture problem are the two major purposes of the further provided information in ARQs. (4) The further provided information in ARQs has several impacts on the quality of answers/architecture solutions, including making architecture solution useful, making architecture solution informative, making architecture solution relevant, among others.

cs.SE↗

arXiv2025

Toward Inclusive Low-Code Development: Detecting Accessibility Issues in User Reviews

Low-code applications are gaining popularity across various fields, enabling non-developers to participate in the software development process. However, due to the strong reliance on graphical user interfaces, they may unintentionally exclude users with visual impairments, such as color blindness and low vision. This paper investigates the accessibility issues users report when using low-code applications. We construct a comprehensive dataset of low-code application reviews, consisting of accessibility-related reviews and non-accessibility-related reviews. We then design and implement a complex model to identify whether a review contains an accessibility-related issue, combining two state-of-the-art Transformers-based models and a traditional keyword-based system. Our proposed hybrid model achieves an accuracy and F1-score of 78% in detecting accessibility-related issues.

cs.SE↗

arXiv2025

Decision Models for Selecting Architecture Patterns and Strategies in Quantum Software Systems

Quantum software is an emerging class of software systems, services, and applications that leverage the principles of quantum mechanics through programmable quantum bits (qubits) and quantum gates to perform computations that offer advantages for certain classes of problems. Quantum software architecture enables quantum software developers to abstract away implementation-specific details (i.e., mapping of qubits and quantum gates to high-level architectural components and connectors). Architecture patterns provide proven solutions to recurring design problems, while architecture strategies are approaches that guide the overall design and evolution of a system to meet specific goals. Such patterns and strategies are needed because quantum software systems involve complex hybrid quantum-classical design concerns. However, quantum software practitioners face significant challenges in selecting and implementing appropriate patterns and strategies. To address these challenges, this study proposes decision models for selecting patterns and strategies in six critical design areas in quantum software systems: Communication, Decomposition, Data Processing, Fault Tolerance, Integration and Optimization, and Algorithm Implementation. These decision models are constructed based on data collected from both a mining study (GitHub and Stack Exchange) and an SLR, which were used to identify relevant patterns and strategies with their involved QAs. We then conducted semi-structured interviews with 30 quantum software practitioners to evaluate the familiarity, understandability, completeness, and usefulness of the proposed decision models. The results show that the proposed decision models can aid practitioners in selecting suitable patterns and strategies to address the challenges related to the architecture design of quantum software systems. The dataset is at https://github.com/shamimaaktar1/DMQSA

cs.SE↗