Searcharxiv⌕ Search

arXiv subjects

Stephen G. MacDonell

Publications and source records attributed to Stephen G. MacDonell.

At least 19 recordsLinked to original sources
arXiv2025

On the need to perform comprehensive evaluations of automated program repair benchmarks: Sorald case study

In supporting the development of high-quality software, especially necessary in the era of LLMs, automated program repair (APR) tools aim to improve code quality by automatically addressing violations detected by static analysis profilers. Previous research tends to evaluate APR tools only for their ability to clear violations, neglecting their potential introduction of new (sometimes severe) violations, changes to code functionality and degrading of code structure. There is thus a need for research to develop and assess comprehensive evaluation frameworks for APR tools. This study addresses this research gap, and evaluates Sorald (a state-of-the-art APR tool) as a proof of concept. Sorald's effectiveness was evaluated in repairing 3,529 SonarQube violations across 30 rules within 2,393 Java code snippets extracted from Stack Overflow. Outcomes show that while Sorald fixes specific rule violations, it introduced 2,120 new faults (32 bugs, 2088 code smells), reduced code functional correctness--as evidenced by a 24% unit test failure rate--and degraded code structure, demonstrating the utility of our framework. Findings emphasize the need for evaluation methodologies that capture the full spectrum of APR tool effects, including side effects, to ensure their safe and effective adoption.

cs.SE↗
arXiv2023

Features that Predict the Acceptability of Java and JavaScript Answers on Stack Overflow

Context: Stack Overflow is a popular community question and answer portal used by practitioners to solve problems during software development. Developers can focus their attention on answers that have been accepted or where members have recorded high votes in judging good answers when searching for help. However, the latter mechanism (votes) can be unreliable, and there is currently no way to differentiate between an answer that is likely to be accepted and those that will not be accepted by looking at the answer's characteristics. Objective: In potentially providing a mechanism to identify acceptable answers, this study examines the features that distinguish an accepted answer from an unaccepted answer. Methods: We studied the Stack Overflow dataset by analyzing questions and answers for the two most popular tags (Java and JavaScript). Our dataset comprised 249,588 posts drawn from 2014-2016. We use random forest and neural network models to predict accepted answers, and study the features with the highest predictive power in those two models. Results: Our findings reveal that the length of code in answers, reputation of users, similarity of the text between questions and answers, and the time lag between questions and answers have the highest predictive power for differentiating accepted and unaccepted answers. Conclusion: Tools may leverage these findings in supporting developers and reducing the effort they must dedicate to searching for suitable answers on Stack Overflow.

cs.SE↗
arXiv2022

A Systematic Mapping Study Addressing the Reliability of Mobile Applications: The Need to Move Beyond Testing Reliability

Intense competition in the mobile apps market means it is important to maintain high levels of app reliability to avoid losing users. Yet despite its importance, app reliability is underexplored in the research literature. To address this need, we identify, analyse, and classify the state-of-the-art in the field of mobile apps' reliability through a systematic mapping study. From the results of such a study, researchers in the field can identify pressing research gaps, and developers can gain knowledge about existing solutions, to potentially leverage them in practice. We found 87 relevant papers which were then analysed and classified based on their research focus, research type, contribution, research method, study settings, data, quality attributes and metrics used. Results indicate that there is a lack of research on understanding reliability with regard to context-awareness, self-healing, ageing and rejuvenation, and runtime event handling. These aspects have rarely been studied, or if studied, there is limited evaluation. We also identified several other research gaps including the need to conduct more research in real-world industrial projects. Furthermore, little attention has been paid towards quality standards while conducting research. Outcomes here show numerous opportunities for greater research depth and breadth on mobile app reliability.

cs.SE↗
arXiv2022

An Empirical Study on the Effectiveness of Data Resampling Approaches for Cross-Project Software Defect Prediction

Crossp-roject defect prediction (CPDP), where data from different software projects are used to predict defects, has been proposed as a way to provide data for software projects that lack historical data. Evaluations of CPDP models using the Nearest Neighbour (NN) Filter approach have shown promising results in recent studies. A key challenge with defect-prediction datasets is class imbalance, that is highly skewed datasets where non buggy modules dominate the buggy modules. In the past, data resampling approaches have been applied to within-projects defect prediction models to help alleviate the negative effects of class imbalance in the datasets. To address the class imbalance issue in CPDP, the authors assess the impact of data resampling approaches on CPDP models after the NN Filter is applied. The impact on prediction performance of five oversampling approaches (MAHAKIL, SMOTE, Borderline-SMOTE, Random Oversampling, and ADASYN) and three undersampling approaches (Random Undersampling, Tomek Links, and Onesided selection) is investigated and results are compared to approaches without data resampling. The authors' examined six defect prediction models on 34 datasets extracted from the PROMISE repository. The authors results show that there is a significant positive effect of data resampling on CPDP performance, suggesting that software quality teams and researchers should consider applying data resampling approaches for improved recall (pd) and g-measure prediction performance. However if the goal is to improve precision and reduce false alarm (pf) then data resampling approaches should be avoided.

cs.SE↗
arXiv2021

What Makes Agile Software Development Agile?

Together with many success stories, promises such as the increase in production speed and the improvement in stakeholders' collaboration have contributed to making agile a transformation in the software industry in which many companies want to take part. However, driven either by a natural and expected evolution or by contextual factors that challenge the adoption of agile methods as prescribed by their creator(s), software processes in practice mutate into hybrids over time. Are these still agile? In this article, we investigate the question: what makes a software development method agile? We present an empirical study grounded in a large-scale international survey that aims to identify software development methods and practices that improve or tame agility. Based on 556 data points, we analyze the perceived degree of agility in the implementation of standard project disciplines and its relation to used development methods and practices. Our findings suggest that only a small number of participants operate their projects in a purely traditional or agile manner (under 15%). That said, most project disciplines and most practices show a clear trend towards increasing degrees of agility. Compared to the methods used to develop software, the selection of practices has a stronger effect on the degree of agility of a given discipline. Finally, there are no methods or practices that explicitly guarantee or prevent agility. We conclude that agility cannot be defined solely at the process level. Additional factors need to be taken into account when trying to implement or improve agility in a software company. Finally, we discuss the field of software process-related research in the light of our findings and present a roadmap for future research.

cs.SE↗
arXiv2021

Analyzing the Stationarity Process in Software Effort Estimation Datasets

Software effort estimation models are typically developed based on an underlying assumption that all data points are equally relevant to the prediction of effort for future projects. The dynamic nature of several aspects of the software engineering process could mean that this assumption does not hold in at least some cases. This study employs three kernel estimator functions to test the stationarity assumption in five software engineering datasets that have been used in the construction of software effort estimation models. The kernel estimators are used in the generation of nonuniform weights which are subsequently employed in weighted linear regression modeling. In each model, older projects are assigned smaller weights while the more recently completed projects are assigned larger weights, to reflect their potentially greater relevance to present or future projects that need to be estimated. Prediction errors are compared to those obtained from uniform models. Our results indicate that, for the datasets that exhibit underlying nonstationary processes, uniform models are more accurate than the nonuniform models; that is, models based on kernel estimator functions are worse than the models where no weighting was applied. In contrast, the accuracies of uniform and nonuniform models for datasets that exhibited stationary processes were essentially equivalent. Our analysis indicates that as the heterogeneity of a dataset increases, the effect of stationarity is overridden. The results of our study also confirm prior findings that the accuracy of effort estimation models is independent of the type of kernel estimator function used in model development.

cs.SE↗
arXiv2021

Differences in Jazz Project Leaders' Competencies and Behaviors: A Preliminary Empirical Investigation

Studying the human factors that impact on software development, and assigning individuals with specific competencies and qualities to particular software roles, have been shown to aid software project performance. For instance, prior evidence suggests that extroverted software project leaders are most successful. Role assignment based on individuals' competencies and behaviors may be especially relevant in distributed software development contexts where teams are often affected by distance, cultural, and personality issues. Project leaders in these environments need to possess high levels of inter-personal, intra-personal and organizational competencies if they are to appropriately manage such issues and maintain positive project performance. With a view to understanding and explaining the specific competencies and behaviors that are required of project leaders in these settings, we used psycholinguistic and directed content analysis to study the way six successful IBM Rational Jazz leaders operated while coordinating their three distributed projects. Contrary to previous evidence reported in personality studies, our results did not reveal universal competencies and behaviors among these Jazz leaders. Instead, Jazz project leaders' competencies and behaviors varied with their project portfolio of tasks. Our findings suggest that a pragmatic approach that considers the nature of the software tasks being developed is likely to be a more effective strategy for assigning leaders to distributed software teams, as against a strategy that promotes a specific personality type. We discuss these findings and outline implications for distributed software project governance.

cs.SE↗
arXiv2021

The True Role of Active Communicators: An Empirical Study of Jazz Core Developers

Context: Interest in software engineering (SE) methodologies and tools has been complemented in recent years by research efforts oriented towards understanding the human processes involved in software development. This shift has been imperative given reports of inadequately performing teams and the consequent growing emphasis on individuals and team relations in contemporary SE methods. Objective: While software repositories have frequently been studied with a view to explaining such human processes, research has tended to use primarily quantitative analysis approaches. There is concern, however, that such approaches can provide only a partial picture of the software process. Given the way human behavior is nuanced within psychological and social contexts, it has been asserted that a full understanding may only be achieved through deeper contextual enquiries. Method: We have followed such an approach and have applied data mining, SNA, psycholinguistic analysis and directed content analysis (CA) to study the way core developers at IBM Rational Jazz contribute their social and intellectual capital, and have compared the attitudes, interactions and activities of these members to those of their less active counterparts. Results: Among our results, we uncovered that Jazz's core developers worked across multiple roles, and were crucial to their teams' organizational, intra-personal and inter-personal processes. Additionally, although these individuals were highly task- and achievement-focused, they were also largely responsible for maintaining positive team atmosphere, and for providing context awareness in support of their colleagues. Conclusion: Our results suggest that high-performing distributed agile teams rely on both individual and collective efforts, as well as organizational environments that promote informal and organic work structures.(Abridged)

cs.SE↗
arXiv2021

How do Globally Distributed Agile Teams Self-organise? Initial Insights from a Case Study

Agile software developers are required to self-organize, occupying various informal roles as needed in order to successfully deliver software features. However, previous research has reported conflicting evidence about the way teams actually undertake this activity. The ability to self-organize is particularly necessary for software development in globally distributed environments, where distance has been shown to exacerbate human-centric issues. Understanding the way successful teams self-organise should inform distributed team composition strategies and software project governance. We have used psycholinguistics to study the way IBM Rational Jazz practitioners enacted various roles, expressed attitudes and shared competencies to successfully self-organize in their global projects. Among our findings, we uncovered that practitioners enacted various roles depending on their teams' cohort of features; and that team leaders were most critical to IBM Jazz teams' self-organisation. We discuss these findings and highlight their implications for software project governance.

cs.SE↗
arXiv2021

Self-organising Roles in Agile Globally Distributed Teams

The ability to self-organise is posited to be a fundamental requirement for successful agile teams. In particular, self-organising teams are said to be crucial in agile globally distributed software development (AGSD) settings, where distance exacerbates team issues. We used contextual analysis to study the specific interaction behaviours and enacted roles of practitioners working in multiple AGSD teams. Our results show that the teams studied were extremely task focussed, and those who occupied team lead or programmer roles were central to their teams' self-organisation. These findings have implications for AGSD teams, and particularly for instances when programmers - or those occupying similar non-leadership positions - may not be willing to accept such responsibilities. We discuss the implications of our findings for information system development (ISD) practice.

cs.SE↗
arXiv2021

A Taxonomy of Data Quality Challenges in Empirical Software Engineering

Reliable empirical models such as those used in software effort estimation or defect prediction are inherently dependent on the data from which they are built. As demands for process and product improvement continue to grow, the quality of the data used in measurement and prediction systems warrants increasingly close scrutiny. In this paper we propose a taxonomy of data quality challenges in empirical software engineering, based on an extensive review of prior research. We consider current assessment techniques for each quality issue and proposed mechanisms to address these issues, where available. Our taxonomy classifies data quality issues into three broad areas: first, characteristics of data that mean they are not fit for modeling; second, data set characteristics that lead to concerns about the suitability of applying a given model to another data set; and third, factors that prevent or limit data accessibility and trust. We identify this latter area as of particular need in terms of further research.

cs.SE↗
arXiv2021

Does class size matter? An in-depth assessment of the effect of class size in software defect prediction

In the past 20 years, defect prediction studies have generally acknowledged the effect of class size on software prediction performance. To quantify the relationship between object-oriented (OO) metrics and defects, modelling has to take into account the direct, and potentially indirect, effects of class size on defects. However, some studies have shown that size cannot be simply controlled or ignored, when building prediction models. As such, there remains a question whether, and when, to control for class size. This study provides a new in-depth examination of the impact of class size on the relationship between OO metrics and software defects or defect-proneness. We assess the impact of class size on the number of defects and defect-proneness in software systems by employing a regression-based mediation (with bootstrapping) and moderation analysis to investigate the direct and indirect effect of class size in count and binary defect prediction. Our results show that the size effect is not always significant for all metrics. Of the seven OO metrics we investigated, size consistently has significant mediation impact only on the relationship between Coupling Between Objects (CBO) and defects/defect-proneness, and a potential moderation impact on the relationship between Fan-out and defects/defect-proneness. Based on our results we make three recommendations. One, we encourage researchers and practitioners to examine the impact of class size for the specific data they have in hand and through the use of the proposed statistical mediation/moderation procedures. Two, we encourage empirical studies to investigate the indirect effect of possible additional variables in their models when relevant. Three, the statistical procedures adopted in this study could be used in other empirical software engineering research to investigate the influence of potential mediators/moderators.

cs.SE↗
arXiv2021

Packaged Software Implementation Requirements Engineering by Small Software Enterprises

Small to medium sized business enterprises (SMEs) generally thrive because they have successfully done something unique within a niche market. For this reason, SMEs may seek to protect their competitive advantage by avoiding any standardization encouraged by the use of packaged software (PS). Packaged software implementation at SMEs therefore presents challenges relating to how best to respond to misfits between the functionality offered by the packaged software and each SME's business needs. An important question relates to which processes small software enterprises - or Small to Medium-Sized Software Development Companies (SMSSDCs) - apply in order to identify and then deal with these misfits. To explore the processes of packaged software (PS) implementation, an ethnographic study was conducted to gain in-depth insights into the roles played by analysts in two SMSSDCs. The purpose of the study was to understand PS implementation in terms of requirements engineering (or 'PSIRE'). Data collected during the ethnographic study were analyzed using an inductive approach. Based on our analysis of the cases we constructed a theoretical model explaining the requirements engineering process for PS implementation, and named it the PSIRE Parallel Star Model. The Parallel Star Model shows that during PSIRE, more than one RE process can be carried out at the same time. The Parallel Star Model has few constraints, because not only can processes be carried out in parallel, but they do not always have to be followed in a particular order. This paper therefore offers a novel investigation and explanation of RE practices for packaged software implementation, approaching the phenomenon from the viewpoint of the analysts, and offers the first extensive study of packaged software implementation RE (PSIRE) in SMSSDCs.

cs.SE↗
arXiv2021

Adopting Softer Approaches in the Study of Repository Data: A Comparative Analysis

Context: Given the acknowledged need to understand the people processes enacted during software development, software repositories and mailing lists have become a focus for many studies. However, researchers have tended to use mostly mathematical and frequency-based techniques to examine the software artifacts contained within them. Objective: There is growing recognition that these approaches uncover only a partial picture of what happens during software projects, and deeper contextual approaches may provide further understanding of the intricate nature of software teams' dynamics. We demonstrate the relevance and utility of such approaches in this study. Method: We use psycholinguistics and directed content analysis (CA) to study the way project tasks drive teams' attitudes and knowledge sharing. We compare the outcomes of these two approaches and offer methodological advice for researchers using similar forms of repository data. Results: Our analysis reveals significant differences in the way teams work given their portfolio of tasks and the distribution of roles. Conclusion: We overcome the limitations associated with employing purely quantitative approaches, while avoiding the time-intensive and potentially invasive nature of field work required in full case studies.

cs.SE↗
arXiv2021

Investigating the Significance of Bellwether Effect to Improve Software Effort Estimation

Bellwether effect refers to the existence of exemplary projects (called the Bellwether) within a historical dataset to be used for improved prediction performance. Recent studies have shown an implicit assumption of using recently completed projects (referred to as moving window) for improved prediction accuracy. In this paper, we investigate the Bellwether effect on software effort estimation accuracy using moving windows. The existence of the Bellwether was empirically proven based on six postulations. We apply statistical stratification and Markov chain methodology to select the Bellwether moving window. The resulting Bellwether moving window is used to predict the software effort of a new project. Empirical results show that Bellwether effect exist in chronological datasets with a set of exemplary and recently completed projects representing the Bellwether moving window. Result from this study has shown that the use of Bellwether moving window with the Gaussian weighting function significantly improve the prediction accuracy.

cs.SE↗
arXiv2021

What can developers' messages tell us? A psycholinguistic analysis of Jazz teams' attitudes and behavior patterns

Reports that communication and behavioral issues contribute to inadequately performing software teams have fuelled a wealth of research aimed at understanding the human processes employed during software development. The increasing level of interest in human issues is particularly relevant for agile and global software development approaches that emphasize the importance of people and their interactions during projects. While mature analysis techniques in behavioral psychology have been recommended for studying such issues, particularly when using archives and artifacts, these techniques have rarely been used in software engineering research. We utilize these techniques under an embedded case study approach to examine whether IBM Rational Jazz practitioners' behaviors change over project duration and whether certain tasks affect teams' attitudes and behaviors. We found highest levels of project engagement at project start and completion, as well as increasing levels of team collectiveness as projects progressed. Additionally, Jazz practitioners were most insightful and perceptive at the time of project scoping. Further, Jazz teams' attitudes and behaviors varied in line with the nature of the tasks they were performing. We explain these findings and discuss their implications for software project governance and tool design.

cs.SE↗
arXiv2021

A Critical Evaluation of Failure in a Nearshore Outsourcing Project: What dilemma analysis can tell us

Global Software Engineering (GSE) research contains few examples consciously applying what Glass and colleagues have termed an 'evaluative-critical' approach. In this study we apply dilemma analysis to conduct a critical review of a major (and ongoing) nearshore Business Process Outsourcing project in New Zealand. The project has become so troubled that a Government Minister has recently been assigned responsibility for troubleshooting it. The 'Novopay' project concerns the implementation of a nationwide payroll system responsible for the payment of some 110,000 teachers and education sector staff. An Australian company won the contract for customizing and implementing the Novopay system, taking over from an existing New Zealand service provider. We demonstrate how a modified form of dilemma analysis can be a powerful technique for highlighting risks and stakeholder impacts from empirical data, and that adopting an evaluative-critical approach to such projects can usefully highlight tensions and barriers to satisfactory project outcomes.

cs.SE↗