arXiv subjects

Stephen G. MacDonell

Publications and source records attributed to Stephen G. MacDonell.

At least 37 records · Page 2Linked to original sources

arXiv2021

Data Quality in Empirical Software Engineering: A Targeted Review

Context: The utility of prediction models in empirical software engineering (ESE) is heavily reliant on the quality of the data used in building those models. Several data quality challenges such as noise, incompleteness, outliers and duplicate data points may be relevant in this regard. Objective: We investigate the reporting of three potentially influential elements of data quality in ESE studies: data collection, data pre-processing, and the identification of data quality issues. This enables us to establish how researchers view the topic of data quality and the mechanisms that are being used to address it. Greater awareness of data quality should inform both the sound conduct of ESE research and the robust practice of ESE data collection and processing. Method: We performed a targeted literature review of empirical software engineering studies covering the period January 2007 to September 2012. A total of 221 relevant studies met our inclusion criteria and were characterized in terms of their consideration and treatment of data quality. Results: We obtained useful insights as to how the ESE community considers these three elements of data quality. Only 23 of these 221 studies reported on all three elements of data quality considered in this paper. Conclusion: The reporting of data collection procedures is not documented consistently in ESE studies. It will be useful if data collection challenges are reported in order to improve our understanding of why there are problems with software engineering data sets and the models developed from them. More generally, data quality should be given far greater attention by the community. The improvement of data sets through enhanced data collection, pre-processing and quality assessment should lead to more reliable prediction models, thus improving the practice of software engineering.

cs.SE↗

arXiv2021

Investigating the Significance of the Bellwether Effect to Improve Software Effort Prediction: Further Empirical Study

Context: In addressing how best to estimate how much effort is required to develop software, a recent study found that using exemplary and recently completed projects [forming Bellwether moving windows (BMW)] in software effort prediction (SEP) models leads to relatively improved accuracy. More studies need to be conducted to determine whether the BMW yields improved accuracy in general, since different sizing and aging parameters of the BMW are known to affect accuracy. Objective: To investigate the existence of exemplary projects (Bellwethers) with defined window size and age parameters, and whether their use in SEP improves prediction accuracy. Method: We empirically investigate the moving window assumption based on the theory that the prediction outcome of a future event depends on the outcomes of prior events. Sampling of Bellwethers was undertaken using three introduced Bellwether methods (SSPM, SysSam, and RandSam). The ergodic Markov chain was used to determine the stationarity of the Bell-wethers. Results: Empirical results show that 1) Bellwethers exist in SEP and 2) the BMW has an approximate size of 50 to 80 exemplary projects that should not be more than 2 years old relative to the new projects to be estimated. Conclusion: The study's results add further weight to the recommended use of Bellwethers for improved prediction accuracy in SEP.

cs.SE↗

arXiv2021

Diagnosable-by-Design Model-Driven Development for IEC 61499 Industrial Cyber-Physical Systems

Integrating the design and creation of fault identification and diagnostic capabilities into Model-Driven Development methodologies is one approach to enhancing the resilience of Industrial Cyber-Physical Systems. We present a Fault Diagnostic Engine designed to recognise and diagnose faults in IEC 61499 Function Block Applications. Using diagnostic agents that interact directly with the target application, we demonstrate fault monitoring and analysis techniques and as well as failure scenario intervention. By designing and building fault diagnostic resources during early phases of Model-Driven Development, both iterative testing and long-term fault management capabilities can be created. While applying and refining appropriate model artifacts, we demonstrate that the concurrent development of function blocks alongside fault management capabilities is both feasible and worthwhile.

cs.SE↗

arXiv2021

Employing Agent Beliefs during Fault Diagnosis for IEC 61499 Industrial Cyber-Physical Systems

We have come to rely on industrial-scale cyber-physical systems more and more to manage tasks and machinery in safety-critical situations. Efficient, reliable fault identification and management has become a critical factor in the design of these increasingly sophisticated and complex devices. Teams of co-operating software agents are one way to coordinate the flow of diagnostic information gathered during fault-finding. By wielding domain knowledge of the software architecture used to construct the system, agents build and refine their beliefs about the location and root cause of faults. This paper examines how agents constructed within the GORITE Multi-Agent Framework create and refine their beliefs. We demonstrate three different belief structures implemented within our Fault Diagnostic Engine, showing how each supports a distinct aspect of the agent's reasoning. Using domain knowledge of the IEC 61499 Function Block architecture, agents are able to examine and rigorously evaluate both individual components and entire subsystems.

cs.SE↗

arXiv2021

Synchronised Visualisation of Software Process and Product Artefacts: Concept, Design and Prototype Implementation

Context: Most prior software visualisation (SV) research has focused primarily on making aspects of intangible software product artefacts more evident. While undoubtedly useful, this focus has meant that software process visualisation has received far less attention. Objective: This paper presents Conceptual Visualisation, a novel SV approach that builds on the well-known CodeCity metaphor by situating software code artefacts alongside their software development processes, in order to link and synchronise these typically separate components. Method: While the majority of prior SV research has focused on representing what is already available in the code (i.e., the implementation) or information derived from it (i.e., various metrics), the presented approach instead makes the design concepts and original developers' intentions -- both significant sources of information in terms of software development and maintenance -- readily and contextually available in a visualisation environment that tightly integrates the code artefacts with their original functional requirements and development activity. Results: Our approach has been implemented in a prototype tool called ScrumCity with the proof of concept being demonstrated using six real-world open source systems. A preliminary case study has further been carried out with real world data. Conclusion: Conceptual Visualisation, as implemented in ScrumCity, shows early promise in enabling developers and managers (and potentially other stakeholders) to traverse and explore multiple aspects of software product and process artefacts in a synchronised manner, achieving traceability between the two.

cs.SE↗

arXiv2021

Technical debt and agile software development practices and processes: An industry practitioner survey

Context: Contemporary software development is typically conducted in dynamic, resource-scarce environments that are prone to the accumulation of technical debt. While this general phenomenon is acknowledged, what remains unknown is how technical debt specifically manifests in and affects software processes, and how the software development techniques employed accommodate or mitigate the presence of this debt. Objectives: We sought to draw on practitioner insights and experiences in order to classify the effects of agile method use on technical debt management. We explore the breadth of practitioners' knowledge about technical debt; how technical debt is manifested across the software process; and the perceived effects of common agile software development practices and processes on technical debt. In doing so, we address a research gap in technical debt knowledge and provide novel and actionable managerial recommendations. Method: We designed, tested and executed a multi-national survey questionnaire to address our objectives, receiving 184 responses from practitioners in Brazil, Finland, and New Zealand. Results: Our findings indicate that: 1) Practitioners are aware of technical debt, although, there was under utilization of the concept, 2) Technical debt commonly resides in legacy systems, however, concrete instances of technical debt are hard to conceptualize which makes it problematic to manage, 3) Queried agile practices and processes help to reduce technical debt; particularly, techniques that verify and maintain the structure and clarity of implemented artifacts. Conclusions: The fact that technical debt instances tend to have characteristics in common means that a systematic approach to its management is feasible. However, notwithstanding the positive effects of some agile practices on technical debt management, competing stakeholders' interests remain a concern.(Abridged)

cs.SE↗

arXiv2021

Revisiting the size effect in software fault prediction models

BACKGROUND: In object oriented (OO) software systems, class size has been acknowledged as having an indirect effect on the relationship between certain artifact characteristics, captured via metrics, and faultproneness, and therefore it is recommended to control for size when designing fault prediction models. AIM: To use robust statistical methods to assess whether there is evidence of any true effect of class size on fault prediction models. METHOD: We examine the potential mediation and moderation effects of class size on the relationships between OO metrics and number of faults. We employ regression analysis and bootstrapping-based methods to investigate the mediation and moderation effects in two widely-used datasets comprising seventeen systems. RESULTS: We find no strong evidence of a significant mediation or moderation effect of class size on the relationships between OO metrics and faults. In particular, size appears to have a more significant mediation effect on CBO and Fan-out than other metrics, although the evidence is not consistent in all examined systems. On the other hand, size does appear to have a significant moderation effect on WMC and CBO in most of the systems examined. Again, the evidence provided is not consistent across all examined systems CONCLUSION: We are unable to confirm if class size has a significant mediation or moderation effect on the relationships between OO metrics and the number of faults. We contend that class size does not fully explain the relationships between OO metrics and the number of faults, and it does not always affect the strength/magnitude of these relationships. We recommend that researchers consider the potential mediation and moderation effect of class size when building their prediction models, but this should be examined independently for each system.

cs.SE↗

arXiv2021

Exploring the links between software development task type, team attitudes and task completion performance: Insights from the Jazz repository

Context: In seeking to better understand the impact of various human factors in software development, and how teams' attitudes relate to their performance, increasing attention is being given to the study of team-related artefacts. In particular, researchers have conducted numerous studies on a range of team communication channels to explore links between developers' language use and the incidence of software bugs in the products they delivered. Comparatively limited attention has been paid, however, to the full range of software tasks that are commonly performed during the development and delivery of software systems, in spite of compelling evidence pointing to the need to understand teams' attitudes more widely. Objective: We were therefore motivated to study the relationships between task type and team attitudes, and how attitudes expressed in teams' communications might be related to their task completion performance when undertaking a range of activities. Method: Our investigation involved artefacts from 474 IBM Jazz practitioners assembled in 149 teams working on around 30,000 software development tasks over a three-year period. We applied linguistic analysis, standard statistical techniques and directed content analysis to address our research objective. Results: Our evidence revealed that teams expressed different attitudes when working on various forms of software tasks, and they were particularly emotional when working to remedy defects. That said, teams' expression of attitudes was not found to be a strong predictor of their task completion performance. Conclusion: Efforts aimed at reducing bug incidence may positively limit teams' emotional disposition when resolving bugs, thereby reducing the otherwise high demand for emotionally stable members. (Abridged)

cs.SE↗

arXiv2021

On Design-time Security in IEC 61499 Systems: Conceptualisation, Implementation, and Feasibility

Cyber-attacks on Industrial Automation and Control Systems (IACS) are rising in numbers and sophistication. Embedded controller devices such as Programmable Logic Controllers (PLCs), which are central to controlling physical processes, must be secured against attacks on confidentiality, integrity and availability. The focus of this paper is to add design-level support for security in IACS applications, especially around inter-PLC communications. We propose an end-to-end solution to develop IACS applications with inherent, and parametric support for security. Built using the IEC 61499 Function Blocks standard, this solution allows us to annotate certain communications as 'secure' during design time. When the application is compiled, these annotations are transformed into a security layer that implements encrypted communication between PLCs. In this paper, we implement a part of this security layer focussed on confidentiality, called Confidentiality Layer for Function Blocks (CL4FB), which provides a range of encryption/decryption and secure key exchange functionalities. We study the impact of using CL4FB in IACS applications with real-time constraints. Through a case study focussing on protection functions in smart-grids, we show that varying levels of confidentiality can be achieved while also meeting hard real-time deadlines.

cs.CR↗

arXiv2021

Exploring software developers' work practices: Task differences, participation, engagement, and speed of task resolution

In seeking to understand the processes enacted during software development, an increasing number of studies have mined software repositories. In particular, studies have endeavored to show how teams resolve software defects. Although much of this work has been useful, we contend that large-scale examinations across the range of activities that are commonly performed, beyond defect-related issues alone, would help us to more fully understand the reasons why defects occur as well as their consequences. More generally, these explorations would reveal how team processes occur during all software development efforts. We thus extend such studies by investigating how software practitioners work while undertaking the range of software tasks that are typically performed. Multiple forms of analyses of a longitudinal case study reveal that software practitioners were mostly involved in fixing defects, and that their engagement covaried depending on the nature of the work they were performing. Furthermore, multiple external factors affected speed of task resolution. Our outcomes suggest that behavioral and intrinsic issues may interact with extrinsic factors becoming significant predictors of the speed of software task resolution.

cs.SE↗

arXiv2021

Evolving a Model for Software Process Context: An Exploratory Study

In the domain of software engineering, our efforts as researchers to advise industry on which software practices might be applied most effectively are limited by our lack of evidence based information about the relationships between context and practice efficacy. In order to accumulate such evidence, a model for context is required. We are in the exploratory stage of evolving a model for context for situated software practices. In this paper, we overview the evolution of our proposed model. Our analysis has exposed a lack of clarity in the meanings of terms reported in the literature. Our base model dimensions are People, Place, Product and Process. Our contributions are a deepening of our understanding of how to scope contextual factors when considering software initiatives and the proposal of an initial theoretical construct for context. Study limitations relate to a possible subjectivity in the analysis and a restricted evaluation base. In the next stage in the research, we will collaborate with academics and practitioners to formally refine the model.

cs.SE↗

arXiv2021

Continuous Transition in Outsourcing: A Case Study

Outsourcing is typically considered to occur in three phases: decision, transition and operation. As outsourcing is now well established the switching of vendors and transitioning from one system to another is common. However, most of the research to date on outsourcing has focused on the decision and operation phases, leaving a gap between theory and practice concerning the transition phase. Transition in outsourcing entails the changing of systems, business processes and/or vendors. If a suitable transition approach is not applied pressures for another transition can immediately build. We present results from a case study carried out on the Novopay Project in which the Ministry of Education in New Zealand changed their vendor from an onshore to a near-shore provider. This project resulted in a sequence of three transitions, with each following a different approach as a direct result of the experiences encountered in the previous transition. In this research we made use of the rich 'data dump' of evidence provided by the Ministry of Education. Our analysis describes how a client organization can become trapped in a continuous transition cycle if a suitable approach is not applied. Transition1 involved the client moving from complete outsourcing to selective insourcing. After realizing that they did not have the capabilities to manage insourcing, Transition2 was initiated. In Transition2 the sourcing approach reverted back to complete outsourcing. When it was realized that the new vendor in Transition2 could not in fact deliver a new service model or support end-users in following new business processes, Transition3 was initiated. In Transition3, the client established an internal company to insource service operations to support end-users. Transition can be a sound business strategy, however, if a flawed sourcing approach is chosen it can result in 'continuous transition'. (Abridged)

cs.SE↗

arXiv2021

Feature Evolution and Reuse -- An Exploratory Study of Eclipse

One of the purported ways to increase productivity and reduce development time is to reuse existing features and modules. If reuse is adopted, logically then, it will have a direct impact on a system's evolution. However, the evidence in the literature is not clear on the extent to which reuse is practiced in real-world projects, nor how it is practiced. In this paper we report the results of an investigation of reuse and evolution of software features in one of the largest open-source ecosystems - Eclipse. Eclipse provides a leading example of how a system can grow dramatically in size and number of features while maintaining its quality. Our results demonstrate the extent of feature reuse and evolution and also patterns of reuse across ten different Eclipse releases (from Europa to Neon).

cs.SE↗

arXiv2021

An empirical study into the relationship between class features and test smells

While a substantial body of prior research has investigated the form and nature of production code, comparatively little attention has examined characteristics of test code, and, in particular, test smells in that code. In this paper, we explore the relationship between production code properties (at the class level) and a set of test smells, in five open source systems. Specifically, we examine whether complexity properties of a production class can be used as predictors of the presence of test smells in the associated unit test. Our results, derived from the analysis of 975 production class-unit test pairs, show that the Cyclomatic Complexity (CC) and Weighted Methods per Class (WMC) of production classes are strong indicators of the presence of smells in their associated unit tests. The Lack of Cohesion of Methods in a production class (LCOM) also appears to be a good indicator of the presence of test smells. Perhaps more importantly, all three metrics appear to be good indicators of particular test smells, especially Eager Test and Duplicated Code. The Depth of the Inheritance Tree (DIT), on the other hand, was not found to be significantly related to the incidence of test smells. The results have important implications for large-scale software development, particularly in a context where organizations are increasingly using, adopting or adapting open source code as part of their development strategy and need to ensure that classes and methods are kept as simple as possible.

cs.SE↗

arXiv2021

Augmenting Text Mining Approaches with Social Network Analysis to Understand the Complex Relationships among Users' Requests: a Case Study of the Android Operating System

Text mining approaches are being used increasingly for business analytics. In particular, such approaches are now central to understanding users' feedback regarding systems delivered via online application distribution platforms such as Google Play. In such settings, large volumes of reviews of potentially numerous apps and systems means that it is infeasible to use manual mechanisms to extract insights and knowledge that could inform product improvement. In this context of identifying software system improvement options, text mining techniques are used to reveal the features that are mentioned most often as being in need of correction (e.g., GPS), and topics that are associated with features perceived as being defective (e.g., inaccuracy of GPS). Other approaches may supplement such techniques to provide further insights for online communities and solution providers. In this work we augment text mining approaches with social network analysis to demonstrate the utility of using multiple techniques. Our outcomes suggest that text mining approaches may indeed be supplemented with other methods to deliver a broader range of insights.

cs.SE↗

arXiv2021

Mobile Data Service Adoption and Use from a Service Supply Perspective: An Empirical Investigation

The paper presents the findings of an empirical study of the views of a selection of mobile data service (MDS) supply chain participants about anticipated MDS customer requirements and expectations, and about the MDS environment. Applying an inductive thematic analysis approach, the study data are first represented as a thematic map; the thematic map is then used to formulate propositions that contribute an MDS supplier perspective to models investigating MDS customer adoption and use.

cs.CY↗

arXiv2021

An Ontological Analysis of a Proposed Theory for Software Development

There is growing acknowledgement within the software engineering community that a theory of software development is needed to integrate the myriad methodologies that are currently popular, some of which are based on opposing perspectives. We have been developing such a theory for a number of years. In this paper, we overview our theory and report on a recent ontological analysis of the theory constructs. We suggest that, once fully developed, this theory, or one similar to it, may be applied to support situated software development, by providing an overarching model within which software initiatives might be categorised and understood. Such understanding would inevitably lead to greater predictability with respect to outcomes.

cs.SE↗

arXiv2021

Adoption and Suitability of Software Development Methods and Practices

In seeking to complement consultants' and tool vendors' reports, there has been an increasing academic focus on understanding the adoption and use of software development methods and practices. We surveyed practitioners working in Brazil, Finland, and New Zealand in a transnational study to contribute to these efforts. Among our findings we observed that most of the 184 practitioners in our sample focused on a small portfolio of projects that were of short duration. In addition, Scrum and Kanban were used most; however, some practitioners also used conventional methods. Coding Standards, Simple Design and Refactoring were used most by practitioners, and these practices were held to be largely suitable for project and process management. Our evidence points to the need to properly understand and support a wide range of software methods.

cs.SE↗