arXiv subjects

Bestoun S. Ahmed

Publications and source records attributed to Bestoun S. Ahmed.

At least 37 records · Page 2Linked to original sources

arXiv2023

Automated and Systematic Digital Twins Testing for Industrial Processes

Digital twins (DT) of industrial processes have become increasingly important. They aim to digitally represent the physical world to help evaluate, optimize, and predict physical processes and behaviors. Therefore, DT is a vital tool to improve production automation through digitalization and becomes more sophisticated due to rapidly evolving simulation and modeling capabilities, integration of IoT sensors with DT, and high-capacity cloud/edge computing infrastructure. However, the fidelity and reliability of DT software are essential to represent the physical world. This paper shows an automated and systematic test architecture for DT that correlates DT states with real-time sensor data from a production line in the forging industry. Our evaluation shows that the architecture can significantly accelerate the automatic DT testing process and improve its reliability. A systematic online DT testing method can significantly detect the performance shift and continuously improve the DT's fidelity. The snapshot creation methodology and testing agent architecture can be an inspiration and can be generally applicable to other industrial processes that use DT to generalize their automated testing.

cs.SE↗

arXiv2022

Quality Assurance in MLOps Setting: An Industrial Perspective

Today, machine learning (ML) is widely used in industry to provide the core functionality of production systems. However, it is practically always used in production systems as part of a larger end-to-end software system that is made up of several other components in addition to the ML model. Due to production demand and time constraints, automated software engineering practices are highly applicable. The increased use of automated ML software engineering practices in industries such as manufacturing and utilities requires an automated Quality Assurance (QA) approach as an integral part of ML software. Here, QA helps reduce risk by offering an objective perspective on the software task. Although conventional software engineering has automated tools for QA data analysis for data-driven ML, the use of QA practices for ML in operation (MLOps) is lacking. This paper examines the QA challenges that arise in industrial MLOps and conceptualizes modular strategies to deal with data integrity and Data Quality (DQ). The paper is accompanied by real industrial use-cases from industrial partners. The paper also presents several challenges that may serve as a basis for future studies.

cs.SE↗

arXiv2022

A Drift Handling Approach for Self-Adaptive ML Software in Scalable Industrial Processes

Most industrial processes in real-world manufacturing applications are characterized by the scalability property, which requires an automated strategy to self-adapt machine learning (ML) software systems to the new conditions. In this paper, we investigate an Electroslag Remelting (ESR) use case process from the Uddeholms AB steel company. The use case involves predicting the minimum pressure value for a vacuum pumping event. Taking into account the long time required to collect new records and efficiently integrate the new machines with the built ML software system. Additionally, to accommodate the changes and satisfy the non-functional requirement of the software system, namely adaptability, we propose an automated and adaptive approach based on a drift handling technique called importance weighting. The aim is to address the problem of adding a new furnace to production and enable the adaptability attribute of the ML software. The overall results demonstrate the improvements in ML software performance achieved by implementing the proposed approach over the classical non-adaptive approach.

cs.SE↗

arXiv2022

Testing of Machine Learning Models with Limited Samples: An Industrial Vacuum Pumping Application

There is often a scarcity of training data for machine learning (ML) classification and regression models in industrial production, especially for time-consuming or sparsely run manufacturing processes. A majority of the limited ground-truth data is used for training, while a handful of samples are left for testing. Here, the number of test samples is inadequate to properly evaluate the robustness of the ML models under test for classification and regression. Furthermore, the output of these ML models may be inaccurate or even fail if the input data differ from the expected. This is the case for ML models used in the Electroslag Remelting (ESR) process in the refined steel industry to predict the pressure in a vacuum chamber. A vacuum pumping event that occurs once a workday generates a few hundred samples in a year of pumping for training and testing. In the absence of adequate training and test samples, this paper first presents a method to generate a fresh set of augmented samples based on vacuum pumping principles. Based on the generated augmented samples, three test scenarios and one test oracle are presented to assess the robustness of an ML model used for production on an industrial scale. Experiments are conducted with real industrial production data obtained from Uddeholms AB steel company. The evaluations indicate that Ensemble and Neural Network are the most robust when trained on augmented data using the proposed testing strategy. The evaluation also demonstrates the proposed method's effectiveness in checking and improving ML algorithms' robustness in such situations. The work improves software testing's state-of-the-art robustness testing in similar settings. Finally, the paper presents an MLOps implementation of the proposed approach for real-time ML model prediction and action on the edge node and automated continuous delivery of ML software from the cloud.

cs.SE↗

arXiv2022

Novel Strategy Generating Variable-length State Machine Test Paths

Finite State Machine is a popular modeling notation for various systems, especially software and electronic. Test paths can be automatically generated from the system model to test such systems using a suitable algorithm. This paper presents a strategy that generates test paths and allows to start and end test paths only in defined states of the finite state machine. The strategy also simultaneously supports generating test paths only of length in a given range. For this purpose, alternative system models, test coverage criteria, and a set of algorithms are developed. The strategy is compared with the best alternative based on the reduction of the test set generated by the established N-switch coverage approach on a mix of 171 industrial and artificially generated problem instances. The proposed strategy outperforms the compared variant in a smaller number of test path steps. The extent varies with the used test coverage criterion and preferred test path length range from none to two and half fold difference. Moreover, the proposed technique detected up to 30% more simple artificial defects inserted into experimental SUT models per one test step than the compared alternative technique. The proposed strategy is well applicable in situations where a possible test path starts and ends in a state machine needs to be reflected and, concurrently, the length of the test paths has to be in a defined range.

cs.SE↗

arXiv2022

IoT Anomaly Detection Methods and Applications: A Survey

Ongoing research on anomaly detection for the Internet of Things (IoT) is a rapidly expanding field. This growth necessitates an examination of application trends and current gaps. The vast majority of those publications are in areas such as network and infrastructure security, sensor monitoring, smart home, and smart city applications and are extending into even more sectors. Recent advancements in the field have increased the necessity to study the many IoT anomaly detection applications. This paper begins with a summary of the detection methods and applications, accompanied by a discussion of the categorization of IoT anomaly detection algorithms. We then discuss the current publications to identify distinct application domains, examining papers chosen based on our search criteria. The survey considers 64 papers among recent publications published between January 2019 and July 2021. In recent publications, we observed a shortage of IoT anomaly detection methodologies, for example, when dealing with the integration of systems with various sensors, data and concept drifts, and data augmentation where there is a shortage of Ground Truth data. Finally, we discuss the present such challenges and offer new perspectives where further research is required.

cs.SE↗

arXiv2022

Prioritized Variable-length Test Cases Generation for Finite State Machines

Model-based Testing (MBT) is an effective approach for testing when parts of a system-under-test have the characteristics of a finite state machine (FSM). Despite various strategies in the literature on this topic, little work exists to handle special testing situations. More specifically, when concurrently: (1) the test paths can start and end only in defined states of the FSM, (2) a prioritization mechanism that requires only defined states and transitions of the FSM to be visited by test cases is required, and (3) the test paths must be in a given length range, not necessarily of explicit uniform length. This paper presents a test generation strategy that satisfies all these requirements. A concurrent combination of these requirements is highly practical for real industrial testing. Six variants of possible algorithms to implement this strategy are described. Using a mixture of 180 problem instances from real automotive and defense projects and artificially generated FSMs, all variants are compared with a baseline strategy based on an established N-switch coverage concept modification. Various properties of the generated test paths and their potential to activate fictional defects defined in FSMs are evaluated. The presented strategy outperforms the baseline in most problem configurations. Out of the six analyzed variants, three give the best results even though a universal best performer is hard to identify. Depending on the application of the FSM, the strategy and evaluation presented in this paper are applicable both in testing functional and non-functional software requirements.

cs.SE↗

arXiv2022

From Concept Drift to Model Degradation: An Overview on Performance-Aware Drift Detectors

The dynamicity of real-world systems poses a significant challenge to deployed predictive machine learning (ML) models. Changes in the system on which the ML model has been trained may lead to performance degradation during the system's life cycle. Recent advances that study non-stationary environments have mainly focused on identifying and addressing such changes caused by a phenomenon called concept drift. Different terms have been used in the literature to refer to the same type of concept drift and the same term for various types. This lack of unified terminology is set out to create confusion on distinguishing between different concept drift variants. In this paper, we start by grouping concept drift types by their mathematical definitions and survey the different terms used in the literature to build a consolidated taxonomy of the field. We also review and classify performance-based concept drift detection methods proposed in the last decade. These methods utilize the predictive model's performance degradation to signal substantial changes in the systems. The classification is outlined in a hierarchical diagram to provide an orderly navigation between the methods. We present a comprehensive analysis of the main attributes and strategies for tracking and evaluating the model's performance in the predictive system. The paper concludes by discussing open research challenges and possible research directions.

cs.LG↗

arXiv2022

Overview of Test Coverage Criteria for Test Case Generation from Finite State Machines Modelled as Directed Graphs

Test Coverage criteria are an essential concept for test engineers when generating the test cases from a System Under Test model. They are routinely used in test case generation for user interfaces, middleware, and back-end system parts for software, electronics, or Internet of Things (IoT) systems. Test Coverage criteria define the number of actions or combinations by which a system is tested, informally determining a potential "strength" of a test set. As no previous study summarized all commonly used test coverage criteria for Finite State Machines and comprehensively discussed them regarding their subsumption, equivalence, or non-comparability, this paper provides this overview. In this study, 14 most common test coverage criteria and seven of their synonyms for Finite State Machines defined via a directed graph are summarized and compared. The results give researchers and industry testing engineers a helpful overview when setting a software-based or IoT system test strategy.

cs.SE↗

arXiv2022

Using Deep Reinforcement Learning for Zero Defect Smart Forging

Defects during production may lead to material waste, which is a significant challenge for many companies as it reduces revenue and negatively impacts sustainability and the environment. An essential reason for material waste is a low degree of automation, especially in industries that currently have a low degree of digitalization, such as steel forging. Those industries typically rely on heavy and old machinery such as large induction ovens that are mostly controlled manually or using well-known recipes created by experts. However, standard recipes may fail when unforeseen events happen, such as an unplanned stop in production, which may lead to overheating and thus material degradation during the forging process. In this paper, we develop a digital twin-based optimization strategy for the heating process for a forging line to automate the development of an optimal control policy that adjusts the power for the heating coils in an induction oven based on temperature data observed from pyrometers. We design a digital twin-based deep reinforcement learning (DTRL) framework and train two different deep reinforcement learning (DRL) models for the heating phase using a digital twin of the forging line. The twin is based on a simulator that contains a heating transfer and movement model, which is used as an environment for the DRL training. Our evaluation shows that both models significantly reduce the temperature unevenness and can help to automate the traditional heating process.

cs.LG↗

arXiv2021

Hybrid Henry Gas Solubility Optimization Algorithm with Dynamic Cluster-to-Algorithm Mapping for Search-based Software Engineering Problems

This paper discusses a new variant of the Henry Gas Solubility Optimization (HGSO) Algorithm, called Hybrid HGSO (HHGSO). Unlike its predecessor, HHGSO allows multiple clusters serving different individual meta-heuristic algorithms (i.e., with its own defined parameters and local best) to coexist within the same population. Exploiting the dynamic cluster-to-algorithm mapping via penalized and reward model with adaptive switching factor, HHGSO offers a novel approach for meta-heuristic hybridization consisting of Jaya Algorithm, Sooty Tern Optimization Algorithm, Butterfly Optimization Algorithm, and Owl Search Algorithm, respectively. The acquired results from the selected two case studies (i.e., involving team formation problem and combinatorial test suite generation) indicate that the hybridization has notably improved the performance of HGSO and gives superior performance against other competing meta-heuristic and hyper-heuristic algorithms.

cs.AI↗

arXiv2021

PatrIoT: IoT Automated Interoperability and Integration Testing Framework

With the rapid growth of the contemporary Internet of Things (IoT) market, the established systems raise a number of concerns regarding the reliability and the potential presence of critical integration defects. In this paper, we present a PatrIoT framework that aims to provide flexible support to construct an effective IoT system testbed to implement automated interoperability and integration testing. The framework allows scaling from a pure physical testbed to a simulated environment using a number of predefined modules and elements to simulate an IoT device or part of the tested infrastructure. PatrIoT also contains a set of reference example testbeds and several sets of example automated tests for a smart street use case.

cs.SE↗

arXiv2021

Review of Specific Features and Challenges in the Current Internet of Things Systems Impacting their Security and Reliability

The current development of the Internet of Things (IoT) technology poses significant challenges to researchers and industry practitioners. Among these challenges, security and reliability particularly deserve attention. In this paper, we provide a consolidated analysis of the root causes of these challenges, their relations, and their possible impacts on IoT systems' general quality characteristics. Further understanding of these challenges is useful for IoT quality engineers when defining testing strategies for their systems and researchers to consider when discussing possible research directions. In this study, twenty specific features of current IoT systems are discussed, divided into five main categories: (1) Economic, managerial and organisational aspects, (2) Infrastructural challenges, (3) Security and privacy challenges, (4) Complexity challenges and (5) Interoperability problems.

cs.CR↗

arXiv2020

Software Module Clustering: An In-Depth Literature Analysis

Software module clustering is an unsupervised learning method used to cluster software entities (e.g., classes, modules, or files) with similar features. The obtained clusters may be used to study, analyze, and understand the software entities' structure and behavior. Implementing software module clustering with optimal results is challenging. Accordingly, researchers have addressed many aspects of software module clustering in the past decade. Thus, it is essential to present the research evidence that has been published in this area. In this study, 143 research papers from well-known literature databases that examined software module clustering were reviewed to extract useful data. The obtained data were then used to answer several research questions regarding state-of-the-art clustering approaches, applications of clustering in software engineering, clustering processes, clustering algorithms, and evaluation methods. Several research gaps and challenges in software module clustering are discussed in this paper to provide a useful reference for researchers in this field.

cs.SE↗

arXiv2020

Quality and Reliability Metrics for IoT Systems: A Consolidated View

Quality and reliability metrics play an important role in the evaluation of the state of a system during the development and testing phases, and serve as tools to optimize the testing process or to define the exit or acceptance criteria of the system. This study provides a consolidated view on the available quality and reliability metrics applicable to Internet of Things (IoT) systems, as no comprehensive study has provided such a view specific to these systems. The quality and reliability metrics categorized and discussed in this paper are divided into three categories: metrics assessing the quality of an IoT system or service, metrics for assessing the effectiveness of the testing process, and metrics that can be universally applied in both cases. In the discussion, recommendations of proper usage of discussed metrics in a testing process are then given.

cs.SE↗

arXiv2020

Code Coverage Aware Test Generation Using Constraint Solver

Code coverage has been used in the software testing context mostly as a metric to assess a generated test suite's quality. Recently, code coverage analysis is used as a white-box testing technique for test optimization. Most of the research activities focus on using code coverage for test prioritization and selection within automated testing strategies. Less effort has been paid in the literature to use code coverage for test generation. This paper introduces a new Code Coverage-based Test Case Generation (CCTG) concept that changes the current practices by utilizing the code coverage analysis in the test generation process. CCTG uses the code coverage data to calculate the input parameters' impact for a constraint solver to automate the generation of effective test suites. We applied this approach to a few real-world case studies. The results showed that the new test generation approach could generate effective test cases and detect new faults.

cs.SE↗

arXiv2020

Interoperability and Integration Testing Methods for IoT Systems: a Systematic Mapping Study

The recent active development of Internet of Things (IoT) solutions in various domains has led to an increased demand for security, safety, and reliability of these systems. Security and data privacy are currently the most frequently discussed topics; however, other reliability aspects also need to be focused on to maintain the smooth and safe operation of IoT systems. Until now, there has been no systematic mapping study dedicated to the topic of interoperability and integration testing of IoT systems specifically; therefore, we present such an overview in this study. We analyze 803 papers from four major primary databases and perform detailed assessment and quality check to find 115 relevant papers. In addition, recently published testing techniques and approaches are analyzed and classified; the challenges and limitations in the field is also identified and discussed. Research trends related to publication time, active researchers, and publication media are presented in this study. The results suggest that studies mainly focus only on general testing methods, which can be applied to integration and interoperability testing of IoT systems; thus, there are research opportunities to develop additional testing methods focused specifically on IoT systems, so that they are more effective in the IoT context.

cs.SE↗

arXiv2020

Alternative Effort-optimal Model-based Strategy for State Machine Testing of IoT Systems

To effectively test parts of the Internet of Things (IoT) systems with a state machine character, Model-based Testing (MBT) approach can be taken. In MBT, a system model is created, and test cases are generated automatically from the model, and a number of current strategies exist. In this paper, we propose a novel alternative strategy that concurrently allows us to flexibly adjust the preferred length of the generated test cases, as well as to mark the states, in which the test case can start and end. Compared with an intuitive N-switch coverage-based strategy that aims at the same goals, our proposal generates a lower number of shorter test cases with fewer test step duplications.

cs.SE↗