arXiv subjects

Bestoun S. Ahmed

Publications and source records attributed to Bestoun S. Ahmed.

At least 55 records · Page 3Linked to original sources

arXiv2020

Testing the Usability and Accessibility of Smart TV Applications Using an Automated Model-based Approach

As the popularity of Smart Televisions (TVs) and interactive Smart TV applications (apps) has recently grown, the usability of these apps has become an important quality characteristic. Previous studies examined Smart TV apps from a usability perspective. However, these methods are mainly manual, and the potential of automated model-based testing methods for usability testing purposes has not yet been fully explored. In this paper, we propose an approach to test the usability of Smart TV apps based on the automated generation of a Smart TV user interaction model from an existing app by a specialized automated crawler. By means of this model, defined user tasks in the Smart TV app can be evaluated automatically in terms of their feasibility and estimated user effort, which reflects the usability of the analyzed app. This analysis can be applied in the context of regular users and users with various specific needs. The findings from this model-based automated analysis approach can be used to optimize the user interface of a Smart TV app to increase its usability, accessibility, and quality.

cs.SE↗

arXiv2020

An Evaluation of Monte Carlo-Based Hyper-Heuristic for Interaction Testing of Industrial Embedded Software Applications

Hyper-heuristic is a new methodology for the adaptive hybridization of meta-heuristic algorithms to derive a general algorithm for solving optimization problems. This work focuses on the selection type of hyper-heuristic, called the Exponential Monte Carlo with Counter (EMCQ). Current implementations rely on the memory-less selection that can be counterproductive as the selected search operator may not (historically) be the best performing operator for the current search instance. Addressing this issue, we propose to integrate the memory into EMCQ for combinatorial t-wise test suite generation using reinforcement learning based on the Q-learning mechanism, called Q-EMCQ. The limited application of combinatorial test generation on industrial programs can impact the use of such techniques as Q-EMCQ. Thus, there is a need to evaluate this kind of approach against relevant industrial software, with a purpose to show the degree of interaction required to cover the code as well as finding faults. We applied Q-EMCQ on 37 real-world industrial programs written in Function Block Diagram (FBD) language, which is used for developing a train control management system at Bombardier Transportation Sweden AB. The results of this study show that Q-EMCQ is an efficient technique for test case generation. Additionally, unlike the t-wise test suite generation, which deals with the minimization problem, we have also subjected Q-EMCQ to a maximization problem involving the general module clustering to demonstrate the effectiveness of our approach.

cs.SE↗

arXiv2020

Avocado: Open-Source Flexible Constrained Interaction Testing for Practical Application

This paper presents the outcome of a research collaboration between academia and industry to implement and utilize the capabilities of constrained interaction testing for an open-source tool for industrial-scale application. The project helps promote flexibility in generating constrained interaction test suites, executing them, and setting up a test oracle to report them--all within the same tool called Avocado. Avocado employs a constraint solver with computational algorithms to generate constrained interaction test suites. The environment of the application under test can be set up to execute the generated test suite with minimum effort. A test oracle can be set up by the tool to report the status and the results of the executed test cases. Avocado represents a comprehensive and flexible solution for conducting combinatorial interaction testing (CIT) and constrained CIT on an industrial application. In this paper, we present the structure of the tool and our method of implementing the algorithms in detail.

cs.SE↗

arXiv2020

An Automated Testing Framework For Smart TV apps Based on Model Separation

Smart TV application (app) is a new technological software app that can deal with smart TV devices to add more functionality and features. Despite its importance nowadays, far too little attention has been paid to present a systematic approach to test this kind of app so far. In this paper, we present a systematic model-based testing approach for smart TV app. We used our new notion of model separation to use sub-models based on the user preference instead of the exhaustive testing to generate the test cases. Based on the constructed model, we generated a set of test cases to assess the selected paths to the chosen destination in the app. We also defined new mutation operators for smart TV app to assess our testing approach. The evaluation results showed that our approach can generate more comprehensive models of smart TV apps with less time as compared to manual exploratory testing. The results also showed that our approach can generate effective test cases in term of fault detection.

cs.SE↗

arXiv2020

Open-source Defect Injection Benchmark Testbed for the Evaluation of Testing

A natural method to evaluate the effectiveness of a testing technique is to measure the defect detection rate when applying the created test cases. Here, real or artificial software defects can be injected into the source code of software. For a more extensive evaluation, the injection of artificial defects is usually needed and can be performed via mutation testing using code mutation operators. However, to simulate complex defects arising from a misunderstanding of design specifications, mutation testing might reach its limit in some cases. In this paper, we present an open-source benchmark testbed application that employs a complement method of artificial defect injection. The application is compiled after artificial defects are injected into its source code from predefined building blocks. The majority of the functions and user interface elements are covered by creating front-end-based automated test cases that can be used in experiments.

cs.SE↗

arXiv2019

Generation and Application of Constrained Interaction Test Suites Using Base Forbidden Tuples With Mixed Neighborhood Tabu Search

Nowadays, ensuring the quality becomes challenging for most modern software systems when constraints are given for the combinations of configurations. Combinatorial interaction strategies can systematically reduce the number of test cases to construct a minimal test suite without affecting the effectiveness of the tests. This paper presents a new efficient search-based strategy to generate constrained interaction test suites to cover all possible combinations. The paper also shows a new application of constrained interaction testing in software fault searches. The proposed strategy initially generates the set of all possible t-tuple combinations; then, it filters out the set by removing the forbidden t-tuples using the base forbidden tuple (BFT) approach. The strategy also utilizes a mixed neighborhood tabu search (TS) to construct optimal or near-optimal constrained test suites. The efficiency of the proposed method is evaluated through a comparison against two well-known state-of-the-art tools. The evaluation consists of three sets of experiments for 35 standard benchmarks. Additionally, the effectiveness and quality of the results are assessed using a real-world case study. Experimental results show that the proposed strategy outperforms one of the competitive strategies, ACTS, for approximately 83% of the benchmarks and achieves similar results to CASA for 65% of the benchmarks when the interaction strength is 2. For an interaction strength of 3, the proposed method outperforms other competitive strategies for approximately 60% and 42% of the benchmarks. The proposed strategy can also generate constrained interaction test suites for an interaction strength of 4, which is not possible for many strategies. Real-world case study shows that the generated test suites can effectively detect injected faults using mutation testing.

cs.SE↗

arXiv2019

Code-Aware Combinatorial Interaction Testing

Combinatorial interaction testing (CIT) is a useful testing technique to address the interaction of input parameters in software systems. In many applications, the technique has been used as a systematic sampling technique to sample the enormous possibilities of test cases. In the last decade, most of the research activities focused on the generation of CIT test suites as it is a computationally complex problem. Although promising, less effort has been paid for the application of CIT. In general, to apply the CIT, practitioners must identify the input parameters for the Software-under-test (SUT), feed these parameters to the CIT tool to generate the test suite, and then run those tests on the application with some pass and fail criteria for verification. Using this approach, CIT is used as a black-box testing technique without knowing the effect of the internal code. Although useful, practically, not all the parameters having the same impact on the SUT. This paper introduces a different approach to use the CIT as a gray-box testing technique by considering the internal code structure of the SUT to know the impact of each input parameter and thus use this impact in the test generation stage. We applied our approach to five reliable case studies. The results showed that this approach would help to detect new faults as compared to the equal impact parameter approach.

cs.SE↗

arXiv2019

Fuzzy adaptive teaching learning-based optimization strategy for the problem of generating mixed strength t-way test suites

The teaching learning-based optimization (TLBO) algorithm has shown competitive performance in solving numerous real-world optimization problems. Nevertheless, this algorithm requires better control for exploitation and exploration to prevent premature convergence (i.e., trapped in local optima), as well as enhance solution diversity. Thus, this paper proposes a new TLBO variant based on Mamdani fuzzy inference system, called ATLBO, to permit adaptive selection of its global and local search operations. In order to assess its performances, we adopt ATLBO for the mixed strength t-way test generation problem. Experimental results reveal that ATLBO exhibits competitive performances against the original TLBO and other meta-heuristic counterparts.

cs.NE↗

arXiv2019

Generating Pairwise Combinatorial Interaction Test Suites Using Single Objective Dragonfly Optimisation Algorithm

Combinatorial interaction testing has been addressed as an effective software testing technique recently. It shows its ability to reduce the number of test cases that have to be considered for software-under-test by taking the combinations of parameters as an interaction of input. This combination could be considered as input-configuration of different software families. Pairwise combinatorial test suite takes the interaction of two input parameters into consideration instead of many parameter interactions. Evidence showed that this test suite could detect most of the faults in the software-under-test as compared to higher interactions. This paper presents a new technique to generate pairwise combinatorial test suites. Also, Dragon Fly (DF), a new swarm intelligent optimization algorithm, is assessed. The design and adaptation of the algorithm are addresses in the paper in detail. The algorithm is evaluated extensively through different experiments and benchmarks. The evaluation shows the efficiency of the proposed technique for test suite generation and the usefulness of DF optimization algorithm for future investigations.

cs.SE↗

arXiv2019

A New Approach to Speed up Combinatorial Search Strategies Using Stack and Hash Table

Owing to the significance of combinatorial search strategies both for academia and industry, the introduction of new techniques is a fast growing research field these days. These strategies have really taken different forms ranging from simple to complex strategies in order to solve all forms of combinatorial problems. Nonetheless, despite the kind of problem these approaches solve, they are prone to heavy computation with the number of combinations and growing search space dimensions. This paper presents a new approach to speed up the generation and search processes using a combination of stack and hash table data structures. This approach could be put to practice for the combinatorial approaches to speed up the generation of combinations and search process in the search space. Furthermore, this new approach proved its performance in diverse stages better than other known strategies.

cs.SE↗

arXiv2019

EvoCreeper: Automated Black-Box Model Generation for Smart TV Applications

Smart TVs are coming to dominate the television market. This accompanied by an increase in the use of smart TV applications (apps). Due to the increasing demand, developers need modeling techniques to analyze these apps and assess their comprehensiveness, completeness, and quality. In this paper, we present an automated strategy for generating models of smart TV apps based on black-box reverse engineering. The strategy can be used to cumulatively construct a model for a given app by exploring the user interface in a manner consistent with the use of a remote control device and extracting the runtime information. The strategy is based on capturing the states of the user interface to create a model during runtime without any knowledge of the internal structure of the app. We have implemented our strategy in a tool called EvoCreeper. The evaluation results show that our strategy can automatically generate unique states and a comprehensive model that represents the real user interactions with an app using a remote control device. The models thus generated can be used to assess the quality and completeness of smart TV apps in various contexts, such as the control of other consumer electronics in smart houses.

cs.SE↗

arXiv2019

Achievement of Minimized Combinatorial Test Suite for Configuration-Aware Software Functional Testing Using the Cuckoo Search Algorithm

Context: Software has become an innovative solution nowadays for many applications and methods in science and engineering. Ensuring the quality and correctness of software is challenging because each program has different configurations and input domains. To ensure the quality of software, all possible configurations and input combinations need to be evaluated against their expected outputs. However, this exhaustive test is impractical because of time and resource constraints due to the large domain of input and configurations. Thus, different sampling techniques have been used to sample these input domains and configurations. Objective: Combinatorial testing can be used to effectively detect faults in software-under-test. This technique uses combinatorial optimization concepts to systematically minimize the number of test cases by considering the combinations of inputs. This paper proposes a new strategy to generate combinatorial test suite by using cuckoo search concepts. Method: Cuckoo Search is used in the design and implementation of a strategy to construct optimized combinatorial sets. The strategy consists of different algorithms for construction. These algorithms are combined to serve the Cuckoo Search. Results: The efficiency and performance of the new technique were proven through different experiment sets. The effectiveness of the strategy is assessed by applying the generated test suites on a real-world case study for the purpose of functional testing. Conclusion: Results show that the generated test suites can detect faults effectively. In addition, the strategy also opens a new direction for the application of Cuckoo Search in the context of software engineering.

cs.SE↗

arXiv2019

Prioritized Process Test: An Alternative to Current Process Testing Strategies

Testing processes and workflows in information and Internet of Things systems is a major part of the typical software testing effort. Consistent and efficient path-based test cases are desired to support these tests. Because certain parts of software system workflows have a higher business priority than others, this fact has to be involved in the generation of test cases. In this paper, we propose a Prioritized Process Test (PPT), which is a model-based test case generation algorithm that represents an alternative to currently established algorithms that use directed graphs and test requirements to model the system under test. The PPT accepts a directed multigraph as a model to express priorities, and edge weights are used instead of test requirements. To determine the test-coverage level of test cases, a test-depth-level concept is used. We compared the presented PPT with five alternatives (i.e., the Process Cycle Test, a naive reduction of test set created by the Process Cycle Test, Brute Force algorithm, Set-covering Based Solution and Matching-based Prefix Graph Solution) for edge coverage and edge-pair coverage. To assess the optimality of the path-based test cases produced by these strategies, we used fourteen metrics based on the properties of these test cases and 59 models that were created for three real-world systems. For all edge coverage, the PPT produced more optimal test cases than the alternatives in terms of the majority of the metrics. For edge-pair coverage, the PPT strategy yielded similar results to those of the alternatives. Thus, the PPT strategy is an applicable alternative, as it reflects both the required test coverage level and the business priority in parallel.

cs.SE↗

arXiv2019

Testing of Smart TV Applications: Key Ingredients, Challenges and Proposed Solutions

Smart TV applications are software applications that have been designed to run on smart TVs which are televisions with integrated Internet features. Nowadays, the smart TVs are going to dominate the television market, and the number of connected TVs is growing exponentially. This growth is accompanied by the increase of consumers and the use of smart TV applications that drive these devices. Due to the increasing demand for smart TV applications especially with the rise of the Internet of Things (IoT) services, it is essential to building an application with a certain level of quality. Despite the analogy between the smart TV and mobile apps, testing smart TV applications is different in many aspects due to the different nature of user interaction and development environment. To develop the field and formulate the concepts of smart TV application testing, this paper aims to provide the essential ingredients, solutions, answers to the most critical questions, and open problems. In addition, we offer initial results and proof of concepts for a creeper algorithm to detect essential views of the applications. This paper serves as an effort to report the key ingredients and challenges of the smart TV application testing systematically to the research community.

cs.SE↗

arXiv2019

Towards an Automated Unified Framework to Run Applications for Combinatorial Interaction Testing

Combinatorial interaction testing (CIT) is a well-known technique, but the industrial experience is needed to determine its effectiveness in different application domains. We present a case study introducing a unified framework for generating, executing and verifying CIT test suites, based on the open-source Avocado test framework. In addition, we present a new industrial case study to demonstrate the effectiveness of the framework. This evaluation showed that the new framework can generate, execute, and verify effective combinatorial interaction test suites for detecting configuration failures (invalid configurations) in a virtualization system.

cs.SE↗

arXiv2019

Software Module Clustering based on the Fuzzy Adaptive Teaching Learning based Optimization Algorithm

Although showing competitive performances in many real-world optimization problems, Teaching Learning based Optimization Algorithm (TLBO) has been criticized for having poor control on exploration and exploitation. Addressing these issues, a new variant of TLBO called Adaptive Fuzzy Teaching Learning based Optimization (ATLBO) has been developed in the literature. This paper describes the adoption of Fuzzy Adaptive Fuzzy Teaching Learning based Optimization (ATLBO) for software module clustering problem. Comparative studies with the original Teaching Learning based Optimization (TLBO) and other Fuzzy TLBO variant demonstrate that ATLBO gives superior performance owing to its adaptive selection of search operators based on the need of the current search.

cs.SE↗

arXiv2019

Dynamic Solution Probability Acceptance within the Flower Pollination Algorithm for t-way Test Suite Generation

Flower Pollination Algorithm (FPA) is the new breed of metaheuristic for the general optimization problem. In this paper, an improved algorithm based on Flower Pollination Algorithm (FPA), called imFPA, has been proposed. In imFPA, the static selection probability is replaced by the dynamic solution selection probability in order to enhance the diversification and intensification of the overall search process. Experimental adoptions on combinatorial t- way test suite generation problem (where t indicates the interaction strength) show that imFPA produces very competitive results as compared to existing strategies.

cs.SE↗

arXiv2019

AMOGA: A Static-Dynamic Model Generation Strategy for Mobile Apps Testing

In the past few years, mobile devices have been increasingly replacing traditional computers as their capabilities such as CPU computation, memory, RAM size, and many more, are being enhanced almost to the level of conventional computers. These capabilities are being exploited by mobile apps developers to produce apps that offer more functionalities and optimized performance. To ensure acceptable quality and to meet their specifications (e.g., design), mobile apps need to be tested thoroughly. As the testing process is often tedious, test automation can be the key to alleviating such laborious activities. In the context of the Android-based mobile apps, researchers and practitioners have proposed many approaches to automate the testing process mainly on the creation of the test suite. Although useful, most existing approaches rely on reverse engineering a model of the application under test for test case creation. Often, such approaches exhibit a lack of comprehensiveness as the application model does not capture the dynamic behavior of the applications extensively due to the incompleteness of reverse engineering approaches. To address this issue, this paper proposes AMOGA, a strategy that uses a hybrid, static-dynamic approach for generating user interface model from mobile apps for model-based testing. AMOGA implements a novel crawling technique that uses the event list of UI element associated with each event to dynamically exercise the events ordering at the run-time to explore the applications' behavior. Results of the experimental assessment showed that AMOGA represents an alternative approach for model-based testing of mobile apps by generating comprehensive models to improve the coverage of the applications.

cs.SE↗