A Systematic Literature Review of Automated Techniques for Functional GUI Testing of Mobile Applications

Abstract

Context. Multiple automated techniques have been proposed and developed for mobile application GUI testing aiming to improve effectiveness, efficiency, and practicality. The effectiveness, efficiency, and practicality are 3 fundamental characteristics which testing techniques are built upon, and need to be continuously improved to deliver useful solutions for researchers and practitioners, and community as a whole. Objective. In this systematic review, we attempt to provide a broad picture of existing mobile testing tools by collating and analysing their conceptual, and also performance characteristics including an estimation of effectiveness, efficiency, and practicality. Method. To achieve our objective, we specify 3 primary, and 14 secondary review questions, and conducted an analysis of 25 primary studies. We first individually analyse each primary study, and next analyse the primary studies as a whole. We developed a review protocol which defines all the details of our systematic review. Results. From effectiveness, we conclude that testing techniques which implement model-checking, symbolic execution, constraint solving, and search-based test generation approach tend to be more effective than those implementing random test generation. From efficiency, we conclude that testing techniques which implement code search-based testing approaches tend to be more efficient than those implementing GUI model-based. From practicality, we conclude that the more effective a testing technique is, the less efficient it will be. Conclusion. For effectiveness, we observe that the existing automated testing techniques are not effective enough, and currently they achieve nearly half of the desired level of effectiveness. For efficiency, we observe that current automated testing techniques are not efficient enough.