A Hint-Based Technique for System Level Model-Based Test Case Prioritization

Abstract

Test Case Prioritization (TCP) techniques aim at proposing new test case execution orders to favor the achievement of certain testing goal, such as fault detection. Current TCP research focus mainly on code-based regression testing; however in the Model-Based Testing (MBT) context, we still need more investigation. General TCP techniques do not use historical information, since this information is often unavailable. Therefore, techniques use different sources of information to guide prioritization. We propose a novel technique that guides its operation using provided hints, the Hint-Based Adaptive Random Prioritization - HARP. Hints are indications or suggestions provided by developers about error-prone functionalities. As hints may be hard to collect automatically, we also propose an approach of collecting them. To validate our findings, we performed an experiment measuring the effect of introducing hints to HARP. It shows that hints can improve HARP's performance comparing to its baseline. Then, we investigated the ability of developers/managers to provide good hints and used them in a case study. This analysis showed that developers and managers are able to provide useful hints, which improves HARP's fault detection comparing to its baseline. Nonetheless, the provided hints should be consensual among the development team members.