Combinatorial-based event sequence testing of Android applications

Abstract

Context: Mobile applications are Event Driven Systems (EDS) that take Graphical User Interface (GUI) event sequences as input and respond by changing their state. EDS are often tested with event sequences that exercise system functionality. Much of prior work focuses on testing random event sequences. Combinatorial-based techniques are often used to systematically generate event combinations and may be extended to test behavior that occurs only when events are executed in a particular order. We expand upon the state-of-the-art by using combinatorial-based techniques to systematically test Android applications with automatically generated GUI event sequences.Objective: This paper describes a combinatorial-based technique for automatic construction of Android application test suites. The goal is to minimize redundant execution of events, maximize coverage of event combinations, and increase the likelihood of testing behavior that occurs when GUI events are executed in a particular order.Method: A greedy online algorithm selects and executes GUI events that maximize coverage of n-way event combinations, where n is a specified event combination strength. We compare our combinatorial-based technique to random and frequency-based techniques. We use a two-hour time budget to generate test suites for ten Android applications and empirically evaluate the test suites in terms of code and event coverage.Results: Our 2-way and 3-way combinatorial-based test suites achieve better code and event coverage compared to random and frequency-based test suites in the majority of our subject applications. The results show that there is no significant difference in code or event coverage between 2-way and 3-way combinatorial-based test suites.Conclusion: Given the time budget, the combinatorial-based technique is more effective than random and frequency-based techniques, but its effectiveness may vary depending on specific characteristics of the application under test.