Experimental Evaluation of Redundancy in Android Mutation Testing

Abstract

Because of the widespread usage of Android devices, the Android ecosystem has the highest numbers of users, developers, and app downloads. Researchers find that many Android apps are not sufficiently tested, which may lead to crashes, incorrect behaviors, and security vulnerabilities. Mutation testing is a syntax-based software testing technique that is very effective at designing high-quality tests and evaluating pre-existing tests. Our prior research designed and implemented Android mutation testing technique, and then used experiments to assess its strength. However, the high computational cost of Android mutation testing possibly limits its industrial application. This paper presents an experimental evaluation that investigates redundant mutation operators in Android mutation analysis. While maintaining the test quality, our goal is to reduce the cost by excluding redundant mutation operators or improving their design and implementation. In our evaluation, we first generate mutants and design mutation-adequate tests for each mutation operator. Then, we compute redundancy scores for each pair of mutation operators. Our evaluation results indicate that three operators (AODU, AOIU, and LOI) are redundant in Android mutation analysis. Other three operators (FOB, TVD, and ORL) are very hard to kill. One operator (MDL) needs improvement in its design to eliminate trivial mutants. We also identity subsumption relationships among operators (BWS subsumes BWD, ODL subsumes CDL, COD, and VDL).