Invasive Software Testing: Mutating Target Programs to Diversify Test Exploration for High Test Coverage

Abstract

Software testing techniques have advanced significantly over several decades; however, most of current techniques still test a target program as it is, and fail to utilize valuable information of diverse test executions on many variants of the original program in test generation. This paper proposes a new direction for software testing – Invasive Software Testing (IST). IST first generates a set of target program variants m1; :::;mn from an original target program p by applying mutation operations 1; :::; n. Second, given a test suite T, IST executes m1; :::;mn with T and records the test runs which increase test coverage compared to p with T. Based on the recorded information, IST generates guideposts for automated test generation on p toward high test coverage. Finally, IST generates test inputs on p with the guideposts and achieves higher test coverage. We developed DEMINER which implements IST for C programs through software mutation and concolic testing. Further, we showed the effectiveness of DEMINER on three real-world target programs Busybox-ls, Busybox-printf, and GNU-find. The experiment results show that the amount of the improved branch coverage by DEMINER is 24.7% relatively larger than those of the conventional concolic testing techniques on average.