Abstract
The number of mobile applications has increased geometrically nowadays, but how to ensure their quality and conduct adequate and effective testing is still a challenge for developers. On the one hand, the number of mobile apps is increasing, and the update speed is faster and faster. Many small and medium-sized companies can hardly test the app adequately before each release. On the other hand, mobile apps play more and more important roles in people’s life, such as financial payment. For the sake of company security and user privacy, most companies will encrypt key codes in their APP. Even third-party testers cannot get source code, which also leads to many researchers cannot carry out further research and effective testing for these widely used mainstream APPs. Code coverage is an important indicator to guide software testing, which plays a crucial role in ensuring the quality of testing. However, it is an urgent problem to find accurate coverage indictors to evaluate these tests. And when testing those existing widely used mainstream closed-source apps, we find that the existing coarse-grained coverage metrics like method coverage is bad coverage indictors for app testing that can exaggerate or minimize the actual coverage rate, which cannot obtain satisfactory results for the evaluation of test effects. To find a more reliable coverage indictor, this paper demonstrates the correctness of instruction coverage indictor and the inaccurate of method coverage in evaluating the test of closed-source APP from the perspective of probability and statistics. Then we shows how inaccurate the method coverage can be through an empirical evaluation on datasets of closed source APPs and open source APPs respectively. It is further verified that instruction coverage is a more effective evaluation indictor than methods coverage or activity coverage in the test of closed source APP for the first time.
Highlights
According to the 2018 global mobile market report released by Newzoo, the number of smartphone users in the world will reach 3 billion in 2018, with the asia-pacific region accounting for more than half
3) LINE COVERAGE AND INSTRUCTION COVERAGE In many industrial widely used APPs, line coverage is often unavailable because many testers, as third-party testers, are unable to access the source code of these closed-source APPs, which hinders the development of testing tools, because the performance of these tools on complex closed-source APPs cannot be judged without accurate criteria
Assuming that the number of lines triggered during testing is n, that the number of instructions corresponding to each code line is Xi the total number of code lines is N, so the instruction coverage and the corresponding code line coverage is Because the test process is random, assuming that the number of instructions corresponding to each code has a mathematical expectationsλ, the instructions covered in the test can be regarded as random sampling from the entire instruction set
Summary
According to the 2018 global mobile market report released by Newzoo, the number of smartphone users in the world will reach 3 billion in 2018, with the asia-pacific region accounting for more than half. In previous review papers [11]–[13], many automated testing tools can only use method coverage or activity coverage to judge the test adequacy [14]–[18], to generate the test case [19], to compare test suites [20], to maximize fault detection by prioritizing test cases [4], [21] or to use it as a fitness function to guide application exploration in testing [22]–[24] when testing closed-source APP. Because of the significant differences among methods in Java, it is often too arbitrary to judge whether or not the code in the method is covered by detecting whether or not the method is called To solve this problem, this paper designs a comparative experiment to prove that instruction coverage is a more useful test criterion than method coverage to guide the automated test of closed source APP. We will summarize and discuss future work
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
