Abstract
While the relation between code coverage measures and fault detection is actively studied, only few works have investigated the correlation between measures of coverage and of reliability. In this work, we introduce a novel approach to measuring code coverage, called the operational coverage, that takes into account how much the program’s entities are exercised so to reflect the profile of usage into the measure of coverage. Operational coverage is proposed as (i) an adequacy criterion, i.e., to assess the thoroughness of a black box test suite derived from the operational profile, and as (ii) a selection criterion, i.e., to select test cases for operational profile-based testing. Our empirical evaluation showed that operational coverage is better correlated than traditional coverage with the probability that the next test case derived according to the user’s profile will not fail. This result suggests that our approach could provide a good stopping rule for operational profile-based testing. With respect to test case selection, our investigations revealed that operational coverage outperformed the traditional one in terms of test suite size and fault detection capability when we look at the average results.
Highlights
Software testing can be based on many criteria (Ammann and Offutt 2016; Bertolino 2007; Zhu et al 1997)
In the case that the derived test suites contain a different number of test cases, for the sake of providing a fair comparison, we reduce the size of the bigger test suite down to the size of the smallest one before computing the metrics described in step 5
We have introduced operational coverage, which measures code coverage taking into account whether and how the entities are relevant with respect to a user’s operational profile
Summary
Software testing can be based on many criteria (Ammann and Offutt 2016; Bertolino 2007; Zhu et al 1997). The preliminary results we obtained seem to sustain our intuition that spectra-based coverage may be taken as both a stopping rule and a selection criterion for operational profile -based testing, better than traditional coverage. Such conclusion can be most usefully applied when the developer can leverage field data collected from profiling usage of the instrumented software (like the scenario described in Orso et al 2003). Related work (Section 7) and Conclusions (Section 8) complete the paper
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
