Software Operational Profile vs. Test Profile

Abstract

Software Operational Profile (SOP) is a software specification based on how users use the software. This specification corresponds to a quantitative representation of software that identifies the most used software parts. As software reliability depends on the context in which users operate the software, ones use SOP in software reliability engineering. However, there are evidence of a misalignment between the software tested parts and SOP. Therefore, this paper investigates a possible misalignment between SOP and the tested software parts to obtain, based on experimental data, more evidence of this misalignment. We performed an exploratory study composed of three activities to verify: a) whether there are significant variations in how users operate the software; b) whether there is a misalignment between SOP and the tested software parts; c) if failures occur in untested SOP parts in case of misalignment. To perform these verifications, we defined the term test profile and presented it in this paper. We instrumented three software to collect data from them while the users were operating this software. Posteriorly, we analyzed these collected data in an attempt to reach the goals of this paper. To evaluate the originality of this research, we performed a Literature Systematic Review (SLR) and presented its conclusions. The obtained results evidence that there are significant variations in how users operate the software and also that there was a misalignment between SOP and the tested software parts when we evaluated the three software mentioned above. There were also indications of the occurrence of failures in the untested SOP parts. These results indicate that SOP becomes relevant not only to software reliability engineering but also to contribute to testing activities, regardless of the adopted strategy.