Abstract
Modern software development increasingly relies on software testing for an ever more frequent delivery of high quality software. This puts high demands on the quality of the central artifacts in software testing, test suites and test cases. We aim to develop a comprehensive model for capturing the dimensions of test case/suite quality, which are relevant for a variety of perspectives. We have carried out a systematic literature review to identify and analyze existing secondary studies on quality aspects of software testing artifacts. We identified 49 relevant secondary studies. Of these 49 studies, less than half did some form of quality appraisal of the included primary studies and only 3 took into account the quality of the primary study when synthesizing the results. We present an aggregation of the context dimensions and factors that can be used to characterize the environment in which the test case/suite quality is investigated. We also provide a comprehensive model of test case/suite quality with definitions for the quality attributes and measurements based on findings in the literature and ISO/IEC 25010:2011. The test artifact quality model presented in the paper can be used to support test artifact quality assessment and improvement initiatives in practice. Furthermore, the model can also be used as a framework for documenting context characteristics to make research results more accessible for research and practice.
Highlights
Software development continues to use testing to ensure that we deliver high-quality software
A systematic literature review of techniques and metrics to reduce the cost of mutation testing
Among the least frequent topics, it is worth to note that we found only one systematic literature reviews (SLR) conducted by Barraood et al (S31) which had its main focus on quality of test cases and test suites
Summary
Software development continues to use testing to ensure that we deliver high-quality software. The role of testing has become even more critical in the context of continuous software engineering [1], as we are increasingly relying on testing to release reliable software more frequently. In continuous software engineering organizations shorten the lead time from ideation to delivery by making features available to customers as soon as development is done. This ambition of continuous delivery/deployment entails that organizations might deliver and deploy new version of a software which makes manual testing challenging. Software developers spend around one quarter of their effort on engineering tests [3]. It is beneficial to quality assure test code and retaining the invested effort, i.e. assure that tests are effective. Test effectiveness (a test case/suite’s ability to identify defects/faults in a system under test) has received a lot of attention in research, but it is only one of the multiple dimensions of test quality [4]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
