On Some Statistical Aspects of Software Testing and Reliability

Abstract

This article discusses the author’s views on possible contributions statistics can make to software testing and reliability. Several difficulties are highlighted and several research challenges are discussed. Overall, the message is that statistical methods cannot provide proper support for software testing or provide useful inferences on software reliability if the statistical methods are considered to be an ‘add-on’; careful treatment of the uncertainties and the adequate use of statistical methods have to be right at the center of the software development and test processes to ensure better tested and more reliable software. In line with this requirement, the development of suitable statistical methods requires collaboration of software developers and testers with statisticians on real-world problems.