Empirical validation for effectiveness of fault prediction technique based on cost analysis framework

Abstract

Prediction of the faulty or non-faulty modules in a software is an important task in software development life cycle and often carried out by validating them through certain empirical techniques. A good number of techniques have been proposed by different authors to develop fault prediction models and determine best suitable techniques for fault prediction based on certain performance criteria. However it appears that there is a scope to extend the research by giving emphasis on feasibility analysis of fault prediction techniques. In this paper, a cost evaluation framework has been proposed to evaluate the effectiveness of developed fault prediction models. Statistical methods such as linear regression, logistic regression, polynomial regression, Naive Bayes and support vector machine have been applied to develop a classifier in order to predict as to whether any module is a faulty or non-faulty one for an embedded software system. The proposed approach has been applied on five different public domain software system chosen from NASA database consisting of different number of faulty models. The performance of the predicted models are evaluated using different performance evaluation parameters. From cost-based analysis, experimental results reveal that our fault prediction model is best suitable for projects with faulty modules less than a certain threshold value depending on cost and fault identification efficiency of different testing phases. The results also show that the model developed using logistic regression technique obtained promising results in terms of performance and cost when compared with other applied techniques.