Software Effort Prediction Research Articles

Integrate the GM(1,1) and Verhulst Models to Predict Software Stage Effort

Software effort prediction clearly plays a crucial role in software project management. In keeping with more dynamic approaches to software development, it is not sufficient to only predict the whole-project effort at an early stage. Rather, the project manager must also dynamically predict the effort of different stages or activities during the software development process. This can assist the project manager to reestimate effort and adjust the project plan, thus avoiding effort or schedule overruns. This paper presents a method for software physical time stage-effort prediction based on grey models GM(1,1) and Verhulst. This method establishes models dynamically according to particular types of stage-effort sequences, and can adapt to particular development methodologies automatically by using a novel grey feedback mechanism. We evaluate the proposed method with a large-scale real-world software engineering dataset, and compare it with the linear regression method and the Kalman filter method, revealing that accuracy has been improved by at least 28% and 50%, respectively. The results indicate that the method can be effective and has considerable potential. We believe that stage predictions could be a useful complement to whole-project effort prediction methods.

Predicting Project Velocity in XP Using a Learning Dynamic Bayesian Network Model

Bayesian networks, which can combine sparse data, prior assumptions and expert judgment into a single causal model, have already been used to build software effort prediction models. We present such a model of an extreme programming environment and show how it can learn from project data in order to make quantitative effort predictions and risk assessments without requiring any additional metrics collection program. The model's predictions are validated against a real world industrial project, with which they are in good agreement.

Bayesian statistical effort prediction models for data-centred 4GL software development

Constructing an accurate effort prediction model is a challenge in Software Engineering. This paper presents three Bayesian statistical software effort prediction models for database-oriented software systems, which are developed using a specific 4GL toolsuite. The models consist of specification-based software size metrics and development team's productivity metric. The models are constructed based on the subjective knowledge of human expert and calibrated using empirical data collected from 17 software systems developed in the target environment. The models' predictive accuracy is evaluated using subsets of the same data, which were not used for the models' calibration. The results show that the models have achieved very good predictive accuracy in terms of MMRE and pred measures. Hence, it is confirmed that the Bayesian statistical models can predict effort successfully in the target environment. In comparison with commonly used multiple linear regression models, the Bayesian statistical models'predictive accuracy is equivalent in general. However, when the number of software systems used for the models' calibration becomes smaller than five, the predictive accuracy of the best Bayesian statistical models are significantly better than the multiple linear regression model. This result suggests that the Bayesian statistical models would be a better choice when software organizations/practitioners do not posses sufficient empirical data for the models' calibration. The authors expect these findings to encourage more researchers to investigate the use of Bayesian statistical models for predicting software effort.

Adaptive fuzzy logic-based framework for software development effort prediction

Algorithmic effort prediction models are limited by their inability to cope with uncertainties and imprecision present in software projects early in the development life cycle. In this paper, we present an adaptive fuzzy logic framework for software effort prediction. The training and adaptation algorithms implemented in the framework tolerates imprecision, explains prediction rationale through rules, incorporates experts knowledge, offers transparency in the prediction system, and could adapt to new environments as new data becomes available. Our validation experiment was carried out on artificial datasets as well as the COCOMO public database. We also present an experimental validation of the training procedure employed in the framework.

Software productivity and effort prediction with ordinal regression

In the area of software cost estimation, various methods have been proposed to predict the effort or the productivity of a software project. Although most of the proposed methods produce point estimates, in practice it is more realistic and useful for a method to provide interval predictions. In this paper, we explore the possibility of using such a method, known as ordinal regression to model the probability of correctly classifying a new project to a cost category. The proposed method is applied to three data sets and is validated with respect to its fitting and predictive accuracy.

Combining techniques to optimize effort predictions in software project management

This paper tackles two questions related to software effort prediction. First, is it valuable to combine prediction techniques? Second, if so, how? Many commentators have suggested the use of more than one technique in order to support effort prediction, but to date there has been little or no empirical investigation to support this recommendation. Our analysis of effort data from a medical records information system reveals that there is little, or even negative, covariance between the accuracy of our three chosen prediction techniques, namely, expert judgment, least squares regression and case-based reasoning. This indicates that when one technique predicts poorly, one or both of the others tends to perform significantly better. This is a particularly striking result given the relative homogeneity of our data set. Consequently, searching for the single “best” technique, at least in this case, leads to a sub-optimal prediction strategy. The challenge then becomes one of identifying a means of determining a priori which prediction technique to use. Unfortunately, despite using a range of techniques including rule induction, we were unable to identify any simple mechanism for doing so. Nevertheless, we believe this remains an important research goal.

Analyzing data sets with missing data: an empirical evaluation of imputation methods and likelihood-based methods

Missing data are often encountered in data sets used to construct software effort prediction models. Thus far, the common practice has been to ignore observations with missing data. This may result in biased prediction models. The authors evaluate four missing data techniques (MDTs) in the context of software cost modeling: listwise deletion (LD), mean imputation (MI), similar response pattern imputation (SRPI), and full information maximum likelihood (FIML). We apply the MDTs to an ERP data set, and thereafter construct regression-based prediction models using the resulting data sets. The evaluation suggests that only FIML is appropriate when the data are not missing completely at random (MCAR). Unlike FIML, prediction models constructed on LD, MI and SRPI data sets will be biased unless the data are MCAR. Furthermore, compared to LD, MI and SRPI seem appropriate only if the resulting LD data set is too small to enable the construction of a meaningful regression-based prediction model.

An investigation of machine learning based prediction systems

Traditionally, researchers have used either off-the-shelf models such as COCOMO, or developed local models using statistical techniques such as stepwise regression, to obtain software effort estimates. More recently, attention has turned to a variety of machine learning methods such as artificial neural networks (ANNs), case-based reasoning (CBR) and rule induction (RI). This paper outlines some comparative research into the use of these three machine learning methods to build software effort prediction systems. We briefly describe each method and then apply the techniques to a dataset of 81 software projects derived from a Canadian software house in the late 1980s. We compare the prediction systems in terms of three factors: accuracy, explanatory value and configurability. We show that ANN methods have superior accuracy and that RI methods are least accurate. However, this view is somewhat counteracted by problems with explanatory value and configurability. For example, we found that considerable effort was required to configure the ANN and that this compared very unfavourably with the other techniques, particularly CBR and least squares regression (LSR). We suggest that further work be carried out, both to further explore interaction between the end-user and the prediction system, and also to facilitate configuration, particularly of ANNs.

Prediction of software effort and project duration

Prediction of software effort and project duration