Software Cost Estimation Models Research Articles

An Empirical Validation of the Checkpoint Software Cost Estimation Model

Abstract As part of an on-going research project began in 1995 at the Air Force Institute of Technology, popular software cost estimation models were calibrated on data sets from a large Air Force database. The purpose was to determine if calibration improved a model's accuracy and to determine the relative accuracy of the calibrated models. The development effort accuracy of each model was evaluated before and after calibration using a hold out sample, or data not used in calibration. Results on a variety of data sets showed that calibrating the CHECKPOINT model improved its accuracy for development effort. The calibrated accuracy of the model on these data sets was significantly greater than that achieved by other calibrated models. The model's accuracy was especially noteworthy on data sets that used function points as a measure of size.

Assessing Software Cost Estimation Models: criteria for accuracy, consistency and regression

One of the problems in software cost estimation is how to evaluate estimation models. Estimation models are usually evaluated against two attributes: estimation accuracy and estimation consistency. A number of measures are reported in the literature, but they have shortcomings. There is no generally accepted standard to evaluate estimation models and the existing measures sometimes are not consistent among themselves. This paper examines existing measures of estimation accuracy and consistency and proposes two new ones: the weighted mean of quartiles of relative errors (WMQ) as a measure of accuracy and the standard deviation of the ratios of the estimate to actual observation (SDR) as a measure of consistency. Besides, a new regression criterion is proposed to determine model parameters. The proposed measures and criterion were tested with a data set from real world software projects. Results obtained show that these new measures and criterion overcome many of the difficulties of the existing ones.

A model for software development effort and cost estimation

Several algorithmic models have been proposed to estimate software costs and other management parameters. Early prediction of completion time is absolutely essential for proper advance planning and aversion of the possible ruin of a project. L.H. Putnam's (1978) SLIM (Software LIfecycle Management) model offers a fairly reliable method that is used extensively to predict project completion times and manpower requirements as the project evolves. However, the nature of the Norden/Rayleigh curve used by Putnam renders it unreliable during the initial phases of the project, especially in projects involving a fast manpower buildup, as is the case with most software projects. In this paper, we propose the use of a model that improves early prediction considerably over the Putnam model. An analytic proof of the model's improved performance is also demonstrated on simulated data.

Cost models for future software life cycle processes: COCOMO 2.0

Current software cost estimation models, such as the 1981 Constructive Cost Model (COCOMO) for software cost estimation and its 1987 Ada COCOMO update, have been experiencing increasing difficulties in estimating the costs of software developed to new life cycle processes and capabilities. These include non-sequential and rapid-development process models; reuse-driven approaches involving commercial off-the-shelf (COTS) packages, re-engineering, applications composition, and applications generation capabilities; object-oriented approaches supported by distributed middleware; and software process maturity initiatives. This paper summarizes research in deriving a baseline COCOMO 2.0 model tailored to these new forms of software development, including rationale for the model decisions. The major new modeling capabilities of COCOMO 2.0 are a tailorable family of software sizing models, involving Object Points, Function Points, and Source Lines of Code; nonlinear models for software reuse and re-engineering; an exponentdriver approach for modeling relative software diseconomies of scale; and several additions, deletions and updates to previous COCOMO effort-multiplier cost drivers. This model is serving as a framework for an extensive current data collection and analysis effort to further refine and calibrate the model's estimation capabilities.

Empirical studies of assumptions that underlie software cost-estimation models

The paper reviews some of the assumptions built into conventional cost models and identifies whether or not there is empirical evidence to support these assumptions. The results indicate that the assumption that there is a nonlinear relationship between size and effort is not supported, but the assumption of a nonlinear relationship between effort and duration is. Second, the assumption that a large number of subjective productivity adjustment factors is necessary is not supported. In addition, it also appears that a large number of size adjustment factors are unnecessary. Third, the assumption that staff experience and/or staff capability are the most significant cost drivers (after allowing for the effect of size) is not supported by the data available to the MERMAID project, but neither can it be confirmed from analysis of the COCOMO data set. Finally, the assumption that compression of schedule decreases productivity was not supported. In fact, none of the models of schedule compression currently included in existing cost models was supported by the data.

Function point analysis: evaluation of a software cost estimation model

The merits of function point analysis are discussed. Function point analysis was chosen because it is one of the most widely used aids for software cost estimation. In the analysis we use data from a large survey of Dutch organizations, from an experiment on the effectiveness of software cost estimation models and from a field study aimed at the adjustment part of the FPA model. Conclusions show that it is indeed widely used and that it performs reasonably well as a product sizer, but that the adjustment part of the model is less useful.

On the use of software cost models

A number of methods and tools have been developed over the years to meet the increasing need to control software development. Among the tools are software cost estimation models. Well-known examples are COCOMO, PRICE S, Estimacs, and Function point analysis. The limited and often unsuccessful use of cost models is the motivation behind this paper. The authors oppose the idea that a model is the solution to the estimation problem, but are convinced that the use of a model can contribute to the control of software development, if it is used properly. The model should be used to generate a second opinion. It will have value as a means of communication, as a checklist, and as it forces the user to collect data on the development process. Certain organizational requirements should be fulfilled to be able to use the model properly. The proper use of the model and the organizational requirements are the main subjects of this paper.

Economics of reuse: issues and alternatives

A major reason for increasing the level of reuse in the software development process is to lower costs. By replacing expensive development activities with the adoption of relatively inexpensive reused parts, software reuse has significant potential as a technique for cutting costs and helping to free developers for work on the more complex or novel components of a software system. Unfortunately, the level of reuse currently in place in the software industry is limited by an inability to predict how and where resources should be invested. The paper describes a software costestimation model developed specifically to deal with decisions about software reuse.

Choosing a software cost estimation model for your organization: A case study

In the last decade or so, many software cost estimation models have been developed. These differ substantially from each other, particularly with respect to the inputs required and their outputs. For a software manager, the problem of selecting a particular model, or a combination of models that can be applied to an individual organization, is not all trivial. This paper describes our efforts to solve this problem for two organizations who had collected data on past development efforts. Statistical correlations between the actual and the estimated efforts calculated by using different cost estimation models were obtained. Regression analysis revealed that the cost model that is used by the organizations is not ideal for their environment. Statistical tests show that the results obtained are indeed statistically significant. The methodology used to perform the case study is applied to predict the development effort for a project, and the result is quite impressive.

Are software cost-estimation models accurate?

The use of a model is one way to estimate a software development project. The paper describes an experiment in which a number of automated versions of estimating models were tested. During the experiment, experienced project leaders were asked to make a number of estimates for a project. This related to a project that had actually been carried out. On the basis of the differences found between the estimates and reality, it is concluded that no proof is given that the models can be used for estimating projects at an early stage of system development. Therefore, only limited confidence should be placed in estimates that are obtained with a model only.

Calibrating a software cost estimation model: why and how

Calibration, has been found to be difficult in practice. Wide experience in using the estimation model is necessary; experience which the beginner naturally lacks. This paper indicates why it is important to calibrate a model and how the inexperienced user can be helped by an expert system. In addition, the development of, and experience with, the prototype of an expert system are described. The system dealt with here is intended for the calibration of the PRICE SP estimation.

An empirical validation of software cost estimation models

Practitioners have expressed concern over their inability to accurately estimate costs associated with software development. This concern has become even more pressing as costs associated with development continue to increase. As a result, considerable research attention is now directed at gaining a better understanding of the software-development process as well as constructing and evaluating software cost estimating tools. This paper evaluates four of the most popular algorithmic models used to estimate software costs (SLIM, COCOMO, Function Points, and ESTIMACS). Data on 15 large completed business data-processing projects were collected and used to test the accuracy of the models' ex post effort estimation. One important result was that Albrecht's Function Points effort estimation model was validated by the independent data provided in this study [3]. The models not developed in business data-processing environments showed significant need for calibration. As models of the software-development process, all of the models tested failed to sufficiently reflect the underlying factors affecting productivity. Further research will be required to develop understanding in this area.

Software cost modeling: Some lessons learned

This paper summarizes certain lessons we learned recently in developing a software cost estimation model for TRW. With respect to general needs in the cost estimation field, it was found particularly important and useful to address the following three needs: 1. 1. to develop a set of well-defined, agreed-on criteria for the “goodness” of a software cost model; 2. 2. to evaluate existing and future models with respect to these criteria; 3. 3. to emphasize “constructive” models that relate their cost estimates to actual software phenomenology and project dynamics. This paper presents a set of criteria found to be particularly important in practical software cost estimation, together with examples of their importance.