Cross-company vs. single-company web effort models using the Tukutuku database: An extended study

Abstract

In 2004 [Kitchenham, B.A., Mendes, E., 2004a. Software productivity measurement using multiple size measures. IEEE Transactions on Software Engineering 30 (12), 1023–1035, Kitchenham, B.A., Mendes, E., 2004b. A comparison of cross-company and single-company effort estimation models for web applications. In: Proceedings Evaluation and Assessment in Software Engineering (EASE’ 04), pp. 47–55] (S1) investigated, using data on 63 Web projects, to what extent a cross-company cost model could be successfully employed to estimate development effort for single-company Web projects. Their effort models were built using Forward Stepwise Regression (SWR) and they found that cross-company predictions were significantly worse than single-company predictions. This study S1 was extended by Mendes and Kitchenham [Mendes, E., Kitchenham, B.A., 2004. Further comparison of cross-company and within company effort estimation models for web applications. In: Proceedings International Software Metrics Symposium (METRICS’04), Chicago, Illinois, September 11–17th, 2004. IEEE Computer Society, pp. 348–357] (S2), who used SWR and Case-based reasoning (CBR), and data on 67 Web projects from the Tukutuku database. They built two cross-company and one single-company models and found that both SWR cross-company models and CBR cross-company data provided predictions significantly worse than single-company predictions. Since 2004 another 83 projects were volunteered to the Tukutuku database, and recently used by Mendes et al. [Mendes, E., Di Martino, S., Ferrucci, F., Gravino, C., in press. Effort estimation: How valuable is it for a web company to use a cross-company data set, compared to using its own single-company data set? In: Proceedings of International World Wide Web Conference (WWW’07), Banff, Canada, 8–12 May] (S3), who partially replicated Mendes and Kitchenham’s study (S2), using SWR and CBR. They corroborated some of S2’s findings (SWR cross-company model and the CBR cross-company data provided predictions significantly worse than single-company predictions) however they replicated only part of S2. The objective of this paper (S4) is therefore to extend Mendes et al.’s work and fully replicate S2. We used the same dataset used in S3, and our results corroborated most of those obtained in S2. The main difference between S2 and our study was that one of our SWR cross-company models showed significantly similar predictions to the single-company model, which contradicts the findings from S2.