Tukutuku Database Research Articles

Using tabu search to configure support vector regression for effort estimation

Recent studies have reported that Support Vector Regression (SVR) has the potential as a technique for software development effort estimation. However, its prediction accuracy is heavily influenced by the setting of parameters that needs to be done when employing it. No general guidelines are available to select these parameters, whose choice also depends on the characteristics of the dataset being used. This motivated the work described in (Corazza et al. 2010), extended herein. In order to automatically select suitable SVR parameters we proposed an approach based on the use of the meta-heuristics Tabu Search (TS). We designed TS to search for the parameters of both the support vector algorithm and of the employed kernel function, namely RBF. We empirically assessed the effectiveness of the approach using different types of datasets (single and cross-company datasets, Web and not Web projects) from the PROMISE repository and from the Tukutuku database. A total of 21 datasets were employed to perform a 10-fold or a leave-one-out cross-validation, depending on the size of the dataset. Several benchmarks were taken into account to assess both the effectiveness of TS to set SVR parameters and the prediction accuracy of the proposed approach with respect to widely used effort estimation techniques. The use of TS allowed us to automatically obtain suitable parameters’ choices required to run SVR. Moreover, the combination of TS and SVR significantly outperformed all the other techniques. The proposed approach represents a suitable technique for software development effort estimation.

Investigating the use of Support Vector Regression for web effort estimation

Support Vector Regression (SVR) is a new generation of Machine Learning algorithms, suitable for predictive data modeling problems. The objective of this paper is twofold: first, to investigate the effectiveness of SVR for Web effort estimation using a cross-company dataset; second, to compare different SVR configurations looking at the one that presents the best performance. In particular, we took into account three variables' preprocessing strategies (no-preprocessing, normalization, and logarithmic), in combination with two different dependent variables (effort and inverse effort). As a result, SVR was applied using six different data configurations. Moreover, to understand the suitability of kernel functions to handle non-linear problems, SVR was applied without a kernel, and in combination with the Radial Basis Function (RBF) and the Polynomial kernels, thus obtaining 18 different SVR configurations. To identify, for each configuration, which were the best values for each of the parameters we defined a procedure based on a leave-one-out cross-validation approach. The dataset employed was the Tukutuku database, which has been adopted in many previous Web effort estimation studies. Three different training and test set splits were used, including respectively 130 and 65 projects. The SVR-based predictions were also benchmarked against predictions obtained using Manual StepWise Regression and Case-Based Reasoning. Our results showed that the configuration corresponding to the logarithmic features' preprocessing, in combination with the RBF kernel provided the best results for all three data splits. In addition, SVR provided significantly superior prediction accuracy than all the considered benchmarking techniques.

Bayesian Network Models for Web Effort Prediction: A Comparative Study

The objective of this paper is to compare, using a cross-company dataset, several Bayesian network (BN) models for Web effort estimation. Eight BNs were built; four automatically using Hugin and PowerSoft tools with two training sets, each with 130 Web projects from the Tukutuku database; four using a causal graph elicited by a domain expert, with parameters automatically fit using the same training sets used in the automated elicitation (hybrid models). Their accuracy was measured using two validation sets, each containing data on 65 projects, and point estimates. As a benchmark, the BN-based estimates were also compared to estimates obtained using manual stepwise regression (MSWR), case-based reasoning (CBR), mean- and median-based effort models. MSWR presented significantly better predictions than any of the BN models built herein, and in addition was the only technique to provide significantly superior predictions to a median-based effort model. This paper investigated data-driven and hybrid BN models using project data from the Tukutuku database. Our results suggest that the use of simpler models, such as the median effort, can outperform more complex models, such as BNs. In addition, MSWR seemed to be the only effective technique for Web effort estimation.

Cross-company vs. single-company web effort models using the Tukutuku database: An extended study

In 2004 [Kitchenham, B.A., Mendes, E., 2004a. Software productivity measurement using multiple size measures. IEEE Transactions on Software Engineering 30 (12), 1023–1035, Kitchenham, B.A., Mendes, E., 2004b. A comparison of cross-company and single-company effort estimation models for web applications. In: Proceedings Evaluation and Assessment in Software Engineering (EASE’ 04), pp. 47–55] (S1) investigated, using data on 63 Web projects, to what extent a cross-company cost model could be successfully employed to estimate development effort for single-company Web projects. Their effort models were built using Forward Stepwise Regression (SWR) and they found that cross-company predictions were significantly worse than single-company predictions. This study S1 was extended by Mendes and Kitchenham [Mendes, E., Kitchenham, B.A., 2004. Further comparison of cross-company and within company effort estimation models for web applications. In: Proceedings International Software Metrics Symposium (METRICS’04), Chicago, Illinois, September 11–17th, 2004. IEEE Computer Society, pp. 348–357] (S2), who used SWR and Case-based reasoning (CBR), and data on 67 Web projects from the Tukutuku database. They built two cross-company and one single-company models and found that both SWR cross-company models and CBR cross-company data provided predictions significantly worse than single-company predictions. Since 2004 another 83 projects were volunteered to the Tukutuku database, and recently used by Mendes et al. [Mendes, E., Di Martino, S., Ferrucci, F., Gravino, C., in press. Effort estimation: How valuable is it for a web company to use a cross-company data set, compared to using its own single-company data set? In: Proceedings of International World Wide Web Conference (WWW’07), Banff, Canada, 8–12 May] (S3), who partially replicated Mendes and Kitchenham’s study (S2), using SWR and CBR. They corroborated some of S2’s findings (SWR cross-company model and the CBR cross-company data provided predictions significantly worse than single-company predictions) however they replicated only part of S2. The objective of this paper (S4) is therefore to extend Mendes et al.’s work and fully replicate S2. We used the same dataset used in S3, and our results corroborated most of those obtained in S2. The main difference between S2 and our study was that one of our SWR cross-company models showed significantly similar predictions to the single-company model, which contradicts the findings from S2.