Effort Prediction Research Articles

Performance tuning for machine learning-based software development effort prediction models

Software development effort estimation is a critical activity of the project management process. In this study, machine learning algorithms were investigated in conjunction with feature transformation, feature selection, and parameter tuning techniques to estimate the development effort accurately and a new model was proposed as part of an expert system. We preferred the most general-purpose algorithms, applied parameter optimization technique (Grid- Search), feature transformation techniques (binning and one-hot-encoding), and feature selection algorithm (principal component analysis). All the models were trained on the ISBSG datasets and implemented by using the scikit-learn package in the Python language. The proposed model uses a multilayer perceptron as its underlying algorithm, applies binning of the features to transform continuous features and one-hot-encoding technique to transform categorical data into numerical values as feature transformation techniques, does feature selection based on the principal component analysis method, and performs parameter tuning based on the GridSearch algorithm. We demonstrate that our effort prediction model mostly outperforms the other existing models in terms of prediction accuracy based on the mean absolute residual parameter.

Threshold-based empirical validation of object-oriented metrics on different severity levels

Software metrics has become desideratum for the fault-proneness, reusability and effort prediction. To enhance and intensify the sufficiency of object-oriented (OO) metrics, it is crucial to perceive the relationship between OO metrics and fault-proneness at distinct severity levels. This paper characterise on the investigation of the software parts with higher probability of occurrence of faults. We examined the effect of thresholds on the OO metrics and build the predictive model based on those threshold values. This paper also instanced on the empirical validation of threshold values calculated for the OO metrics for predicting faults at different severity levels and builds the statistical model using logistic regression. This paper depicts the detection of fault-proneness by extracting the relevant OO metrics and focus on those projects that falls outside the specified risk level for allocating the more resources to them. We presented the effects of threshold values at different risk levels and also validated results on the KC1 dataset using machine learning and different classifiers.

Threshold-based empirical validation of object-oriented metrics on different severity levels

Software metrics has become desideratum for the fault-proneness, reusability and effort prediction. To enhance and intensify the sufficiency of object-oriented (OO) metrics, it is crucial to perceive the relationship between OO metrics and fault-proneness at distinct severity levels. This paper characterise on the investigation of the software parts with higher probability of occurrence of faults. We examined the effect of thresholds on the OO metrics and build the predictive model based on those threshold values. This paper also instanced on the empirical validation of threshold values calculated for the OO metrics for predicting faults at different severity levels and builds the statistical model using logistic regression. This paper depicts the detection of fault-proneness by extracting the relevant OO metrics and focus on those projects that falls outside the specified risk level for allocating the more resources to them. We presented the effects of threshold values at different risk levels and also validated results on the KC1 dataset using machine learning and different classifiers.

A Generic Data Mining Model for Software Cost Estimation Based on Novel Input Selection Procedure

It is always preferable for any estimation model to be inclusive as accuracy in estimation models inherently lie with their inclusiveness. Software cost estimation is the prediction of development effort and time required to develop a software project and being predictive in nature, it demands for inclusiveness, which will accordingly bring the accuracy in it. In this study, a generic model for software cost estimation using an input selection procedure is proposed. The proposed model brings inclusiveness into the already available data mining techniques of software cost estimation by sensitively choosing a subset of highly relevant project attributes and ignoring the less relevant ones. In this article, a diverse set of data mining techniques for software cost estimation are considered. All these techniques are experimented on five data sets before and after passed through the proposed procedure. The obtained results showed that newly generated techniques after being passed through the proposed procedure offer accurate results up in the way of efficiency in software cost estimation.

MINN: A Missing Data Imputation Technique for Analogy-based Effort Estimation

Success and failure of a complex software project are strongly associated with the accurate estimation of development effort. There are numerous estimation models developed but the most widely used among those is Analogy-Based Estimation (ABE). ABE model follows human nature as it estimates the future project’s effort by making analogies with the past project's data. Since ABE relies on the historical datasets, the quality of the datasets affects the accuracy of estimation. Most of the software engineering datasets have missing values. The researchers either delete the projects containing missing values or avoid treating the missing values which reduce the ABE performance. In this study, Numeric Cleansing (NC), K-Nearest Neighbor Imputation (KNNI) and Median Imputation of the Nearest Neighbor (MINN) methods are used to impute the missing values in Desharnais and DesMiss datasets for ABE. MINN technique is introduced in this study. A comparison among these imputation methods is performed to identify the suitable missing data imputation method for ABE. The results suggested that MINN imputes more realistic values in the missing datasets as compared to values imputed through NC and KNNI. It was also found that the imputation treatment method helped in better prediction of the software development effort on ABE model.

Support vector regression‐based imputation in analogy‐based software development effort estimation

AbstractMissing data (MD) is a widespread problem that can affect the ability to use data to construct effective software development effort estimation (SDEE) techniques. To deal with this challenge, several imputation techniques have been investigated in SDEE and k‐nearest neighbors (KNN)‐based imputation is still the most frequently used. To the best of our knowledge, no study has used support vector regression (SVR)‐based imputation to construct accurate estimation techniques, in particular those based on analogy. This paper introduces a new imputation technique based on SVR for handling MD in two analogy‐based SDEE techniques: classical analogy and fuzzy analogy. More specifically, we investigate whether the use of SVR instead of KNN in imputing MD improves the predictive performance of these two analogy‐based techniques. A total of 1134 experiments were conducted involving seven datasets, SVR/KNN MD imputation techniques (KNN with Euclidean and Manhattan distances), three missingness mechanisms (missing completely at random, missing at random, non‐ignorable missing), and MD percentages from 10% to 90%. The results suggest that the use of SVR imputation, rather than KNN imputation, may improve the prediction performance of both analogy‐based techniques. Furthermore, we found that the impact of MD percentage upon effort prediction performance is reduced when using SVR rather than KNN. Moreover, fuzzy analogy generates better estimates in terms of the standardized accuracy measure than classical analogy regardless of the MD technique, the dataset used, the missingness mechanism, or the MD percentage.

Comparison of hard limiting and soft computing methods for predicting software effort estimation: In reference to Small Scale Visualization Projects

It is a well known fact that software effort estimation is exceptionally critical in every software industry, particular during the development of projects. It is hard to estimate the parameters involved due to ambiguity and uncertainty associated with the parameters. It is exactly here the hard limiting techniques, soft computing techniques comes to play. In this unique circumstance, this paper, presents an attempt to that compare the two paradigms for effort estimation. For this, we have considered fifty real time small visualization projects thrive by post graduate students. The prototype development involves following stages: i) Elicitation of seven novel parameters namely Lines of Code, Cumulative Grade Point Average, New and changed code, Reused code, Cyclomatic Complexity, Algorithmic Complexity and Functional Points. ii) Developing of hard limiting methods and soft computing methods for prediction of software effort involved in terms of duration in minutes.For the validation of the models error metrics namely: Mean Absolute Error (MAE), Mean Magnitude of Relative Error (MMRE), Mean of Magnitude of error Relative to the Estimate (MMER) and Root Mean Square Error (RMSE) have been used. The result showed that the models compared very well with marginal difference in terms of predict values of error matrix.

Comparison of hard limiting and soft computing methods for predicting software effort estimation: In reference to Small Scale Visualization Projects

It is a well known fact that software effort estimation is exceptionally critical in every software industry, particular during the development of projects. It is hard to estimate the parameters involved due to ambiguity and uncertainty associated with the parameters. It is exactly here the hard limiting techniques, soft computing techniques comes to play. In this unique circumstance, this paper, presents an attempt to that compare the two paradigms for effort estimation. For this, we have considered fifty real time small visualization projects thrive by post graduate students. The prototype development involves following stages: i) Elicitation of seven novel parameters namely Lines of Code, Cumulative Grade Point Average, New and changed code, Reused code, Cyclomatic Complexity, Algorithmic Complexity and Functional Points. ii) Developing of hard limiting methods and soft computing methods for prediction of software effort involved in terms of duration in minutes.For the validation of the models error metrics namely: Mean Absolute Error (MAE), Mean Magnitude of Relative Error (MMRE), Mean of Magnitude of error Relative to the Estimate (MMER) and Root Mean Square Error (RMSE) have been used. The result showed that the models compared very well with marginal difference in terms of predict values of error matrix.

An ensemble-based model for predicting agile software development effort

To support agile software development projects, an array of tools and systems is available to plan, design, track, and manage the development process. In this paper, we explore a critical aspect of agile development i.e., effort prediction, that cuts across these tools and agile project teams. Accurate effort prediction can improve the planning of a sprint by enabling optimal assignments of both stories and developers. We develop a model for story-effort prediction using variables that are readily available when a story is created. We use seven predictive algorithms to predict a story’s effort. Interestingly, none of the predictive algorithms consistently outperforms others in predicting story effort across our test data of 423 stories. We develop an ensemble-based method based on our model for predicting story effort. We conduct computational experiments to show that our ensemble-based approach performs better in comparison to other ensemble-based benchmarking approaches. We then demonstrate the practical application of our predictive model and our ensemble-based approach by optimizing sprint planning for two projects from our dataset using an optimization model.

Software effort prediction with algorithm based frameworks

The need for accurate effort predictions for projects is one of the most critical and complex issues in the software industry. Effort estimation is an essential activity in planning and monitoring software project development to deliver the product on time and within budget. The competitiveness of software organizations depends on their ability to predict the effort required for developing software systems accurately. Several algorithmic approaches have been proposed in the literature to support software engineers in improving the accuracy of their estimations. The decision of how to select different techniques considering the characteristics of a specific dataset through genetic algorithms could be viewed as a search-based problem for software engineering. Some groups use well known evolutionary learning algorithms (MMRE, PRED (25), PRED (50) and PRED (75), while others use machine learning methods like artificial neural network and fuzzy logic. Analogy estimation is also being utilized for estimating software effort, which has been shown to predict accurate results consistently. Genetic frameworks are becoming famous for selecting best learning schemes for effort prediction using the elitism techniques, utilizing some well-known parameters like Spearman's rank correlation, the median of the magnitude of relative error (MdMRE), and mean of the absolute residuals (MMAR). The technique of using fuzzy rough sets for the analysis is also becoming popular. This article aims to compare various algorithm based frameworks for predicting software development effort which can generate the best-optimized software product.

Investigating the Significance of the Bellwether Effect to Improve Software Effort Prediction: Further Empirical Study

Context: In addressing how best to estimate how much effort is required to develop software, a recent study found that using exemplary and recently completed projects [forming Bellwether moving windows (BMW)] in software effort prediction (SEP) models leads to relatively improved accuracy. More studies need to be conducted to determine whether the BMW yields improved accuracy in general, since different sizing and aging parameters of the BMW are known to affect accuracy. Objective: To investigate the existence of exemplary projects (Bellwethers) with defined window size and age parameters, and whether their use in SEP improves prediction accuracy. Method: We empirically investigate the moving window assumption based on the theory that the prediction outcome of a future event depends on the outcomes of prior events. Sampling of Bellwethers was undertaken using three introduced Bellwether methods (SSPM, SysSam, and RandSam). The ergodic Markov chain was used to determine the stationarity of the Bell-wethers. Results: Empirical results show that 1) Bellwethers exist in SEP and 2) the BMW has an approximate size of 50 to 80 exemplary projects that should not be more than 2 years old relative to the new projects to be estimated. Conclusion: The study's results add further weight to the recommended use of Bellwethers for improved prediction accuracy in SEP.

Test effort estimation and prediction of traditional and rapid release models using machine learning algorithms

Test effort estimation and prediction of traditional and rapid release models using machine learning algorithms

Models for Predicting Development Effort of Small-Scale Visualization Projects

AbstractSoftware project effort estimation is one of the important aspects of software engineering. Researchers in this area are still striving hard to come out with the best predictive model that has befallen as a greatest challenge. In this work, the effort estimation for small-scale visualization projects all rendered on engineering, general science, and other allied areas developed by 60 postgraduate students in a supervised academic setting is modeled by three approaches, namely, linear regression, quadratic regression, and neural network. Seven unique parameters, namely, number of lines of code (LOC), new and change code (N&C), reuse code (R), cumulative grade point average (CGPA), cyclomatic complexity (CC), algorithmic complexity (AC), and function points (FP), which are considered to be influential in software development effort, are elicited along with actual effort. The three models are compared with respect to their prediction accuracy via the magnitude of error relative to the estimate (MER) for each project and also its mean MER (MMER) in all the projects in both the verification and validation phases. Evaluations of the models have shown MMER of 0.002, 0.006, and 0.009 during verification and 0.006, 0.002, and 0.002 during validation for the multiple linear regression, nonlinear regression, and neural network models, respectively. Thus, the marginal differences in the error estimates have indicated that the three models can be alternatively used for effort computation specific to visualization projects. Results have also suggested that parameters such as LOC, N&C, R, CC, and AC have a direct influence on effort prediction, whereas CGPA has an inverse relationship. FP seems to be neutral as far as visualization projects are concerned.

Path associations between trait personality, enjoyment, and effort by gender in high school physical education

Various combinations or dimensions of personality traits are associated with achievement and its related behaviours in a host of domains including academia, physical activity, and sport. Little is yet known about how these trait personality dimensions relate to vital outcomes such as enjoyment and effort in school-age physical education students. The primary intent of this study was to test a proposed path model wherein personality and effort relate indirectly through enjoyment by gender in 316 students in 9th and 10th-grade physical education. Secondary objectives were to assess gender-specific levels of and relations among the six (HEXACO) personality dimensions, the prediction of effort by personality dimensions, and relations between enjoyment, effort, and the facets (sub-scales) of the personality dimensions. Surveys were completed that assessed students' effort, enjoyment, and HEXACO trait personality (honesty-humility, emotionality, extraversion, agreeableness, conscientiousness, and openness to experience). In both boys and girls, the data fit the proposed paths with lower openness to experience and higher extraversion predicting enjoyment and effort via enjoyment. In boys, personality also predicted effort whereas honesty-humility predicted enjoyment. Girls' agreeableness showed a higher predictive effect on effort through enjoyment compared to that of boys. Further investigation is needed to determine whether students' particular profiles of trait personality (especially those lower in extraversion and higher in openness to experience; and, girls who are less agreeable) are more susceptible to compromised enjoyment and/or effort in high school physical education.

Performance Exploratory Comparative Study for Software Change and Effort Prediction in Object-Oriented Systems Using Statistical and Machine Learning Algorithms

Precise software effort estimation is a vital sector of the software procedure. An excess as well as lessening software effort can prompt to unsafe concludes. Additionally, software venture supervisors need toward create evaluations of exactly how abundant a software development has been accepted cost. The predominant price of some software is the price of ascertaining effort. In this way, effort assessment has been exceptionally urgent as well as an effective dependably a necessity to enhance individual’s precision however as much at the time that could be expected. Different effort estimation models are, yet effortful to define and that model contributes too much precise assessment on that dataset. Above research experimentally assesses as well as analyzes the capability of Linear Regression Model, Bagging, Decision Table, Decision Stump, IBk, KStar, Locally weighted learning (LWL), Multilayer Perceptron, Simple Linear Regression, Stacking and Vote on software venture dataset. The dataset has been attained taken away from 3518 instances of java open source software. The outcomes demonstrate that Root relative squared error of K-Star technique was just 71.54%. Hence, the performance of K-Star technique has proved to be superior to the various other strategies. Above research, on the utilization of statistical and machine learning conclusion aimed at exploring the outcomes acquired in an analysis in the arena of computational intelligence has been concentrated. An investigation which includes a set of strategies in classification tasks has been shown and along with a set of procedures which are helpful to break down the conduct of a strategy as for a set of algorithms has been concentrated, for example, the structure in which another proposition is created.

Cuckoo search based hybrid models for improving the accuracy of software effort estimation

This research proposes a new approach which is based on Cuckoo Search algorithm for the prediction of software development effort. It uses Cuckoo Search for discovering the best possible parameters of COCOMO II model and then further hybridizes with ANN for increasing the accuracy to better predict the software development effort. The proposed hybrid models have been tested on two standard datasets. During experimentation, it has been seen that the proposed hybrid models provide more accurate and effective results than other existing models. The result has been analyzed with MMRE and three different types of PRED 25, 30 and 40% that shows the efficiency and capability of the proposed hybrid models. A comparative study of computational complexity with other existing approach has also been done which shows the superiority of the proposed model over existing approaches.

The role and value of replication in empirical software engineering results

ContextConcerns have been raised from many quarters regarding the reliability of empirical research findings and this includes software engineering. Replication has been proposed as an important means of increasing confidence. ObjectiveWe aim to better understand the value of replication studies, the level of confirmation between replication and original studies, what confirmation means in a statistical sense and what factors modify this relationship. MethodWe perform a systematic review to identify relevant replication experimental studies in the areas of (i) software project effort prediction and (ii) pair programming. Where sufficient details are provided we compute prediction intervals. ResultsOur review locates 28 unique articles that describe replications of 35 original studies that address 75 research questions. Of these 10 are external, 15 internal and 3 internal-same-article replications. The odds ratio of internal to external (conducted by independent researchers) replications of obtaining a ‘confirmatory’ result is 8.64. We also found incomplete reporting hampered our ability to extract estimates of effect sizes. Where we are able to compute replication prediction intervals these were surprisingly large. ConclusionWe show that there is substantial evidence to suggest that current approaches to empirical replications are highly problematic. There is a consensus that replications are important, but there is a need for better reporting of both original and replicated studies. Given the low power and incomplete reporting of many original studies, it can be unclear the extent to which a replication is confirmatory and to what extent it yields additional knowledge to the software engineering community. We recommend attention is switched from replication research to meta-analysis.

Support vector regression for predicting software enhancement effort

ContextSoftware maintenance (SM) has to be planned, which involves SM effort prediction. One type of SM is enhancement, where new functionality is added or existing functionality changed or deleted. ObjectiveAnalyze the prediction accuracy of two types of support vector regression (ε-SVR and ʋ-SVR) when applied to predict software enhancement effort. MethodBoth types of support vector regression used linear, polynomial, radial basis function, and sigmoid kernels. Prediction accuracies for ε-SVR and ʋ-SVR were compared with those of statistical regressions, neural networks, association rules, and decision trees. The models were trained and tested with five data sets of enhancement projects from Release 11 of the International Software Benchmarking Standards Group (ISBSG). Each data set was selected on the basis of data quality, development platform, programming language generation, and levels of effort recording. ResultsThe polynomial kernel ε-SVR (PKε-SVR) was statistically better than statistical regression, neural networks, association rules and decision trees, with 95% confidence. ConclusionsA PKε-SVR could be used for predicting software enhancement effort in mainframe platforms and coded in a third-generation programming languages, and when enhancement effort recording includes the efforts of the development team, its support personnel, the computer operations involvement, and end users.

Assessing Usability of Wealth Management Portals using Neurophysiological Tools: Eye-tracking and fNIRS Study

Assessing Usability of Wealth Management Portals using Neurophysiological Tools: Eye-tracking and fNIRS Study

Evaluating subset selection methods for use case points estimation

When the Use Case Points method is used for software effort estimation, users are faced with low model accuracy which impacts on its practical application. This study investigates the significance of using subset selection methods for the prediction accuracy of Multiple Linear Regression models, obtained by the stepwise approach. K-means, Spectral Clustering, the Gaussian Mixture Model and Moving Window are evaluated as appropriate subset selection techniques. The methods were evaluated according to several evaluation criteria and then statistically tested. Evaluation was performing on two independent datasets - which differ in project types and size. Both were cut by the hold-out method. If clustering were used, the training sets were clustered into 3 classes; and, for each of class, an independent regression model was created. These were later used for the prediction of testing sets. If Moving Window was used, then window of sizes 5, 10 and 15 were tested.The results show that clustering techniques decrease prediction errors significantly when compared to Use Case Points or moving windows methods. Spectral Clustering was selected as the best-performing solution, because it achieves a Sum of Squared Errors reduction of 32% for the first dataset, and 98% for the second dataset. The Mean Absolute Percentage Error is less than 1% for the second dataset for Spectral Clustering; 9% for moving window; and 27% for Use Case Points. When the first dataset is used, then prediction errors are significantly higher – 53% for Spectral Clustering, but Use Case Points produces a 165% result.It can be concluded that this study proves subset selection techniques as a significant method for improving the prediction ability of linear regression models - which are used for software development effort prediction. It can also be concluded that the clustering method performs better than the moving window method.