Mean Magnitude Of Relative Error Research Articles

Enhanced Software Effort Estimation Using Multi Layered Feed Forward Artificial Neural Network Technique

Software Effort Estimation models are hot topic of study over 3 decades. Several models have been developed in these decades. Providing accurate estimations of software is still very challenging. The major reason for such disappointments in projects are because of inaccurate software development norms; effort estimation is one such practice. Dynamically fluctuating environment of technology in software development industry make effort estimation further perplexing. One of the most commonly used algorithmic model for estimating effort in industry is COCOMO. Capability of machine learning particularly Artificial Neural Networks is to adjust a complex set of bond among the various independent and dependent variables. The paper proposes usage of ANN (Artificial Neural Network) based model technologically advanced using Multi Layered Feed Forward Neural Network which is given training with Back Propagation training method. COCOMO data-set is accustomed to test and train the network. Mean-Square-Error (MSE) and Mean Magnitude of Relative-Error (MMRE) are used as performance measurement indices. The experiment outputs suggest that the suggested model can provide better results and accurately forecast the software development effort.

Prediction accuracy measurements as a fitness function for software effort estimation.

This paper evaluates the usage of analytical programming and different fitness functions for software effort estimation. Analytical programming and differential evolution generate regression functions. These functions are evaluated by the fitness function which is part of differential evolution. The differential evolution requires a proper fitness function for effective optimization. The problem is in proper selection of the fitness function. Analytical programming and different fitness functions were tested to assess insight to this problem. Mean magnitude of relative error, prediction 25 %, mean squared error (MSE) and other metrics were as possible candidates for proper fitness function. The experimental results shows that means squared error performs best and therefore is recommended as a fitness function. Moreover, this work shows that analytical programming method is viable method for calibrating use case points method. All results were evaluated by standard approach: visual inspection and statistical significance.

A method for generating membership function from numerical data

Construction of membership functions from numerical data is very important in various applications of the fuzzy set theory. In last two decade, many methods of membership function generation have been developed. Majority of the methods are application domain dependent and complex. In this paper, a simple method for the construction of membership functions from numerical data is proposed. To validate the proposed method, commonly used and suggested evaluation measures: average error rate, mean magnitude of relative error (MMRE), balanced mean magnitude of relative error (BMMRE), and coefficient of determination (R 2 ), have been taken. The validating results show that proposed method has a higher accuracy than existing methods. The sensitivity analysis has been performed to analyze the impact of input variable on the output variable.

Tuning of COCOMO Model Parameters by using Bee Colony Optimization

Constructive Cost Model (COCOMO) used parameters for software effort estimation, which were calculated in 1981 by regression analysis of 63 types of project data; therefore applying these parameters to current project development will not generate accurate results. The objective of current research is applying Bee Colony Optimization (BCO) metaheuristic approach to optimize the parameters of COCOMO model for improving software cost estimation. The Bee Colony Optimization (BCO) is a new branch of Swarm Intelligence and has been applied successfully to various engineering disciplines. BCO approach is a “bottom-up” approach to modeling where special kinds of artificial agents are created by analogy with bees. These artificial agents or bees are used to solve complex combinatorial optimization problems. The proposed model validation is carried out using Interactive Voice Response software project dataset of a company. The results generated by the proposed model are compared to those obtained by methods proposed in the literature using Walston-Felix, SEL, Bailey-Basil, COCOMO II and Halstead models. The BCO approach generates various partial solutions and best solution is selected based on Mean Magnitude of Relative Error. The results obtained show that the proposed BCO based model is able to improve the accuracy of cost estimation and also outperform other models.

Software cost estimating for CMMI Level 5 developers

This article provides analysis results of Capability Maturity Model Integrated Level 5 projects for developers earning the highest level possible, using actual software data from their initial project estimates. Since there were no measures to verify software performance, this level was used a proxy for high quality software. Ordinary least squares regression was used to predict final effort hours with initially estimated variables obviates the need to estimate growth or shrinkage for typical changes occurring in software projects, regardless of software developer (contracted or in-house). The OLS equations, or cost estimating relationship equations, were evaluated by a series of standards: statistical significance, visual inspection, goodness of fit measures, and academically set thresholds for accuracy measures used in software cost estimating: mean magnitude of relative error and prediction (for determining the percentage of records with 25%, or less, based on their magnitude of relative error score). As several initial estimated variables were strongly correlated to the reported final effort hours and each other, each variable was examined separately. Thirty records from software projects completed in 2003–2008 for the highest process maturity level were used to compute statistically significant equations with implicit growth or shrinkage in their make-up.

A fuzzy logic based approach for phase-wise software defects prediction using software metrics

ContextThe software defect prediction during software development has recently attracted the attention of many researchers. The software defect density indicator prediction in each phase of software development life cycle (SDLC) is desirable for developing a reliable software product. Software defect prediction at the end of testing phase may not be more beneficial because the changes need to be performed in the previous phases of SDLC may require huge amount of money and effort to be spent in order to achieve target software quality. Therefore, phase-wise software defect density indicator prediction model is of great importance. ObjectiveIn this paper, a fuzzy logic based phase-wise software defect prediction model is proposed using the top most reliability relevant metrics of the each phase of the SDLC. MethodIn the proposed model, defect density indicator in requirement analysis, design, coding and testing phase is predicted using nine software metrics of these four phases. The defect density indicator metric predicted at the end of the each phase is also taken as an input to the next phase. Software metrics are assessed in linguistic terms and fuzzy inference system has been employed to develop the model. ResultsThe predictive accuracy of the proposed model is validated using twenty real software project data. Validation results are satisfactory. Measures based on the mean magnitude of relative error and balanced mean magnitude of relative error decrease significantly as the software project size increases. ConclusionIn this paper, a fuzzy logic based model is proposed for predicting software defect density indicator at each phase of the SDLC. The predicted defects of twenty different software projects are found very near to the actual defects detected during testing. The predicted defect density indicators are very helpful to analyze the defect severity in different artifacts of SDLC of a software project.

Impact of Modification Made in Re-UCP on Software Effort Estimation

This research work evaluates the performance of Re-UCP model and compares the results with the UCP and e-UCP method of software effort estimation. In this research work, an attempt has been made to highlight the accuracy of results by using MRE (Magnitude of Relative Error), MMRE (Mean Magnitude Relative Error), MdMRE (Median of Magnitude Relative Error) tools to check the error rate and PRED (20) and PRED (10) method to find out the predictability of accuracy of Re-UCP software effort estimation method. The observations made from the results are based on the comparison of Re-UCP, e-UCP and UCP models of software effort estimation.

A Novel PSO based Approach with Hybrid of Fuzzy C-Means and Learning Automata in Software Cost Estimation

We used Particle Swarm Optimization (PSO) algorithm hybrid with Fuzzy C-Means (FCM) and Learning Automata (LA) algorithms for Software Cost Estimation (SCE). In this paper we test and evaluate PSO-FCM and PSO-LA hybrid models on NASA dataset software projects. The obtained results showed that in the hybrid models the values of Magnitude of Relative Error (MRE) and Mean Magnitude of Relative Error (MMRE) were reduced compared with COCOMO model and also the accuracy of Percentage of Relative Error Deviation (PRED) was higher in the hybrid models.

A Novel PSO based Approach with Hybrid of Fuzzy C-Means and Learning Automata in Software Cost Estimation

We used Particle Swarm Optimization (PSO) algorithm hybrid with Fuzzy C-Means (FCM) and Learning Automata (LA) algorithms for Software Cost Estimation (SCE). In this paper we test and evaluate PSO-FCM and PSO-LA hybrid models on NASA dataset software projects. The obtained results showed that in the hybrid models the values of Magnitude of Relative Error (MRE) and Mean Magnitude of Relative Error (MMRE) were reduced compared with COCOMO model and also the accuracy of Percentage of Relative Error Deviation (PRED) was higher in the hybrid models.

Software Effort Estimation Using Multilayer Perceptron and Adaptive Neuro Fuzzy Inference System

Accurate software effort estimation has a big importance for software companies for the reason that management of the project, control of the project, financial matters and timely deliveries are achieved with effort estimation. Thus, effort estimation plays vital role for software companies. In this study, software effort estimation is predicted by using Multilayer Perceptron and Adaptive Neuro Fuzzy Inference System. As a dataset, NASA 93 with 93 projects and Desharnais with 77 projects are used. The results show that Mean Magnitude Relative Error of Adaptive Neuro Fuzzy Inference System is lower than Multilayer Perceptron. In addition, it is seen that PRED(0.25) value of Adaptive Neuro Fuzzy Inference System is higher than Multilayer Perceptron. Thus, performance of Adaptive Neuro Fuzzy Inference System is better when compared to performance of Multilayer Perceptron.

A HYBRID INTELLIGENT MODEL FOR SOFTWARE COST ESTIMATION

Accurate software development effort estimation is critical to the success of software projects. Altho ugh many techniques and algorithmic models have been developed and implemented by practitioners, accurate software development effort prediction is still a c hallenging endeavor in the field of software engine ering, especially in handling uncertain and imprecise inpu ts and collinear characteristics. In this study, a hybrid intelligent model combining a neural network model integrated with fuzzy model (neuro-fuzzy model) has been used to improve the accuracy of estimating sof tware cost. The performance of the proposed model i s assessed by designing and conducting evaluation wit h published project and industrial data. Results ha ve shown that the proposed model demonstrates the ability of improving the estimation accuracy by 18% bas ed on the Mean Magnitude of Relative Error (MMRE) criterion.

Interval kernel regression

Kernel regression is more attractive when it is not possible to determine explicit parametric form of the model and moreover, it does not depend on probabilistic distribution. This paper introduces kernel regression in which the input data set is described by interval-value variables. Two model families are considered. The first family estimates the bounds of the intervals regarding either a smooth function for center variables of the intervals (first model) or two smooth functions for range and center variables, respectively (second model). The second family performs the estimates of the intervals based on regression mixtures. These mixtures assume either a smooth function for center variables and a linear function based on least squares for range variables (third model) or a smooth function for range variables and a linear function for center variables (fourth model). The predictions of the lower and upper bounds of new intervals are computed and two different simulation studies are carried out to validate these predictions. Five real-life interval data sets are also considered. The prediction quality is assessed by a mean magnitude of relative error calculated from a test data set.

A novel fuzzy based approach for effort estimation in software development

Accurate and credible software effort estimation is always a challenge for academic research and software industry. In the beginning, estimation was carried out using only human expertise or algorithmic models, but more recently, interest has turned to a range of Soft Computing techniques. New paradigms such as Fuzzy Logic enable a choice for software effort estimation. Constructive Cost Model (COCOMO) is considered to be the most widely used model for effort estimation. Effort drivers have immense influence on COCOMO and this paper investigates the role of cost drivers (effort features) in improving the precision of effort estimation using Fuzzy Logic. Fuzzy logic-based estimation models are more appropriate when indistinct and incorrect information is to be used. This paper aims at estimating effort in an efficient way using a Fuzzy technique. For this purpose, the COCOMO81 dataset and the Fuzzy Inference System (FIS) of MATLAB are used for implementation. At the end, the outcomes are compared against traditional methods using parameters like Mean Magnitude of Relative Error (MMRE) and Pred (25).

Performance Evaluation of Regression Techniques for Effort Estimation

Software effort estimation assesses the quantity of work required to develop a software project. It is a well known fact that the software industry is unable to give proper an estimate of effort, time and development cost and this is described in reports in various reports including those from project management consultancy companies through case studies on failed projects, and surveys. In this paper, we propose to investigate the Mean Magnitude Relative Error (MMRE) and Median Magnitude Relative Error (MdMRE) using various techniques such as M5, Linear regression, SMO Polykernel and RBF kernel. The dataset COCOMO is used for the investigations. General Terms Performance Evaluation, Regression analysis, Algorithms.

Robust regression with application to symbolic interval data

This paper presents a robust regression model that deals with cases that have interval-valued outliers in the input data set. Each interval of the input data is represented by its range and midpoint and the fitting to interval-valued data is not sensible in the presence of midpoint and/or range outliers on the interval response. The predictions of the lower and upper bounds of new intervals are performed and simulation studies are carried out to validate these predictions. Two applications with real-life interval data sets are considered. The prediction quality is assessed by a mean magnitude of relative error calculated from a test data set.

Investigating Effort Prediction of Software Projects on the ISBSG Dataset

Many cost estimation models have been proposed over the last three decades. In this study, we investigate fuzzy ID3 decision tree as a method for software effort estimation. Fuzzy ID software effort estimation model is designed by incorporating the principles of ID3 decision tree and the concepts of the fuzzy settheoretic; permitting the model to handle uncertain and imprecise data whenpresenting the software projects. MMRE (Mean Magnitude of Relative Error) and Pred(l) (Prediction at level l) are used, as measures of prediction accuracy, for this study. A series of experiments is reported using ISBSG software projects dataset. Fuzzy trees are grown using different fuzziness control thresholds. Results showed that optimizing the fuzzy ID3 parameters can improve greatl y the accuracy of the generated software cost estimate.

Increasing the accuracy of software development effort estimation using projects clustering

Software development effort is one of the most important metrics that must be correctly estimated in software projects. Analogy-based estimation (ABE) and artificial neural networks (ANN) are the most popular methods used widely in this field. These methods suffer from inconsistent and irrelevant projects that exist in the software project datasets. In this paper, a new hybrid method is proposed to increase the accuracy of development effort estimation based on the combination of fuzzy clustering, ABE and ANN methods. In the proposed method, the effect of irrelevant and inconsistent projects on estimates is decreased by designing a new framework, in which all the projects are clustered. The quality of training in ANN and the consistency of historical data in ABE are improved using the proposed framework. Two large and real datasets are utilised in order to evaluate the performance of the proposed method and the obtained results are compared to eight other estimation methods. The promising results showed that the proposed method outperformed the other methods on both datasets. The performance metrics of mean magnitude of relative error (MMRE) and the percentage of the prediction (PRED) (0.25) have been improved by average of 51 and 127% in the first dataset, as well as 52 and 94% in the second dataset.

An analogy based estimation framework for design rework efforts

Analogy based estimation framework is developed aiming to estimate design rework efforts which are unnecessary repetition design efforts. Design rework factors were captured from literature reviews and then they were developed to be design rework drivers by mapping with results from semi structure interviews with three companies in automotive industries. There are six design rework drivers for probability occurrence, and there is only one driver for efforts required. Design structure matrix is used to identify interactions and addressed a key critical component or subsystem. The analogy based estimation framework is applied with the water pump development project from an engine manufacturer. The initial statistical validation method called mean magnitude of relative error is implemented and the results are in acceptable range, while experts from company C agreed on the validity of design rework drivers and opportunity to improve the framework. The ultimate goal of this development is to develop a framework to estimate design rework efforts in the product design and development projects at the early stage.

Accuracy Comparison of Data Imputation Estimation Methods between the Unconstrained Structural Equation Modeling and K-Nearest Neighbors

This study aimed to the accuracy comparison of data imputation estimation methods between the unconstrained structural equation modeling (Uncon-SEM) and k-nearest neighbors (K-NN). The measurement accuracy of the model based on the mean magnitude of relative error (MMRE). The model is developed by using the online database from University of California, Irvine (UCI) which is a data set on waveform generators. Indicators 21 (1,200 sets) methods were as follows: 1) Data set was divided into two groups (experimental group of 1,000 sets and test group of 200 sets); 2) The experimental group was analyzed by three main factors (F1, F2, F3); 3) Uncon-SEM method: It created a SEM with three main factors, then the remaining factors to be created new the relationships with the unconstrained approach and created new SEM. The test data was substituted in the equation to find the MMRE which was 34.29% (accuracy was 65.71%); 4) K-NN method: It selected the main factor was the relationship of the missing data (F2). Measure the Euclidean distance between the test group and experimental group and selected 5 (K=5) of data sets were nearest to the missing data for the estimate by mean. The MMRE which was 57.00% (accuracy was 43.00%). Thus, comparing estimates of missing data showed that using the Uncon-SEM method were more accuracy, and MMRE declined about 22.71% than K-NN method.

A Comparison of Accuracy Based on Data Imputation between Unconstrained Structural Equation Modeling and Weighted Least Squares Regression

This research proposed a comparison of accuracy based on data imputation between unconstrained structural equation modeling (Uncon-SEM) and weighted least squares (WLS) regression. This model is developed by University of California, Irvine (UCI) and measured using the mean magnitude of relative error (MMRE). Experimental data set is created using the waveform generator that contained 21 indicators (1,200 samples) and divided into two groups (1,000 for training and 200 for testing groups). In fact, training group was analyzed by three main factors (F1, F2, and F3) for creating the models. The result of the experiment show MMRE of Uncon-SEM method based on the testing group is 34.29% (accuracy is 65.71%). In contrast, WLS method produces MMRE for testing group is 55.54% (accuracy is 44.46%). So, Uncon-SEM is high accuracy and MMRE than WLS method that is 21.25%.