Quantitative software project management with mixed data: A comparison of radial, nonradial, and ensemble data envelopment analysis models

Abstract

AbstractData envelopment analysis (DEA) models are often used for benchmarking software projects, and traditional DEA models only allow for the use of continuous variables. This study considers the use of DEA for datasets with mixed continuous and discrete variables to ranking software projects. It uses the existing radial DEA model and extends the nonradial DEA model to allow for the use of mixed variables. Further efficiency scores from the two DEA models are averaged in an ensemble DEA score. Using three real‐world software engineering datasets, this study finds that the nonradial DEA and the ensemble DEA models have better discriminating power (lower tied efficiency scores) to rank software projects, and the radial DEA model generates more general ranking distribution (higher entropy) of normalized efficiency scores. The choice between selecting radial and nonradial DEA models for ranking software projects appears to depend on the extent to which managers want to introduce bias into the efficiency score ranking distribution. Radial models appear to have a lower bias than nonradial models. The ensemble DEA model appears to be the best performing DEA model for datasets containing two or more discrete and continuous variables.