Evaluating the performance of open source software projects using data envelopment analysis

Abstract

PurposeThe purpose of this paper is to develop and test a model of the relative performance of open source software (OSS) projects.Design/methodology/approachThis paper evaluates the relative performance of OSS projects by evaluating multiple project inputs and multiple project outputs by using a data envelopment analysis (DEA) model. The DEA model produces an efficiency score for each project based on project inputs and outputs. The method of producing an efficiency score is based on the convex envelopment technology structure. The efficiency measure quantifies a “distance” to an efficient frontier.FindingsThe DEA model produced an index of corresponding intensities linking an inefficient project to its benchmark efficient project(s). The inefficiency measures produced an ordering of inefficient projects. Eight projects were found to be “efficient” and used as benchmarking projects.Research limitations/implicationsThis research is limited to only security‐based OSS projects. Future research on other areas of OSS projects is warranted.Practical implicationsThe result of this research is a practical model that can be used by OSS project developers to evaluate the relative performance of their projects and make resource decisions.Originality/valueThis research extends the work of previous studies that have examined the relative performance of software development projects in a traditional development environment. As a result of this research, OSS projects can now be adequately benchmarked and evaluated according to project performance. An OSS project manger can effectively use these results to critically evaluate resources for their project and judge the relative efficiency of the resources.