Novel Multi-Perspective Usability Evaluation Framework for Selection of Open Source Software Based on BWM and Group VIKOR Techniques

Abstract

Increasing demand for open-source software (OSS) has raised the value of efficient selection in terms of quality; usability is an essential quality factor that significantly affects system acceptability and sustainability. Most large and complex software packages partitioned across multiple portals and involve many users — each with their role in the software package; those users have different perspectives on the software package, defined by their knowledge, responsibilities, and commitments. Thus, a multi-perspective approach has been used in usability evaluation to overcome the challenge of inconsistency between users’ perspectives; the inconsistency challenge would lead to an ill-advised decision on the selection of a suitable OSS. This study aimed to assist the public and private organizations in evaluating and selecting the most suitable OSS. The evaluation of the OSS software packages to choose the best one is a challenging task owing to (a) multiple evaluation criteria, (b) criteria importance, and (c) data variation; thus, it is considered a sophisticated multi-criteria decision making (MCDM) problem; moreover, the multi-perspective usability evaluation framework for OSS selection lacks in the current literature. Hence, this study proposes a novel multi-perspective usability evaluation framework for the selection of OSS based on the multi-criteria analysis. Integration of best-worst method (BWM) and VIKOR MCDM techniques has been used for weighting and ranking OSS alternatives. BWM is utilized for weighting of evaluation criteria, whereas VIKOR is applied to rank OSS-LMS alternatives. Individual and group decision-making contexts, and the internal and external groups aggregation were used to demonstrate the efficiency of the proposed framework. A well-organized algorithmic procedure is presented in detail, and a case study was examined to illustrate the validity and feasibility of the proposed framework. The results demonstrated that BWM and VIKOR integration works effectively to solve the OSS software package benchmarking/selection problems. Furthermore, the ranks of OSS software packages obtained from the VIKOR internal and external group decision making were similar; the best OSS-LMS based on the two ways was ‘Moodle’ software package. Among the scores of groups in the objective validation, significant differences were identified; this indicated that the ranking results of internal and external VIKOR group decision making were valid, which pointed to the validation of the framework.