Exploring three-way group decisions with consensus evolution network for software ecosystem hierarchical criteria health rating

Abstract

The appropriate rating of software ecosystem health is to the benefit of its sustainable development. The rating method based on an expert panel with specialised knowledge is reliable for rating software ecosystem health. Complexity often causes obstacles for expert panels to give appropriate ratings. Facing complexity, hierarchical criteria system provides evaluation guides with multiple levels and multiple perspectives. However, at present, effective group decision-making under hierarchical criteria system remains a challenge. Thus, this article systematically investigates consensus reaching process and rating method under hierarchical criteria system of software ecosystem health, which are two important components of group decision-making. To master the consensus situation of all criteria and consensus relationships of pair-wise experts under hierarchical criteria system, we construct hierarchical consensus evolution network. Then, we further propose a novel consensus degree measurement and design a consensus reaching process with pair-wise adjustment by utilising hierarchical characteristics. For rating software ecosystem health efficiently, there are usually three ratings including health, sub-health and ill-health. Three-way decisions exactly provide semantic interpretation for these three ratings. Meanwhile, three-way decisions can reduce the risk of incorrect ratings. Considering these advantages, we develop three-way group decisions with hierarchical consensus evolution network to comprehensively rate software ecosystem health under hierarchical criteria system. Finally, take the health rating of the software ecosystem on GitHub as an example, we develop a series of experiment analysis to verify the effectiveness of the proposed method.