RSFIN: A Rule Search-based Fuzzy Inference Network for performance prediction of configurable software systems

Abstract

Many modern software systems provide numerous configuration options to users and different configurations often lead to different performances. Due to the complex impact of a configuration on the system performance, users have to experimentally evaluate the performance for different configurations. However, it is practically infeasible to exhaust the almost infinite configuration space. To address this issue, various approaches have been proposed for performance prediction based on a limited number of configurations and corresponding performance measurements. Many of such efforts attempt to achieve a reasonable trade-off between experiment effort and prediction accuracy. In this paper, we propose a novel performance prediction model using a Rule Search-based Fuzzy Inference Network (RSFIN) based on ANFIS and NAS. One intuition is that, in systems, similar configurations produce similar performance. We experimentally validate this intuition based on data and introduce a configuration space under entropy. This view suggests the use of RSFIN to capture hidden distributions in configuration space. We implement and evaluate RSFIN using eleven real-world configurable software systems. Experimental results show that RSFIN achieves a better trade-off between measurement effort and prediction accuracy compared to other algorithms. In addition, the results also confirm that the evaluation of configuration space complexity based on data entropy is beneficial.