Ensemble belief rule base modeling with diverse attribute selection and cautious conjunctive rule for classification problems

Abstract

Belief rule-based systems have demonstrated its advantages in solving complicated problems with uncertain information. However, the rule combinatorial explosion problem is still a great challenge for belief rule bases (BRBs) when a problem involves a large number of attributes, because existing attempts have not addressed this challenge adequately, e.g., utilization of single attribute selection method to downsize BRBs without considering its inherent weakness, or adjustment of referential values to optimize BRBs without attribute selection. Thus, inspired by ensemble learning, the objective of this paper is to propose an ensemble BRB modeling method to deal with classification problems. First, six attribute selection methods that have different advantages are introduced to select diverse sets of antecedent attributes for constructing multiple BRBs, and all of these BRBs are further trained by parameter learning for diverse belief rule-based systems. Second, due to the fact that each belief rule-based system has different importance and hardly satisfies the assumption of independence, a weight learning method is proposed to determine the weight of each belief rule-based system, and a new analytical cautious conjunctive rule (CCR) is deduced from the recursive CCR, that is suitable for the combination of non- independent individuals, to combine the outputs of all belief rule-based systems. Eight classification datasets from the well- known UCI database are adopted to verify the effectiveness of the proposed BRB modeling method in comparison with the belief rule-based systems constructed by single attribute selection, conventional fuzzy rule-based classifiers, and machine learning-based classifiers.