Non-overlapping Rules Research Articles

An extended belief rule-based system with hybrid sampling strategy for imbalanced rule base

Imbalanced dataset is an important focus for classification. As the mainstream of addressing imbalanced dataset, data-level methods are trapped in facing difficultly-determined subjective parameters, and the inconsistency between new minority samples and original minority samples simultaneously. To address it, this paper develops an extended belief-rule-based (EBRB) system with hybrid sampling strategy, which is a white-box classifier. The hybrid sampling strategy is composed of an under-sampling process and an oversampling process, in which subjective parameters are not involved. The under-sampling is to identify and remove overlapping majority rules by iteratively determining an appropriate objective threshold for calculating the inconsistency degree of rule base, and to determine and remove redundant non-overlapping majority rules by using the density of non-overlapping rules in clustered groups. The oversampling is to design a differential evolution based iterative process to generate new minority rules in groups by minimizing the inconsistency of rule base. The distribution of original dataset is maintained extremely by balancing rules in clustered majority and minority groups, respectively. This EBRB system is used for the auxiliary diagnosis of thyroid nodules, and its superior performance is highlighted by the comparison with existing EBRB systems, representative data-level methods, and algorithm-level methods.

Interpretable Decision Sets: A Joint Framework for Description and Prediction.

One of the most important obstacles to deploying predictive models is the fact that humans do not understand and trust them. Knowing which variables are important in a model's prediction and how they are combined can be very powerful in helping people understand and trust automatic decision making systems. Here we propose interpretable decision sets, a framework for building predictive models that are highly accurate, yet also highly interpretable. Decision sets are sets of independent if-then rules. Because each rule can be applied independently, decision sets are simple, concise, and easily interpretable. We formalize decision set learning through an objective function that simultaneously optimizes accuracy and interpretability of the rules. In particular, our approach learns short, accurate, and non-overlapping rules that cover the whole feature space and pay attention to small but important classes. Moreover, we prove that our objective is a non-monotone submodular function, which we efficiently optimize to find a near-optimal set of rules. Experiments show that interpretable decision sets are as accurate at classification as state-of-the-art machine learning techniques. They are also three times smaller on average than rule-based models learned by other methods. Finally, results of a user study show that people are able to answer multiple-choice questions about the decision boundaries of interpretable decision sets and write descriptions of classes based on them faster and more accurately than with other rule-based models that were designed for interpretability. Overall, our framework provides a new approach to interpretable machine learning that balances accuracy, interpretability, and computational efficiency.

A Hyper-Heuristic for Descriptive Rule Induction

There are in general three approaches to rule induction: exhaustive search, divide-and conquer, and separate-and-conquer (or its extension as weighted covering). Among them, the third approach, according to different rule search heuristics, can avoid the problem of producing many redundant rules (limitation of the first approach) or non-overlapping rules (limitation of the second approach). In this chapter, we propose a hyper-heuristic to construct rule search heuristics for weighted covering algorithms that allows producing rules of desired generality. The hyper-heuristic is based on a PN space, a new ROC-like tool for analysis, evaluation, and visualization of rules. Well-known rule search heuristics such as entropy, Laplacian, weight relative accuracy, and others are equivalent to ones proposed by the hyper-heuristic. Moreover, it can present new non-linear rule search heuristics, some are especially appropriate for description tasks. The non-linear rule search heuristics have been experimentally compared with others on the generality of rules induced from UCI datasets and used to learn regulatory rules from microarray data.