Evolutionary Multiobjective Design of Fuzzy Rule-Based Classifiers

Abstract

The main goal in classifier design has been accuracy maximization on unseen patterns [23]. A number of learning algorithms have been proposed to minimize the classification errors on training patterns in various fields such as neural networks [84], fuzzy systems [76] and machine learning [81]. It is well-known that neural networks are universal approximators of nonlinear functions [34, 35]. Since fuzzy systems are also universal approximators [70, 73, 92], we can design fuzzy rule-based classifiers that can correctly classify all training patterns. Such a fuzzy rule-based classifier, however, does not usually have high accuracy on test patterns, as shown in Fig. 1, where a typical accuracy-complexity tradeoff relation is depicted. We can decrease the error rate of classifiers on training patterns by increasing their complexity (for example, by increasing the number of fuzzy rules in fuzzy rule-based classifiers), as shown by the dotted curve in Fig. 1. The classification accuracy on test patterns is, however, degraded by increasing the complexity too much, as shown by the solid curve in Fig. 1. Such an undesirable deterioration in the classification accuracy on test patterns is known as ‘overfitting to training patterns’ [23]. Finding the optimal complexity with the maximum accuracy on test patterns (that is, S * in Fig. 1) is one of the main research issues in machine learning, especially in the field of statistical learning theory [10].