Reinforced Fuzzy Clustering-Based Ensemble Neural Networks

Abstract

In this paper, we propose reinforced fuzzy clustering-based ensemble neural networks (FCENNs) classifier. The objective of this paper is focused on the development of the design methodologies of ensemble neural networks classifier for constructing the network structure and enhancing the learning methods of fuzzy clustering-based neural networks through the combination of the probabilistic model and its learning mechanism. The proposed FCENNs classifier takes into consideration a cross-entropy error function to improve learning while $L_2$ -norm regularization is used to reduce overfitting as well as enhance generalization abilities. The essential points of the proposed reinforced FCENNs classifier can be enumerated as follows: First, in the proposed classifier, the cross-entropy error function is used as a cost function; to do this, a softmax function is applied to represent a categorical distribution located at the nodes of the output layer. Second, the learning mechanism is composed of two parts. First, fuzzy C-means clustering forms the connections (weights) of the hidden layer while the connections of the output layer are adjusted with the aid of the nonlinear least squares method using Newton's method-based learning. Third, $L_2$ norm-regularization is considered to avoid the degradation of generalization ability caused by overfitting. The learning mechanism similar to ridge regression is realized by adding $L_2$ penalty term to the cross-entropy error function. From the viewpoint of performance improvement achieved through the proposed novel learning method, the design methodology for the ensemble neural networks classifier is discussed and analyzed with the aid of a diversity of two-dimensional synthetic data and machine learning datasets.