Ensemble decision forest of RBF networks via hybrid feature clustering approach for high-dimensional data classification

Abstract

Classification of the high-dimensional data is challenging due to the curse of dimensionality, heavy computational burden and decreasing precision of algorithms. In order to mitigate these effects, feature selection approaches that can determine an efficient subset of features are utilized in the processing. However, most of these techniques attain just one subset of non-redundant features including the best ones. Alternatively, clustering approaches can be used to find the most informative clusters of features instead of generating just a single subset. So called, Hybrid Feature Clustering (HFC) method is capable of maximizing the classification accuracy while keeping the amount of redundant features in each cluster low. The patterns of each cluster are classified by a neural tree that employs Radial Basis Function (RBF) for the nodes. Within each neural tree, a hierarchical approach is proposed to transfer the knowledge of synaptic weights from a parent RBF node to each child. A gating network is applied on the forest of these neural trees in order to aggregate the results. By assessing the classification accuracy and the computational complexity on high-dimensional datasets it can be shown that the proposed solution has outperformed the state of the art classifiers. Furthermore, the computational complexity and the convergence of this method are theoretically proven and the robustness analysis under noisy conditions is conducted.