Unsupervised Pre-training Classifier Based on Restricted Boltzmann Machine with Imbalanced Data

Abstract

Many learning algorithms can suffer from a performance bias for classification with imbalanced data. This paper proposes the pre-training the deep structure neural network by restricted Boltzmann machine (RBM) learning algorithm, which is pre-sampled with standard SMOTE methods for imbalanced data classification. Firstly, a new training data set can be generated by a pre-sampling method from original examples; secondly the deep neural network structure is trained on the sampled data and all unlabelled data sets by RBM greedy algorithm, which is called “coarse tuning”. Then the neural networks are fined tuned by BP algorithm. The effectiveness of the RBM pre-training neural network (RBMPT) classifier is demonstrated on a number of benchmark data sets. Compared with only BP classifier, pre-sampling BP classifier and RBMPT classifier, it has shown that pre-training procedure can learn more representations of data better with unlabelled data and has better classification performance for classification with imbalanced data sets.