A new regularized restricted Boltzmann machine based on class preserving

Abstract

It is known that an Restricted Boltzmann machine (RBM) can be used as a feature extractor to automatically extract data features in a completely unsupervised learning manner. In this paper, we develop a new regularized RBM by adding the class information, referred to as class preserving RBM (CPr-RBM). Specifically, we impose two constraints on RBM to make the class information clearly reflected in extracted features. One constraint can decrease the distance between the features of the same class and the other one can increase the distance between the features of different classes. The two constraints introduce class information to RBM and make the extracted features contain more category information which contributes to a better classification result. Experiments are conducted on MNIST dataset and 20-newgroup dataset, which show that CPr-RBM learns more discriminate representations and outperforms other related state-of-the-art models in dealing with classification problems.