Point-wise gated restricted Boltzmann machines using clean data

Abstract

The point-wise gated restricted Boltzmann machine (pgRBM), an RBM variant, finds the task-relevant patterns from data containing irrelevant patterns and thus achieves satisfied classification results. Given that the training data composes of noisy data and clean data, how the clean data is applied to promote the performance of the pgRBM is a problem. To address the problem, this paper proposes a method, named as pgRBM based on random noisy data and clean data (pgrncRBM). The pgrncRBM makes use of RBM and the clean data to obtain the initial values of the task-relevant weights. However, if the noise is an image, the pgrncRBM cannot learn the task-relevant patterns from the noisy data. Therefore, this paper combines the Spike-and-Slab RBM with the pgRBM and proposes a method, named as pgRBM based on image noisy data and clean data (pgincRBM). The pgincRBM uses the spike-and-slab RBM to model the noise, so it can learn the 'clean' data from the data containing image noisy. Finally, we discuss the validities of these methods on MNIST variation datasets.