Comparison of different variants of Restricted Boltzmann Machines

Abstract

Restricted Boltzmann Machines (RBMs) have been developed for a lot of applications in the past few years, and many of its variants have also appeared. In this paper, RBM model and its learning algorithm with contrastive divergence algorithm will be introduced firstly. Then three important variants of RBM are presented in details, which are sparse RBM, discriminative RBM, and the Deep Boltzmann Machines (DBM). All the variants including original RBM are tested on MNIST handwriting digit dataset for classification task. Our empirical results demonstrate the advantage of RBM models and show that compared with other variants, the DBM is the best one in terms of the classification accuracy.