DBM Optimization with Additional Category Information

Abstract

For unconstrained training of restricted Boltzmann machine (RBM), it is easy to appear that feature homogenization leads to poor generalization ability. This paper introduces category conditions into DBM training, proposes label conditional RBM which is used in the construction of DBM. According to unsupervised training characteristics of RBM, this paper adds the category information as the model hidden unit training condition to the implicit unit posterior activation probability calculation. This paper applies the model as the underlying structure of the deep Boltzmann machine (DBM) to the deep network construction. Through handwritten digit recognition set test, compared with the shallow model, the new model after adding the category condition has a great improvement in the model training speed and feature extraction effectiveness, and can effectively enhance the feature learning ability of the deep model.