Sparse Restricted Boltzmann Machine Based on Multiobjective Optimization

Abstract

This article proposes an efficient method for Restricted Boltzmann Machine (RBM) to learn sparse feature. Deep learning algorithms are used more and more often. The Deep Belief Network (DBN) model, which is composed of RBM, is considered as one of the most effective deep learning algorithms. RBM or auto-encoder (AE) is the basic model to build deep networks. However, RBM may produce redundant features without any constraints, then much improved RBM were proposed by added a regularization term to control sparsity of hidden units. Most of the proposed algorithms need a parameter to control the sparseness of the code. In this paper, we proposed a multiobjective optimization model to avoid user-defined constant that is a trade-off between the regularization term and the reconstruction error based on SR-RBM. We employ evolutionary algorithm to optimize the distortion function and the sparsity of hidden units simultaneously. Experimental results show that our novel approach can learn useful sparse feature without a user-define constant and it performs better than other feature learning models.