A structure-self-organizing DBN for image recognition

Abstract

A structural self-organized DBN (S-DBN) is proposed in this paper to improve the ability of feature learning in unsupervised training. In S-DBN, the strategy of dropout is designed for unsupervised learning to reduce inner cooperation between feature detectors. Then, the regularization-reinforced transfer function is put forward, in order to further reduce the insignificant weights, and to raise the abilities of feature learning and generation. The fast training method of contrastive divergence is designed, and backpropagation is used in supervised training. Finally, two experiments on regression and classification using MNIST show that S-DBN has better generation and faster convergence rate than other methods, in particular, in regression experiment, the proposed model beats traditional DBN by 1.50%; in image classification, the proposed model achieves smaller testing error in much less computing time.