Convolutional Neural Network Based on Principal Component Analysis Initialization for Image Classification

Abstract

One kind of Deep Learning models-convolutional neural network, which can reduce the complexity of network structure and the number of parameters to be determined through local receptive fields, weight sharing and pooling operation has achieved state of art results in image classification problems. But this model has gradient diffusion problem, which can cause slow updating of the underlying parameters during the process of training. To solve the problem above and make improvements, this paper presents a model of convolutional neural network based on principal component analysis initialization for image classification. Principal component analysis is usually used to reduce the dimension of the raw input images and the complexity of calculating. This paper proposes a use of principal component analysis to extract eigenvectors without supervision and initialize the convolutional kernels, which is combined with the training process of the convolutional neural network. Such kind of initialization values contains image information and reduces the effect of gradient diffusion problem due to the bad initial parameters. According to the image classification experiments on Mnist and Cifar-10 datasets, the model proposed in this paper reduces the processes of iteration and optimization. It also has simple structure as well as less training time compared with the models of traditional convolutional neural network and using Auto-Encoders to initialize.