Efficient construction and convergence analysis of sparse convolutional neural networks

Abstract

Listen

Translate article

In this paper, a new variant of convolutional neural network (CNN) based on L1 regularization is proposed. The main consideration is to generate a sparse weight matrix, i.e., to generate a sparse neural network model. To some extent, L1 regularization prevents the occurrence of overfitting and effectively improves the learning efficiency of the network. Unfortunately, the L1 regularization term is non-smooth, which leads to the occurrence of oscillations in experiments and poses a great challenge to the theoretical analysis. So, we overcome this drawback by using polishing techniques and rigorously prove the monotonicity of the error function and the strong and weak convergence theorems of the algorithm. Finally, numerical simulations on several data sets support our theoretical results and the superiority of the proposed algorithm.