Research on compression pruning methods based on deep learning

Abstract

In the past decade or so, deep neural networks have continuously refreshed the best performance in tasks such as computer vision and natural language processing, and have become the most concerned research directions. Although deep neural networks have significant performance, they are difficult to deploy on hardware-constrained embedded and mobile devices due to their huge number of parameters, computational and storage costs. Therefore, in recent years, the lightweight design of network structure has gradually become a cutting-edge and popular direction, improving the operation speed and optimizing the storage space under the premise of maintaining the accuracy of the neural network. In this paper, the results achieved by domestic and foreign scholars in deep learning model compression are sorted and classified, and the methods of network pruning, model quantification, knowledge distillation, and lightweight network design are summarized, and the compression effect of related methods on known public models is summarized. Finally, the possible directions, application directions and development trends of future neural network model lightweight research are prospected.