• A multi-scale feature extraction module (MFE) is designed by using multi-layer dilated convolution to obtain the multi-scale feature information of pest image with a large receptive field without increasing the amount of model parameters. • A deep feature extraction module (DFE) is proposed. Firstly, the deep feature information of pest image is obtained by multiple convolution operations, and then the obtained deep feature image is restored to the size of the original image by deconvolution. At the same time, the deep and shallow feature information are fused by multiple skip-connection structure ( Zhang et al., 2019 ) to reduce the loss of feature information. • We propose a network model based on multi-scale feature fusion to accurately identify and classify crop pests. Experiments on 12 types of pest datasets show that our method has better performance in pest identification and classification than other advanced methods, its classification accuracy (ACC) reaches 98.2%, and the model training time is only 197 min. Crop diseases and insect pests are a serious natural disaster, which needs to be predicted and monitored in time to ensure the output of crops. Due to the wide variety of pests and the similar morphology of crops in the early stages of growth, it is difficult for agricultural workers to accurately identify various types of pests. Crop insects have brought huge challenges to the prevention and control of plant diseases and insect pests. In response to this problem, we propose a way of classification of crop pests based on multi-scale feature fusion(MFFNet) to accurately recognizes and classifies crop pests. First, the multi-scale feature extraction module (MFE) is designed by using dilated convolution to obtain the multi-scale feature map of the pest image. At the same time, extracted the deep feature information of the image by the feature extraction module (DFE). Finally, the features extracted separately by the multi-scale feature extraction module (MFE) and the feature extraction module (DFE) were fused thus achieving accurately classified and identified the crops insects by the way of end-to-end. Experiments show that our proposed method has obtained excellent classification performance on the dataset of 12 types of pests, its classification accuracy rate (ACC) reached 98.2%.