Real-time defects detection for apple sorting using NIR cameras with pruning-based YOLOV4 network

Abstract

In order to realize on-line detection of defective apples on a two-lane fruit sorting machine, an inspection module was constructed using NIR cameras and a diffuse illumination chamber. A real-time apple defects inspection method was proposed based on YOLO V4 deep learning algorithm. The input images were generated by combining NIR images in three consecutive rubber roller stations. Channel pruning and layer pruning methods were used to simplify the YOLO V4 network and accelerate the detection speed. A non-maximum suppression (NMS) method based on L1 norm is proposed to remove redundant prediction box after fine-tuning the pruned network. The test results indicated that the model size and inference time of the pruning-based YOLO V4 network was decreased by 241.24 megabyte (MB) and 10.82 ms, respectively, and the mean average precision (mAP) was increased from 91.82% to 93.74%, compared with the YOLO V4 network before pruning. The pruning-based YOLO V4 network based on NIR images was not affected by the variation of skin color and suitable for detects identification of different cultivars including ‘Fuji’ apple covered in red-yellow striping and red blush, ‘Golden Delicious’, and ‘Granny Smith’, with the average detection accuracy of 93.9% at the on-line test assessing five fruit per second. The overall results showed that the proposed pruning-based YOLO V4 network combined with the developed inspection module, had great potential to be implemented in commercial fruit packing line for fruit defects identification.