Infield corn kernel detection using image processing, machine learning, and deep learning methodologies under natural lighting

Abstract

Machine vision has been increasingly used to address agricultural issues. One such case is corn field harvest losses and image-based object detection approaches, namely image processing, machine learning, and deep learning were investigated to detect and count infield corn kernels, immediately after harvest for combine harvester performance evaluation. A hand-held low-cost RGB camera was used to collect images with kernels of different backgrounds, based on which a 420 images dataset (200, 40, and 180 for training, validation, and testing, respectively) was generated. Three different models for kernel detection were constructed based on image processing, machine learning, and deep learning. For the imaging processing method, the images were preprocessed (color thresholding, graying, and erosion), followed by Hough circle detection to identify kernels. For the machine learning (cascade detector) and deep learning (Mask R-CNN, EfficientDet, YOLOv5, and YOLOX), models were trained, validated, and tested. Experimental results showed the overall performance of the deep learning network YOLOv5 was superior to the other approaches, with a small model size (89.3 MB) and a high model average precision (78.3 %) for object detection. The detection accuracy, undetection rate and F1 value were 90.7 %, 9.3 %, and 91.1 %, respectively, and the average detection rate was 55 fps. This study demonstrates that the YOLOv5 model has the potential to be used as a real-time, reliable, and robust method for infield corn kernel detection.