Machine vision system for real-time debris detection on mechanical wild blueberry harvesters

Abstract

Improving fruit quality is important to wild blueberry producers. Plant debris like leaves and stems caused a reduction in fruit quality during mechanical harvesting. A debris detection system was developed using two Logitech C920 webcam cameras mounted before and after the blower fan on a commercial mechanical wild blueberry harvester. Images were collected from two commercially managed wild blueberry fields located in central Nova Scotia to develop a dataset of 1000 images. Two convolutional neural networks (CNNs), YOLOv3 and YOLOv3-SPP, were used and compared for performance analysis. The CNNs were trained using no augmented images, images with five augmentations (sharpening, brightness, contrast, gamma correction, and saturation), and images with five augmentations that were more heavily weighted towards gamma. Four different computer hardware packages (Hewlett-Packard, Shuttle XPC, Jetson TX2, and GPU-based Desktop) were used to determine the appropriate hardware and CNN model combination for real-time performance. YOLOv3 and YOLOv3-SPP were trained to mAP scores of 72.87 and 74.38% using the dataset with more gamma-augmented images. This was an improvement over the dataset with no augmentation (67.75; 68.49%) and equally numbered augmentations (71.26%; 73.03%). YOLOv3-SPP combined with an Intel® Core™ i9-7900X CPU and Nvidia GeForce RTX™ 2080 Ti GPU in a desktop computer achieved the fastest detection rate of 33.30 ms. The detection speed on all other hardware configurations exceeded 33.30 ms, indicating that a powerful desktop GPU is required for real-time performance of this task. With adequate processing hardware, the developed technology could be integrated into a control system to automatically adjust brushes based on conveyor information from mechanical wild blueberry harvesters to make cleaning berries easier and enhance fruit quality.