Development of a computer vision approach as a useful tool to assist producers in harvesting yellow melon in northeastern Brazil

Abstract

This paper presents a Computer Vision (CV) approach to harvest decision of yellow melon (hybrid Natal®) based on prediction of Soluble Solids Content (SSC, as °Brix) from digital image. At this point, it is worth remembering that the minimum SSC for harvesting this type of melon is 9°Brix. In this context, melons with SSC ≥ 9°Brix should be classified as “suitable for harvesting” (SFH), whereas melons with SCC < 9°Brix should be classified as “unsuitable for harvesting” (UFH). Nonetheless, the visual decision of this harvest point is difficult due to the uniform yellow color of this melon's rind. To circumvent this problem and ensure quality (SSC ≥ 9°Brix), growers use the practice of taking pre-harvest melon samples to measure SSC by refractometry. However, this practice presents two problems: as it analyzes a limited number of samples, the result does not reflect the totality of the fruits; and because it is destructive, it causes the loss of marketable melons. Based on digital images, the hypothesis of this work states that it is possible to develop a non-destructive CV-based technique, which will allow growers to decide in real time whether each melon is suitable for harvesting or not. From the foregoing, the aim was to develop a simple, fast and inexpensive CV technique, based on the texture differences in the yellow color of this melon's rind, able of analyzing SSC (as °Brix) of all melons at harvest moment, classifying them as SFH or UFH. For this purpose, we used a digital camera to capture the image and a portable refractometer to quantify the SSC of each melon (n = 144). Melons were then divided into two classes with 72 melons each - SFH (SSC ≥ 9° Brix) or UFH (SSC < 9° Brix). In each image, we manually selected three (3) small regions (205 × 205 pixels), totaling 432 images (Image Database). From Image Database, 302 images (70%) were used for training, being one half (156 images) from SFH and the other half of the UFH. For the test, we used 130 images (30%) of the Image Database, being 67 images from SFH and 63 from UFH. Based on color filters (RGB average, Channel H, and Channel Y), textures (using Local Binary Pattern, LBP) and two classifiers (KNN or MLP), the developed CV-technique proved useful to predict SSC still in the field, as well as to classify melons into the two-kwon classes (SFH or UFH). The classifiers' performance has been verified by confusion matrix associated with the Receiver Operating Characteristics (ROC) analysis. The MLP obtained 95% accuracy, while the KNN obtained 94%. In addition, the combination of MLP (classifier) with the RGB average (color filter) presented the highest hits (accuracy), as well as the lowest false positive values. From the results obtained in this work, it is possible to conclude that the developed CV-method is useful for growers to classify yellow melon Natal® at the harvest moment.