Deep learning-based accurate grapevine inflorescence and flower quantification in unstructured vineyard images acquired using a mobile sensing platform

Abstract

Early grapevine yield forecasting at satisfactory accuracy is among the major trends in precision viticulture research. Conventionally, yield is estimated manually through extrapolation right before harvest, which is mostly inaccurate, requires considerable labor as well as resources, and is often destructive. The number of flowers per vine is one of the main determinants of grapevine yield and can be used as an early indicator for viticulture yield forecasting. In the present study, a non-invasive, automated image analysis framework was proposed for quantifying flowers in unstructured RGB images of grapevine canopies. Images were automatically acquired under field conditions at night, using a mobile sensing platform equipped with artificial illumination. Due to the small shape and dense distribution of individual flowers (a few hundred per inflorescence) and the similar hue of all plant organs in the fore- and background, efficient flower quantification in images is challenging. To overcome this, we adopted a two-step segmentation approach in our algorithm. First, image regions containing inflorescences were recognized and extracted by segmenting each instance of inflorescences in the image using a mask region-based convolutional neural network (Mask R-CNN), followed by test-time augmentation post-processing to achieve high accuracy. Finally, individual flowers in the extracted inflorescences were detected and quantified using another Mask R-CNN model preceded by a contrast enhancement pre-processing operation. For efficient segmentation and quantification of individual flowers, a high-resolution full image was split down into smaller patches and processed in multiple iterations during inference. The algorithm yielded significant performances, with F1 score values of 0.943 and 0.903 for inflorescence segmentation and single flower detection tasks, respectively, against a test set of 75 images from three different cultivars. A determination coefficient (R2) of 0.98 and a normalized root mean square error of 12.24% were obtained in the test set between the automatic flower number quantification and manual counts. In conclusion, the proposed algorithm constitutes a promising approach for automatically predicting yield potential in the early stages of grapevine development, and it can be used for objective monitoring and optimal management of commercial vineyards.