Determining grapevine cordon shape for automated green shoot thinning using semantic segmentation-based deep learning networks

Abstract

Canopy management practices are carried out annually in vineyards for establishing and maintaining healthy canopies. Green shoot thinning is an integral part of canopy management practices in wine grapes (Vitis vinifera), used to reduce crop load to desired level for optimizing wine quality. Mechanical thinning can reduce labor requirements by 25 times compared to manual operation. However, due to difficulty in adjusting position and orientation of thinning end-effector to the shape of the cordons, cluster removal efficiency with mechanical green shoot thinning varies from 10 to 85%. Automating mechanical thinning could help to substantially increase its efficiency and performance. For performing an automated operation, the first step is to determine the shapes of vine cordons. In this work, methods were investigated to accurately determine the cordon shapes using deep learning networks in natural environment of commercial vineyards. A color camera was used to acquire canopy images, and different deep learning-based semantic segmentation techniques (SegNet and FCN) were used to do this cordon detection and determination work. Results show that the FCN-based model initialized from VGG16 weights (FCN-VGG16) achieved the highest Boundary-F1 score compared to the same with other networks investigated (SegNet-VGG16, SegNet-VGG19, and FCN-AlexNet). Following the cordon segmentation, different mathematical models (Polynomial, Gaussian, Fourier, and sum of sines) were fitted on the cordon segments obtained from FCN-VGG16 network. The results showed that a polynomial model of 6th degree could fit about 80% of cordons trajectories with an R-square value of 0.98 and more. This model could be used to determine cordon trajectories in field operations when the cordons were heavily occluded by shoots/leaves to precisely position and orient thinning end-effectors/rollers for automated green shoot thinning.