Supporting table grape berry thinning with deep neural network and augmented reality technologies

Abstract

Berry thinning is a crucial process in table grape cultivation. Such visual features as bunch compactness, bunch form, and berry size are important factors affecting market value. Moreover, sufficient space for each berry to grow also largely influences the final product’s quality, such as the sugar concentration. Berry thinning requires professional skills, and it is usually accomplished only by experienced farmers. Furthermore, the appropriate period for berry thinning is limited to two weeks; hence, berry-thinning tasks have led to a bottleneck in terms of increasing the yield of table grape products. This paper addresses the aforementioned issue by proposing a system for empowering unskilled farmers to begin berry thinning without in-person coaching from expert farmers. The proposed system employs a deep neural network model to learn the knowledge required for identifying berries to be removed, and it uses augmented reality technology to display instructions based on this knowledge to naïve farmers through smart glasses. The proposed system was validated throughout the entire growing season in a real table grape field in Yamanashi Prefecture, Japan. It was confirmed that unskilled farmers can execute berry-thinning tasks immediately without training. Furthermore, they can become familiar with the proposed system quickly. The grape products from unskilled farmers who used the proposed system also had an 8.18 % higher average quality score than those from the skilled farmers.