Vision on the bog: Cranberry crop risk evaluation with deep learning

Abstract

Computer vision and AI for smart agriculture have exciting potential in optimizing crop yield while reducing resource use for better environmental and commercial outcomes. The goal of this work is to develop state-of-the-art computer vision algorithms for image-based crop evaluation and weather-related risk assessment to support real-time decision-making for growers. We develop a cranberry bog monitoring system that maps cranberry density and also predicts short-term cranberry internal temperatures. We have two important algorithm contributions. First, we develop a method for cranberry instance segmentation that provides the number of sun-exposed cranberries (not covered by the crop canopy) that are at risk of overheating. The algorithm is based on a novel weakly supervised framework using inexpensive point-click annotations, avoiding time-consuming annotations of fully-supervised methods. The second algorithmic contribution is an in-field joint solar irradiation and berry temperature prediction in an end-to-end differentiable network. The combined system enables over-heating risk assessment to inform irrigation decisions. To support these algorithms, we employ drone-based crop imaging and ground-based sky imaging systems to obtain a large-scale dataset at multiple time points. Through extensive experimental evaluation, we demonstrate high accuracy in cranberry segmentation, irradiance prediction and internal berry temperature prediction. This work is a pioneering step in using computer vision and machine learning for rapid, short-term decision-making that can assist growers in irrigation decisions in response to complex time-sensitive risk factors. Datasets collected over two growing seasons are made publicly available to support further research. The methods can be extended to additional crops beyond cranberries, such as grapes, olives, and grain, where irrigation management is increasingly challenging as climate changes.