Blackberry Fruit Classification in Underexposed Images Combining Deep Learning and Image Fusion Methods

Abstract

Berry production is increasing worldwide each year; however, high production leads to labor shortages and an increase in wasted fruit during harvest seasons. This problem opened new research opportunities in computer vision as one main challenge to address is the uncontrolled light conditions in greenhouses and open fields. The high light variations between zones can lead to underexposure of the regions of interest, making it difficult to classify between vegetation, ripe, and unripe blackberries due to their black color. Therefore, the aim of this work is to automate the process of classifying the ripeness stages of blackberries in normal and low-light conditions by exploring the use of image fusion methods to improve the quality of the input image before the inference process. The proposed algorithm adds information from three sources: visible, an improved version of the visible, and a sensor that captures images in the near-infrared spectra, obtaining a mean F1 score of 0.909±0.074 and 0.962±0.028 in underexposed images, without and with model fine-tuning, respectively, which in some cases is an increase of up to 12% in the classification rates. Furthermore, the analysis of the fusion metrics showed that the method could be used in outdoor images to enhance their quality; the weighted fusion helps to improve only underexposed vegetation, improving the contrast of objects in the image without significant changes in saturation and colorfulness.