Fully convolutional networks for blueberry bruising and calyx segmentation using hyperspectral transmittance imaging

Abstract

Early detection of internal bruises in blueberries is a significant challenge for the blueberry industry. The main goal of this study was to detect blueberries’ internal bruising accurately, after mechanical damage from hyperspectral transmittance images (HSTIs), using the deep learning-based method of fully convolutional networks (FCNs) for segmentation tasks. To improve detection accuracy, a total of three classes (bruised tissue, unbruised tissue, and calyx end of blueberries), were treated as segmentation targets. A near-infrared hyperspectral imaging system was used to acquire transmittance images of 1200 blueberries, and the images were divided randomly to form training, validation and testing sets. Three categories of input HSTIs were used to evaluate the FCN models using pre-trained weights (transfer learning) and random initialisation. Random forests and linear discriminant analysis were applied to generate 9-channel and 3-channel input images along with full-wavelength multi-channel images. The results indicate that when using the deep learning approach, blueberry bruises and calyx ends can be segmented from the blueberry fruit as early as 30 min after mechanical damage has been inflicted on the blueberries. The new full-wavelength model with random initialisation had the highest accuracy (81.2% over the entire test set), and can be used to research the resistance of blueberry fruit to mechanical damage. The new 3-channel and 9-channel models show potential for application to packing-line detection and online inspection.