Improving blueberry firmness classification with spectral and textural features of microstructures using hyperspectral microscope imaging and deep learning

Abstract

Postharvest blueberry softening was investigated by observing microstructural changes of blueberry in spectrum and texture using hyperspectral microscope imaging and deep learning technology. More specifically, textural features were examined from grey-level co-occurrence matrix (GLCM) computed from hypercube of blueberry cells. GLCMs predetermined with nine different pixel distances and four orientations were extracted from three input image sources (i.e., single-band images of 530 nm, 680 nm, and double-band images pairing of these two bands). The optimum GLCM features were mean, variance, homogeneity, contrast, dissimilarity, entropy, energy, and correlation. With these GLCM features, the parenchyma cell textures were visually and statistically characterized over different firmness. To confirm the effectiveness of textural features, Fusion-Nets combining 1D-CNN for spectra and ResNet50 for GLCMs were trained and evaluated with four different distances (i.e., 16, 32, 64, 128 in pixels) and three image sources. According to the results of textural feature analysis, contrast, entropy, variance, dissimilarity, homogeneity, and energy were different (p < 0.05) over two firmness categories (1.96–3.92 N and 3.92–9.81 N in shear force) for average GLCMs with 16–64 pixel distance calculated from a single-band image source (i.e., 530 nm or 680 nm band images). A Fusion-Net with spectra of cell walls and GLCMs with 64-pixel distance from 680 nm band images distinguished the firmness categories with 95% classification accuracy and 90% Matthew's correlation coefficient (MCC), which outperformed the previous Fusion-Net with spectra and band images, which were 85% test accuracy with 73% test MCC. While implying a close relationship between blueberry softening and textural change in hyperspectral images of blueberry microstructures, these results provide a basis for further research on development of non-destructive methods to measure blueberry firmness with macroscopic imaging platforms.