Deep learning-based restoration of nonlinear motion blurred images for plant classification using multi-spectral images

Abstract

There have been various plant image-based studies for segmentation, deblurring, super-resolution reconstruction, and classification. However, nonlinear motion blur in thermal images was not considered in the existing studies on plant classification. Nonlinear motion blur occurs in images due to camera or plant movements, and it causes the degradation of plant classification accuracy. Moreover, nonlinear motion blur in images gets worse when both camera and plant movements occur simultaneously. In this case, it becomes difficult to recognize plants, and the performance of plant image classification becomes very low. Therefore, to reduce the nonlinear motion blur, a thermal and visible light plant images-based deblurring network (TVPD-Net) is proposed in this study. In addition, a thermal and visible light plant images-based classification network (TVPC-Net) is also proposed to improve the plant classification performance on deblurred images. Experimental results revealed that the proposed TVPD-Net achieved 21.21 and 22.53 of the peak signal-to-noise ratio (PSNR), and 0.726 and 0.737 of the structural similarity index measure (SSIM) on both visible light and thermal plant image datasets which were self-collected, respectively. Moreover, the proposed TVPC-Net with deblurred images by TVPD-Net achieved 92.52 % (top-1 accuracy) and 87.73 % (harmonic mean of precision and recall (F1-score)). In addition, the experimental results on an open dataset named Hyperspectral Flower Dataset (HFD100) revealed that the proposed plant classification method achieved 90.94 % of top-1 accuracy and 86.21 % of F1-score. The accuracies of the proposed methods are greater than those of the state-of-the-art methods.