Wheat growth stage identification method based on multimodal data

Abstract

Accurate identification of crop growth stages is a crucial basis for implementing effective cultivation management. With the development of deep learning techniques in image understanding, research on intelligent real-time recognition of crop growth stages based on RGB images has garnered significant attention. However, the small differences and high similarity in crop morphological characteristics during the transition between adjacent growth stages pose challenges for accurate identification. To address this issue, this study proposes a multi-scale convolutional neural network model, termed MultiScalNet-Wheat (MSN-W), which enhances the algorithm's ability to learn complex features by utilizing multi-scale convolution and attention mechanisms. This model extracts key information from redundant data to identify winter wheat growth stages in complex field environments. Experimental results show that the MSN-W model achieves a recognition accuracy of 97.6 %, outperforming typical convolutional neural network models such as VGG19, ResNet50, MobileNetV3, and DenseNet. To further address the difficulty in recognizing growth stages during transition periods, where canopy morphological features are highly similar and show small differences, this paper introduces an innovative approach by incorporating sequential environmental data related to wheat growth stages. By extracting these features and performing multi-modal collaborative inference, a multi-modal feature-based wheat growth stage recognition model, termed MultiModalNet-Wheat (MMN-W), is constructed on the basis of the MSN-W model. Experimental results indicate that the MMN-W model achieves a recognition accuracy of 98.5 %, improving by 0.9 % over the MSN-W model. Both the MSN-W and MMN-W models provide accurate methods for observing wheat growth stages, thereby supporting the scientific management of winter wheat at different growth stages.