3D-based precise evaluation pipeline for maize ear rot using multi-view stereo reconstruction and point cloud semantic segmentation

Abstract

Maize ear rot poses a severe threat to maize yield and quality. Breeding and cultivating highly resistant maize varieties is a crucial approach for preventing and controlling maize ear rot. However, traditional methods of visually grading the severity of maize ear infection and resistance lack objectivity and repeatability. To meet the requirement of precise breeding and resistance assessment scenarios, a novel pipeline based on three-dimensional (3D) point clouds of maize ear was developed for ear rot precise evaluation. First, multi-view stereo (MVS) reconstruction was employed to obtain high-precision dense point clouds of maize ears. And the coordinate correction and circular sampling approaches were proposed to optimize the data structure of the input maize ear samples. Next, a specialized network called the ear rot segmentation network (ERSegNet) was proposed to detect the infected area of maize ears. This network incorporated an orientation-encoding (OE) module and point transformer (PT) attention, which effectively boosted the performance of PointNet++. The proposed ERSegNet achieved impressive results, including a mean intersection over union (mIoU) of 85.83%, a mean precision (mPrec) of 92.34%, a mean recall (mRec) of 92.23%, a mean F1-score of 92.28%, and an overall accuracy (OA) of 93.76%. This demonstrated the feasibility of using semantic segmentation algorithms to predict 3D point clouds of maize ears. Furthermore, a point cloud resampling method was suggested to enhance the spatial uniformity of maize ear point clouds and a point-level quantitative assessment approach based on the 3D point cloud data was provided for evaluating the severity of ear rot. The results showed an average evaluation error of 1.55% in the testing set, indicating the accuracy of the proposed method. This study provides a reliable and objective method for maize ear rot precise assessment, offering potential and valuable support for the identification of resistant varieties in breeding programs.