Automatic segmentation of stem and leaf components and individual maize plants in field terrestrial LiDAR data using convolutional neural networks

Abstract

High-throughput maize phenotyping at both organ and plant levels plays a key role in molecular breeding for increasing crop yields. Although the rapid development of light detection and ranging (LiDAR) provides a new way to characterize three-dimensional (3D) plant structure, there is a need to develop robust algorithms for extracting 3D phenotypic traits from LiDAR data to assist in gene identification and selection. Accurate 3D phenotyping in field environments remains challenging, owing to difficulties in segmentation of organs and individual plants in field terrestrial LiDAR data. We describe a two-stage method that combines both convolutional neural networks (CNNs) and morphological characteristics to segment stems and leaves of individual maize plants in field environments. It initially extracts stem points using the PointCNN model and obtains stem instances by fitting 3D cylinders to the points. It then segments the field LiDAR point cloud into individual plants using local point densities and 3D morphological structures of maize plants. The method was tested using 40 samples from field observations and showed high accuracy in the segmentation of both organs (F-score =0.8207) and plants (F-score =0.9909). The effectiveness of terrestrial LiDAR for phenotyping at organ (including leaf area and stem position) and individual plant (including individual height and crown width) levels in field environments was evaluated. The accuracies of derived stem position (position error =0.0141 m), plant height (R2 >0.99), crown width (R2 >0.90), and leaf area (R2 >0.85) allow investigating plant structural and functional phenotypes in a high-throughput way. This CNN-based solution overcomes the major challenges in organ-level phenotypic trait extraction associated with the organ segmentation, and potentially contributes to studies of plant phenomics and precision agriculture.