Automated segmentation of soybean plants from 3D point cloud using machine learning

Abstract

Image-based plant phenotyping has become a promising method for high-throughput measurement of plant traits in breeding programs. Plant geometric features that are essential for understanding plant growth can be obtained from the point cloud built using three-dimensional (3D) reconstruction of plant imagery data. A key task in the data processing pipeline is the automated and accurate segmentation of individual plants. Machine learning is a promising approach due to its strong ability in the extraction of details from images and has been successfully applied in plant leaf segmentation from two-dimensional (2D) images. The aim of this paper was to evaluate the performance of three machine learning methods, i.e. boosting, Support Vector Machine (SVM) and K-means clustering, in the segmentation of non-overlapped and overlapped soybean plants at early growth stages using 3D point cloud. Images of 75 soybean plants at two growth stages in a greenhouse were collected using an image-based high-throughput phenotyping platform and were used to develop 3D point cloud using the Structure from Motion (SfM) method. Plant features including position (coordinate x, y, and z), and color (Red, Green, Blue, hue, saturation and Triangular Greenness Index) were used for background removal and the separation of non-overlapped plants. A Histogram of Oriented Gradient (HOG) descriptor was used for the separation of overlapped plants. The percentage of mismatched points between manual and automated segmentation was calculated and results showed that K-means clustering had the least mean error rates (0.36% and 0.20%) for the background removal and the non-overlapped plant separation. The least mean error rate for the separation of overlapped plants was 2.57% using SVM with labeled HOG descriptor. The developed image segmentation pipeline was evaluated in a case study where 69 plants at different growth stages were continuously monitored. Results showed that it took three minutes on average for completing all procedures in the pipeline and the extracted features (i.e. height and shooting area) were able to quantify the plant growth.