Measuring Stem Diameter of Sorghum Plants in the Field Using a High-Throughput Stereo Vision System

Abstract

HighlightsA custom-built camera module named PhenoStereo was developed for high-throughput field-based plant phenotyping.Novel integration of strobe lights facilitated application of PhenoStereo in various environmental conditions.Image-derived stem diameters were found to have high correlations with ground truth, which outperformed any previously reported sensing approach.PhenoStereo showed promising potential to characterize a broad spectrum of plant phenotypes.Abstract. The stem diameter of sorghum plants is an important trait for evaluation of stalk strength and biomass potential, but it is a challenging sensing task to automate in the field due to the complexity of the imaging object and the environment. In recent years, stereo vision has offered a viable three-dimensional (3D) solution due to its high spatial resolution and wide selection of camera modules. However, the performance of in-field stereo imaging for plant phenotyping is adversely affected by textureless regions, occlusion of plants, variable outdoor lighting, and wind conditions. In this study, a portable stereo imaging module named PhenoStereo was developed for high-throughput field-based plant phenotyping. PhenoStereo features a self-contained embedded design, which makes it capable of capturing images at 14 stereoscopic frames per second. In addition, a set of customized strobe lights is integrated to overcome lighting variations and enable the use of high shutter speed to overcome motion blur. PhenoStereo was used to acquire a set of sorghum plant images, and an automated point cloud data processing pipeline was developed to automatically extract the stems and then quantify their diameters via an optimized 3D modeling process. The pipeline employed a mask region convolutional neural network (Mask R-CNN) for detecting stalk contours and a semi-global block matching (SGBM) stereo matching algorithm for generating disparity maps. The correlation coefficient (r) between the image-derived stem diameters and the ground truth was 0.97 with a mean absolute error (MAE) of 1.44 mm, which outperformed any previously reported sensing approach. These results demonstrate that, with proper customization, stereo vision can be an effective sensing method for field-based plant phenotyping using high-fidelity 3D models reconstructed from stereoscopic images. Based on the results from sorghum plant stem diameter sensing, this proposed stereo sensing approach can likely be extended to characterize a broad range of plant phenotypes, such as the leaf angle and tassel shape of maize plants and the seed pods and stem nodes of soybean plants. Keywords: Field-based high-throughput phenotyping, Point cloud, Stem diameter, Stereo vision.