<i>PhenoStereo: a high-throughput stereo vision system for field-based plant phenotyping - with an application in sorghum stem diameter estimation</i>

Abstract

Abstract. In recent years, three-dimensional (3D) sensing has gained a great interest in plant phenotyping because it can represent the 3D nature of plant architecture. Among all available 3D imaging technologies, stereo vision offers a viable solution due to its high spatial resolution and wide selection of camera modules. However, the performance of in-field stereo imaging for plant phenotyping has been adversely affected by textureless regions and occlusions of plants, and variable outdoor lighting and wind conditions. In this research, a portable stereo imaging module namely PhenoStereo was developed for high-throughput field-based plant phenotyping. PhenoStereo featured a self-contained embedded design, which made it capable of capturing images at 14 stereoscopic frames per second. In addition, a set of customized strobe lights was integrated to overcome lighting variations and enable the use of high shutter speed to overcome motion blurs. The stem diameter of sorghum plants is an important trait for stalk strength and biomass potential evaluation but has been identified as a challenging sensing task to automated in the field due to the complexity of the imaging object and the environment. To that connection, PhenoStereo was used to acquire a set of sorghum plant images and an automated point cloud data processing pipeline was also developed to automatically extract the stems and then quantify their diameters via an optimized 3D modeling process. The pipeline employed a Mask R-CNN deep learning network for detecting stalk contours and a Semi-Global Block Matching stereo matching algorithm for generating disparity maps. The correlation coefficient (r) between the image-derived stem diameters and the ground truth was 0.97 with a mean absolute error (MAE) of 1.44 mm, which outperformed any previously reported sensing approaches. These results demonstrated that with proper customization stereo vision can be a highly desirable sensing method for field-based plant phenotyping using high-fidelity 3D models reconstructed from stereoscopic images. With the proving results from sorghum plant stem diameter sensing, this proposed stereo sensing approach can likely be extended to characterize a broad spectrum of plant phenotypes such as leaf angle and tassel shape of maize plants and seed pods and stem nodes of soybean plants.