3D reconstruction of plants using probabilistic voxel carving

Abstract

We propose a probabilistic voxel carving algorithm to efficiently reconstruct 3D models of maize plants and extract leaf traits for phenotyping. Traditional voxel carving algorithm is restricted to a limited number of views and usually requires multiple coordinated cameras in the imaging setup. They are also not robust small movements of the object, which introduce noise into the data. These imperfections in data collection can lead to large regions of the object being carved away during the voxel carving process, leading to incomplete and disjoint objects. We have developed a probabilistic voxel carving algorithm to overcome these challenges. In this approach, instead of carving out or keeping a voxel in a binary manner, we associate a probability of a voxel corresponding to it being part of the plant. We then use a user-defined probability cutoff to obtain the final voxelized plant geometry. We optimize the data collection procedure by adopting a rotating base to hold the plant and then capturing videos of the rotating plants, thereby obtaining an arbitrary number of views by extracting the image frames. Additionally, we leverage GPU computing to implement our voxel carving and trait extraction pipeline for a large dataset with over 1000 maize plants with high voxel resolutions (such as 10243). Our results demonstrate that our algorithm is robust and can handle an arbitrary number of views, and can automatically extract plant traits such as the number of leaves and leaf angles. Our approach shows that 3D reconstructions of plants from multi-view images can accurately extract multiple phenotypic traits, enabling better plant breeding programs.