Segmentation-Based 4D Registration of Plants Point Clouds for Phenotyping

Abstract

Plant phenotyping, i.e., the task of measuring plant traits to describe the anatomy and physiology of plants, is a central task in crop science and plant breeding. Standard methods often require intrusive or time-consuming operations involving a lot of manual labor. Cameras or range sensors, paired with 3D reconstructions methods, can support phenotyping but the task yields several challenges in practice such as plant growth over time. In this paper, we address the problem of finding correspondences between plants recorded at different points in time to track phenotypic traits in an automated fashion. Our approach makes use of semantic segmentation and unsupervised clustering to compute keypoints from plant point clouds. We extract a compact representation of the considered scan that encodes both, topology and semantic information. Through our approach, we are able to tackle the data association problem for 4D point cloud data of plants effectively. We tested our approach on different 3D plus time, i.e., 4D, sequences of plant point clouds of different plant species. The experiments presented in this paper suggest that our 4D matching approach allows for non-rigid registration of the plants that change over time. Moreover, we show that our method allows for tracking different phenotyping traits at an organ level, forming a basis for automated temporal phenotyping.