Point Clouds Of Plants Research Articles

DFSP: A fast and automatic distance field-based stem-leaf segmentation pipeline for point cloud of maize shoot.

The 3D point cloud data are used to analyze plant morphological structure. Organ segmentation of a single plant can be directly used to determine the accuracy and reliability of organ-level phenotypic estimation in a point-cloud study. However, it is difficult to achieve a high-precision, automatic, and fast plant point cloud segmentation. Besides, a few methods can easily integrate the global structural features and local morphological features of point clouds relatively at a reduced cost. In this paper, a distance field-based segmentation pipeline (DFSP) which could code the global spatial structure and local connection of a plant was developed to realize rapid organ location and segmentation. The terminal point clouds of different plant organs were first extracted via DFSP during the stem-leaf segmentation, followed by the identification of the low-end point cloud of maize stem based on the local geometric features. The regional growth was then combined to obtain a stem point cloud. Finally, the instance segmentation of the leaf point cloud was realized using DFSP. The segmentation method was tested on 420 maize and compared with the manually obtained ground truth. Notably, DFSP had an average processing time of 1.52 s for about 15,000 points of maize plant data. The mean precision, recall, and micro F1 score of the DFSP segmentation algorithm were 0.905, 0.899, and 0.902, respectively. These findings suggest that DFSP can accurately, rapidly, and automatically achieve maize stem-leaf segmentation tasks and could be effective in maize phenotype research. The source code can be found at https://github.com/syau-miao/DFSP.git.

Generating 3D Multispectral Point Clouds of Plants with Fusion of Snapshot Spectral and RGB-D Images.

Accurate and high-throughput plant phenotyping is important for accelerating crop breeding. Spectral imaging that can acquire both spectral and spatial information of plants related to structural, biochemical, and physiological traits becomes one of the popular phenotyping techniques. However, close-range spectral imaging of plants could be highly affected by the complex plant structure and illumination conditions, which becomes one of the main challenges for close-range plant phenotyping. In this study, we proposed a new method for generating high-quality plant 3-dimensional multispectral point clouds. Speeded-Up Robust Features and Demons was used for fusing depth and snapshot spectral images acquired at close range. A reflectance correction method for plant spectral images based on hemisphere references combined with artificial neural network was developed for eliminating the illumination effects. The proposed Speeded-Up Robust Features and Demons achieved an average structural similarity index measure of 0.931, outperforming the classic approaches with an average structural similarity index measure of 0.889 in RGB and snapshot spectral image registration. The distribution of digital number values of the references at different positions and orientations was simulated using artificial neural network with the determination coefficient (R 2) of 0.962 and root mean squared error of 0.036. Compared with the ground truth measured by ASD spectrometer, the average root mean squared error of the reflectance spectra before and after reflectance correction at different leaf positions decreased by 78.0%. For the same leaf position, the average Euclidean distances between the multiview reflectance spectra decreased by 60.7%. Our results indicate that the proposed method achieves a good performance in generating plant 3-dimensional multispectral point clouds, which is promising for close-range plant phenotyping.

Eff-3DPSeg: 3D Organ-Level Plant Shoot Segmentation Using Annotation-Efficient Deep Learning.

Reliable and automated 3-dimensional (3D) plant shoot segmentation is a core prerequisite for the extraction of plant phenotypic traits at the organ level. Combining deep learning and point clouds can provide effective ways to address the challenge. However, fully supervised deep learning methods require datasets to be point-wise annotated, which is extremely expensive and time-consuming. In our work, we proposed a novel weakly supervised framework, Eff-3DPSeg, for 3D plant shoot segmentation. First, high-resolution point clouds of soybean were reconstructed using a low-cost photogrammetry system, and the Meshlab-based Plant Annotator was developed for plant point cloud annotation. Second, a weakly supervised deep learning method was proposed for plant organ segmentation. The method contained (a) pretraining a self-supervised network using Viewpoint Bottleneck loss to learn meaningful intrinsic structure representation from the raw point clouds and (b) fine-tuning the pretrained model with about only 0.5% points being annotated to implement plant organ segmentation. After, 3 phenotypic traits (stem diameter, leaf width, and leaf length) were extracted. To test the generality of the proposed method, the public dataset Pheno4D was included in this study. Experimental results showed that the weakly supervised network obtained similar segmentation performance compared with the fully supervised setting. Our method achieved 95.1%, 96.6%, 95.8%, and 92.2% in the precision, recall, F1 score, and mIoU for stem-leaf segmentation for the soybean dataset and 53%, 62.8%, and 70.3% in the AP, AP@25, and AP@50 for leaf instance segmentation for the Pheno4D dataset. This study provides an effective way for characterizing 3D plant architecture, which will become useful for plant breeders to enhance selection processes. The trained networks are available at https://github.com/jieyi-one/EFF-3DPSEG.

Point Cloud Completion of Plant Leaves under Occlusion Conditions Based on Deep Learning.

The utilization of 3-dimensional point cloud technology for non-invasive measurement of plant phenotypic parameters can furnish important data for plant breeding, agricultural production, and diverse research applications. Nevertheless, the utilization of depth sensors and other tools for capturing plant point clouds often results in missing and incomplete data due to the limitations of 2.5D imaging features and leaf occlusion. This drawback obstructed the accurate extraction of phenotypic parameters. Hence, this study presented a solution for incomplete flowering Chinese Cabbage point clouds using Point Fractal Network-based techniques. The study performed experiments on flowering Chinese Cabbage by constructing a point cloud dataset of their leaves and training the network. The findings demonstrated that our network is stable and robust, as it can effectively complete diverse leaf point cloud morphologies, missing ratios, and multi-missing scenarios. A novel framework is presented for 3D plant reconstruction using a single-view RGB-D (Red, Green, Blue and Depth) image. This method leveraged deep learning to complete localized incomplete leaf point clouds acquired by RGB-D cameras under occlusion conditions. Additionally, the extracted leaf area parameters, based on triangular mesh, were compared with the measured values. The outcomes revealed that prior to the point cloud completion, the R2 value of the flowering Chinese Cabbage's estimated leaf area (in comparison to the standard reference value) was 0.9162. The root mean square error (RMSE) was 15.88 cm2, and the average relative error was 22.11%. However, post-completion, the estimated value of leaf area witnessed a significant improvement, with an R2 of 0.9637, an RMSE of 6.79 cm2, and average relative error of 8.82%. The accuracy of estimating the phenotypic parameters has been enhanced significantly, enabling efficient retrieval of such parameters. This development offers a fresh perspective for non-destructive identification of plant phenotypes.

PST: Plant segmentation transformer for 3D point clouds of rapeseed plants at the podding stage

Segmentation of plant point clouds to obtain high-precise morphological traits is essential for plant phenotyping. Although the fast development of deep learning has boosted much research on segmentation of plant point clouds, previous studies mainly focus on the hard voxelization-based or down-sampling-based methods, which are limited to segmenting simple plant organs. Segmentation of complex plant point clouds with a high spatial resolution still remains challenging. In this study, we proposed a deep learning network plant segmentation transformer (PST) to achieve the semantic segmentation of rapeseed plants point clouds acquired by handheld laser scanning (HLS) with the high spatial resolution, which can characterize the tiny siliques as the main traits targeted. PST is composed of: (i) a dynamic voxel feature encoder (DVFE) to aggregate the point features with the raw spatial resolution; (ii) the dual window sets attention blocks to capture the contextual information; and (iii) a dense feature propagation module to obtain the final dense point feature map. We then integrated PST with an instance segmentation head in the point grouping network (PointGroup) and developed PST-PointGroup (PG) to achieve the instance segmentation of the siliques. The results proved that PST and PST-PG achieved superior performance in semantic and instance segmentation tasks. For the semantic segmentation, the mean IoU, mean Precision, mean Recall, mean F1-score, and overall accuracy of PST were 93.96%, 97.29%, 96.52%, 96.88%, and 97.07%, achieving an improvement of 7.62, 3.28, 4.8, 4.25, and 3.88 percentage points compared to the second-best state-of-the-art network position adaptive convolution (PAConv). For instance segmentation, PST-PG reached 89.51%, 89.85%, 88.83% and 82.53% in mCov, mWCov, mPerc90, and mRec90, achieving an improvement of 2.93, 2.21, 1.99, and 5.9 percentage points compared to the original instance segmentation network PointGroup. This study extends the phenotyping of rapeseed plants in an end-to-end way and proves that the deep-learning-based point cloud segmentation method has a great potential for resolving dense plant point clouds with complex morphological traits.

MASPC_Transform: A Plant Point Cloud Segmentation Network Based on Multi-Head Attention Separation and Position Code

Plant point cloud segmentation is an important step in 3D plant phenotype research. Because the stems, leaves, flowers, and other organs of plants are often intertwined and small in size, this makes plant point cloud segmentation more challenging than other segmentation tasks. In this paper, we propose MASPC_Transform, a novel plant point cloud segmentation network base on multi-head attention separation and position code. The proposed MASPC_Transform establishes connections for similar point clouds scattered in different areas of the point cloud space through multiple attention heads. In order to avoid the aggregation of multiple attention heads, we propose a multi-head attention separation loss based on spatial similarity, so that the attention positions of different attention heads can be dispersed as much as possible. In order to reduce the impact of point cloud disorder and irregularity on feature extraction, we propose a new point cloud position coding method, and use the position coding network based on this method in the local and global feature extraction modules of MASPC_Transform. We evaluate our MASPC_Transform on the ROSE_X dataset. Compared with the state-of-the-art approaches, the proposed MASPC_Transform achieved better segmentation results.

Estimation method of urban green space living vegetation volume based on backpack light detection and ranging

Living vegetation volume (LVV) can objectively and accurately reflect the urban greenery quality, and provide a reliable data foundation for the quantitative study aiming to reveal the mechanisms underlying urban greenery ecological functions. According to the characteristics of dispersion and small scale of unit affiliated green space, we proposed a LVV estimation scheme for such urban green space, which included data acquisition, processing, entity segmentation, classification, single tree canopy extraction, and LVV calculation. First, point cloud data was obtained with a backpack LiDAR system, and the ground point clouds were eliminated by a multi-scale algorithm. Second, the Density Based Spatial Clustering of Application with Noise (DBSCAN) algorithm was used to cluster the non-ground point clouds, and density feature-based competitive algorithm was used to re-segmented for the overlapping area to generate independent objects. Third, the PointNet++ network model was used to extracted plant point clouds. Then, the canopy point clouds were extracted using the similarity of principal direction between neighboring points and distribution density of branch and leaf points. Finally, the LVV of individual tree canopy was calculated by the convex hull method, and then the LVV of the accessory greenland was summed up. Taking a science and technology park as an example, its total LVV was 21034.95 m3, among which the number of mango trees was the highest, and the total LVV was the largest (4868.64 m3, accounting for 23.2%). The tree species with the largest LVV per plant was Terminalia neotaliala tree, with an average of 120.37 m3 per plant. The relative error between LVV of trees estimated by this scheme compared with traditional method and convex hull method was 10.7%-33.7% and 2.7%-16.0%, with average value of 20.9% and 8.7%, respectively. This scheme could make full use of the characteristics of the three-dimensional point cloud and use a convex polyhedron to simulate the original form of the tree crown, which was more consistent with the actual situation of trees. The measurement and estimation solution of the LVV provided new ideas for rapid and accurate estimation of urban LVV.

Using LiDAR technique and modified Community Land Model for calculating water interception of cherry tree canopy

In precision agriculture, methods for analysing 3D point clouds of plants have been introduced, particularly pointing to the high accuracy of light detection and ranging (LiDAR) laser scanning under field conditions. In the present work, LiDAR-based 3D point clouds of cherry trees (n = 255) were analysed for estimating the leaf area as the main factor for water interception. Canopies were scanned for segmenting leaf area pointing to a high variability of canopy surface. The derived tree-specific data of leaf area index (LAI) were implemented into the Community Land Model (CLM), which takes into account canopy interception processes during rainfall events. During canopy development of perennial trees the LAI increased resulting in increased water interception. Events with low rain fall the interception reached 38–100 % capturing LAI of 0.76 – 2.11 m2/m2, respectively. In high rainfall events, interception varied 10–14 % capturing the same LAI range. An equation for describing the varying effects of rainfall intensity and LAI is proposed. The evapotranspiration and water interception data point to a substantial decrease of effective water supply that varies tree-individually during the season. In commercial fruit production, the proposed method can support precise irrigation management.

RoseSegNet: An attention-based deep learning architecture for organ segmentation of plants

An important component for the advancement of plant breeding, genetics, and genomics research is the rapid and accurate measurement of phenotypic traits of large plant populations. The phenotypic data that are of interest can be at multiple levels of plant organization including organ-level geometric characteristics as well as the spatial organization of the organs. 3D computer vision enabling 3D geometry acquisition and processing promises to supply fast, automated phenotypic data collection. One important component of the processing pipeline is the segmentation of the plant into its structural components, such as leaves, stems, and flowers. In this paper, a novel 3D point-based deep learning network, namely RoseSegnet, is proposed for segmentation of point clouds of rosebush plants to their organs. The network is equipped with two attention-based modules, one for extracting contextual features at the encoder phase, another for feature propagation at the decoder phase. The network processes regions of points in a hierarchical manner, where at each level, point features are aggregated using attention-based operators. The aggregation is performed by incorporating point relations both within and between the receptive fields, defined by the hierarchical organization of points. RoseSegNet outperforms the widely-used architecture PointNet++ by 4% in terms of MIoU on the publicly available ROSE-X data set. Also, it is demonstrated that introducing local surface features together with the spatial coordinates of each 3D point at the input level boosts the segmentation performance of both networks by 9% in terms of MIoU . • A new 3D point-based deep learning architecture for organ segmentation of plants. • Attention-based modules for aggregation and propagation of features. • Exploiting point interactions within and between regions at multiple scales. • Incorporation of local surface features to the deep learning framework.

Banana plant counting and morphological parameters measurement based on terrestrial laser scanning

BackgroundThe number of banana plants is closely related to banana yield. The diameter and height of the pseudo-stem are important morphological parameters of banana plants, which can reflect the growth status and vitality. To address the problems of high labor intensity and subjectivity in traditional measurement methods, a fast measurement method for banana plant count, pseudo-stem diameter, and height based on terrestrial laser scanning (TLS) was proposed.ResultsFirst, during the nutritional growth period of banana, three-dimensional (3D) point cloud data of two measured fields were obtained by TLS. Second, the point cloud data was preprocessed. And the single plant segmentation of the canopy closed banana plant point cloud was realized furtherly. Finally, the number of banana plants was obtained by counting the number of pseudo-stems, and the diameter of pseudo-stems was measured using a cylindrical segmentation algorithm. A sliding window recognition method was proposed to determine the junction position between leaves and pseudo-stems, and the height of the pseudo-stems was measured. Compared with the measured value of artificial point cloud, when counting the number of banana plants, the precision,recall and percentage error of field 1 were 93.51%, 94.02%, and 0.54% respectively; the precision,recall and percentage error of field 2 were 96.34%, 92.00%, and 4.5% respectively; In the measurement of pseudo-stem diameter and height of banana, the root mean square error (RMSE) of pseudo-stem diameter and height of banana plant in field 1 were 0.38 cm and 0.2014 m respectively, and the mean absolute percentage error (MAPE) were 1.30% and 5.11% respectively; the RMSE of pseudo-stem diameter and height of banana plant in field 2 were 0.39 cm and 0.2788 m respectively, and the MAPE were 1.04% and 9.40% respectively.ConclusionThe results show that the method proposed in this paper is suitable for the field measurement of banana count, pseudo-stem diameter, and height and can provide a fast field measurement method for banana plantation management.

Point cloud registration method for maize plants based on conical surface fitting\u2014ICP

Reconstructing three-dimensional (3D) point cloud model of maize plants can provide reliable data for its growth observation and agricultural machinery research. The existing data collection systems and registration methods have low collection efficiency and poor registration accuracy. A point cloud registration method for maize plants based on conical surface fitting—iterative closest point (ICP) with automatic point cloud collection platform was proposed in this paper. Firstly, a Kinect V2 was selected to cooperate with an automatic point cloud collection platform to collect multi-angle point clouds. Then, the conical surface fitting algorithm was employed to fit the point clouds of the flowerpot wall to acquire the fitted rotation axis for coarse registration. Finally, the interval ICP registration algorithm was used for precise registration, and the Delaunay triangle meshing algorithm was chosen to triangulate the point clouds of maize plants. The maize plant at the flowering and kernel stage was selected for reconstruction experiments, the results show that: the full-angle registration takes 57.32 s, and the registration mean distance error is 1.98 mm. The measured value’s relative errors between the reconstructed model and the material object of maize plant are controlled within 5%, the reconstructed model can replace maize plants for research.

Segmentation of Individual Leaves of Field Grown Sugar Beet Plant Based on 3D Point Cloud

Accurate segmentation of individual leaves of sugar beet plants is of great significance for obtaining the leaf-related phenotypic data. This paper developed a method to segment the point clouds of sugar beet plants to obtain high-quality segmentation results of individual leaves. Firstly, we used the SFM algorithm to reconstruct the 3D point clouds from multi-view 2D images and obtained the sugar beet plant point clouds after preprocessing. We then segmented them using the multiscale tensor voting method (MSTVM)-based region-growing algorithm, resulting in independent leaves and overlapping leaves. Finally, we used the surface boundary filter (SBF) method to segment overlapping leaves and obtained all leaves of the whole plant. Segmentation results of plants with different complexities of leaf arrangement were evaluated using the manually segmented leaf point clouds as benchmarks. Our results suggested that the proposed method can effectively segment the 3D point cloud of individual leaves for field grown sugar beet plants. The leaf length and leaf area of the segmented leaf point clouds were calculated and compared with observations. The calculated leaf length and leaf area were highly correlated with the observations with R2 (0.80–0.82). It was concluded that the MSTVM-based region-growing algorithm combined with SBF can be used as a basic segmentation step for high-throughput plant phenotypic data extraction of field sugar beet plants.

Segmentation of Plant Point Cloud based on Deep Learning Method

Computer-Aided Design and Applications is an international journal on the applications of CAD and CAM. It publishes papers in the general domain of CAD plus in emerging fields like bio-CAD, nano-CAD, soft-CAD, garment-CAD, PLM, PDM, CAD data mining, CAD and the internet, CAD education, genetic algorithms and CAD engines. The journal is aimed at all developers and users of CAD technology to ptovide CAD solutions for various stages of design and manufacturing. The journal publishes all about Computer-Aided Design and Computer-Aided technologies.

Automatic organ-level point cloud segmentation of maize shoots by integrating high-throughput data acquisition and deep learning

• High-throughput data acquisition platform was used to build high-quality dataset. • Automatic maize shoot segmentation was realized using deep learning. • Six phenotypes of maize shoot can be extracted with satisfactory accuracy. • An pipeline was established from data acquisition to phenotypes extraction. • The pipeline can deal with large amount of 3D data processing for plant phenomics. Point cloud segmentation is essential for studying the 3D spatial characteristics of plants. Notably, the segmentation accuracy greatly impacts subsequent 3D plant phenotypes extraction and 3D plant reconstruction. Automated segmentation approaches for plant point clouds are a bottleneck in achieving big data processing of 3D plant phenotypes. Using maize as a representative crop, this study developed DeepSeg3DMaize, a technique for plant point cloud segmentation that integrates high-throughput data acquisition and deep learning. A high-throughput data acquisition platform for individual plants and an association mapping panel containing 515 inbred lines were used to construct the training dataset. Specifically, the MVS-Pheno platform was used to acquire high-throughput data, and Label3DMaize was used for point cloud data labeling. Based on the dataset, PointNet was introduced to implement stem-leaf and organ instance segmentation, and six phenotypes were extracted. According to the results, the mean precision and F1-Score of stem-leaf segmentation were 0.91 and 0.85, respectively. Meanwhile, the mean precision and F1-Score for organ instance segmentation were 0.94 and 0.93, respectively. The correlations of the six parameters (leaf length, leaf width, leaf inclination, leaf growth height, plant height, and stem height) extracted from the segmentation results with the measured values were 0.90, 0.82, 0.94, 0.95, 0.99, and 0.94, respectively. High-throughput data acquisition, automatic organ segmentation, and phenotypic data extraction form an automatic phenotypic data processing pipeline, which is practical for dealing with large amounts of initial data. Besides, it provides a systematic reference for the automated analysis of 3D phenotypic features at the individual plant level.

PSegNet: Simultaneous Semantic and Instance Segmentation for Point Clouds of Plants.

Phenotyping of plant growth improves the understanding of complex genetic traits and eventually expedites the development of modern breeding and intelligent agriculture. In phenotyping, segmentation of 3D point clouds of plant organs such as leaves and stems contributes to automatic growth monitoring and reflects the extent of stress received by the plant. In this work, we first proposed the Voxelized Farthest Point Sampling (VFPS), a novel point cloud downsampling strategy, to prepare our plant dataset for training of deep neural networks. Then, a deep learning network—PSegNet, was specially designed for segmenting point clouds of several species of plants. The effectiveness of PSegNet originates from three new modules including the Double-Neighborhood Feature Extraction Block (DNFEB), the Double-Granularity Feature Fusion Module (DGFFM), and the Attention Module (AM). After training on the plant dataset prepared with VFPS, the network can simultaneously realize the semantic segmentation and the leaf instance segmentation for three plant species. Comparing to several mainstream networks such as PointNet++, ASIS, SGPN, and PlantNet, the PSegNet obtained the best segmentation results quantitatively and qualitatively. In semantic segmentation, PSegNet achieved 95.23%, 93.85%, 94.52%, and 89.90% for the mean Prec, Rec, F1, and IoU, respectively. In instance segmentation, PSegNet achieved 88.13%, 79.28%, 83.35%, and 89.54% for the mPrec, mRec, mCov, and mWCov, respectively.

Boosting plant-part segmentation of cucumber plants by enriching incomplete 3D point clouds with spectral data

Plant scientists require high quality phenotypic datasets. Computer-vision based methods can improve the objectiveness and the accuracy of phenotypic measurements. In this paper, we focus on 3D point clouds for measuring plant architecture of cucumber plants, using spectral data and deep learning (DL). More specifically, the focus of this paper is on the segmentation of the point clouds, such that for each point it is known to which plant part (e.g. leaf or stem) it belongs. It was shown that the availability of spectral data can improve the segmentation, with the mean intersection-over-union rising from 0.90 to 0.95. Furthermore, we analysed the effect of uncertainty in the collection of ground truth data. For this purpose, we hand-labelled 264 point clouds of cucumber plants twice and show that the intra-observer variability between those two annotation sets can be as low as 0.49 for difficult classes, while it was 0.99 for the class with the least uncertainty. Adding the second set of hand-labelled data to the training of the network improved the segmentation performance slightly. Finally, we show the improved performance of a 4-class segmentation over an 8-class segmentation, emphasizing the need for a careful design of plant phenotyping experiments. The results presented in this paper contribute to further development of automated phenotyping methods for complex plant traits. • Adding spectral data boosts 3D point cloud segmentation for plant phenotyping. • High intra-observer variability shows that ground-truth collection is challenging. • Labelling the same dataset multiple times can improve the segmentation performance. • Careful design of phenotyping experiments makes a computer scientist's life easier.

Branch-Pipe: Improving Graph Skeletonization around Branch Points in 3D Point Clouds

Modern plant phenotyping requires tools that are robust to noise and missing data, while being able to efficiently process large numbers of plants. Here, we studied the skeletonization of plant architectures from 3D point clouds, which is critical for many downstream tasks, including analyses of plant shape, morphology, and branching angles. Specifically, we developed an algorithm to improve skeletonization at branch points (forks) by leveraging the geometric properties of cylinders around branch points. We tested this algorithm on a diverse set of high-resolution 3D point clouds of tomato and tobacco plants, grown in five environments and across multiple developmental timepoints. Compared to existing methods for 3D skeletonization, our method efficiently and more accurately estimated branching angles even in areas with noisy, missing, or non-uniformly sampled data. Our method is also applicable to inorganic datasets, such as scans of industrial pipes or urban scenes containing networks of complex cylindrical shapes.

Pheno4D: A spatio-temporal dataset of maize and tomato plant point clouds for phenotyping and advanced plant analysis.

Understanding the growth and development of individual plants is of central importance in modern agriculture, crop breeding, and crop science. To this end, using 3D data for plant analysis has gained attention over the last years. High-resolution point clouds offer the potential to derive a variety of plant traits, such as plant height, biomass, as well as the number and size of relevant plant organs. Periodically scanning the plants even allows for performing spatio-temporal growth analysis. However, highly accurate 3D point clouds from plants recorded at different growth stages are rare, and acquiring this kind of data is costly. Besides, advanced plant analysis methods from machine learning require annotated training data and thus generate intense manual labor before being able to perform an analysis. To address these issues, we present with this dataset paper a multi-temporal dataset featuring high-resolution registered point clouds of maize and tomato plants, which we manually labeled for computer vision tasks, such as for instance segmentation and 3D reconstruction, providing approximately 260 million labeled 3D points. To highlight the usability of the data and to provide baselines for other researchers, we show a variety of applications ranging from point cloud segmentation to non-rigid registration and surface reconstruction. We believe that our dataset will help to develop new algorithms to advance the research for plant phenotyping, 3D reconstruction, non-rigid registration, and deep learning on raw point clouds. The dataset is freely accessible at https://www.ipb.uni-bonn.de/data/pheno4d/.

Automatic stem-leaf segmentation of maize shoots using three-dimensional point cloud

The application of 3D point cloud data in maize research is increasingly extensive. Currently, there are many approaches to acquiring three-dimensional (3D) point clouds of maize plants. However, automatic stem-leaf segmentation of maize shoots from 3D point clouds remains challenging, especially for new emerging leaves that are wrapped very closely together during the seedling stage. To address this issue, we propose an automatic segmentation method consisting of three steps: skeleton extraction, coarse segmentation based on the skeleton, and fine segmentation based on stem-leaf classification. The segmentation method was tested on 75 maize seedlings and compared with the manually obtained ground truth. The mean precision, mean recall, mean micro F1 score, and mean overall accuracy of our segmentation algorithm were 0.944, 0.956, 0.950 and 0.953, respectively. Using the segmentation results, two applications were also developed in this study, namely, phenotypic trait extraction and skeleton optimization. Six phenotypic parameters, namely, plant height, crown diameter, stem height and diameter, leaf width, and length, can be accurately and automatically measured. Furthermore, the values of R2 for the six phenotypic traits were all above 0.92. We also propose a skeleton optimization method that can extract the skeletons of the upper leaves completely and clearly. The results indicate that the proposed algorithm can automatically and precisely segment not only the fully expanded leaves but also the new leaves wrapped closely together. The proposed approach can play an important role in further maize research and applications, such as genotype-to-phenotype study, geometric reconstruction, and dynamic growth animation. We released the source code and test data at the web site https://github.com/syau-miao/seg4maize.git.

High resolution 3D terrestrial LiDAR for cotton plant main stalk and node detection

Dense three-dimensional point clouds provide opportunities to retrieve detailed characteristics of plant organ-level phenotypic traits, which are helpful to better understand plant architecture leading to its improvements via new plant breeding approaches. In this study, a high-resolution terrestrial LiDAR was used to acquire point clouds of plants under field conditions, and a data processing pipeline was developed to detect plant main stalks and nodes, and then to extract two phenotypic traits including node number and main stalk length. The proposed method mainly consisted of three steps: first, extract skeletons from original point clouds using a Laplacian-based contraction algorithm; second, identify the main stalk by converting a plant skeleton point cloud to a graph; and third, detect nodes by finding the intersection between the main stalk and branches. Main stalk length was calculated by accumulating the distance between two adjacent points from the lowest to the highest point of the main stalk. Experimental results based on 26 plants showed that the proposed method could accurately measure plant main stalk length and detect nodes; the average R2 and mean absolute percentage error were 0.94 and 4.3% for the main stalk length measurements and 0.7 and 5.1% for node counting, respectively, for point numbers between 80,000 and 150,000 for each plant. Three-dimensional point cloud-based high throughput phenotyping may expedite breeding technologies to improve crop production.