Light Detection And Ranging Point Research Articles

High-precision deformation analysis of yingxian wooden pagoda based on UAV image and terrestrial LiDAR point cloud

The monitoring of wooden pagodas is a very important task in the restoration of wooden pagodas. Traditionally, this labor has always been carried out by surveying personnel, who manually check all parts of the pagoda, which not only consumes huge manpower, but also suffers from low efficiency and measurement errors. This article evaluates the feasibility of combining portable 3D light detection and ranging (LiDAR) scanning and unmanned aerial vehicle (UAV) photogrammetry to perform these inspection tasks easily and accurately. The wooden pagoda's exterior picture and inside point cloud are acquired using a UAV and a LiDAR scanner, respectively. We propose a feature−based global alignment method to register the site point cloud. The error equation of the column of observed values is utilized as the beginning value of the feature constraint for global leveling. The beam method leveling model solves the spatial transformation parameters and the unknown point leveling values. Then, the Structure from Motion (SfM) algorithm of computer vision is used to realize the fusion of the dense point cloud of the exterior of the wooden pagoda generated from multiple non−measured images by global optimization and the LiDAR point cloud of the interior of the wooden pagoda to obtain the complete point cloud of the wooden pagoda, which makes the deformation monitoring of the pagoda more detailed and comprehensive. After experimental verification, the overall registration accuracy of the Yingxian wooden pagoda reaches 0.006 m. Compared with the scanning point cloud data in 2018, the model is more accurate and complete. By analyzing and comparing the data of the second floor of the wooden pagoda, we knew that the inclination of a second bright layer and a second dark layer is still developing steadily. Overall, the western outer trough inclines thoughtfully, and the column frame slopes from southwest to northeast. Some internal columns showed a negative offset in 2020, and the deformation analysis of a single column was realized by comparing it with the standard column model. The main contribution of this method lies in the effective integration of UAV images and point cloud data to provide accurate data sources for good modeling. This research will provide theoretical and methodological support for the digital protection of architectural heritage and GIS data modeling. The analysis results can provide a scientific basis for the restoration scheme design.

Winter adverse driving dataset for autonomy in inclement winter weather

The availability of public datasets with annotated light detection and ranging (LiDAR) point clouds has advanced autonomous driving tasks, such as semantic and panoptic segmentation. However, there is a lack of datasets focused on inclement weather. Snow and rain degrade visibility and introduce noise in LiDAR point clouds. In this article, summarize a 3-year winter weather data collection effort and introduce the winter adverse driving dataset. It is the first multimodal dataset featuring moderate to severe winter weather—weather that would cause an experienced driver to alter their driving behavior. Our dataset features exclusively events with heavy snowfall and occasional white-out conditions. Data are collected using high-resolution LiDAR, visible as well as near infrared (IR) cameras, a long wave IR camera, forward-facing radio detection and ranging, and Global Navigation Satellite Systems/Inertial Measurement Unit units. Our dataset is unique in the range of sensors and the severity of the conditions observed. It is also one of the only data sets to focus on rural and semi-rural environments. Over 36 TB of adverse winter data have been collected over 3 years. We also provide dense point-wise labels to sequential LiDAR scans collected in severe winter weather. We have labeled and will make available around 1000 sequential LiDAR scenes, amounting to over 7 GB or 3.6 billion labeled points. This is the first point-wise semantically labeled dataset to include falling snow.

A Category-Contrastive Guided-Graph Convolutional Network Approach for the Semantic Segmentation of Point Clouds

The semantic segmentation of light detection and ranging (LiDAR) point clouds plays an important role in 3D scene intelligent perception and semantic modeling. The unstructured, sparse and uneven characteristics of point clouds pose great challenges to the representation of the local geometric shapes, which degrades semantic segmentation performance. To address the challenges of describing local geometric shapes due to unstructured and sparse 3D point clouds, this paper proposes a category-contrastive-guided graph convolutional network (CGGC-Net) for the semantic segmentation of LiDAR point clouds. First, a detailed geometric structure of the raw point clouds is encoded to represent the inherent geometric pattern within the local neighborhood. At the same time, the geometric structures information is transmitted across multiple layers, so that the geometric structure encoding information containing different receptive fields and richer neighborhood spatial structure can be aggregated. Following this, the graph convolution neural network uses the edge convolution layer to adaptively describe the semantic correlation between the query point and its neighboring points, and combines the attention mechanism to gather the surrounding feature information to the query point. As a result, the graph convolution neural network and attention mechanism are iteratively stacked for the aggregation and fusion of spatial context semantic information, to generate highly discriminative semantic feature representation. Finally, the super parameters of the model are learned through a multitask optimization strategy guided by category-aware contrastive loss and cross-entropy loss. Experiments are conducted on the public SemanticKITTI dataset and the Stanford Large-Scale 3D Indoor Spaces dataset to demonstrate the effectiveness and reliability of the proposed CGGC-Net from both quantitative and qualitative perspectives. The results indicate its capability of automatically classifying LiDAR point clouds, with a mean intersection-over-union (mIoU) of 58.4%. Moreover, multiple comparative experiments also demonstrate the superior performance of the proposed method, exceeding state-of-the-art methods.

3D object detection for autonomous driving from image: a survey——benchmarks, constraints and error analysis

从高分辨率图像中获取周边目标的精准3D位置和尺寸信息是实现自动驾驶控制和行为决策的基础，因此基于图像的3D目标检测是自动驾驶领域中的研究热点。已有学者对该领域方法论及成果进行了比较详细的综述，但对于导致现有方法检测精度不尽如意的制约因素未能进行深入系统的分析。考虑自动驾驶领域在工程应用方面的要求高，且现有方法以数据驱动类型为主，本文从常用数据集和评价基准、数据影响、方法论的制约因素和误差等角度，对学术界和产业界在3D目标检测方面的研究成果及行业应用进行较为系统的阐述。首先，从学术界探索成果以及自动驾驶行业的应用角度进行概要介绍。然后，从数据采集设备、数据精度和标注信息3方面详细分析总结了KITTI等4个通用数据集，并对这些数据集提出的主要评价指标进行对比分析。接着，从数据和方法论方面分析制约算法性能的主要因素及由此造成的误差影响。在数据方面，制约因素主要是数据精度、样本差异、标注数据量和标注规范；在方法论方面，制约因素主要包括先验几何关系、深度预测误差和数据模态等。最后，对国内外研究现状进行总结，并在数据集、评价指标和目标深度预测等方面提出了未来需要重点关注的研究方向。;Autonomous driving-oriented accurate perception and measurement of the three-dimensional(3D) spatial position and scale can be as the basis for realizing the control ability and decision-making level. Sensing technology-driven autonomous vehicles are equipped with high-resolution camera, light detection and ranging (LiDAR), radar, global positioning system (GPS)/inertial measurement unit (IMU) and other related sensors. Current LiDAR or multi-modal data-based 3D object detection algorithms are challenged for its deployment and application because of the shortcomings of LiDAR sensors like high price, limited sensing range, and sparse point clouds data. In contrast, such high-resolution cameras are commonly-used and featured by its lower price, and it can obtain high-resolution spatial information, richer shape, and appearance details as well. The emerging image-based 3D object detection is focused on further. At present, constraints of detection accuracy of the existing methods are still to be analyzed thoroughly and systematically. We summary the research results and industrial applications in relevance to such 1) perspectives of commonly used datasets and evaluation criteria, 2) data impact, 3) methodological constraints and prediction errors. First, a brief introduction is linked to perspective of academic domain and application of autonomous driving industry. We briefly review latest growths of Baidu Apollo, Google Waymo, Tesla and other related autonomous driving companies, and the thread of 3D object detection methods for autonomous driving. Then, we analyzed and summarized four popular datasets like KITTI, nuScenes, Waymo open dataset, and DAIR-V2X dataset from three aspects of:1) data acquisition/sensors, data accuracy and data label information;2) key evaluation standards proposed by these data sets, and 3) pros/cons and applicability of these evaluation standards. Third, main constraints of the image-based 3D object detection algorithm and the errors are derived from two sides of:data and methodology. Such main data constraints are originated from their data accuracy, sample difference, data volume, and data annotation. The data accuracy is mainly limited by equipment performance. The sample difference is mainly restricted by such image processing problems in related to object distance difference, angle difference, occlusion, and truncation. Data volume is affected by variety of 3D data types and high difficulty of labeling. The volume of 3D object detection data set is much smaller in comparison with the 2D object detection data set. Data annotation is mainly focused on 3D bounding box labeling, the labeling details, and quality of the dataset, especially for image annotation used in image-based 3D object detection. For non-rigid objects like pedestrians, the annotation error is larger, and there are some optimal for improving the labeling method. The general framework of image-based 3D object detection can be classified as one-stage methods and two-stage methods, and the limitations consists of 1) the prior geometric relationship, 2) depth prediction accuracy, and 3) data modality. The prior geometric relationship is focused on 2D-3D geometric constraints for 2D images-projected 3D objects and objects-between position relationships. The image-based 3D object detection methods face such problems as:prior 2D-3D geometric constraints and occluded and truncated objects. The prediction of depth information from 2D images is an ill conditioned problem, and dimension collapse will cause depth prediction error-relevant loss of depth information in the image. On the one hand, the depth prediction is often not accurate due to the influence of projection relationship. On the other hand, the performance of continuous depth prediction is often poor at the depth mutation of the image(such as edge of objects). When the prediction depth is discretized, there is a problem that the classification of depth is relatively rough, and the accuracy classification cannot be arbitrarily divided. The limitation of single image-based data modality is mainly reflected via large error of depth prediction. The detection performance of the algorithm can be optimized by 1) simulating the stereo signal and LiDAR point clouds, or 2) using stereo image as the aided input, or 3) leveraging point clouds data with accurate 3D information as supervision signal. In addition, video data can be adopted to improve the detection accuracy to a certain extent. Forth, current research situation is summarized and compared from academic and industrial domain. Finally, some future research directions are predicted in terms of such factors of datasets, evaluation indicators, and depth prediction.

Frame-Level Rate Control for Geometry-Based LiDAR Point Cloud Compression

The state-of-the-art compression method for Light Detection And Ranging (LiDAR) point clouds is the geometry-based point cloud compression (G-PCC) standard developed by Moving Pictures Experts Group immersive media working group (MPEG-I). However, there are currently no rate control algorithms designed specifically for Geometry-based LiDAR point cloud compression (G-LPCC). In this paper, we propose the first frame-level rate control algorithm for G-LPCC. We mainly have the following contributions in our proposed rate control algorithm. First, we model the rate-distortion (R-D) relationship for both the geometry and attribute. As the geometry bitrate is mainly determined by the frame-level geometry quantizer <formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex>$Q_G$</tex></formula> , we propose a relationship between the geometry bitrate and <formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex>$Q_G$</tex></formula> . In addition, as the attribute bitrate can be influenced by both the attribute quantizer <formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex>$Q_A$</tex></formula> and <formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex>$Q_G$</tex></formula> , we build a relationship among the attribute bitrate, <formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex>$Q_G$</tex></formula> , and <formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex>$Q_A$</tex></formula> . Second, we propose a bit allocation algorithm between the geometry and attribute based on the R-D modeling. The <formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex>$Q_G$</tex></formula> and <formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex>$Q_A$</tex></formula> are modeled into a proper relationship to obtain geometry and attribute bits to achieve good R-D performance. Third, we propose using the point density of LiDAR point clouds to estimate the geometry model parameters. The point density is calculated using the average distance between each point and its nearest neighbor after excluding some noisy points. The proposed rate control algorithm is implemented in the G-PCC reference software. The experimental results show that the proposed rate control algorithm can control the bitrate accurately with satisfactory R-D performance.

Rate-Distortion Optimized Geometry Compression for Spinning LiDAR Point Cloud

Point cloud is a major representation format of 3D objects and scenes. It has been increasingly applied in various applications due to the rapid advances in 3D sensing and rendering technologies. In the field of autonomous driving, point clouds captured by spinning Light Detection And Ranging (LiDAR) devices have become an informative data source for road environment perception and intelligent vehicle control. On the other hand, the massive data volume of point clouds also brings huge challenges to point cloud transmission and storage. Therefore, establishing compression frameworks and algorithms that conform to the characteristics of point cloud data has become an important research topic for both academia and industry. In this paper, a geometry compression method dedicated to spinning LiDAR point cloud was proposed taking advantage of the prior information of the LiDAR acquisition procedure. Rate-distortion optimizations were further integrated into the coding pipeline according to the characteristics of the prediction residuals. Experimental results obtained on different datasets show that the proposed method consistently outperforms the state-of-the-art G-PCC predictive geometry coding method with reduced runtime at both the encoder and decoder sides.

SemanticFlow: Semantic Segmentation of Sequential LiDAR Point Clouds From Sparse Frame Annotations

Sequential point clouds acquired by light detection and ranging (LiDAR) technology provide accurate spatial information for environmental sensing. However, semantic segmentation of point cloud sequences relies on many manual point-wise annotations, which are error-prone and expensive. Existing mainstream weakly supervised methods tackle this by reducing the percentage of labeled points, but they are mostly designed for static indoor scenes and are hard to apply practically. From the viewpoint of realistic annotation procedures and the nature of point cloud sequences, this paper proposes a novel semantic segmentation method, SemanticFlow, for LiDAR point cloud sequences using sparse frames with annotations. The proposed method achieves competitive performance compared with fully supervised methods. Specifically, we designed a bidirectional cross-frame pseudo label propagation module that uses scene flow to learn the correlation and propagate pseudo labels across neighboring frames. In addition, a label refinement mechanism is proposed to select reliable pseudo labels for learning. Extensive experiments on SemanticKITTI, SemanticPOSS, and Synthia 4D datasets demonstrate that our sparse frame annotation method is compatible with some fully supervised counterparts.

Multi-Source Data Fusion Improves Time-Series Phenotype Accuracy in Maize under a Field High-Throughput Phenotyping Platform.

The field phenotyping platforms that can obtain high-throughput and time-series phenotypes of plant populations at the 3-dimensional level are crucial for plant breeding and management. However, it is difficult to align the point cloud data and extract accurate phenotypic traits of plant populations. In this study, high-throughput, time-series raw data of field maize populations were collected using a field rail-based phenotyping platform with light detection and ranging (LiDAR) and an RGB (red, green, and blue) camera. The orthorectified images and LiDAR point clouds were aligned via the direct linear transformation algorithm. On this basis, time-series point clouds were further registered by the time-series image guidance. The cloth simulation filter algorithm was then used to remove the ground points. Individual plants and plant organs were segmented from maize population by fast displacement and region growth algorithms. The plant heights of 13 maize cultivars obtained using the multi-source fusion data were highly correlated with the manual measurements (R2 = 0.98), and the accuracy was higher than only using one source point cloud data (R2 = 0.93). It demonstrates that multi-source data fusion can effectively improve the accuracy of time series phenotype extraction, and rail-based field phenotyping platforms can be a practical tool for plant growth dynamic observation of phenotypes in individual plant and organ scales.

Synthetic LiDAR-Labeled Data Set Generation to Train Deep Neural Networks for Object Classification in IoT at the Edge

Light detection and ranging (LiDAR) sensors are increasing in popularity due to the advantages they provide over 2-D sensors in IoT object detection and classification applications, because of their ability to provide very precise distances to objects. Deep learning algorithms need a huge amount of data during training to obtain high accuracy results. When using 2-D images, a vast quantity of data sets are publicly available, but this is not the case for LiDAR point clouds. Each LiDAR model generates a point cloud with unique properties, which causes the data sets not to be compatible between different LiDAR models. As a result, when using deep learning with LiDARs, it is necessary to generate the data sets manually. For this purpose, the data must be captured and then labeled one by one, which is a very time and cost-consuming process. To overcome this issue and to reduce the development time when using LiDAR sensors with deep learning algorithms, a methodology is proposed in this article to automatically generate point cloud data sets using a 3-D simulator for autonomous cars. In this regard, a data set can be generated for any LiDAR model by adding the specific LiDAR parameters to the simulator. Besides, custom scenarios can be designed and generated, based on the final deployment location, to provide a simulated solution very close to the final implementation. With the proposed methodology, a simulation can be performed to select the LiDAR that best fits certain application requirements, in contrast to the traditional approach where the LiDAR must first be purchased.

Nonparametric Background Model-Based LiDAR SLAM in Highly Dynamic Urban Environments

In urban environments, simultaneous localization and mapping (SLAM) are essential for autonomous driving. Most light detection and ranging (LiDAR) SLAM methodologies have been developed for relatively static environments, despite real-world environments having many dynamic objects such as vehicles, bicycles, and pedestrians. This paper proposes an efficient and robust LiDAR SLAM. Our SLAM framework leverages the estimated background model to achieve robust motion estimation in dynamic urban environments. Based on probabilistic object estimation, the dynamic removal module estimates a nonparametric background model to recognize dynamic objects. This module estimates the probability of the difference of the range values from the accumulated LiDAR frames. Then, dynamic objects are removed by adapting the sensor velocity from the estimated ego-motion. In the local mapping module, our method optimizes the LiDAR motion considering the dynamic characteristics of LiDAR point clouds. Finally, the proposed method results in a global map with static point clouds and accurate LiDAR motion with global pose optimization. We tested the proposed method on the well-known public dataset (KITTI) and the custom dataset with complex environments, including various moving objects. Comparisons with state-of-the-art (SOTA) methods demonstrate that the our approach is more robust and efficient. For example, the proposed method performed an average 0.63% and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0.18^{\circ }/100\;m$ </tex-math></inline-formula> errors on the KITTI dataset with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0.96ms$ </tex-math></inline-formula> processing time that convinces real-time processing.

Enabling 3-D Object Detection With a Low-Resolution LiDAR

Light detection and ranging (LiDAR) has been widely used in autonomous vehicles for perception and localization. However, the cost of a high-resolution LiDAR is still prohibitively expensive, while its low-resolution counterpart is much more affordable. Therefore, using low-resolution LiDAR for autonomous driving is an economically viable solution, but the point cloud sparsity makes it extremely challenging. In this letter, we propose a two-stage neural network framework that enables 3-D object detection using a low-resolution LiDAR. Taking input from a low-resolution LiDAR point cloud and a monocular camera image, a depth completion network is employed to produce dense point cloud that is subsequently processed by a voxel-based network for 3-D object detection. Evaluated with KITTI dataset for 3-D object detection in bird-eye view (BEV), the experimental result shows that the proposed approach performs significantly better than directly applying the 16-line LiDAR point cloud for object detection. For both easy and moderate cases, our 3-D vehicle detection results are close to those using 64-line high-resolution LiDARs.

Estimation of ship berthing parameters based on Multi-LiDAR and MMW radar data fusion

Achieving autonomous and safe berthing can reflect the intelligence level of unmanned ships, which is also an important manifestation to ensure the safe navigation of ships. As high-precision sensors, LiDAR (Light Detection and Ranging) and MMW Radar (millimeter wave radar) can provide environmental data for berthing. However, the use of one of them has certain shortcomings, and the fusion of the data from both sensors can provide more accurate and ample berthing information. Therefore, a novel data fusing method is proposed for assisting berthing. This method combines multi-solid-state LiDAR point cloud and MMW Radar data to obtain berthing parameters. Firstly, the collected data are pre-processed according to the characteristics of asynchronous sensors separately, and then the berthing parameters under the two sensors are obtained by the relative spatial position between the ship and the berth; In addition, the data-level fusion of multiple LiDARs point cloud is achieved by the point cloud registration-based 3D-NDT (Normal Distributions Transform) algorithm, and the decision-level fusion of berthing parameters acquired from the multi-LiDAR point cloud and MMW Radar data is performed by the weighted data fusion algorithm. Two sensors were used as perceptual devices to effectively monitor the berthing process of the experimental Ro-Ro ship at Pikou Port. The results demonstrate that the proposed method can make the data of the two sensors complement each other and effectively estimate model-free berthing parameters, which is important to assist in achieving efficient and safe berthing of ships.

Terrain-Net: A Highly-Efficient, Parameter-Free, and Easy-to-Use Deep Neural Network for Ground Filtering of UAV LiDAR Data in Forested Environments

In recent years, a rise in interest in using Unmanned Aerial Vehicles (UAV) with LiDAR (Light Detection and Ranging) to capture the 3D structure of forests for forestry and ecosystem monitoring applications has been witnessed. Since the terrain is an essential basis for the vertical structure modeling of a forest, the point cloud filtering delivering a highly accurate Digital Terrain Model (DTM) contributes significantly to forest studies. Conventional point cloud filtering algorithms require users to select suitable parameters according to the knowledge of the algorithm and the characteristics of scanned scenes, which are normally empirical and time-consuming. Deep learning offers a novel method in classifying and segmenting LiDAR point cloud, while there are only few studies reported on utilizing deep learning to filter non-ground LiDAR points of forested environments. In this study, we proposed an end-to-end and highly-efficient network named Terrain-net which combines the 3D point convolution operator and self-attention mechanism to capture local and global features for UAV point cloud ground filtering. The network was trained with over 15 million labeled points of 70 forest sites and was evaluated at 17 sites covering various forested environments. Terrain-net was compared with four classical filtering algorithms and one of the most well-recognized point convolution-based deep learning methods (KP-FCNN). Results indicated that Terrain-net achieved the best performance in respect of the Kappa coefficient (0.93), MIoU (0.933) and overall accuracy (98.0%). Terrain-net also performed well in transferring to an additional third-party open dataset for ground filtering in large-scale scenes and other vegetated environments. No parameters need to be tuned in transferring predictions. Terrain-net will hopefully be widely applied as a new highly-efficient, parameter-free, and easy-to-use tool for LiDAR data ground filtering in varying forest environments.

Analysing individual 3D tree structure using the R package ITSMe

Abstract Detailed 3D quantification of tree structure plays a crucial role in understanding tree‐ and plot‐level biophysical processes. Light detection and ranging (LiDAR) has led to a revolution in tree structural measurements and its 3D data are increasingly becoming publicly available. Yet, calculating structural metrics from LiDAR data can often be complex and time‐consuming and potentially requires expert knowledge. We present the R package Individual Tree Structural Metrics (ITSMe), a toolbox that works with LiDAR tree point clouds and quantitative structure models (QSMs) derived from LiDAR point clouds to obtain individual tree structural metrics. It serves as a robust synthesis framework for researchers who want to readily obtain structural information from 3D data of individual trees. The package includes functions to determine basic structural metrics (tree height, diameter at breast height, diameter above buttresses, projected crown area, 3D alpha crown volume) from individual tree point clouds, as well as more complex structural metrics (individual tree component volumes, branch angle‐, radius‐ and length‐related metrics) from QSMs. The ITSMe package is an open‐source package hosted on GitHub that will make the use of 3D data more straightforward and transparent for a range of end‐users interested in exploiting tree structure information.

Density Awareness and Neighborhood Attention for LiDAR-Based 3D Object Detection

Light detection and ranging (LiDAR) is widely used in the automotive industry as it can provide point cloud information containing precise distances. Three-dimensional (3D) object detection based on LiDAR point clouds is significant for environment perception tasks. However, feature learning for point clouds remains challenging. This paper proposes a two-stage voxel-based LiDAR 3D object detector, referred to as density-aware and neighborhood attention (DenNet), that focuses on the neighborhood information of objects. DenNet mainly integrates two modules: voxel density-aware (VDA) and neighborhood attention (NA). VDA introduces density information of the point cloud. Here, point cloud density information was added as voxel features in the voxel-based framework to alleviate the information loss during voxelization. Additionally, to extract neighbor information, the characteristics of 3D objects were analyzed for traffic scenes. The NA mechanism was adopted, which localizes the receptive field for each query to its nearest neighboring points. DenNet yielded competitive results, as compared with state-of-the-art methods, for the KITTI and One Million Scenes (ONCE) datasets; notably, it afforded an improvement of 3.96% relative to the baseline mean average precision on the more challenging ONCE dataset.

Automated extraction of building instances from dual-channel airborne LiDAR point clouds

With the rapid development of Light Detection And Ranging (LiDAR) systems, the novel dual-channel airborne LiDAR systems have emerged to provide more complete and precise data than traditional scanners for building instance extraction since 2013. RIEGL VQ-1560i, launched in 2016, is a state-of-the-art dual-channel LiDAR system, which is capable of capturing dense points on building rooftops and façades simultaneously, due to the unique and innovative bidirectional scanning angle. Our proposed method is the first ever to use dual-channel airborne LiDAR data for subsequent point clouds processing. The main challenges of the new LiDAR data are significant amount of points, complex data structure and multi-class targets. We proposed a preprocessing-free building instance extraction method consisting of three steps, i.e., point cloud reorganization, rasterization, and constraint-based labeling for improving the extraction performance. First, point cloud reorganization, consisting of point distribution-based slicing, coarse 3D semantic segmentation, and top-down merging, is used to reorganize point cloud scene into interrelated point groups. This greatly reduces the processing difficulty and computational burden of complex structures while removing multiple classes of non-building points. Second, we rasterize the point groups into images to further reduce computational complexity while improving processing efficiency. Finally, we utilize the upper and lower structural relationship of buildings to label them and then remap into 3D buildings. Experimental results on six test point cloud scenes demonstrate the outstanding performance of the proposed preprocessing-free method. For semantic-level performance, our method achieves 95.36% in average recall and 93.59% in average F1-score. While for instance-level performance, our approach reaches 92.86% and 98.31% in quality on two public test scenes, respectively. • Dual-channel ALS data is used to build instances extraction for the first time. • The proposed method is preprocessing-free and fully automated. • The input scene is reorganized based on point distribution and top-down merging. • Both semantic-based and instance-based building segmentation were conducted.

Recognition of ecological vegetation fairy circles in intertidal salt marshes from UAV LiDAR point clouds

• Fairy circles are detected from 3D point clouds of UAV LiDAR for the first time. • A new model is proposed to correct the intensity data of UAV LiDAR. • Merely corrected intensity data can reliably be used for point cloud filtering. • Absent instantaneous UAV LiDAR scanning distances and scan angles are recovered. • The proposed method is promising for spatial patterns recognition in other scenes. Fairy circles (FC) are a type of spatial self-organized patterns that widely exist in various vegetation ecosystems and the accurate detection and quantitative characterization of these mysterious circles remain a technical challenge. In this study, vegetation FC in intertidal salt marshes are recognized from the derived reflectance information (backscattered intensity) and geometric quantities of light detection and ranging (LiDAR) carried on unmanned aerial vehicle (UAV). The specular effect on the UAV LiDAR intensity data over nadir regions of wet salt marshes is eliminated using the laser radar equation and Phong model where the absent distances and incidence angles are approximately retrieved on the basis of geometric and temporal relations in data collection. The FC are progressively recovered through three interconnected procedures. First, the retrieved reflectance information is used to discriminate the mudflat and vegetation points. Second, a spatial connectivity clustering algorithm is utilized on the extracted vegetation points to form individual spatially disconnected clusters. Finally, FC and regular vegetation are successfully recognized by jointly using the salient, size, and circularity features of the generated clusters. A multi-echo UAV LiDAR system is used for data collection at an intertidal salt marsh to assess the feasibility and prospects of the proposed method. Taking the manual detection results from the orthophoto generated by images of a UAV camera system as a reference, the missing detection rate, false detection rate, and area detection error of the proposed method are 6%, 9%, and 5%, respectively. Results suggest that UAV LiDAR is an extremely promising technique to characterize the geometric properties (e.g., location, size, and quantity) of FC from a holistic perspective.

Information fusion approach for biomass estimation in a plateau mountainous forest using a synergistic system comprising UAS-based digital camera and LiDAR

Forest land plays a vital role in global climate, ecosystems, farming and human living environments. Therefore, forest biomass estimation methods are necessary to monitor changes in the forest structure and function, which are key data in natural resources research. Although accurate forest biomass measurements are important in forest inventory and assessment, high-density measurements that involve airborne light detection and ranging (LiDAR) at a low flight height in large mountainous areas are expensive. The objective of this study was to quantify the aboveground biomass (AGB) of a plateau mountainous forest reserve using a system that synergistically combines an unmanned aircraft system (UAS)-based digital aerial camera and LiDAR to leverage their complementary advantages. In this study, we utilized digital aerial photogrammetry (DAP), which has the unique advantages of speed, high spatial resolution, and low cost, to compensate for the deficiency of forestry inventory using UAS-based LiDAR that requires terrain-following flight for high-resolution data acquisition. Combined with the sparse LiDAR points acquired by using a high-altitude and high-speed UAS for terrain extraction, dense normalized DAP point clouds can be obtained to produce an accurate and high-resolution canopy height model (CHM). Based on the CHM and spectral attributes obtained from multispectral images, we estimated and mapped the AGB of the region of interest with considerable cost efficiency. It is proved that sparse LiDAR point cloud could serve as important sources for terrain estimation. The accuracy of the AGB estimates could be improved by 9% and 5% in terms of the RMSE and R2, respectively, by adding spectral metrics to point cloud structural metrics. Compared with results obtained using only spectral metrics, the results obtained using the combination of spectral and point cloud metrics were better by 37% in terms of the RMSE and 55% in R2. Our study supports the development of predictive models for large-scale wall-to-wall AGB mapping by leveraging the complementarity between DAP and LiDAR measurements. This work also reveals the potential of utilizing a UAS-based digital camera and LiDAR synergistically in a plateau mountainous forest area. In this future, we will further investigate different establishing strategies for combining different sensors in a synergistic system.

Tree stem volume estimation from terrestrial LiDAR point cloud by unwrapping

Estimating the volume of standing trees is a fundamental concern in forestry and is typically accomplished using one or more measurements of stem diameter along with formulae that assume geometric primitives. In contrast, technologies such as terrestrial Light Detection And Ranging (LiDAR) can record very detailed spatial information on the actual surface of an object, such as a tree bole. We present a method using LiDAR that provides accurate volume estimates of tree stems, as well as 2D rasters that display details of stem surfaces, which we call the “unwrapping method.” This method combines the concepts of cylinder fitting, voxelization, and digital elevation models. The method is illustrated and tested using a sample of standing trees, whereby we are able to generate accurate volume estimates from the point cloud, as well as accurate visualization of the scanned stem sections. When compared to volume estimates derived from Huber's, Smalian's, and Newton's formulae, the differences are consistent with previous studies comparing formula-derived volume estimates and water-displacement-derived volume estimates, suggesting the unwrapping method has comparable accuracy to water displacement.

Rapeseed Leaf Estimation Methods at Field Scale by Using Terrestrial LiDAR Point Cloud

Exploring the key technologies of agricultural robots is an inevitable trend in the development of smart agriculture. It is significant to continuously transplant and develop novel algorithms and models to update agricultural robots that use light detection and ranging (LiDAR) as a remote sensing method. This paper implements a method for extracting and estimating rapeseed leaves through agricultural robots based on LiDAR point cloud, taking leaf area (LA) measurement as an example. Firstly, the three-dimensional (3D) point cloud obtained with a terrestrial laser scanner (TLS) were used to extract crop phenotypic information. We then imported the point cloud within the study area into a custom hybrid filter, from which the rapeseed point cloud was segmented. Finally, a new LA estimation model, based on the Delaunay triangulation (DT) algorithm was proposed, namely, LA-DT. In this study, a crop canopy analyzer, LAI-2200C, was used to measure rapeseed LA in farmland. The measured values were employed as standard values to compare with the calculated results obtained using LA-DT, and the differences between the two methods were within 3%. In addition, 100 individual rapeseed crops were extracted, and the output of the LA-DT model was subjected to linear regression analysis. The R² of the regression equation was 0.93. The differences between the outputs of the LAI-2200C and LA-DT in these experiments passed the paired samples t-test with significant correlation (p &lt; 0.01). All the results of the comparison and verification showed that the LA-DT has excellent performance in extracting LA parameters under complex environments. These results help in coping with the complex working environment and special working objects of agricultural robots. This is of great significance for expanding the interpretation methods of agricultural 3D information.