Image-based laser point cloud building facade structure extraction method by considering semantic information

Facade structures from three‐dimensional (3D) point cloud data (PCD) and two‐dimensional (2D) optical images can provide significant information for 3D building modeling. However, a unified data model for integrating 2D imagery pixels and 3D PCD is absent in current methods, leading to a complex implementation process, large calculations, and inefficiency. An efficient facade structure extraction method for building facades is proposed in this study. Based on the conversion matrix, 2D image and 3D PCD information are merged to build an image‐based laser point cloud (ILPC) data model first. Second, both the line segment detection and random sample consensus algorithms are improved according to the structure and characteristics of the ILPC data model. Finally, building facade structures are extracted and optimized. Facade structures can be extracted accurately and efficiently by the proposed method, which contains rich information support from the ILPC data model. The proposed method extracts fine building facade structures with accuracy over 0.68 in all experiments and recall up to 0.81, which are better than the Wang method. Extracted structures constitute valuable support for numerous fields, such as 3D building modeling and building information modeling construction.

In recent years, point cloud data generated with RGB-D cameras, 3D lasers, and 3D LiDARs have been employed frequently in robotic applications. In indoor environments, RGB-D cameras, which have short-range and can only describe the vicinity of the robots, generally are opted due to their low cost. On the other hand, 3D lasers and LiDARs can capture long-range measurements and generally are used in outdoor applications. In this study, we deal with the segmentation of indoor planar surfaces such as wall, floor, and ceiling via point cloud data. The segmentation methods, which are situated in Point Cloud Library (PCL) were executed with 3D laser point cloud data. The experiments were conducted to evaluate the performance of these methods with the publicly available Fukuoka indoor laser dataset, which has point clouds with different noise levels. The test results were compared in terms of segmentation accuracy and the time elapsed for segmentation. Besides, the general characteristics of each method were discussed. In this way, we revealed the positive and negative aspects of these methods for researchers that plan to apply them to 3D laser point cloud data.

High resolution satellite imagery and 3D laser point cloud data provide precise geometry, rich spectral information and clear texture of feature. The segmentation of high resolution remote sensing images and 3D laser point cloud is the basis of object-oriented remote sensing image analysis, for the segmentation results will directly influence the accuracy of subsequent analysis and discrimination. Currently, there still lacks a common segmentation theory to support these algorithms. So when we face a specific problem, we should determine applicability of the segmentation method through segmentation accuracy assessment, and then determine an optimal segmentation. To today, the most common method for evaluating the effectiveness of a segmentation method is subjective evaluation and supervised evaluation. For providing a more objective evaluation result, we have carried out following work. Analysis and comparison previous proposed image segmentation accuracy evaluation methods, which are area-based metrics, location-based metrics and combinations metrics. 3D point cloud data, which was gathered by Reigl VZ1000, was used to make two-dimensional transformation of point cloud data. The object-oriented segmentation result of aquaculture farm, building and farmland polygons were used as test object and adopted to evaluate segmentation accuracy.

Repairing point cloud holes has become an important problem in the research of 3D laser point cloud data, which ensures the integrity and improves the precision of point cloud data. However, for the point cloud data with non-characteristic holes, the boundary data of point cloud holes cannot be used for repairing. Therefore, this paper introduces photogrammetry technology and analyzes the density of the image point cloud data with the highest precision. The 3D laser point cloud data are first formed into hole data with sharp features. The image data are calculated into six density image point cloud data. Next, the barycenterization Bursa model is used to fine-register the two types of data and to delete the overlapping regions. Then, the cross-section is used to evaluate the precision of the combined point cloud data to get the optimal density. A three-dimensional model is constructed for this data and the original point cloud data, respectively and the surface area method and the deviation method are used to compare them. The experimental results show that the ratio of the areas is less than 0.5%, and the maximum standard deviation is 0.0036 m and the minimum is 0.0015 m.

Building facade structures serve as important data support for three-dimensional (3D) building modelling. A building facade structure extraction method based on 3D Laser Point Cloud Data (LPCD) by considering semantic information is proposed. The proposed method mainly involves three steps. First, an improved 3D LPCD semantic segmentation method is introduced to extract and label point clouds as the building structure and walls. Second, structure line segments are recognized from the initial building facade structure based on the Regulated Block RANSAC (RB-RANSAC) algorithm. Third, geometric information such as distance, vector and positional relationship between structure line segments is considered to optimize the initial building facade structure. The proposed method can effectively extract and fit segment structures by considering the semantic and geometric information, obtaining significantly accurate and concise building facade structures. Case studies on 3D LPCD of local buildings and open-source datasets (Semantic3D) prove the extraction accuracy and efficiency of the proposed method.

Quality assessment is one of the essential aspects of the acceptance of transmission line project completion. To resolve the challenge of efficiency and accuracy, we presents a multi-modal data fusion approach established on two new technology, which are the 3D laser point cloud technology and the BIM technology. The scanned 3D laser point cloud data of transmission lines are firstly filtered and classified to build the laser point cloud model; The BIM based 3D model of transmission lines are then converted to the BIM point cloud model; the laser point cloud model and the BIM point cloud model are registered through application of both a coarse registration and a fine registration; Differences between the laser and the BIM point cloud models are assessed for the acceptance of transmission line project. Validation results showed that the multi-modal fusion algorithm had highly satisfactory performance.

Since its appearance, airborne laser scanning technology has become more convenient and modern with the continuous progress and development of science and technology. Airborne LiDAR has a very wide range of applications in various fields. It can obtain the three-dimensional coordinates of the measured object comprehensively and with high quality. It has the advantages of high efficiency and high precision in DEM production. In this paper, DEM is produced in an area near Guantao County, Handan City. Firstly, airborne LiDAR is used to collect 3D laser point cloud data, and then software is used to process the point cloud. TerraSolid software is used to perform coordinate conversion and point cloud classification on the point cloud, and finally DEM is produced on the classified ground points. The result is a realistic and intuitive 3D model. The results show that the 3D point cloud data collected by airborne LiDAR can fully and accurately reflect the topographic and geomorphic information, and the software tools and algorithms in the processing process can effectively improve the availability and accuracy of the data, and the generated DEM data can provide important data support for terrain analysis, urban planning, natural resource management and other fields.

The segmentation of crop organs from 3D laser point clouds is an important prerequisite work of crop phenotypic parameters in non-destructive measurement. This paper respectively selected the 3D point cloud data of the rapeseed plant with leaf stage and pod stage as the research materials. A novel normal vector-based method for segmentation of the 3D point cloud is presented. First, a 3D scanner, HandyScan 300, was used to obtain 3D point cloud data. Second, using the voxel-based grid method, the original point cloud data were down-sampled at the premise of keeping the shape of point cloud unchanged. Third, according to the characteristics of the point cloud, the two conditions of the normal vector difference and the Euclidean distance between each point could be merged into two necessary conditions of the current class. Finally, the nearest point was searched with a set of labeled point cloud growth and through each point cloud of European radius until the collection of point cloud and the adjacent candidate was in accordance with the current conditions of the finished classification process. Results showed that the angle difference threshold of the normal vector was [0.91, 0.95]. The segmentation effect of the point cloud data of the leaves of the rapeseed plant was the best, which avoided the problem of misclassification and the appearance of over-segmentation. The angle difference threshold of the normal vector was [0.88, 0.91]. The segmentation effect of the point cloud data of the pod of the rapeseed plant was the best, and the accuracy rate reached 97%. Therefore, the validity and feasibility of the method was verified. Accurate segmentation of the plant organ is another foundation for the nondestructive measurement of the phenotypic parameters in the later stage.

The convex – concave feature of 3D laser point cloud data (3D LPCD) is introduced in fragmentation calculation for blast muck piles in open-pit copper mines to improve the calculation efficiency and accuracy of the blast fragmentation of muck piles (BFMP). First, a supervoxel segmentation method based on point cloud curvature feature (named as CFSS method) is constructed to overcome drawbacks, such as the color difference of the same blast muck pile surface is not evident and has little effect on the 3D LPCD segmentation. Then, a supervoxel clustering method based on boundary extraction (named as BESC method) is constructed to cluster the supervoxel data of the blast muck piles. Finally, on the basis of the supervoxel clustering results, the automatic fragment recognition and fragmentation calculation of the blast muck piles are realized, and the evaluation metrics for the calculation results of the BFMP are introduced. Dexing Copper Mine, the largest open-pit copper mine in China, is considered the study area. Case study results and discussion tests reveal that the ratios of big blocks in the three selected test areas are 0.91%, 0.21%, and 0.83%, and more than 80% of the fragments in the blast muck piles are less than 300 mm, satisfying the actual situation and confirming the effectiveness and accuracy of the proposed method, which is significant for optimizing blast designs and improving mining efficiency in open-pit operations.

Zhao, X.; Wang, X.; Zhao, J., and Zhou, F., 2021. An improved water-land discriminator using laser waveform amplitudes and point cloud elevations of airborne LIDAR. Journal of Coastal Research, 37(6), 1158–1172. Coconut Creek (Florida), ISSN 0749-0208. Laser waveforms or point clouds of airborne LIDAR are used to distinguish water and land. However, false alarms often occur in coastal areas with complex environments when laser waveforms are used. Elevations of three-dimensional (3D) point clouds can be used to help discriminate ocean and land but fail to identify inland waters. An improved water-land discriminator that uses laser waveform amplitudes and point cloud elevations is proposed in this study. First, 3D point cloud elevations derived using infrared (IR) laser are used as features to conduct ocean-land discrimination with fuzzy c-means clustering. Second, amplitudes of IR laser waveforms are used to identify inland waters from land derived by ocean-land discrimination. The proposed method is applied to data collected using Optech coastal zone mapping and imaging LIDAR (CZMIL). Land boundary derived from digital orthophoto map is used as a reference to evaluate different water-land discriminators. Results showed that consistency of the water-land interface derived using the improved water-land discriminator is higher with the reference interface than that derived by traditional waveform saturation, waveform clustering, and 3D point cloud methods. Overall accuracy of water-land points discriminated via waveform saturation, waveform clustering, 3D point cloud, and improved water-land discriminator is 93.80%, 93.84%, 95.66%, and 99.27%, respectively. High accuracy of the water-land interface and points indicates the effectiveness of the improved water-land discriminator for Optech CZMIL.

Point cloud segmentation is necessary for obtaining highly precise morphological traits in plant phenotyping. Although a huge development has occurred in point cloud segmentation, the segmentation of point clouds from complex plant leaves still remains challenging. Rapeseed leaves are critical in cultivation and breeding, yet traditional two-dimensional imaging is susceptible to reduced segmentation accuracy due to occlusions between plants. The current study proposes the use of binocular stereo-vision technology to obtain three-dimensional (3D) point clouds of rapeseed leaves at the seedling and bolting stages. The point clouds were colorized based on elevation values in order to better process the 3D point cloud data and extract rapeseed phenotypic parameters. Denoising methods were selected based on the source and classification of point cloud noise. However, for ground point clouds, we combined plane fitting with pass-through filtering for denoising, while statistical filtering was used for denoising outliers generated during scanning. We found that, during the seedling stage of rapeseed, a region-growing segmentation method was helpful in finding suitable parameter thresholds for leaf segmentation, and the Locally Convex Connected Patches (LCCP) clustering method was used for leaf segmentation at the bolting stage. Furthermore, the study results show that combining plane fitting with pass-through filtering effectively removes the ground point cloud noise, while statistical filtering successfully denoises outlier noise points generated during scanning. Finally, using the region-growing algorithm during the seedling stage with a normal angle threshold set at 5.0/180.0* M_PI and a curvature threshold set at 1.5 helps to avoid the under-segmentation and over-segmentation issues, achieving complete segmentation of rapeseed seedling leaves, while the LCCP clustering method fully segments rapeseed leaves at the bolting stage. The proposed method provides insights to improve the accuracy of subsequent point cloud phenotypic parameter extraction, such as rapeseed leaf area, and is beneficial for the 3D reconstruction of rapeseed.

Three dimension (3D) point cloud data in fog-filled environments were measured using light detection and ranging (LIDAR). Disaster response robots cannot easily navigate through such environments because this data contain false data and distance errors caused by fog. We propose a method for recognizing and removing fog based on 3D point cloud features and a distance correction method for reducing measurement errors. Laser intensity and geometrical features are used to recognize false data. However, these features are not sufficient to measure a 3D point cloud in fog-filled environments with 6 and 2 m visibility, as misjudgments occur. To reduce misjudgment, laser beam penetration features were added. Support vector machine (SVM) and K-nearest neighbor (KNN) are used to classify point cloud data into ‘fog’ and ‘objects.’ We evaluated our method in heavy fog (6 and 2 m visibility). SVM has a better F-measure than KNN; it is higher than 90% in heavy fog (6 and 2 m visibility). The distance error correction method reduces distance errors in 3D point cloud data by a maximum of 4.6%. A 3D point cloud was successfully measured using LIDAR in a fog-filled environment. Our method’s recall (90.1%) and F-measure (79.4%) confirmed its robustness.

In recent years, due to the significant advancements in hardware sensors and software technologies, 3D environmental point cloud modeling has gradually been applied in the automation industry, autonomous vehicles, and construction engineering. With the high-precision measurements of 3D LiDAR, its point clouds can clearly reflect the geometric structure and features of the environment, thus enabling the creation of high-density 3D environmental point cloud models. However, due to the enormous quantity of high-density 3D point clouds, storing and processing these 3D data requires a considerable amount of memory and computing time. In light of this, this paper proposes a real-time 3D point cloud environmental contour modeling technique. The study uses the point cloud distribution from the 3D LiDAR body frame point cloud to establish structured edge features, thereby creating a 3D environmental contour point cloud map. Additionally, unstable objects such as vehicles will appear during the mapping process; these specific objects will be regarded as not part of the stable environmental model in this study. To address this issue, the study will further remove these objects from the 3D point cloud through image recognition and LiDAR heterogeneous matching, resulting in a higher quality 3D environmental contour point cloud map. This 3D environmental contour point cloud not only retains the recognizability of the environmental structure but also solves the problems of massive data storage and processing. Moreover, the method proposed in this study can achieve real-time realization without requiring the 3D point cloud to be organized in a structured order, making it applicable to unorganized 3D point cloud LiDAR sensors. Finally, the feasibility of the proposed method in practical applications is also verified through actual experimental data.

The digital documentation of cultural relics plays an important role in archiving, protection, and management. In the field of cultural heritage, three-dimensional (3D) point cloud data is effective at expressing complex geometric structures and geometric details on the surface of cultural relics, but lacks semantic information. To elaborate the geometric information of cultural relics and add meaningful semantic information, we propose a modeling and processing method of smart point clouds of cultural relics with complex geometries. An information modeling framework for complex geometric cultural relics was designed based on the concept of smart point clouds, in which 3D point cloud data are organized through the time dimension and different spatial scales indicating different geometric details. The proposed model allows smart point clouds or a subset to be linked with semantic information or related documents. As such, this novel information modeling framework can be used to describe rich semantic information and high-level details of geometry. The proposed information model not only expresses the complex geometric structure of the cultural relics and the geometric details on the surface, but also has rich semantic information, and can even be associated with documents. A case study of the Dazu Thousand-Hand Bodhisattva Statue, which is characterized by a variety of complex geometries, reveals that our proposed framework is capable of modeling and processing the statue with excellent applicability and expansibility. This work provides insights into the sustainable development of cultural heritage protection globally.

Many place recognition approaches, which identify previously visited places or locations by matching current sensory data, such as 2D RGB images and 3D point clouds, have been proposed to achieve accurate and robust localization and loop closure detection in global positioning system (GPS)-denied environments. Since visual place recognition (VPR) methods that rely on images captured by camera sensors are highly sensitive to variations in appearance, including changes in lighting, surface color, and shadows, they can lead to poor place recognition accuracy. In contrast, light detection and ranging (LiDAR)-based place recognition (LPR) approaches based on 3D point cloud data that captures the shape and geometric structure of the environment are robust to changes in place appearance and can therefore provide more reliable place recognition results than VPR methods. This work presents an indoor LPR method called PointNetVLAD-based indoor pedestrian localization (PIPL). PIPL is a deep network model that uses PointNetVLAD to learn to extract global descriptors from 3D LiDAR point cloud data. PIPL can recognize places previously visited by a pedestrian using point clouds captured by a low-cost LiDAR sensor on a smartphone in small-scale indoor environments, while PointNetVLAD performs place recognition for vehicles using high-cost LiDAR, GPS, and inertial measurement unit (IMU) sensors in large-scale outdoor areas. For place recognition on 3D point cloud reference maps generated from LiDAR scans, PointNetVLAD exploits the universal transverse mercator (UTM) coordinate system based on GPS and IMU measurements, whereas PIPL uses a virtual coordinate system designed in this study due to the unavailability of GPS indoors. In experiments conducted in campus buildings, PIPL shows significant advantages over NetVLAD (known as a convolutional neural network (CNN)-based VPR method). Particularly in indoor environments with repetitive scenes where geometric structures are preserved and image-based appearance features are sparse or unclear, PIPL achieved 39% higher top-1 accuracy and 10% higher top-3 accuracy compared to NetVLAD. Furthermore, PIPL achieved place recognition accuracy comparable to NetVLAD even with a small number of points in a 3D point cloud and outperformed NetVLAD even with a smaller model training dataset. The experimental results also indicate that PIPL requires over 76% less place retrieval time than NetVLAD while maintaining robust place classification performance.