Urban Point Clouds Research Articles

LightUrban: Similarity Based Fine‐grained Instancing for Lightweighting Complex Urban Point Clouds

AbstractLarge‐scale urban point clouds play a vital role in various applications, while rendering and transmitting such data remains challenging due to its large volume, complicated structures, and significant redundancy. In this paper, we present LightUrban, the first point cloud instancing framework for efficient rendering and transmission of fine‐grained complex urban scenes. We first introduce a segmentation method to organize the point clouds into individual buildings and vegetation instances from coarse to fine. Next, we propose an unsupervised similarity detection approach to accurately group instances with similar shapes. Furthermore, a fast pose and size estimation component is applied to calculate the transformations between the representative instance and the corresponding similar instances in each group. By replacing individual instances with their group's representative instances, the data volume and redundancy can be dramatically reduced. Experimental results on large‐scale urban scenes demonstrate the effectiveness of our algorithm. To sum up, our method not only structures the urban point clouds but also significantly reduces data volume and redundancy, filling the gap in lightweighting urban landscapes through instancing.

A precise registration method for large-scale urban point clouds based on phased and spatial geometric features

Abstract The dense high-rise buildings and multipath effects in urban areas significantly reduce the positioning signal accuracy of laser scanning systems, leading to layering and offset issues in the collected point cloud data on the same road. In order to acquire comprehensive and consistent three-dimensional information on the objects, thereby providing field inspection data for large-scale road traffic network scenarios, in this paper, an improved point cloud registration method is proposed to divide the registration process into two stages: elevation registration and plane registration. Elevation registration takes the ground point cloud as the registration primitive, reduces the number of point clouds through curvature down-sampling, and constrains the feature point sequence with a fixed range to provide a good initial pose for fine registration. The plane registration first inherits the elevation registration parameters, combining the dynamic distance parameters of spherical region step based on the median, using robust multi-scale loss functions to address residual points, effective adjacent point pairs are selected to obtain the spatial transformation matrix, and realizes accurate registration. Experimental results with multiple sets of urban point cloud data show that the root mean square error of point cloud registration can be controlled within 0.06 m, achieving a relatively superior registration accuracy, it can provide detailed prior data for measurement information analysis.

Enhancing explainability of deep learning models for point cloud analysis: a focus on semantic segmentation

ABSTRACT Semantic segmentation of point clouds plays a critical role in various applications, such as urban planning, infrastructure management, environmental analyses and autonomous navigation. Understanding the behaviour of deep neural networks (DNNs) in analysing point cloud data is essential for improving segmentation accuracy and developing effective network architectures and acquisition strategies. In this paper, we investigate the traits of some state-of-the-art neural networks using indoor and urban outdoor point cloud datasets. We compare PointNet, DGCNN, and BAAF-Net on specifically selected datasets, including synthetic and real-world environments. The chosen datasets are S3DIS, SynthCity, Semantic3D, and KITTI. We analyse the impact of different factors such as dataset type (synthetic vs. real), scene type (indoor vs. outdoor), and acquisition system (static vs. mobile sensors). Through detailed analyses and comparisons, we provide insights into the strengths and limitations not only of different network architectures in handling urban point clouds but also of their data structure. This study contributes to going beyond the mere and unconditional use of AI algorithms, trying to explain DNNs behaviour in point cloud analysis and paving the way for future research to enhance segmentation accuracy and develop possible guidelines both for network design and data acquisition in the geomatics field.

Multi-source point cloud registration for urban areas using a coarse-to-fine approach

ABSTRACT The foundational step in constructing a digital twin involves transforming the physical world into an immersive 3D virtual environment. Unmanned aerial vehicle (UAV)-based photogrammetry, mobile mapping system (MMS), and terrestrial laser scanning (TLS) serve as powerful tools for 3D reconstruction of urban environments. Given the distinctive strengths and weaknesses of each sensor, a multi-source point cloud registration technique is imperative to achieve a detailed and precisely geolocated 3D urban model. Hence, we developed coarse-to-fine approach methods, called building exterior wall-based (BEWB) and building outline-based (BOB) algorithms, to register point clouds captured by diverse sensors in urban scenes. The BEWB algorithm coarsely registers urban point clouds by extracting building exterior walls, establishing correspondence points, and effectively removing outliers within this correspondence point set. The BOB algorithm precisely registers the urban point clouds acquired from multiple sensors by leveraging building outlines and points corresponding to the ground of the point clouds. To validate the proposed algorithms, we initially registered a UAV-based photogrammetry point cloud with the MMS point cloud and subsequently registered the TLS point clouds. Comprehensive quantitative and qualitative analyzes of the results demonstrate that our algorithms outperform existing methods in achieving precise registration. By successfully registering three distinct point clouds, we generated a comprehensive urban scene point cloud characterized by enhanced precision, point density, and geolocation accuracy.

SEMANTIC POINT CLOUD SEGMENTATION IN URBAN ENVIRONMENTS WITH 1D CONVOLUTIONAL NEURAL NETWORKS

Abstract. Convolutional Neural Networks (CNNs) have been widely recognized for their efficacy in image analysis tasks. This paper investigates the application of the 1D-CNN variant CNNs for the semantic segmentation of urban point clouds obtained through Mobile Laser Scanning. Ten well-known local geometric features of point clouds were used as input for the 1D CNN. Through an empirical analysis on the Santiago Urban Dataset, the 1D CNN was optimized in terms of numbers of convolution layers, neurons, pooling layers, dropout layers, dense layers, training epochs, and batch size. The performance of the proposed 1D CNN was compared with Support Vector Machine (SVM), Random Forest (RF), and PointNet++. Despite demonstrating a F1-score weighted at 70.3%, outperforming SVM but slightly lagging RF (71.6%) and significantly trailing PointNet++ (90.3%), the proposed 1D-CNN showcases a cost-effective potential for the segmentation of road and building classes. The relative computational requirements of the models were also discussed, highlighting the practical advantages and limitations of each approach.

The impact of electrical hazards from overhead power lines on urban search and rescue operations during extreme flood events

Accurate flood forecasting and efficient emergency response operations are vital, especially in the case of urban flash floods. The dense distribution of power lines in urban areas significantly impacts search and rescue operations during extreme flood events. However, no existing emergency response frameworks have incorporated the impacts of overhead power lines on lifeboat rescue operations. This study aims to determine the necessity and feasibility of incorporating overhead power line information into an emergency response framework using Manville, New Jersey during Hurricane Ida as a test bed. We propose an integrated framework, which includes a building-scale flood model, urban point cloud data, a human vulnerability model, and network analysis, to simulate rescue operation feasibility during Hurricane Ida. Results reveal that during the most severe point of the flood event, 46% of impacted buildings became nonrescuable due to complete isolation from the road network, and a significant 67.7% of the municipality’s areas that became dangerous for pedestrians also became inaccessible to rescue boats due to overhead power line obstruction. Additionally, we identify a continuous 10-hour period during which an average of 43.4% of the 991 impacted buildings faced complete isolation. For these structures, early evacuation emerges as the sole means to prevent isolation. This research highlights the pressing need to consider overhead power lines in emergency response planning to ensure more effective and targeted flood resilience measures for urban areas facing increasingly frequent extreme precipitation events.

A high-accuracy multi-temporal laser point cloud registration method for providing a detailed expression of the urban environment

In consideration of the significant changes in urban environmental scenes and the impact of factors such as signal obstruction on positioning, the spatial positions of multi-temporal Mobile Laser Scanning (MLS) point clouds collected in the same area are inconsistent, which makes it challenging to provide a complete expression of the urban road environment. To address this issue, a precise registration algorithm based on pole-like targets is proposed. The core steps include registration primitive extraction: a cylindrical spatial neighborhood is established based on the distinctive key points, and precise extraction of pole-like geospatial point cloud data is performed using the Euclidean clustering method with the normal curvature constraint. Achieving optimal spatial transformation: a circle center fitting method with additional Mean Absolute Error (MAE) loss function constraints was developed based on the stable elemental unit which was utilized to accurately construct homonymous feature point pairs. Validation results from multiple real urban point cloud datasets collected from different experimental areas and platforms demonstrate that our method achieves superior registration performance, with mean residuals below 5 cm in both the planar and elevation directions. The proposed method exhibits strong universality and registration robustness for multi-temporal point cloud data of the homologous or cross-sources.

Semantic Segmentation and Roof Reconstruction of Urban Buildings Based on LiDAR Point Clouds

In urban point cloud scenarios, due to the diversity of different feature types, it becomes a primary challenge to effectively obtain point clouds of building categories from urban point clouds. Therefore, this paper proposes the Enhanced Local Feature Aggregation Semantic Segmentation Network (ELFA-RandLA-Net) based on RandLA-Net, which enables ELFA-RandLA-Net to perceive local details more efficiently by learning geometric and semantic features of urban feature point clouds to achieve end-to-end building category point cloud acquisition. Then, after extracting a single building using clustering, this paper utilizes the RANSAC algorithm to segment the single building point cloud into planes and automatically identifies the roof point cloud planes according to the point cloud cloth simulation filtering principle. Finally, to solve the problem of building roof reconstruction failure due to the lack of roof vertical plane data, we introduce the roof vertical plane inference method to ensure the accuracy of roof topology reconstruction. The experiments on semantic segmentation and building reconstruction of Dublin data show that the IoU value of semantic segmentation of buildings for the ELFA-RandLA-Net network is improved by 9.11% compared to RandLA-Net. Meanwhile, the proposed building reconstruction method outperforms the classical PolyFit method.

LFEA-Net: semantic segmentation for urban point cloud scene via local feature extraction and aggregation

Considering the increasing prominence of 3D real city construction technology, 3D urban point cloud scene data merit further investigation. However, achieving finegrained semantic segmentation of urban scenes remains highly challenging due to the natural orderlessness and unstructured nature of acquired point clouds, along with their large-scale points and non-uniform distributions. In this study, we present LFEA-Net, a novel neural network specifically designed for semantic segmentation of large-scale urban point cloud scenes. The network comprises two main components: (1) The local feature extraction (LFE) module, which fully exploits local spatial, color and semantic information to enhance and preserve crucial information, consisting of bilateral feature encoding unit and multidimensional feature encoding unit. (2) The local feature aggregation (LFA) module, designed to bridge the semantic gap between local information and emphasize both local significant features and the entire local neighbor, consisting of soft cross operation and united pooling operation. We have evaluated the performance of LFEA-Net with state-of-the-art networks using the photogrammetric point cloud dataset SensatUrban, achieving 61.6 of mIoU score. The results demonstrate the superior efficacy of LFEA-Net in accurately segmenting and classifying large-scale urban point cloud scenes, highlighting its potential to advance environmental information perception.

METAMORPHISM OF ALS POINT DATA FOR MULTITUDE APPLICATION

Abstract. Technologically assisted decision-making in urban planning and governance is significant to envisage Sustainable Development Goal (SDG) 11, of developing sustainable cities and communities. In the current millennium, planners and decision-makers require knowledge-rich virtual models for managing the man-made and natural resources in the city. To effectively utilize the technological advancement for sustainable urban development, there is a need for expeditious entail towards accurate urban resource mapping, development of flexible monitoring and information extraction framework, and enticing visualization. Airborne LiDAR Scanning (ALS) is capable of producing very precise 3D geometric data over expansive urban areas in a timely and economically efficient manner. However, it is challenging to derive viable outcomes from the unstructured and voluminous point cloud. This work proposes an intermediary metamorphosed point cloud storage framework to enhance the utility of point clouds for multiple pragmatic applications. The proposed methodological approach transforms the unstructured, massive point cloud into an ontologically stored urban object collection to utilize the large-scale urban point cloud in decision-making. Further, the paper demonstrates the direct applicability of the metamorphosed point cloud storage framework for two specific applications related to sustainable urban development. Experiments carried out using the proposed framework on DALES benchmark dataset show promising results.

Combining point cloud and surface methods for modeling partial shading impacts of trees on urban solar irradiance

Although trees and urban vegetation have a significant influence on solar irradiation in the built environment, their impact on daylight and energy consumption is often not considered in building performance and urban environment simulation studies. This paper presents a novel method for comprehensive solar irradiance assessment that considers the dynamic partial shading impacts from trees. The proposed method takes urban point clouds as input and consists of three subsequent steps: (a) DGCNN-based segmentation, (b) fusion model generation, (c) matrix-based irradiance calculation. The method is validated by comparing model outputs with field measurement data, and an inter-model comparison with eleven state-of-the-art tree shading modeling approaches. Analyses carried out on daily and long-term basis show that the proposed fusion model can significantly reduce simulation errors compared to alternative approaches, while limiting the required input data to a minimum. The primary source of uncertainty stems from mismatches between tree morphology in the fusion model and reality, attributable to phenological growth and seasonal variations.

An Accurate and Efficient Supervoxel Re-Segmentation Approach for Large-Scale Point Clouds Using Plane Constraints

The accurate and efficient segmentation of large-scale urban point clouds is crucial for many higher-level tasks, such as boundary line extraction, point cloud registration, and deformation measurement. In this paper, we propose a novel supervoxel segmentation approach to address the problem of under-segmentation in local regions of point clouds at various resolutions. Our approach introduces distance constraints from boundary points to supervoxel planes in the merging stage to enhance boundary segmentation accuracy between non-coplanar supervoxels. Additionally, supervoxels with roughness above a threshold are re-segmented using random sample consensus (RANSAC) to address multi-planar coupling within local areas of the point clouds. We tested the proposed method on two publicly available large-scale point cloud datasets. The results show that the new method outperforms two classical methods in terms of boundary recall, under-segmentation error, and average entropy in urban scenes.

Classification of Large-Scale Mobile Laser Scanning Data in Urban Area with LightGBM

Automatic point cloud classification (PCC) is a challenging task in large-scale urban point clouds due to the heterogeneous density of points, the high number of points and the incomplete set of objects. Although recent PCC studies rely on automatic feature extraction through deep learning (DL), there is still a gap for traditional machine learning (ML) models with hand-crafted features, particularly after emerging gradient boosting machine (GBM) methods. In this study, we are using the traditional ML framework for the problem of PCC in large-scale datasets following the steps of neighborhood definition, multi-scale feature extraction, and classification. Different from others, our framework takes advantage of the fast feature calculation with multi-scale radius neighborhood and a recent state-of-the-art GBM classifier, LightGBM. We tested our framework using three mobile urban datasets, Paris–Rau–Madame, Paris–Rue–Cassette and Toronto3D. According to the results, our framework outperforms traditional machine learning models and competes with DL-based methods.

A discordance analysis in manual labelling of urban mobile laser scanning data used for deep learning based semantic segmentation

Labelled point clouds are crucial to train supervised Deep Learning (DL) methods used for semantic segmentation. The objective of this research is to quantify discordances between the labels made by different people in order to assess whether such discordances can influence the success rates of a DL based semantic segmentation algorithm. An urban point cloud of 30 m road length in Santiago de Compostela (Spain) was labelled two times by ten persons. Discordances and its significance in manual labelling between individuals and rounds were calculated. In addition, a ratio test to signify discordance and concordance was proposed. Results show that most of the points were labelled accordingly with the same class by all the people. However, there were many points that were labelled with two or more classes. Class curb presented 5.9% of discordant points and 3.2 discordances for each point with concordance by all people. In addition, the percentage of significative labelling differences of the class curb was 86.7% comparing all the people in the same round and 100% comparing rounds of each person. Analysing the semantic segmentation results with a DL based algorithm, PointNet++, the percentage of concordance points are related with F-score value in R2 = 0.765, posing that manual labelling has significant impact on results of DL-based semantic segmentation methods.

Instance recognition of street trees from urban point clouds using a three-stage neural network

As one of the most important components of urban space, the geometric and semantic properties of road trees are crucial for the assessment and upgrade of urban environments. Accurate positioning of individual trees is a critical procedure for this task; however, traditional manual methods require considerable personnel and material resources. By contrast, individual tree segmentation based on a vehicle-mounted mobile laser scanner (MLS) operating along urban streets can provide a feasible and efficient solution. Recent studies (especially those based on deep learning) on point cloud semantic segmentation and forest tree detection have achieved significant progress; however, they rarely perform instance recognition of urban trees. In addition, extracting individual urban trees from point clouds to balance accuracy and efficiency remains challenging. To this end, we propose an effective pipeline to segment individual trees from urban MLS data. The proposed approach encompasses two main steps: tree-point extraction and individual-tree segmentation. First, a patch-guided semi-supervised semantic segmentation network was designed to separate non-tree points from entire urban scene. The extracted tree points are further divided into instance-level trees using a three-stage segmentation network for the individual tree segmentation. Specifically, a tree-centroid prediction module is proposed to ensure the accurate localization of street trees, even with irregular positions, in complex urban environments. Subsequently, a special confidence-aware attention mechanism is developed to resolve challenging ambiguities from complementary perspectives. Finally, a dynamic shifting-based clustering algorithm is introduced to speed up the segmentation of each tree guided by the predicted centroid and directly transfer the IDs back to the point clouds. The proposed method is evaluated and validated on MLS point clouds collected in two urban areas, robustly producing accurate instance-level roadside trees. The instance-level segmentation results in average precision and recall of (96.62%, 96.17%) and (95.62%, 94.58%), respectively, on these two MLS point cloud datasets. Point-level segmentation also exhibits mean precision, recall, and weighted coverage exceeding 90%. We also verify that the proposed method could achieve a promising result on an airborne laser scanning point cloud dataset without retraining, thereby demonstrating a good generalization ability. To further verify the quality of the segmented tree instances, a reconstruction method is adopted to employ these independent street trees in trunk point cloud modeling. Our contribution may assist in developing intelligent urban green space planning and management.

Road-Side Individual Tree Segmentation from Urban MLS Point Clouds Using Metric Learning

As one of the most important components of urban space, an outdated inventory of road-side trees may misguide managers in the assessment and upgrade of urban environments, potentially affecting urban road quality. Therefore, automatic and accurate instance segmentation of road-side trees from urban point clouds is an important task in urban ecology research. However, previous works show under- or over-segmentation effects for road-side trees due to overlapping, irregular shapes and incompleteness. In this paper, a deep learning framework that combines semantic and instance segmentation is proposed to extract single road-side trees from vehicle-mounted mobile laser scanning (MLS) point clouds. In the semantic segmentation stage, the ground points are filtered to reduce the processing time. Subsequently, a graph-based semantic segmentation network is developed to segment road-side tree points from the raw MLS point clouds. For the individual tree segmentation stage, a novel joint instance and semantic segmentation network is adopted to detect instance-level roadside trees. Two complex Chinese urban point cloud scenes are used to evaluate the individual urban tree segmentation performance of the proposed method. The proposed method accurately extract approximately 90% of the road-side trees and achieve better segmentation results than existing published methods in both two urban MLS point clouds. Living Vegetation Volume (LVV) calculation can benefit from individual tree segmentation. The proposed method provides a promising solution for ecological construction based on the LVV calculation of urban roads.

Vegetation Extraction from Airborne Laser Scanning Data of Urban Plots Based on Point Cloud Neighborhood Features

This study proposes an accurate vegetation extraction method used for airborne laser scanning data of an urban plot based on point cloud neighborhood features to overcome the deficiencies in the current research on the precise extraction of vegetation in urban plots. First, the plane features in the R-neighborhood are combined with Euclidean distance clustering to extract the building point cloud accurately, and the rough vegetation point cloud is extracted using the discrete features in the R-neighborhood. Then, under the building point cloud constraints, combined with the Euclidean distance clustering method, the remaining building boundary points in the rough vegetation point cloud are removed. Finally, based on the vegetation point cloud after removing the building boundary point cloud, points within a specific radius r are extracted from the vegetation point cloud in the original data, and a complete urban plot vegetation extraction result is obtained. Two urban plots of airborne laser scanning data are selected to calculate the point cloud plane features and discrete features with R = 0.6 m and accurately extract the vegetation point cloud from the urban point cloud data. The visual effect and accuracy analysis results of vegetation extraction are compared under four different radius ranges of r = 0.5 m, r = 1 m, r = 1.5 m and r = 2 m. The best vegetation extraction results of the two plots are obtained for r = 1 m. The recall and precision are obtained as 92.19% and 98.74% for plot 1 and 94.30% and 98.73% for plot 2, respectively.

Strict rule-based automatic training data extraction using Mobile laser scanning in urban area

To reduce the cost of manually annotating training data for supervised classifiers, we propose an automated approach to extract training data of urban objects in six classes: buildings, fences, man-made poles, vegetation, vehicles, and low objects. In this study, two segmentation algorithms are firstly implemented to generate meaningful objects from the non-ground point cloud. Then, we generated valid strict rules to label partial RANSAC (Random Sample Consensus) planes and meaningful objects as training data. The strict rules are built upon the semantic knowledge formed by the features of geometric, eigenvalue, RANSAC plane, multidimensional slice, and relative location. The accuracy of strict rule-based (SRB) training data is higher than 98.5 % for buildings, man-made poles, vegetation, and vehicles. The accuracy of low objects and fences reaches 97.10 % and 94.99 %, respectively. Finally, we compared the performance of the KPConv and PointNET++ networks trained by SRB and manually labeled training data to evaluate the effectiveness of our training data. The KPConv overall accuracy using manually labeled and (SRB) training data are 91.5 % and 86.8 % in the Paris dataset, 95.6 % and 92.0 % in the Freiburg dataset, respectively. The experiments demonstrate that automatically labeled training data can achieve similar accuracy compared to manual labels when coupled with two deep learning networks. Therefore, SRB training data extraction can effectively deal with the problem of training data scarcity and provide significant advancements in urban point cloud classification, where manual labeling of training data remains a crucial challenge.

Sampling-attention deep learning network with transfer learning for large-scale urban point cloud semantic segmentation

Targeting the development of smart cities to facilitate the significant analysis of large-scale urban for construction and update. This research develops a new method named transfer learning-based sampling-attention network (TSANet) to act on 3D urban point clouds for semantic segmentation. The main contributions of this research are a segmentation model and a transfer learning protocol, where the segmentation model adopts the point downsampling–upsampling structure for time efficiency, the embedding method and an attention mechanism for point cloud feature processing, and the transfer learning protocol is employed to reduce the data requirements and labeling efforts by using prior knowledge for practical application. In addition, a focal loss is designed to assist the model for feature extraction and learning with handling data imbalance. To demonstrate the efficiency and effectiveness of the developed method, a realistic point cloud dataset of Cambridge and Birmingham cities is utilized as a case study. The results demonstrate that (1) the developed method has promising performance with overall accuracy (OA) of 0.9133 and Mean Intersection over Union (MIoU) of 0.5588; (2) the proposed transfer learning protocol contributes to the core model performance by fully combining accuracy and time efficiency, offering a 74.91% improvement in time efficiency; (3) the developed TSANet outperforms other state-of-the-art models, such as PointNet++ and DGCNN, by comparing the accuracy and time efficiency. Overall, the method developed in this research is capable of great potential as a practical application tool by presenting accurate, effective, and efficient results for semantic segmentation of large-scale 3D urban point clouds.

EFFICIENT SPARSE STREET FURNITURE EXTRACTION FROM MOBILE LASER SCANNING POINT CLOUDS

Abstract. Current mobile systems are capable of efficiently acquiring dense urban point clouds. Still, operational use of such data is hampered by the lack of efficient object extraction methodology. Notably methodology is lacking for automatically extracting objects that do not belong to the road furniture like street signs and light poles but do belong to the street furniture. As an example, object we consider public garbage bins, that are installed and should be maintained in public areas in every city. However, information about types, locations, and condition of these public garbage bins are rarely updated and only obtained through manual measurements. Therefore, an efficient way of collecting information on such public objects is of interest not only for urban management but also when developing digital twins of a city. This study proposes a new method to automatically extract public garbage bins from large urban mobile laser scanning (MLS) point clouds. The proposed method consists of three main steps: (1) cell-, (2) sub-cell-, and (3) surface-based filtering, in which both spatial information of the point clouds and contextual knowledge of the public garbage bins are incorporated to efficiently remove irrelevant 3D points at an early phase and identify and classify different types of public garbage bins. Contextual knowledge includes shape and dimensions, and the relationship between the public garbage bins and the ground surface. A MLS dataset of the city centre of Rotterdam, the Netherlands, consisting of 2.84 billion points organised in 166 tiles of 50 × 75m, and covering an area of about 750 × 750m was used to test the proposed method. Results show that the method can automatically extract ~90 public garbage bins with an overall detection rate of 89.1%. Moreover, the executing time for the entire dataset was only about 163.6 minutes, which is equivalent to 3.46 seconds per one million points. Although the method was tested here one public garbage bins, it can be easily tuned for the detection of other street furniture objects, like benches, post boxes or bollards.