3D LiDAR Data Research Articles

Recurrent-OctoMap: Learning State-Based Map Refinement for Long-Term Semantic Mapping With 3-D-Lidar Data

This paper presents a novel semantic mapping approach, Recurrent-OctoMap, learned from long-term 3D Lidar data. Most existing semantic mapping approaches focus on improving semantic understanding of single frames, rather than 3D refinement of semantic maps (i.e. fusing semantic observations). The most widely-used approach for 3D semantic map refinement is a Bayesian update, which fuses the consecutive predictive probabilities following a Markov-Chain model. Instead, we propose a learning approach to fuse the semantic features, rather than simply fusing predictions from a classifier. In our approach, we represent and maintain our 3D map as an OctoMap, and model each cell as a recurrent neural network (RNN), to obtain a Recurrent-OctoMap. In this case, the semantic mapping process can be formulated as a sequence-to-sequence encoding-decoding problem. Moreover, in order to extend the duration of observations in our Recurrent-OctoMap, we developed a robust 3D localization and mapping system for successively mapping a dynamic environment using more than two weeks of data, and the system can be trained and deployed with arbitrary memory length. We validate our approach on the ETH long-term 3D Lidar dataset [1]. The experimental results show that our proposed approach outperforms the conventional "Bayesian update" approach.

Automatic object classification according to 3d-lidar data based on single-photon counting technology

Subject of Research. We compare the effectiveness of the procedure for automatic object classification according to 3d-lidar data, built on the basis of single-photon counting technology, with the efficiency of the classification procedure with the usage of the conventional linear lidar data. Lidars based on the single-photons counting technology (SPCT) significantly exceed the ordinary linear lidars for a whole range of target parameters, including: spatial resolution - by more than an order of magnitude; the performance of the land surface scanning process - by an order of magnitude; by weight and size indicators - few times less. Unlike linear lidars, each laser point in the case of a SPCT-lidar is described not only by its coordinates and the intensity of the reflected signal, but by coordinates and an additional data block that characterizes the surface relief of the probed object in the direction of the probing photon flux. The presence of this additional data block for each laser point makes it possible to consider the received images as 3d-images, that simplifies the solution of not only the photogrammetric problem, but also the task of automatic objects classification on SPCT-image. Method. We consider the automatic object classification task solution based on SPCT-data with the use of the machine learning methods (XGBoost and multilayered neural networks - ANN). Main Results. The results of the numerical simulation carried out within the framework of the present study showed that, other things being equal, the efficiency of solving the classification problem based on SPCT-data, in practically important cases, increases up to 20% compared with the case where the linear-lidar data was used. Practical Relevance. The obtained results can be used in the design of the mobile SPCT-lidars enabling the detection and classification of objects on the Earth's surface in real time with high reliability.

Spatially sensitive statistical shape analysis for pedestrian recognition from LIDAR data

Range-based pedestrian recognition is instrumental towards the development of autonomous driving and driving assistance systems. This work introduces encoding methods for pedestrian recognition, based on statistical shape analysis of 3D LIDAR data. The proposed approach has two variants, based on the encoding of local shape descriptors either in a spatially agnostic or spatially sensitive fashion. The latter method derives more detailed cues, by enriching the ‘gross’ information reflected by overall statistics of local shape descriptors, with ‘fine-grained’ information reflected by statistics associated with spatial clusters. Experiments on artificial LIDAR datasets, which include challenging samples, as well as on a large scale dataset of real LIDAR data, lead to the conclusion that both variants of the proposed approach (i) obtain high recognition accuracy, (ii) are robust against low-resolution sampling, (iii) are robust against increasing distance, and (iv) are robust against non-standard shapes and poses. On the other hand, the spatially-sensitive variant is more robust against partial occlusion and bad clustering.

Iterative individual plant clustering in maize with assembled 2D LiDAR data

A two dimensional (2D) laser scanner was mounted at the front part of a small 4-wheel autonomous robot with differential steering, at an angle of 30 ° pointing downwards. The machine was able to drive between maize rows and collect concurrent time-stamped data. A robotic total station tracked the position of a prism mounted on the vehicle. The total station and laser scanner data were fused to generate a three dimensional (3D) point cloud. This 3D representation was used to detect individual plant positions, which are of particular interest for applications such as phenotyping, individual plant treatment and precision weeding. Two different methodologies were applied to the 3D point cloud to estimate the position of the individual plants. The first methodology used the Euclidian Clustering on the entire point cloud. The second methodology utilised the position of an initial plant and the fixed plant spacing to search iteratively for the best clusters. The two algorithms were applied at three different plant growth stages. For the first method, results indicated a detection rate up to 73.7% with a root mean square error of 3.6 cm. The second method was able to detect all plants (100% detection rate) with an accuracy of 2.7–3.0 cm, taking the plant spacing of 13 cm into account.

TITAN to Google Earth: Workflow for Data Processing for Mobile Terrestrial LIDAR

Terrestrial LIDAR scanners are pushing the boundaries of accurate urban modelling. Automation and the usability of tools used in feature abstraction and, to a lesser degree, presentation have become the chief concerns with this new technology. To broaden the use and impact of LIDAR in the geomatics, LiDAR datasets must be converted to feature-based representations without loss of precision. One approach, taken here, is to simultaneously examine the overall path that data takes through an organization and the operatordriven tasks carried out on the data as it is transformed from a raw point cloud to final product. We present a review of the current practices in LiDAR data processing and a foundation for future efforts to optimize. We examine alternative LIDAR processing workflows with two key questions in mind: computational efficiency - whether the process can be done using the tools at all - and tool complexity - what operator skill level is needed at each step. Using these workflows the usability of the specific software tools and the required knowledge to effectively carry out the procedures using the tools are examined. Preliminary results have yielded workflows that successfully translate LIDAR to 3D object models, highly decimated point representations of street data represented in Google Earth, and large volume point data flythroughs in ESRI ArcScene. We are documenting the pragmatic limits on each of these workflows and tools for endusers. Terrestrial LIDAR brings with it new innovations for spatial visualizations, but also questions of viability. The technology has proved valuable for specialized applications for experts, but can it be useful as a tool for proliferating 3d spatial information by and to non-experts. This study illustrates the issues associated with preparing 3d LIDAR data for presentation in mainstream visualization environments.

Multimodal vehicle detection: fusing 3D-LIDAR and color camera data

Most of the current successful object detection approaches are based on a class of deep learning models called Convolutional Neural Networks (ConvNets). While most existing object detection researches are focused on using ConvNets with color image data, emerging fields of application such as Autonomous Vehicles (AVs) which integrates a diverse set of sensors, require the processing for multisensor and multimodal information to provide a more comprehensive understanding of real-world environment. This paper proposes a multimodal vehicle detection system integrating data from a 3D-LIDAR and a color camera. Data from LIDAR and camera, in the form of three modalities, are the inputs of ConvNet-based detectors which are later combined to improve vehicle detection. The modalities are: (i) up-sampled representation of the sparse LIDAR’s range data called dense-Depth Map (DM), (ii) high-resolution map from LIDAR’s reflectance data hereinafter called Reflectance Map (RM), and (iii) RGB image from a monocular color camera calibrated wrt the LIDAR. Bounding Box (BB) detections in each one of these modalities are jointly learned and fused by an Artificial Neural Network (ANN) late-fusion strategy to improve the detection performance of each modality. The contribution of this paper is two-fold: (1) probing and evaluating 3D-LIDAR modalities for vehicle detection (specifically the depth and reflectance map modalities), and (2) joint learning and fusion of the independent ConvNet-based vehicle detectors (in each modality) using an ANN to obtain more accurate vehicle detection. The obtained results demonstrate that (1) DM and RM are very promising modalities for vehicle detection, and (2) experiments show that the proposed fusion strategy achieves higher accuracy compared to each modality alone in all the levels of increasing difficulty (easy, moderate, hard) in KITTI object detection dataset.

Automatic Detection and Parameter Estimation of Trees for Forest Inventory Applications Using 3D Terrestrial LiDAR

Forest inventory plays an important role in the management and planning of forests. In this study, we present a method for automatic detection and estimation of trees, especially in forest environments using 3D terrestrial LiDAR data. The proposed method does not rely on any predefined tree shape or model. It uses the vertical distribution of the 3D points partitioned in a gridded Digital Elevation Model (DEM) to extract out ground points. The cells of the DEM are then clustered together to form super-clusters representing potential tree objects. The 3D points contained in each of these super-clusters are then classified into trunk and vegetation classes using a super-voxel based segmentation method. Different attributes (such as diameter at breast height, basal area, height and volume) are then estimated at individual tree levels which are then aggregated to generate metrics for forest inventory applications. The method is validated and evaluated on three different data sets obtained from three different types of terrestrial sensors (vehicle-borne, handheld and static) to demonstrate its applicability and feasibility for a wide range of applications. The results are evaluated by comparing the estimated parameters with real field observations/measurements to demonstrate the efficacy of the proposed method. Overall segmentation and classification accuracies greater than 84 % while average parameter estimation error ranging from 1 . 6 to 9 % were observed.

Voxel-Based Neighborhood for Spatial Shape Pattern Classification of Lidar Point Clouds with Supervised Learning.

Improving the effectiveness of spatial shape features classification from 3D lidar data is very relevant because it is largely used as a fundamental step towards higher level scene understanding challenges of autonomous vehicles and terrestrial robots. In this sense, computing neighborhood for points in dense scans becomes a costly process for both training and classification. This paper proposes a new general framework for implementing and comparing different supervised learning classifiers with a simple voxel-based neighborhood computation where points in each non-overlapping voxel in a regular grid are assigned to the same class by considering features within a support region defined by the voxel itself. The contribution provides offline training and online classification procedures as well as five alternative feature vector definitions based on principal component analysis for scatter, tubular and planar shapes. Moreover, the feasibility of this approach is evaluated by implementing a neural network (NN) method previously proposed by the authors as well as three other supervised learning classifiers found in scene processing methods: support vector machines (SVM), Gaussian processes (GP), and Gaussian mixture models (GMM). A comparative performance analysis is presented using real point clouds from both natural and urban environments and two different 3D rangefinders (a tilting Hokuyo UTM-30LX and a Riegl). Classification performance metrics and processing time measurements confirm the benefits of the NN classifier and the feasibility of voxel-based neighborhood.

Classification of Outdoor 3D Lidar Data Based on Unsupervised Gaussian Mixture Models

Three-dimensional point clouds acquired with lidars are an important source of data for the classification of outdoor environments by autonomous terrestrial robots. We propose a two-layer classification model. The first layer consists of a Gaussian mixture model. This model is determined in a training step in an unsupervised manner and classified into a large set of classes. The second layer consists of a grouping of these classes. This grouping is determined by an expert during the training step and leads to a smaller set of classes that are interpretable in a considered target task. Because the first layer relies on unsupervised learning, manual labeling of data is not required. Supervision is necessary only for the second layer and in this case is assisted by the classes provided by the first layer. The evaluation is done for two data sets acquired with different lidars and possessing different characteristics. It is done quantitatively using one of the data sets and qualitatively using another. The system design follows a standard learning procedure with training, validation, and test steps. The operation follows a standard classification pipeline. The system is simple with no requirement of preprocessing or postprocessing stages.

Late Wisconsinan deglaciation and proglacial lakes development in the Charlevoix region, southeastern Québec, Canada

The Charlevoix region, in southeastern Quebec, is characterized by a dramatic landscape formed by the junction of the Laurentian Highlands, the Charlevoix Astrobleme and the St Lawrence Estuary. At the Last Glacial Maximum (LGM), the region was completely covered by the Laurentide Ice Sheet (LIS). The complex topography of the region was the stage of many of the major deglacial events of southern Quebec (e.g. Goldthwait Sea Invasion, St Lawrence Ice-Stream, Saint-Narcisse Episode). We present a detailed reconstruction of the pattern of retreat of the LIS in the Charlevoix region based on the interpretation of ice-marginal features (e.g. moraines, fans) and glaciolacustrine landforms and deposits, two extensive field campaigns, and the interpretation of high-resolution 3D digital aerial photographs and LiDAR data. Our results indicate five moraine complexes in the region: the Rochette, the Brulee, the Sainte-Anne, the Saint-Narcisse and the Mars-Batiscan complexes. Deltas, fans, fine-grained sediments, littoral deposits, drainage breaches and deposits were used to identify 91 palaeo-proglacial lakes. The identification of these lakes and their relation to moraine complexes enabled the reconstruction of six stages of lake development during the Charlevoix deglaciation. The development of proglacial lakes occurred in all types of terrain (highlands, lowlands, transitory levels above marine limit). We conclude that local topography had a decisive effect on promoting both moraine deposition and lake development. We suggest that similar topographical regions (hilly-mountainous) that were affected by major ice-margin stabilizations during glacial retreat should have experience small lakes dominating valleys and topographical lows.

Global–local articulation pattern-based pedestrian detection using 3D Lidar data

ABSTRACTHighly variable human poses and pedestrian occlusion make light detection and ranging (Lidar)-based pedestrian detection challenging. This letter proposes a novel framework to address these issues. Other than dividing humans into arbitrary number of parts and using the same features for all part detectors, we represent humans with global–local articulated parts and formulate new features relying on each part’s own character. Articulated parts are effective because they each usually maintain a relatively consistent shape across a broader range of body poses. In addition, to extract visible human segments from cluttered surroundings with the presence of pedestrian occlusion, both 3D information and 2D spatial information are used in a coarse-to-fine manner, making the interaction of human part and its neighbouring objects better analysed. The algorithm is evaluated over a busy street dataset and is shown to be competitive with the state-of-the-art Lidar-based algorithms. Remarkably, even at long distances, up to 20 m, it can handle pedestrian occlusion efficiently and effectively.

Palaeogeographical reconstruction of the Sierra de Atapuerca Pleistocene sites (Burgos, Spain)

Abstract The Sierra de Atapuerca is an anticlinal ridge of Mesozoic carbonate rocks on the NW edge of the Iberian Chain (Northern Spain, Burgos), surrounded by subhorizontal continental sediments of the NE Duero Cenozoic Basin under endorheic conditions. The shift to exorheic conditions in the Duero Basin lead to the onset of an episodic downcutting phase and the development of the Atapuerca multilevel cave system, containing several sites from the Early and Middle Pleistocene. In this work, we have reconstructed the Pleistocene palaeogeographical evolution of the SW flank of the Sierra de Atapuerca, where these archaeological sites are located. The study is based on a detailed geomorphological and geological analysis, combined with Global Navigation Satellite System (GNSS) and 3D LiDAR data, and GIS modeling. These techniques have been applied to analyse the small valleys and the interfluve on the SW flank of the SW Sierra de Atapuerca. The results were combined with the regional base levels recorded by fluvial terraces and the chronostratigraphic sequences of the Sierra de Atapuerca sites. These reconstructions have allowed us to model the palaeogeographical evolution in the nearby area of the cave sites during the Early–Middle Pleistocene, coupling the main formation phases of the sites with the local physical landscape changes that occurred outside the caves. Surface processes are defined by incision phases entailing mitigate knickpoint recession and slope retreatment, and local aggradational phases associated with caves opened and captured by fluvial incision. This reconstruction provides the local physical palaeogeographical habitats developed during the Pleistocene hominid occupation of the Sierra de Atapuerca.

Automatic Object Extraction from Electrical Substation Point Clouds

The reliability of power delivery can be profoundly improved by preventing wildlife-related power outages. This can be achieved by insulating electrical substation components with non-conductive covers. The manufacture of custom-built covers requires as-built models of the salient components. This study presents new, automated methodology to recognize key components of electrical substations from 3D LiDAR data acquired using terrestrial laser scanning. The proposed methodology includes six novel algorithms to recognize key components (fence, cables, circuit breakers, bushings and bus pipes) of electrical substations. Three datasets with different resolutions and configurations are used in this study. A Leica HDS 6100 laser scanner was used to acquire the first dataset and a Faro Focus3D laser scanner was employed to collect the second and third datasets. The obtained results indicate that 178 and 171 out of 181 electrical substation elements were successfully recognized in the first and second dataset, respectively, and 183 out of 191 components were identified in the third dataset. The results also demonstrate that an average 97.8% accuracy and average 98.8% precision at the point cloud level can be achieved.

Large-scale mapping in complex field scenarios using an autonomous car

In this paper, we present an end-to-end framework for precise large-scale mapping with applications in autonomous driving. In special, the problem of mapping complex environments, with features changing from tree-lined streets to urban areas with dense traffic, is studied. The robotic car is equipped with an odometry sensor, a 3D LiDAR Velodyne HDL-32E, a IMU, and a low cost GPS, and the data generated by these sensors are integrated in a pose-based GraphSLAM estimator. A new strategy for identification and correction of odometry data using evolutionary algorithms is presented. This new strategy makes odometry data significantly more consistent with GPS. Loop closures are detected using GPS data, and GICP, a 3D point cloud registration algorithm, is used to estimate the displacement between the different travels over the same region. After path estimation, 3D LiDAR data is used to build an occupancy grid mapping of the environment. A detailed mathematical description of how occupancy evidence can be calculated from the point clouds is given, and a submapping strategy to handle memory limitations is presented as well. The proposed framework is tested in three real world environments with different sizes, and features: a parking lot, a university beltway, and a city neighborhood. In all cases, satisfactory maps were built, with precise loop closures even when the vehicle traveled long distances between them.

Automated Recognition of Railroad Infrastructure in Rural Areas from LIDAR Data

This study is aimed at developing automated methods to recognize railroad infrastructure from 3D LIDAR data. Railroad infrastructure includes rail tracks, contact cables, catenary cables, return current cables, masts, and cantilevers. The LIDAR dataset used in this study is acquired by placing an Optech Lynx mobile mapping system on a railcar, operating at 125 km/h. The acquired dataset covers 550 meters of Austrian rural railroad corridor comprising 31 railroad key elements and containing only spatial information. The proposed methodology recognizes key components of the railroad corridor based on their physical shape, geometrical properties, and the topological relationships among them. The developed algorithms managed to recognize all key components of the railroad infrastructure, including two rail tracks, thirteen masts, thirteen cantilevers, one contact cable, one catenary cable, and one return current cable. The results are presented and discussed both at object level and at point cloud level. The results indicate that 100% accuracy and 100% precision at the object level and an average of 96.4% accuracy and an average of 97.1% precision at point cloud level are achieved.

Computing and monitoring potential of public spaces by shading analysis using 3d lidar data and advanced image analysis

Abstract. The paper regards specific context of public spaces in “shadow” of tall buildings located in European cities. Majority of tall buildings in European cities were built in last 15 years. Tall buildings appear mainly in city centres, directly at important public spaces being viable environment for inhabitants with variety of public functions (open spaces, green areas, recreation places, shops, services etc.). All these amenities and services are under direct impact of extensive shading coming from the tall buildings. The paper focuses on analyses and representation of impact of shading from tall buildings on various public spaces in cities using 3D city models. Computer environment of 3D city models in cityGML standard uses 3D LiDAR data as one of data types for definition of 3D cities. The structure of cityGML allows analytic applications using existing computer tools, as well as developing new techniques to estimate extent of shading coming from high-risers, affecting life in public spaces. These measurable shading parameters in specific time are crucial for proper functioning, viability and attractiveness of public spaces – finally it is extremely important for location of tall buildings at main public spaces in cities. The paper explores impact of shading from tall buildings in different spatial contexts on the background of using cityGML models based on core LIDAR data to support controlled urban development in sense of viable public spaces. The article is prepared within research project 2TaLL: Application of 3D Virtual City Models in Urban Analyses of Tall Buildings, realized as a part of Polish-Norway Grants.

Obstacle Detection in Hybrid Cross-Country Environment Based on Markov Random Field for Unmanned Ground Vehicle

In order to detect the obstacle from the large amount of 3D LIDAR data in hybrid cross-country environment for unmanned ground vehicle, a new graph approach based on Markov random field was presented. Firstly, the preprocessing method based on the maximum blurred line is applied to segment the projection of every laser scan line inx-yplane. Then, based onK-means clustering algorithm, the same properties of the line are combined. Secondly, line segment nodes are precisely positioned by using corner detection method, and the next step is to take advantage of line segment nodes to build an undirected graph for Markov random field. Lastly, the energy function is calculated by means of analyzing line segment features and solved by graph cut. Two types of line mark are finally classified into two categories: ground and obstacle. Experiments prove the feasibility of the approach and show that it has better performance and runs in real time.

Semantic 3D scene interpretation: A framework combining optimal neighborhood size selection with relevant features

Abstract. 3D scene analysis by automatically assigning 3D points a semantic label has become an issue of major interest in recent years. Whereas the tasks of feature extraction and classification have been in the focus of research, the idea of using only relevant and more distinctive features extracted from optimal 3D neighborhoods has only rarely been addressed in 3D lidar data processing. In this paper, we focus on the interleaved issue of extracting relevant, but not redundant features and increasing their distinctiveness by considering the respective optimal 3D neighborhood of each individual 3D point. We present a new, fully automatic and versatile framework consisting of four successive steps: (i) optimal neighborhood size selection, (ii) feature extraction, (iii) feature selection, and (iv) classification. In a detailed evaluation which involves 5 different neighborhood definitions, 21 features, 6 approaches for feature subset selection and 2 different classifiers, we demonstrate that optimal neighborhoods for individual 3D points significantly improve the results of scene interpretation and that the selection of adequate feature subsets may even further increase the quality of the derived results.

Robust curb detection with fusion of 3D-Lidar and camera data.

Curb detection is an essential component of Autonomous Land Vehicles (ALV), especially important for safe driving in urban environments. In this paper, we propose a fusion-based curb detection method through exploiting 3D-Lidar and camera data. More specifically, we first fuse the sparse 3D-Lidar points and high-resolution camera images together to recover a dense depth image of the captured scene. Based on the recovered dense depth image, we propose a filter-based method to estimate the normal direction within the image. Then, by using the multi-scale normal patterns based on the curb's geometric property, curb point features fitting the patterns are detected in the normal image row by row. After that, we construct a Markov Chain to model the consistency of curb points which utilizes the continuous property of the curb, and thus the optimal curb path which links the curb points together can be efficiently estimated by dynamic programming. Finally, we perform post-processing operations to filter the outliers, parameterize the curbs and give the confidence scores on the detected curbs. Extensive evaluations clearly show that our proposed method can detect curbs with strong robustness at real-time speed for both static and dynamic scenes.

Mapping shotcrete thickness using LiDAR and photogrammetry data: Correcting for over-calculation due to rockmass convergence

The benchmark method of measuring shotcrete thickness from 3D LiDAR and photogrammetry data involves scanning sequential blast rounds, aligning the data in a 3D environment, and calculating the spatial difference between the two models. The calculated difference between the two 3D surface models is measured as the thickness of the sprayed concrete. This methodology does not account for the convergence of the rockmass that naturally occurs between the scanning protocols, nor is it included in the difference equation, and resultantly the user over-measures the thickness of the sprayed concrete. The over-measurement can be corrected through changing the time of scanning with respect to the excavation sequence or calibrating the solution based on known rockmass convergence rates or numerical modelling. The use of 3D imaging data for the calculation of shotcrete thickness will remain a useful tool for geotechnical engineers, but corrections must made to the state-of-practice methodology in order to achieve accurate results.