Lidar Scene Research Articles

Diffusion Models-Based Purification for Common Corruptions on Robust 3D Object Detection.

LiDAR sensors have been shown to generate data with various common corruptions, which seriously affect their applications in 3D vision tasks, particularly object detection. At the same time, it has been demonstrated that traditional defense strategies, including adversarial training, are prone to suffering from gradient confusion during training. Moreover, they can only improve their robustness against specific types of data corruption. In this work, we propose LiDARPure, which leverages the powerful generation ability of diffusion models to purify corruption in the LiDAR scene data. By dividing the entire scene into voxels to facilitate the processes of diffusion and reverse diffusion, LiDARPure overcomes challenges induced from adversarial training, such as sparse point clouds in large-scale LiDAR data and gradient confusion. In addition, we utilize the latent geometric features of a scene as a condition to assist the generation of diffusion models. Detailed experiments show that LiDARPure can effectively purify 19 common types of LiDAR data corruption. Further evaluation results demonstrate that it can improve the average precision of 3D object detectors to an extent of 20% in the face of data corruption, much higher than existing defence strategies.

Effective Training and Inference Strategies for Point Classification in LiDAR Scenes

Light Detection and Ranging systems serve as robust tools for creating three-dimensional representations of the Earth’s surface. These representations are known as point clouds. Point cloud scene segmentation is essential in a range of applications aimed at understanding the environment, such as infrastructure planning and monitoring. However, automating this process can result in notable challenges due to variable point density across scenes, ambiguous object shapes, and substantial class imbalances. Consequently, manual intervention remains prevalent in point classification, allowing researchers to address these complexities. In this work, we study the elements contributing to the automatic semantic segmentation process with deep learning, conducting empirical evaluations on a self-captured dataset by a hybrid airborne laser scanning sensor combined with two nadir cameras in RGB and near-infrared over a 247 km2 terrain characterized by hilly topography, urban areas, and dense forest cover. Our findings emphasize the importance of employing appropriate training and inference strategies to achieve accurate classification of data points across all categories. The proposed methodology not only facilitates the segmentation of varying size point clouds but also yields a significant performance improvement compared to preceding methodologies, achieving a mIoU of 94.24% on our self-captured dataset.

MOVES: Movable and moving LiDAR scene segmentation in label-free settings using static reconstruction

Accurate static structure reconstruction and segmentation of non-stationary objects is of vital importance for autonomous navigation applications. These applications assume a LiDAR scan to consist of only static structures. In the real world however, LiDAR scans consist of non-stationary dynamic structures — moving and movable objects. Current solutions use segmentation information to isolate and remove moving structures from LiDAR scan. This strategy fails in several important use-cases where segmentation information is not available. In such scenarios, moving objects and objects with high uncertainty in their motion i.e. movable objects, may escape detection. This violates the above assumption. We present MOVES, a novel GAN based adversarial model that segments out moving as well as movable objects in the absence of segmentation information. We achieve this by accurately transforming a dynamic LiDAR scan to its corresponding static scan. This is obtained by replacing dynamic objects and corresponding occlusions with static structures which were occluded by dynamic objects. We leverage corresponding static-dynamic LiDAR pairs. We design a novel discriminator, coupled with a contrastive loss on a smartly selected LiDAR scan triplet. For datasets lacking paired information, we propose MOVES-MMD that integrates Unsupervised Domain Adaptation into the network. We perform rigorous experiments to demonstrate state of the art dynamic to static translation performance on a sparse real world industrial dataset, an urban and a simulated dataset. MOVES also segments out movable and moving objects without using segmentation information. Without utilizing segmentation labels, MOVES performs better than segmentation based navigation baseline in highly dynamic and long LiDAR sequences. The code is available here.

IDet3D: Towards Efficient Interactive Object Detection for LiDAR Point Clouds

Accurately annotating multiple 3D objects in LiDAR scenes is laborious and challenging. While a few previous studies have attempted to leverage semi-automatic methods for cost-effective bounding box annotation, such methods have limitations in efficiently handling numerous multi-class objects. To effectively accelerate 3D annotation pipelines, we propose iDet3D, an efficient interactive 3D object detector. Supporting a user-friendly 2D interface, which can ease the cognitive burden of exploring 3D space to provide click interactions, iDet3D enables users to annotate the entire objects in each scene with minimal interactions. Taking the sparse nature of 3D point clouds into account, we design a negative click simulation (NCS) to improve accuracy by reducing false-positive predictions. In addition, iDet3D incorporates two click propagation techniques to take full advantage of user interactions: (1) dense click guidance (DCG) for keeping user-provided information throughout the network and (2) spatial click propagation (SCP) for detecting other instances of the same class based on the user-specified objects. Through our extensive experiments, we present that our method can construct precise annotations in a few clicks, which shows the practicality as an efficient annotation tool for 3D object detection.

Adaptive LiDAR Reconstruction by Convolutional Compressive Sensing Network and Multivariate Empirical Mode Decomposition

With the recent advancement of LiDAR data processing pipelines integrated with signal processing techniques, it is now possible to rapidly acquire, process, and infer multidimensional information of 3D surroundings. This work proposes a novel compressive sensing (CS) framework (named MEMD-CSNet), based on Multivariate Empirical Mode Decomposition (MEMD) for adaptive 3D reconstruction of airborne urban LiDAR point clouds. EMD is a data-driven transform for decomposing the input data into Intrinsic Mode Functions (IMFs), capturing the implicit features in terms of variation in spatial frequency. Our approach generates a new filtered 3D signal constituting an optimal subset of IMFs based on the understanding of their statistical significance. Further, this filtered signal hierarchically extracts the overall shape in the last IMF (Residual function). Our proposed MEMD-CSNet is a deep convolutional network that (1) computes a multivariate EMD on input raw 3D LiDAR point clouds for generating data-dependent feature-rich filtered signal and (2) additionally augments this signal with convolutional features, thus learning a data-centric representation function (Ψ) for transforming the input into a low-dimensional sparse basis. We present extensive qualitative and quantitative analysis using the proposed MEMD-CSNet for a set of chosen CS measurement ratios. MEMD-CSNet achieves a maximum reconstruction PSNR of 108.22 dB (approx. 30 dB/1.36x better than without using MEMD) for reconstruction fidelity on selected urban LiDAR scenes. We envisage that our proposed MEMD-CSNet has a high potential (as a data-centric framework) for generating data-driven sparse feature maps and reconstructing entire 3D LiDAR point clouds from a few sparse measurements.

Rapid Localization and Mapping Method Based on Adaptive Particle Filters.

With the development of autonomous vehicles, localization and mapping technologies have become crucial to equip the vehicle with the appropriate knowledge for its operation. In this paper, we extend our previous work by prepossessing a localization and mapping architecture for autonomous vehicles that do not rely on GPS, particularly in environments such as tunnels, under bridges, urban canyons, and dense tree canopies. The proposed approach is of two parts. Firstly, a K-means algorithm is employed to extract features from LiDAR scenes to create a local map of each scan. Then, we concatenate the local maps to create a global map of the environment and facilitate data association between frames. Secondly, the main localization task is performed by an adaptive particle filter that works in four steps: (a) generation of particles around an initial state (provided by the GPS); (b) updating the particle positions by providing the motion (translation and rotation) of the vehicle using an inertial measurement device; (c) selection of the best candidate particles by observing at each timestamp the match rate (also called particle weight) of the local map (with the real-time distances to the objects) and the distances of the particles to the corresponding chunks of the global map; (d) averaging the selected particles to derive the estimated position, and, finally, using a resampling method on the particles to ensure the reliability of the position estimation. The performance of the newly proposed technique is investigated on different sequences of the Kitti and Pandaset raw data with different environmental setups, weather conditions, and seasonal changes. The obtained results validate the performance of the proposed approach in terms of speed and representativeness of the feature extraction for real-time localization in comparison with other state-of-the-art methods.

Reconfigurable optical time delay array for 3D lidar scene projector.

3D lidar scene projector (LSP) plays an important role in the hardware-in-the-loop (HIL) simulation for autonomous driving system (ADS). It generates a simulated 3D lidar scene in laboratory by generating a 2D array of optical time delay signals. The reconfigurable optical time delay array (ROTDA) is crucial for LSP. However, current ROTDA solutions cannot support a LSP with a spatial resolution more than 10×10. In this paper, we proposed a novel ROTDA design based on the time slicing method. The optical signals with the same time delay but different spatial coordinates were treated as one time slice. Different time slices were superimposed into a composite image by a microlens-array-based imaging system to obtain a 3D lidar scene. And a spatial light modulator (SLM) was utilized to configure the time delay of each lidar scene pixel. We developed a ROTDA prototype with 64×64 pixels, each pixel can be reconfigured with up to 180 different time delays in one frame. The time delay resolution is 1 ns, the maximum time delay is 5000 s, and the 3D frame rate is 20Hz. The prototype can generate a continuous lidar scene with a distance span of 27 m, and can also generate up to 8 short scenes that are separated from each other along the lidar observation direction, each short scene covers a distance span of 3 m or 3.75 m. The design method proposed in this paper can also be applied to other occasions that demand a large number of time delay generators.

Range-Image: Incorporating Sensor Topology for Lidar Point Cloud Processing

This paper proposes a novel methodology for LiDAR point cloud processing that takes advantage of the implicit topology of various LiDAR sensors to derive 2D images from the point cloud while bringing spatial structure to each point. The interest of such a methodology is then proved by addressing the problems of segmentation and disocclusion of mobile objects in 3D LiDAR scenes acquired via street-based Mobile Mapping Systems (MMS). Most of the existing lines of research tackle those problems directly in the 3D space. This work promotes an alternative approach by using this image representation of the 3D point cloud, taking advantage of the fact that the problem of disocclusion has been intensively studied in the 2D image processing community over the past decade. Using the image derived from the sensor data by exploiting the sensor topology, a semi-automatic segmentation procedure based on depth histograms is presented. Then, a variational image inpainting technique is introduced to reconstruct the areas that are occluded by objects. Experiments and validation on real data prove the effectiveness of this methodology both in terms of accuracy and speed.

LOCALIZED SEGMENT BASED PROCESSING FOR AUTOMATIC BUILDING EXTRACTION FROM LiDAR DATA

Abstract. The current methods of object segmentation and extraction and classification of aerial LiDAR data is manual and tedious task. This work proposes a technique for object segmentation out of LiDAR data. A bottom-up geometric rule based approach was used initially to devise a way to segment buildings out of the LiDAR datasets. For curved wall surfaces, comparison of localized surface normals was done to segment buildings. The algorithm has been applied to both synthetic datasets as well as real world dataset of Vaihingen, Germany. Preliminary results show successful segmentation of the buildings objects from a given scene in case of synthetic datasets and promissory results in case of real world data. The advantages of the proposed work is non-dependence on any other form of data required except LiDAR. It is an unsupervised method of building segmentation, thus requires no model training as seen in supervised techniques. It focuses on extracting the walls of the buildings to construct the footprint, rather than focussing on roof. The focus on extracting the wall to reconstruct the buildings from a LiDAR scene is crux of the method proposed. The current segmentation approach can be used to get 2D footprints of the buildings, with further scope to generate 3D models. Thus, the proposed method can be used as a tool to get footprints of buildings in urban landscapes, helping in urban planning and the smart cities endeavour.

DISOCCLUSION OF 3D LIDAR POINT CLOUDS USING RANGE IMAGES

Abstract. This paper proposes a novel framework for the disocclusion of mobile objects in 3D LiDAR scenes aquired via street-based Mobile Mapping Systems (MMS). Most of the existing lines of research tackle this problem directly in the 3D space. This work promotes an alternative approach by using a 2D range image representation of the 3D point cloud, taking advantage of the fact that the problem of disocclusion has been intensively studied in the 2D image processing community over the past decade. First, the point cloud is turned into a 2D range image by exploiting the sensor’s topology. Using the range image, a semi-automatic segmentation procedure based on depth histograms is performed in order to select the occluding object to be removed. A variational image inpainting technique is then used to reconstruct the area occluded by that object. Finally, the range image is unprojected as a 3D point cloud. Experiments on real data prove the effectiveness of this procedure both in terms of accuracy and speed.

Optimizing embedded sensor network design for catchment-scale snow-depth estimation using LiDAR and machine learning

PUBLICATIONS Water Resources Research RESEARCH ARTICLE 10.1002/2016WR018896 Key Points: A machine-learning algorithm for optimizing snow sensor placements is compared to expert placements in an existing sensor network The spatial and temporal transferability of the algorithm is then assessed in 14 total LiDAR surveys in two uninstrumented catchments The accuracy of the snow depth estimated from the sensor measurements is higher than expert and randomized placements in each of the surveys Correspondence to: C. A. Oroza, coroza@berkeley.edu Citation: Oroza, C. A., Z. Zheng, S. D. Glaser, and D. Tuia (2016), Optimizing embedded sensor network design for catchment- scale snow-depth estimation using LiDAR and machine learning, Water Resour. Res., 52, 8174–8189, doi:10.1002/2016WR018896. Received 11 MAR 2016 Accepted 23 SEP 2016 Accepted article online 27 SEP 2016 Published online 22 OCT 2016 Optimizing embedded sensor network design for catchment-scale snow-depth estimation using LiDAR and machine learning Carlos A. Oroza 1 , Zeshi Zheng 1 , Steven D. Glaser 1 , Devis Tuia 2 , and Roger C. Bales 1,3 Department of Civil and Environmental Engineering, University of California, Berkeley, California, USA, 2 Department of Geography, University of Zurich, Zurich, Switzerland, 3 Sierra Nevada Research Institute, University of California, Merced, California, USA Abstract We evaluate the accuracy of a machine-learning algorithm that uses LiDAR data to optimize ground-based sensor placements for catchment-scale snow measurements. Sampling locations that best represent catchment physiographic variables are identified with the Expectation Maximization algorithm for a Gaussian mixture model. A Gaussian process is then used to model the snow depth in a 1 km 2 area sur- rounding the network, and additional sensors are placed to minimize the model uncertainty. The aim of the study is to determine the distribution of sensors that minimizes the bias and RMSE of the model. We com- pare the accuracy of the snow-depth model using the proposed placements to an existing sensor network at the Southern Sierra Critical Zone Observatory. Each model is validated with a 1 m 2 LiDAR-derived snow-depth raster from 14 March 2010. The proposed algorithm exhibits higher accuracy with fewer sensors (8 sensors, RMSE 38.3 cm, bias 5 3.49 cm) than the existing network (23 sensors, RMSE 53.0 cm, bias 5 15.5 cm) and randomized placements (8 sensors, RMSE 63.7 cm, bias 5 24.7 cm). We then evaluate the spatial and temporal transferability of the method using 14 LiDAR scenes from two catchments within the JPL Airborne Snow Observatory. In each region, the optimized sensor placements are determined using the first available snow raster for the year. The accuracy in the remaining LiDAR surveys is then com- pared to 100 configurations of sensors selected at random. We find the error statistics (bias and RMSE) to be more consistent across the additional surveys than the average random configuration. 1. Introduction A major challenge of hydrologic science in montane regions relates to estimating the spatial variability of snow cover [Bales et al., 2006; Trujillo and Lehning, 2015; McCreight et al., 2014; Guan et al., 2013]. Multiple independent variables govern the distribution of snow cover, including elevation, slope, aspect, and the dis- tribution of canopy [Faria et al., 2000; Musselman et al., 2008; Lehning et al., 2011; Helfricht et al., 2014]. Non- stationary effects such as climate warming and changes in vegetation structure may significantly alter the timing and magnitude of storage and runoff in these watersheds [Goulden and Bales, 2014; Flanner et al., 2009]. Existing regression-based hydrologic models, which use statistical relations from historical hydro- graphs to predict runoff and inform allocation decisions [Perkins et al., 2009; Rosenberg et al., 2011; Rango and Martinec, 1995] will have limited accuracy as conditions deviate from historical norms and thus may prove to be inadequate for predictions in water management. C 2016. American Geophysical Union. V All Rights Reserved. OROZA ET AL. Recent research has focused on improving hydrologic models by assimilating remote-sensing and in situ measurements with distributed energy-balance models to better estimate storage and runoff [Guan et al., 2013]. These methods use well-developed remote-sensing [Painter et al., 2003; Rosenthal and Dozier, 1996; € lli et al., 2002; Egli et al., 2012] and energy-balance models [Marks et al., 1992; Link and Marks, Dozier, 1989; P a 1999; Brubaker et al., 1996] to estimate snow and snowmelt processes across basins. In situ measurements for these methods are presently limited to snow pillows and snow courses, which largely sample flat, open terrain [Molotch and Bales, 2006], yet the distribution of snow cover can vary considerably as a function of topographic features. To address this, in situ sensor measurements can be deployed to capture the mean and variance of the snow depth, which can be used to inform models that use these statistics as inputs [e.g., Essery and Pomeroy, 2004]. Alternatively, individual sensor measurements can be used together with OPTIMIZING SNOW SENSOR PLACEMENTS

Coseismic fault zone deformation revealed with differential lidar: Examples from Japanese [formula omitted] ∼7 intraplate earthquakes

We use two recent Japanese earthquakes to demonstrate the rich potential, as well as some of the challenges, of differencing repeat airborne Light Detection and Ranging (lidar) topographic data to measure coseismic fault zone deformation. We focus on densely-vegetated sections of the 14 June 2008 Iwate–Miyagi (Mw 6.9) and 11 April 2011 Fukushima–Hamadori (Mw 7.1) earthquake ruptures, each covered by 2 m-resolution pre-event and 1 m-resolution post-event bare Earth digital terrain models (DTMs) obtained from commercial lidar providers. Three-dimensional displacements and rotations were extracted from these datasets using an adaptation of the Iterative Closest Point (ICP) algorithm. These displacements remain coherent close to surface fault breaks, as well as within dense forest, despite intervals of ∼2 years (Iwate–Miyagi) and ∼4 years (Fukushima–Hamadori) encompassed by the lidar scenes. Differential lidar analysis is thus complementary to Interferometric Synthetic Aperture Radar (InSAR) and sub-pixel correlation techniques which often break down under conditions of long time intervals, dense vegetation or steep displacement gradients. Although the ICP displacements are much noisier than overlapping InSAR line-of-sight displacements, they still provide powerful constraints on near-surface fault slip. In the Fukushima–Hamadori case, near-fault displacements and rotations are consistent with decreased primary fault slip at very shallow depths of a few tens of meters, helping to account for the large, along-strike heterogeneity in surface offsets observed in the field. This displacement field also captures long-wavelength deformation resulting from the 11 March 2011 Tohoku great earthquake.

FEATURES AND GROUND AUTOMATIC EXTRACTION FROM AIRBORNE LIDAR DATA

Abstract. The aim of the research has been the developing and implementing an algorithm for automated extraction of features from LIDAR scenes with varying terrain and coverage types. This applies the moment of third order (Skweness) and fourth order (Kurtosis). While the first has been applied in order to produce an initial filtering and data classification, the second, through the introduction of the weights of the measures, provided the desired results, which is a finer classification and less noisy. The process has been carried out in Matlab but to reduce processing time, given the large data density, the analysis has been limited at a mobile window. It was, therefore, arranged to produce subscenes in order to covers the entire area. The performance of the algorithm, confirm its robustness and goodness of results. Employment of effective processing strategies to improve the automation is a key to the implementation of this algorithm. The results of this work will serve the increased demand of automation for 3D information extraction using remotely sensed large datasets. After obtaining the geometric features from LiDAR data, we want to complete the research creating an algorithm to vector features and extraction of the DTM.

An airborne lidar sampling strategy to model forest canopy height from Quickbird imagery and GEOBIA

High-resolution digital canopy models derived from airborne lidar data have the ability to provide detailed information on the vertical structure of forests. However, compared to satellite data of similar spatial resolution and extent, the small footprint airborne lidar data required to produce such models remain expensive. In an effort to reduce these costs, the primary objective of this paper is to develop an airborne lidar sampling strategy to model full-scene forest canopy height from optical imagery, lidar transects and Geographic Object-Based Image Analysis (GEOBIA). To achieve this goal, this research focuses on (i) determining appropriate lidar transect features (i.e., location, direction and extent) from an optical scene, (ii) developing a mechanism to model forest canopy height for the full-scene based on a minimum number of lidar transects, and (iii) defining an optimal mean object size (MOS) to accurately model the canopy composition and height distribution. Results show that (i) the transect locations derived from our optimal lidar transect selection algorithm accurately capture the canopy height variability of the entire study area; (ii) our canopy height estimation models have similar performance in two lidar transect directions (i.e., north–south and west–east); (iii) a small lidar extent (17.6% of total size) can achieve similar canopy height estimation accuracies as those modeled from the full lidar scene; and (iv) different MOS can lead to distinctly different canopy height results. By comparing the best canopy height estimate with the full lidar canopy height data, we obtained average estimation errors of 6.0 m and 6.8 m for conifer and deciduous forests at the individual tree crown/small tree cluster level, and an area weighted combined error of 6.2 m, which is lower than the provincial forest inventory height class interval (i.e., ≈ 9.0 m).