3D LiDAR Sensor Research Articles

An efficient LiDAR-based localization method for self-driving cars in dynamic environments

AbstractReal-time localization is an important mission for self-driving cars and it is difficult to achieve precise pose information in dynamic environments. In this paper, a novel localization method is proposed to estimate the pose of self-driving cars using a 3D-LiDAR sensor. First, the multi-frame curb features and laser intensity features are extracted. Meanwhile, based on the high-precision curb map generated offline, obstacles on road are detected using region segmentation methods and their features are removed. Furthermore, a map-matching method is proposed to match the features to the map, a robust iterative closest point algorithm is utilized to deal with curb features along with a probability search method dealing with intensity features. Finally, two separate Kalman filters are used to fuse the low-cost global positioning systems and map-matching results. Both offline and online experiments are carried out in dynamic environments and the results demonstrate the accuracy and robustness of the proposed method.

R-LOAM: Improving LiDAR Odometry and Mapping With Point-to-Mesh Features of a Known 3D Reference Object

LiDAR-based odometry and mapping is used in many robotic applications to retrieve the robot's position in an unknown environment and allows for autonomous operation in GPS-denied (e.g., indoor) environments. With a 3D LiDAR sensor, highly accurate localization becomes possible, which enables high quality 3D reconstruction of the environment. In this letter we extend the well-known LOAM framework by leveraging prior knowledge about a reference object in the environment to further improve the localization accuracy. This requires a known 3D model of the reference object and its known position in a global coordinate frame. Instead of only relying on the point features in the mapping module of LOAM, we also include mesh features extracted from the 3D triangular mesh of the reference object in the optimization problem. For fast correspondence computation of mesh features, we use the Axis-Aligned-Bounding-Box-Tree (AABB) structure. Essentially, our approach not only makes use of the previously built map for absolute localization in the environment, but also takes the relative position to the reference object into account, effectively reducing long-term drift. To validate the proposed concept, we generated datasets using the Gazebo simulation environment in exemplary visual inspection scenarios of an airplane inside a hangar and the Eiffel Tower. An actuated 3D LiDAR sensor is mounted via a 1-DoF gimbal on a UAV capturing 360° scans. We benchmark our approach against the state-of-the-art open-source LOAM framework. The results show that the proposed joint optimization using both point and mesh features yields a significant reduction in Absolute Pose Error (APE) and therefore improves the map and 3D reconstruction quality during long-term operations.

3D laser scanning and surveying adjustment in traffic infrastructure management

In view of difficulties in automatic and refined management of traffic infrastructure, road lamps and signboard in the test road section are taken as examples, an application framework for automatic and refined management of traffic infrastructure is proposed by using 3D laser scanning technology combined with GPS (Global Positioning System) and QGIS (Quantum Geographic Information System). The advantages of 3D lidar sensor, such as high precision, strong anti-interference ability and good real-time performance are fully used. Through data fitting by using total least squares method, digital management of infrastructure is carried out and its precision is high. Combined with GPS and QGIS, the spatial information of infrastructure is developed. In this way, manpower and efficiency of management of infrastructure are saved and improved.

3D Object Detection Using Frustums and Attention Modules for Images and Point Clouds

Three-dimensional (3D) object detection is essential in autonomous driving. Three-dimensional (3D) Lidar sensor can capture three-dimensional objects, such as vehicles, cycles, pedestrians, and other objects on the road. Although Lidar can generate point clouds in 3D space, it still lacks the fine resolution of 2D information. Therefore, Lidar and camera fusion has gradually become a practical method for 3D object detection. Previous strategies focused on the extraction of voxel points and the fusion of feature maps. However, the biggest challenge is in extracting enough edge information to detect small objects. To solve this problem, we found that attention modules are beneficial in detecting small objects. In this work, we developed Frustum ConvNet and attention modules for the fusion of images from a camera and point clouds from a Lidar. Multilayer Perceptron (MLP) and tanh activation functions were used in the attention modules. Furthermore, the attention modules were designed on PointNet to perform multilayer edge detection for 3D object detection. Compared with a previous well-known method, Frustum ConvNet, our method achieved competitive results, with an improvement of 0.27%, 0.43%, and 0.36% in Average Precision (AP) for 3D object detection in easy, moderate, and hard cases, respectively, and an improvement of 0.21%, 0.27%, and 0.01% in AP for Bird’s Eye View (BEV) object detection in easy, moderate, and hard cases, respectively, on the KITTI detection benchmarks. Our method also obtained the best results in four cases in AP on the indoor SUN-RGBD dataset for 3D object detection.

Motion Analysis and Performance Improved Method for 3D LiDAR Sensor Data Compression

Continuous point cloud data is being used more and more widely in practical applications such as mapping, localization and object detection in autonomous driving systems, but due to the huge volume of data involved, sharing and storing this data is currently expensive and difficult. One possible solution is the development of more efficient methods of compressing the data. Other researchers have proposed converting 3D point cloud data into 2D images, or using tree structures to store the data. In a previous study targeting streaming point cloud data, we proposed an MPEG-like compression method which utilizes simultaneous localization and mapping (SLAM) results to simulate LiDAR’s operating process. In this paper, instead of imitating MPEG, we propose new strategy for more efficient reference frame distribution and more natural frame prediction, and use a different algorithm to encode the residual, greatly improving the algorithm’s performance and its stability in different scenarios. We also discuss how various parameters affect compression performance. Using our proposed method, streaming point cloud data collected by LiDAR sensors can be compressed to 1/50th of its original size, with only 2 cm of Root Mean Square Error for each detected point. We evaluate our proposed method by comparing its performance with several other existing point cloud compression methods in three different driving scenarios, demonstrating that our proposed method outperforms them.

Ground and Aerial Collaborative Mapping in Urban Environments

A heterogeneous multi-robot system consisting of Unmanned Ground Vehicles (UGVs) and Unmanned Aerial Vehicles (UAVs) have advantages over a single-robot system in efficiency and flexibility, enabling them to perform a larger range of tasks. To allow heterogeneous platforms to work together in GPS-denied scenarios, it is crucial to build a complete 3D map of the environment. In this letter, a novel method is presented to perform ground and aerial collaborative mapping leveraging visual and range data collected by cameras and 3D LiDAR sensors. In the proposed system, a visual-LiDAR ego-motion estimation module that considers point, line and planar constraints can provide robust odometry information. Thumbnail images representing obstacle outlines are generated and descriptors are extracted using a neural network to help perform data association between separate runs. Map segments and the robot poses are organized together and are updated during a pose graph optimization procedure. The proposed ground-aerial collaborative mapping approach is evaluated on both synthetic and real-world datasets comparing with other methods. Experiment results demonstrate that our method can achieve outstanding mapping results.

A Deep-Learning-based Strategy for Kidnapped Robot Problem in Similar Indoor Environment

We present a deep-learning-based strategy that only uses a 2D LiDAR sensor to solve the kidnapped robot problem in similar indoor environments. First, we converted a set of 2D laser data into an RGB-image and an occupancy grid map and stacked them into a multi-channel image. Then, a neural network structure with five convolutional layers and four fully connected layers was designed to regress the 3-DOF robot pose. Finally, the network was trained using multi-channel images as input. We also improved the network structure to identify the scene where the robot is localized. Extensive experiments have been conducted in practice with a real mobile robot, verifying the effectiveness of the proposed strategy. Our network can obtain approximately 2m and 5∘ accuracy indoors, and the scene classification accuracy of our network reaches up to 98%.

Procuring cooperative intelligence in autonomous vehicles for object detection through data fusion approach

In an autonomous vehicle (AV), in order to efficiently exploit the acquired resources, big data analyses will be a reliable source for extracting valuable information from various sensors and actuators. The data extracted with the combined ability of telematics and real-time investigation forms the vibrant asset for self-driving cars. To demonstrate the significances of big data analysis, this study proposes a competent architecture for real-time big data analysis for an AV, which indeed keeps pace with the latest trends and advancement concerning an emerging paradigm. There are a massive amount of sensors and independent systems needed to be realised for better competence in an AV, and the proposed model focuses on independent sensors that distinguish objects and handles visual information to decide the path. In order to attain the objective as mentioned above, a sensor fusion mechanism is proposed, which combines 3D camera sensor data and Lidar sensor information to provide an optimised solution for path selection. Furthermore, three algorithms, namely overlapping algorithm, sequential adding algorithm, the distance-focused algorithm is designed for higher efficiency in sensor fusion mechanism. The proposed methodology is for the best exploitation of the enormous dataset, meant for real-time processing for an AV.

Single 2D lidar based follow-me of mobile robot on hilly terrains

It would be a very challenging task for mobile robot to track a person walking ahead on hilly terrains with a 2D lidar sensor instead of a 3D lidar or a rotating 2D lidar sensor. But it is clear that the 2D lidar based method is much more efficient in terms of computational costs and sensor cost only. For successful leader-following of the mobile robot with the 2D lidar sensor on such hilly roads, it is necessary to develop a classifier capable of recognizing a specific body part of the leader differently from other parts. In this study, the Mahalanobis distance kernel based on two features extracted from the leader’s torso profiles is used to construct the elliptical decision boundaries for the classification problem. In addition, based on that classifier, the roll and pitch angle of the 2D lidar sensor are controlled to continuously track the leader’s body part and to estimate the relative distance between the leader and the robot. As a result, even when the leader walks along paths with flat-to-hill transitions in day and night, the mobile robot successfully follows the leader with 80 % detection rate by stable pitching not exceed 1° of the 2D lidar sensor for keeping the torso inside its field of view. The proposed approach will be useful to improve the performances of multi-sensor based leader detection system using not only lidar sensor but also vision sensor with switching control scheme.

MOVING OBJECTS AWARE SENSOR MESH FUSION FOR INDOOR RECONSTRUCTION FROM A COUPLE OF 2D LIDAR SCANS

Abstract. Indoor mapping attracts more attention with the development of 2D and 3D camera and Lidar sensor. Lidar systems can provide a very high resolution and accurate point cloud. When aiming to reconstruct the static part of the scene, moving objects should be detected and removed which can prove challenging. This paper proposes a generic method to merge meshes produced from Lidar data that allows to tackle the issues of moving objects removal and static scene reconstruction at once. The method is adapted to a platform collecting point cloud from two Lidar sensors with different scan direction, which will result in different quality. Firstly, a mesh is efficiently produced from each sensor by exploiting its natural topology. Secondly, a visibility analysis is performed to handle occlusions (due to varying viewpoints) and remove moving objects. Then, a boolean optimization allows to select which triangles should be removed from each mesh. Finally, a stitching method is used to connect the selected mesh pieces. Our method is demonstrated on a Navvis M3 (2D laser ranger system) dataset and compared with Poisson and Delaunay based reconstruction methods.

Evaluation of 3D LiDAR Sensor Setup for Heterogeneous Robot Team

We encounter a world where the role of robots has increased in society. Day by day, we can observe that problems related to autonomous vehicles are being solved. The focused problem that we have recently confronted is based on collaborative mapping with multi-robots since collaboratively generated map provides more information about the environment than with a single robot. Moreover, the mapping framework for robot team can be more efficient if 3D laser sensors are placed to cover wider area. However, laser sensors are commonly positioned intuitively and depended mostly on the designer’s choice. To overcome designer-independent laser sensor setup for efficient collaborative mapping, a subjective evaluation of laser sensor placement is developed as another key aspect. This paper proposed a collaborative mapping framework for heterogeneous robot teams to make full use of collected information from different viewpoints and explore large areas more efficiently. It’s comprised of two modules, localization of multi-robots and collaborative map construction. In addition, to avoid arbitrary setup of laser sensors in existing methods whose performance is heavily affected by designers’ experience, a metric evaluation system was defined to determine the optimal placements of laser sensors and achieve maximum sweep area. The proposed methods were verified on both simulation and experiments which were carried out on a heterogeneous robot team (including an unmanned aerial vehicle and a ground vehicle). According to the metric evaluation, best placements were chosen for the laser sensors, and presented collaborative mapping technique was shown to work more efficiently compared with existing methods.

SYSTEMATIC EVALUATION AND CHARACTERIZATION OF 3D SOLID STATE LIDAR SENSORS FOR AUTONOMOUS GROUND VEHICLES

Abstract. 3D LiDAR sensors play an important part in several autonomous navigation and perception systems with the technology evolving rapidly over time. This work presents the preliminary evaluation results of a 3D solid state LiDAR sensor. Different aspects of this new type of sensor are studied and their data are characterized for their effective utilization for object detection for the application of Autonomous Ground Vehicles (AGV). The paper provides a set of evaluations to analyze the characterizations and performances of such LiDAR sensors. After characterization of the sensor, the performance is also evaluated in real environment with the sensors mounted on top of a vehicle and used to detect and classify different objects using a state-of-the-art Super-Voxel based method. The 3D point cloud obtained from the sensor is classified into three main object classes “Building”, “Ground” and “Obstacles”. The results evaluated on real data, clearly demonstrate the applicability and suitability of the sensor for such type of applications.

Extrinsic Calibration of an Eye-In-Hand 2D LiDAR Sensor in Unstructured Environments Using ICP

We propose a calibration method for the six degrees of freedom (DOF) extrinsic pose of a 2D laser rangefinder mounted to a robot arm. Our goal is to design a system that allows on-site re-calibration without requiring any kind of special environment or calibration objects. By moving the sensor we generate 3D scans of the surrounding area on which we run a iterative closest point (ICP) variant to estimate the missing part of the kinematic chain. With this setup we can simply scale the density and format of our 3D scan by adjusting the robot speed and trajectory, allowing us to exploit the power of a high resolution 3D scanner for a variety of tasks such as mapping, object recognition and grasp planning. Our evaluation, performed on synthetic datasets as well as from real-data shows that the presented approach provides good results both in terms of convergence on crude initial parameters as well as in the precision of the final estimate.

Crash course learning: an automated approach to simulation-driven LiDAR-basedtraining of neural networks for obstacle avoidance in mobile robotics

This paper proposes and implements a self-supervised simulation-driven approach to data collection used for training of perception-based shallow neural networks for mobile robot obstacle avoidance. In the approach, a 2D LiDAR sensor was used as an information source for training neural networks. The paper analyzes neural network performance in terms of numbers of layers and neurons, as well as the amount of data needed for reliable robot operation. Once the best architecture is identified, it is trained using only data obtained in simulation and then implemented and tested on a real robot (Turtlebot 2) in several simulations and real-world scenarios. Based on obtained results it is shown that this fast and simple approach is very powerful with good results in a variety of challenging environments, with both static and dynamic obstacles.

Real-Time Mine Road Boundary Detection and Tracking for Autonomous Truck.

Road boundary detection is an important part of the perception of the autonomous driving. It is difficult to detect road boundaries of unstructured roads because there are no curbs. There are no clear boundaries on mine roads to distinguish areas within the road boundary line and areas outside the road boundary line. This paper proposes a real-time road boundary detection and tracking method by a 3D-LIDAR sensor. The road boundary points are extracted from the detected elevated point clouds above the ground point cloud according to the spatial distance characteristics and the angular features. Road tracking is to predict and update the boundary point information in real-time, in order to prevent false and missed detection. The experimental verification of mine road data shows the accuracy and robustness of the proposed algorithm.

Multimodal localization: Stereo over LiDAR map

AbstractIn this paper, we present a real‐time high‐precision visual localization system for an autonomous vehicle which employs only low‐cost stereo cameras to localize the vehicle with a priori map built using a more expensive 3D LiDAR sensor. To this end, we construct two different visual maps: a sparse feature visual map for visual odometry (VO) based motion tracking, and a semidense visual map for registration with the prior LiDAR map. To register two point clouds sourced from different modalities (i.e., cameras and LiDAR), we leverage probabilistic weighted normal distributions transformation (ProW‐NDT), by particularly taking into account the uncertainty of source point clouds. The registration results are then fused via pose graph optimization to correct the VO drift. Moreover, surfels extracted from the prior LiDAR map are used to refine the sparse 3D visual features that will further improve VO‐based motion estimation. The proposed system has been tested extensively in both simulated and real‐world experiments, showing that robust, high‐precision, real‐time localization can be achieved.

Extrinsic Calibration between Camera and LiDAR Sensors by Matching Multiple 3D Planes.

This paper proposes a simple extrinsic calibration method for a multi-sensor system which consists of six image cameras and a 16-channel 3D LiDAR sensor using a planar chessboard. The six cameras are mounted on a specially designed hexagonal plate to capture omnidirectional images and the LiDAR sensor is mounted on the top of the plates to capture 3D points in 360 degrees. Considering each camera–LiDAR combination as an independent multi-sensor unit, the rotation and translation between the two sensor coordinates are calibrated. The 2D chessboard corners in the camera image are reprojected into 3D space to fit to a 3D plane with respect to the camera coordinate system. The corresponding 3D point data that scan the chessboard are used to fit to another 3D plane with respect to the LiDAR coordinate system. The rotation matrix is calculated by aligning normal vectors of the corresponding planes. In addition, an arbitrary point on the 3D camera plane is projected to a 3D point on the LiDAR plane, and the distance between the two points are iteratively minimized to estimate the translation matrix. At least three or more planes are used to find accurate external parameters between the coordinate systems. Finally, the estimated transformation is refined using the distance between all chessboard 3D points and the LiDAR plane. In the experiments, quantitative error analysis is done using a simulation tool and real test sequences are also used for calibration consistency analysis.

A Socially Aware SLAM Technique Augmented by Person Tracking Module

In recent years the development of Simultaneous Localization and Mapping (SLAM) techniques have enabled social robots to autonomously navigate in routine human workplaces. However, most common SLAM techniques are developed for mapping and localization in static worlds. In this paper, we have developed and analyzed a novel augmentation of the FastSLAM algorithm, including a person tracking module using 2D LiDAR sensor data. Utilizing this module, the SLAM algorithm is capable of filtering measurements coming from walking people who produce noises due to their intrinsic dynamic and unstableness. This augmentation was developed and then tested on our socially assistive mobile robot platform, Arash, while moving in populated environments utilizing Robotic Operating System (ROS) as a middleware. This new approach demonstrated a clearer representation of the mapped environment and therefore more accurate localization and navigation compared to its static SLAM predecessor.

Mapless Collaborative Navigation for a Multi-Robot System Based on the Deep Reinforcement Learning

Compared with the single robot system, a multi-robot system has higher efficiency and fault tolerance. The multi-robot system has great potential in some application scenarios, such as the robot search, rescue and escort tasks, and so on. Deep reinforcement learning provides a potential framework for multi-robot formation and collaborative navigation. This paper mainly studies the collaborative formation and navigation of multi-robots by using the deep reinforcement learning algorithm. The proposed method improves the classical Deep Deterministic Policy Gradient (DDPG) to address the single robot mapless navigation task. We also extend the single-robot Deep Deterministic Policy Gradient algorithm to the multi-robot system, and obtain the Parallel Deep Deterministic Policy Gradient (PDDPG). By utilizing the 2D lidar sensor, the group of robots can accomplish the formation construction task and the collaborative formation navigation task. The experiment results in a Gazebo simulation platform illustrates that our method is capable of guiding mobile robots to construct the formation and keep the formation during group navigation, directly through raw lidar data inputs.

Online learning for 3D LiDAR-based human detection: experimental analysis of point cloud clustering and classification methods

This paper presents a system for online learning of human classifiers by mobile service robots using 3D LiDAR sensors, and its experimental evaluation in a large indoor public space. The learning framework requires a minimal set of labelled samples (e.g. one or several samples) to initialise a classifier. The classifier is then retrained iteratively during operation of the robot. New training samples are generated automatically using multi-target tracking and a pair of “experts” to estimate false negatives and false positives. Both classification and tracking utilise an efficient real-time clustering algorithm for segmentation of 3D point cloud data. We also introduce a new feature to improve human classification in sparse, long-range point clouds. We provide an extensive evaluation of our the framework using a 3D LiDAR dataset of people moving in a large indoor public space, which is made available to the research community. The experiments demonstrate the influence of the system components and improved classification of humans compared to the state-of-the-art.