High-definition Maps Research Articles

A BOUNDARY-ENHANCED SUPERVOXEL METHOD FOR EXTRACTION OF ROAD EDGES IN MLS POINT CLOUDS

Abstract. Road extraction plays a significant role in production of high definition maps (HD maps). This paper presents a novel boundary-enhanced supervoxel segmentation method for extracting road edge contours from MLS point clouds. The proposed method first leverages normal feature judgment to obtain 3D point clouds global geometric information, then clusters points according to an existing method with global geometric information to enhance the boundaries. Finally, it utilizes the neighbor spatial distance metric to extract the contours and drop out existing outliers. The proposed method is tested on two datasets acquired by a RIEGL VMX-450 MLS system that contain the major point cloud scenes with different types of road boundaries. The experimental results demonstrate that the proposed method provides a promising solution for extracting contours efficiently and completely. Results show that the precision values are 1.5 times higher and approximately equal than the other two existing methods when the recall value is 0 for both tested two road datasets.

HDMI-Loc: Exploiting High Definition Map Image for Precise Localization via Bitwise Particle Filter

In this letter, we propose a method for accurately estimating the 6-Degree Of Freedom (DOF) pose in an urban environment when a High Definition (HD) map is available. An HD map expresses 3D geometric data with semantic information in a compressed format and thus is more memory-efficient than point cloud maps. The small capacity of HD maps can be a significant advantage for autonomous vehicles in terms of map storage and updates within a large urban area. Unfortunately, existing approaches failed to sufficiently exploit HD maps by only estimating partial pose. In this study, we present a full 6-DOF localization against an HD map using an onboard stereo camera with semantic information from roads. We introduce an 8-bit representation for road information, which allow for effective bitwise operation when matching between query data and the HD map. For the pose estimation, we leverage a particle filter followed by a full 6-DOF pose optimization. Our experimental results show a median error of approximately 0.3 m in the lateral and longitudinal directions for a drive of approximately 11 km. These results can be used by autonomous vehicles to correct the global position without using Global Positioning System (GPS) data in highly complex urban environments. The median operation speed is approximately 60 msec supporting 10 Hz.

GENERATING SYNTHETIC TRAINING DATA FOR OBJECT DETECTION USING MULTI-TASK GENERATIVE ADVERSARIAL NETWORKS

Abstract. Nowadays, digitizing roadside objects, for instance traffic signs, is a necessary step for generating High Definition Maps (HD Map) which remains as an open challenge. Rapid development of deep learning technology using Convolutional Neural Networks (CNN) has achieved great success in computer vision field in recent years. However, performance of most deep learning algorithms highly depends on the quality of training data. Collecting the desired training dataset is a difficult task, especially for roadside objects due to their imbalanced numbers along roadside. Although, training the neural network using synthetic data have been proposed. The distribution gap between synthetic and real data still exists and could aggravate the performance. We propose to transfer the style between synthetic and real data using Multi-Task Generative Adversarial Networks (SYN-MTGAN) before training the neural network which conducts the detection of roadside objects. Experiments focusing on traffic signs show that our proposed method can reach mAP of 0.77 and is able to improve detection performance for objects whose training samples are difficult to collect.

Automated Algorithm for Removing Clutter Objects in Mms Point Cloud for 3D Road Mapping.

Road information high definition maps (HD map) contain information about the facilities around the roads and are often constructed through a mobile mapping system (MMS). Although constructing an HD map is essential for road maintenance and the application of autonomous driving in the future, it is problematic to acquire the data of objects other than the facilities in an unstructured form while operating the MMS. In this study, the researchers define this object data as clutter objects and present a method of automatic removal using characteristics of the MMS and image segmentation techniques. By applying the method to 10 KITTI (Karlsruhe Institute of Technology and Toyota Technological Institute at Chicago) datasets, clutter objects were removed with an average overall accuracy of 91% with 0% (0.448%) error of commission for the complete point cloud map.

High Definition Map-Based Localization Using ADAS Environment Sensors for Application to Automated Driving Vehicles

This paper presents high definition (HD) map-based localization using advanced driver assistance system (ADAS) environment sensors for application to automated driving vehicles. A variety of autonomous driving technologies are being developed using expensive and high-performance sensors, but limitations exist due to several practical issues. In respect of the application of autonomous driving cars in the near future, it is necessary to ensure autonomous driving performance by effectively utilizing sensors that are already installed for ADAS purposes. Additionally, the most common localization algorithm, which is usually used lane information only, has a highly unstable disadvantage in the absence of that information. Therefore, it is essential to ensure localization performance with other road features such as guardrails when there are no lane markings. In this study, we would like to propose a localization algorithm that could be implemented in the near future by using low-cost sensors and HD maps. The proposed localization algorithm consists of several sections: environment feature representation with low-cost sensors, digital map analysis and application, position correction based on map-matching, designated validation gates, and extended Kalman filter (EKF)-based localization filtering and fusion. Lane information is detected by monocular vision in front of the vehicle. A guardrail is perceived by radar by distinguishing low-speed object measurements and by accumulating several steps to extract wall features. These lane and guardrail information are able to correct the host vehicle position by using the iterative closest point (ICP) algorithm. The rigid transformation between the digital high definition map (HD map) and environment features is calculated through ICP matching. Each corrected vehicle position by map-matching is selected and merged based on EKF with double updating. The proposed algorithm was verified through simulation based on actual driving log data.

정밀지도 객체 분류 및 네트워크를 이용한 교정 최적화

This paper aims to classify objects in high-definition maps used for routing, localizing, and planning the movement of unmanned vehicles. Currently, noise removal and object annotation are performed manually in such maps, which is a time-consuming process. To solve this problem, we propose automating this process through object classification algorithms using point clouds and images. First, our method measures the data, and as the sensor moves, it generates noise. We remove the noise by merging the two data sets using deep learning. Second, to classify objects, the objects were isolated using the Image Only. The image was resolved to classify objects using deep learning.

Lane-Marker-Based Map Construction and Map Precision Evaluation Methods Using On-Board Cameras for Autonomous Driving

This report describes a map construction and evaluation method based on lane-marker information for autonomous driving. Autonomous driving systems typically require digital high-definition (HD) maps to correct the current position of autonomous vehicles by using localization techniques. However, an HD map is usually costly to generate because it requires a special-purpose vehicle and mapping system with precise and expensive sensors. This report presents a map construction method that uses cost-efficient on-board cameras. We implement two types of map construction methods with two different cameras in terms of range and field of view and test their performances to determine the minimum sensor specification required for autonomous driving. This report also presents a constructed map evaluation method to determine the “usability” of the map for autonomous driving. Given that the system cannot obtain “true” positions of landmarks, the method judges whether the constructed map contains sufficient information for localization via the presented indices “lateral-distance error.” The methods are verified based on mapping and localization errors determined via manual driving tests. Furthermore, the smoothness of steering maneuvers is determined by conducting autonomous driving tests on a proving ground. The results reveal the necessary conditions of sensor requirements, i.e., the constant visibility of landmarks is one of the key factors for ego-localization to conduct autonomous driving.

A Hierarchical Control System for Autonomous Driving towards Urban Challenges

In recent years, the self-driving car technologies have been developed with many successful stories in both academia and industry. The challenge for autonomous vehicles is the requirement of operating accurately and robustly in the urban environment. This paper focuses on how to efficiently solve the hierarchical control system of a self-driving car into practice. This technique is composed of decision making, local path planning and control. An ego vehicle is navigated by global path planning with the aid of a High Definition map. Firstly, we propose the decision making for motion planning by applying a two-stage Finite State Machine to manipulate mission planning and control states. Furthermore, we implement a real-time hybrid A* algorithm with an occupancy grid map to find an efficient route for obstacle avoidance. Secondly, the local path planning is conducted to generate a safe and comfortable trajectory in unstructured scenarios. Herein, we solve an optimization problem with nonlinear constraints to optimize the sum of jerks for a smooth drive. In addition, controllers are designed by using the pure pursuit algorithm and the scheduled feedforward PI controller for lateral and longitudinal direction, respectively. The experimental results show that the proposed framework can operate efficiently in the urban scenario.

Capsule-Based Networks for Road Marking Extraction and Classification From Mobile LiDAR Point Clouds

Accurate road marking extraction and classification play a significant role in the development of autonomous vehicles (AVs) and high-definition (HD) maps. Due to point density and intensity variations from mobile laser scanning (MLS) systems, most of the existing thresholding-based extraction methods and rule-based classification methods cannot deliver high efficiency and remarkable robustness. To address this, we propose a capsule-based deep learning framework for road marking extraction and classification from massive and unordered MLS point clouds. This framework mainly contains three modules. Module I is first implemented to segment road surfaces from 3D MLS point clouds, followed by an inverse distance weighting (IDW) interpolation method for 2D georeferenced image generation. Then, in Module II, a U-shaped capsule-based network is constructed to extract road markings based on the convolutional and deconvolutional capsule operations. Finally, a hybrid capsule-based network is developed to classify different types of road markings by using a revised dynamic routing algorithm and large-margin Softmax loss function. A road marking dataset containing both 3D point clouds and manually labeled reference data is built from three types of road scenes, including urban roads, highways, and underground garages. The proposed networks were accordingly evaluated by estimating robustness and efficiency using this dataset. Quantitative evaluations indicate the proposed extraction method can deliver 94.11% in precision, 90.52% in recall, and 92.43% in F <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -score, respectively, while the classification network achieves an average of 3.42% misclassification rate in different road scenes.

Lane Detection: A Survey with New Results

Lane detection is essential for many aspects of autonomous driving, such as lane-based navigation and high-definition (HD) map modeling. Although lane detection is challenging especially with complex road conditions, considerable progress has been witnessed in this area in the past several years. In this survey, we review recent visual-based lane detection datasets and methods. For datasets, we categorize them by annotations, provide detailed descriptions for each category, and show comparisons among them. For methods, we focus on methods based on deep learning and organize them in terms of their detection targets. Moreover, we introduce a new dataset with more detailed annotations for HD map modeling, a new direction for lane detection that is applicable to autonomous driving in complex road conditions, a deep neural network LineNet for lane detection, and show its application to HD map modeling.

Self-Driving Car Location Estimation Based on a Particle-Aided Unscented Kalman Filter.

Localization is one of the key components in the operation of self-driving cars. Owing to the noisy global positioning system (GPS) signal and multipath routing in urban environments, a novel, practical approach is needed. In this study, a sensor fusion approach for self-driving cars was developed. To localize the vehicle position, we propose a particle-aided unscented Kalman filter (PAUKF) algorithm. The unscented Kalman filter updates the vehicle state, which includes the vehicle motion model and non-Gaussian noise affection. The particle filter provides additional updated position measurement information based on an onboard sensor and a high definition (HD) map. The simulations showed that our method achieves better precision and comparable stability in localization performance compared to previous approaches.

Lane-Level Map-Matching Method for Vehicle Localization Using GPS and Camera on a High-Definition Map.

Accurate vehicle localization is important for autonomous driving and advanced driver assistance systems. Existing precise localization systems based on the global navigation satellite system cannot always provide lane-level accuracy even in open-sky environments. Map-based localization using high-definition (HD) maps is an interesting method for achieving greater accuracy. We propose a map-based localization method using a single camera. Our method relies on road link information in the HD map to achieve lane-level accuracy. Initially, we process the image—acquired using the camera of a mobile device—via inverse perspective mapping, which shows the entire road at a glance in the driving image. Subsequently, we use the Hough transform to detect the vehicle lines and acquire driving link information regarding the lane on which the vehicle is moving. The vehicle position is estimated by matching the global positioning system (GPS) and reference HD map. We employ iterative closest point-based map-matching to determine and eliminate the disparity between the GPS trajectories and reference map. Finally, we perform experiments by considering the data of a sophisticated GPS/inertial navigation system as the ground truth and demonstrate that the proposed method provides lane-level position accuracy for vehicle localization.

Automatic Building and Labeling of HD Maps with Deep Learning

In a world where autonomous driving cars are becoming increasingly more common, creating an adequate infrastructure for this new technology is essential. This includes building and labeling high-definition (HD) maps accurately and efficiently. Today, the process of creating HD maps requires a lot of human input, which takes time and is prone to errors. In this paper, we propose a novel method capable of generating labelled HD maps from raw sensor data. We implemented and tested our methods on several urban scenarios using data collected from our test vehicle. The results show that the proposed deep learning based method can produce highly accurate HD maps. This approach speeds up the process of building and labeling HD maps, which can make meaningful contribution to the deployment of autonomous vehicles.

High Definition Map Based Motion Plan and Control of Autonomous Vehicle on Structured Road

A high definition (HD) map based motion plan and control method of autonomous vehicle on structured road is proposed in this paper. The system is designed in a multi-layer structure: a motion planner and a motion controller, furthermore, the motion planner is consisted of a global path planner and a local trajectory planner. The inputs of the global path planner are HD map, start and goal state, Dynamic Program (DP) is used to plan a global path that connect these two states based on the lane-level structures and traffic rules extracted from HD map. The inputs of local motion planner are the planned global path, current host vehicle state and the surrounding dynamic object information, several candidate trajectories are generated in Frenet frame, and then the optimal one is chosen by a multi-term cost function calculation. Taking the vehicle dynamics under high speed into consideration, Model Predictive Control (MPC) is used in motion controller to track the planned target trajectory. Results of the conducted simulation tests show that the designed motion plan and control method works well in velocity keeping, vehicle following, stopping and going, and vehicle passing scenarios.

Monocular Localization with Vector HD Map (MLVHM): A Low-Cost Method for Commercial IVs.

Real-time vehicle localization (i.e., position and orientation estimation in the world coordinate system) with high accuracy is the fundamental function of an intelligent vehicle (IV) system. In the process of commercialization of IVs, many car manufacturers attempt to avoid high-cost sensor systems (e.g., RTK GNSS and LiDAR) in favor of low-cost optical sensors such as cameras. The same cost-saving strategy also gives rise to an increasing number of vehicles equipped with High Definition (HD) maps. Rooted upon these existing technologies, this article presents the concept of Monocular Localization with Vector HD Map (MLVHM), a novel camera-based map-matching method that efficiently aligns semantic-level geometric features in-camera acquired frames against the vector HD map in order to achieve high-precision vehicle absolute localization with minimal cost. The semantic features are delicately chosen for the ease of map vector alignment as well as for the resiliency against occlusion and fluctuation in illumination. The effective data association method in MLVHM serves as the basis for the camera position estimation by minimizing feature re-projection errors, and the frame-to-frame motion fusion is further introduced for reliable localization results. Experiments have shown that MLVHM can achieve high-precision vehicle localization with an RMSE of 24 cm with no cumulative error. In addition, we use low-cost on-board sensors and light-weight HD maps to achieve or even exceed the accuracy of existing map-matching algorithms.

Artificial intelligence applications in the development of autonomous vehicles: a survey

The advancement of artificial intelligence &#x0028 AI &#x0029 has truly stimulated the development and deployment of autonomous vehicles &#x0028 AVs &#x0029 in the transportation industry. Fueled by big data from various sensing devices and advanced computing resources, AI has become an essential component of AVs for perceiving the surrounding environment and making appropriate decision in motion. To achieve goal of full automation &#x0028 i.e., self-driving &#x0028 , it is important to know how AI works in AV systems. Existing research have made great efforts in investigating different aspects of applying AI in AV development. However, few studies have offered the research community a thorough examination of current practices in implementing AI in AVs. Thus, this paper aims to shorten the gap by providing a comprehensive survey of key studies in this research avenue. Specifically, it intends to analyze their use of AIs in supporting the primary applications in AVs: 1&#x0029 perception; 2&#x0029 localization and mapping; and 3&#x0029 decision making. It investigates the current practices to understand how AI can be used and what are the challenges and issues associated with their implementation. Based on the exploration of current practices and technology advances, this paper further provides insights into potential opportunities regarding the use of AI in conjunction with other emerging technologies: 1&#x0029 high definition maps, big data, and high performance computing; 2&#x0029 augmented reality&#x0028 AR &#x0029 &#x002F virtual reality &#x0028 VR &#x0029 enhanced simulation platform; and 3&#x0029 5G communication for connected AVs. This paper is expected to offer a quick reference for researchers interested in understanding the use of AI in AV research.

High Definition 3D Map Creation Using GNSS/IMU/LiDAR Sensor Integration to Support Autonomous Vehicle Navigation.

Recent developments in sensor technologies such as Global Navigation Satellite Systems (GNSS), Inertial Measurement Unit (IMU), Light Detection and Ranging (LiDAR), radar, and camera have led to emerging state-of-the-art autonomous systems, such as driverless vehicles or UAS (Unmanned Airborne Systems) swarms. These technologies necessitate the use of accurate object space information about the physical environment around the platform. This information can be generally provided by the suitable selection of the sensors, including sensor types and capabilities, the number of sensors, and their spatial arrangement. Since all these sensor technologies have different error sources and characteristics, rigorous sensor modeling is needed to eliminate/mitigate errors to obtain an accurate, reliable, and robust integrated solution. Mobile mapping systems are very similar to autonomous vehicles in terms of being able to reconstruct the environment around the platforms. However, they differ a lot in operations and objectives. Mobile mapping vehicles use professional grade sensors, such as geodetic grade GNSS, tactical grade IMU, mobile LiDAR, and metric cameras, and the solution is created in post-processing. In contrast, autonomous vehicles use simple/inexpensive sensors, require real-time operations, and are primarily interested in identifying and tracking moving objects. In this study, the main objective was to assess the performance potential of autonomous vehicle sensor systems to obtain high-definition maps based on only using Velodyne sensor data for creating accurate point clouds. In other words, no other sensor data were considered in this investigation. The results have confirmed that cm-level accuracy can be achieved.

Large-volume data dissemination for cellular-assisted automated driving with edge intelligence

Automated driving emerges as a potential technology for safe, convenient, and efficient transportation systems. In order to assist autonomous vehicles with large-volume data dissemination such as high definition (HD) map, cellular network becomes more inevitable to provide vehicles with the ability of accessing Internet. However, due to the bottleneck of wireless bandwidth and the high mobility of vehicles, efficient data dissemination assisted by cellular network is a challenging task. In this work, we investigate the large-volume data dissemination problem for cellular-assisted automated driving within given delay limit. Firstly, a data dissemination scheme is designed that transmits data from the base stations to the target vehicle via a collection of relay vehicles. Secondly, a novel user satisfaction model is derived by considering the vehicular mobility patterns and erasure coding, and the data dissemination is formulated into an optimization problem. Then, to solve this NP-hard problem, a low-complexity dynamic programming based data dissemination solution with edge intelligence is proposed. Finally, by carrying out extensive simulation, the effectiveness of proposed data dissemination scheme for cellular-assisted automated driving is demonstrated, in comparison with the optimal solution based on CPLEX as well as other conventional schemes.

DLT-Net: Joint Detection of Drivable Areas, Lane Lines, and Traffic Objects

Perception is an essential task for self-driving cars, but most perception tasks are usually handled independently. We propose a unified neural network named DLT-Net to detect drivable areas, lane lines, and traffic objects simultaneously. These three tasks are most important for autonomous driving, especially when a high-definition map and accurate localization are unavailable. Instead of separating tasks in the decoder, we construct context tensors between sub-task decoders to share designate influence among tasks. Therefore, each task can benefit from others during multi-task learning. Experiments show that our model outperforms the conventional multi-task network in terms of the task-wise accuracy and the overall computational efficiency, in the challenging BDD dataset.

Multi-Scale Point-Wise Convolutional Neural Networks for 3D Object Segmentation From LiDAR Point Clouds in Large-Scale Environments

Although significant improvement has been achieved in fully autonomous driving and semantic high-definition map (HD) domains, most of the existing 3D point cloud segmentation methods cannot provide high representativeness and remarkable robustness. The principally increasing challenges remain in completely and efficiently extracting high-level 3D point cloud features, specifically in large-scale road environments. This paper provides an end-to-end feature extraction framework for 3D point cloud segmentation by using dynamic point-wise convolutional operations in multiple scales. Compared to existing point cloud segmentation methods that are commonly based on traditional convolutional neural networks (CNNs), our proposed method is less sensitive to data distribution and computational powers. This framework mainly includes four modules. Module I is first designed to construct a revised 3D point-wise convolutional operation. Then, a U-shaped downsampling-upsampling architecture is proposed to leverage both global and local features in multiple scales in Module II. Next, in Module III, high-level local edge features in 3D point neighborhoods are further extracted by using an adaptive graph convolutional neural network based on the K-Nearest Neighbor (KNN) algorithm. Finally, in Module IV, a conditional random field (CRF) algorithm is developed for postprocessing and segmentation result refinement. The proposed method was evaluated on three large-scale LiDAR point cloud datasets in both urban and indoor environments. The experimental results acquired by using different point cloud scenarios indicate our method can achieve state-of-the-art semantic segmentation performance in feature representativeness, segmentation accuracy, and technical robustness.