High-definition Maps Research Articles

THE IMPLEMENTATION OF SEMI-AUTOMATED ROAD SURFACE MARKINGS EXTRACTION SCHEMES UTILIZING MOBILE LASER SCANNED POINT CLOUDS FOR HD MAPS PRODUCTION

Abstract. As research on autonomous driving deepens, High-definition Maps (HD Maps) have gradually become an auxiliary information for the new generation of autonomous driving technology. Compared to traditional electronic navigation maps, HD Maps have higher accuracy requirements and more information. Multi-road environment information and road elements are included. In the production of HD Maps, the on-board Mobile Laser Scanning (MLS) system has the ability to quickly collect environmental information, with high precision, thus making the system a widely used data collection method today. However, subsequent map building, digitization, and other mapping work still rely on manual operation, which is time-consuming and laborious. Therefore, this research is dedicated to developing a semi-automatic algorithm to generate HD Maps from the acquired point cloud data. This research focuses on the extraction of road surface markings, using the Cloth Simulation Filter (CSF) to obtain the road surface point cloud to improve the extraction efficiency. The road markings are extracted using the characteristic of high intensity values, and the commonly used Otsu threshold filter in image processing is used to extract point clouds with high reflectance intensity, eliminating the need for manual setting of point clouds. And based on geometric conditions, the objects are classified, such as arrow lines, pedestrian crossings, stop lines, and lane lines, which are convenient for further mapping HD Maps.

Road Shape Classification-Based Matching Between Lane Detection and HD Map for Robust Localization of Autonomous Cars

Many map matching-based localization algorithms estimate the autonomous car's pose using a registration between lanes measured by a camera and the High-Definition (HD) map. However, the registration methods based on numerical optimization, such as the Iterative Closest Point (ICP) and Normal Distributions Transform (NDT), can cause underdetermined problems due to no unique solution when matching under-constrained lane shapes such as straight lines and circular arcs. This paper proposes a robust localization algorithm with centimeter-level accuracy by applying different matching techniques depending on the road shape. We proposed a road shape classification-based map matching algorithm to overcome the under-constrained problems, which have no unique solution due to insufficient constraints. The proposed algorithm classifies lane segments into line, arc, and clothoid curves considering their curvature characteristics. After that, we find correction information through a map matching and covariance estimation method using lane pairs with the same type of their shape. For the under-constrained shapes, the geometry-based map-matching algorithm and covariance estimation method are applied to avoid underdetermined results. Finally, the measurement calculated from the correction information and predicted pose of the egovehicle is exploited for the measurement update of the Extended Kalman Filter (EKF). The proposed method was quantitatively evaluated in the simulation environment, which contains various road shapes, and qualitatively validated for experimental data from an autonomous driving platform. The proposed algorithm shows more robust matching capability, efficient computation time, and higher accuracy than the localization system based on ICP-based lane matching.

Time Efficient Offloading Optimization in Automotive Multi-Access Edge Computing Networks Using Mean-Field Games

Emerging connected vehicular services, such as intelligent driving and high-definition (HD) map, are gaining increasing interest with the fast development of multi-access edge computing (MEC). For most time-sensitive and computation-intensive vehicular services, the data offloading process significantly influences the capacity and performance of MEC, especially when the number of connected vehicles is enormous. In this work, we consider data offloading optimization for a large-scale automotive MEC network. The problem is challenging due to the large number of connected vehicles and the complicated interaction between vehicles and edge servers. To tackle the scalability problem, we reformulate the original offloading optimization problem into a Mean-Field-Game (MFG) problem by abstracting the interaction among the connected vehicles as a distribution over their state spaces of task sizes, known as the mean-field term. To solve the problem efficiently, we propose a G-prox Primal-Dual-Hybrid-Gradient (PDHG) algorithm that transforms the MFG problem into a saddle-point problem. Based on our developed MFG model and G-prox PDHG algorithm, we propose the first data offloading scheme whose computation time is independent of the number of connected vehicles in automotive MEC systems. Extensive evaluation results corroborate the superior performance of our proposed scheme compared with the state-of-the-art methods.

Lymphocyte networks are dynamic cellular communities in the immunoregulatory landscape of lung adenocarcinoma.

Lymphocytes are key for immune surveillance of tumors, but our understanding of the spatial organization and physical interactions that facilitate lymphocyte anti-cancer functions is limited. We used multiplexed imaging, quantitative spatial analysis, and machine learning to create high-definition maps of lung tumors from a Kras/Trp53-mutant mouse model and human resections. Networks of interacting lymphocytes ("lymphonets") emerged as a distinctive feature of the anti-cancer immune response. Lymphonets nucleated from small Tcell clusters and incorporated B cells with increasing size. CXCR3-mediated trafficking modulated lymphonet size and number, but Tcell antigen expression directed intratumoral localization. Lymphonets preferentially harbored TCF1+ PD-1+ progenitor CD8+ Tcells involved in responses to immune checkpoint blockade (ICB) therapy. Upon treatment of mice with ICB or an antigen-targeted vaccine, lymphonets retained progenitor and gained cytotoxic CD8+ Tcell populations, likely via progenitor differentiation. These data show that lymphonets create a spatial environment supportive of CD8+ Tcell anti-tumor responses.

An automated driving systems data acquisition and analytics platform

In this paper, an automated driving system (ADS) data acquisition and analytics platform for vehicle trajectory extraction, reconstruction, and evaluation based on connected automated vehicle (CAV) cooperative perception are presented. This platform presents a holistic pipeline from the raw advanced sensory data collection to data processing, which is capable of processing the sensor data from multi-CAVs and extracting the objects’ Identity (ID) number, position, speed, and orientation information in the map and Frenet coordinates. First, the ADS data acquisition and analytics platform are presented. Specifically, the experimental CAVs platform and sensor configuration are shown, and the processing software, including a deep-learning-based object detection algorithm using LiDAR information, a late fusion scheme to leverage cooperative perception to fuse the detected objects from multi-CAVs, and a multi-object tracking method is introduced. To further enhance the object detection and tracking results, high-definition maps consisting of point cloud and vector maps are generated and forwarded to a world model to filter out the objects off the road and extract the objects’ coordinates in Frenet coordinates and the lane information. In addition, to refine trajectories from the object tracking algorithms, a post-processing method is proposed. Given the objects’ information from the object detection and tracking and the world model, a Kalman filter and Chi-square test method are applied to reduce the noise and remove the outlier in the trajectories. Aiming at tackling the ID switch issue of the object tracking algorithm, a fuzzy-logic-based approach is proposed to detect the discontinuous trajectories belonging to the same object. Then, a vehicle-kinematics-based trajectory prediction method is used, and a forward–backward-smoothing technique is applied to reconstruct the trajectory between the discontinuous trajectories. Finally, results, including object detection and tracking and a late fusion scheme, are presented, and the improvements by the post-processing algorithm in terms of noise level and outlier removal are discussed, which confirm the functionality and effectiveness of the proposed holistic data collection and processing platform. In another aspect, the extracted objects’ information and generated HD maps can be used for several purposes in the transportation research community and ADS development community: analyzing the interaction between human-driven vehicles and ADS-equipped vehicles, car-following behavior analysis of ADS-equipped vehicles, traffic flow status analysis and modeling, and scenario generation for ADS testing.

A new method for constructing roads map in forest area using UAV images

High definition map is a map used for automatic driving. Its absolute precision is sub meter level, that is, the error between the position of an element on the map and the position of the element in the real world is less than 100 cm. In addition, compared with ordinary maps, high definition maps contain more geographic information, such as road width, slope, curvature, etc. At present, high definition roads map construction is largely for the main traffic road. However, there are few studies on the construction of high definition forest roads map for forestry production and management. Thus, it is imperative and helpful to explore an efficient and low-cost method for constructing high definition forest roads map that can provide accurate basic map services for forestry management activities. In this study, firstly, the DJI Phantom 4 RTK UAV was used for aerial photography to obtain the forest oblique images. Then, combined with close range photogrammetry, the high-resolution images under the forest are acquired and generated into dense point cloud images. Next, the two images are fused using iterative closest point (ICP) algorithm combined with manual registration; Finally, the fine-grained 3D scene model is generated by using the Context Capture 3D modeling software, based on which a high definition forest roads map was constructed using human-computer interaction model. Taking Menggongshan Forest Farm as an example, the construction of the fine 3D scene model was carried out to solve the problem of loopholes and ambiguities of the road model under forest, based on which the high definition forest roads map was constructed. To evaluate the accuracy of the forest roads map, the accuracy check was conducted. The results showed that the point position on the forest roads map, compared to the actual point, had a medium, minimum and maximum error of 0.076 m, 0.020 m and 0.170 m in plane, respectively, a medium, minimum and maximum error of 0.122 m, 0.005 m and -0.214 m in elevation, respectively. The accuracy of the high definition forest roads map meets the need of forestry production and management activities. It is feasible to adopt the method of UAV oblique photogrammetry combined with close-range photogrammetry, fine 3D scene model construction and human-computer interactive for constructing high definition forest road map. And this method can provide a low-cost and high-efficiency way for constructing high definition forest roads map, and also can better provide basic map data for forest management activities.

Real-Time Dynamic Map With Crowdsourcing Vehicles in Edge Computing

Autonomous driving perceives surroundings with line-of-sight sensors that are compromised under environmental uncertainties. To achieve real time global information in high definition map, we investigate to share perception information among connected and automated vehicles. However, it is challenging to achieve real time perception sharing under varying network dynamics in automotive edge computing. In this paper, we propose a novel real time dynamic map, named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LiveMap</i> to detect, match, and track objects on the road. We design the data plane of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LiveMap</i> to efficiently process individual vehicle data with multiple sequential computation components, including detection, projection, extraction, matching and combination. We design the control plane of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LiveMap</i> to achieve adaptive vehicular offloading with two new algorithms (central and distributed) to balance the latency and coverage performance based on deep reinforcement learning techniques. We conduct extensive evaluation through both realistic experiments on a small-scale physical testbed and network simulations on an edge network simulator. The results suggest that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LiveMap</i> significantly outperforms existing solutions in terms of latency, coverage, and accuracy.

Integrity Management of the Reachable Space With Lane Grid Maps

For an autonomous vehicle, reliable situation understanding is a key component of safe navigation. An incorrect prediction of an upcoming situation means that erroneous information may be supplied to the decision-making process, leading to hazardous outcomes. It is therefore of great importance to estimate the driving areas that are reachable by other interacting road users, without introducing misleading information. This paper presents a means of handling the integrity of prediction information, given the imperfection of object prediction, via a Lane Grid Map, that is to say a spatial representation of the situation at a tactical level, based on the topological layer of a high-definition map. We demonstrate experimentally, using real data, how the spatial sampling step of the grid representation can be used to manage the integrity of prediction information. Moreover, addressing interactions during the prediction makes it possible to handle some particular situations safely. We show how some interactions can be utilized via the concept of neutralization. To quantify the integrity of the prediction, we propose the use of two metrics, namely False Negative Rate and Neutralized Time Interval. Experiments were carried out with three vehicles to evaluate the integrity of the prediction using these metrics.

A review of high-definition map creation methods for autonomous driving

Autonomous driving has been among the most popular and challenging topics in the past few years. Among all modules in autonomous driving, High-definition (HD) map has drawn lots of attention in recent years due to its high precision and informative level in localization. Since localization is a significant module for automated vehicles to navigate an unknown environment, it has immediately become one of the most critical components of autonomous driving. Big organizations like HERE, NVIDIA, and TomTom have created HD maps for different scenes and purposes for autonomous driving. However, such HD maps are not open-source and are only available for internal research or automotive companies. Even though researchers have proposed various methods to create HD maps using different types of sensor data, there are few papers that review and summarize those methods. New researchers do not have a clear insight into the current state of HD map creation methods to work on their HD map research. Due to the reason above, reviewing, classifying, comparing, and summarizing the state-of-the-art techniques for HD map creation is necessary. This paper reviews recent HD map creation methods that leverage both 2D and 3D map generation. This review introduces the concept of HD maps and their usefulness in autonomous driving and gives a detailed overview of HD map creation methods. We will also discuss the limitations of the current HD map creation methods to motivate future research. Additionally, a chronological overview is created with the most recent HD map creation methods in this paper.

High Definition Map Aided Object Detection for Autonomous Driving in Urban Areas

High Definition Map Aided Object Detection for Autonomous Driving in Urban Areas

Bust Portraits Matting Based on Improved U-Net

Extracting complete portrait foregrounds from natural images is widely used in image editing and high-definition map generation. When making high-definition maps, it is often necessary to matte passers-by to guarantee their privacy. Current matting methods that do not require additional trimap inputs often suffer from inaccurate global predictions or blurred local details. Portrait matting, as a soft segmentation method, allows the creation of excess areas during segmentation, which inevitably leads to noise in the resulting alpha image as well as excess foreground information, so it is not necessary to keep all the excess areas. To overcome the above problems, this paper designed a contour sharpness refining network (CSRN) that modifies the weight of the alpha values of uncertain regions in the prediction map. It is combined with an end-to-end matting network for bust matting based on the U-Net target detection network containing Residual U-blocks. An end-to-end matting network for bust matting is designed. The network can effectively reduce the image noise without affecting the complete foreground information obtained by the deeper network, thus obtaining a more detailed foreground image with fine edge details. The network structure has been tested on the PPM-100, the RealWorldPortrait-636, and a self-built dataset, showing excellent performance in both edge refinement and global prediction for half-figure portraits.

Traffic lights detection and tracking for HD map creation.

HD-maps are one of the core components of the self-driving pipeline. Despite the effort of many companies to develop a completely independent vehicle, many state-of-the-art solutions rely on high-definition maps of the environment for localization and navigation. Nevertheless, the creation process of such maps can be complex and error-prone or expensive if performed via ad-hoc surveys. For this reason, robust automated solutions are required. One fundamental component of an high-definition map is traffic lights. In particular, traffic light detection has been a well-known problem in the autonomous driving field. Still, the focus has always been on the light state, not the features (i.e., shape, orientation, pictogram). This work presents a pipeline for lights HD-map creation designed to provide accurate georeferenced position and description of all traffic lights seen by a camera mounted on a surveying vehicle. Our algorithm considers consecutive detection of the same light and uses Kalman filtering techniques to provide each target's smoother and more precise position. Our pipeline has been validated for the detection and mapping task using the state-of-the-art dataset DriveU Traffic Light Dataset. The results show that our model is robust even with noisy GPS data. Moreover, for the detection task, we highlight how our model can correctly identify even far-away targets which are not labeled in the original dataset.

Minimum required accuracy for HD maps

AbstractAutonomous vehicles rely on a combination of sensors for safe navigation around the world. For precise localisation, high-definition (HD) maps are used. These maps are a representation of the world containing information about objects on the road infrastructure. Currently, there are tens of HD map makers, however, no rigorous description of the requirements for the accuracy of HD maps has been published yet. This study fills the gap and offers a mathematical description of the minimum required accuracy for HD maps. In the first part, we identify factors that influence the quality of a map. Based on that, we proceed to present our solution for determining the minimum required accuracy for HD maps, both for static and dynamic models, and present a new formula for the minimum necessary accuracy for HD maps.

VIS-MM: a novel map-matching algorithm with semantic fusion from vehicle-borne images

Conventional map-matching (MM) algorithms take blind eyes to the complexity in realistic traffic conditions and hence present significant limitations in distinguishing the detailed driving paths of vehicles within complex urban road networks. The popularity of vehicle-borne cameras and advances in image recognition technologies provide an opportunity to remedy the gap through integrating vehicle-borne image semantic information with MM algorithms. Following this logic, this article proposes a novel MM algorithm with semantic fusion from vehicle-borne images (VIS-MM) suited to the parallel road scenes. First, a multipath output algorithm is developed using the hidden Markov model to obtain candidate paths. Second, image recognition techniques are employed to extract vehicle-borne image semantics. Finally, the entropy weight method is performed to determine the most promising driving path among the candidate paths. The experimental results show that semantic fusion from vehicle-borne images contributes to a significant improvement of accuracy from 66.18% to 99.88% against the parallel road scenes. The proposed map-matching algorithm can be applied into the fields of unmanned autonomous navigation and crowdsourcing updating of high-definition maps.

Investigating the Improvement of Autonomous Vehicle Performance through the Integration of Multi-Sensor Dynamic Mapping Techniques

The emergence of autonomous vehicles marks a shift in mobility. Conventional vehicles have been designed to prioritize the safety of drivers and passengers and increase fuel efficiency, while autonomous vehicles are developing as convergence technologies with a focus on more than just transportation. With the potential for autonomous vehicles to serve as an office or leisure space, the accuracy and stability of their driving technology is of utmost importance. However, commercializing autonomous vehicles has been challenging due to the limitations of current technology. This paper proposes a method to build a precision map for multi-sensor-based autonomous driving to improve the accuracy and stability of autonomous vehicle technology. The proposed method leverages dynamic high-definition maps to enhance the recognition rates and autonomous driving path recognition of objects in the vicinity of the vehicle, utilizing multiple sensors such as cameras, LIDAR, and RADAR. The goal is to improve the accuracy and stability of autonomous driving technology.

Love of Variety Based Latency Analysis for High Definition Map Updating: Age of Information and Distributional Robust Perspectives

High definition (HD) map is a key technology that enables autonomous driving, which has the characteristic of frequent updates and low delay requirements. In order to minimize the HD map updating latency, this paper jointly considers the generation and transmission processes of HD map. The generation process of HD map relies heavily on the data captured by different types of sensors, which is modeled by the federated analytics process. Also, in order to quantify the diversity of utilized sensors, the concept of love of variety is introduced in this paper. Then, considering HD map is composed of dynamic and static layers that have different latency requirements during the transmission process, this paper proposes a method to allocate edge server capacity to each HD map layer such that the overall information staleness can be minimized. Firstly, the deterministic edge capacity case is discussed and the solution is derived by obtaining the Karush—Kuhn—Tucker conditions. Then, considering the practice that an edge server provides services to multiple attached devices simultaneously, contentions among these devices make available capacity for the autonomous vehicle variational. Therefore, an uncertain edge capacity case is discussed as well, where the uncertainty is described by Wasserstein metrics and the problem is reformulated into distributional robust chance constrained optimization problem. And for the low latency purpose, we utilize an inner approximation to reduce the complexity of the original problem and find a suboptimal solution. Finally, simulation results demonstrate the effectiveness of our proposed methods that achieves the lowest latency for HD map updating.

Camera-Based Lane-Aided Multi-Information Integration for Land Vehicle Navigation

Accurate positioning, especially lateral positioning, is an essential requirement of autonomous driving. Although high accuracy vehicle positioning can be conducted in open-sky environments by global navigation satellite system (GNSS), the positioning accuracy cannot be guaranteed in GNSS-denied environments such as urban canyons and tunnels. To solve this issue, inertial navigation systems, vehicle speed sensors, and vehicle motion constraints are often fused to mitigate positioning errors. However, this integration cannot meet the decimeter-level lateral positioning accuracy of autonomous vehicles&#x2019; localization requirements. This article proposes a multi-information integration method aided by lane distance to further eliminate lateral error. In the proposed method, the lateral vehicle-to-lane distance measurements from camera-based systems, and the map-matching lane distance based on high-definition map were utilized to provide absolute lane distance measurement corrections in GNSS-denied environments. Field vehicular tests with low-cost integration systems were conducted to evaluate the navigation performance of the proposed multi-information integration method, and the results indicated that it is feasible for the proposed method to maintain continuous and reliable lateral positioning accuracy of better than 0.6 m under reliable lane line detection conditions in GNSS-denied environments.

Real-Time Scan-to-Map Matching Localization System Based on Lightweight Pre-Built Occupancy High-Definition Map

High-precision and robust localization in GNSS-denied areas is crucial for autonomous vehicles and robots. Most state-of-the-art localization methods are based on simultaneous localization and mapping (SLAM) with a camera or light detection and ranging (LiDAR). However, SLAM will suffer from drift during long-term running without loop closure or prior constraints. Lightweight, high-precision environmental maps have gradually become an indispensable part of future autonomous driving. In order to solve the problem of real-time global localization for autonomous vehicles and robots, we propose a precise and robust LiDAR localization system based on a pre-built, occupied high-definition (HD) map called the Extended QuadTree (EQT) map. It makes use of a planar quadtree for block division and a Z-sequence index structure within the block cells. Then, a four-level occupancy probability cell value model is adopted. It will save about eight times the storage space compared with Google Cartographer, and the EQT map can be extended to store other information. For efficient scan-to-map matching with our specialized EQT map, the Bursa linearized model is used in the Gauss–Newton iteration of our algorithm, which makes the calculation of partial derivatives fast. All the above improvements lead to optimal storage and efficient querying for real-time scan-to-map matching localization. Field tests in an industrial park and road environment prove that positioning accuracy of about 6–13 cm and attitude accuracy of about 0.15° were achieved using a VLP-16 LiDAR. They also show that the method proposed in this paper is significantly better than the NDT method. For the long and narrow environment of an underground mine tunnel, high-resolution maps are also helpful for accurate and robust localization.

Targeting Lane-Level Map Matching for Smart Vehicles: Construction of High-Definition Road Maps Based on GIS

The development of smart vehicles has increased the demand for high-definition road maps. However, traditional road maps for vehicle navigation systems are not sufficient to meet the requirements of intelligent vehicle systems (e.g., autonomous driving). The present work comes up with a method of generating high-definition map models based on the geographic information system (GIS). A systematic map construction framework including the road layer, intersection connection layer, and lane layer is proposed based on the GIS database. Specifically, the constrained Delaunay triangular network method is applied to extract road layer network models, which are then used as linear reference networks to construct lane-level road maps. To further examine the feasibility of the proposed framework, a field experiment is then conducted to build a high-definition road map. Furthermore, a lane-level map matching test is conducted in the constructed road map using the trajectory data collected from a probe vehicle. The results show that the proposed method provides an efficient way of extracting lane-level information from urban road networks and can be applied for lane-level map matching with good performance.

Development of a semantic map for an unmanned vehicle using a simultaneous localisation and mapping method

Introduction: The field of unmanned technologies is rapidly developing and a lot of research is being conducted on the practical application of artificial intelligence algorithms to solve complex problems on the road. The difficulties in the perception of the surrounding world by the machine led to the appearance of special High definition maps. These maps are used to simplify and improve the quality and reliability of other subsystems from the stack of autonomous technologies, such as localization, prediction, navigation and planning modules. In modern literature, there are mainly works on the practical application of such maps, and the process of developing a map remains outside the scope of consideration.The aim of the work is to create a methodology for designing semantic maps for autonomous vehicles with a detailed description of each of the development stages.Materials and methods: The article describes the methodology for creation of HD maps, which includes the stages of data collection using SLAM (Simultaneous localization and mapping) approach, its further processing and the development of the semantics of the road network. The described algorithm is applied in practice to develop the semantic map of Innopolis city area using SLAM approach with LIDAR inertial odometry via smoothing and mapping (LIO-SAM).Results: The main stages of the methodology for creating HD maps for autonomous vehicles have been proposed and investigated. Authors implemented the proposed concept in practice and described in detail the process of creating a semantic map for the Innopolis city area.Conclusions: The proposed methodology can be used for any type of autonomous robots (ground vehicles, unmanned aerial vehicle, water transport) and can be implemented in different road conditions (city, off-road), depending on the information the map should provide for the implementation of the goals and objectives set for the autonomous vehicle.