Localization Of Autonomous Vehicles Research Articles

New Monte Carlo Localization Using Deep Initialization: A Three-Dimensional LiDAR and a Camera Fusion Approach

Fast and accurate global localization of autonomous ground vehicles is often required in indoor environments and GPS-shaded areas. Typically, with regard to global localization problem, the entire environment should be observed for a long time to converge. To overcome this limitation, a new initialization method called deep initialization is proposed and it is applied to Monte Carlo localization (MCL). The proposed method is based on the combination of a three-dimensional (3D) light detection and ranging (LiDAR) and a camera. Using a camera, pose regression based on a deep convolutional neural network (CNN) is conducted to initialize particles of MCL. Particles are sampled from the tangent space to a manifold structure of the group of rigid motion. Using a 3D LiDAR as a sensor, a particle filter is applied to estimate the sensor pose. Furthermore, we propose a re-localization method for performing initialization whenever a localization failure or the situation of robot kidnapping is detected. Either the localization failure or the kidnapping is detected by combining the outputs from a camera and 3D LiDAR. Finally, the proposed method is applied to a mobile robot platform to prove the method's effectiveness in terms of both the localization accuracy and time consumed for estimating the pose correctly.

A Novel Online Approach for Drift Covariance Estimation of Odometries Used in Intelligent Vehicle Localization.

Localization is the fundamental problem of intelligent vehicles. For a vehicle to autonomously operate, it first needs to locate itself in the environment. A lot of different odometries (visual, inertial, wheel encoders) have been introduced through the past few years for autonomous vehicle localization. However, such odometries suffers from drift due to their reliance on integration of sensor measurements. In this paper, the drift error in an odometry is modeled and a Drift Covariance Estimation (DCE) algorithm is introduced. The DCE algorithm estimates the covariance of an odometry using the readings of another on-board sensor which does not suffer from drift. To validate the proposed algorithm, several real-world experiments in different conditions as well as sequences from Oxford RobotCar Dataset and EU long-term driving dataset are used. The effect of the covariance estimation on three different fusion-based localization algorithms (EKF, UKF and EH-infinity) is studied in comparison with the use of constant covariance, which were calculated based on the true variance of the sensors being used. The obtained results show the efficacy of the estimation algorithm compared to constant covariances in terms of improving the accuracy of localization.

ACK-MSCKF: Tightly-Coupled Ackermann Multi-State Constraint Kalman Filter for Autonomous Vehicle Localization.

Visual-Inertial Odometry (VIO) is subjected to additional unobservable directions under the special motions of ground vehicles, resulting in larger pose estimation errors. To address this problem, a tightly-coupled Ackermann visual-inertial odometry (ACK-MSCKF) is proposed to fuse Ackermann error state measurements and the Stereo Multi-State Constraint Kalman Filter (S-MSCKF) with a tightly-coupled filter-based mechanism. In contrast with S-MSCKF, in which the inertial measurement unit (IMU) propagates the vehicle motion and then the propagation is corrected by stereo visual measurements, we successively update the propagation with Ackermann error state measurements and visual measurements after the process model and state augmentation. This way, additional constraints from the Ackermann measurements are exploited to improve the pose estimation accuracy. Both qualitative and quantitative experimental results evaluated under real-world datasets from an Ackermann steering vehicle lead to the following demonstration: ACK-MSCKF can significantly improve the pose estimation accuracy of S-MSCKF under the special motions of autonomous vehicles, and keep accurate and robust pose estimation available under different vehicle driving cycles and environmental conditions. This paper accompanies the source code for the robotics community.

Practical Modeling of GNSS for Autonomous Vehicles in Urban Environments.

Autonomous navigation technology is used in various applications, such as agricultural robots and autonomous vehicles. The key technology for autonomous navigation is ego-motion estimation, which uses various sensors. Wheel encoders and global navigation satellite systems (GNSSs) are widely used in localization for autonomous vehicles, and there are a few quantitative strategies for handling the information obtained through their sensors. In many cases, the modeling of uncertainty and sensor fusion depends on the experience of the researchers. In this study, we address the problem of quantitatively modeling uncertainty in the accumulated GNSS and in wheel encoder data accumulated in anonymous urban environments, collected using vehicles. We also address the problem of utilizing that data in ego-motion estimation. There are seven factors that determine the magnitude of the uncertainty of a GNSS sensor. Because it is impossible to measure each of these factors, in this study, the uncertainty of the GNSS sensor is expressed through three variables, and the exact uncertainty is calculated. Using the proposed method, the uncertainty of the sensor is quantitatively modeled and robust localization is performed in a real environment. The approach is validated through experiments in urban environments.

3D 라이다 센서와 도로 환경 지도의 비교를 통한 자율주행 자동차 위치인식 기법 검증

This paper describes our study of localization for autonomous vehicles based on a particle filter with road feature maps and lidar sensor inputs. We propose this system for vehicle localization in tunnels and underground roads when satellite signals from the US GPS, the Russian GLONASS, and the Japanese QZSS are not available. We studied sensor-based localization methods for autonomous vehicles using radar, lidar, and cameras. In this paper, we describe a localization method using a particle filter that correlates a high-resolution map with a local map constructed by 3D lidar and vehicle odometry. We validate the developed method with a real-time computing environment using actual measurements obtained from public road experiments.

자율주행자동차의 실시간 고정밀 위치추정을 위한 파티클필터와 확장칼만필터 융합 위치추정 알고리즘 개발

자율주행자동차의 실시간 고정밀 위치추정을 위한 파티클필터와 확장칼만필터 융합 위치추정 알고리즘 개발

Estimating Autonomous Vehicle Localization Error Using 2D Geographic Information

Accurately and precisely knowing the location of the vehicle is a critical requirement for safe and successful autonomous driving. Recent studies suggest that error for map-based localization methods are tightly coupled with the surrounding environment. Considering this relationship, it is therefore possible to estimate localization error by quantifying the representation and layout of real-world phenomena. To date, existing work on estimating localization error have been limited to using self-collected 3D point cloud maps. This paper investigates the use of pre-existing 2D geographic information datasets as a proxy to estimate autonomous vehicle localization error. Seven map evaluation factors were defined for 2D geographic information in a vector format, and random forest regression was used to estimate localization error for five experiment paths in Shinjuku, Tokyo. In the best model, the results show that it is possible to estimate autonomous vehicle localization error with 69.8% of predictions within 2.5 cm and 87.4% within 5 cm.

Neural Network Based Uncertainty Prediction for Autonomous Vehicle Application

This paper proposes a framework for uncertainty prediction in complex fusion networks, where signals become available sporadically. Assuming there is no information of the sensor characteristics available, a surrogated model of the sensor uncertainty is yielded directly from data through artificial neural networks. The strategy developed is applied to autonomous vehicle localization through odometry sensors (speed and orientation), so as to determine the location uncertainty in the trajectory. The results obtained allow for fusion of autonomous vehicle location measurements, and effective correction of the accumulated odometry error in most scenarios. The neural networks applicability and generalization capacity are proven, evidencing the suitability of the presented methodology for uncertainty estimation in non-linear and intractable processes.

Artificial Landmarks for Trusted Localization of Autonomous Vehicles Based on Magnetic Sensors.

Magnetic sensors provide an advantageous alternative localization method, primarily focusing on localization in surroundings where GPS, other radio frequency-based, as well as optical localization do not work or has severe limitations. Suitable for distances in the meter range, such magnetic localization may in particular be useful as artificial landmarks, e.g., for automatic drift correction. To easily use such artificial landmarks, we propose an integration process based on Transducer Electronic Data Sheets. With this approach, the landmarks can be used by passing autonomous vehicles, e.g., UAVs, for re-orientation and re-calibration. During this process, all necessary information such as data formats, reference coordinates, calibration data, provider of the landmark etc. is made known to the vehicle passing by. Based on the provided so-called meta-information, the vehicle itself can decide whether and how to use the provided sensory information. To provide a certain level of trust in the landmarks and their provided information, the corresponding data sheets are certified using a digital signature.

A Particle Filter Localization Method Using 2D Laser Sensor Measurements and Road Features for Autonomous Vehicle

This paper presents a method of particle filter localization for autonomous vehicles, based on two-dimensional (2D) laser sensor measurements and road features. To navigate an urban environment, an autonomous vehicle should be able to estimate its location with a reasonable accuracy. By detecting road features such as curbs and road markings, a grid-based feature map is constructed using 2D laser range finder measurements. Then, a particle filter is employed to accurately estimate the position of the autonomous vehicle. Finally, the performance of the proposed method is verified and compared to accurate Differential Global Positioning Systems (DGPS) data through real road driving experiments.

Visual Semantic Landmark-Based Robust Mapping and Localization for Autonomous Indoor Parking.

Autonomous parking in an indoor parking lot without human intervention is one of the most demanded and challenging tasks of autonomous driving systems. The key to this task is precise real-time indoor localization. However, state-of-the-art low-level visual feature-based simultaneous localization and mapping systems (VSLAM) suffer in monotonous or texture-less scenes and under poor illumination or dynamic conditions. Additionally, low-level feature-based mapping results are hard for human beings to use directly. In this paper, we propose a semantic landmark-based robust VSLAM for real-time localization of autonomous vehicles in indoor parking lots. The parking slots are extracted as meaningful landmarks and enriched with confidence levels. We then propose a robust optimization framework to solve the aliasing problem of semantic landmarks by dynamically eliminating suboptimal constraints in the pose graph and correcting erroneous parking slots associations. As a result, a semantic map of the parking lot, which can be used by both autonomous driving systems and human beings, is established automatically and robustly. We evaluated the real-time localization performance using multiple autonomous vehicles, and an repeatability of 0.3 m track tracing was achieved at a 10 kph of autonomous driving.

Autonomous Vehicle Control based on HoloLens Technology and Raspberry Pi Platform: an Educational Perspective

Localization and mapping is an essential part of an autonomous vehicle control. Microsoft HoloLens is a sophisticated augmented reality device using many advanced technologies. As such HoloLens has high appeal for young people and its use in teaching may increase the attractiveness of control classes. In this paper we describe how this device can be used in advanced control education for experiments with autonomous vehicles localization, mapping and control. Among other capabilities HoloLens can provide 3D position information by its spatial mapping feature. This facilitates the simultaneous localization and mapping (SLAM) problem. Furthermore, based on HoloLens position information the students can design vehicle longitudinal and lateral control systems and compare the performance achieved with different strategies such as PID and Model Predictive control. Example comparisons are given in this paper.

Iterative parameter identification method of a vehicle odometry model

The paper proposes an off-line iterative estimation algorithm for wheel circumference estimation of autonomous vehicles. The motivation of the paper is that the signals of the GPS are not available in parking garages or in several urban areas, e.g. next to the buildings with high walls. Therefore, in these situations the wheel encoder based odometry can be an appropriate choice for autonomous vehicle localization, which requires the precise estimation of the wheel circumference. The proposed novel method has three layers, in which the Kalman-filtering with an enhanced tuning method and the least squares algorithm are performed in an iterative loop. Since the off-line methods uses all of the measurements at once, a highly accurate estimation with low sensitivity on the noise can be reached. The efficiency of the algorithm is presented through CarMaker simulations.

Байесовская вероятностная локализация автономного транспортного средства путем ассимиляции сенсорных данных и информации о дорожных знаках

Локализация транспортного средства является важной задачей в области интеллектуальных транспортных систем. Хорошо известно, что слияние показаний с разных датчиков (англ. Sensor Fusion) позволяет создавать более робастные и точные навигационные системы для автономных транспортных средств. Стандартные подходы, такие как расширенный фильтр Калмана или многочастичный фильтр, либо неэффективны при работе с сильно нелинейными данными, либо потребляют значительные вычислительные ресурсы, что осложняет их использование во встроенных системах. При этом точность сливаемых сенсоров может сильно различаться. Значительный прирост точности, особенно в ситуации, когда GPS (англ. Global Positioning System) не доступен, может дать использование ориентиров, положение которых заранее известно, — таких как дорожные знаки, светофоры, или признаки SLAM (англ. Simultaneous Localization and Mapping). Однако такой подход может быть неприменим в случае, если априорные локации неизвестны или неточны. Мы предлагаем новый подход для уточнения координат транспортного средства с использованием визуальных ориентиров, таких как дорожные знаки. Наша система представляет собой байесовский фреймворк, уточняющий позицию автомобиля с использованием внешних данных о прошлых наблюдениях дорожных знаков, собранных методом краудсорсинга (англ. Crowdsourcing — сбор данных широким кругом лиц). Данная статья представляет также подход к комбинированию траекторий, полученных с помощью глобальных GPS-координат и локальных координат, полученных с помощью акселерометра и гироскопа (англ. Inertial Measurement Unit, IMU), для создания траектории движения транспортного средства в неизвестной среде. Дополнительно мы собрали новый набор данных, включающий в себя 4 проезда на автомобиле в городской среде по одному маршруту, при которых записывались данные GPS и IMU смартфона, видеопоток с камеры, установленной на лобовом стекле, а также высокоточные данные о положении с использованием специализированного устройства Real Time Kinematic Global Navigation Satellite System (RTK-GNSS), которые могут быть использованы для валидации. Помимо этого, с использованием той же системы RTK-GNSS были записаны точные координаты знаков, присутствующих на маршруте. Результаты экспериментов показывают, что байесовский подход позволяет корректировать траекторию движения транспортного средства и дает более точные оценки при увеличении количества известной заранее информации. Предложенный метод эффективен и требует для своей работы, кроме показаний GPS/IMU, только информацию о положении автомобилей в моменты прошлых наблюдений дорожных знаков.

Autonomous Ground Vehicle Localization Filter Design Using Landmarks with Non-Unique Features

Autonomous Ground Vehicle Localization Filter Design Using Landmarks with Non-Unique Features

SOFT‐SLAM: Computationally efficient stereo visual simultaneous localization and mapping for autonomous unmanned aerial vehicles

AbstractAutonomous navigation of unmanned aerial vehicles (UAVs) in GPS‐denied environments is a challenging problem, especially for small‐scale UAVs characterized by a small payload and limited battery autonomy. A possible solution to the aforementioned problem is vision‐based simultaneous localization and mapping (SLAM), since cameras, due to their dimensions, low weight, availability, and large information bandwidth, circumvent all the constraints of UAVs. In this paper, we propose a stereo vision SLAM yielding very accurate localization and a dense map of the environment developed with the aim to compete in the European Robotics Challenges (EuRoC) targeting airborne inspection of industrial facilities with small‐scale UAVs. The proposed approach consists of a novel stereo odometry algorithm relying on feature tracking (SOFT), which currently ranks first among all stereo methods on the KITTI dataset. Relying on SOFT for pose estimation, we build a feature‐based pose graph SLAM solution, which we dub SOFT‐SLAM. SOFT‐SLAM has a completely separate odometry and mapping threads supporting large loop‐closing and global consistency. It also achieves a constant‐time execution rate of 20 Hz with deterministic results using only two threads of an onboard computer used in the challenge. The UAV running our SLAM algorithm obtained the highest localization score in the EuRoC Challenge 3, Stage IIa–Benchmarking, Task 2. Furthermore, we also present an exhaustive evaluation of SOFT‐SLAM on two popular public datasets, and we compare it to other state‐of‐the‐art approaches, namely ORB‐SLAM2 and LSD‐SLAM. The results show that SOFT‐SLAM obtains better localization accuracy on the majority of datasets sequences, while also having a lower runtime.

Random Set Approach to Distributed Multivehicle SLAM

This paper deals with the simultaneous localization and mapping (SLAM) problem for autonomous vehicles or mobile robots. More specifically, a multi-vehicle scenario is considered wherein a team of vehicles explore the scene of interest in order to cooperatively construct the map of the environment by locally updating and exchanging map information in a neighbor-wise fashion. A random-set approach is undertaken by regarding the map as a random finite set (RFS) and updating the first-order moment, called probability hypothesis density (PHD), of its multi-object density. Consensus on PHDs is adopted in order to spread the map information through the team of vehicles also taking into account the different and time-varying fields of view (FoVs) of the team members. The developed algorithm represents - to the best of the authors’ knowledge - the first attempt to solve in a fully decentralized way the multi-vehicle SLAM problem within the RFS framework. The effectiveness of the proposed approach is tested by means of simulation experiments.

Efficient ConvNet Feature Extraction with Multiple RoI Pooling for Landmark-Based Visual Localization of Autonomous Vehicles

Efficient and robust visual localization is important for autonomous vehicles. By achieving impressive localization accuracy under conditions of significant changes, ConvNet landmark-based approach has attracted the attention of people in several research communities including autonomous vehicles. Such an approach relies heavily on the outstanding discrimination power of ConvNet features to match detected landmarks between images. However, a major challenge of this approach is how to extract discriminative ConvNet features efficiently. To address this challenging, inspired by the high efficiency of the region of interest (RoI) pooling layer, we propose a Multiple RoI (MRoI) pooling technique, an enhancement of RoI, and a simple yet efficient ConvNet feature extraction method. Our idea is to leverage MRoI pooling to exploit multilevel and multiresolution information from multiple convolutional layers and then fuse them to improve the discrimination capacity of the final ConvNet features. The main advantages of our method are (a) high computational efficiency for real-time applications; (b) GPU memory efficiency for mobile applications; and (c) use of pretrained model without fine-tuning or retraining for easy implementation. Experimental results on four datasets have demonstrated not only the above advantages but also the high discriminating power of the extracted ConvNet features with state-of-the-art localization accuracy.

1 year, 1000 km: The Oxford RobotCar dataset

We present a challenging new dataset for autonomous driving: the Oxford RobotCar Dataset. Over the period of May 2014 to December 2015 we traversed a route through central Oxford twice a week on average using the Oxford RobotCar platform, an autonomous Nissan LEAF. This resulted in over 1000 km of recorded driving with almost 20 million images collected from 6 cameras mounted to the vehicle, along with LIDAR, GPS and INS ground truth. Data was collected in all weather conditions, including heavy rain, night, direct sunlight and snow. Road and building works over the period of a year significantly changed sections of the route from the beginning to the end of data collection. By frequently traversing the same route over the period of a year we enable research investigating long-term localization and mapping for autonomous vehicles in real-world, dynamic urban environments. The full dataset is available for download at: http://robotcar-dataset.robots.ox.ac.uk

Hierarchical Neighborhood Based Precise Localization for Intelligent Vehicles in Urban Environments

High-precision localization has drawn more and more attention in recent research of intelligent vehicle systems and autonomous robot navigation technology. In most methods, the approaches are only effective in some specific situations. In other words, these methods can only perform well with obvious features, like tall building walls, road curbs, etc . In this paper, a novel framework for precise localization of autonomous vehicle applying to different scenes especially some typical urban environments is proposed. The main procedures of this method include mapping and localization. During mapping process, inertial measurement unit, odometry, and high-precision GPS are fused together with the data sensed by LIDAR, a high-precision map that could provide global position and pose is generated using rolling window. When localizing, live laser data align with the prior-map. A particle filter based point cloud matching method is utilized here. Based on this, a hierarchical localizing method is proposed, which is more accurate and faster than the original matching method. With this method, the sampling guided by proposal density is propagated upward every hierarchy. Besides that, some accelerating algorithms are utilized to make this approach real time. Finally, decimeter-level localization is achieved in different environments, which is proven by some experiments.