Global Localization Problem Research Articles

An experimental evaluation of Siamese Neural Networks for robot localization using omnidirectional imaging in indoor environments

The objective of this paper is to address the localization problem using omnidirectional images captured by a catadioptric vision system mounted on the robot. For this purpose, we explore the potential of Siamese Neural Networks for modeling indoor environments using panoramic images as the unique source of information. Siamese Neural Networks are characterized by their ability to generate a similarity function between two input data, in this case, between two panoramic images. In this study, Siamese Neural Networks composed of two Convolutional Neural Networks (CNNs) are used. The output of each CNN is a descriptor which is used to characterize each image. The dissimilarity of the images is computed by measuring the distance between these descriptors. This fact makes Siamese Neural Networks particularly suitable to perform image retrieval tasks. First, we evaluate an initial task strongly related to localization that consists in detecting whether two images have been captured in the same or in different rooms. Next, we assess Siamese Neural Networks in the context of a global localization problem. The results outperform previous techniques for solving the localization task using the COLD-Freiburg dataset, in a variety of lighting conditions, specially when using images captured in cloudy and night conditions.

Highly robust and accurate multi-sensor fusion localization system for complex and challenging scenarios

Highly robust and accurate localization using on-board sensors in the absence of GNSS is a key issue for long-term autonomous navigation of vehicles. However, the limited sensing capabilities of a single sensor cannot resist the effects of feature degradation and high-speed motion in complex and challenging scenarios, which results in the fragile of existing LiDAR-based and visual-based localization algorithms. To solve these problems, we propose a highly robust and accurate localization system by fusing information from multi-sensors. Firstly, we propose a global localization method that leverages deep learning models for processing visual information to provide an accurate initial position for vehicles, which solves the problem of global localization of vehicles in the absence of GNSS. Then, we propose a highly robust and accurate LiDAR-Vision-IMU localization method based on pose graph optimization, which integrates LiDAR-based poses and visual-inertial-based poses. The proposed method detects sensor degradation in real time to dynamically adjust the fusion weights, which solves the problem of low localization performance of a single sensor. Finally, we conduct experiments in complex and challenging scenarios, such as urban, tunnels, and unstructured mountains. Experimental results show that our method can achieve a global localization error of less than 0.5 m in complex scenes. In unstructured mountainous scenarios of length 21.1 km, the root mean square errors (RMSE) of translation and rotation are 0.67 m and 0.048rad, respectively. Compared to most existing algorithms, our method achieves the most robust and accurate localization performance.

Localization and tracking of stationary users for augmented reality

In augmented reality applications it is essential to know the position and orientation of the user to correctly register virtual 3D content in the user’s field of view. For this purpose, visual tracking through simultaneous localization and mapping (SLAM) is often used. However, when applied to the commonly occurring situation where the users are mostly stationary, many methods presented in previous research have two key limitations. First, SLAM techniques alone do not address the problem of global localization with respect to prior models of the environment. Global localization is essential in many applications where multiple users are expected to track within a shared space, such as spectators at a sporting event. Secondly, these methods often assume significant translational movement to accurately reconstruct and track from a local model of the environment, causing challenges for many stationary applications. In this paper, we extend recent research on Spherical Localization and Tracking to support relocalization after tracking failure, as well as global localization in large shared environments, and optimize the method for operation on mobile hardware. We also evaluate various state-of-the-art localization approaches, the robustness of our visual tracking method, and demonstrate the effectiveness of our system in real-life scenarios.

Real-Time Scan-to-Map Matching Localization System Based on Lightweight Pre-Built Occupancy High-Definition Map

High-precision and robust localization in GNSS-denied areas is crucial for autonomous vehicles and robots. Most state-of-the-art localization methods are based on simultaneous localization and mapping (SLAM) with a camera or light detection and ranging (LiDAR). However, SLAM will suffer from drift during long-term running without loop closure or prior constraints. Lightweight, high-precision environmental maps have gradually become an indispensable part of future autonomous driving. In order to solve the problem of real-time global localization for autonomous vehicles and robots, we propose a precise and robust LiDAR localization system based on a pre-built, occupied high-definition (HD) map called the Extended QuadTree (EQT) map. It makes use of a planar quadtree for block division and a Z-sequence index structure within the block cells. Then, a four-level occupancy probability cell value model is adopted. It will save about eight times the storage space compared with Google Cartographer, and the EQT map can be extended to store other information. For efficient scan-to-map matching with our specialized EQT map, the Bursa linearized model is used in the Gauss–Newton iteration of our algorithm, which makes the calculation of partial derivatives fast. All the above improvements lead to optimal storage and efficient querying for real-time scan-to-map matching localization. Field tests in an industrial park and road environment prove that positioning accuracy of about 6–13 cm and attitude accuracy of about 0.15° were achieved using a VLP-16 LiDAR. They also show that the method proposed in this paper is significantly better than the NDT method. For the long and narrow environment of an underground mine tunnel, high-resolution maps are also helpful for accurate and robust localization.

LiDAR-Based Global Localization Using Histogram of Orientations of Principal Normals

Global localization on LiDAR point cloud maps is a challenging task because of the sparse nature of point clouds and the large size difference between LiDAR scans and the maps. In this paper, we solve the LiDAR-based global localization problem based upon the plane-motion assumption. We first project the clouds into Bird’s-eye View (BV) images and transform the problem into a BV image matching problem. We then introduce a novel local descriptor, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , Histogram of Orientations of Principal Normals (HOPN), to perform matching. The HOPN descriptor encodes the point normals of the clouds, and is more effective in matching BV images than the common image descriptors. In addition, we present the consensus set maximization algorithm to robustly estimate a rigid pose from the HOPN matches in the case of the low inlier ratio. The experimental results on three large-scale datasets show that our method achieves state-of-the-art global localization performance when using either single LiDAR scans or local maps.

Research on rapid location method of mobile robot based on semantic grid map in large scene similar environment

AbstractAiming at the problem that adaptive Monte Carlo localization (AMCL) algorithm is difficult to localize in large scenes and similar environments. This paper uses a semantic information-assisted approach to improve the AMCL algorithm. This method realizes the robust localization of the robot in the large scenes and similar environments. Firstly, the 2D grid map created by simultaneous localization and mapping using lidar can obtain highly accurate indoor environmental contour information. Secondly, the semantic object capture is achieved by using a depth camera combined with an instance segmentation algorithm. Then, the semantic grid map is created by mapping the semantic point cloud through the back-projection process of the pinhole camera. Finally, semantic grid maps are used as a priori information to assist in localization, which will be used to improve the initial particle swarm distribution in the global localization of the AMCL algorithm and thus will solve the robot localization problem in this environment. The experimental evidence shows that the semantic grid map solves the environmental information degradation problem caused by 2D lidar as well as improves the robot’s perception of the environment. In addition, this paper improves the localization robustness of the AMCL algorithm in large scenes and similar environments, resulting in an average localization success rate of about 90% or even higher, and further reduces the number of iterations. The global localization problem of robots in large scenes and similar environments is effectively solved.

Filtering Algorithms for Determining the Coordinates of the Object in Decision Support Systems

Methods for estimating the parameters and states of dynamical systems are an urgent task, the results of which are used in various fields, including processes in technical systems, cosmological and physical research, medical diagnostic systems, economics, finance, biotechnology, ecology and others. Despite significant scientific and practical advances in this area, researchers in many countries around the world continue to search for new methods of assessing the parameters and states of the studied objects and improving existing ones. An example of such methods is digital and optimal filtering, which have been widely used in technical systems since the middle of the last century, in particular, in the processing of financial and economic data, physical experiments and other information technologies for various purposes. The model and algorithms of granular filtering are considered on a practical example - a variant of the problem of global localization of mobile robot (global localization for mobile robots) or the problem of hijacked robot (hijacked robot problem). In the general embodiment, it is to determine the position of the robot according to the data from the sensor. This problem was generally solved by a number of probabilistic methods in the late 1990s and early 2000s. The task is important and finds application in mobile robotics and industry. The tasks of positioning submarines, aircraft, cars, etc. are essentially similar.

Infrastructure-Free Hierarchical Mobile Robot Global Localization in Repetitive Environments

Repetitive or ambiguous environment, where structures are highly similar and distinct geometric features are not sufficient, is one of the critical and challenging scenarios for mobile robots to perform global localization or simultaneous localization and mapping (SLAM) tasks. The robots are easy to get lost or mismatched to wrong places in such environments. The existing solutions either rely heavily on pre-installed infrastructures which are inflexible and expensive or rely on single sensor-based global localization methods whose initialization module is not capable enough to provide distinctive information. Thus, this article proposes a hierarchical probabilistic framework that addresses the problem of infrastructure-free mobile robot global localization in GPS-challenged repetitive environments by leveraging both the measured magnetic field (MF) and Light Detection and Ranging (LiDAR) information. The proposed hierarchical system mainly consists of: 1) coarse localization: MF-based localization and 2) fine localization: LiDAR-based localization (LBL). The ambient MF functions as a substitute for the GPS signal, which is considered not available or severely challenged in indoor/semi-indoor local environments. Based on the pre-built MF database, multiple coarse candidate robot poses can first be determined by the proposed initial pose estimation algorithm, which incorporates multivariate Gaussian observation model and random sample consensus (RANSAC) algorithm. Then, using the obtained multiple candidate poses as initialization, the robot can be localized more accurately by LiDAR-based fine localization. Extensive real-world experiments demonstrate that the proposed system achieves over 95% localization success rate and takes less than 2.0-m average traveling distance to localize the mobile robot.

LSTM and Filter Based Comparison Analysis for Indoor Global Localization in UAVs

Deep learning (DL) based localization and Simultaneous Localization and Mapping (SLAM) has recently gained considerable attention demonstrating remarkable results. Instead of constructing hand-crafted algorithms through geometric theories, DL based solutions provide a data-driven solution to the problem. Taking advantage of large amounts of training data and computing capacity, these approaches are increasingly developing into a new field that offers accurate and robust localization systems. In this work, the problem of global localization for unmanned aerial vehicles (UAVs) is analyzed by proposing a sequential, end-to-end, and multimodal deep neural network based monocular visual-inertial localization framework. More specifically, the proposed neural network architecture is three-fold; a visual feature extractor convNet network, a small IMU integrator bi-directional long short-term memory (LSTM), and a global pose regressor bi-directional LSTM network for pose estimation. In addition, by fusing the traditional IMU filtering methods instead of LSTM with the convNet, a more time-efficient deep pose estimation framework is presented. It is worth pointing out that the focus in this study is to evaluate the precision and efficiency of visual-inertial (VI) based localization approaches concerning indoor scenarios. The proposed deep global localization is compared with the various state-of-the-art algorithms on indoor UAV datasets, simulation environments and real-world drone experiments in terms of accuracy and time-efficiency. In addition, the comparison of IMU-LSTM and IMU-Filter based pose estimators is also provided by a detailed analysis. Experimental results show that the proposed filter-based approach combined with a DL approach has promising performance in terms of accuracy and time efficiency in indoor localization of UAVs.

Модель та алгоритми гранулярної фільтрації у задачі глобальної локалізації мобільного робота

Методи оцінювання параметрів і станів динамічних систем – актуальна задача, результати розв’язання якої знаходять своє застосування у різних галузях діяльності, включаючи дослідження процесів у технічних системах, космологічних та фізичних дослідженнях, медичних діагностичних системах, економіці, фінансах, біотехнологіях, екології та інших. Незважаючи на значні наукові і практичні досягнення у цьому напрямі, дослідники багатьох країн світу продовжують пошуки нових методів оцінювання параметрів і станів досліджуваних об’єктів та удосконалення існуючих. Прикладом таких методів є цифрова та оптимальна фільтрація, які знайшли широке застосування у технічних системах ще у середині минулого століття, зокрема, у обробці фінансово-економічних даних, фізичних експериментах та інших інформаційних технологіях самого різного призначення.&#x0D; Розглядається модель та алгоритми гранулярної фільтрації на практичному прикладі – варіанті задачі глобальної локалізації мобільного робота (global localization for mobile robots) або задачі про викраденого робота (hijacked robot problem). В загальному варіанті вона полягає у визначенні положення робота за даними з сенсора. Ця задача була в цілому розв’язана рядом імовірнісних методів в кінці 90-х-початку 2000-х років. Задача є важливою і знаходить застосування у мобільній робототехніці та промисловості. Схожими за суттю є задачі позиціонування підводних човнів, літальних апаратів, автомобілів тощо. Також розглядається задача позиціонування робота. Нехай у темному лабіринті увімкнувся робот. Він має карту лабіринту та компас. У лабіринті в деяких точках встановлені позначені на карті станції, які можуть приймати і відбивати сигнал. Робот не знає, в якому місці лабіринту він знаходиться, але він може в кожний момент часу відправляти сигнал і з деякою похибкою дізнаватись відстань до найближчої до нього станції. Робот починає блукати лабіринтом, роблячи кожний крок у новому випадково обраному напрямку, але його компас також дає деяку несистематичну похибку. На кожному кроці робот визначає відстань до найближчої станції. Мета – з’ясувати координати робота у лабіринті в системі відліку, введеній на карті.

Improved Global Localization and Resampling Techniques for Monte Carlo Localization Algorithm

Global indoor localization algorithms enable the robot to estimate its pose in pre-mapped environments using sensor measurements when its initial pose is unknown. The conventional Adaptive Monte Carlo Localization (AMCL) is a highly efficient localization algorithm that can successfully cope with global uncertainty. Since the global localization problem is paramount in mobile robots, we propose a novel approach that can significantly reduce the amount of time it takes for the algorithm to converge to true pose. Given the map and initial scan data, the proposed algorithm detects regions with high likelihood based on the observation model. As a result, the suggested sample distribution will expedite the process of localization. In this study, we also present an effective resampling strategy to deal with the kidnapped robot problem that enables the robot to recover quickly when the sample weights drop-down due to unmapped dynamic obstacles within the sensor’s field of view. The proposed approach distributes the random samples within a circular region centered around the robot’s pose by taking into account the prior knowledge about the most recent successful pose estimation. Since the samples are distributed over the region with high probabilities, it will take less time for the samples to converge to the actual pose. The percentage of improvement for the small sample set (500 samples) exceeded 90% over the large maps and played a big role in reducing computational resources. In general, the results demonstrate the localization efficacy of the proposed scheme, even with small sample sets. Consequently, the proposed scheme significantly increases the real-time performance of the algorithm by 85.12% on average in terms of decreasing the computational cost.

New Monte Carlo Localization Using Deep Initialization: A Three-Dimensional LiDAR and a Camera Fusion Approach

Fast and accurate global localization of autonomous ground vehicles is often required in indoor environments and GPS-shaded areas. Typically, with regard to global localization problem, the entire environment should be observed for a long time to converge. To overcome this limitation, a new initialization method called deep initialization is proposed and it is applied to Monte Carlo localization (MCL). The proposed method is based on the combination of a three-dimensional (3D) light detection and ranging (LiDAR) and a camera. Using a camera, pose regression based on a deep convolutional neural network (CNN) is conducted to initialize particles of MCL. Particles are sampled from the tangent space to a manifold structure of the group of rigid motion. Using a 3D LiDAR as a sensor, a particle filter is applied to estimate the sensor pose. Furthermore, we propose a re-localization method for performing initialization whenever a localization failure or the situation of robot kidnapping is detected. Either the localization failure or the kidnapping is detected by combining the outputs from a camera and 3D LiDAR. Finally, the proposed method is applied to a mobile robot platform to prove the method's effectiveness in terms of both the localization accuracy and time consumed for estimating the pose correctly.

On the Problem of Global Localization of Discontinuity Lines for a Function of Two Variables

We consider the ill-posed problem of localizing (finding the position of) the discontinuity lines of a function of two variables that is smooth outside the discontinuity lines and has a discontinuity of the first kind at each point of such lines. A uniform square grid with step τ is considered, and it is assumed that the mean values of a perturbed function over squares with side τ are known at each node of the grid. The perturbed function approximates the exact function in the space L2(ℝ2). The perturbation level δ is known. To solve the problem under consideration, we design and study global discrete algorithms that are based on averaging procedures and approximate the discontinuity lines by a set of points of a uniform grid. The main result of the paper is the development of an approach to the global study of localization algorithms. We formulate conditions for the exact function, thus defining a class of correctness. Within this class, we perform a theoretical study of the proposed algorithms, introduce the characteristics to be estimated (the concept of approximating a set of discontinuity lines by a set of points of a uniform grid), and develop methods for deriving the estimates. To achieve this goal, we use a simplified statement: the discontinuity lines are straight line segments, and the proposed localization algorithm has the simplest thinning block. It is established that the localization error of the algorithm has order O(δ). Estimates of other important parameters characterizing the localization algorithm are given.

A Novel Global Localization Approach Based on Structural Unit Encoding and Multiple Hypothesis Tracking

In this paper, we present a novel 2-D laser-based global localization approach for mobile robots, which is composed of geometrical relationship construction, a new structural unit encoding scheme (SUES), and an extended multiple hypothesis tracking (MHT) algorithm. Different from existing methods, we construct a 3-D “directional endpoint” feature encapsulating both the endpoint and the direction of a line segment; on this basis, a novel and efficient online structural unit encoding scheme (SUES) is proposed to describe the geometric relationship between the two directional endpoint features with some robustness to dynamic disturbances. Note that SUES presented in this paper is different from the bag-of-words scheme in two aspects: 1) SUES quantizes the geometrical relationship without offline training for vocabulary and 2) SUES is independent of the quality of the vocabulary. By factoring the global localization problem into a discrete pose estimation problem, the MHT is extended on the basis of SUES and odometry information to gradually restore the global robot pose. Different from the classical MHT framework, the extended MHT takes the independent observation results as the a priori , while the likelihood term is composed of consecutive candidate poses and the odometry information. Evaluations are carried out by using both publicly available data sets and self-recorded data sets. Comparative experimental results with respect to the adaptive Monte Carlo localization are presented to show the superior performance of the proposed approach in terms of success ratio and efficiency.

A Study of Sensor-Fusion Mechanism for Mobile Robot Global Localization

SummaryEstimating the robot state within a known map is an essential problem for mobile robot; it is also referred to “localization”. Even LiDAR-based localization is practical in many applications, it is difficult to achieve global localization with LiDAR only for its low-dimension feedback, especially in environments with repetitive geometric features. A sensor-fusion-based localization system is introduced in this paper, which has the capability of addressing the global localization problem. Both LiDAR and vision sensors are integrated, making use of the rich information introduced by vision sensor and the robustness from LiDAR. A hybrid grid-map is built for global localization, and a visual global descriptor is applied to speed up the localization convergence, combined with a pose refining pipeline for improving the localization accuracy. Also, a trigger mechanism is introduced to solve kidnapped problem and verify the relocalization result. The experiments under different conditions are designed to evaluate the performance of the proposed approach, as well as a comparison with the existing localization systems. According to the experimental results, our system is able to solve the global localization problem, and the sensor-fusion mechanism in our system has an improved performance.

Mobile Robot Localization Based on the Novel Leader-Based Bat Algorithm

This work proposes a new approach to the well-known method bat algorithm for solving the mobile robots global localization problem. The proposed method is leader-based bat algorithm (LBBA). The LBBA uses a small number of better micro-bats as leaders to influence the colony in the search for the best position, dealing satisfactorily with ambiguities during the localization process. The tests covered different scenarios aiming at comparing the proposed algorithm with other methods, such as the standard BA, the particle swarm optimization and particle filter. The results outperformed the compared methods, presenting a fast response and errors below the intended tolerance. The algorithm was tested in the robot kidnapping scenario and shows fast recovery in both simulation and in a real environment. In addition, the proposed technique showed 21% lower average error when compared with an algorithm that presents a variable quantity of particles, i.e. the adaptive Monte Carlo localization algorithm.

Seamless navigation and mapping using an INS/GNSS/grid-based SLAM semi-tightly coupled integration scheme

Mobile Mapping Systems (MMS) with Inertial Navigation System / Global Navigation Satellite System (INS/GNSS) and mapping sensors have been widely developed in recent years. However current systems and results are still prone to errors, especially in GNSS-denied or multipath environments. To provide robust and stable navigation information, particularly for mapping in long-term GNSS-denied environments, we propose a semi-tightly coupled integration scheme which integrates INS/GNSS with grid-based Simultaneous Localization and Mapping (SLAM). Although traditional SLAM using LiDAR can map the GNSS-denied environment efficiently, it is only in local localization. The proposed integration scheme is based on the Extended Kalman Filter (EKF) with motion constraints. In this scheme, a measurement model for grid-based SLAM is aided by the heading and velocity information. A special innovation of this scheme is the improved fusion of GNSS/INS with the use of grid-based SLAM serves like virtual odometer and virtual compass, thus gaining reliable measurements and error models to maintain good performance during INS-only mode. In addition, the initial values for example position and heading, are given to solve global localization and loop closure problems in SLAM. Finally, a smoothing and multi-resolution map strategy are applied offline to increase the robustness and performance of the proposed grid-based SLAM. Evaluation based on experimental data shows the significant improvement by the proposed semi-tightly coupled integration scheme with low-cost INS/GNSS and LiDAR, which is able to achieve 1–2 m’ accuracy in terms of positioning and mapping. An approximately 60% improvement was achieved during long-term GNSS-denied environments using the proposed integration scheme.

PlaneLoc: Probabilistic global localization in 3-D using local planar features

The global localization problem concerns situations when a map of the environment is known but there is no initial guess of the agent position. Whereas the ability to perform global localization is required in many practical situations, it is still an open problem, particularly if the agent requires to find an accurate estimate of its 3-D pose. In this article, we describe PlaneLoc, a novel probabilistic approach to 3-D global localization, which integrates multiple local cues to construct a probability distribution that describes the likelihood of the agent pose. This framework enables to incorporate various types of localization cues but we demonstrate its feasibility using segmented planes abstracted from RGB-D data. We use multiple triplets of planar segments to generate candidate probability distribution and employ it to find the most probable pose with respect to a global map of planar segments. The PlaneLoc implementation uses the ORB-SLAM2 system that serves as visual odometry and makes it possible to generate observation in a form of sets of local segments online. The proposed approach can be used for global localization with a known map or for loop closing and re-localization in Simultaneous Localization and Mapping. The implemented system is validated in experiments using publicly available RGB-D data sets, including our own data set acquired specifically for testing localization methods based on planar features.

Methods for the construction of image descriptors for the global visual localization problem

Methods for the construction of image descriptors for the global visual localization problem

Using the Jensen-Shannon, Density Power, and Itakura-Saito Divergences to Implement an Evolutionary-Based Global Localization Filter for Mobile Robots

One of the most demanding skills for a mobile robot is to be intelligent enough to know its own location. The global localization problem consists of obtaining the robot’s pose (position and orientation) in a known map if the initial location is unknown. This task is addressed applying evolutionary computation concepts (Differential Evolution). In the current approach, the distances obtained from the laser sensors are combined with the predicted scan (in the known map) from possible locations to implement a cost function that is optimized by an evolutionary filter. The laser beams (sensor information) are modeled using a combination of probability distributions to implement a non-symmetric fitness function. The main contribution of this paper is to apply the probabilistic approach to design three different cost functions based on known divergences (Jensen–Shannon, Itakura–Saito, and density power). The three metrics have been tested in different experiments and the localization module performance is exceptional in regions with occlusions caused by different obstacles. This fact validates that the non-symmetric probabilistic approach is a suitable technique to be applied to multiple metrics.