Unmanned Aerial Vehicles Localization Research Articles

A semantic knowledge database-based localization method for UAV inspection in perceptual-degraded underground mine

Abstract In recent years, unmanned aerial vehicles (UAVs) have been applied in underground mine inspection and other similar works depending on their versatility and mobility. However, accurate localization of UAVs in perceptually degraded mines is full of challenges due to the harsh light conditions and similar roadway structures. Due to the unique characteristics of the underground mines, this paper proposes a semantic knowledge database-based localization method for UAVs. By minimizing the spatial point-to-edge distance and point-to-plane distance, the relative pose constraint factor between keyframes is designed for UAV continuous pose estimation. To reduce the accumulated localization errors during the long-distance flight in a perceptual-degraded mine, a semantic knowledge database is established by segmenting the intersection point cloud from the prior map of the mine. The topological feature of the current keyframe is detected in real time during the UAV flight. The intersection position constraint factor is constructed by comparing the similarity between the topological feature of the current keyframe and the intersections in the semantic knowledge database. Combining the relative pose constraint factor of LiDAR keyframes and the intersection position constraint factor, the optimization model of the UAV pose factor graph is established to estimate UAV flight pose and eliminate the cumulative error. Two UAV localization experiments conducted on the simulated large-scale Edgar Mine and a mine-like indoor corridor indicate that the proposed UAV localization method can realize accurate localization during long-distance flight in degraded mines.

A graphics-based digital twin (GBDT) framework for accurate UAV localization in GPS-denied environments

Autonomous navigation of Unmanned Aerial Vehicles (UAVs) is crucial for effective assessment of large-scale civil infrastructure, as manual UAV control is time consuming and prone to mishaps. Autonomous navigation outdoors typically employs GPS signals to enable accurate localization and reduce long-term drifts. However, in many civil engineering applications, GPS signals are either poor or unavailable due to interference and/or multi-path effects. Current approaches for UAV localization in GPS-denied environments, track geometric features such as corners; however, these natural features can be misrepresented due to the presence of occlusions and/or image processing errors, resulting in significant localization errors that can grow with time. AprilTags can reduce localization errors, but installation on large-scale civil infrastructure can be challenging. Therefore, this paper proposes a framework that leverages the wealth of visual and geometric information encoded in a Graphics-Based Digital Twin (GBDT) of a target infrastructure to provide accurate localization of a UAV. The GBDT is comprised of a computer graphics model that faithfully represents a target structure’s geometry, structural features, and visual textures. The visual details of the GBDT are exploited to design an object recognition algorithm that detects GBDTtags, which are distinctive objects (or a collection of components) on the target structure in advance; GBDTtags provide functionality similar to AprilTags. When the camera attached to a UAV detects one or more GBDTtags, the vertices of the GBDTtags are mapped from the image plane into the GBDT coordinate system, allowing for the UAV to be localized. The framework, termed herein as GBDTpose, is first validated numerically using Blender, Gazebo, and Mavros software-in-the-loop (SITL). Subsequently, field validation is carried out using the Kavita and Lalit Bahl Smart Bridge at the University of Illinois Urbana-Champaign (UIUC). Results show that localization in GPS-denied environments can be achieved with 5-50 cm accuracy without the need for physical markers being placed on the structure.

Jamming avoidance trajectory planning and load balancing user association in mmWave UAV-assisted HetECN

Emergency communication network (ECN) can provide fast, efficient and high-capacity communication services for specific areas by using mmWave transmission and unmanned aerial vehicles (UAVs) serving as aerial base stations (ABSs) or relay nodes. Now, in order to satisfy diverse demands, ECN should support different types of nodes, access methods, and traffic distributions, which is referred to as heterogeneous ECN (HetECN). Therefore, inappropriate trajectory planning and unbalanced traffic loading can lead to UAV flight collisions and network congestion. In this article, we jointly optimize UAV jamming avoidance trajectory and user association strategy aimed to load balancing, to maximize the utilization of HetECN. Specifically, an improved artificial potential field (APF) method along with mmWave beam forming technology is used to obtain the jamming avoidance trajectory of UAVs, and the optimal deployment location of UAVs are determined based on the distribution of ground users (GUs). Subsequently, the matching game and alliance game are comprehensively used to determine the load balancing based GU-UAV associated strategy under various GU demands, thereby ensuring traffic load balancing and resource optimization allocation. In addition, altitude fine-tuning have been made to further power consumption, thereby improving overall network efficiency. Simulation results demonstrate that the proposed method can achieve the expected performance in network utilities such as coverage rate, network capacity, load balancing effect of mmWave UAV-assisted HetECNs.

IRS-assisted UAV wireless powered communication network for sustainable federated learning

Intelligent reflective surface (IRS) and unmanned aerial vehicle (UAV) communication are indispensable potential technologies in the future sixth-generation mobile communication technology (6 G). Utilizing the high beamforming gain of IRS and the high mobility of UAV can achieve ubiquitous network coverage and ensure a high-quality communication environment. Wireless Powered Communication Network (WPCN) is an emerging green communication technology network that converts received radio frequency (RF) signals into electrical energy to power Federated Learning (FL) users. FL users perform local computing and model transmission through the collected energy, ensuring the sustainability of FL. In order to solve the problems of complex communication environment, privacy protection, and energy constraints on terminal devices, we design an FL system based on an IRS-assisted UAV wireless power communication network, which minimizes the UAV transmission energy by jointly optimizing UAV location, IRS phase shift, and resource allocation strategies. We use a low-complexity iterative algorithm to solve this complex non-convex problem. The simulation results show that the performance of the proposed algorithm is obviously better than that of other benchmark schemes, indicating that joint optimization plays an essential role in improving the performance of the system.

MMHCA: Multi-feature representations based on multi-scale hierarchical contextual aggregation for UAV-view geo-localization

In global navigation satellite system denial environment, cross-view geo-localization based on image retrieval presents an exceedingly critical visual localization solution for Unmanned Aerial Vehicle (UAV) systems. The essence of cross-view geo-localization resides in matching images containing the same geographical targets from disparate platforms, such as UAV-view and satellite-view images. However, images of the same geographical targets may suffer from occlusions and geometric distortions due to variations in the capturing platform, view, and timing. The existing methods predominantly extract features by segmenting feature maps, which overlook the holistic semantic distribution and structural information of objects, resulting in loss of image information. To address these challenges, dilated neighborhood attention Transformer is employed as the feature extraction backbone, and Multi-feature representations based on Multi-scale Hierarchical Contextual Aggregation (MMHCA) is proposed. In the proposed MMHCA method, the multi-scale hierarchical contextual aggregation method is utilized to extract contextual information from local to global across various granularity levels, establishing feature associations of contextual information with global and local information in the image. Subsequently, the multi-feature representations method is utilized to obtain rich discriminative feature information, bolstering the robustness of model in scenarios characterized by positional shifts, varying distances, and scale ambiguities. Comprehensive experiments conducted on the extensively utilized University-1652 and SUES-200 benchmarks indicate that the MMHCA method surpasses the existing techniques, showing outstanding results in UAV localization and navigation.

Rotor unmanned aerial vehicle localization in the building sheltered area based on millimetre‐wave frequency‐modulated continuous wave radar

AbstractRotor unmanned aerial vehicles (UAVs) play an important role in both military and civilian fields nowadays. The safety risks associated with the UAVs increase the urgent need for detecting UAVs in urban environments as well. Moreover, UAVs are easily located in the non‐line‐of‐sight (NLOS) building sheltered area relative to the radar, making it very challenging to detect and localise. A novel algorithm for localising the rotor UAV in the common L‐shaped street building sheltered area is proposed. First, the authors establish a multipath signal model for a rotor UAV hovering over an L‐shaped street scene, leveraging the frequency‐modulated continuous wave signal. Then, the multipath information of the UAV is extracted by identifying the rotating periodicity of the blade embedded in the time‐frequency spectrum of the received signal. The back projection imaging is then conducted on the UAV‐related multipath. After extracting multipath ghosts in the image, the street area, where the UAV locates, can be determined, and the UAV is further localised using the path reflection characteristics of this area. Simulations and practical experiments based on millimetre waves indicate that the proposed method can enable high‐accuracy estimation of rotor UAV in the NLOS building sheltered area.

Multi-sensor fusion for robust indoor localization of industrial UAVs using particle filter

Abstract Robotic platforms including Unmanned Aerial Vehicles (UAVs) require an accurate and reliable source of position information, especially in indoor environments where GNSS cannot be used. This is typically accomplished by using multiple independent position sensors. This paper presents a UAV position estimation mechanism based on a particle filter, that combines information from visual odometry cameras and visual detection of fiducial markers. The article proposes very compact, lightweight and robust method for indoor localization, that can run with high frequency on the UAV’s onboard computer. The filter is implemented such that it can seamlessly handle sensor failures and disconnections. Moreover, the filter can be extended to include inputs from additional sensors. The implemented approach is validated on data from real-life UAV test flights, where average position error under 0.4 m was achieved.

Vision-Based UAV Detection and Localization to Indoor Positioning System.

In recent years, the technological landscape has undergone a profound metamorphosis catalyzed by the widespread integration of drones across diverse sectors. Essential to the drone manufacturing process is comprehensive testing, typically conducted in controlled laboratory settings to uphold safety and privacy standards. However, a formidable challenge emerges due to the inherent limitations of GPS signals within indoor environments, posing a threat to the accuracy of drone positioning. This limitation not only jeopardizes testing validity but also introduces instability and inaccuracies, compromising the assessment of drone performance. Given the pivotal role of precise GPS-derived data in drone autopilots, addressing this indoor-based GPS constraint is imperative to ensure the reliability and resilience of unmanned aerial vehicles (UAVs). This paper delves into the implementation of an Indoor Positioning System (IPS) leveraging computer vision. The proposed system endeavors to detect and localize UAVs within indoor environments through an enhanced vision-based triangulation approach. A comparative analysis with alternative positioning methodologies is undertaken to ascertain the efficacy of the proposed system. The results obtained showcase the efficiency and precision of the designed system in detecting and localizing various types of UAVs, underscoring its potential to advance the field of indoor drone navigation and testing.

Downlink Transmissions of UAV-RIS-Assisted Cell-Free Massive MIMO Systems: Location and Trajectory Optimization.

In this paper, we investigate a cell-free massive multiple-input multiple-output (CF-mMIMO) system with a reconfigurable intelligent surface (RIS) carried by an unmanned aerial vehicle (UAV), called the UAV-RIS. Compared with the RIS located on the ground, the UAV-RIS has a wider coverage that can reflect all signals from access points (APs) and user equipment (UE). By correlating the UAV location with the Rician K-factor, we derive a closed-form approximation of the UE achievable downlink rate. Based on this, we obtain the optimal UAV location and RIS phase shift that can maximize the UE sum rate through an alternating optimization method. Simulation results have verified the accuracy of the derived approximation and shown that the UE sum rate can be significantly improved with the obtained optimal UAV location and RIS phase shift. Moreover, we find that with a uniform UE distribution, the UAV-RIS should fly to the center of the system, while with an uneven UE distribution, the UAV-RIS should fly above the area where UEs are gathered. In addition, we also design the best trajectory for the UAV-RIS to fly from its initial location to the optimal destination while maintaining the maximum UE sum rate per time slot during the flight.

Enhanced Unmanned Aerial Vehicle Localization in Dynamic Environments Using Monocular Simultaneous Localization and Mapping and Object Tracking

This work proposes an innovative approach to enhance the localization of unmanned aerial vehicles (UAVs) in dynamic environments. The methodology integrates a sophisticated object-tracking algorithm to augment the established simultaneous localization and mapping (ORB-SLAM) framework, utilizing only a monocular camera setup. Moving objects are detected by harnessing the power of YOLOv4, and a specialized Kalman filter is employed for tracking. The algorithm is integrated into the ORB-SLAM framework to improve UAV pose estimation by correcting the impact of moving elements and effectively removing features connected to dynamic elements from the ORB-SLAM process. Finally, the results obtained are recorded using the TUM RGB-D dataset. The results demonstrate that the proposed algorithm can effectively enhance the accuracy of pose estimation and exhibits high accuracy and robustness in real dynamic scenes.

Sound Source Localization for Unmanned Aerial Vehicles in Low Signal-to-Noise Ratio Environments

Sound Source Localization for Unmanned Aerial Vehicles in Low Signal-to-Noise Ratio Environments

Visual SLAM for Unmanned Aerial Vehicles: Localization and Perception.

Localization and perception play an important role as the basis of autonomous Unmanned Aerial Vehicle (UAV) applications, providing the internal state of movements and the external understanding of environments. Simultaneous Localization And Mapping (SLAM), one of the critical techniques for localization and perception, is facing technical upgrading, due to the development of embedded hardware, multi-sensor technology, and artificial intelligence. This survey aims at the development of visual SLAM and the basis of UAV applications. The solutions to critical problems for visual SLAM are shown by reviewing state-of-the-art and newly presented algorithms, providing the research progression and direction in three essential aspects: real-time performance, texture-less environments, and dynamic environments. Visual-inertial fusion and learning-based enhancement are discussed for UAV localization and perception to illustrate their role in UAV applications. Subsequently, the trend of UAV localization and perception is shown. The algorithm components, camera configuration, and data processing methods are also introduced to give comprehensive preliminaries. In this paper, we provide coverage of visual SLAM and its related technologies over the past decade, with a specific focus on their applications in autonomous UAV applications. We summarize the current research, reveal potential problems, and outline future trends from academic and engineering perspectives.

Research on a Sound Source Localization Method for UAV Detection Based on Improved Empirical Mode Decomposition.

To address the challenge of accurately locating unmanned aerial vehicles (UAVs) in situations where radar tracking is not feasible and visual observation is difficult, this paper proposes an innovative acoustic source localization method based on improved Empirical Mode Decomposition (EMD) within an adaptive frequency window. In this study, the collected flight signals of UAVs undergo smoothing filtering. Additionally, Robust Empirical Mode Decomposition (REMD) is applied to decompose the signals into Intrinsic Mode Function (IMF) components for spectrum analysis. We introduce a sliding frequency window with adjustable bandwidth, which is automatically determined using a Grey Wolf Optimizer (GWO) with a sliding index. This window is used to lock and extract specific frequencies from the IMFs. Based on predefined criteria, the extracted IMF components are reconstructed, and trigger signal times are analyzed and recorded from these reconstructed IMFs. The time differences between sensor receptions are then calculated. Furthermore, this study introduces the Chan-Taylor localization algorithm based on weighted least squares. This advanced algorithm takes sensor time delay parameters as input and solves a set of nonlinear equations to determine the target's location. Simulations and real-world signal tests are used to validate the robustness and performance of the proposed method. The results indicate that the localization error remains below 5% within a 15 m × 15 m measurement area. This provides an efficient and real-time method for detecting the location of small UAVs.

Distributed Localization for UAV–UGV Cooperative Systems Using Information Consensus Filter

In the evolving landscape of autonomous systems, the integration of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) has emerged as a promising solution for improving the localization accuracy and operational efficiency for diverse applications. This study introduces an Information Consensus Filter (ICF)-based decentralized control system for UAVs, incorporating the Control Barrier Function–Control Lyapunov Function (CBF–CLF) strategy aimed at enhancing operational safety and efficiency. At the core of our approach lies an ICF-based decentralized control algorithm that allows UAVs to autonomously adjust their flight controls in real time based on inter-UAV communication. This facilitates cohesive movement operation, significantly improving the system resilience and adaptability. Meanwhile, the UAV is equipped with a visual recognition system designed for tracking and locating the UGV. According to the experiments proposed in the paper, the precision of this visual recognition system correlates significantly with the operational distance. The proposed CBF–CLF strategy dynamically adjusts the control inputs to maintain safe distances between the UAV and UGV, thereby enhancing the accuracy of the visual system. The effectiveness and robustness of the proposed system are demonstrated through extensive simulations and experiments, highlighting its potential for widespread application in UAV operational domains.

Decentralized triangular relative localization of multiple UAVs based on relative range and inertial measurements

In this paper, a novel algorithm for cooperative relative localization of multiple Unmanned Aerial Vehicles (UAVs) is proposed based on relative range and inertial measurements. In this algorithm, a relative motion estimation model is established for each group of three UAVs that can form a triangle in space. Each group member estimates the relative position and heading angle of the other group members and shares the estimation results with the other group members. When members satisfy the triangle law among their estimated relative position vectors, they can estimate more accurately. After analyzing the observability of the presented model, the necessary conditions for observability are also given, which are less restrictive compared to previous methods. Then, the presented method is compared with the decentralized consensus-based Kalman filter approach. The results of both methods are analyzed in the presence of noise and disturbances, and under the condition of disconnection between the members.

An omnidirectional spatial monocular visual localization and tracking method for indoor unmanned aerial vehicles based on the two-axis rotary table

Aiming at the complexity and poor adaptability of the calibration process in the traditional unmanned aerial vehicles (UAV) indoor visual positioning, this paper proposes an omnidirectional spatial tracking and localization method for indoor UAV based on the two-axis rotary table. Firstly, the position of the UAV fuselage feature points in the camera coordinate system of the turntable camera is computed by the Pespective-n-Point algorithm utilizing known position information of a plurality of feature points with pixel coordinate information in the corresponding image. Pixel coordinate information is extracted by obtaining UAV body-specific feature points from rotary table camera shots. Then, UAV localization in omnidirectional space can be obtained by using the calibrated rotary axis parameters of the rotary table and the rotation angle of the rotary table and substituting them into Rodriguez’s formula to unify the UAV position information acquired by the rotary table camera at different positions into a unified coordinate system. Finally, the angle at which the rotary table should rotate is calculated from the obtained UAV pose and the spatial position of the camera optical center and the rotary axis of the rotary table. The calculated angle is fed back to the turntable as feedback information. The rotary table receiving the feedback information is rotated to a position where the UAV is located at the center of the camera image. Thereby the tracking and localization of the UAV is realized. The experimental results show that the spatial range of localization is greatly expanded with the localization accuracy reaching the level of binocular visual localization. The omnidirectional spatial tracking and localization of indoor UAV can be conveniently realized by this method.

Privacy-Preserving Location Authentication for Low-altitude UAVs: A Blockchain-based Approach

Efficient and trusted regulation of unmanned aerial vehicles (UAVs) is an essential but challenging issue in the future era of Internet of Low-altitude Intelligence, due to the difficulties in UAVs' identity recognition and location matching, potential for falsified information reporting, etc. To address this challenging issue, in this paper, we propose a blockchain-based UAV location authentication scheme, which employs a distance bounding protocol to establish a location proof, ensuring the authenticity of UAV positions. To preserve the privacy of UAVs, anonymous certificates and zero-knowledge proof are used. The security of the proposed scheme is analyzed. Experiments demonstrate the efficiency and feasibility of the proposed scheme.

Real-Time Optical Localization and Tracking of UAV Using Ellipse Detection

A real-time three-dimensional (3D) spatial localization system to track a UAV (unmanned aerial vehicle) using a single off-board conventional camera and an onboard circle-shaped colored Light-Emitting Diode (LED) marker is presented. The image from the camera is color-segmented and then morphologically closed before being sent to the ellipse detector. The location of the drone is then estimated with geometric/optical calculations using the pixel coordinates provided by the real-time ellipse detector. The location data generated by this system was validated by a Motion-Capture System (MCS) that was simultaneously tracking the system in real-time. The average position error was of 6cm, while the processing speed achieved 15 locations/second, which are comparable with recent research references. The usage of a low-cost camera makes this method promising for most mobile UAV tracking applications.

İnsansız Hava Araçları (İHA) İçin Küresel Navigasyon Uydu Sistemi (GNSS) Bağımsız Navigasyon

The ability of Unmanned Aerial Vehicles (UAV) to perform autonomous navigation depends on the accurate determination of their positions provided by the Global Navigation Satellite System (GNSS). For position determination and environmental orientation during flight, UAVs are usually equipped with electronic equipment such as GNSS, Inertial Measurement Unit (IMU), gyroscope and accelerometer. However, the GNSS signal may be lost or distorted due to poor weather, obstacles or terrain, the unfavorable position of satellites, spoofing and jamming. In such cases of GNSS signal loss or deterioration, the IMU alone becomes unable to provide reliable UAV location information. Especially in cases where there is not enough visibility and the UAV cannot be brought to the take-off point by manual operation, the loss of the GNSS signal causes great losses. In this paper, GNSS independent flight and navigation studies are included. It is seen that the use of hybrid navigation solutions has great importance in GNSS independent UAV flights.

A Novel Approach for Simultaneous Localization and Dense Mapping Based on Binocular Vision in Forest Ecological Environment

The three-dimensional reconstruction of forest ecological environment by low-altitude remote sensing photography from Unmanned Aerial Vehicles (UAVs) provides a powerful basis for the fine surveying of forest resources and forest management. A stereo vision system, D-SLAM, is proposed to realize simultaneous localization and dense mapping for UAVs in complex forest ecological environments. The system takes binocular images as input and 3D dense maps as target outputs, while the 3D sparse maps and the camera poses can be obtained. The tracking thread utilizes temporal clue to match sparse map points for zero-drift localization. The relative motion amount and data association between frames are used as constraints for new keyframes selection, and the binocular image spatial clue compensation strategy is proposed to increase the robustness of the algorithm tracking. The dense mapping thread uses Linear Attention Network (LANet) to predict reliable disparity maps in ill-posed regions, which are transformed to depth maps for constructing dense point cloud maps. Evaluations of three datasets, EuRoC, KITTI and Forest, show that the proposed system can run at 30 ordinary frames and 3 keyframes per second with Forest, with a high localization accuracy of several centimeters for Root Mean Squared Absolute Trajectory Error (RMS ATE) on EuRoC and a Relative Root Mean Squared Error (RMSE) with two average values of 0.64 and 0.2 for trel and Rrel with KITTI, outperforming most mainstream models in terms of tracking accuracy and robustness. Moreover, the advantage of dense mapping compensates for the shortcomings of sparse mapping in most Smultaneous Localization and Mapping (SLAM) systems and the proposed system meets the requirements of real-time localization and dense mapping in the complex ecological environment of forests.