Pose Of Unmanned Aerial Vehicle Research Articles

An omnidirectional spatial monocular visual localization and tracking method for indoor unmanned aerial vehicles based on the two-axis rotary table

Aiming at the complexity and poor adaptability of the calibration process in the traditional unmanned aerial vehicles (UAV) indoor visual positioning, this paper proposes an omnidirectional spatial tracking and localization method for indoor UAV based on the two-axis rotary table. Firstly, the position of the UAV fuselage feature points in the camera coordinate system of the turntable camera is computed by the Pespective-n-Point algorithm utilizing known position information of a plurality of feature points with pixel coordinate information in the corresponding image. Pixel coordinate information is extracted by obtaining UAV body-specific feature points from rotary table camera shots. Then, UAV localization in omnidirectional space can be obtained by using the calibrated rotary axis parameters of the rotary table and the rotation angle of the rotary table and substituting them into Rodriguez’s formula to unify the UAV position information acquired by the rotary table camera at different positions into a unified coordinate system. Finally, the angle at which the rotary table should rotate is calculated from the obtained UAV pose and the spatial position of the camera optical center and the rotary axis of the rotary table. The calculated angle is fed back to the turntable as feedback information. The rotary table receiving the feedback information is rotated to a position where the UAV is located at the center of the camera image. Thereby the tracking and localization of the UAV is realized. The experimental results show that the spatial range of localization is greatly expanded with the localization accuracy reaching the level of binocular visual localization. The omnidirectional spatial tracking and localization of indoor UAV can be conveniently realized by this method.

N-Cameras-Enabled Joint Pose Estimation for Auto-Landing Fixed-Wing UAVs

We propose a novel 6D pose estimation approach tailored for auto-landing fixed-wing unmanned aerial vehicles (UAVs). This method facilitates the simultaneous tracking of both position and attitude using a ground-based vision system, regardless of the number of cameras (N-cameras), even in Global Navigation Satellite System-denied environments. Our approach proposes a pipeline consisting of a Convolutional Neural Network (CNN)-based detection of UAV anchors which, in turn, drives the estimation of UAV pose. In order to ensure robust and precise anchor detection, we designed a Block-CNN architecture to mitigate the influence of outliers. Leveraging the information from these anchors, we established an Extended Kalman Filter to continuously update the UAV’s position and attitude. To support our research, we set up both monocular and stereo outdoor ground view systems for data collection and experimentation. Additionally, to expand our training dataset without requiring extra outdoor experiments, we created a parallel system that combines outdoor and simulated setups with identical configurations. We conducted a series of simulated and outdoor experiments. The results show that, compared with the baselines, our method achieves 3.0% anchor detection precision improvement and 19.5% and 12.7% accuracy improvement of position and attitude estimation. Furthermore, these experiments affirm the practicality of our proposed architecture and algorithm, meeting the stringent requirements for accuracy and real-time capability in the context of auto-landing fixed-wing UAVs.

UAVPNet: A balanced and enhanced UAV object detection and pose recognition network

With the increasing popularity of unmanned aerial vehicles (UAVs), accurate positioning and pose recognition of UAVs by target images based on photoelectric detection become a research hotspot. To solve this issue, a multi-scale UAV-Pose dataset consisting of 1400 UAV images is contributed in this paper. In addition, a balanced and enhanced network (UAVPNet) is proposed. UAVPNet has two major features: (1) balanced feature pyramid (BFP) feature fusion structure to improve unbalanced multi-scale features; (2) VarifocalNet detection head to alleviate the foreground-background imbalance. A comparative study demonstrates that UAVPNet is superior to some state-of-the-art object detection models (such as Faster R-CNN-CARAFE, and Yolov8, etc.) in terms of detection accuracy and robustness. Specifically, UAVPNet achieves state-of-the-art 0.885 mAP on the newly-created UAV-Pose dataset, together with nearly 33.49 M parameters, 139.9G FLOPs, and 9.8 FPS. It could fully fulfill the requirements of UAV positioning and pose recognition in the intricate environment.

Assessment of Accuracy in Unmanned Aerial Vehicle (UAV) Pose Estimation with the REAL-Time Kinematic (RTK) Method on the Example of DJI Matrice 300 RTK.

The growing possibilities offered by unmanned aerial vehicles (UAV) in many areas of life, in particular in automatic data acquisition, spur the search for new methods to improve the accuracy and effectiveness of the acquired information. This study was undertaken on the assumption that modern navigation receivers equipped with real-time kinematic positioning software and integrated with UAVs can considerably improve the accuracy of photogrammetric measurements. The research hypothesis was verified during field measurements with the use of a popular Enterprise series drone. The problems associated with accurate UAV pose estimation were identified. The main aim of the study was to perform a qualitative assessment of the pose estimation accuracy of a UAV equipped with a GNSS RTK receiver. A test procedure comprising three field experiments was designed to achieve the above research goal: an analysis of the stability of absolute pose estimation when the UAV is hovering over a point, and analyses of UAV pose estimation during flight along a predefined trajectory and during continuous flight without waypoints. The tests were conducted in a designated research area. The results were verified based on direct tachometric measurements. The qualitative assessment was performed with the use of statistical methods. The study demonstrated that in a state of apparent stability, horizontal deviations of around 0.02 m occurred at low altitudes and increased with a rise in altitude. Mission type significantly influences pose estimation accuracy over waypoints. The results were used to verify the accuracy of the UAV's pose estimation and to identify factors that affect the pose estimation accuracy of an UAV equipped with a GNSS RTK receiver. The present findings provide valuable input for developing a new method to improve the accuracy of measurements performed with the use of UAVs.

A Robust Learned Feature-Based Visual Odometry System for UAV Pose Estimation in Challenging Indoor Environments

Unmanned Aerial Vehicles (UAVs) are becoming popular nowadays due to their versatility and flexibility for indoor applications, such as the autonomous visual inspection for the inner surface of a pressure vessel. Nevertheless, robust and reliable position estimation is critical for completing these tasks. Visual Odometry (VO) and Visual Simultaneous Localisation and Mapping (VSLAM) allow the UAV to estimate its position in unknown environments. However, traditional feature-based VO/VSLAM systems struggle to deal with complex scenes such as low illumination and textureless environments. Replacing the traditional features with deep learning-based features provides the advantage of handling the challenging environment, but the efficiency is ignored. In this work, an efficient VO system based on a novel lightweight feature extraction network for UAV onboard platforms has been developed. The Deformable Convolution (DFConv) is utilised to improve the feature extraction capability. Owing to the limited onboard computing capability, the Depth-wise Separable Convolution (DWConv) is adopted to calculate the offsets for the deformable convolution and construct the backbone network to improve the feature extraction efficiency. Experiments on public datasets indicate that the efficiency of the VO system is improved by 30.03% while preserving the accuracy on embedded platforms with the feature points and descriptors detected by the proposed Convolutional Neural Network (CNN). Moreover, the proposed VO system is verified through UAV flying tests in a real-world scenario. The results prove that the proposed VO system is able to handle the challenging environments where both the latest traditional and deep learning feature-based VO/VSLAM systems fail, and it is feasible for UAV self-localisation and autonomous navigation in the confined, low illumination and textureless indoor environment.

Visual-Inertial Image-Odometry Network (VIIONet): A Gaussian process regression-based deep architecture proposal for UAV pose estimation

This study estimates the pose of Unmanned Aerial Vehicle (UAV) through artificial intelligence-based approaches and combines visual-inertial information in a different way than previous studies. For an effective fusion, the inertial data between both frames is normalized after denoising with the Savitzky-Golay technique and finally converted from numerical value to image. To strengthen these inertial image features with the change of motion between two frames, frames of Optical Flow (OF) are obtained and OF frames are combined with inertial images. Simultaneously, a parallel thread combines this OF frame with two consecutive raw frames. After features are extracted from inertial and camera data via Inception-v3, these features are fused and actual UAV poses are estimated via Gaussian Process Regression (GPR). Thanks to the smoothing process applied to these estimated values, a more stable pose estimation is provided. This proposed method is applied to the EuRoC dataset and our dataset produced in the Gazebo environment. The pose estimation results reveal that the proposed method has high performance compared to many previous studies.

UAV Pose Estimation in GNSS-Denied Environment Assisted by Satellite Imagery Deep Learning Features

With the growing maturity of unmanned aerial vehicle (UAV) technology, its applications have widened to many spheres of life. The prerequisite for a UAV to perform air tasks smoothly is an accurate localization of its own position. Traditional UAV navigation relies on the Global Navigation Satellite System (GNSS) for localization; however, this system has disadvantages of instability and susceptibility to interference. Therefore, to obtain accuracy in UAV pose estimation in GNSS-denied environments, a UAV localization method that is assisted by deep learning features of satellite imagery is proposed. With the inclusion of a top-view optical camera to the UAV, localization is achieved based on satellite imageries with geographic coordinates and a digital elevation model (DEM). By utilizing the difference between the UAV frame and satellite imagery, the convolutional neural network is used to extract deep learning features between the two images to achieve stable registration. To improve the accuracy and robustness of the localization method, a local optimization method based on bundle adjustment (BA) is proposed. Experiments demonstrate that when the UAV’s relative altitude is 0.5 km, the average localization error of this method under different trajectories is within 15 m.

Trajectory and image-based detection and identification of UAV

Much more attentions have been attracted to the inspection and prevention of unmanned aerial vehicle (UAV) in the wake of increasing high frequency of security accident. Many factors like the interferences and the small fuselage of UAV pose challenges to the timely detection of the UAV. In our work, we present a system that is capable of detecting, recognizing, and tracking an UAV using single camera automatically. For our method, a single pan–tilt–zoom (PTZ) camera detects flying objects and gets their trajectories; then, the trajectory identified as a UAV guides the camera and PTZ to capture the detailed region image of the target. Therefore, the images can be classified into the UAV and interference classes (such as birds) by the convolution neural network classifier trained with our image dataset. For the target recognized as a UAV with the double verification, the radio jammer emits the interferential radio to disturb its control radio and GPS. This system could be applied in some complex environment where many birds and UAV appear simultaneously.

Real-time georeferencing of thermal images using small fixed-wing UAVs in maritime environments

This article considers real-time georeferencing using a fixed-wing unmanned aerial vehicle (UAV) with a thermal camera. A flexible system for direct georeferencing is proposed without the need for ground reference points. Moreover, as the system is tailored for highly maneuverable and agile fixed-wing UAVs, no restrictions on the motion are assumed. The system is designed with a solution for accurate time synchronization between sensors. This feature enables tracking of objects with low uncertainty. Sensors for navigation, permitting estimation of the UAV pose with a nonlinear observer, are employed in addition to a thermal camera. The estimated UAV pose is utilized in georeferencing to acquire Earth-fixed coordinates of objects. The main examples studied in this research are georeferencing of a static object and of a moving marine vessel. To obtain the desired accuracy, thermal camera calibration and compensation of mounting misalignment errors are discussed. The entire system is validated in two independent field experiments with a thorough analysis of the results. Georeferencing of a static object is conducted with centimeter accuracy when the average position of all measurements is used. The position of a moving marine vessel is obtained with mean accuracy of two meters.

A novel measurement model based on abBRIEF for global localization of a UAV over satellite images

This paper presents a method for global localization and tracking of an Unmanned Aerial Vehicle (UAV) over satellite images. We propose a new measurement model based on a novel version of BRIEF descriptor and apply it in a Monte Carlo Localization system that estimates the UAV pose in 4 degrees of freedom. The model is used to compare images obtained from the UAV downward looking camera against patches of satellite images such as the ones available on Google™ Earth. The proposed method was validated using real flights sequences and has yield good results with different maps of the same region spawning many years and covering large areas.

A hierarchical vision-based localization of rotor unmanned aerial vehicles for autonomous landing

The vision-based localization of rotor unmanned aerial vehicles for autonomous landing is challenging because of the limited detection range. In this article, to extend the vision detection and measurement range, a hierarchical vision-based localization method is proposed for unmanned aerial vehicle autonomous landing. In such a hierarchical framework, the landing is defined into three phases: “Approaching,”“Adjustment,” and “Touchdown,” in which visual artificial features at different scales can be detected from the designed object pattern for unmanned aerial vehicle pose recovery. The corresponding feature detection and pose estimation algorithms are also presented. In the end, typical simulation and field experiments have been carried out to illustrate the proposed method. The results show that our hierarchical vision-based localization has the ability to a consecutive unmanned aerial vehicle localization in a wider working range from far to near, which is significant for autonomous landing.

Alignment of UAV-hyperspectral bands using keypoint descriptors in a spectrally complex environment

ABSTRACTNear earth imaging spectroscopy has gained popularity in the recent past due to its increasing capabilities to acquire data with unprecedented abundance of spatial resolution and spectral purity. Unmanned Aerial Vehicle (UAV) based portable hyperspectral sensors employing snapshot based scanning mechanisms further advances these capabilities with improved spatial co-registration of pixels. However, unavoidable motion of UAV pose challenges in spectral co-registration which is pronounced in spectrally complex environments of heterogeneous ecosystems. In this study, the different feature descriptor techniques such as Harris-Stephens Features (HSF), Min Eigen Features (MEF), Scale Invariant Feature Transformation (SIFT), Speeded-Up Robust Features (SURF), Binary Robust Invariant Scalable Keypoints (BRISK), and Features from Accelerated Segment Test (FAST) were evaluated to align hyperspectral bands in a spectrally complex environment. A band alignment workflow was devised and operated in different band-wise arrangements (spectral order and temporal order) of the acquired hypercube. The co-registration accuracy between the adjacent band pairs (reference-transformed) upon registration was estimated using Root-Mean-Square Error (RMSE) and Pearson’s Correlation Coefficient (PCC) based approaches. Furthermore, a standard transformation workflow was used to evaluate the efficiency of the different feature descriptor based band-to-band registration approaches.

A Ground-based Vision System for UAV Pose Estimation

We present a vision system based on a single frame of standard RGB digital camera to estimate the pose of an unmanned aerial vehicle (UAV). The envisaged application is of ground-based automatic landing, where the vision system is located on the ship’s deck and is used to estimate the UAV pose (3D position and orientation) during the landing process. Using a vision system located on the ship makes it possible to use an UAV with less processing power, decreasing its size and weight. The proposed method uses a 3D model based pose estimation approach that requires the 3D CAD model of the UAV. Pose is estimated in a particle filtering framework. The implemented particle filter is inspired in the evolution strategies present in the genetic algorithms avoiding sample impoverishment. Results show position and angular errors are compatible with automatic landing system requirements, even without temporal filtering. The algorithm is suitable for real time implementation in standard workstations with graphical processing units.

Visual lock-on to invisible target for unmanned aerial vehicle

Presented is a robust visual lock-on framework for an unmanned aerial vehicle (UAV) that utilises geometric relations between the UAV pose and the 3D local map defined by the positions of the target and natural landmarks. Experimental results using real datasets demonstrate the robustness of the proposed method compared to state-of-the-art visual tracking methods.