Drogue Detection Research Articles

Semi-Supervised Image Domain Adaption for Aerial Refueling Drogue Detection on Embedded Chip Under Foggy Conditions

Semi-Supervised Image Domain Adaption for Aerial Refueling Drogue Detection on Embedded Chip Under Foggy Conditions

Drogue detection for autonomous aerial refueling via hybrid pigeon-inspired optimized color opponent and saliency aggregation

Drogue detection is one of the challenging tasks in autonomous aerial refueling due to the requirement for accuracy and rapidity. Saliency detection based on image intrinsic cues can achieve fast detection, but with poor accuracy. Recent studies reveal that optimization-based methods provide accurate and quick solutions for saliency detection. This paper presents a hybrid pigeon-inspired optimization method, the optimized color opponent, that aims to adjust the weight of color opponent channels to detect the drogue region. It can optimize the weights in the selected aerial refueling scene offline, and the results are applied for drogue detection in the scene. A novel algorithm aggregated by the optimized color opponent and robust background detection is presented to provide better precision and robustness. Experimental results on benchmark datasets and aerial refueling images show that the proposed method successfully extracts the saliency region or drogue and exhibits superior performance against the other saliency detection methods with intrinsic cues. The algorithm designed in this paper is competent for the drogue detection task of autonomous aerial refueling.

Research of an Unmanned Aerial Vehicle Autonomous Aerial Refueling Docking Method Based on Binocular Vision

In this paper, a visual navigation method based on binocular vision and a deep learning approach is proposed to solve the navigation problem of the unmanned aerial vehicle autonomous aerial refueling docking process. First, to meet the requirements of high accuracy and high frame rate in aerial refueling tasks, this paper proposes a single-stage lightweight drogue detection model, which greatly increases the inference speed of binocular images by introducing image alignment and depth-separable convolution and improves the feature extraction capability and scale adaptation performance of the model by using an efficient attention mechanism (ECA) and adaptive spatial feature fusion method (ASFF). Second, this paper proposes a novel method for estimating the pose of the drogue by spatial geometric modeling using optical markers, and further improves the accuracy and robustness of the algorithm by using visual reprojection. Moreover, this paper constructs a visual navigation vision simulation and semi-physical simulation experiments for the autonomous aerial refueling task, and the experimental results show the following: (1) the proposed drogue detection model has high accuracy and real-time performance, with a mean average precision (mAP) of 98.23% and a detection speed of 41.11 FPS in the embedded module; (2) the position estimation error of the proposed visual navigation algorithm is less than ±0.1 m, and the attitude estimation error of the pitch and yaw angle is less than ±0.5°; and (3) through comparison experiments with the existing advanced methods, the positioning accuracy of this method is improved by 1.18% compared with the current advanced methods.

Automated Aerial Docking System Using Onboard Vision-Based Deep Learning

This paper proposes an automated aerial docking system for unmanned aerial vehicles. The proposed automated aerial docking system consists of the following two subsystems: 1) docking mechanical system and 2) vision-based deep learning detection/tracking system. A fundamental challenge during the midair docking phase is integration between the leader and follower vehicles with a robust target detection/tracking strategy. This paper presents not only the design of a probe-and-drogue-type aerial docking system, but it also presents the development of an onboard machine learning system for aerial target detection and tracking, which is limited from 0.1 to 3.5 m. The aerial docking system is designed based on a bistable mechanism to increase the robustness of integration and separation between two vehicles. The prototype of the proposed docking mechanism is built by a 3D printer. To employ effective drogue detection in the air, a deep convolutional neural network-based single-stage detector algorithm, YOLOv4-tiny, is applied. Furthermore, to track the moving drogue in the air simultaneously, a point cloud-based tracking algorithm with an RGB-D camera system is developed. The developed deep learning-based detection/tracking system is implemented to be operated in the Arm architecture-based onboard machine learning computer. For performance validation of the proposed automated aerial docking system, a ground test using two robot arms and an indoor flight test using quadcopter drone and an unmanned ground vehicle were conducted.

A real-time embedded drogue detection method based on lightweight convolution neural network for autonomous aerial refueling

A real-time embedded drogue detection method based on lightweight convolution neural network for autonomous aerial refueling

Study on robust aerial docking mechanism with deep learning based drogue detection and docking

This paper proposes a simple and a robust bistable docking system with a deep learning based real-time drogue detection and tracking system for Unmanned Aircraft Systems (UAS) for mid-air autonomous aerial docking. Secure aerial docking mechanisms between the leader and follower aerial vehicles with effective drogue detection and tracking strategies are fundamental challenges during the air-to-air docking phase of autonomous aerial docking. To confront those issues, this paper not only presents the design of a bistable-based aerial docking mechanism, but also proposes effective deep learning based real-time drogue detection using a convolutional neural network (CNN) and real-time tracking algorithm using a point cloud algorithm. To ensure novelty and robustness for the aerial docking mechanism, a foldable bistable gripper-type mechanism is designed to increase the grasping performance with simplicity and adaptability. The proposed gripper acts as a drogue by itself to grasp a probe which is attached to the follower aerial vehicle. To employ an effective drogue detection method, the deep learning based real-time object detection algorithm, YOLOv3, is used to implement the drogue detection system. The proposed new probe-and-drogue type bistable docking system has the advantages of being simple and robust. The deep learning based real-time drogue detection method increases the detection rate. Moreover, the real-time tracking algorithm with a depth camera system does not require a GPS/INS system and many other sensors to follow the drogue movement in the air.

A Novel Vision‐Based PRPL Multistage Image Processing Algorithm for Autonomous Aerial Refueling

Autonomous aerial refueling (AAR) technology can increase the flight endurance of unmanned air vehicles (UAVs) effectively. Drogue detection and target tracking method are significant for probe‐drogue refueling system in the docking stage. This paper proposes a novel vision‐based multistage image processing algorithm of drogue detection and target tracking for AAR. This algorithm divides the whole task into four stages: preprocessor, recognizer, predictor, and locker (PRPL). The adaptive threshold segmentation (ATS) algorithm and support vector machine (SVM) classifier are utilized in preprocessor and recognizer for drogue detection. An improved kernelized correlation filter (IKCF) tracking algorithm and scale adaptive method by window position as well as image resolution adjusted are adopted in predictor and locker for target tracking in complex dynamic environments. Finally, the proposed PRPL multistage image processing strategy is tested using an autonomous aerial refueling testbed. The results indicate that the proposed algorithm achieves high precision, good reliability, and real‐time capability compared with conventional algorithms. The average processing time is within 11 ms in various environments, which can meet the requirement for drogue detection and tracking in AAR.

Materials and design for drogue detection in air-to-air refueling

Abstract Drogue detection is the primary issue during the air to air refueling particularly at hazy and night conditions. It is difficult for receiver aircraft pilot to detect the probe due to hazy and dark conditions at night. There has already been valuable research in probe position tracking, system modelling, manned and unmanned refueling with the help of sensors. All these methods work only when there is proper accuracy of drogue detection. So there is a drogue detection model with the opto-mechanical system that provides better visual accuracy and navigation to detect the port and real time performance of air refueling. This method is based on direct or manual inspection of fuel port. Experiments were conducted on ground level to demonstrate the effectiveness of proposed method and results were matched with the software data. This paper emphasizes over the availability of proper light intensity at drogue area so that receiver pilot must recognize the drogue position from a distance of 50 m with view angle of 40°. Results show that opto-mechanical system achieves light intensity of 1.0307 fc by calculations at 50 m distance.

A cascade adaboost and CNN algorithm for drogue detection in UAV autonomous aerial refueling

To promote the combat capability of unmanned aerial vehicles (UAVs) in the future battlefield, the autonomous aerial refueling (AAR) technology becomes a challenging research issue. An accurate position relationship between the tanker and the receiver is significant for AAR. A novel drogue detection method is presented in this paper. The Adaptive boosting (Adaboost) and the convolutional neural networks (CNN) classifier with the improved focal loss (IFL) function are utilized to detect the drogue in complex environments. The sample imbalance during the training stage of the CNN classifier is solved by the IFL function. The PyTorch deep learning framework is employed to implement the software system with the graphics processing units (GPUs). Real scenario images with a mimetic drogue on the tanker are captured for training and testing dataset by the airborne camera on the receiver. The experimental results indicate that the presented algorithm can accelerate the detection speed and improve the detection accuracy.

A novel autonomous aerial refueling drogue detection and pose estimation method based on monocular vision

This paper proposes a novel drogue measurement method based on monocular vision for autonomous aerial refueling task in unmanned aerial vehicles (UAV). Firstly, to solve the problem of detecting the drogue without artificial features, an arc-level drogue detection and recognition algorithm is proposed. Secondly, a pose estimation algorithm based on the structural features of the drogue is proposed measuring the pose of the drogue’s 3D space. Finally, the proposed method is proved to be valid according to the real air refueling task data set, ground acquisition data, and simulation experiments.

Real-time drogue detection and template tracking strategy for autonomous aerial refueling

Autonomous aerial refueling technology is an effective solution to extend flight duration of unmanned aerial vehicles, and also a great challenge due to its high risk. A novel real-time drogue detection and tracking strategy of monocular vision system for autonomous aerial refueling is proposed. It uses a direct image registration-based tracking method to ensure reliable and real-time tracking, and an ROI detection based on edge features to solve the tracking drift problem. A multiply patches fusion structure is adopted in the tracking method to improve the tracking accuracy and slow the divergence speed. Finally, various experiments are conducted to validate the proposed image processing strategy. These results show that the proposed strategy obtains a high accuracy as well as the real-time performance, and achieves a better performance than state-of-the-art methods.

Detecting the drogue presence of SVP drifters from wind slippage in the Mediterranean Sea

The Surface Velocity Program drifter data, collected in the Mediterranean Sea since 1989, are analysed in order to evaluate and, possibly, improve the accuracy of the drogue (sea anchor, attached to the buoy by a long thin tether) presence detection system. Wind products and altimetry data are used to estimate the wind slip from the drifter velocities. Complex correlations between the drifter residual and wind velocities reveal potential errors of the SVP drogue detection system and allow to define the drogue status for drifters without information about the drogue presence. A simple procedure is applied to update the drogue status by matching the drogue detection system data and the wind slippage correction. This method allows assigning a drogue status to about 88% of the drifter observations which were lacking this information and to detect/correct anomalous behaviours of the drogue detection system in 5.2% of the analysed cases.

Drogue detection for autonomous aerial refueling based on convolutional neural networks

Drogue detection is a fundamental issue during the close docking phase of autonomous aerial refueling (AAR). To cope with this issue, a novel and effective method based on deep learning with convolutional neural networks (CNNs) is proposed. In order to ensure its robustness and wide application, a deep learning dataset of images was prepared by utilizing real data of “Probe and Drogue” aerial refueling, which contains diverse drogues in various environmental conditions without artificial features placed on the drogues. By employing deep learning ideas and graphics processing units (GPUs), a model for drogue detection using a Caffe deep learning framework with CNNs was designed to ensure the method’s accuracy and real-time performance. Experiments were conducted to demonstrate the effectiveness of the proposed method, and results based on real AAR data compare its performance to other methods, validating the accuracy, speed, and robustness of its drogue detection ability.

Real-time drogue recognition and 3D locating for UAV autonomous aerial refueling based on monocular machine vision

Drogue recognition and 3D locating is a key problem during the docking phase of the autonomous aerial refueling (AAR). To solve this problem, a novel and effective method based on monocular vision is presented in this paper. Firstly, by employing computer vision with red-ring-shape feature, a drogue detection and recognition algorithm is proposed to guarantee safety and ensure the robustness to the drogue diversity and the changes in environmental conditions, without using a set of infrared light emitting diodes (LEDs) on the parachute part of the drogue. Secondly, considering camera lens distortion, a monocular vision measurement algorithm for drogue 3D locating is designed to ensure the accuracy and real-time performance of the system, with the drogue attitude provided. Finally, experiments are conducted to demonstrate the effectiveness of the proposed method. Experimental results show the performances of the entire system in contrast with other methods, which validates that the proposed method can recognize and locate the drogue three dimensionally, rapidly and precisely.

Vision Based Measurement of Refueling Drogue for Autonomous Aerial Refueling

Autonomous aerial refueling (AAR) can improve the long-distance and rapid strike capacities of an unmanned aerial vehicle (UAV) significantly by extending its range as well as loiter time. This paper has mainly studied on the measurement of refueling drogue during the docking phase of the probe-drogue UAV-AAR. Firstly, the monocular vision based drogue detection and recognition method is presented. Secondly, the measuring method of relative position between the drogue and UAV based on the model of camera calibration is researched. Finally, the semi-physical experimental system of drogue spacial location for UAV-AAR during short-range docking phase is constructed. The drogue measuring experiment based on monocular vision is implemented, and the performance of the monocular vision method for drogue measurement and the validity of the experimental scheme are verified.

Drogue detection for vision-based autonomous aerial refueling via low rank and sparse decomposition with multiple features

The technology of vision-based probe-and-drogue autonomous aerial refueling is an amazing task in modern aviation for both manned and unmanned aircraft. A key issue is to determine the relative orientation and position of the drogue and the probe accurately for relative navigation system during the approach phase, which requires locating the drogue precisely. Drogue detection is a challenging task due to disorderly motion of drogue caused by both the tanker wake vortex and atmospheric turbulence. In this paper, the problem of drogue detection is considered as a problem of moving object detection. A drogue detection algorithm based on low rank and sparse decomposition with local multiple features is proposed. The global and local information of drogue is introduced into the detection model in a unified way. The experimental results on real autonomous aerial refueling videos show that the proposed drogue detection algorithm is effective.