Real Unmanned Aerial Vehicles Research Articles

Determining UAV Flight Trajectory for Target Recognition Using EO/IR and SAR.

The paper presents the concept of planning the optimal trajectory of fixed-wing unmanned aerial vehicle (UAV) of a short-range tactical class, whose task is to recognize a set of ground objects as a part of a reconnaissance mission. Tasks carried out by such systems are mainly associated with an aerial reconnaissance using Electro-Optical/Infrared (EO/IR) systems and Synthetic Aperture Radars (SARs) to support military operations. Execution of a professional reconnaissance of the indicated objects requires determining the UAV flight trajectory in the close neighborhood of the target, in order to collect as much interesting information as possible. The paper describes the algorithm for determining UAV flight trajectories, which is tasked with identifying the indicated objectives using the sensors specified in the order. The presence of UAV threatening objects is taken into account. The task of determining the UAV flight trajectory for recognition of the target is a component of the planning process of the tactical class UAV mission, which is also presented in the article. The problem of determining the optimal UAV trajectory has been decomposed into several subproblems: determining the reconnaissance flight method in the vicinity of the currently recognized target depending on the sensor used and the required parameters of the recognition product (photo, film, or SAR scan), determining the initial possible flight trajectory that takes into account potential UAV threats, and planning detailed flight trajectory considering the parameters of the air platform based on the maneuver planning algorithm designed for tactical class platforms. UAV route planning algorithms with time constraints imposed on the implementation of individual tasks were used to solve the task of determining UAV flight trajectories. The problem was formulated in the form of a Mixed Integer Linear Problem (MILP) model. For determining the flight path in the neighborhood of the target, the optimal control algorithm was also presented in the form of a MILP model. The determined trajectory is then corrected based on the construction algorithm for determining real UAV flight segments based on Dubin curves.

Study of UAV tracking based on CNN in noisy environment

Recently, there are lots of tracking methods proposed to improve the performance of visual tracking in videos with challenging situations, such as background clutter, severe occlusion, rotation, and so on. In real unmanned aerial vehicle (UAV) based tracking systems, there are various noises occurring during video capturing, transmission, and processing. However, most existing studies pay attention to improve the robustness and accuracy of visual tracking while ignoring the performance of tracking methods on videos with noise. In this paper, we investigate the performance evaluation of existing tracking methods on videos with noise. A group of noisy UAV based tracking video datasets are constructed and used to the benchmark datasets for analysis of tracking methods. Furthermore, we propose an algorithm for robustness tracking in noisy videos. The performance of 9 tracking methods is evaluated on the proposed dataset. We provide the detailed analysis and discussion on the robustness analysis of different tracking methods on videos with different variance of noises. Our investigation shows that it is still challenging for effective tracking for existing methods on videos with noise. And our proposed method shows promising results in noisy videos.

Airborne Sensor Data-Based Unsupervised Recursive Identification for UAV Flight Phases

With the widespread application of Unmanned Aerial Vehicle (UAV), more and more attention has been paid to the analysis of flight data, which can realize the condition monitoring and help improve operational safety. The UAV flight phase is dynamically switched, and flight phase identification is the vital basis for accurate flight data analysis. However, UAV flight data do not always contain a parameter that can be used for the direct division of flight phases, and it is difficult to give uniform thresholds for the flight phase identification. To realize the automatic identification of UAV flight phases based on airborne sensor data, an unsupervised Gaussian Mixture Model Recursive Clustering (GMMRC) method is proposed. GMMRC divides a complex flight phase identification task into simple subtasks which are corresponding to UAV operations. It can not only improve the performance of identification, but also enhance the interpretability of the results. In the recursive method, GMM is used as an unsupervised identification operator to obtain the desired identification results in each subtask. To further improve the identification effect, an interval detection method is used to eliminate false results. Experiments based on simulated flight data and real small fixed-wing UAV flight data demonstrate the potential of this method for accurate and robust flight phase identification.

Motion-encoded particle swarm optimization for moving target search using UAVs

This paper presents a novel algorithm named the motion-encoded particle swarm optimization (MPSO) for finding a moving target with unmanned aerial vehicles (UAVs). From the Bayesian theory, the search problem can be converted to the optimization of a cost function that represents the probability of detecting the target. Here, the proposed MPSO is developed to solve that problem by encoding the search trajectory as a series of UAV motion paths evolving over the generation of particles in a PSO algorithm. This motion-encoded approach allows for preserving important properties of the swarm including the cognitive and social coherence, and thus resulting in better solutions. Results from extensive simulations with existing methods show that the proposed MPSO improves the detection performance by 24% and time performance by 4.71 times compared to the original PSO, and moreover, also outperforms other state-of-the-art metaheuristic optimization algorithms including the artificial bee colony (ABC), ant colony optimization (ACO), genetic algorithm (GA), differential evolution (DE), and tree-seed algorithm (TSA) in most search scenarios. Experiments have been conducted with real UAVs in searching for a dynamic target in different scenarios to demonstrate MPSO merits in a practical application.

Real-Time Fault Detection for UAV Based on Model Acceleration Engine

With the wide applications of the unmanned aerial vehicle (UAV) in the civilian and military fields, its operational safety has drawn much attention. A series of fault detection methods are studied to avoid disasters. Due to the capabilities of strong feature extraction and massive flight data processing, the deep learning-based methods have received extensive attention. However, restricted by UAV airborne size, weight, and power consumption, a significant challenge is posed to deploy these complicated detection methods in the airborne application, which requires to run in real time. In this article, a fault detection model acceleration engine (FDMAE) for UAV real-time fault detection is realized under the airborne constraint. First, a high-performance detection model is designed based on stacked long short-term memory networks, and fault detection is achieved by a statistical threshold in this method. Second, a model pruning method based on principal component analysis is proposed to improve computing efficiency. Finally, the pruned fault detection method is optimized and integrated as a flexible acceleration engine through high-level synthesis and deployed on an airborne embedded computing platform based on a field-programmable gate array. Real UAV flight data are used to verify the proposed FDMAE. By comparing accuracy, the area under the receiver operating characteristic curve, speed, and power consumption, the effectiveness of the FDMAE is proven.

Deep Reinforcement Learning Approach with Multiple Experience Pools for UAV's Autonomous Motion Planning in Complex Unknown Environments.

Autonomous motion planning (AMP) of unmanned aerial vehicles (UAVs) is aimed at enabling a UAV to safely fly to the target without human intervention. Recently, several emerging deep reinforcement learning (DRL) methods have been employed to address the AMP problem in some simplified environments, and these methods have yielded good results. This paper proposes a multiple experience pools (MEPs) framework leveraging human expert experiences for DRL to speed up the learning process. Based on the deep deterministic policy gradient (DDPG) algorithm, a MEP–DDPG algorithm was designed using model predictive control and simulated annealing to generate expert experiences. On applying this algorithm to a complex unknown simulation environment constructed based on the parameters of the real UAV, the training experiment results showed that the novel DRL algorithm resulted in a performance improvement exceeding 20% as compared with the state-of-the-art DDPG. The results of the experimental testing indicate that UAVs trained using MEP–DDPG can stably complete a variety of tasks in complex, unknown environments.

Person Re-Identification in Aerial Imagery

Nowadays, with the rapid development of consumer Unmanned Aerial Vehicles (UAVs), visual surveillance by utilizing the UAV platform has been very attractive. Most of the research works for UAV captured visual data are mainly focused on the tasks of object detection and tracking. However, limited attention has been paid to the task of person Re-identification (ReID) which has been widely studied in ordinary surveillance cameras with fixed emplacements. In this paper, to facilitate the research of person ReID in aerial imagery, we collect a large scale airborne person ReID dataset named as Person ReID in Aerial Imagery (PRAI-1581), which consists of 39,461 images of 1581 person identities. The images of the dataset are shot by two DJI consumer UAVs flying at an altitude ranging from 20 to 60 meters above the ground, which covers most of the real UAV surveillance scenarios. In addition, we propose to utilize subspace pooling of convolution feature maps to represent the input person images. Our method can learn a discriminative and compact feature representation for ReID in aerial imagery and can be trained in an end-to-end fashion efficiently. We conduct extensive experiments on the proposed dataset and the experimental results demonstrate that re-identifying persons in aerial imagery is a challenging problem, where our method performs favorably against state of the arts.

UAV Mission Planning with SAR Application.

The paper presents the concept of mission planning for a short-range tactical class Unmanned Aerial Vehicle (UAV) that recognizes targets using the sensors it has been equipped with. Tasks carried out by such systems are mainly associated with aerial reconnaissance employing Electro Optical (EO)/Near Infra-Red (NIR) heads, Synthetic Aperture Radar (SAR), and Electronic Intelligence (ELINT) systems. UAVs of this class are most often used in NATO armies to support artillery actions, etc. The key task, carried out during their activities, is to plan a reconnaissance mission in which the flight route will be determined that optimally uses the sensors’ capabilities. The paper describes the scenario of determining the mission plan and, in particular, the UAV flight routes to which the recognition targets are assigned. The problem was decomposed into several subproblems: assigning reconnaissance tasks to UAVs with choosing the reconnaissance sensors and designating an initial UAV flight plan. The last step is planning a detailed flight route taking into account the time constraints imposed on recognition and the characteristics of the reconnaissance sensors. The final step is to generate the real UAV flight trajectory based on its technical parameters. The algorithm for determining exact flight routes for the indicated reconnaissance purposes was also discussed, taking into account the presence of enemy troops and available air corridors. The task scheduling algorithm—Vehicle Route Planning with Time Window (VRPTW)—using time windows is formulated in the form of the Mixed Integer Linear Problem (MILP). The MILP formulation was used to solve the UAV flight route planning task. The algorithm can be used both when planning individual UAV missions and UAV groups cooperating together. The approach presented is a practical way of establishing mission plans implemented in real unmanned systems.

3D Trajectory Planning Method for UAVs Swarm in Building Emergencies

The development in Multi-Robot Systems (MRS) has become one of the most exploited fields of research in robotics in recent years. This is due to the robustness and versatility they present to effectively undertake a set of tasks autonomously. One of the essential elements for several vehicles, in this case, Unmanned Aerial Vehicles (UAVs), to perform tasks autonomously and cooperatively is trajectory planning, which is necessary to guarantee the safe and collision-free movement of the different vehicles. This document includes the planning of multiple trajectories for a swarm of UAVs based on 3D Probabilistic Roadmaps (PRM). This swarm is capable of reaching different locations of interest in different cases (labeled and unlabeled), supporting of an Emergency Response Team (ERT) in emergencies in urban environments. In addition, an architecture based on Robot Operating System (ROS) is presented to allow the simulation and integration of the methods developed in a UAV swarm. This architecture allows the communications with the MavLink protocol and control via the Pixhawk autopilot, for a quick and easy implementation in real UAVs. The proposed method was validated by experiments simulating building emergences. Finally, the obtained results show that methods based on probability roadmaps create effective solutions in terms of calculation time in the case of scalable systems in different situations along with their integration into a versatile framework such as ROS.

A Rational Polynomial Tracking Control Approach to a Common System Representation for Unmanned Aerial Vehicles

This article presents a rational polynomial tracking control approach to a common system representation for unmanned aerial vehicles (UAVs). First, we newly provide a common system representation of kinematic models for straight and orbit paths. A polynomial representation is introduced to describe the common system. To stabilize the polynomial system, we design a rational polynomial controller using a sum-of-squares (SOS)-based design framework. A set of stabilization conditions considering a real actuator saturation is represented in terms of SOS, where a relaxation for the SOS design conditions is brought by considering practical operation domains. The SOS-based design framework is applied to path stabilization for a real UAV platform (a parafoil wing-type UAV). Experimental results demonstrate the capabilities of the proposed SOS-based design framework in path stabilization of real-world complex UAVs.

Cyber Attacks on Healthcare Devices Using Unmanned Aerial Vehicles.

The growing use of wireless technology in healthcare systems and devices makes these systems particularly open to cyber-based attacks, including denial of service and information theft via sniffing (eaves-dropping) and phishing attacks. Evolving technology enables wireless healthcare systems to communicate over longer ranges, which opens them up to greater numbers of possible threats. Unmanned aerial vehicles (UAV) or drones present a new and evolving attack surface for compromising wireless healthcare systems. An enumeration of the types of wireless attacks capable via drones are presented, including two new types of cyber threats: a stepping stone attack and a cloud-enabled attack. A real UAV is developed to test and demonstrate the vulnerabilities of healthcare systems to this new threat vector. The UAV successfully attacked a simulated smart hospital environment and also a small collection of wearable healthcare sensors. Compromise of wearable or implanted medical devices can lead to increased morbidity and mortality.

Accurate Landing of Unmanned Aerial Vehicles Using Ground Pattern Recognition

Over the last few years, several researchers have been developing protocols and applications in order to autonomously land unmanned aerial vehicles (UAVs). However, most of the proposed protocols rely on expensive equipment or do not satisfy the high precision needs of some UAV applications such as package retrieval and delivery or the compact landing of UAV swarms. Therefore, in this work, a solution for high precision landing based on the use of ArUco markers is presented. In the proposed solution, a UAV equipped with a low-cost camera is able to detect ArUco markers sized 56 × 56 cm from an altitude of up to 30 m. Once the marker is detected, the UAV changes its flight behavior in order to land on the exact position where the marker is located. The proposal was evaluated and validated using both the ArduSim simulation platform and real UAV flights. The results show an average offset of only 11 cm from the target position, which vastly improves the landing accuracy compared to the traditional GPS-based landing, which typically deviates from the intended target by 1 to 3 m.

Interference-Compensating Magnetometer Calibration With Estimated Measurement Noise Covariance for Application to Small-Sized UAVs

This article proposes a new interference-compensating magnetometer calibration scheme aided by a gyroscope sensor, which could reduce the effect of induced magnetic interference due to nearby onboard current flow. By using the innovation process and a linear matrix inequality approach, mean and variance of the induced magnetic interference are estimated to ensure that their physical meaningful values are taken to be closest to the empirically measured ones. The values are reflected in the update step of the extended Kalman filter in order to accurately estimate the calibration parameters and the corresponding direction. Results of experiments performed using a real unmanned aerial vehicle (UAV) demonstrate that the proposed calibration scheme compensates well for the disturbing magnetic interference arising from the UAV's onboard current and hence reduces the estimation error of its yaw angle, or its heading direction, by approximately two-thirds of that obtained using the existing scheme.

Unmanned aerial vehicle (UAV) manufacturing materials: Synthesis, spectroscopic characterization and dynamic mechanical analysis (DMA)

Abstract This study aimed to trace the best prepared composites material to the unmanned aerial vehicles (UAVs) real sample. Carbon composite materials have extraordinary material properties in UAVs manufacturing. In this work, vacuum bag technique is used to prepare rectangular strips of four different carbon fiber composites densities. Spectroscopic characterization and investigating their dynamic mechanical properties have been deployed on them. Fourier transform infrared (FT-IR), energy dispersive X-ray (EDX), and laser induced breakdown spectroscopy (LIBS) techniques were used. The FTIR spectrum of carbon fiber exhibited multiple bands in the region between 1500 and 1000 cm−1 related to stretching vibration of C–O bond from carbonyl functional groups and C–C bond in aromatic rings due to the presence of aromatic molecules in the epoxy resin in the structure of carbon fiber. EDX and LIBS techniques revealed the quantitative and qualitative characteristics of the main constituted elements of prepared fiber. The dynamic mechanical properties were measured under cyclic uniaxial tensile loading at different frequencies where activation energy was calculated. Finally, the results were then compared to the measurements obtained from real UAVs material.

Combined weightless neural network FPGA architecture for deforestation surveillance and visual navigation of UAVs

This work presents a combined weightless neural network architecture for deforestation surveillance and visual navigation of Unmanned Aerial Vehicles (UAVs). Binary images, which are required for position estimation and UAV navigation, are provided by the deforestation surveillance circuit. Learned models are evaluated in a real UAV flight over a green countryside area, while deforestation surveillance is assessed with an Amazon forest benchmarking image data. Small utilization percentage of Field Programmable Gate Arrays (FPGAs) allows for a higher degree of parallelization and block processing of larger regions of input images.

Control Design for UAV Quadrotors via Embedded Model Control

In this paper, a control system for unmanned aerial vehicles (UAVs) is designed, tested in simulation by means of a high-fidelity simulator, and then applied to a real quadrotor UAV. A novel approach is proposed for the control design, based on the combination of two methodologies: feedback linearization (FL) and embedded model control (EMC). FL allows us to properly transform the UAV dynamics into a form suitable for EMC; EMC is then used to control the transformed system. A key feature of EMC is that it encompasses a so-called extended state observer (ESO), which not only recovers the system state but also gives a real-time estimate of all the disturbances/uncertainties affecting the system. This estimate is used by the FL-EMC control law to reject the aforementioned disturbances/uncertainties, including those collected via the FL, allowing a robustness and performance enhancement. This approach allows us to combine FL and EMC strengths. Most notably, the entire process is made systematic and application oriented. To set-up a reliable UAV attitude observer, an effective attitude sensors fusion is proposed and also benchmarked with an enhanced complementary filter. Finally, to enhance the closed-loop performance, a complete tuning procedure, encompassing frequency requirements, is outlined, based on suitably defined stability and performance metrics.

Drones Chasing Drones: Reinforcement Learning and Deep Search Area Proposal

Unmanned aerial vehicles (UAVs) are very popular and increasingly used in different applications. Today, the use of multiple UAVs and UAV swarms are attracting more interest from the research community, leading to the exploration of topics such as UAV cooperation, multi-drone autonomous navigation, etc. In this work, we propose two approaches for UAV pursuit-evasion. The first approach uses deep reinforcement learning to predict the actions to apply to the follower UAV to keep track of the target UAV. The second approach uses a deep object detector and a search area proposal (SAP) to predict the position of the target UAV in the next frame for tracking purposes. The two approaches are promising and lead to a higher tracking accuracy with an intersection over union (IoU) above the selected threshold. We also show that the deep SAP-based approach improves the detection of distant objects that cover small areas in the image. The efficiency of the proposed algorithms is demonstrated in outdoor tracking scenarios using real UAVs.

An Adaptive Framework for Multi-Vehicle Ground Speed Estimation in Airborne Videos

With the rapid development of unmanned aerial vehicles (UAVs), UAV-based intelligent airborne surveillance systems represented by real-time ground vehicle speed estimation have attracted wide attention from researchers. However, there are still many challenges in extracting speed information from UAV videos, including the dynamic moving background, small target size, complicated environment, and diverse scenes. In this paper, we propose a novel adaptive framework for multi-vehicle ground speed estimation in airborne videos. Firstly, we build a traffic dataset based on UAV. Then, we use the deep learning detection algorithm to detect the vehicle in the UAV field of view and obtain the trajectory in the image through the tracking-by-detection algorithm. Thereafter, we present a motion compensation method based on homography. This method obtains matching feature points by an optical flow method and eliminates the influence of the detected target to accurately calculate the homography matrix to determine the real motion trajectory in the current frame. Finally, vehicle speed is estimated based on the mapping relationship between the pixel distance and the actual distance. The method regards the actual size of the car as prior information and adaptively recovers the pixel scale by estimating the vehicle size in the image; it then calculates the vehicle speed. In order to evaluate the performance of the proposed system, we carry out a large number of experiments on the AirSim Simulation platform as well as real UAV aerial surveillance experiments. Through quantitative and qualitative analysis of the simulation results and real experiments, we verify that the proposed system has a unique ability to detect, track, and estimate the speed of ground vehicles simultaneously even with a single downward-looking camera. Additionally, the system can obtain effective and accurate speed estimation results, even in various complex scenes.

Mixed reality and remote sensing application of unmanned aerial vehicle in fire and smoke detection

Abstract This paper proposes the development of a system incorporating inertial measurement unit (IMU), a consumer-grade digital camera and a fire detection algorithm simultaneously with a nano Unmanned Aerial Vehicle (UAV) for inspection purposes. The video streams are collected through the monocular camera and navigation relied on the state-of-the-art indoor/outdoor Simultaneous Localisation and Mapping (SLAM) system. It implements the robotic operating system (ROS) and computer vision algorithm to provide a robust, accurate and unique inter-frame motion estimation. The collected onboard data are communicated to the ground station and used the SLAM system to generate a map of the environment. A robust and efficient re-localization was performed to recover from tracking failure, motion blur, and frame lost in the data received. The fire detection algorithm was deployed based on the color, movement attributes, temporal variation of fire intensity and its accumulation around a point. The cumulative time derivative matrix was utilized to analyze the frame-by-frame changes and to detect areas with high-frequency luminance flicker (random characteristic). Color, surface coarseness, boundary roughness, and skewness features were perceived as the quadrotor flew autonomously within the clutter and congested area. Mixed Reality system was adopted to visualize and test the proposed system in a physical environment, and the virtual simulation was conducted through the Unity game engine. The results showed that the UAV could successfully detect fire and flame, autonomously fly towards and hover around it, communicate with the ground station and simultaneously generate a map of the environment. There was a slight error between the real and virtual UAV calibration due to the ground truth data and the correlation complexity of tracking real and virtual camera coordinate frames.

ADMOST: UAV Flight Data Anomaly Detection and Mitigation via Online Subspace Tracking

Since the control laws require sensor feedback to set the current dynamic state of the unmanned aerial vehicle (UAV), incorrect readings may lead to potentially catastrophic conditions. Thus, automated detection and mitigation of UAV flight data anomaly is an important problem in the aviation domain. However, the conventional anomaly detection algorithms simply detect the outlier points and cannot provide estimated values. To address this challenge, anomaly detection and mitigation algorithm based on online subspace tracking, an online algorithm for flight data anomaly detection and mitigation is proposed. At every time instant, data subspace matrix is used as a meaningful data representation of raw multivariate heterogeneous flight data. Besides, in terms of outlier points, the subspace matrix is tracked with incomplete partial observation. Anomaly score is calculated based on the identification of a change in the underlying subspace. Utilizing the tracked subspace matrix, the detected outlier points are replaced with reasonable recovered estimations, which will help mitigate the influence of anomaly to UAV system control. Experimental results on real UAV flight data demonstrate its ability to maintain high accuracy for anomaly detection and low error for data recovery.