Autonomous Unmanned Aerial Vehicles Research Articles

A Comprehensive Survey of Unmanned Aerial Vehicles Detection and Classification Using Machine Learning Approach: Challenges, Solutions, and Future Directions

Autonomous unmanned aerial vehicles (UAVs) have several advantages in various fields, including disaster relief, aerial photography and videography, mapping and surveying, farming, as well as defense and public usage. However, there is a growing probability that UAVs could be misused to breach vital locations such as airports and power plants without authorization, endangering public safety. Because of this, it is critical to accurately and swiftly identify different types of UAVs to prevent their misuse and prevent security issues arising from unauthorized access. In recent years, machine learning (ML) algorithms have shown promise in automatically addressing the aforementioned concerns and providing accurate detection and classification of UAVs across a broad range. This technology is considered highly promising for UAV systems. In this survey, we describe the recent use of various UAV detection and classification technologies based on ML and deep learning (DL) algorithms. Four types of UAV detection and classification technologies based on ML are considered in this survey: radio frequency-based UAV detection, visual data (images/video)-based UAV detection, acoustic/sound-based UAV detection, and radar-based UAV detection. Additionally, this survey report explores hybrid sensor- and reinforcement learning-based UAV detection and classification using ML. Furthermore, we consider method challenges, solutions, and possible future research directions for ML-based UAV detection. Moreover, the dataset information of UAV detection and classification technologies is extensively explored. This investigation holds potential as a study for current UAV detection and classification research, particularly for ML- and DL-based UAV detection approaches.

EFP: Efficient Frontier-Based Autonomous UAV Exploration Strategy for Unknown Environments

EFP: Efficient Frontier-Based Autonomous UAV Exploration Strategy for Unknown Environments

CHALLENGES AND BARRIERS FOR UNMANNED AERIAL VEHICLE (UAV) IMPLEMENTATION IN MALAYSIAN INFRASTRUCTURE PROJECTS

The usage of Unmanned Aerial Vehicles (UAV) in infrastructure projects has inclined drastic recently due to its benefits. However, there is a slowness in transition from conventional construction approach to new innovation. Besides, there are lots of issues in current practices such as cost overrun, dangerous, delay of time completion and lack of an overall comprehensive scientific system. Furthermore, although there was high level of awareness on the advantages of using UAVs, the Malaysian construction industry are still reluctant for the changes. In response, to overcome issues that the construction industry faces, such as infrastructure projects thus, this study aims to develop a conceptual framework for promoting the applications of UAV in Malaysia infrastructure projects. The methods adopted include explanatory sequential research methods. For instance, questionnaire surveys and interviews are used to collect quantitative and qualitative data, respectively. Besides that, a refined conceptual framework of the challenges and barriers of UAV adoption in Malaysia infrastructure projects were mainly contributed by the restrictive regulatory environment and industry point of view. For example, the privacy issues and guidelines on autonomous UAV and the industry lack of confidence in the adoption of UAV and education on the adoption of the UAV. In addition, it offers some solutions for the issues that have been revealed in the preliminary conceptual framework for the challenges on UAV adoption in Malaysia infrastructure projects. For instance, amaze the contractor with the final data from the UAV. Lastly, the identify solution served as a reference for other Malaysian construction stakeholders that have not used UAVs before. The recommendations for future scholarly research include the roles of UAV technology with other technologies such as Artificial Intelligent (AI) or integration of data with other software.

An oil and gas pipeline inspection UAV based on improved YOLOv7

This study proposes a method of autonomous navigation UAV for oil and gas pipeline (OGP) dial detection based on the improved YOLOv7 model. The canny edge detection algorithm is applied in identifying the edges of the pipeline, and the Hough transform algorithm is used to detect the pipeline in a straight line. The intelligent UAV P600 is guided to patrol the oil and gas dials (OGD) along the pipeline, and the trained improved YOLOv7-based model is adopted to identify the OGD data. Dial recognition is divided into two stages, that is, dial contour detection and dial reading recognition. For the dial recognition rate (RR), the Levenstein distance, a commonly used method, is introduced, thereby calculating the distance between two character sequences. Meanwhile, an integrated global attention mechanism (GAM) is proposed based on the YOLOv7 model, aiming at extracting more informative features. With this mechanism, the channel and spatial aspects of the features are effectively captured, and the importance of cross-dimensional interactions is increased. By introducing GAM attention mechanism in the backbone and head of YOLOv7, the network’s ability in efficiently extracting depth and primary features is enhanced. ACmix (a hybrid model combining the advantages of self-attentiveness and convolution) is also included, with ACmix module improved. The improved ACmix module has the objectives of enhancing feature extraction capability of backbone network and accelerating network convergence. By substituting 3 × 3 convolutional block with 3 × 3 ACmixBlock and adding a jump connection and a 1 × 1 convolutional structure between the ACmixBlock modules, E-ELAN module in YOLOv7 network is also improved, thus optimizing E-ELAN network, enriching features extracted by E-ELAN network, and reducing inference time of YOLOv7 model. As indicated by comparing the experimental results of the six model algorithms (improved YOLOv7, YOLOv7, YOLOX, YOLOv5, YOLOv6 and Faster R-CNN), the improved YOLOv7 model has higher mAP, faster RR, faster network convergence, and higher IOU. In addition, a generic real dataset, called custom dial reading dataset, is presented. With well-defined evaluation protocol, this dataset allows for a fair comparison of various methods in future work.

Trajectory Interception Classification for Prediction of Collision Scope between Moving Objects

In the fields of autonomous navigation and vehicle safety, accurately predicting potential collision field points between moving objects is a significant challenge. A novel computing technique to enhance trajectory interception analysis is presented in this paper. Our objective is to develop a field model that can accurately forecast collision zones, improving road transportation safety and the use of autonomous cars. Our main contribution is a binary classification model called PCSMO (Prediction of Collision Scope between Moving Objects), which is based on zero-shot learning. Gann angles, which are typically 45 degrees, are used to analyze the trajectories of moving objects. This method is inspired by GANN (Gann Angle Numeric Nomenclature). Compared to earlier techniques, this model more accurately identifies potential collision collision interception zones. The technique computes Gann angles for trajectory analysis and extracts GPS coordinates of moving objects from video data using OpenCV. It offers a more sophisticated comprehension of object movement patterns and points of interception. To assess the precision, recall, F1-score, and prediction accuracy of our model, we employ 10-fold cross-validation. Comparing the PCSMO model to existing models, these metrics demonstrate how well the PCSMO model predicts potential collision zones. Our approach, we discovered, enhances trajectory analysis—a critical component of safer autonomous navigation systems. With potential applications in autonomous vehicle and UAV safety, the PCSMO model improves field interception classification.

Towards Fully Autonomous UAV: Damaged Building-Opening Detection for Outdoor-Indoor Transition in Urban Search and Rescue

Autonomous unmanned aerial vehicle (UAV) technology is a promising technology for minimizing human involvement in dangerous activities like urban search and rescue missions (USAR), both in indoor and outdoor. Automated navigation from outdoor to indoor environments is not trivial, as it encompasses the ability of a UAV to automatically map and locate the openings in a damaged building. This study focuses on developing a deep learning model for the detection of damaged building openings in real time. A novel damaged building-opening dataset containing images and mask annotations, as well as a comparison between single and multi-task learning-based detectors are given. The deep learning-based detector used in this study is based on YOLOv5. First, this study compared the different versions of YOLOv5 (i.e., small, medium, and large) capacity to perform damaged building-opening detections. Second, a multitask learning YOLOv5 was trained on the same dataset and compared with the single-task detector. The multitask learning (MTL) was developed based on the YOLOv5 object detection architecture, adding a segmentation branch jointly with the detection head. This study found that the MTL-based YOLOv5 can improve detection performance by combining detection and segmentation losses. The YOLOv5s-MTL trained on the damaged building-opening dataset obtained 0.648 mAP, an increase of 0.167 from the single-task-based network, while its inference speed was 73 frames per second on the tested platform.

Progress in artificial intelligence-based visual servoing of autonomous unmanned aerial vehicles (UAVs)

Unmanned aerial vehicles (UAVs) have attracted massive attention in many engineering and practical applications in the last years for their characteristics and operation flexibility. For the UAV system, suitable control systems are required to operate appropriately and efficiently. An emerging control technique is visual servoing utilizing the onboard camera systems for inspecting the UAV's environment and autonomously controlling the UAV's operation. Artificial intelligence (AI) techniques are widely deployed in the visual servoing of autonomous UAV applications. Despite the increasing research in the field of AI-based visual control of UAV systems, comprehensive review articles that showcase the general trends and future directions in this field of research are limited. This work comprehensively examines the application and advancements of AI-enhanced visual servoing in autonomous UAV systems, covering critical control tasks and offering insights into future research directions for enhancing performance and applicability which is limited in the current literature. The paper first reviews the application of intelligent visual servoing systems for autonomously executing various UAV control tasks, including 3D UAV positioning, aerial and ground object following, obstacle avoidance, and autonomous landing. Second, the research progresses in applying AI techniques in the visual servoing of autonomous UAV systems are discussed and analyzed. Finally, future directions and critical research gaps for further improving the performance and applicability of intelligent visual servoing systems are included.

Framework for Autonomous UAV Navigation and Target Detection in Global-Navigation-Satellite-System-Denied and Visually Degraded Environments

Autonomous Unmanned Aerial Vehicles (UAVs) have possible applications in wildlife monitoring, disaster monitoring, and emergency Search and Rescue (SAR). Autonomous capabilities such as waypoint flight modes and obstacle avoidance, as well as their ability to survey large areas, make UAVs the prime choice for these critical applications. However, autonomous UAVs usually rely on the Global Navigation Satellite System (GNSS) for navigation and normal visibility conditions to obtain observations and data on their surrounding environment. These two parameters are often lacking due to the challenging conditions in which these critical applications can take place, limiting the range of utilisation of autonomous UAVs. This paper presents a framework enabling a UAV to autonomously navigate and detect targets in GNSS-denied and visually degraded environments. The navigation and target detection problem is formulated as an autonomous Sequential Decision Problem (SDP) with uncertainty caused by the lack of the GNSS and low visibility. The SDP is modelled as a Partially Observable Markov Decision Process (POMDP) and tested using the Adaptive Belief Tree (ABT) algorithm. The framework is tested in simulations and real life using a navigation task based on a classic SAR operation in a cluttered indoor environment with different visibility conditions. The framework is composed of a small UAV with a weight of 5 kg, a thermal camera used for target detection, and an onboard computer running all the computationally intensive tasks. The results of this study show the robustness of the proposed framework to autonomously explore and detect targets using thermal imagery under different visibility conditions. Devising UAVs that are capable of navigating in challenging environments with degraded visibility can encourage authorities and public institutions to consider the use of autonomous remote platforms to locate stranded people in disaster scenarios.

An autonomous Unmanned Aerial Vehicle exploration platform with a hierarchical control method for post‐disaster infrastructures

AbstractCatastrophic natural disasters like earthquakes can cause infrastructure damage. Emergency response agencies need to assess damage precisely while repeating this process for infrastructures with different shapes and types. The authors aim for an autonomous Unmanned Aerial Vehicle (UAV) platform equipped with a 3D LiDAR sensor to comprehensively and accurately scan the infrastructure and map it with a predefined resolution r. During the inspection, the UAV needs to decide on the Next Best View (NBV) position to maximize the gathered information while avoiding collision at high speed. The authors propose solving this problem by implementing a hierarchical closed‐loop control system consisting of a global planner and a local planner. The global NBV planner decides the general UAV direction based on a history of measurements from the LiDAR sensor, and the local planner considers the UAV dynamics and enables the UAV to fly at high speed with the latest LiDAR measurements. The proposed system is validated through the Regional Scale Autonomous Swarm Damage Assessment simulator, which is built by the authors. Through extensive testing in three unique and highly constrained infrastructure environments, the autonomous UAV inspection system successfully explored and mapped the infrastructures, demonstrating its versatility and applicability across various shapes of infrastructure.

A Fuzzy-Based System for Autonomous Unmanned Aerial Vehicle Ship Deck Landing.

This paper introduces a fuzzy logic-based autonomous ship deck landing system for fixed-wing unmanned aerial vehicles (UAVs). The ship is assumed to maintain a constant course and speed. The aim of this fuzzy logic landing model is to simplify the task of landing UAVs on moving ships in challenging maritime conditions, relieving operators from this demanding task. The designed UAV ship deck landing model is based on a fuzzy logic system (FLS), which comprises three interconnected subsystems (speed, lateral motion, and altitude components). Each subsystem consists of three inputs and one output incorporating various fuzzy rules to account for external factors during ship deck landings. Specifically, the FLS receives five inputs: the range from the deck, the relative wind direction and speed, the airspeed, and the UAV's flight altitude. The FLS outputs provide data on the speed of the UAV relative to the ship's velocity, the bank angle (BA), and the angle of descent (AOD) of the UAV. The performance of the designed intelligent ship deck landing system was evaluated using the standard configuration of MATLAB Fuzzy Toolbox.

Rule-Based Verification of Autonomous Unmanned Aerial Vehicles

Automation enhances the capabilities of unmanned aerial vehicles (UAVs) by enabling self-determined behavior, while reducing the need for extensive human involvement. Future concepts envision a single human operator commanding multiple autonomous UAVs with minimal supervision. Despite advances in automation, there remains a demand for a “human in command” to assume overall responsibility, driven by concerns about UAV safety and regulatory compliance. In response to these challenges, a method for runtime verification of UAVs using a knowledge-based system is introduced. This method empowers human operators to identify unsafe behavior without assuming full control of the UAV. Aspects of automated formalization, updating and processing of knowledge elements at runtime, coupled with an automatic reasoning process, are considered. The result is an ontology-based approach for runtime verification, addressing the growing complexity of UAVs and the need to ensure safety in the context of evolving aviation regulations.

An autonomous UAV system based on adaptive LiDAR inertial odometry for practical exploration in complex environments

AbstractUnmanned aerial vehicles (UAVs) offer many advantages over ground vehicles, including quadruped robots, based on high maneuverability when performing exploration in complex and unknown environments. However, due to their limited computational capability, UAVs require lightweight but accurate state estimation algorithms for reliable exploration. In this paper, we propose a segmented map‐based exploration system based on light detection and ranging (LiDAR)‐based state estimation for UAVs. The proposed system includes capabilities such as exploration, obstacle avoidance, and object detection with localization using three‐dimensional (3D) dense maps generated by tightly coupled LiDAR Inertial Odometry (LIO). Our proposed system is a hybrid system that can switch between guided and exploration modes, making it practical for search and rescue missions in disaster scenarios. The proposed LIO algorithm adapts to its surroundings, allowing for fast and accurate state estimation in complex environments. The proposed exploration algorithm is designed to cover specific regions in the 3D dense map generated by the proposed LIO, with the UAV determining if map points are included within the coverage area. We tested the proposed system in both simulation and real‐world environments and validated that the proposed system outperforms state‐of‐the‐art algorithms in various aspects such as localization accuracy and exploration efficiency in complex environments.

A fast, lightweight deep learning vision pipeline for autonomous UAV landing support with added robustness

Despite massive development in aerial robotics, precise and autonomous landing in various conditions is still challenging. This process is affected by many factors, such as terrain shape, weather conditions, and the presence of obstacles. This paper describes a deep learning-accelerated image processing pipeline for accurate detection and relative pose estimation of the UAV with respect to the landing pad. Moreover, the system provides increased safety and robustness by implementing human presence detection and error estimation for both landing target detection and pose computation. Human presence and landing pad location are performed by estimating the presence probability via segmentation. This is followed by the landing pad keypoints’ location regression algorithm, which, in addition to coordinates, provides the uncertainty of presence for each defined landing pad landmark. To perform the aforementioned tasks, a set of lightweight neural network models was selected and evaluated. The resulting measurements of the system’s performance and accuracy are presented for each component individually and for the whole processing pipeline. The measurements are performed using onboard embedded UAV hardware and confirm that the method can provide accurate, low-latency feedback information for safe landing support.

A review of perception sensors, techniques, and hardware architectures for autonomous low-altitude UAVs in non-cooperative local obstacle avoidance

Unmanned Aerial Vehicles (UAVs) can detect and communicate with cooperative obstacles through established protocols. However, non-cooperative obstacles pose a significant threat to UAVs during low-flight operations. These obstacles include static obstacles like buildings, trees, or communication towers and dynamic objects like other UAVs. The application of autonomous UAVs in low-altitude surveillance has motivated research into non-cooperative local obstacle avoidance. This paper provides an overview of such solutions that have been proposed within the last decade. Unlike most literature that limits obstacle avoidance to algorithms, this work provides an in-depth review of obstacle avoidance components, namely the perception sensor, techniques, and hardware architecture of the obstacle avoidance system. This review categorizes the non-cooperative obstacle avoidance techniques into four groups: gap-based methods, geometric methods, repulsive force-based methods, and Artificial Intelligence (AI) based methods. This paper provides a comprehensive resource for researchers working on collision-free surveillance by autonomous UAVs at low altitudes.

UAVs for forestry: Metric-semantic mapping and diameter estimation with autonomous aerial robots

To properly monitor the growth of forests and administer effective methods for their cultivation, forestry researchers require access to quantitative metrics such as diameter at breast height and stem taper profile of trees. These metrics are tedious and labor-intensive to measure by hand, especially at the scale of vast forests with thick undergrowth. Autonomous mobile robots can help to scale up such operations and provide an efficient method to capture the data. We present a set of algorithms for autonomous navigation and fine-grained metric-semantic mapping with a team of aerial robots in under-canopy forest environments. Our autonomous UAV system has 3D flight capabilities and relies only on a LIDAR and an IMU for state estimation and mapping. This allows each robot to accurately navigate in challenging forest environments with drastic terrain changes regardless of illumination conditions. Our deep-learning-driven fine-grained metric-semantic mapping module is capable of detecting and extracting detailed information such as the position, orientation, and stem taper profile of trees. This map of tree trunks is represented as a set of sparse cylinder models. Our semantic place recognition module leverages this sparse representation to efficiently estimate the relative transformation between multiple robots, and merge their information to build a globally consistent large-scale map. This ultimately allows us to scale up operations with multiple robots. Our system is able to achieve a mean absolute error of 1.45 cm for diameter estimation and 13.2 cm for relative position estimation between a pair of robots after place recognition and map merging

Learning Optimizer-Based Visual Analytics Method to Detect Targets in Autonomous Unmanned Aerial Vehicles

Visual analytics is vital for identifying targets by differentiating their structure and texture from unmanned aerial vehicles (UAVs). Object sensing disturbance and swift texture differentiation are tedious due to the UAV displacements. For improving the accuracy of target detection, this article introduces a learning optimizer-based visual analytics method. The proposed method assimilates deep learning and a gradient descent algorithm for feature differentiation and error minimization concurrently. The captured images are identified using multiple structural feature variations and are correlated with similar stored images. The features are extracted at different displacement and structural changing instances for leveraging accuracy. The learning process trains the similarity features during different differentiation factors. In the feature extraction, the minimum slope points are identified using a gradient descent algorithm by assigning random weights. As the differentiation increases using similar features, the minimum similarity value is detection. Postdetection, the weights are incremental and linear across different feature slopes. Therefore, the accuracy increases under varying displacement instances, preventing target misdetection. The gradient function is invariable between the minimum and maximum values for identifying high-precision features. This ensures optimal detection of different buildings and structures with high accuracy.

Control System for Quadcopter UAV based SMC-RBFNN with External Disturbance

Unmanned aerial vehicles (UAVs) can either be flown autonomously or remotely by a pilot. Due to its many benefits, including the capacity to take off and land vertically and the ability to take off and land in a small space, this form of UAV quadcopter is currently the subject of extensive research. An autonomous UAV is being developed to reduce the likelihood of pilot operating errors when managing the UAV. The quadcopter dynamic system in this study was controlled primarily by a radial basis function neural network (RBFNN), and its performance was evaluated using simulation on a test track with outside disturbances. One test track is used for the simulation, and there are no outside disturbances. Input of external noise occurs concurrently for x, y, and z coordinates. The average of error for the control system SMC and SMC-RBFNN without disturbance is 0 according to the simulation results. Additionally, the SMC control system’s of error with external disturbances is 0.74, whereas it is 0.54 for the SMC-RBFNN control system. This is demonstrated by the system’s ability to return to the test track at the present within 9 seconds while employing the SMC-RBFNN controller. In contrast, the system can reach the test track in 18 seconds while using the SMC. The SMC- RBFNN is one of the suitable control strategies for flight missions with external disturbances, it may be inferred.

ТАКСОНОМІЧНИЙ МЕТОД ВИБОРУ СПОСОБУ ВЕКТОРИЗАЦІЇ ПОТОЧНОГО ЗОБРАЖЕННЯ ДЛЯ ОПТИКО-ЕЛЕКТРОННОГО МОДУЛЯ БЕЗПІЛОТНОГО ЛІТАЛЬНОГО АПАРАТА

In the article, a taxonomic method for selecting the optimal vectorization approach for processing current images in the optical-electronic module of unmanned aerial vehicles (UAVs) was developed. The primary goal of this method is to optimize UAV navigation systems, which is particularly relevant due to the need to improve data processing efficiency in conditions of limited access to satellite navigation systems. This is critically important for ensuring high autonomy and accuracy of UAVs during the performance of various tasks. The need for such a method arose from the specific challenges faced by UAVs when operating in complex and unpredictable conditions. Analysis has shown that the use of vector images can significantly reduce the time required for information processing. This is possible due to the smaller volume of data that needs to be processed compared to traditional raster images. Reducing information processing time is critically important for the operational capability of UAVs to respond to dynamic changes in the operational environment, especially in critical situations. The taxonomic approach applied in the study allows for systematic classification and evaluation of different vectorization methods in terms of their processing speed, accuracy of results, and data retention efficiency. This enables the identification of the most suitable method for the specific operational conditions in which the UAV is used, thereby optimizing the overall system performance. Furthermore, the results of this research are significant not only in the scientific field but also in practical applications. By improving autonomous navigation systems for UAVs, the study directly contributes to enhancing their effectiveness in complex and unpredictable conditions. This is particularly relevant for missions that require a high level of precision and reliability, such as those performed by military UAVs and emergency response. Enhanced navigation capabilities allow these UAVs to perform complex maneuvers and make real-time decisions based on processed vectorized images, thereby reducing the risk of errors and increasing mission success rates. An improved vectorization method could lead to significant enhancements in UAV navigation systems, making them more resistant to failures and capable of operating in less structured environments without heavy reliance on satellite signals. This enhancement is vitally important for both military and civilian UAV applications, where operational flexibility and reliability are extremely important.

Topical issues of countering modern autonomous unmanned aerial vehicles and FPV drones

The high level of development of unmanned aviation has predetermined the possibility of its use to solve a wide range of tasks. At the same time, it should be noted that achievements in this area are not always used for peaceful purposes. The results of the analysis of the practice of using unmanned aerial vehicles (UAVs) in modern military conflicts and information about terrorist acts with their use allow us to conclude that the most difficult targets resistant to various methods of influence are modern autonomous UAVs and FPV drones with explosive devices. Therefore, the search for effective ways to counteract them seems to be an urgent area of research. The aim of the work is to develop the means for effective counteraction to modern autonomous UAVs and FPV drones by electromagnetic, laser and mechanical ways. The methods of system analysis are used in the work. The article notes the increasing level of threats associated with the mass use of autonomous UAVs and FPV drones, presents the results of the "revision" and critical analysis of the main methods of countering modern UAVs, reflecting their characteristics, advantages and disadvantages. Based on the results obtained, a comparative analysis of methods of countering autonomous UAVs and FPV drones was carried out. It is concluded that the most effective of them are the methods of electromagnetic, laser and mechanical action. The required parameters for the effective application of these types of impacts on modern autonomous UAVs and FPV drones are presented. The results of the research can be used as initial data for the creation of new and improvement of existing means of countering UAVs as part of physical protection systems (SFZ). The scientific novelty of the work consists in the development of a scientific and methodological apparatus for substantiating the SFZ of objects in terms of taking into account the functioning features and vulnerabilities of modern autonomous UAVs and FPV drones, as well as determining ways to improve systems to counter them based on the use of electromagnetic, laser and mechanical means.

UAVs for Disaster Management - An Exploratory Review

The following research article constitutes a comprehensive review of state-of-the-art methodologies and problems addressed by current academic literature in the field of deploying autonomous Unmanned Aerial Vehicles (UAVs) for disaster management. The work compares several papers to identify the current trend of the field. It also presents an in-depth analysis of possible future research directions in the domain.