Real-time Unmanned Aerial Vehicles Research Articles

Smooth prediction and spatio-temporal regularization correlation filters for real-time unmanned aerial vehicle tracking

Smooth prediction and spatio-temporal regularization correlation filters for real-time unmanned aerial vehicle tracking

NavBLIP: a visual-language model for enhancing unmanned aerial vehicles navigation and object detection

IntroductionIn recent years, Unmanned Aerial Vehicles (UAVs) have increasingly been deployed in various applications such as autonomous navigation, surveillance, and object detection. Traditional methods for UAV navigation and object detection have often relied on either handcrafted features or unimodal deep learning approaches. While these methods have seen some success, they frequently encounter limitations in dynamic environments, where robustness and computational efficiency become critical for real-time performance. Additionally, these methods often fail to effectively integrate multimodal inputs, which restricts their adaptability and generalization capabilities when facing complex and diverse scenarios.MethodsTo address these challenges, we introduce NavBLIP, a novel visual-language model specifically designed to enhance UAV navigation and object detection by utilizing multimodal data. NavBLIP incorporates transfer learning techniques along with a Nuisance-Invariant Multimodal Feature Extraction (NIMFE) module. The NIMFE module plays a key role in disentangling relevant features from intricate visual and environmental inputs, allowing UAVs to swiftly adapt to new environments and improve object detection accuracy. Furthermore, NavBLIP employs a multimodal control strategy that dynamically selects context-specific features to optimize real-time performance, ensuring efficiency in high-stakes operations.Results and discussionExtensive experiments on benchmark datasets such as RefCOCO, CC12M, and Openlmages reveal that NavBLIP outperforms existing state-of-the-art models in terms of accuracy, recall, and computational efficiency. Additionally, our ablation study emphasizes the significance of the NIMFE and transfer learning components in boosting the model's performance, underscoring NavBLIP's potential for real-time UAV applications where adaptability and computational efficiency are paramount.

Relay Selection Technique for Energy-Efficient Data Transmission Policy for UAV-Assisted LoRaWAN

An unmanned aerial vehicle (UAV), is an aircraft that does not have a human pilot and is controlled either independently by the aircraft's computers or by remote control. Because of their high mobility, FANETs are often used in UAVs because they can disrupt communications as well as network stability due to frequent topology changes. In-depth exploration and implementation of UAV network development, concentrating on diverse topologies such as star, multi-star, and network configurations, we developed the Energy-Efficient Data Transmission Policy for UAV (EEDTP-UAV) model. It emphasizes mobility, energy efficiency, UAV distance control, and road availability in the context of Flying Ad-hoc Network (FANET) technology development. Special attention is given to path optimization techniques and different link types, which are crucial for effective UAV communication. By using a relay-based energy-efficient process, off-grid user connections are optimized according to energy consumption meters, and data is collected cooperatively between the UAV cluster and the mobile synchronous node, providing equal energy usage and efficient data transmission. Due to its flexibility and adaptive nature, the proposed algorithm is well-suited for real-time UAV swarm operations that tackle dynamic path planning, and energy efficient communication methods such as airborne navigation or crowd control. Compared with existing model EEDS, DSSPCA, EEUCH, and ESRD-PDCA models with our proposed EEDTP-UAV model. By the following parameters communication delay, energy efficiency, data success rate, throughput, and routing overhead have been calculated.

Real-Time UAV Recognition Through Advanced Machine Learning for Enhanced Military Surveillance

In an era where the military utilization of Unmanned Aerial Vehicles (UAVs) has become essential for surveillance and operational operations, our study tackles the growing demand for real-time, accurate UAV recognition. The rise of UAVs presents numerous safety hazards, requiring systems that distinguish UAVs from non-threatening phenomena, such as birds. This research study conducts a comparative examination of advanced machine learning models, aiming to address the challenge of real-time aerial classification in diverse environmental conditions without model retraining. This research employs extensive datasets to train and validate models such as Neural Networks, Support Vector Machines, ensemble methods, and Gradient Boosting Machines. The fashions are evaluated based on accuracy, forgetfulness, and processing efficiency—criteria determining the viability of real-time operational scenarios. The findings indicate that Neural Networks exhibit enhanced performance, demonstrating exceptional accuracy in distinguishing UAVs from birds. This culminates in our primary assertion: Neural Networks possess vital operational security ramifications and can markedly enhance the allocation of defense resources. The findings significantly improve surveillance systems, highlighting the effectiveness of machine-learning methods in real-time UAV identification. Moreover, incorporating Neural Network systems into military defenses is recommended to enhance decision-making capabilities and security operations. Foresee forthcoming UAV developments and advocate for regular model updates to keep up with increasingly nimble and perhaps stealthier drone designs.

ANALYSIS OF EXISTING OBJECT DETECTION MODELS FOR UAV DETECTION ON THERMAL IMAGERY

Nowadays, unmanned aerial vehicles (UAVs) are becoming widely used. They have proven their effectiveness, reliability, and feasibility. However, improper use or abuse of this technology can lead to significant human rights violations and threaten public safety. In connection with this, appropriate methods of countering UAVs should be made. One of the areas that is actively developing nowadays is the detection of UAVs based on optoelectronic radiation. Since most of such systems are designed for the visible range, the IR range has not been so widely investigated. UAV detection based on thermal imaging can be performed using artificial intelligence. Currently, there are ready-made approaches and models that perform object detection tasks, but they are more general purpose. To review the problem of UAV detection based on thermal imaging and AI, it is necessary to analyze the most effective object detection models for the problem of UAV detection in the IR range. In this work, an analysis and comparison of such object detection models as YOLOv5 and YOLOv8, Faster RCNN, DETR was carried out. Datasets were used for model training, both in the IR range and in the visible range. Preliminary digital processing of the dataset of UAV images in the visible range was carried out to convert them into pseudo-thermal images. This was done in order to increase the amount of training data for the models and thereby improve their accuracy. The results showed that models such as YOLOv8 and DETR are the most effective for UAV detection tasks in thermal imaging, but their accuracy is still insufficient to be effectively used for real-time UAV defense systems.

Adaptive Path Planning for Multi-UAV Systems in Dynamic 3D Environments: A Multi-Objective Framework

This study evaluates and compares the computational performance and practical applicability of advanced path planning algorithms for Unmanned Aerial Vehicles (UAVs) in dynamic and obstacle-rich environments. The Adaptive Multi-Objective Path Planning (AMOPP) framework is highlighted for its ability to balance multiple objectives, including path length, smoothness, collision avoidance, and real-time responsiveness. Through experimental analysis, AMOPP demonstrates superior performance, with a 15% reduction in path length compared to A*, achieving an average path length of 450 m. Its angular deviation of 8.0° ensures smoother trajectories than traditional methods like Genetic Algorithm and Particle Swarm Optimization (PSO). Moreover, AMOPP achieves a 0% collision rate across all simulations, surpassing heuristic-based methods like Cuckoo Search and Bee Colony Optimization, which exhibit higher collision rates. Real-time responsiveness is another key strength of AMOPP, with an average re-planning time of 0.75 s, significantly outperforming A* and RRT*. The computational complexities of each algorithm are analyzed, with AMOPP exhibiting a time complexity of O(k·n) and a space complexity of O(n), ensuring scalability and efficiency for large-scale operations. The study also presents a comprehensive qualitative and quantitative comparison of 14 algorithms using 3D visualizations, highlighting their strengths, limitations, and suitable application scenarios. By integrating weighted optimization with penalty-based strategies and spline interpolation, AMOPP provides a robust solution for UAV path planning, particularly in scenarios requiring smooth navigation and adaptive re-planning. This work establishes AMOPP as a promising framework for real-time, efficient, and safe UAV operations in dynamic environments.

A Digital Management System for Monitoring Epidemics and the Management of Pine Wilt Disease in East China

The precise monitoring of forest pest and disease outbreaks is a crucial prerequisite for efficient prevention and control. With the extensive application of remote sensing monitoring technology in the forest, a large amount of data on pest and disease outbreaks has been collected. It is highly necessary to practically apply these data and improve the efficiency of forest pest and disease monitoring and management. In this study, a Digital Forest Protection (DFP) system based on the geographic information system (GIS) was designed and developed for pine wilt disease (PWD) monitoring and management, a devastating forest disease caused by the pine wood nematode, Bursaphelenchus xylophilus. The DFP system consists of a mobile app for data collection and a web-based data analysis platform. Meanwhile, artificial intelligence and deep-learning methods had been conducted to integrate a real-time unmanned aerial vehicle (UAV) remote sensing monitoring with PWD detection. This system was implemented in PWD monitoring and management in Zhejiang Province, China, and has been applied in data collection under certain circumstances, including the manual epidemic survey, the UAV epidemic survey, and eradication monitoring, as well as trunk injection. Based on DFP system, the effective monitoring of PWD outbreaks could be achieved, and corresponding efficient management strategies could be formulated in a timely manner. This allows for the possibility to optimize the integrated management strategy of PWD on a large geographic scale.

Kinodynamic Model-Based UAV Trajectory Optimization for Wireless Communication Support of Internet of Vehicles in Smart Cities

Unmanned aerial vehicles (UAVs) are utilized for wireless communication support of Internet of Intelligent Vehicles (IoVs). Intelligent vehicles (IVs) need vehicle-to-vehicle (V2V) and vehicle-to-everything (V2X) wireless communication for real-time perception knowledge exchange and dynamic environment modeling for safe autonomous driving and mission accomplishment. UAVs autonomously navigate through dense urban areas to provide aerial line-of-sight (LoS) communication links for IoVs. Real-time UAV trajectory design is required for minimum energy consumption and maximum channel performance. However, this is multidisciplinary research including (1) dynamic-aware kinematic (kinodynamic) planning by considering UAVs’ motion and nonholonomic constraints; (2) channel modeling and channel performance improvement in future wireless networks (i.e., beyond 5G and 6G) that are limited to beamforming to LoS links with the aid of reconfigurable intelligent surfaces (RISs); and (3) real-time obstacle-free crash avoidance 3D trajectory optimization in dense urban areas by modeling obstacles and LoS paths in convex programming. Modeling and solving this multilateral problem in real-time are computationally prohibitive unless extensive computational and overhead processing costs are imposed. To pave the path for computationally efficient yet feasible real-time trajectory optimization, this paper presents UAV kinodynamic modeling. Then, it proposes a convex trajectory optimization problem with the developed linear kinodynamic models. The optimality and smoothness of the trajectory optimization problem are improved by utilizing model predictive control and quadratic state feedback control. Simulation results are provided to validate the methodology.

AI in the Sky: Developing Real-Time UAV Recognition Systems to Enhance Military Security

In an era where Unmanned Aerial Vehicles (UAVs) have become crucial in military surveillance and operations, the need for real-time and accurate UAV recognition is increasingly critical. The widespread use of UAVs presents various security threats, requiring systems that can differentiate between UAVs and benign objects, such as birds. This study conducts a comparative analysis of advanced machine learning models to address the challenge of aerial classification in diverse environmental conditions without system redesign. Large datasets were used to train and validate models, including Neural Networks, Support Vector Machines, ensemble methods, and Random Forest Gradient Boosting Machines. These models were evaluated based on accuracy and computational efficiency, key factors for real-time application. The results indicate that Neural Networks provide the best performance, demonstrating high accuracy in distinguishing UAVs from birds. The findings emphasize that Neural Networks have significant potential to enhance operational security and improve the allocation of defense resources. Overall, this research highlights the effectiveness of machine learning in real-time UAV recognition and advocates for the integration of Neural Networks into military defense systems to strengthen decision-making and security operations. Regular updates to these models are recommended to keep pace with advancements in UAV technology, including more agile and stealthier designs

Small Object Detection in UAV Images Based on YOLOv8n

With the rapid development of unmanned aerial vehicle (UAV) technology, there is an urgent need for high-performance aerial object detection algorithms that are tailored for deployment on drones with limited computing capabilities. This paper proposes a series of improvements to the state-of-the-art YOLOv8 object detector to enhance its detection accuracy and speed for small, partially occluded objects in complex environments. Specifically, we introduce multi-scale feature fusion through additional detection layers, employ conditionally parameterized convolutions to increase representational capacity, and import a dynamic non-monotonic loss function named Wise-IoU to enable more effective regression of bounding boxes. Experiments conducted on the large-scale UAV benchmark dataset VisDrone demonstrate that the improved model achieves state-of-the-art accuracy of 37.6% mAP with 3 M parameters, outperforming other lightweight YOLO detectors and two-stage detectors like Faster R-CNN. The improved model also reaches 40 FPS on an embedded edge device, validating its efficiency and suitability for real-time UAV applications. Through comprehensive quantitative experiments and visual results, this work provides valuable insights and techniques to tailor object detection algorithms for robust and efficient deployment on UAVs with limited onboard computing power.

UAV-Assisted Mobile Edge Computing: Dynamic Trajectory Design and Resource Allocation.

The recent advancements of mobile edge computing (MEC) technologies and unmanned aerial vehicles (UAVs) have provided resilient and flexible computation services for ground users beyond the coverage of terrestrial service. In this paper, we focus on a UAV-assisted MEC system in which the UAV equipped with MEC servers is used to assist user devices in computing their tasks. To minimize the weighted average energy consumption and delay in the UAV-assisted MEC system, a LQR-Lagrange-based DDPG (LLDDPG) algorithm, which jointly optimizes the user task offloading and the UAV trajectory design, is proposed. To be specific, the LLDDPG algorithm consists of three subproblems. The DDPG algorithm is used to address the issue of UAV desired trajectory planning, and subsequently, the LQR-based algorithm is employed to achieve the real-time tracking control of UAV desired trajectory. Finally, the Lagrange duality method is proposed to solve the optimization problem of computational resource allocation. Simulation results indicate that the proposed LLDDPG algorithm can effectively improve the system resource management and realize the real-time UAV trajectory design.

Light-weight Fusion Network for UAV Visible-light and Infrared Images based on Real-time Brain-like Intelligent Computing

Abstract. Multiple sensors equipped on the Unmanned Aerial Vehicle (UAV) enables the acquisition of multi-modal and multi-source remote sensing data. UAV remote sensing usually faces with real-time or near-real-time tasks in complex and highly dynamic environments, such as disaster monitoring, traffic management, border patrol and so on. Under these conditions, the image fusion algorithm needs to be high efficiency, precision and reliability. In this paper, we proposed an intelligent real-time fusion network for UAV multi-source remote sensing data based on AI brain-like chips, and deployed the algorithm on the UAV platform to achieve online high-efficiency computing. Firstly, we have developed a novel image fusion algorithm named SFNet for infrared and visible image fusion based on ShuffleNetv2. Then, we use ZCA and l1-norm to process the remodeled deep feature. The weight maps are generated by bi-cubic interpolation and soft-max operation. Finally, the fused image is reconstructed by weighted-average operation. The proposed SFNet is deployed on the Lynxi KA200 brain computing chip, and a comprehensive inference test is carried out with UAV remote sensing data. Several State-Of-The-Art (SOTA) data fusion algorithms are deployed on the same chip for experimental comparison. The proposed SFNet is proved to have faster inference speed and better feature extraction results on brain-like chips. It is more suitable for real-time UAV remote sensing image fusion tasks.

Development of Real-Time Unmanned Aerial Vehicle Urban Object Detection System with Federated Learning

In this paper, an urban object detection system via unmanned aerial vehicles (UAVs) is developed to collect real-time traffic information, which can be further utilized in many applications such as traffic monitoring and urban traffic management. The system includes an object detection algorithm, deep learning model training, and deployment on a real UAV. For the object detection algorithm, the Mobilenet-SSD model is applied owing to its lightweight and efficiency, which make it suitable for real-time applications on an onboard microprocessor. For model training, federated learning (FL) is used to protect privacy and increase efficiency with parallel computing. Last, the FL-trained object detection model is deployed on a real UAV for real-time performance testing. The experimental results show that the object detection algorithm can reach a speed of 18 frames per second with good detection performance, which shows the real-time computation ability of a resource-limited edge device and also validates the effectiveness of the developed system.

AI Driven Applications for Precision Weed Management in Agricultural Crops

Agriculture plays a most important role in our Indian economy and therefore lowering the cost of production and improving the quality of agricultural products is highly demanded. A weed is a plant which grows in wrong place at the wrong time and doing mostly harm than crops. Weed competes with the crops for water, light, nutrients and space, and therefore it prevents crop yields. This paper proposes a new method in a contrary way, which combines deep learning and image processing technology to prevent these weeds. Machine learning technologies, are becoming crucial in agriculture to increase productivity, where advanced automation and control have been required. Based on large training datasets and pre-trained models, (Deep Learning) DL-based Convolutional Neural Networks (CNN) methods have proven to be more accurate than previous traditional techniques. Recently, Deep Learning (DL) has gained much attention due to its advantages in object detection, classification, and feature extraction. The system implementation of image processing technique for weed detection, a trained image is taken as a sample in order to demonstrate the difference between weed and the crop. Yolo frame work is used for annotate boundary boxes to the image with datasets. The effectiveness of the (You Only Live Once) YOLO-WEED system for real-time Unmanned Aerial Vehicle (UAV) weed detection, given its high speed and high accuracy in detection. After certain steps, we get desired output, where the weeds are separated from the crop that has been taken in the sample image. Key Words: DeepLearning,CNN, Unmanned Aerial Vehicle (UAV), Precision Weed Detection.

Real-Time 3D Routing Optimization for Unmanned Aerial Vehicle using Machine Learning

In the realm of Unmanned Aerial Vehicles (UAVs) for civilian applications, the surge in demand has underscored the need for sophisticated technologies. The integration of Unmanned Aerial Systems (UAS) with Artificial Intelligence (AI) has become paramount to address challenges in urban environments, particularly those involving obstacle collision risks. These UAVs are equipped with advanced sensor arrays, incorporating LiDAR and computer vision technologies. The AI algorithm undergoes comprehensive training on an embedded machine, fostering the development of a robust spatial perception model. This model enables the UAV to interpret and navigate through the intricate urban landscape with a human-like understanding of its surroundings. During mission execution, the AI-driven perception system detects and localizes objects, ensuring real-time awareness. This study proposes an innovative real-time three-dimensional (3D) path planner designed to optimize UAV trajectories through obstacle-laden environments. The path planner leverages a heuristic A* algorithm, a widely recognized search algorithm in artificial intelligence. A distinguishing feature of this proposed path planner is its ability to operate without the need to store frontier nodes in memory, diverging from conventional A* implementations. Instead, it relies on relative object positions obtained from the perception system, employing advanced techniques in simultaneous localization and mapping (SLAM). This approach ensures the generation of collision-free paths, enhancing the UAV's navigational efficiency. Moreover, the proposed path planner undergoes rigorous validation through Software-In-The-Loop (SITL) simulations in constrained environments, leveraging high-fidelity UAV dynamics models. Preliminary real flight tests are conducted to assess the real-world applicability of the system, considering factors such as wind disturbances and dynamic obstacles. The results showcase the path planner's effectiveness in providing swift and accurate guidance, thereby establishing its viability for real-time UAV missions in complex urban scenarios.

Integrated the Artificial Potential Field with the Leader–Follower Approach for Unmanned Aerial Vehicles Cooperative Obstacle Avoidance

For the formation and obstacle avoidance challenges of UAVs (unmanned aerial vehicles) in complex scenarios, this paper proposes an improved collaborative strategy based on APF (artificial potential field). This strategy combines graph theory, the Leader–Follower method, and APF. Firstly, we used graph theory to design formation topology and dynamically adjust the distances between UAVs in real time. Secondly, we introduced APF to avoid obstacles in complicated environments. This algorithm innovatively integrates the Leader–Follower formation method. The design of this attractive field is replaced by the leader’s attraction to the followers, overcoming the problem of unreachable targets in APF. Meanwhile, the introduced Leader–Follower mode reduces information exchange within the swarm, realizing a more efficient “few controlling many” paradigm. Afterwards, we incorporated rotational force to assist the swarm in breaking free from local minima. Ultimately, the stability of the integrated formation strategy was demonstrated using Lyapunov functions. The feasibility and effectiveness of the proposed strategy were validated across multiple platforms.

AI-Embedded UAV System for Detecting and Pursuing Unwanted UAVs

In recent years, the use of unmanned aerial vehicle (UAV) platforms in civil and military applications has surged, highlighting the critical role of artificial intelligence (AI) embedded UAV systems in the future. This study introduces the Autonomous Drone (Vechür-SIHA), a novel AI-embedded UAV system designed for real-time detection and tracking of other UAVs during flight sequences. Leveraging advanced object detection algorithms and an LSTM-based tracking mechanism, our system achieves an impressive 80% accuracy in drone detection, even in challenging conditions like varying backgrounds and adverse weather.
 Our system boasts the capability to simultaneously track multiple drones within its field of view, maintaining flight for up to 35 minutes, making it ideal for extended missions that require continuous UAV tracking. Moreover, it can lock onto and track other UAVs in mid-air for durations of 4-10 seconds without losing contact, a feature with significant potential for security applications.
 This research marks a substantial contribution to the development of AI-embedded UAV systems, with broad implications across diverse domains such as search and rescue operations, border security, and forest fire prevention. These results provide a solid foundation for future research, fostering the creation of similar systems tailored to different applications, ultimately enhancing the efficiency and safety of UAV operations. The novel approach to real-time UAV detection and tracking presented here holds promise for driving innovations in UAV technology and its diverse applications.

Geolocalization from Aerial Sensing Images Using Road Network Alignment

Estimating the geographic positions in GPS-denied environments is of great significance to the safe flight of unmanned aerial vehicles (UAVs). In this paper, we propose a novel geographic position estimation method for UAVs after road network alignment. We discuss the generally overlooked issue, namely, how to estimate the geographic position of the UAV after successful road network alignment, and propose a precise robust solution. In our method, the optimal initial solution of the geographic position of the UAV is first estimated from the road network alignment result, which is typically presented as a homography transformation between the observed road map and the reference one. The geographic position estimation is then modeled as an optimization problem to align the observed road with the reference one to improve the estimation accuracy further. Experiments on synthetic and real flight aerial image datasets show that the proposed algorithm can estimate more accurate geographic position of the UAV in real time and is robust to the errors from homography transformation estimation compared to the currently commonly-used method.

A Statistical Study of Arien Sequences for an H.264 Prediction Module Lower Complexity

Abstract Advanced video coding, or H.264/AVC has been proved to possess a highly complex encoder with a block motion estimation (ME) scheme, entropy coding and a prediction module. Typical applications of unmanned aerial vehicles (UAVs) require real-time video compression. As a result, the current study aims to develop an approach to decrease the computing complexity of the prediction module. The software algorithm that was developed helps basic functions to be achieved in the prediction module. In fact, the user can set up, manage, deploy, and monitor the UAVs in real time. Evaluations obtained in this paper have shown that the adopted approach can significantly improve the execution time of coding and, consequently, the network performance. When comparing the original method of the Laboratory of Electronics and Information’s software with the suggested approach, experimental results show that the proposed method had achieved an increase in execution time from 3.544–15.574 %.

Spatial Reliability Enhanced Correlation Filter: An Efficient Approach for Real-Time UAV Tracking

Traditional discriminative correlation filter (DCF) has received great popularity due to its high computational efficiency. However, the lightweight framework of DCF cannot promise robust performance when the tracker faces appearance variations within the background. These unpredictable appearance variations always distract the filter. Most existing DCF-based trackers either utilize deep convolutional features or incorporate additional constraints to elevate tracking robustness. Despite some improvements, both of them hamper the tracking speed and can only roughly alleviate the distractions of appearance variations. In this paper, a novel spatial reliability enhanced learning strategy is proposed to handle the problems aforementioned. By monitoring the variation of response produced in detection phase, a dynamic reliability map is generated to indicate the reliability of each background subregion. Then, label adjustment is conducted to repress the distractions of these unreliable areas. Compared with the conventional way of constraint where a new term is always added to realize the desired goal, label adjustment is simultaneously more efficient and effective. Moreover, to promise the accuracy and dependability of the reliability map, an adaptively updated response pool recording reliable historical response values is proposed. Extensive and exhaustive experiments on three challenging unmanned aerial vehicle (UAV) benchmarks, i.e., UAV123@10fps, DTB70 and UAVDT, which totally include 243 video sequences, validate the superiority of the proposed method against other state-of-the-art trackers and exhibit a remarkable generality in a variety of scenarios. Meanwhile, the tracking speed of 65.2FPS on a cheap CPU makes it suitable for real-time UAV applications.