UAV Video Research Articles

AI-based Precision Enhancement for Change Detection in Drone/UAV Videos

The rapid advancement in drone/UAV technology has led to their extensive use in various domains including surveillance, agriculture, and environmental monitoring. Change detection in UAV videos is crucial for identifying significant alterations in the observed areas. However, challenges such as varying scales, resolutions, and orientations of video frames pose significant hurdles. This paper addresses these challenges by leveraging the SIFT algorithm for panoramic stitching and change detection, followed by the application of the YOLOv8 model for high-precision anomaly detection

The Legacy of Sycamore Gap: The Potential of Photogrammetric AI for Reverse Engineering Lost Heritage with Crowdsourced Data

Abstract. The orientation of crowdsourced and multi-temporal image datasets presents a challenging task for traditional photogrammetry. Indeed, traditional image matching approaches often struggle to find accurate and reliable tie points in images that appear significantly different from one another. In this paper, in order to preserve the memory of the Sycamore Gap tree, a symbol of Hadrian's Wall that was felled in an act of vandalism in September 2023, deep-learning-based features trained specifically on challenging image datasets were employed to overcome limitations of traditional matching approaches. We demonstrate how unordered crowdsourced images and UAV videos can be oriented and used for 3D reconstruction purposes, together with a recently acquired terrestrial laser scanner point cloud for scaling and referencing. This allows the memory of the Sycamore Gap tree to live on and exhibits the potential of photogrammetric AI (Artificial Intelligence) for reverse engineering lost heritage.

Semantic segmentation of oblique UAV video based on ConvLSTM in complex urban area

Semantic segmentation of oblique UAV video based on ConvLSTM in complex urban area

ULSR-UV: an ultra-lightweight super-resolution networks for UAV video

ULSR-UV: an ultra-lightweight super-resolution networks for UAV video

Critical Examination of 3D Building Modelling through UAV Frame and Video Imaging

Abstract. Data capture in UAV photogrammetry is carried out using two main methodologies: frame frame-based and video video-based. Frame Frame-based data gathering is the preferred method among UAV projects because to its inherent reliability in calibration. Nonetheless, circumstances involving moving objects or occlusions inside the measured region may produce unsatisfactory results utilizing this method. I In response to these challenges, video video-based data collecting appears as a potential option, capable of creating a series of successive images that together alleviate the constraints outlined above. In this study we aim to compare the usefulness of frame and video images in building 3D models, using both oblique and vertical image orientations. Rigorous evaluations produced many outputs, including dense point clouds, digital surface models (DSM), meshes, and orthophotographs. The evaluation criteria included da ta acquisition velocity, processing efficiency, calibration precision, distortion analysis, residual plots, scale correctness, and reprojection error. The empirical results demonstrated the advantages of video video-frame captures in improving the quality of the resulting 3D models. Notably, the use of video frames resulted in a significant reduction in reprojection error by 16%, calibration residuals by 36%, distortions by up to 51%, and processing time by 27%. Thus, it seems that integrating video frames improves data gathering accuracy and speeds up processing, replacing standard frame images with video counterparts for increased efficacy.

A maneuver indicator and ensemble learning-based risky driver recognition approach for highway merging areas

Abstract Due to the complex traffic characteristics in highway merging areas, drivers tend to exhibit high-risk driving behaviors. To address the characteristics of driving behavior in highway merging areas, we have developed a real-time identification model for risky drivers by combining a driver risk level labeling method with load balancing-ensemble learning (LB-EL). In this paper, we explore four types of maneuver indicator indexes (MIIs)—acute direction, stomp pedal, dangerous following, and dangerous lane changing—that can describe the negative behaviors of both individual vehicles and vehicle platoons in highway merging areas. To quantize the label driver risk level, we use the interquartile range (IQR) method and Criteria Importance Though Intercriteria Correlation (CRITIC), while we evaluate the reliability of the MII using spatial analysis. Furthermore, we balance the dataset using three load balancing (LB) algorithms and create nine ensemble strategies by pairing adaptive boosting (AdaBoost), extreme gradient boosting (XGBoost), and light gradient boosting machine (LGBM) with the three LB algorithms. Finally, we validate the proposed model using trajectory data extracted from UAV videos. The results indicate that the distribution laws of risky driving behaviors in the acute direction and stomp pedal show a high degree of similarity and good matching with the distribution laws of traffic conflict points in existing research. Moreover, the SMOTE-LGBM ensemble model achieves the best performance, reaching an accuracy rate of 93.4% and a recall rate of 92.1%, which demonstrates the validity of our proposed model. This model can be widely applied to recognize risky drivers in video-based surveillance systems.

Development of software for contactless control of UAVs

The article is devoted to the development of a program for contactless control of a UAV using a neural network that tracks the position of an object. For these purposes, a corresponding model was trained on the ultralytics YOLOv8 neural network. The graphs of the training of this model are presented, demonstrating the change in the magnitude of the loss function over the bounding box and class, as well as the values of the mAP50-95 metric. The training was completed when the value of the mAP50-95 metric was 0.855. Software has been developed to control the UAV using hand movements, its block diagram and description are given. The program reads the coordinates of the hand in each frame, calculates its area, evaluates the received data and sends control commands to the copter, which moves in the appropriate direction for a given step, including a certain group of LEDs. At the same time, the screen displays a simulation of the drone›s movement in two projections (front and top) and displays the relative coordinates of the drone. The software was tested on a Geoscan pioneer mini quadcopter. It can be used for educational, demonstration purposes, in agriculture, UAV sports competitions, aerial photography and video filming and other fields of activity.

Human activity recognition from uav videos using an optimized hybrid deep learning model

Human activity recognition from uav videos using an optimized hybrid deep learning model

Developing a More Reliable Framework for Extracting Traffic Data From a UAV Video

Developing a More Reliable Framework for Extracting Traffic Data From a UAV Video

Real-Time Traffic End-of-Queue Detection and Tracking in UAV Video

Real-Time Traffic End-of-Queue Detection and Tracking in UAV Video

Vision-based investigation of road traffic and violations at urban roundabout in India using UAV video: A case study

In the past decade, the number of vehicles in India has increased exponentially; however, road infrastructure has not scaled proportionately. As a result, road traffic problems such as congestion on urban roads, dangerous traffic violations, and road accidents have increased significantly. Due to limited road infrastructure, traffic violations (human errors) have intensified in densely populated urban areas. This paper presents a case study (at a multi-lane urban roundabout in Ahmedabad city, India) and the methodology based on computer vision to investigate road traffic and violations using drone/UAV-based aerial video. You Only Look Once-YOLOv7 is used for vehicle detection, and Simple Online and Real Time Tracking-SORT for tracking vehicles. Our methodology divides the road scene (roundabout) into certain zones. We then formulated the dictionary, which maps the traffic violations under Motor Vehicle Driving Regulations - MVDR/ Motor Vehicle Act - MVA and the movement of the vehicle (zone traversal sequences). Using the zone-based methodology, we could also probe other road traffic data such as the count of vehicles, speed of vehicles, rate of traffic flow, and congestion. Based on our results, we also infer some of the possible causes of traffic violations in terms of problems/limitations of road infrastructure. As per our analysis, around 23.26% of vehicles committed traffic violations. We detected traffic violations related to lane indiscipline, driving against the authorized flow of traffic, parking violations, and over-speeding within the roundabout. Our methodology of investigating road traffic and violations can be used for road infrastructure improvement, law enforcement drives, and policy making, for road traffic safety, in developing and densely populated countries.

Real-Time Intersection Vehicle Turning Movement Counts from Live UAV Video Stream Using Multiple Object Tracking

Real-Time Intersection Vehicle Turning Movement Counts from Live UAV Video Stream Using Multiple Object Tracking

A robust vehicle tracking in low-altitude UAV videos

A robust vehicle tracking in low-altitude UAV videos

A Block-Based Concatenated LDPC-RS Code for UAV-to-Ground SC-FDE Communication Systems

To enhance the reliability of UAV-to-Ground data transmission in SC-FDE, this study proposes a combined MMSE equalization and LDPC-RS concatenated code approach, utilizing source redundancy to mitigate errors in low-elevation G2A communication. The proposed coding scheme combines a binary LDPC code as the outer code and an improved RS code as the inner code, resulting in improved resistance to burst and random errors. To address the coding aspect, this paper designs a coding table using RS coding as the underlying logic, enabling efficient coding. For decoding, a decoding scheme based on the minimum Euclidean distance criterion is devised, enabling high-performance decoding. Simulation results demonstrate that the proposed LDPC-RS concatenated coding scheme significantly enhances the performance of the communication system, thereby offering considerable potential for application in UAV video transmission systems.

Research on the Method of Counting Wheat Ears via Video Based on Improved YOLOv7 and DeepSort.

The number of wheat ears in a field is an important parameter for accurately estimating wheat yield. In a large field, however, it is hard to conduct an automated and accurate counting of wheat ears because of their density and mutual overlay. Unlike the majority of the studies conducted on deep learning-based methods that usually count wheat ears via a collection of static images, this paper proposes a counting method based directly on a UAV video multi-objective tracking method and better counting efficiency results. Firstly, we optimized the YOLOv7 model because the basis of the multi-target tracking algorithm is target detection. Simultaneously, the omni-dimensional dynamic convolution (ODConv) design was applied to the network structure to significantly improve the feature-extraction capability of the model, strengthen the interaction between dimensions, and improve the performance of the detection model. Furthermore, the global context network (GCNet) and coordinate attention (CA) mechanisms were adopted in the backbone network to implement the effective utilization of wheat features. Secondly, this study improved the DeepSort multi-objective tracking algorithm by replacing the DeepSort feature extractor with a modified ResNet network structure to achieve a better extraction of wheat-ear-feature information, and the constructed dataset was then trained for the re-identification of wheat ears. Finally, the improved DeepSort algorithm was used to calculate the number of different IDs that appear in the video, and an improved method based on YOLOv7 and DeepSort algorithms was then created to calculate the number of wheat ears in large fields. The results show that the mean average precision (mAP) of the improved YOLOv7 detection model is 2.5% higher than that of the original YOLOv7 model, reaching 96.2%. The multiple-object tracking accuracy (MOTA) of the improved YOLOv7-DeepSort model reached 75.4%. By verifying the number of wheat ears captured by the UAV method, it can be determined that the average value of an L1 loss is 4.2 and the accuracy rate is between 95 and 98%; thus, detection and tracking methods can be effectively performed, and the efficient counting of wheat ears can be achieved according to the ID value in the video.

CapERA: Captioning Events in Aerial Videos

In this paper, we introduce the CapERA dataset, which upgrades the Event Recognition in Aerial Videos (ERA) dataset to aerial video captioning. The newly proposed dataset aims to advance visual–language-understanding tasks for UAV videos by providing each video with diverse textual descriptions. To build the dataset, 2864 aerial videos are manually annotated with a caption that includes information such as the main event, object, place, action, numbers, and time. More captions are automatically generated from the manual annotation to take into account as much as possible the variation in describing the same video. Furthermore, we propose a captioning model for the CapERA dataset to provide benchmark results for UAV video captioning. The proposed model is based on the encoder–decoder paradigm with two configurations to encode the video. The first configuration encodes the video frames independently by an image encoder. Then, a temporal attention module is added on the top to consider the temporal dynamics between features derived from the video frames. In the second configuration, we directly encode the input video using a video encoder that employs factorized space–time attention to capture the dependencies within and between the frames. For generating captions, a language decoder is utilized to autoregressively produce the captions from the visual tokens. The experimental results under different evaluation criteria show the challenges of generating captions from aerial videos. We expect that the introduction of CapERA will open interesting new research avenues for integrating natural language processing (NLP) with UAV video understandings.

Super pixels transmission map-based object detection using deep neural network in UAV video

ABSTRACT Object detection has become a very prominent subject for research in recent times. This study's main goal is to suggest a technique for video saliency object detection. It seems to sense that using the depth information in photos to detect salient things. Since depth offers abundant information about scene structure, object forms, and other 3D cues. This information is very compatible to distinguish between objects in the foreground and background. As a result of the high object density, small object size, and cluttered background, aerial photos and movies provide results with low precision. In this paper, the proposed SPTM (Super Pixel Transmission Map)-YOLO model, the input RGB image has applied Dark Channel Prior (DCP) method for estimating the transmission map. From the transmission map only, the background probability is estimated with the help of SLIC (simple linear iterative clustering algorithm) superpixel segmentation. That foreground extracted image is further learned with YOLO architecture to detect the objects effectively. For object detection in aerial images, this proposed SPTM-YOLO approach outperforms classic YOLO by up to 6% accuracy. Accurate detection of things that are small in size, partially occluded, and out of view is possible.

Object Tracking in UAV Videos by Multifeature Correlation Filters With Saliency Proposals

Object Tracking in UAV Videos by Multifeature Correlation Filters With Saliency Proposals

Bounding box initialization and optimization for ground target tracking in UAV Videos

Bounding box initialization and optimization for ground target tracking in UAV Videos

Person Tracking by Fusing Posture Data From UAV Video and Wearable Sensors

In this article, a novel framework that fuses the posture data taken by a drone (or unmanned aerial vehicle, UAV) camera and the wearable sensors data recorded by smartwatches is proposed. The framework is designed for continuously tracking persons in a drone view by analyzing location-independent human posture features and correctly tagging smartwatch identities (IDs) and personal profiles to video human objects, thus conquering the former work in requiring ground markers. Person detection, ID assignment, and pose estimation are integrated into our framework to obtain recognized human postures. These recognized postures are then paired with those from the wearable sensors. Through fusing common postures, such as standing, walking, jumping, and falling down, person tracking accuracy by UAV up to 95.36% can be attained in our testing scenarios.