Unmanned Aerial Vehicle Video Research Articles

Algorithm for measuring parameters of radio link of an unmanned aircraft in fpv mode of the centimeter frequency range

The mass use of unmanned aerial vehicles (UAVs) in the mode of remote control "from the first person" (First Person View, FPV) in combat operations radically changes the picture of modern war. Their use makes it possible to damage or destroy enemy equipment, the cost of which is hundreds of times higher than the cost of UAVs. To successfully perform the UAV flight mission in FPV mode, it is necessary to ensure stable radio communication of the UAV with the ground control station (GSC). In the work to organize a radio line between the GSC and the UAV, it is proposed to use radio lines of the centimeter frequency range. At the same time, at the stage of flight mission planning, the UAV operator needs to have information about the parameters of the receiving and transmitting devices of the unmanned aerial vehicle complex (UAVС) in order to calculate the maximum flight range. The purpose of the work is to develop an algorithm for measuring parameters of the radio link of an unmanned aerial vehicle in the FPV mode of the centimeter frequency range. The use of this information will allow the UAVC operator to create a flight map and prepare for a combat mission. The material presented in the article contains practical value for UAV operators and can be used in the organization and planning of the flight mission of multi-rotor UAVs of the tactical level at a given distance. Also, the material can be used by UAV service units for the purpose of checking the technical parameters of the radio link between the UAV and GSC and the optical systems installed on board the UAV (video cameras, thermal imaging cameras, etc.).

Video saliency prediction for First-Person View UAV videos: Dataset and benchmark

Visual saliency prediction plays a crucial role in Unmanned Aerial Vehicle (UAV) video analysis tasks. In this paper, an eye-tracking dataset of the immersive viewing of videos captured from a First-Person View (FPV) of UAVs is developed, which consists of 200 video clips captured by DJI FPV drones, with a resolution of 4K QHD. The videos cover six different genres and fourteen unique scenes. To study human visual attention in watching FPV videos, fixation points are recorded using an eye tracker integrated into a VR headset. Based on the dataset, a simple yet effective FPV UAV video Saliency prediction model (FUAVSal) is proposed as a baseline, considering spatial–temporal feature, camera motion information and FPV prior. To establish benchmarks for saliency prediction in immersive FPV UAV video viewing, sixteen computational models are evaluated on this dataset. Detailed quantitative and qualitative comparisons are provided. The developed dataset and benchmarks aim to facilitate research on visual saliency prediction for First-Person View UAV videos.

Efficient Structure from Motion for Large-Size Videos from an Open Outdoor UAV Dataset.

Modern UAVs (unmanned aerial vehicles) equipped with video cameras can provide large-scale high-resolution video data. This poses significant challenges for structure from motion (SfM) and simultaneous localization and mapping (SLAM) algorithms, as most of them are developed for relatively small-scale and low-resolution scenes. In this paper, we present a video-based SfM method specifically designed for high-resolution large-size UAV videos. Despite the wide range of applications for SfM, performing mainstream SfM methods on such videos poses challenges due to their high computational cost. Our method consists of three main steps. Firstly, we employ a visual SLAM (VSLAM) system to efficiently extract keyframes, keypoints, initial camera poses, and sparse structures from downsampled videos. Next, we propose a novel two-step keypoint adjustment method. Instead of matching new points in the original videos, our method effectively and efficiently adjusts the existing keypoints at the original scale. Finally, we refine the poses and structures using a rotation-averaging constrained global bundle adjustment (BA) technique, incorporating the adjusted keypoints. To enrich the resources available for SLAM or SfM studies, we provide a large-size (3840 × 2160) outdoor video dataset with millimeter-level-accuracy ground control points, which supplements the current relatively low-resolution video datasets. Experiments demonstrate that, compared with other SLAM or SfM methods, our method achieves an average efficiency improvement of 100% on our collected dataset and 45% on the EuRoc dataset. Our method also demonstrates superior localization accuracy when compared with state-of-the-art SLAM or SfM methods.

Promoting UAV autonomous aerial video to improve cross-cultural understanding, historical knowledge, and students’ perception

ABSTRACT We designed a cross-cultural learning activity leveraging unmanned aerial vehicle (UAV) technology to enhance historical and cultural understanding among twenty university students from Indonesia and China. By recording and exchanging aerial videos of local cultural and historical sites using UAVs equipped with the latest AI features, students gained unique perspectives on these landmarks, unavailable through traditional ground-level photography. This study examined how such activities foster cross-cultural understanding and deepen historical knowledge, alongside participants' perceptions of UAV video production. Employing a mixed method approach, including video analysis, questionnaires, and interviews, we discovered that UAV technology not only facilitated a unique learning experience but also significantly impacted participants' cross-cultural and historical insights. The ease of UAV use and its effectiveness in cross-cultural education were particularly noted, with participants expressing intentions to use UAV technology in future learning contexts. Based on these findings, we offer recommendations for educators and researchers to integrate UAV technology into educational settings.

Rapid full-field deformation measurements of tall buildings using UAV videos and deep learning

Full-field deformation of tall buildings is informative and imperative for structural health condition assessments. This study explores the feasibility of using videos recorded by an unmanned aerial vehicle (UAV) to measure the vibration profiles of tall buildings with the aid of deep neural networks. First, a Transformer network with a self-attention mechanism is used, and its outcome is employed for the homography transformation to minimize the drifting problems induced by the UAV. Second, a boundary-aware salient object detection network is adopted to extract the boundary of the buildings in the stabilized videos for further full-field deformation measurements. The accuracy of the proposed method is experimentally validated using a wood frame structure under harmonic excitations. Under a measuring distance of approximately 4.2 m, the proposed method achieves averages of root-mean-square-errors (RMSEs) for the displacement and rotation measurements of 2.13 mm and 1.74 × 10−3 rad, respectively, as compared to that determined by an infrared light tracking system. The study further analyzes the field-test data recorded from the vibration of a 247-m tower. Under a measuring distance of about 55 m, the proposed method is comparable to the vibration measurement at a specific story and estimates the full-field deformation within the region of interest. Overall, the proposed method with rapid UAV deployment is feasible and accurate for monitoring the full-field deformation using the UAV-recorded videos.

A robust and efficient citrus counting approach for large-scale unstructured orchards

CONTEXTAccurately detecting and counting fruits is essential for orchard yield estimation and smart management. However, two drawbacks are inherent in the current citrus counting algorithms: First, the robustness needs to be improved, particularly illumination changes and dense occlusion. Secondly, the actual operating efficiency of the system is low. OBJECTIVETo tackle the above issues, this paper proposed a robust and efficient fruit-counting pipeline based on Unmanned Aerial Vehicle (UAV). METHODSFirst, to obtain UAV video streaming data online, a live broadcast platform and a flight control Application called FlyCounter were developed. Secondly, the Illumination-Adaptive-Transformer network is used to enhance the low-illumination citrus image in real-time. Then, for the specific challenging scenario, a novel model named Fruit-YOLO is designed to accurately detect citrus in data streams. Finally, the DeepSORT is adopted to track and count fruits in video sequences. RESULTS AND CONCLUSIONSThe results indicate that for detection performance, the P, R and mAP of the Fruit-YOLO of input enhanced image are 0.898, 0.854 and 0.929 respectively, which are 1.7%, 4.6% and 3.5% higher than the original image respectively. Regarding counting performance, for the daytime and evening scenes, the ID switch, MOTA and Error Mean of the pipeline are 63.7, 0.86 and 9.81% respectively. The MAPE of the offline and online operations of the pipeline are 4.36% and 12.66% respectively. The time and resources consumed by the system under parallel operations with different numbers of UAVs were analyzed. SIGNIFICANCEIn this study, an online counting pipeline based on UAVs that can work in low-light scenarios is implemented for the first time. The system has good performance in both daytime and nighttime scenarios, enabling efficient counting in orchards and extending operating time.

Productivity Analysis of Mid-Size Cable Yarding Operation based on UAV-captured Video Sequences

Cable yarding is one of the most preferred timber extraction methods in steep and mountainous terrain conditions. In recent years in Turkiye, mid-size cable yarders have begun to be used frequently in steep forest areas where intensive forestry is carried out. In order to make more effective use of cable yarders and achieve maximum productivity, timber extraction operations must be planned most appropriately and implemented in the field accordingly. The productivity of mechanized equipment used in harvesting operations is mainly determined by using time study analysis. In traditional methods, time measurements are carried out by observing the work stages with time recording devices (stopwatches) in the area where the machine operates. Since timber extraction by using a cable yarder occur in steep and rugged terrain conditions, difficulties may be encountered in applying time analysis in the field. In this study, it was aimed to obtain high-resolution digital video images of the activities of cable yarding operation with an Unmanned Aerial Vehicle (UAV) and to calculate productivity through time study analysis on these images. For the study, the Tajfun MOZ 500 GR model mid-size cable yarder was evaluated during a timber extraction operation in Köyceğiz Forest Enterprise Chief (FEC), located in Muğla City, Turkiye. Besides, statistical analysis was performed to estimate the effects of factors such as average product length, product diameter, lateral yarding distance, number of pieces, and total product volume per turn on the productivity of the cable yarder. The results indicated that UAV video images can be an effective alternative method to conduct time study analysis and productivity estimations. It was found that the productivity of the mid-size cable yarder was 11.76 m3/hour. During the operation, the pulling mainline to the product and choker setting were the most time-consuming work stages, followed by the time-loaded carriage reaching the landing. The statistical analysis indicated a significant relationship (p<0.001) between product length, product diameter, total volume and productivity of the cable yarder. The results showed that there was a negative relationship between lateral yarding distance and productivity. The regression model regarding the productivity of the cable yarder and the independent variables was also significant.

High-Performance Detection-Based Tracker for Multiple Object Tracking in UAVs

As a result of increasing urbanization, traffic monitoring in cities has become a challenging task. The use of Unmanned Aerial Vehicles (UAVs) provides an attractive solution to this problem. Multi-Object Tracking (MOT) for UAVs is a key technology to fulfill this task. Traditional detection-based-tracking (DBT) methods begin by employing an object detector to retrieve targets in each image and then track them based on a matching algorithm. Recently, the popular multi-task learning methods have been dominating this area, since they can detect targets and extract Re-Identification (Re-ID) features in a computationally efficient way. However, the detection task and the tracking task have conflicting requirements on image features, leading to the poor performance of the joint learning model compared to separate detection and tracking methods. The problem is more severe when it comes to UAV images due to the presence of irregular motion of a large number of small targets. In this paper, we propose using a balanced Joint Detection and Re-ID learning (JDR) network to address the MOT problem in UAV vision. To better handle the non-uniform motion of objects in UAV videos, the Set-Membership Filter is applied, which describes object state as a bounded set. An appearance-matching cascade is then proposed based on the target state set. Furthermore, a Motion-Mutation module is designed to address the challenges posed by the abrupt motion of UAV. Extensive experiments on the VisDrone2019-MOT dataset certify that our proposed model, referred to as SMFMOT, outperforms the state-of-the-art models by a wide margin and achieves superior performance in the MOT tasks in UAV videos.

STDFormer: Spatial-Temporal Motion Transformer for Multiple Object Tracking

Mainstream multi-object tracking methods exploit appearance information and/or motion information to achieve interframe association. However, dealing with similar appearance and occlusion is a challenge for appearance information, while motion information is limited by linear assumptions and is prone to failure in nonlinear motion patterns. In this work, we disregard appearance clues and propose a pure motion tracker to address the above issues. It dexterously utilizes Transformer to estimate complex motion and achieves high-performance tracking with low computing resources. Furthermore, contrastive learning is introduced to optimize feature representation for robust association. Specifically, we first exploit the long-range modeling capability of Transformer to mine intention information in temporal motion and decision information in spatial interaction and introduce prior detection to constrain the range of motion estimation. Then, we introduce contrastive learning as an auxiliary task to extract reliable motion features to compute affinity and introduce bidirectional matching to improve the affinity computation distribution. In addition, given that both tasks are dedicated to narrowing the embedding distance between the motion features of the tracked object and the detection features, we design a joint-motion-and-association framework to unify the above two tasks in one framework for optimization. The experimental results achieved with three benchmark datasets, MOT17, MOT20 and DanceTrack, verify the effectiveness of our proposed method. Compared with state-of-the-art methods, the proposed STDFormer sets a new state-of-the-art on DanceTrack and achieves competitive performance on MOT17 and MOT20. This demonstrates the advantage of our method in handling associations under similar appearance, occlusion or nonlinear motion. At the same time, the significant advantages of the proposed method over Transformer-based and contrastive learning-based methods suggest a new direction for the application of Transformer and contrastive learning in MOT. In addition, to verify the generalization of STDFormer in unmanned aerial vehicle (UAV) videos, we also evaluate STDFormer on VisDrone2019. The results show that STDFormer achieves state-of-the-art performance on VisDrone2019, which proves that it can handle small-scale object associations in UAV videos well. The code is available at https://github.com/Xiaotong-Zhu/STDFormer.

Feasibility of Live Video Feed Transmission from Unmanned Aerial Vehicles for Medical Surveillance During the 2022 Montreal Marathon.

In recent years, unmanned aerial vehicles (UAVs) have been increasingly used for medical surveillance purposes in mass-gathering events. No studies have investigated the reliability of live video transmission from UAVs for accurate identification of distressed race participants in need of medical attention. The aim of this study was to determine the proportion of time during which live medical surveillance UAV video feed was successfully transmitted and considered of sufficient quality to identify acute illness in runners participating in the 2022 Montreal Marathon (Canada). Four UAVs equipped with high-resolution cameras were deployed at two pre-defined high-risk areas for medical incidents located within the last 500 meters of the race. The video footage was transmitted in real-time during four consecutive hours to a remote viewing station where four research assistants monitored it on large screens. Interruptions in live feed transmission and moments with inadequate field of view (FOV) on runners were documented. On September 25, 2022, a total of 6,916 athletes ran during the Montreal Marathon and Half Marathon. Out of the eight hours of video footage analyzed (four hours per high-risk area), 91.7% represented uninterrupted live video feed with an adequate view of the runners passing through the high-risk areas. There was a total of 18 live feed interruptions leading to a total interruption time of 22 minutes and 19 seconds (median interruption time of 32 seconds) and eight distinct moments with inadequate FOV on runners which accounted for 17 minutes and 33 seconds (median of 1 minute 47 seconds per moments with inadequate FOV). Active surveillance of drone-captured footage allowed early identification of two race participants in need of medical attention. Appropriate resources were dispatched, and UAV repositioning allowed for real-time viewing of the medical response. Live video transmission from UAVs for medical surveillance of runners passing through higher risk segments of a marathon for four consecutive hours is feasible. Live feed interruptions and moments with inadequate FOV could be minimized through practice and additional equipment redundancy.

Attention-Enhanced One-Shot Attack against Single Object Tracking for Unmanned Aerial Vehicle Remote Sensing Images

Recent studies have shown that deep-learning-based models for processing Unmanned Aerial Vehicle (UAV) remote sensing images are vulnerable to artificially designed adversarial examples, which can lead to incorrect predictions of deep models when facing adversarial examples. Previous adversarial attack methods have mainly focused on the classification and detection of UAV remote sensing images, and there is still a lack of research on adversarial attacks for object tracking in UAV video. To address this challenge, we propose an attention-enhanced one-shot adversarial attack method for UAV remote sensing object tracking, which perturbs only the template frame and generates adversarial samples offline. First, we employ an attention feature loss to make the original frame’s features dissimilar to those of the adversarial frame, and an attention confidence loss to either suppress or enhance different confidence scores. Additionally, by forcing the tracker to concentrate on the background information near the target, a background distraction loss is used to mismatch templates with subsequent frames. Finally, we add total variation loss to generate adversarial examples that appear natural to humans. We validate the effectiveness of our method against popular trackers such as SiamRPN, DaSiamRPN, and SiamRPN++ on the UAV123 remote sensing dataset. Experimental results verify the superior attack performance of our proposed method.

Modified Siamese Network Based on Feature Enhancement and Dynamic Template for Low-Light Object Tracking in UAV Videos

Unmanned aerial vehicles (UAVs) visual object tracking under low-light conditions serves as a crucial component for applications, such as night surveillance, indoor searches, night combat, and all-weather tracking. However, the majority of the existing tracking algorithms are designed for optimal lighting conditions. In low-light environments, images captured by UAV typically exhibit reduced contrast, brightness, and a signal-to-noise ratio, which hampers the extraction of target features. Moreover, the target’s appearance in low-light UAV video sequences often changes rapidly, rendering traditional fixed template tracking mechanisms inadequate, and resulting in poor tracker accuracy and robustness. This study introduces a low-light UAV object tracking algorithm (SiamLT) that leverages image feature enhancement and a dynamic template-updating Siamese network. Initially, the algorithm employs an iterative noise filtering framework-enhanced low-light enhancer to boost the features of low-light images prior to feature extraction. This ensures that the extracted features possess more critical target characteristics and minimal background interference information. Subsequently, the fixed template tracking mechanism, which lacks adaptability, is enhanced by dynamically updating the tracking template through the fusion of the reference and base templates. This improves the algorithm’s capacity to address challenges associated with feature changes. Furthermore, the Average Peak-to-Correlation Energy (APCE) is utilized to filter the templates, mitigating interference from low-quality templates. Performance tests were conducted on various low-light UAV video datasets, including UAVDark135, UAVDark70, DarkTrack2021, NAT2021, and NAT2021L. The experimental outcomes substantiate the efficacy of the proposed algorithm in low-light UAV object-tracking tasks.

Hedge transfer learning routing for dynamic searching and reconnoitering applications in 3D multimedia FANETs.

With the fast development of unmanned aerial vehicles (UAVs) and the user increasing demand of UAV video transmission, UAV video service is widely used in dynamic searching and reconnoitering applications. Video transmissions not only consider the complexity and instability of 3D UAV network topology but also ensure reliable quality of service (QoS) in flying ad hoc networks (FANETs). We propose hedge transfer learning routing (HTLR) for dynamic searching and reconnoitering applications to address this problem. Compared with the previous transfer learning framework, HTRL has the following innovations. First, hedge principle is introduced into transfer learning. Online model is continuously trained on the basis of offline model, and their weight factors are adjusted in real-time by transfer learning, so as to adapt to the complex 3D FANETs. Secondly, distributed multi-hop link state scheme is used to estimate multi-hop link states in the whole network, thus enhancing the stability of transmission links. Among them, we propose the multiplication rule of multi-hop link states, which is a new idea to evaluate link states. Finally, we use packet delivery rate (PDR) and energy efficiency rate (EER) as two main evaluation metrics. In the same NS3 experimental scenario, the PDR of HTLR is at least 5.11% higher and the EER is at least 1.17 lower than compared protocols. Besides, we use Wilcoxon test to compare HTLR with the simplified version of HTLR without hedge transfer learning (N-HTLR). The results show that HTRL is superior to N-HTRL, effectively ensuring QoS.

Adaptive Image Dehazing and Object Tracking in UAV Videos Based on the Template Updating Siamese Network

Visual object tracking for unmanned aerial vehicles (UAVs) is extensively used in civil and military applications such as intelligent transportation, disaster rescue, and military reconnaissance. However, when a UAV encounters haze weather, its tracking performance deteriorates greatly because of problems such as blurring and quality degradation of the collected images. Most trackers are designed to operate in good weather conditions, this greatly limits the robustness of UAV tracking in haze weather. To solve this problem, this paper proposes an adaptive dehazing Siamese network, which consists of an adaptive dehazing module, a dynamic template update branch, and feature extraction and bounding box prediction subnetworks. The dehazing module yields haze-free images and preserves the edge information of the object, so that the tracker can extract the object features with better characterization. Subsequently, to mitigate the risk of tracking drift caused by the changing appearance of the object, the dynamic template update branch dynamically selects more reliable template images during tracking. In addition, to solve the problem of insufficient UAV object tracking testing datasets in haze scenes, a haze testing dataset for UAV video object tracking is obtained through artificial synthesis and data collection in real-world haze scenes. Furthermore, experiments are conducte on haze testing datasets, namely UAV123-AH, UAV20L-AH, UAVDT-AH, DTB70-AH, and UAVhaze; the results verified that the proposed tracker yields more competitive performance and more robustness in haze scenes compared with the existing algorithms.

Optimization of Vehicle Object Detection Based on UAV Dataset: CNN Model and Darknet Algorithm

This study was conducted to identify several types of vehicles taken using drone technology or Unmanned Aerial Vehicles (UAV). The introduction of vehicles from above an altitude of more than 300-400 meters that pass the highway above ground level becomes a problem that needs optimum investigation so that there are no errors in determining the type of vehicle. This study was conducted at mining sites to identify the class of vehicles that pass through the highway and how many types of vehicles pass through the road for vehicle recognition using a deep learning algorithm using several CNN models such as Yolo V4, Yolo V3, Densenet 201, CsResNext –Panet 50 and supported by the Darknet algorithm to support the training process. In this study, several experiments were carried out with other CNN models, but with peripherals and hardware devices, only 4 CNN models resulted in optimal accuracy. Based on the experimental results, the CSResNext-Panet 50 model has the highest accuracy and can detect 100% of the captured UAV video data, including the number of detected vehicle volumes, then Densenet and Yolo V4, which can detect up to 98% - 99%. This research needs to continue to be developed by knowing all classes affordable by UAV technology but must be supported by hardware and peripheral technology to support the training process.

Feasibility of Live Video Feed Transmission from UAVs for Medical Surveillance During the 2022 Montreal Marathon

Introduction:In recent years, unmanned aerial vehicles (UAVs) have been increasingly used for medical surveillance purposes in mass gathering events. No studies have investigated the reliability of live video transmission from UAVs for accurate identification of distressed race participants in need of medical attention. During the 2022 Montreal marathon, the aim of this study was to determine the proportion of data collection time during which live medical surveillance UAV video feed was successfully transmitted and considered of sufficient quality to identify acute illness.Method:Four UAVs equipped with high resolution cameras were deployed at two predefined high-risk areas for medical incidents located within the last 800m of the race. The video footage was transmitted in real-time during four consecutive hours to a remote viewing station where four research assistants monitored it on large screens. Interruptions in live feed transmission and moments with inadequate field of view on runners were documented.Results:On September 25, 2022, 8,577 athletes registered in the Montreal marathon and half marathon. Out of the eight hours of video footage analyzed (four hours per high-risk area), 91.7% represented uninterrupted live video feed with an adequate view of the runners passing through the high-risk areas. The total interruption time was 22 minutes and 19 seconds, and the field of view was considered inadequate for a total of 17 minutes and 33 seconds. Active surveillance of drone-captured footage allowed identification of two race participants in need of medical attention. Appropriate resources were dispatched, and UAV repositioning allowed for real-time viewing of the medical response.Conclusion:Live video transmission from UAVs for medical surveillance of runners passing through higher-risk segments of a marathon race for four consecutive hours is feasible. Live feed interruptions and segments with an inadequate field of view could be minimized through practice and additional equipment redundancy.

Surveying of Nearshore Bathymetry Using UAVs Video Stitching

In this paper, we extended video stitching to nearshore bathymetry for videos that were captured for the same coastal field simultaneously by two unmanned aerial vehicles (UAVs). In practice, a video captured by a single UAV often shows a limited coastal zone with a lack of a wide field of view. To solve this problem, we proposed a framework in which video stitching and bathymetric mapping were performed in sequence. Specifically, our method listed the video acquisition strategy and took two overlapping videos captured by two UAVs as inputs. Then, we adopted a unified video stitching and stabilization optimization to compute the stitching and stabilization of one of the videos separately. In this way, we can obtain the best stitching result. At the same time, background feature points identification on the shore plays the role of short-time visual odometry. Through the obtained panoramic video in Shuang Yue Bay, China, we used the temporal cross-correlation analysis based on the linear dispersion relationship to estimate the water depth. We selected the region of interest (ROI) area from the panoramic video, performed an orthorectification transformation and extracted time-stack images from it. The wave celerity was then estimated from the correlation of the signal through filtering processes. Finally, the bathymetry results were compared with the cBathy. By applying this method to two UAVs, a wider FOV was created and the surveying area was expanded, which provided effective input data for the bathymetry algorithms.

Realtime wide-area vehicle trajectory tracking using millimeter-wave radar sensors and the open TJRD TS dataset

• A system based on millimeter wave radar is introduced for tracking vehicles. • Trajectory splicing is applied for tracking vehicles in wide-area level. • Validation results show a reliable and stable performance of the system. • The system performs well in detecting speeding fractions through pilot project. • TJRD TS Platform is introduced in sharing wide-area trajectory dataset (open). This paper presented a real-time millimeter wave radar-based system for tracking vehicle trajectories in a wide area (continuously along a roadway with essentially no length limit in practice). The trajectory tracking results were first validated for single vehicle trajectory tracking using the Real-Time Kinematic positioning technology based on the Beidou satellite navigation systems. The validation showed that the vehicle positions were captured with a mean lateral offset of −0.284 m and a mean longitudinal offset of −0.352 m. The mean estimated speeds were found to have a difference of only −0.048 km/h from the ground truths. The trajectory tracking results were also validated through multi-object tracking using video data from an unmanned aerial vehicle (UAV). Compared with the UAV video footage, the millimeter wave-based system was found to correctly capture around 92% of the total number of vehicles. For the correctly captured vehicles, their positions were found to be within 0.99 m (about a quarter of the width of a regular traffic lane) of the ground truth. In this paper, we also would like to share openly the entire validated datasets which has been recently published online as the TJRD TS platform. Additionally, a demonstration in using the dataset to detect aggressive driving behaviors, such as speeding was presented. It is expected that the open dataset will help enable researchers and practitioners to further explore the behaviors of road users, track the dynamics of safety risks and congestion formation, and evaluate incident impacts in a more microscopic but comprehensive way.

Vision-Based HAR in UAV Videos Using Histograms and Deep Learning Techniques.

Activity recognition in unmanned aerial vehicle (UAV) surveillance is addressed in various computer vision applications such as image retrieval, pose estimation, object detection, object detection in videos, object detection in still images, object detection in video frames, face recognition, and video action recognition. In the UAV-based surveillance technology, video segments captured from aerial vehicles make it challenging to recognize and distinguish human behavior. In this research, to recognize a single and multi-human activity using aerial data, a hybrid model of histogram of oriented gradient (HOG), mask-regional convolutional neural network (Mask-RCNN), and bidirectional long short-term memory (Bi-LSTM) is employed. The HOG algorithm extracts patterns, Mask-RCNN extracts feature maps from the raw aerial image data, and the Bi-LSTM network exploits the temporal relationship between the frames for the underlying action in the scene. This Bi-LSTM network reduces the error rate to the greatest extent due to its bidirectional process. This novel architecture generates enhanced segmentation by utilizing the histogram gradient-based instance segmentation and improves the accuracy of classifying human activities using the Bi-LSTM approach. Experimental outcomes demonstrate that the proposed model outperforms the other state-of-the-art models and has achieved 99.25% accuracy on the YouTube-Aerial dataset.

Development of a Novel Lightweight CNN Model for Classification of Human Actions in UAV-Captured Videos

There has been increased attention paid to autonomous unmanned aerial vehicles (UAVs) recently because of their usage in several fields. Human action recognition (HAR) in UAV videos plays an important role in various real-life applications. Although HAR using UAV frames has not received much attention from researchers to date, it is still a significant area that needs further study because of its relevance for the development of efficient algorithms for autonomous drone surveillance. Current deep-learning models for HAR have limitations, such as large weight parameters and slow inference speeds, which make them unsuitable for practical applications that require fast and accurate detection of unusual human actions. In response to this problem, this paper presents a new deep-learning model based on depthwise separable convolutions that has been designed to be lightweight. Other parts of the HarNet model comprised convolutional, rectified linear unit, dropout, pooling, padding, and dense blocks. The effectiveness of the model has been tested using the publicly available UCF-ARG dataset. The proposed model, called HarNet, has enhanced the rate of successful classification. Each unit of frame data was pre-processed one by one by different computer vision methods before it was incorporated into the HarNet model. The proposed model, which has a compact architecture with just 2.2 million parameters, obtained a 96.15% success rate in classification, outperforming the MobileNet, Xception, DenseNet201, Inception-ResNetV2, VGG-16, and VGG-19 models on the same dataset. The proposed model had numerous key advantages, including low complexity, a small number of parameters, and high classification performance. The outcomes of this paper showed that the model’s performance was superior to that of other models that used the UCF-ARG dataset.