Comparison of Neural Network Architectures for Spatial Orientation of Unmanned Aerial Vehicles and Ground Drones

Unmanned aerial vehicles (UAVs), as aerial base stations, are a promising solution for providing wireless communications, thanks to their high flexibility and autonomy. Moreover, emerging services, such as extended reality, require high-capacity communications. To achieve this, millimeter wave (mmWave), and recently, terahertz bands have been considered for UAV communications. However, communication at these high frequencies requires a line-of-sight (LoS) to the terminals, which may be located in 3D space and may have extremely limited direct-line-of-view (LoV) due to blocking objects, like buildings and trees. In this paper, we investigate the problem of determining 3D placement and orientation of UAVs such that users have guaranteed LoS coverage by at least one UAV and the signal-to-noise ratio (SNR) between the UAV-user pairs are maximized. We formulate the problem as an integer linear programming(ILP) problem and prove its NP-hardness. Next, we propose a low-complexity geometry-based greedy algorithm to solve the problem efficiently. Our simulation results show that the proposed algorithm (almost) always guarantees LoS coverage to all users in all considered simulation settings.

Object detection on hardware platforms plays a very significant role in developing driver assistance systems (DASs) with limited computational resources. Object detection for DAS is a multiclass detection problem that involves detecting various objects like cars, auto, traffic lights, bicycles, pedestrians, etc. DAS also requires accuracy, speed, and sensitivity for detecting these objects in various challenging conditions. The lighting and weather conditions pose a serious challenge for accurate object detection for DAS. This paper proposes a speed-efficient and lightweight fully convolutional neural network (CNN) architecture for object detection in adverse rainy conditions. The proposed architecture uses a CNN-based deraining network with a custom SSIM loss function in the object detection pipeline, which can give an accurate performance using limited computational and memory resources. The object detection architecture contains some architectural modifications to the existing single shot multibox detector (SSD) architecture to make it more hardware efficient and improve accuracy on small objects. It uses a trainable color transformation module using [Formula: see text] convolutions for handling the adverse lighting conditions encountered in DAS. The architecture uses feature fusion and the dilated convolution approach to enhance the accuracy of the proposed architecture on small objects. The datasets available for object detection in DAS are very imbalanced with cars as a predominant object. The class weight penalization technique is used to improve the performance of the architecture on scarcely present objects. The performance of the architecture is evaluated on well-known datasets like Kitti, Udacity, Indian Driving Dataset (IDD), and DAWN. The architecture achieves satisfactory performance in terms of mean average precision (mAP) and detection time on all these datasets. It requires three times fewer hardware resources compared to existing architectures. The lightweight nature of the proposed architecture and modification of CNN architecture with TensorRT allow the efficient implementation on the jetson nanohardware platform for prototyping, which can be integrated with other intelligent transportation systems.

UAVs are widely used for multiple tasks, which in many cases require autonomous processing and decision making. This autonomous function often requires significant computational capabilities that cannot be integrated into the UAV due to weight or cost limitations, making the distribution of the workload and the combination of the results produced necessary. In this paper, a dual-stage processing architecture for object detection and tracking in Unmanned Aerial Vehicles (UAVs) is presented, focusing on efficient resource utilization and real-time performance. The proposed system delegates lightweight detection tasks to onboard hardware while offloading computationally intensive processes to a ground server. The UAV is equipped with a Raspberry Pi for onboard data processing, utilizing an Intel Neural Compute Stick 2 (NCS2) for accelerated object detection. Specifically, YOLOv5n is selected as the onboard model. The UAV transmits selected frames to the ground server, which handles advanced tracking, trajectory prediction, and target repositioning using state-of-the-art deep learning models. Communication between the UAV and the server is maintained through a high-speed Wi-Fi link, with a fallback to a 4G connection when needed. The ground server, equipped with an NVIDIA A40 GPU, employs YOLOv8x for object detection and DeepSORT for multi-object tracking. The proposed architecture ensures real-time tracking with minimal latency, making it suitable for mission-critical UAV applications such as surveillance and search and rescue. The results demonstrate the system’s robustness in various environments, highlighting its potential for effective object tracking under limited onboard computational resources. The system achieves recall and accuracy scores as high as 0.53 and 0.74, respectively, using the remote server, and is capable of re-identifying a significant portion of objects of interest lost by the onboard system, measured at approximately 70%.

Millimeter-wave (mmWave) communication offers a promising solution for high-throughput, low-latency unmanned aerial vehicle (UAV) networks. However, maintaining strong received signal strength (RSS) remains a challenge due to UAV mobility. While existing studies have largely focused on beam alignment, they often overlook another critical issue: polarization mismatch caused by UAV orientation changes. This problem is particularly severe in cost-sensitive commercial off-the-shelf (COTS) mmWave devices, which typically employ linearly polarized (LP) antenna arrays. Our measurements reveal that even with perfect beam alignment, UAV orientation can still cause significant signal degradation due to polarization mismatch. To address this challenge, we propose PolarFix, a practical metasurface solution that enables real-time polarization matching without requiring any modifications to existing transceiver hardware. Specifically, we design a linear-to-circular polarization (L2C) metasurface that transforms linearly polarized (LP) waves into circularly polarized signals, allowing LP antennas to maintain consistent signal power despite changes in UAV orientation.

Formulation of the problem. Currently, unmanned aerial vehicles (UAVs) are actively used to solve cross-industry tasks, such as photo and video filming of the surrounding space, remote analysis of surrounding objects, cargo delivery, monitoring the state of transport routes and mining complexes. UAVs are also used in emergency areas where the presence of people is associated with a threat to life. Since good orientation in space is necessary in all of these areas, UAV orientation in urban infrastructure is of great practical importance. For example, for accident-free driving of UAVs, determining the distance to obstacles (image depth maps) and constructing the trajectory of their flight is of great importance, which is relevant for precise UAV orientation in urban infrastructure. Target. Propose a modified method for post-processing the image depth map for the problems of accident-free driving of UAVs in urban infrastructure. Results. Existing methods and algorithms for converting from 2D signals to 3D signals in image analysis and display systems are considered. A study of the method of post-processing of the image depth map was carried out and its modification was proposed. To evaluate the effectiveness of the developed post-processing method, a comparative analysis of depth maps for various test images obtained by the original method and its modification was performed. Root-mean-square errors are obtained for the proposed and original post-processing methods for the considered test images. Practical significance. The presented modified image depth map post-processing method makes it possible to build sharper image depth maps, which provides more accurate UAV orientation in urban infrastructure.

This paper presents a system applied to unmanned aerial vehicles based on Robot Operating Systems (ROSs). The study addresses the challenges of efficient object detection and real-time target tracking for unmanned aerial vehicles. The system utilizes a pruned YOLOv4 architecture for fast object detection and the SiamMask model for continuous target tracking. A Proportional Integral Derivative (PID) module adjusts the flight attitude, enabling stable target tracking automatically in indoor and outdoor environments. The contributions of this work include exploring the feasibility of pruning existing models systematically to construct a real-time detection and tracking system for drone control with very limited computational resources. Experiments validate the system’s feasibility, demonstrating efficient object detection, accurate target tracking, and effective attitude control. This ROS-based system contributes to advancing UAV technology in real-world environments.

Abstract. The paper presents the results of the prediction for the parameters of the position and orientation of the unmanned aerial vehicle (UAV) equipped with compact digital camera. Issue focus in this paper is to achieve optimal accuracy and reliability of the geo-referenced video frames on the basis of data from the navigation sensors mounted on UAV. In experiments two mathematical models were used for the process of the prediction: the polynomial model and the trigonometric model. The forecast values of position and orientation of UAV were compared with readings low cost GPS and INS sensors mounted on the unmanned Trimble UX-5 platform. Research experiment was conducted on the preview of navigation data from 23 measuring epochs. The forecast coordinate values and angles of the turnover and the actual readings of the sensor Trimble UX-5 were compared in this paper. Based on the results of the comparison it was determined that: the best results of co-ordinate comparison of an unmanned aerial vehicle received for the storage with, whereas worst for the coordinate Y on the base of both prediction models, obtained value of standard deviation for the coordinate XYZ from both prediction models does not cross over a admissible criterion 10 m for the term of the exactitudes of the position of a unmanned aircraft. The best results of the comparison of the angles of the turn of a unmanned aircraft received for the angle Pitch, whereas worst for the angles Heading and Roll on the base of both prediction models. Obtained value of standard deviation for the angles of turn HPR from both prediction models does not exceed a admissible exactitude 5° only for the angle Pitch, however crosses over this value for the angles Heading and Roll.

Object detection is a method of identifying and locating items in a continuous image or video stream using computer vision. Background subtraction, optical flow, and frame difference are object detecting approaches. In this study, we will explore Object Detection Algorithms, such as Convolutional Neural Network algorithm families (CNN, R-CNN, Fast R-CNN, Faster R-CNN) and YOLO as well as associated applications and frameworks. Object Detection Algorithms will be used by real-time applications and will get their goals.This book chapter discusses the Introduction about computer vision, object detection, object detection architectures, object detection methods, object detection techniques, applications of object detection, and implementation sources for object detection.

Object detection systems are central to the autonomy and safety of intelligent transportation systems. Yet, the accuracy of object detection models can suffer under environmental noise or adverse weather. This paper tested the robustness of four object detection architectures: YOLOv5s, YOLOv8m, YOLOv10n, and Faster R-CNN, to visual degradation (real-world weather and artificial noise). We utilize the DAWN dataset, a benchmark of 1,000 high-resolution traffic images captured under fog, rain, snow, and sandstorms, with further augmentations of Gaussian noise, salt-and-pepper noise, blurriness, and overlays applied with artificial fog. We standardized all annotations to YOLO and COCO annotation formats for multi-framework interoperability. Our quantitative analysis used mAP@0.5, mAP@0.5:0.95, Precision, and Recall to compare the models, alongside some qualitative analysis through visual overlays and plotting training loss. The findings of our analysis showed YOLOv8m achieved the highest baseline accuracy on clean data, while Faster R-CNN proved resilient in noisy environments. YOLOv10n achieves a good trade-off between efficient and robust detection. The results of this study highlight the necessity of adaptive training pipelines and environment-aware benchmarks to enhance the real-world reliability of vision-based detection systems.

The application of principal component analysis to simplify the classification of pixels of images obtained with the help of unmanned aerial vehicle for the organization received information processing in real time, and simplify the process of orientation of unmanned aerial vehicle on the ground. We consider the neural network of the principal component analysis. The results of experiments showing the effectiveness of such use of principal component analysis.

An algorithm for determining the position and orientation of unmanned aerial vehicle (UAVs) by photos of the area

The results of development of formalized model of process of formation of decisive function of combined correlation-extreme navigation system of unmanned aerial vehicles are presented. The development of the model is carried out taking into account the state of the operational and tactical situation, as well as the effect of changing the geometry of the sighting. The peculiarity of the developed model consists in the formation of a decisive function using a set of reference images, which is necessary for the formation of a decisive function, the set of reference images is built taking into account changes in navigation parameters of an unmanned aerial vehicle and angular orientation parameters. The originality of the formalized model consists in the determined order of formation of the decisive function of the combined correlation-extreme navigation system, which, unlike the known ones, consists in step-by-step determination of coordinates of the reference object by clarifying the navigation parameters of unmanned aerial vehicles with subsequent refinement of the angular orientation parameters according to the following sequence: determining the height of formation of the current image, determining the angles of view and determining the angular orientation parameters. Each of the individual reference images corresponds to its own multidimensional matrix of luminance values of the corresponding pixels or radio-thermal temperature. This takes into account the area of objects located on the sighting surface, which leads to refinement of the coordinates of the reference object in the current image. Changes in the spatial position and orientation of unmanned aerial vehicles, geometric properties of the sighting surface, as well as taking into account variations in informative features of the sighting surface objects depending on external factors and the influence of the medium on the propagation of signals in the optical and radiometric ranges of waves and obstacles, which will reduce or completely eliminate the discrepancy of coordinates of the reference object to the current image determined in advance in the reference image.

The conventional method for crop insect detection based on visual judgment of the field is time-consuming, laborious, subjective, and error prone. The early detection and accurate localization of agricultural insect pests can significantly improve the effectiveness of pest control as well as reduce the costs, which has become an urgent demand for crop production. Maize Spodoptera frugiperda is a migratory agricultural pest that has severely decreased the yield of maize, rice, and other kinds of crops worldwide. To monitor the occurrences of maize Spodoptera frugiperda in a timely manner, an end-to-end Spodoptera frugiperda detection model termed the Pest Region-CNN (Pest R-CNN) was proposed based on the Faster Region-CNN (Faster R-CNN) model. Pest R-CNN was carried out according to the feeding traces of maize leaves by Spodoptera frugiperda. The proposed model was trained and validated using high-spatial-resolution red–green–blue (RGB) ortho-images acquired by an unmanned aerial vehicle (UAV). On the basis of the severity of feeding, the degree of Spodoptera frugiperda invasion severity was classified into the four classes of juvenile, minor, moderate, and severe. The degree of severity and specific feed location of S. frugiperda infestation can be determined and depicted in the frame forms using the proposed model. A mean average precision (mAP) of 43.6% was achieved by the proposed model on the test dataset, showing the great potential of deep learning object detection in pest monitoring. Compared with the Faster R-CNN and YOLOv5 model, the detection accuracy of the proposed model increased by 12% and 19%, respectively. Further ablation studies showed the effectives of channel and spatial attention, group convolution, deformable convolution, and the multi-scale aggregation strategy in the aspect of improving the accuracy of detection. The design methods of the object detection architecture could provide reference for other research. This is the first step in applying deep-learning object detection to S. frugiperda feeding trace, enabling the application of high-spatial-resolution RGB images obtained by UAVs to S. frugiperda-infested object detection. The proposed model will be beneficial with respect to S. frugiperda pest stress monitoring to realize precision pest control.

Unmanned Aerial Vehicles (UAVs) require frequent transmissions to a ground station. In this paper, we consider Time Division Multiple Access (TDMA) and equip the ground station with a Successive Interference Cancellation (SIC) radio, and thereby, allowing it to receive multiple transmissions simultaneously. A key problem is to identify a suitable TDMA transmission schedule that allows UAVs to transmit successfully and frequently to the ground station. Moreover, in order for SIC to operate, there must be a sufficient difference in received power at the ground station. However, in practice, due to the mobility and orientation of UAVs, the ground station experiences random channel gains. To this end, we adopt a discrete optimization approach to select a transmission schedule that yields the highest expected number of successes against random channel gains. We prove our approach is able to find the optimal transmission schedule. In addition, we propose a novel heuristic approach to generate a subset of transmission schedules for use in large-scale UAV networks. Our results show both proposed approaches yield high throughput under various network conditions. The average number of successful transmissions for schedules generated by our solutions is greater than 70%. In contrast, a competing approach only has an average success rate of less than 50%. Lastly, we conducted a trace-based simulation using data from a testbed with three static or mobile UAVs. Our results show that using a transmission schedule where at most two UAVs transmit yields more transmission successes.

In recent years, the use of unmanned aerial vehicles (UAVs) is widespread in urban traffic monitoring. However, the challenges faced while practical implementation from a fixed-wing UAV's involves compensating for the device vibration, real-time video transmission lag, small object sizes tracking while sending a command from Ground Control Station (GCS) to UAV and vice versa. The constantly changing position and orientation of UAV during different flight maneuvers lead to the loss of tracked objects from the field of view. The paper's primary objective is to develop a gimbal control algorithm for real-time object tracking and object location from a small-unmanned fixed-wing aircraft. The proposed algorithm deals with two scenarios, the first scenario offers gimbal pointing at a given coordinate, while second incorporates gimbal pointing such as to align the optical axis with a certain point in the image plane. The solution proposed is to turn the camera head in azimuth and elevation angles towards the object. The alternative suggested includes rotating the camera head in azimuth and elevation angles towards the object, and the suitable angles were calculated based on the camera's distance from the object, Global Positioning System (GPS) and Inertial Measurement Unit (IMU) (camera positioning and orientation) information. The algorithm was first simulated using MATLAB/Simulink using the same parameter of the actual aircraft UAV and further implemented in fixed-wing UAV. The result advocates the applicability of the algorithm and the future work to be done for robust object tracking.