RSP-YOLOv11n multi-module optimized algorithm for insulator defect detection in UAV images

In a complex background, insulator fault is the main factor behind transmission accidents. With the wide application of unmanned aerial vehicle (UAV) photography, digital image recognition technology has been further developed to detect the position and fault of insulators. There are two mainstream methods based on deep learning: the first is the “two-stage” example for a region convolutional neural network and the second is the “one-stage” example such as a single-shot multibox detector (SSD), both of which pose many difficulties and challenges. However, due to the complex background and various types of insulators, few researchers apply the “two-stage” method for the detection of insulator faults in aerial images. Moreover, the detection performance of “one-stage” methods is poor for small targets because of the smaller scope of vision and lower accuracy in target detection. In this article, the authors propose an accurate and real-time method for small object detection, an example for insulator location, and its fault inspection based on a mixed-grouped fire single-shot multibox detector (MGFSSD). Based on SSD and deconvolutional single-shot detector (DSSD) networks, the MGFSSD algorithm solves the problems of inaccurate recognition in small objects of the SSD and complex structure and long running time of the DSSD. To resolve the problems of some target repeated detection and small-target missing detection of the original SSD, the authors describe how to design an effective and lightweight feature fusion module to improve the performance of traditional SSDs so that the classifier network can take full advantage of the relationship between the pyramid layer features without changing the base network closest to the input data. The data processing results show that the method can effectively detect insulator faults. The average detection accuracy of insulator faults is 92.4% and the average recall rate is 91.2%.

The power grid plays a vital role in the construction of livelihood projects by transmitting electrical energy. In the event of insulator explosions on power grid towers, these insulators may detach, presenting potential safety risks to transmission lines. The identification of such failures relies on the examination of images captured by unmanned aerial vehicles (UAVs). However, accurately detecting insulator defects remains challenging, particularly when dealing with variations in size. Existing methods exhibit limited accuracy in detecting small objects. In this paper, we propose a novel detection method that incorporates the convolutional block attention module (CBAM) as an attention mechanism into the backbone of the "you only look once" version 5 (YOLOv5) model. Additionally, we integrate a residual structure into the model to learn additional information and features related to insulators, thereby enhancing detection efficiency. Experimental results demonstrate that our proposed method achieved F1 scores of 0.87 for insulator detection and 0.89 for insulator defect detection. The improved YOLOv5 network shows promise in detecting insulators and their defects in UAV images.

Comparison of UAV and WorldView-2 imagery for mapping leaf area index of mangrove forest

Less effective information is obtained by the object detection network, due to the small size of the detection object in the entire image, the complex background, and the dense object in unmanned aerial vehicle (UAV) images. In response to the difficulties encountered, a small object detection method in UAV images is proposed as an improved YOLOv5-based algorithm in this paper. First, the space-to-depth(SPD) conv module is introduced into the basic feature extraction network, to improve significant loss of image information during downsampling. Then, various attention mechanisms are added, to intensify the acquisition of regions of interest in UAV images. Finally, the multiscale detection module is improved, to enhance the network's ability to detect small objects in UAV images. By conducting experiments on the VisDrone-DET2019 dataset, the test results of the established model show. The improved algorithm achieved a Mean Average Precision (mAP) of 41.8%, which is 7.8% better than the baseline network. In addition, the detection performance is better than most current mainstream target detection algorithms and is of some practical value.

Insulators are critical electric components in transmission lines. Recognizing insulators and detecting the faults timely and accurately is essential for maintaining the safety and stability of transmission lines. Traditional methods have low accuracy and poor applicability in insulator recognition and fault detection. An insulator recognition and fault detection model was proposed in the article aiming at improving the insulator recognition and fault detection accuracy. First, based on the faster region convolutional neural network (RCNN), the feature pyramid networks (FPNs) were used to improve the Faster RCNN model and locate the insulators with complex background image. Then, the target area was clipped to remove the redundant background noise, and the hue, saturation, and value (HSV) color space adaptive threshold algorithm was applied for image segmentation due to the influence of light, background noise, and shooting angle. Finally, line detection, image rotation, and vertical projection were used to finish the insulator fault detection. The experimental results show that the proposed insulator recognition and fault detection model can recognize the insulators and detect fault types with better accuracy and achieve a mean average precision (mAP) of 90.8% for glass insulators and 91.7% for composite insulators on the testing dataset. Additionally, the proposed method meets the intelligent inspection of insulator faults in transmission lines and has good engineering application value.

Insulator defect detection of transmission line insulators is an important task for unmanned aerial vehicle (UAV) inspection, which is of immense importance in ensuring the stable operation of transmission lines. Transmission line insulators exist in complex weather scenarios, with small and inconsistent shapes. These insulators under various weather conditions could result in low-quality images captured, limited data numbers, and imbalanced sample problems. Traditional detection methods often struggle to accurately identify defect information, resulting in missed or false detections in real-world scenarios. In this paper, we propose a weather domain synthesis network for extracting cross-modality discriminative information on multi-domain insulator defect detection and classification tasks. Firstly, we design a novel weather domain synthesis (WDSt) module to convert various weather-conditioned insulator images to the uniform weather domain to decrease the existing domain gap. To further improve the detection performance, we leverage the attention mechanism to construct the Cross-modality Information Attention YOLO (CIA-YOLO) model to improve the detection capability for insulator defects. Here, we fuse both shallow and deep feature maps by adding the extra object detection layer, increasing the accuracy for detecting small targets. The experimental results prove the proposed Cross-modality Information Attention YOLO with the weather domain synthesis algorithm can achieve superior performance in multi-domain insulator datasets (MD-Insulator). Moreover, the proposed algorithm also gives a new perspective for decreasing the multi-domain insulator modality gap with weather-domain transfer, which can inspire more researchers to focus on the field.

Regular inspection of the insulator operating status is essential to ensure the safe and stable operation of the power system. Unmanned aerial vehicle (UAV) inspection has played an important role in transmission line inspection, replacing former manual inspection. With the development of deep learning technologies, deep learning-based insulator defect detection methods have drawn more and more attention and gained great improvement. However, former insulator defect detection methods mostly focus on designing complex refined network architecture, which will increase inference complexity in real applications. In this paper, we propose a novel efficient cross-modality insulator augmentation algorithm for multi-domain insulator defect detection to mimic real complex scenarios. It also alleviates the overfitting problem without adding the inference resources. The high-resolution insulator cross-modality translation (HICT) module is designed to generate multi-modality insulator images with rich texture information to eliminate the adverse effects of existing modality discrepancy. We propose the multi-domain insulator multi-scale spatial augmentation (MMA) module to simultaneously augment multi-domain insulator images with different spatial scales and leverage these fused images and location information to help the target model locate defects with various scales more accurately. Experimental results prove that the proposed cross-modality insulator augmentation algorithm can achieve superior performance in public UPID and SFID insulator defect datasets. Moreover, the proposed algorithm also gives a new perspective for improving insulator defect detection precision without adding inference resources, which is of great significance for advancing the detection of transmission lines.

Recent years have witnessed the fast development of UAVs (unmanned aerial vehicles). As an alternative to traditional image acquisition methods, UAVs bridge the gap between terrestrial and airborne photogrammetry and enable flexible acquisition of high resolution images. However, the georeferencing accuracy of UAVs is still limited by the low-performance on-board GNSS and INS. This paper investigates automatic geo-registration of an individual UAV image or UAV image blocks by matching the UAV image(s) with a previously taken georeferenced image, such as an individual aerial or satellite image with a height map attached or an aerial orthophoto with a DSM (digital surface model) attached. As the biggest challenge for matching UAV and aerial images is in the large differences in scale and rotation, we propose a novel feature matching method for nadir or slightly tilted images. The method is comprised of a dense feature detection scheme, a one-to-many matching strategy and a global geometric verification scheme. The proposed method is able to find thousands of valid matches in cases where SIFT and ASIFT fail. Those matches can be used to geo-register the whole UAV image block towards the reference image data. When the reference images offer high georeferencing accuracy, the UAV images can also be geolocalized in a global coordinate system. A series of experiments involving different scenarios was conducted to validate the proposed method. The results demonstrate that our approach achieves not only decimeter-level registration accuracy, but also comparable global accuracy as the reference images.

In recent years, unmanned aerial vehicle (UAV) inspection has become one of the main means of daily operation and maintenance of overhead transmission lines. Since the key components of the UAV inspection system are composed of electronic components, it relies on data communication to realize data link transmission. In the actual operation process, the complex electromagnetic field environment has an obvious impact on its control and communication performance, especially when inspecting near the DC line. In this paper, combined with the research on the safety assurance technology of UAV inspection operations, the electromagnetic field simulation model of the UAV inspection operation on DC transmission lines is established. The electromagnetic field distribution around the wire is analyzed when the UAV is at different inspection distances, and the influence range and degree of the DC electromagnetic field on UAV inspection system are obtained. A true test platform for the inspection safety distance of the ±500 kV line UAV inspection was built, and the inspection safety distance test and research of the small rotary-wing UAV with a typical size, structure, and material was carried out. Through the test, when the current of the transmission line is 3 kA, the minimum safety distance of the UAV inspection operation of the ±500 kV DC transmission line straight tower is 3 m, and the distance is corrected in combination with the current. The research results of this paper can provide technical support for the inspection operation of small multi-rotor UAVs on ±500 kV transmission lines.

With special consideration for complex scenes and densely distributed small objects, this frequently leads to serious false and missed detections for unmanned aerial vehicle (UAV) images in small object detection scenarios. Consequently, we propose a UAV image small object detection algorithm, termed SMA-YOLO. Firstly, a parameter-free simple slicing convolution (SSC) module is integrated in the backbone network to slice the feature maps and enhance the features so as to effectively retain the features of small objects. Subsequently, to enhance the information exchange between upper and lower layers, we design a special multi-cross-scale feature pyramid network (M-FPN). The C2f-Hierarchical-Phantom Convolution (C2f-HPC) module in the network effectively reduces information loss by fine-grained multi-scale feature fusion. Ultimately, adaptive spatial feature fusion detection Head (ASFFDHead) introduces an additional P2 detection head to enhance the resolution of feature maps to better locate small objects. Moreover, the ASFF mechanism is employed to optimize the detection process by filtering out information conflicts during multi-scale feature fusion, thereby significantly optimizing small object detection capability. Using YOLOv8n as the baseline, SMA-YOLO is evaluated on the VisDrone2019 dataset, achieving a 7.4% improvement in mAP@0.5 and a 13.3% reduction in model parameters, and we also verified its generalization ability on VAUDT and RSOD datasets, which demonstrates the effectiveness of our approach.

Object detection in Unmanned Aerial Vehicle (UAV) images has emerged as a focal area of research, which presents two significant challenges: i) objects are typically small and dense within vast images; ii) computational resource constraints render most models unsuitable for real-time deployment. Current real-time object detectors are not optimized for UAV images, and complex methods designed for small object detection often lack real-time capabilities. To address these challenges, we propose a novel detector, RemDet (Reparameter efficient multiplication Detector). Our contributions are as follows: 1) Rethinking the challenges of existing detectors for small and dense UAV images, and proposing information loss as a design guideline for efficient models. 2) We introduce the ChannelC2f module to enhance small object detection performance, demonstrating that high-dimensional representations can effectively mitigate information loss. 3) We design the GatedFFN module to provide not only strong performance but also low latency, effectively addressing the challenges of real-time detection. Our research reveals that GatedFFN, through the use of multiplication, is more cost-effective than feed-forward networks for high-dimensional representation. 4) We propose the CED module, which combines the advantages of ViT and CNN downsampling to effectively reduce information loss. It specifically enhances context information for small and dense objects. Extensive experiments on large UAV datasets, Visdrone and UAVDT, validate the real-time efficiency and superior performance of our methods. On the challenging UAV dataset VisDrone, our methods not only provided state-of-the-art results, improving detection by more than 3.4%, but also achieve 110 FPS on a single 4090.

Optimization of unmanned aerial vehicle inspection strategy for infrastructure based on model-enabled diagnostics and prognostics

Object detection in unmanned aerial vehicle (UAV) images has become a popular research topic in recent years. However, UAV images are captured from high altitudes with a large proportion of small objects and dense object regions, posing a significant challenge to small object detection. To solve this issue, we propose an efficient YOLOv7-UAV algorithm in which a low-level prediction head (P2) is added to detect small objects from the shallow feature map, and a deep-level prediction head (P5) is removed to reduce the effect of excessive down-sampling. Furthermore, we modify the bidirectional feature pyramid network (BiFPN) structure with a weighted cross-level connection to enhance the fusion effectiveness of multi-scale feature maps in UAV images. To mitigate the mismatch between the prediction box and ground-truth box, the SCYLLA-IoU (SIoU) function is employed in the regression loss to accelerate the training convergence process. Moreover, the proposed YOLOv7-UAV algorithm has been quantified and compiled in the Vitis-AI development environment and validated in terms of power consumption and hardware resources on the FPGA platform. The experiments show that the resource consumption of YOLOv7-UAV is reduced by 28%, the mAP is improved by 3.9% compared to YOLOv7, and the FPGA implementation improves the energy efficiency by 12 times compared to the GPU.

The Sentinel-2 multispectral (S2-MS) images, equipped with three red-edge (Red-E) bands, serve as an optimal data source for vegetation monitoring. However, its spatial resolution of 10–20 m restricts greatly its utility for local, precise monitoring. The widely used consumer-grade unmanned aerial vehicle (UAV) provides much finer spatial resolution images but typically only in the visible and near-infrared spectral bands. UAV and S2-MS images have strong complementarity in spatial, temporal, and spectral resolution. This paper establishes a spatio-temporal-spectral (STS) fusion framework for downscaling S2-MS images using UAV images. First, the spatio-temporal (ST) fusion method of Spatial and Temporal Adaptive Reflectance Fusion Model (STARFM) is applied to the spatio-spectral (SS) fusion of UAV and S2-MS images, and it is verified to perform better than the existing SS fusion methods and exhibits robustness across spatial scales. Then, CA-STARFM is generated by coupling STARFM with Consistent Adjustment of the Climatology to Actual Observations (CACAO) and used to further optimize SS fusion results, yielding more competent performance. Moreover, the applicability of CA-STARFM to STS fusion is further verified based on the UAVlike image generated by the ST fusion of UAV and S2-MS images. The results indicate that STARFM is competent for SS fusion at large spatial scales, while CA-STARFM can not only optimize the ST fusion of UAV and satellite images but also be promising for SS fusion. Therefore, the proposed fusion framework provides a potential solution to integrate spatial, temporal, and spectral information of UAV and S2-MS images for precise monitoring.

Objects in unmanned aerial vehicle (UAV) images are generally small due to the high-photography altitude. Although many efforts have been made in object detection, how to accurately and quickly detect small objects is still one of the remaining open challenges. In this paper, we propose a feature fusion and scaling-based single shot detector (FS-SSD) for small object detection in the UAV images. The FS-SSD is an enhancement based on FSSD, a variety of the original single shot multibox detector (SSD). We add an extra scaling branch of the deconvolution module with an average pooling operation to form a feature pyramid. The original feature fusion branch is adjusted to be better suited to the small object detection task. The two feature pyramids generated by the deconvolution module and feature fusion module are utilized to make predictions together. In addition to the deep features learned by the FS-SSD, to further improve the detection accuracy, spatial context analysis is proposed to incorporate the object spatial relationships into object redetection. The interclass and intraclass distances between different object instances are computed as a spatial context, which proves effective for multiclass small object detection. Six experiments are conducted on the PASCAL VOC dataset and the two UAV image datasets. The experimental results demonstrate that the proposed method can achieve a comparable detection speed but an accuracy superior to those of the six state-of-the-art methods.