Deformable Convolutional Networks Research Articles

GDB‐YOLOv5s: Improved YOLO‐based model for ship detection in SAR images

AbstractIn recent years, deep learning methods were good solutions for object detection in synthetic aperture radar (SAR) images. However, the problems of complex scenarios, large object scale differences and imperfect fine‐grained classification in ship detection were intractable. In response, an improved model GDB‐YOLOv5s (Improved YOLOv5s model incorporating global attention mechanism (GAM), DCN‐v2 and BiFusion) is designed. This model introduces deformable convolution networks (DCN‐v2) into the Backbone to enhance the adaptability of the receptive field. It replaces the original Neck's PANet structure with a BiFusion structure to better fuse the extracted multiscale features. Additionally, it integrates GAM into the network to reduce information loss and improve global feature interaction. Experiments were conducted on single‐class dataset SSDD and multi‐class dataset SRSSD‐V1.0. The results show that the GDB‐YOLOv5s model improves mean average precision scores (mAP) significantly, outperforming the original YOLOv5s model and other traditional methods. GDB‐YOLOv5s also improves Precision‐score (P) and Recall‐score (R) for fine‐grained classification to some extent, thereby reducing false alarms and missed detections. It has been proved that the improved model is relatively effective.

SD-YOLOv8: An Accurate Seriola dumerili Detection Model Based on Improved YOLOv8.

Accurate identification of Seriola dumerili (SD) offers crucial technical support for aquaculture practices and behavioral research of this species. However, the task of discerning S. dumerili from complex underwater settings, fluctuating light conditions, and schools of fish presents a challenge. This paper proposes an intelligent recognition model based on the YOLOv8 network called SD-YOLOv8. By adding a small object detection layer and head, our model has a positive impact on the recognition capabilities for both close and distant instances of S. dumerili, significantly improving them. We construct a convenient S. dumerili dataset and introduce the deformable convolution network v2 (DCNv2) to enhance the information extraction process. Additionally, we employ the bottleneck attention module (BAM) and redesign the spatial pyramid pooling fusion (SPPF) for multidimensional feature extraction and fusion. The Inner-MPDIoU bounding box regression function adjusts the scale factor and evaluates geometric ratios to improve box positioning accuracy. The experimental results show that our SD-YOLOv8 model achieves higher accuracy and average precision, increasing from 89.2% to 93.2% and from 92.2% to 95.7%, respectively. Overall, our model enhances detection accuracy, providing a reliable foundation for the accurate detection of fishes.

OD-YOLO: Robust Small Object Detection Model in Remote Sensing Image with a Novel Multi-Scale Feature Fusion.

As remote sensing technology has advanced, the use of satellites and similar technologies has become increasingly prevalent in daily life. Now, it plays a crucial role in hydrology, agriculture, and geography. Nevertheless, because of the distinct qualities of remote sensing, including expansive scenes and small, densely packed targets, there are many challenges in detecting remote sensing objects. Those challenges lead to insufficient accuracy in remote sensing object detection. Consequently, developing a new model is essential to enhance the identification capabilities for objects in remote sensing imagery. To solve these constraints, we have designed the OD-YOLO approach that uses multi-scale feature fusion to improve the performance of the YOLOv8n model in small target detection. Firstly, traditional convolutions have poor recognition capabilities for certain geometric shapes. Therefore, in this paper, we introduce the Detection Refinement Module (DRmodule) into the backbone architecture. This module utilizes Deformable Convolutional Networks and the Hybrid Attention Transformer to strengthen the model's capability for feature extraction from geometric shapes and blurred objects effectively. Meanwhile, based on the Feature Pyramid Network of YOLO, at the head of the model framework, this paper enhances the detection capability by introducing a Dynamic Head to strengthen the fusion of different scales features in the feature pyramid. Additionally, to address the issue of detecting small objects in remote sensing images, this paper specifically designs the OIoU loss function to finely describe the difference between the detection box and the true box, further enhancing model performance. Experiments on the VisDrone dataset show that OD-YOLO surpasses the compared models by at least 5.2% in mAP50 and 4.4% in mAP75, and experiments on the Foggy Cityscapes dataset demonstrated that OD-YOLO improved mAP by 6.5%, demonstrating outstanding results in tasks related to remote sensing images and adverse weather object detection. This work not only advances the research in remote sensing image analysis, but also provides effective technical support for the practical deployment of future remote sensing applications.

Research on Bearing Surface Scratch Detection Based on Improved YOLOV5.

Bearings are crucial components of machinery and equipment, and it is essential to inspect them thoroughly to ensure a high pass rate. Currently, bearing scratch detection is primarily carried out manually, which cannot meet industrial demands. This study presents research on the detection of bearing surface scratches. An improved YOLOV5 network, named YOLOV5-CDG, is proposed for detecting bearing surface defects using scratch images as targets. The YOLOV5-CDG model is based on the YOLOV5 network model with the addition of a Coordinate Attention (CA) mechanism module, fusion of Deformable Convolutional Networks (DCNs), and a combination with the GhostNet lightweight network. To achieve bearing surface scratch detection, a machine vision-based bearing surface scratch sensor system is established, and a self-made bearing surface scratch dataset is produced as the basis. The scratch detection final Average Precision (AP) value is 97%, which is 3.4% higher than that of YOLOV5. Additionally, the model has an accuracy of 99.46% for detecting defective and qualified products. The average detection time per image is 263.4 ms on the CPU device and 12.2 ms on the GPU device, demonstrating excellent performance in terms of both speed and accuracy. Furthermore, this study analyzes and compares the detection results of various models, demonstrating that the proposed method satisfies the requirements for detecting scratches on bearing surfaces in industrial settings.

A bearing surface defect detection method based on multi-attention mechanism Yolov8

Surface defects in bearings not only affect the appearance but also impact the service life and performance. Therefore, it is imperative for bearing manufacturers to conduct quality inspections before bearings leave the factory. However, traditional visual inspection methods exhibit shortcomings such as high omission rates, insufficient feature fusion and oversized models when dealing with multiple target defects in bearings. To address these challenges, this paper proposes a surface defect detection method for bearings based on an improved Yolov8 algorithm (G-Yolov8). Firstly, a C3Ghost convolutional module based on the Ghost module is constructed in YOLOv8 to simplify model computational costs. Secondly, a global attention mechanism module is designed at the end of the backbone network to increase sensitivity to implicit small target area features and optimize feature extraction efficiency. Subsequently, a deep deformable convolution feature pyramid network is constructed by introducing the deformable convolutional networks version 2 (DCNv2) and the lightweight content-aware reassembly of features upsampling operator to reduce sampling information loss and improve the fusion of multi-scale target defects. Finally, different attention mechanisms are embedded in the detection network to construct a multi-attention detection head to replace the decoupled head, refining classification and localization tasks, reducing feature confusion, and improving the model’s detection accuracy. Experimental results demonstrate that the improved algorithm achieves a 3.5% increase in mean average precision on a self-made small-scale train bearing surface defect dataset, with a 17.3% reduction in model size. This improvement not only enhances accuracy but also addresses the requirement for lightweight deployment in subsequent stages.

Fire Smoke Target Detection Incorporating PBCA

Background: Fire incidents occur in complex scenarios, where the dynamic positions and varying scales of flames and smoke pose challenges for fire detection. To improve the stability, localization accuracy, and detection precision of small targets in fire detection, a fused PBCA method for fire and smoke object detection has been proposed in this paper, called FS-YOLOv8. Background: Fire incidents occur in complex scenarios, where the dynamic positions and varying scales of flames and smoke pose challenges for fire detection. To improve the stability, localization accuracy, and detection precision of small targets in fire detection, a fused PBCA method for fire and smoke object detection has been proposed in this paper, called FS-YOLOv8. Objective: The objective of this approach was to improve the detection accuracy of flames and smoke, enhance the robustness of the system, and strengthen the feature representation capability. It aimed to optimize the contribution of feature maps at different scales, allowing the network to capture inter-channel correlations while preserving precise localization information of the targets. Furthermore, it aimed to enhance the learning ability of small-scale flame and smoke objects. Method: Firstly, DCN (Deformable Convolutional Network) was integrated into the CSPDarknet backbone network to extract features from flame and smoke images. Subsequently, a module called PBCA was designed by combining BiFPN (Bidirectional Feature Pyramid Network) and coordinate attention. Finally, a small object detection layer, YOLO HEAD-4, was constructed. Results: The experimental results of our proposed FS-YOLOv8 method on a self-made dataset demonstrated improved detection accuracy compared to other conventional methods. Therefore, the FS-YOLOv8 method effectively enhanced the performance of object detection in fire and smoke scenarios. Conclusion: The FS-YOLOv8 method has been found to effectively improve the performance of object detection in fire and smoke scenarios, enhance the robustness of the system, strengthen the feature representation capability, and amplify the learning ability of small-scale flame and smoke objects. other: NO

An Integrated Gather-and-Distribute Mechanism and Attention-Enhanced Deformable Convolution Model for Pig Behavior Recognition.

The behavior of pigs is intricately tied to their health status, highlighting the critical importance of accurately recognizing pig behavior, particularly abnormal behavior, for effective health monitoring and management. This study addresses the challenge of accommodating frequent non-rigid deformations in pig behavior using deformable convolutional networks (DCN) to extract more comprehensive features by incorporating offsets during training. To overcome the inherent limitations of traditional DCN offset weight calculations, the study introduces the multi-path coordinate attention (MPCA) mechanism to enhance the optimization of the DCN offset weight calculation within the designed DCN-MPCA module, further integrated into the cross-scale cross-feature (C2f) module of the backbone network. This optimized C2f-DM module significantly enhances feature extraction capabilities. Additionally, a gather-and-distribute (GD) mechanism is employed in the neck to improve non-adjacent layer feature fusion in the YOLOv8 network. Consequently, the novel DM-GD-YOLO model proposed in this study is evaluated on a self-built dataset comprising 11,999 images obtained from an online monitoring platform focusing on pigs aged between 70 and 150 days. The results show that DM-GD-YOLO can simultaneously recognize four common behaviors and three abnormal behaviors, achieving a precision of 88.2%, recall of 92.2%, and mean average precision (mAP) of 95.3% with 6.0MB Parameters and 10.0G FLOPs. Overall, the model outperforms popular models such as Faster R-CNN, EfficientDet, YOLOv7, and YOLOv8 in monitoring pens with about 30 pigs, providing technical support for the intelligent management and welfare-focused breeding of pigs while advancing the transformation and modernization of the pig industry.

Radar Signal Classification with Multi-Frequency Multi-Scale Deformable Convolutional Networks and Attention Mechanisms

Radar Signal Classification with Multi-Frequency Multi-Scale Deformable Convolutional Networks and Attention Mechanisms

A deep learning-based dynamic deformable adaptive framework for locating the root region of the dynamic flames.

Traditional optical flame detectors (OFDs) in flame detection are susceptible to environmental interference, which will inevitably cause detection errors and miscalculations when confronted with a complex environment. The conventional deep learning-based models can mitigate the interference of complex environments by flame image feature extraction, which significantly improves the precision of flame recognition. However, these models focus on identifying the general profile of the static flame, but neglect to effectively locate the source of the dynamic flame. Therefore, this paper proposes a novel dynamic flame detection method named Dynamic Deformable Adaptive Framework (DDAF) for locating the flame root region dynamically. Specifically, to address limitations in flame feature extraction of existing detection models, the Deformable Convolution Network v2 (DCNv2) is introduced for more flexible adaptation to the deformations and scale variations of target objects. The Context Augmentation Module (CAM) is used to convey flame features into Dynamic Head (DH) to feature extraction from different aspects. Subsequently, the Layer-Adaptive Magnitude-based Pruning (LAMP) where the connection with the smallest LAMP score is pruned sequentially is employed to further enhance the speed of model detection. More importantly, both the coarse- and fine-grained location techniques are designed in the Inductive Modeling (IM) to accurately delineate the flame root region for effective fire control. Additionally, the Temporal Consistency-based Detection (TCD) contributes to improving the robustness of model detection by leveraging the temporal information presented in consecutive frames of a video sequence. Compared with the classical deep learning method, the experimental results on the custom flame dataset demonstrate that the AP0.5 value is improved by 4.4%, while parameters and FLOPs are reduced by 25.3% and 25.9%, respectively. The framework of this research extends applicability to a variety of flame detection scenarios, including industrial safety and combustion process control.

Series arc fault detection method based on a residual deformable convolutional network for complex branch circuit

Series arc fault detection method based on a residual deformable convolutional network for complex branch circuit

Analysis of interactive impact and student behavior of teaching online courses based on gamified learning

The potential of gamified learning as an emerging force for educational change has been widely recognized, to address the design deficiencies in the automatic recognition methods in traditional gamified learning models and the low accuracy of learning behavior recognition. Based on the improved gamified learning model, this paper jointly proposes an optimal design and recognition model of learning mode based on a deformable convolutional type network. Based on the gamified machine learning theory and interactive feedback theory, it comprehensively analyzes the procedural improvement of the gamified interactive intelligence model to provide a theoretical and practical basis for carrying out the analysis of the interactive impact and student behavior of teaching online courses based on gamified learning. The research results show that: (1) with the increase of the number of gamified learning samples, the value of iterative loss function decreases, the accuracy of recognition improves, and gradually converges and tends to be stable when the number of training times is 500; the 800 training samples setting can fully meet the network training requirements. (2) The deformable convolutional neural learning network constructed with the proposed joint algorithm reduces the network error during the training process and introduces the radical change function to improve the data processing capability. This improved gamified learning model is not only superior in recognition accuracy but also shows great advantages in recognition time.

Motion sensitive network for action recognition in control and decision-making of autonomous systems.

Spatial-temporal modeling is crucial for action recognition in videos within the field of artificial intelligence. However, robustly extracting motion information remains a primary challenge due to temporal deformations of appearances and variations in motion frequencies between different actions. In order to address these issues, we propose an innovative and effective method called the Motion Sensitive Network (MSN), incorporating the theories of artificial neural networks and key concepts of autonomous system control and decision-making. Specifically, we employ an approach known as Spatial-Temporal Pyramid Motion Extraction (STP-ME) module, adjusting convolution kernel sizes and time intervals synchronously to gather motion information at different temporal scales, aligning with the learning and prediction characteristics of artificial neural networks. Additionally, we introduce a new module called Variable Scale Motion Excitation (DS-ME), utilizing a differential model to capture motion information in resonance with the flexibility of autonomous system control. Particularly, we employ a multi-scale deformable convolutional network to alter the motion scale of the target object before computing temporal differences across consecutive frames, providing theoretical support for the flexibility of autonomous systems. Temporal modeling is a crucial step in understanding environmental changes and actions within autonomous systems, and MSN, by integrating the advantages of Artificial Neural Networks (ANN) in this task, provides an effective framework for the future utilization of artificial neural networks in autonomous systems. We evaluate our proposed method on three challenging action recognition datasets (Kinetics-400, Something-Something V1, and Something-Something V2). The results indicate an improvement in accuracy ranging from 1.1% to 2.2% on the test set. When compared with state-of-the-art (SOTA) methods, the proposed approach achieves a maximum performance of 89.90%. In ablation experiments, the performance gain of this module also shows an increase ranging from 2% to 5.3%. The introduced Motion Sensitive Network (MSN) demonstrates significant potential in various challenging scenarios, providing an initial exploration into integrating artificial neural networks into the domain of autonomous systems.

Power System to Prevent External Damage Detection Method

An improved YOLOv5s multi-scale target detection algorithm is proposed to address the increasingly threatening problem of foreign object intrusion in power transmission lines, which poses a risk to the safety and stable operation of the power grid. To enhance the model's ability to extract features of different sizes, a new C3 feature extraction module was designed. It uses deformable convolutional networks to replace the original fixed convolutional layers, and then introduces the ECA attention mechanism to take into account global context information, reducing the decrease in model detection accuracy caused by feature loss. An improved form of NWD fusion Wise IoU is proposed as the network's predicted box regression loss function, improving the model's generalization ability and its ability to detect features of small targets, and achieving faster network convergence speed. Experimental results show that the improved algorithm has increased the accuracy P, recall rate R, and average precision mAP by 5.21%, 4.71%, and 6.24%, respectively, compared to the original YOLOv5s model in detecting foreign object intrusion in power transmission lines, while also enhancing the detection performance of small targets.

Damage identification of wind turbine blades based on deep learning and ultrasonic testing

ABSTRACT Ultrasonic testing (UT) is an effective method for detecting internal damage in wind turbine blades. However, manual interpretation is time-consuming and subjective. To automate UT in the wind turbine industry, an image detection model based on deep learning called UCD-YOLO (Ultrasonic C-scan image Detection You Only Look Once) is proposed to detect internal damage such as stratified and lack of glue in wind turbine blades. In order to enhance the feature extraction ability under complex damage forms, the use of deformable convolutional networks and context augmentation module are proposed. Designing a lightweight cross stage partial spatial pyramid pool fast (LCSP_SPPF) module, which allows the model to learn more information and improve detection accuracy. The introduction of wise intersection over union (WIoU) improves the convergence speed while reducing the regression error. Using UT devices to collect damage images of wind turbine blades and conducting comprehensive evaluation experiments on the detection performance of the proposed method. Results show that compared to the YOLOv5, UCD-YOLO is not only lighter but also achieves a 7.5% increase in mean average precision (mAP50). And compared with other detection methods, further verifying that the proposed method can quickly and effectively detect wind turbine blade internal damage, improving power generation efficiency.

Dual contrastive attention-guided deformable convolutional network for single image super-resolution

With its powerful ability to model geometric transformations, the deformable convolutional network brings great improvements for single image super-resolution (SISR). Nevertheless, its location-variant sampling method leads to an escalation in spatial variance as the deformable convolutional layers are stacked, consequently resulting in limited performance. Hence, we propose a novel and effective approach called dual contrastive attention-guided deformable convolutional network (DCADCN) for SISR modeling. Specifically, we propose an attention-guided deformable convolutional module with joint inner and external attention mechanisms to fully exploit the correspondences between input and deformation features and preserve spatial characteristics to the extent possible. Additionally, we propose a dual mixed feature extractor consisting of two parallel sub-paths. This design allows for the learning of diverse and complementary spatial features. Furthermore, contrastive learning is applied to further amplify the role of key features and mitigate the interference of noisy features. Extensive experimental results demonstrate that DCADCN is capable of effectively handling classic SISR, SISR with blind noise, and real-world SISR tasks. Moreover, our method achieves comparable or even better performance with lower computational cost compared to state-of-the-art methods.

Vision-Based Multiscale Construction Object Detection under Limited Supervision

Contemporary multiscale construction object detection algorithms rely predominantly on fully-supervised deep learning, requiring arduous and time-consuming labeling process. This paper presents a novel semisupervised multiscale construction objects detection (SS-MCOD) by harnessing nearly infinite unlabeled images along with limited labels, achieving more accurate and robust detection results. SS-MCOD uses a deformable convolutional network (DCN)-based teacher-student joint learning framework. DCN uses deformable advantages to extract and fuse multiscale construction object features. The teacher module generates pseudolabels for construction objects in unlabeled images, while the student module learns the location and classification of construction objects in both labeled images and unlabeled images with pseudolabels. Experimental validation using commonly used construction datasets demonstrates the accuracy and generalization performance of SS-MCOD. This research can provide insights for other detection tasks with limited labels in the construction domain.

A hyperspectral image classification method based on feature enhancement and a hybrid deformable convolution network

ABSTRACT In recent years, some hyperspectral image (HSI) classification methods based on deep models have shown excellent performance. Most deep models receive three-dimensional (3D) block structures as input to extract spectral-spatial features from HSI data. However, the existence of background pixel information in 3D blocks affects the spectral-spatial feature representation ability. In addition, directly performing overall dimension reduction on high-dimensional spectral information lacks analysis of the original data, which weakens the constraint relationship between spectral data and increases the difficulty of feature extraction. Therefore, an HSI classification method based on feature enhancement and hybrid deformable convolution network is proposed. Firstly, a novel 3D block is constructed to enhance spectral-spatial feature representation ability by replacing background pixel information with non-background pixel information. Secondly, based on the analysis results of the original spectral data, this paper proposes a regional dimension reduction mechanism to enhance the constraint relationship between HSI data. Finally, the proposed method is applied to hybrid deformable convolution network for experimental verification. The experimental results of three standard datasets show that the classification effect of the proposed method is better than that of several classification methods, which proves the effectiveness of the proposed method.

YOLOv5s maritime distress target detection method based on swin transformer

AbstractIn recent years, the task of maritime emergency rescue has increased, while the cost of time for traditional methods of search and rescue is pretty long with poor effect subject to the constraints of the complex circumstances around the sea, the effective conditions, and the support capability. This paper applies deep learning and proposes a YOLOv5s‐SwinDS algorithm for target detection in distress at sea. Firstly, the backbone network of the YOLOv5s algorithm is replaced by swin transformer, and a multi‐level feature fusion module is introduced to enhance the feature expression ability for maritime targets. Secondly, deformable convolutional networks v2 (DCNv2) is used instead of traditional convolution to improve the recognition capability for irregular targets when the neck network features are output. Finally, the CIoU loss function is replaced with SIoU to reduce the redundant box effectively while accelerating the convergence and regression of the predicted box. Experimenting on the publicly dataset SeaDronesSee, the , , and of YOLOv5s‐SwinDS model are 87.9%, 75.8%, 79.1% and 42.9%, respectively, which get higher results than the original YOLOv5s model, the YOLOv7 series of models, and the YOLOv8 series of models. The experiments verifies that the algorithm has good performance in detecting maritime distress targets.

SGR: An Improved Point-Based Method for Remote Sensing Object Detection via Dual-Domain Alignment Saliency-Guided RepPoints

With the advancement of deep neural networks, several methods leveraging convolution neural networks (CNNs) have gained prominence in the field of remote sensing object detection. Acquiring accurate feature representations from feature maps is a critical step in CNN-based object detection methods. Previously, region of interest (RoI)-based methods have been widely used, but of late, deformable convolution network (DCN)-based approaches have started receiving considerable attention. A significant challenge in the use of DCN-based methods is the inefficient distribution patterns of sampling points, stemming from a lack of effective and flexible guidance. To address this, our study introduces Saliency-Guided RepPoints (SGR), an innovative framework designed to enhance feature representation quality in remote sensing object detection. SGR employs a dynamic dual-domain alignment (DDA) training strategy to mitigate potential misalignment issues between spatial and feature domains during the learning process. Furthermore, we propose an interpretable visualization method to assess the alignment between feature representation and classification performance in DCN-based methods, providing theoretical analysis and validation for the effectiveness of sampling points. In this study, we assessed the proposed SGR framework through a series of experiments conducted on four varied and rigorous datasets: DOTA, HRSC2016, DIOR-R, and UCAS-AOD, all of which are widely employed in remote sensing object detection. The outcomes of these experiments substantiate the effectiveness of the SGR framework, underscoring its potential to enhance the accuracy of object detection within remote sensing imagery.

Hyperspectral image classification based on channel perception mechanism and hybrid deformable convolution network

Hyperspectral image classification based on channel perception mechanism and hybrid deformable convolution network