Feature Pyramid Network Research Articles

Application of improved deep learning algorithm in obstacle avoidance of UAV

Abstract To enhance the intelligent obstacle avoidance capabilities of unmanned aerial vehicles (UAVs), we introduce a refined version of the You Only Look Once (YOLO) algorithm, termed Improved YOLO (I-YOLO). This novel algorithm not only maintains the swift detection speed inherent in the original YOLO but also augments its feature enhancement network. Specifically, we propose a three-branch parallel feature pyramid network (TBPFP) to capture richer semantic information pertaining to the context of small obstacles. Furthermore, we incorporate a generative adversarial network (GAN) in a cascaded manner to synthesize more realistic super-resolution images, thereby bolstering detection accuracy. Experimental evaluations demonstrate that, in comparison to the baseline YOLO algorithm, our I-YOLO variant exhibits superior detection precision and feature extraction capabilities in the perception of intricate environmental targets. While there is a modest decrement in detection speed, it remains adequate to fulfill real-time requirements, particularly in the context of video stream processing.

GLIC: Underwater target detection based on global–local information coupling and multi-scale feature fusion

With the rapid development of object detection technology, underwater object detection has attracted widespread attention. Most of the existing underwater target detection methods are built based on convolutional neural networks (CNNs), which still have some limitations in the utilization of global information and cannot fully capture the key information in the images. To overcome the challenge of insufficient global–local feature extraction, an underwater target detector (namely GLIC) based on global–local information coupling and multi-scale feature fusion is proposed in this paper. Our GLIC consists of three main components: spatial pyramid pooling, global–local information coupling, and multi-scale feature fusion. Firstly, we embed spatial pyramid pooling, which improves the robustness of the model while retaining more spatial information. Secondly, we design the feature pyramid network with global–local information coupling. The global context of the transformer branch and the local features of the CNN branch interact with each other to enhance the feature representation. Finally, we construct a Multi-scale Feature Fusion (MFF) module that utilizes balanced semantic features integrated at the same depth for multi-scale feature fusion. In this way, each resolution in the pyramid receives equal information from others, thus balancing the information flow and making the features more discriminative. As demonstrated in comprehensive experiments, our GLIC, respectively, achieves 88.46%, 87.51%, and 74.94% mAP on the URPC2019, URPC2020, and UDD datasets.

Underwater Target Detection Using Side-Scan Sonar Images Based on Upsampling and Downsampling

Side-scan sonar (SSS) images present unique challenges to computer vision due to their lower resolution, smaller targets, and fewer features. Although the mainstream backbone networks have shown promising results on traditional vision tasks, they utilize traditional convolution to reduce the dimensionality of feature maps, which may cause information loss for small targets and decrease performance in SSS images. To address this problem, based on the yolov8 network, we proposed a new underwater target detection model based on upsampling and downsampling. Firstly, we introduced a new general downsampling module called shallow robust feature downsampling (SRFD) and a receptive field convolution (RFCAConv) in the backbone network. Thereby multiple feature maps extracted by different downsampling techniques can be fused to create a more robust feature map with a complementary set of features. Additionally, an ultra-lightweight and efficient dynamic upsampling module (Dysample) is introduced to improve the accuracy of the feature pyramid network (FPN) in fusing different levels of features. On the underwater shipwreck dataset, our improved model’s mAP50 increased by 4.4% compared to the baseline model.

A Ghost and Attention Mechanism-Based Deep Learning Approach for SAR Small Target Image Detection

Sy nthetic aperture radar (SAR) images play an important role in the task of small target imaging and detection. However, due to its blurred target imaging, complex image background, and challenging feature representation, SAR detection is difficult to get high precision in a complex imaging environment. Therefore, by integrating the ghost model and the Attention-Mechanism (AM), this paper presents a new feature network model based on the improved YOLOv5s for detecting small targets in SAR images efficiently. The key is to design a Ghost_AM module by integrating the ghost bottleneck with the simple AM. This Ghost_AM module is introduced into to the YOLOv5s’ network for improving the YOLOv5s’ extraction performance for small target’s feature from a complex SAR image. Furthermore, to improve the YOLOv5s’ recognition ability for the features extracted from the Ghost_AM, a bidirectional feature pyramid network is use to replace the YOLOv5s’ feature fusion network and get a precise detection. The experiments using some SAR ship image datasets have demonstrated the presented model is better than some traditional SAR target detection algorithms and the original YOLOv5s. Moreover, the presented model’s parameters are reduced by 25.1% compared with the original YOLOv5s, which contributes to its simpler implementation.

SOD-YOLOv8-Enhancing YOLOv8 for Small Object Detection in Aerial Imagery and Traffic Scenes.

Object detection, as a crucial aspect of computer vision, plays a vital role in traffic management, emergency response, autonomous vehicles, and smart cities. Despite the significant advancements in object detection, detecting small objects in images captured by high-altitude cameras remains challenging, due to factors such as object size, distance from the camera, varied shapes, and cluttered backgrounds. To address these challenges, we propose small object detection YOLOv8 (SOD-YOLOv8), a novel model specifically designed for scenarios involving numerous small objects. Inspired by efficient generalized feature pyramid networks (GFPNs), we enhance multi-path fusion within YOLOv8 to integrate features across different levels, preserving details from shallower layers and improving small object detection accuracy. Additionally, we introduce a fourth detection layer to effectively utilize high-resolution spatial information. The efficient multi-scale attention module (EMA) in the C2f-EMA module further enhances feature extraction by redistributing weights and prioritizing relevant features. We introduce powerful-IoU (PIoU) as a replacement for CIoU, focusing on moderate quality anchor boxes and adding a penalty based on differences between predicted and ground truth bounding box corners. This approach simplifies calculations, speeds up convergence, and enhances detection accuracy. SOD-YOLOv8 significantly improves small object detection, surpassing widely used models across various metrics, without substantially increasing the computational cost or latency compared to YOLOv8s. Specifically, it increased recall from 40.1% to 43.9%, precision from 51.2% to 53.9%, mAP0.5 from 40.6% to 45.1%, and mAP0.5:0.95 from 24% to 26.6%. Furthermore, experiments conducted in dynamic real-world traffic scenes illustrated SOD-YOLOv8's significant enhancements across diverse environmental conditions, highlighting its reliability and effective object detection capabilities in challenging scenarios.

Computer-aided diagnosis of pituitary microadenoma on dynamic contrast-enhanced MRI based on spatio-temporal features

Computer-aided diagnosis (CAD) of pituitary microadenoma (PM) can assist doctors in decision-making, leading to improved lesion detection rates and diagnostic accuracy. However, the performance of existing CAD methods for PM detection has been hindered by the difficulty in obtaining high-quality segmentation results. This is primarily due to the small size of PM lesions and the relatively low resolution of Magnetic Resonance Imaging (MRI) images. To address these challenges, this paper proposes a new medical image detection and segmentation model based on spatio-temporal information. The proposed model aims to addresses the disease classification of PM by designing a network module based on multi-scale feature fusion. This module ensures comprehensive extraction of target semantic information while retaining clear spatial information, achieving classification from dynamic contrast-enhanced MRI(DCE-MRI) to identify positive PM samples. For the lesion segmentation of PM, after ROI Align alignment, the model further adds a semantic segmentation module named Dual-path Semantic Segmentation Module (DSSM) behind the mask head and classification head. This module captures more precise spatio-temporal semantic information, reducing accuracy loss and achieving pituitary segmentation. Finally, leveraging the results of pituitary detection, a feature pyramid network (FPN) layer is redesigned named Reuse Underlying Information Module (RUIM) to reuse low-level information, enhancing the detection capability for PM and thus achieving precise object detection and segmentation. The proposed model achieves an accuracy of 97.10% for PM, mAP of 50.24%, which is superior to multiple representative deep models for medical data. The code is available at https://github.com/BUCT-IUSRC/Research__PM-CAD.

Three-dimensional reconstruction of porous media by fusing multi-grid image features based on extended feature pyramid network

Microstructural analysis of porous media has significant research value in the study of their macroscopic properties. An accurate reconstruction of the digital microstructure model is a prerequisite for understanding their physical properties and transport properties. Deep learning can accurately extract features from training images for a fast and precise reconstruction of porous media. The reconstruction methods based on deep learning have yielded important achievements in the reconstruction of porous media. Aiming at the complex morphology and multi-scale image information of porous media, this paper proposes a generic end-to-end deep learning-based encoder–decoder generative method for the reconstruction of porous media from a single 2D image. This paper designs an extended feature pyramid network encoder to extract image features of different resolutions and different scales, and a generator network incorporating multi-scale image information. Through the training process of the network, the final three-dimensional images that meets certain requirements are generated by means of confrontation. By introducing the loss function of the pattern density function and the loss function of Wasserstein distance with gradient penalty, we constrain the reconstructed 3D structures. Finally, the reconstructed 3D structures and the target 3D structures are visualized to compare their similar morphologies. The morphological parameter indicators of the reconstructed 3D structures were line with those of the target 3D structure. The microstructure parameters of the 3D structures were also compared and analyzed. For complex porous media images, our method can achieve diverse, accurate, and stable 3D reconstruction.

YOLO-BFRV: An Efficient Model for Detecting Printed Circuit Board Defects.

The small area of a printed circuit board (PCB) results in densely distributed defects, leading to a lower detection accuracy, which subsequently impacts the safety and stability of the circuit board. This paper proposes a new YOLO-BFRV network model based on the improved YOLOv8 framework to identify PCB defects more efficiently and accurately. First, a bidirectional feature pyramid network (BIFPN) is introduced to expand the receptive field of each feature level and enrich the semantic information to improve the feature extraction capability. Second, the YOLOv8 backbone network is refined into a lightweight FasterNet network, reducing the computational load while improving the detection accuracy of minor defects. Subsequently, the high-speed re-parameterized detection head (RepHead) reduces inference complexity and boosts the detection speed without compromising accuracy. Finally, the VarifocalLoss is employed to enhance the detection accuracy for densely distributed PCB defects. The experimental results demonstrate that the improved model increases the mAP by 4.12% compared to the benchmark YOLOv8s model, boosts the detection speed by 45.89%, and reduces the GFLOPs by 82.53%, further confirming the superiority of the algorithm presented in this paper.

A novel target detection method with dual‐domain multi‐frequency feature in side‐scan sonar images

AbstractSide‐scan sonar (SSS) detection is a key method in underwater environmental security and subsea resource development. However, many detection approaches primarily concentrate on tracking the evolution path of optical image object detection tasks when using acoustic images, resulting in complex structures and limited versatility. To tackle this issue, we introduce a pioneering dual‐domain multi‐frequency network (D2MFNet) meticulously crafted to harness the distinct characteristics of SSS image detection. In D2MFNet, a novel method for optimizing and improving the detection sensitivity in different frequency ranges called multi‐frequency combined attention mechanism (MFCAM) is proposed. This mechanism amplifies the relevance of dual‐domain features across different channels and spaces. Moreover, we introduce a dual‐domain feature pyramid network (D2FPN) significantly augments the depth and breadth of feature information in underwater small datasets. The methods offer plug‐and‐play functionality with substantial performance enhancements. Extensive experiments are conducted to validate the efficacy of the proposed techniques, and the results showcase their state‐of‐the‐art performance. MFCAM improves the mAP by 16.9% in the KLSG dataset and 15.5% in the SCTD dataset. The mAP of D2FPN was improved by 8.4% in the KLSG dataset and by 9.8% in the SCTD dataset. The code and models will be publicly available at https://dagshub.com/estrellaww00/D2MFNet.

M-YOLOv8s: An improved small target detection algorithm for UAV aerial photography

The object of UAV target detection usually means small target with complicated backgrounds. In this paper, an object detection model M-YOLOv8s based on UAV aerial photography scene is proposed. Firstly, to solve the problem that the YOLOv8s model cannot adapt to small target detection, a small target detection head (STDH) module is introduced to fuse the location and appearance feature information of the shallow layers of the backbone network. Secondly, Inner-Wise intersection over union (Inner-WIoU) is designed as the boundary box regression loss, and auxiliary boundary calculation is used to accelerate the regression speed of the model. Thirdly, the structure of multi-scale feature pyramid network (MS-FPN) can effectively combine the shallow network information with the deep network information and improve the performance of the detection model. Furthermore, a multi-scale cross-spatial attention (MCSA) module is proposed to expand the feature space through multi-scale branch, and then achieves the aggregation of target features through cross-spatial interaction, which improves the ability of the model to extract target features. Finally, the experimental results show that our model does not only possess fewer parameters, but also the values of mAP0.5 are 6.6% and 5.4% higher than the baseline model on the Visdrone2019 validation dataset and test dataset, respectively. Then, as a conclusion, the M-YOLOv8s model achieves better detection performance than some existing ones, indicating that our proposed method can be more suitable for detecting the small targets.

HPNet: Text Detection Network with Hybrid Attention and Pixel Aggregation for Irregularly-Shaped Nearby Texts

Scene text detection is a challenging topic in computer vision, characterized by complex illumination, irregular shape, and arbitrary size. While recent advancements have been made in scene text detection, it remains difficult to simultaneously distinguish nearby text and accommodate irregularly shaped text. Therefore, this paper introduces HPNet, an enhanced text detector, based on the segmentation method that predicts two-scale results. To improve the shape robustness, the Hybrid Attentional Feature Fusion (HAFF) module is integrated into Feature Pyramid Networks (FPN) to dynamically perform feature fusion. Additionally, to distinguish nearby text, the model predicts the text region covering text instances and the text kernel covering the central region of the text. The improved Pixel Aggregation (PA) algorithm is then utilized to guide the expansion from the text kernel to the text region. Experiments on IC15, Total-Text, and CTW1500 validate the effectiveness of these improvements and the superiority of HPNet. Compared with the previous method PSENet for nearby texts, the proposed HPNet has improved inference speed by 63.6% and F-measure metric by 2.6%, 3.7%, and 2.5% on three datasets, respectively.

Denoising Diffusion Implicit Model for Camouflaged Object Detection

Camouflaged object detection (COD) is a challenging task that involves identifying objects that closely resemble their background. In order to detect camouflaged objects more accurately, we propose a diffusion model for the COD network called DMNet. DMNet formulates COD as a denoising diffusion process from noisy boxes to prediction boxes. During the training stage, random boxes diffuse from ground-truth boxes, and DMNet learns to reverse this process. In the sampling stage, DMNet progressively refines random boxes to prediction boxes. In addition, due to the camouflaged object’s blurred appearance and the low contrast between it and the background, the feature extraction stage of the network is challenging. Firstly, we proposed a parallel fusion module (PFM) to enhance the information extracted from the backbone. Then, we designed a progressive feature pyramid network (PFPN) for feature fusion, in which the upsample adaptive spatial fusion module (UAF) balances the different feature information by assigning weights to different layers. Finally, a location refinement module (LRM) is constructed to make DMNet pay attention to the boundary details. We compared DMNet with other classical object-detection models on the COD10K dataset. Experimental results indicated that DMNet outperformed others, achieving optimal effects across six evaluation metrics and significantly enhancing detection accuracy.

SDD-Net: Soldering defect detection network for printed circuit boards

The rapid detection of soldering defects in printed circuit boards (PCBs) is crucial and a challenge for quality control. Thus, a novel soldering defect detection network (SDD-Net) is proposed based on improvements in YOLOv7-tiny. A fast spatial pyramid pooling block integrating a cross-stage partial network is designed to expand the receptive field and feature extraction ability of the model. A hybrid combination attention mechanism is proposed to boost feature representation. A residual feature pyramid network is subsequently presented to reinforce the capability of multilevel feature fusion to overcome the scale variance issue in PCB soldering defects. Finally, efficient intersection over union loss is applied for bounding box regression to accelerate model convergence while improving localisation precision. SDD-Net achieves a stunning mean average precision of 99.1% on the dataset, producing a 1.8% increase compared with the baseline. The detection speed is boosted to 102 frames/s for input images of 640 × 640 pixels using a mediocre processor. In addition, SDD-Net exhibits outstanding generalisation ability in two public surface defect datasets.

A deep learning approach for automated PCB defect detection: A comprehensive review

The ever-expanding market for electronic devices has significantly heightened the demand for high-quality printed circuit boards (PCBs). Even minor defects in PCBs can pose substantial safety risks for end-users. This article provides a comprehensive review on deep learning-based approaches for PCB defect detection. Our exploration covers various critical aspects, including the classification of PCB defects, automated vision inspection (AVI) techniques, object detection methodologies, and the widespread adoption of deep learning models. Specifically, we focus on the state-of-the-art approach known as region-based Fully Convolutional Network with Feature Pyramid Networks (FPN-RFCN). Additionally, we discuss effective data augmentation techniques and commonly used evaluation metrics in this domain. This review provides valuable insights for researchers, practitioners, and industry professionals engaged in PCB quality assurance.

SAR-NTV-YOLOv8: A Neural Network Aircraft Detection Method in SAR Images Based on Despeckling Preprocessing

Monitoring aircraft using synthetic aperture radar (SAR) images is a very important task. Given its coherent imaging characteristics, there is a large amount of speckle interference in the image. This phenomenon leads to the scattering information of aircraft targets being masked in SAR images, which is easily confused with background scattering points. Therefore, automatic detection of aircraft targets in SAR images remains a challenging task. For this task, this paper proposes a framework for speckle reduction preprocessing of SAR images, followed by the use of an improved deep learning method to detect aircraft in SAR images. Firstly, to improve the problem of introducing artifacts or excessive smoothing in speckle reduction using total variation (TV) methods, this paper proposes a new nonconvex total variation (NTV) method. This method aims to ensure the effectiveness of speckle reduction while preserving the original scattering information as much as possible. Next, we present a framework for aircraft detection based on You Only Look Once v8 (YOLOv8) for SAR images. Therefore, the complete framework is called SAR-NTV-YOLOv8. Meanwhile, a high-resolution small target feature head is proposed to mitigate the impact of scale changes and loss of depth feature details on detection accuracy. Then, an efficient multi-scale attention module was proposed, aimed at effectively establishing short-term and long-term dependencies between feature grouping and multi-scale structures. In addition, the progressive feature pyramid network was chosen to avoid information loss or degradation in multi-level transmission during the bottom-up feature extraction process in Backbone. Sufficient comparative experiments, speckle reduction experiments, and ablation experiments are conducted on the SAR-Aircraft-1.0 and SADD datasets. The results have demonstrated the effectiveness of SAR-NTV-YOLOv8, which has the most advanced performance compared to other mainstream algorithms.

Dense pedestrian face detection in complex environments

To address the problem of dense crowd face detection in complex environments, this paper proposes a face detection model named Deep and Compact Face Detection (DCFD), which adopts an improved lightweight EfficientNetV2 network to replace the backbone network of RetinaFace. A large kernel attention mechanism is introduced to address the face detection task more accurately. The backbone network, an improved efficient channel attention (ECA) mechanism, is added to further improve the algorithm performance. The feature fusion module is an improved neural architecture search feature pyramid network (NAS-FPN) that significantly improves the face detection accuracy in different scenes. To balance the training process of positive and negative samples, we use the focus loss function to replace the traditional cross-entropy loss function. In different environments, the DCFD algorithm has shown efficient face detection performance. This algorithm provides not only a feasible and effective solution for solving the problem of face detection in dense groups but also an important basis for improving the accuracy of face detection models in practical applications.

Tunnel lining quality detection based on the YOLO-LD algorithm

Tunnel lining defects, such as discontinuous reinforcing steel, concrete dehollowing, and incomplete pouring, can substantially undermine structural durability and stability. Addressing limitations such as strong subjectivity and low accuracy in traditional quality assessment methods, we introduce the YOLO-LD tunnel lining quality detection algorithm. This model is an adaptation of the original YOLOv7 algorithm, where the original feature pyramid network is substituted by an asymptotic feature pyramid network, and a convolutional block attention module is added subsequently to backbone extraction. Electromagnetic wave propagation is simulated using gprMax3.0, while the tunnel lining structure is imaged through ground-penetrating radar employing the finite-difference time-domain method. These simulations yield a comprehensive radar image dataset for tunnel lining quality evaluation. A performance comparison of YOLO-LD with four other established algorithms reveals its superiority in detecting the three aforementioned defects, yielding an mF1 score of 91.07 % and a mAP score of 94.13 %. The model demonstrates robust performance in comprehensive defect detection and generalization.

Research on Lightweight Method of Insulator Target Detection Based on Improved SSD.

Aiming at the problems of a large volume, slow processing speed, and difficult deployment in the edge terminal, this paper proposes a lightweight insulator detection algorithm based on an improved SSD. Firstly, the original feature extraction network VGG-16 is replaced by a lightweight Ghost Module network to initially achieve the lightweight model. A Feature Pyramid structure and Feature Pyramid Network (FPN+PAN) are integrated into the Neck part and a Simplified Spatial Pyramid Pooling Fast (SimSPPF) module is introduced to realize the integration of local features and global features. Secondly, multiple Spatial and Channel Squeeze-and-Excitation (scSE) attention mechanisms are introduced in the Neck part to make the model pay more attention to the channels containing important feature information. The original six detection heads are reduced to four to improve the inference speed of the network. In order to improve the recognition performance of occluded and overlapping targets, DIoU-NMS was used to replace the original non-maximum suppression (NMS). Furthermore, the channel pruning strategy is used to reduce the unimportant weight matrix of the model, and the knowledge distillation strategy is used to fine-adjust the network model after pruning, so as to ensure the detection accuracy. The experimental results show that the parameter number of the proposed model is reduced from 26.15 M to 0.61 M, the computational load is reduced from 118.95 G to 1.49 G, and the mAP is increased from 96.8% to 98%. Compared with other models, the proposed model not only guarantees the detection accuracy of the algorithm, but also greatly reduces the model volume, which provides support for the realization of visible light insulator target detection based on edge intelligence.

Segmentation Method of Zanthoxylum bungeanum Cluster Based on Improved Mask R-CNN

The precise segmentation of Zanthoxylum bungeanum clusters is crucial for developing picking robots. An improved Mask R-CNN model was proposed in this study for the segmentation of Zanthoxylum bungeanum clusters in natural environments. Firstly, the Swin-Transformer network was introduced into the model’s backbone as the feature extraction network to enhance the model’s feature extraction capabilities. Then, the SK attention mechanism was utilized to fuse the detailed information into the mask branch from the low-level feature map of the feature pyramid network (FPN), aiming to supplement the image detail features. Finally, the distance intersection over union (DIOU) loss function was adopted to replace the original bounding box loss function of Mask R-CNN. The model was trained and tested based on a self-constructed Zanthoxylum bungeanum cluster dataset. Experiments showed that the improved Mask R-CNN model achieved 84.0% and 77.2% in detection mAP50box and segmentation mAP50mask, respectively, representing a 5.8% and 4.6% improvement over the baseline Mask R-CNN model. In comparison to conventional instance segmentation models, such as YOLACT, Mask Scoring R-CNN, and SOLOv2, the improved Mask R-CNN model also exhibited higher segmentation precision. This study can provide valuable technology support for the development of Zanthoxylum bungeanum picking robots.

A Deep Learning Model for Cervical Optical Coherence Tomography Image Classification.

Objectives: Optical coherence tomography (OCT) has recently been used in gynecology to detect cervical lesions in vivo and proven more effective than colposcopy in clinical trials. However, most gynecologists are unfamiliar with this new imaging technique, requiring intelligent computer-aided diagnosis approaches to help them interpret cervical OCT images efficiently. This study aims to (1) develop a clinically-usable deep learning (DL)-based classification model of 3D OCT volumes from cervical tissue and (2) validate the DL model's effectiveness in detecting high-risk cervical lesions, including high-grade squamous intraepithelial lesions and cervical cancer. Method: The proposed DL model, designed based on the convolutional neural network architecture, combines a feature pyramid network (FPN) with texture encoding and deep supervision. We extracted, represent, and fused four-scale texture features to improve classification performance on high-risk local lesions. We also designed an auxiliary classification mechanism based on deep supervision to adjust the weight of each scale in FPN adaptively, enabling low-cost training of the whole model. Results: In the binary classification task detecting positive subjects with high-risk cervical lesions, our DL model achieved an 81.55% (95% CI, 72.70-88.51%) F1-score with 82.35% (95% CI, 69.13-91.60%) sensitivity and 81.48% (95% CI, 68.57-90.75%) specificity on the Renmin dataset, outperforming five experienced medical experts. It also achieved an 84.34% (95% CI, 74.71-91.39%) F1-score with 87.50% (95% CI, 73.20-95.81%) sensitivity and 90.59% (95% CI, 82.29-95.85%) specificity on the Huaxi dataset, comparable to the overall level of the best investigator. Moreover, our DL model provides visual diagnostic evidence of histomorphological and texture features learned in OCT images to assist gynecologists in making clinical decisions quickly. Conclusions: Our DL model holds great promise to be used in cervical lesion screening with OCT efficiently and effectively.