Feature Pyramid Network Research Articles

Attention-based digital filter with anchor-free feature pyramid learning model for pedestrian detection

Ensuring real-time performance while leveraging pedestrian detection is a crucial prerequisite for intelligent driving technology. The development of lightweight models with good detection accuracy is also crucial. This work proposes a novel method, the Attention Digital Filter with Anchor-Free Feature Pyramid Learning Model (ADFAFPLM), to meet these needs. The suggested method consists of combining two networks: one is a digital filter based on an attention network that eliminates noise and other picture distortions. The attention-based residual network digital filters are chosen for their enhanced filtering performance, adaptability, efficient learning through residual connections, noise suppression, interpretability, and generalization capabilities. Next, from the input crowded and occluded photos, the pedestrian is identified using an anchor-free feature pyramid network. The Eurocity person dataset was used to train the model, and it was also tested on other datasets like CityPersons, INRIA, PennFudan, and Eurocity. The investigation was expanded to include images in hazy, noisy, and occlusion environments, among other environmental conditions. The image resolutions were also considered for analysis and it was observed that with increasing image resolution, the mAP increases. Based on the ablation study, the ADF-AFPLM adopted YOLOv8n with batch size 16, and image size 640 is considered for efficient result with different testing datasets. The model achieved a mean average precision (mAP) of approx. 87% and shows its efficacy over state-of-art models.

DEEP LEARNING FOR VISUAL RECOGNITION

The design and development of an advanced object detection system are presented in this work, which was guided by a thorough literature review and feasibility assessment. The literature review emphasises how object detection techniques have evolved, highlighting the shift from conventional techniques to deep learning approaches. Important developments are highlighted, such as feature pyramid networks, anchor-based localization, and region-based and single-stage detectors. Furthermore, offered are insights about assessment metrics, transfer learning, and data augmentation. A feasibility study assesses the suggested systems operational, technological, and financial viability and finds that it is highly feasible in each of these areas. The architecture of the system is modular and scalable, including backend services, data management, an object detection engine, and a user interface among its constituent parts. Specifications for both functional and non-functional needs are provided, which direct the development of the system. The development phases, resource allocation, development process, and quality assurance procedures are all outlined in the implementation plan. Through the integration of deep learning techniques, the suggested system seeks to achieve high-performance object identification capabilities that are appropriate for a variety of applications while being scalable, reliable, and user-friendly.

Combining transformer global and local feature extraction for object detection

Convolutional neural network (CNN)-based object detectors perform excellently but lack global feature extraction and cannot establish global dependencies between object pixels. Although the Transformer is able to compensate for this, it does not incorporate the advantages of convolution, which results in insufficient information being obtained about the details of local features, as well as slow speed and large computational parameters. In addition, Feature Pyramid Network (FPN) lacks information interaction across layers, which can reduce the acquisition of feature context information. To solve the above problems, this paper proposes a CNN-based anchor-free object detector that combines transformer global and local feature extraction (GLFT) to enhance the extraction of semantic information from images. First, the segmented channel extraction feature attention (SCEFA) module was designed to improve the extraction of local multiscale channel features from the model and enhance the discrimination of pixels in the object region. Second, the aggregated feature hybrid transformer (AFHTrans) module combined with convolution is designed to enhance the extraction of global and local feature information from the model and to establish the dependency of the pixels of distant objects. This approach compensates for the shortcomings of the FPN by means of multilayer information aggregation transmission. Compared with a transformer, these methods have obvious advantages. Finally, the feature extraction head (FE-Head) was designed to extract full-text information based on the features of different tasks. An accuracy of 47.0% and 82.76% was achieved on the COCO2017 and PASCAL VOC2007 + 2012 datasets, respectively, and the experimental results validate the effectiveness of our method.

SCB-YOLOv5: a lightweight intelligent detection model for athletes’ normative movements

Intelligent detection of athlete behavior is beneficial for guiding sports instruction. Existing mature target detection algorithms provide significant support for this task. However, large-scale target detection algorithms often encounter more challenges in practical application scenarios. We propose SCB-YOLOv5, to detect standardized movements of gymnasts. First, the movements of aerobics athletes were captured, labeled using the labelImg software, and utilized to establish the athlete normative behavior dataset, which was then enhanced by the dataset augmentation using Mosaic9. Then, we improved the YOLOv5 by (1) incorporating the structures of ShuffleNet V2 and convolutional block attention module to reconstruct the Backbone, effectively reducing the parameter size while maintaining network feature extraction capability; (2) adding a weighted bidirectional feature pyramid network into the multiscale feature fusion, to acquire precise channel and positional information through the global receptive field of feature maps. Finally, SCB-YOLOv5 was lighter by 56.9% than YOLOv5. The detection precision is 93.7%, with a recall of 99% and mAP value of 94.23%. This represents a 3.53% improvement compared to the original algorithm. Extensive experiments have verified that our method. SCB-YOLOv5 can meet the requirements for on-site athlete action detection. Our code and models are available at https://github.com/qingDu1/SCB-YOLOv5.

Multiple-in-Single-Out Object Detector Leveraging Spiking Neural Membrane Systems and Multiple Transformers.

Most existing multi-scale object detectors depend on multi-level feature maps. The Feature Pyramid Networks (FPN) is a significant architecture for object detection that utilizes these multi-level feature maps. However, the use of FPN also increases the detector's complexity. For object detection methods that only use a single-level feature map, the detection performance is limited to some extent because the single-level feature map cannot balance deep semantic information and shallow detail information. We introduce a novel detector- the Spiking Neural P Multiple-in-Single-out (SNPMiSo) detector to address these challenges. The SNPMiSo detector is constructed based on SNP-like neurons. In SNPMiSo, we employ two kinds of Transformers to boost the important features across different-level feature maps separately. After enhancing the features, we use an incremental upsampling module to upsample and merge the two feature maps. This combined feature map is input into the NAF dilated residual module and the NAF dual-branch detection head. This process allows us to extract multi-scale features and carry out detection tasks. Our tests show promising results: On the COCO dataset, SNPMiSo attains an Average Precision (AP) of 38.7, an improvement of 1.0 AP over YOLOF.In addition, SNPMiSo demonstrates a quicker detection speed, outperforming some advanced multi-level and single-level object detectors.

Real-time task parameter selection method of accounting system based on multi-objective optimization and genetic algorithm.

The progress of the digital economy has promoted the enterprise accounting system. To accelerate the update and evolution of accounting systems, we propose a parameter selection method based on multi-objective optimization and genetic algorithm. Firstly, this article proposes an accounting feature extraction method based on multimodal information embedding. The dual-branch structure and feature pyramid network are used to realize the feature extraction of the information involved in accounting. Then, this article proposes a multi-objective parameter selection method based on a parallel genetic algorithm. By embedding a genetic algorithm in the process of dual-branch model training, the model's ability to sense accounting information is improved. Finally, using the above two methods, an accounting system evaluation method upon recurrent Transformer is proposed to improve the financial situation of enterprises. Our experiments have proven that our approach attains a remarkable performance with an 87.6% F-value, 83.5% mAP value, and 83.4% accuracy. These results position our method at an advanced level globally, showcasing its capability to enhance accounting systems.

Color Hint-guided Ink Wash Painting Colorization with Ink Style Prediction Mechanism

We propose an end-to-end generative adversarial network that allows for controllable ink wash painting generation from sketches by specifying the colors via color hints. To the best of our knowledge, this is the first study for interactive Chinese ink wash painting colorization from sketches. To help our network understand the ink style and artistic conception, we introduced an ink style prediction mechanism for our discriminator, which enables the discriminator to accurately predict the style with the help of a pre-trained style encoder. We also designed our generator to receive multi-scale feature information from the feature pyramid network for detail reconstruction of ink wash painting. Experimental results and user study show that ink wash paintings generated by our network have higher realism and richer artistic conception than existing image generation methods.

Tomato leaf disease detection based on attention mechanism and multi-scale feature fusion.

When detecting tomato leaf diseases in natural environments, factors such as changes in lighting, occlusion, and the small size of leaf lesions pose challenges to detection accuracy. Therefore, this study proposes a tomato leaf disease detection method based on attention mechanisms and multi-scale feature fusion. Firstly, the Convolutional Block Attention Module (CBAM) is introduced into the backbone feature extraction network to enhance the ability to extract lesion features and suppress the effects of environmental interference. Secondly, shallow feature maps are introduced into the re-parameterized generalized feature pyramid network (RepGFPN), constructing a new multi-scale re-parameterized generalized feature fusion module (BiRepGFPN) to enhance feature fusion expression and improve the localization ability for small lesion features. Finally, the BiRepGFPN replaces the Path Aggregation Feature Pyramid Network (PAFPN) in the YOLOv6 model to achieve effective fusion of deep semantic and shallow spatial information. Experimental results indicate that, when evaluated on the publicly available PlantDoc dataset, the model's mean average precision (mAP) showed improvements of 7.7%, 11.8%, 3.4%, 5.7%, 4.3%, and 2.6% compared to YOLOX, YOLOv5, YOLOv6, YOLOv6-s, YOLOv7, and YOLOv8, respectively. When evaluated on the tomato leaf disease dataset, the model demonstrated a precision of 92.9%, a recall rate of 95.2%, an F1 score of 94.0%, and a mean average precision (mAP) of 93.8%, showing improvements of 2.3%, 4.0%, 3.1%, and 2.7% respectively compared to the baseline model. These results indicate that the proposed detection method possesses significant detection performance and generalization capabilities.

NeuroSeg-III: efficient neuron segmentation in two-photon Ca2+ imaging data using self-supervised learning

Two-photon Ca2+ imaging technology increasingly plays an essential role in neuroscience research. However, the requirement for extensive professional annotation poses a significant challenge to improving the performance of neuron segmentation models. Here, we present NeuroSeg-III, an innovative self-supervised learning approach specifically designed to achieve fast and precise segmentation of neurons in imaging data. This approach consists of two modules: a self-supervised pre-training network and a segmentation network. After pre-training the encoder of the segmentation network via a self-supervised learning method without any annotated data, we only need to fine-tune the segmentation network with a small amount of annotated data. The segmentation network is designed with YOLOv8s, FasterNet, efficient multi-scale attention mechanism (EMA), and bi-directional feature pyramid network (BiFPN), which enhanced the model's segmentation accuracy while reducing the computational cost and parameters. The generalization of our approach was validated across different Ca2+ indicators and scales of imaging data. Significantly, the proposed neuron segmentation approach exhibits exceptional speed and accuracy, surpassing the current state-of-the-art benchmarks when evaluated using a publicly available dataset. The results underscore the effectiveness of NeuroSeg-III, with employing an efficient training strategy tailored for two-photon Ca2+ imaging data and delivering remarkable precision in neuron segmentation.

Research on steel surface defect classification method based on deep learning

Surface defects on steel, arising from factors like steel composition and manufacturing techniques, pose significant challenges to industrial production. Efficient and precise detection of these defects is crucial for enhancing production efficiency and product quality. In accordance with these requisites, this paper elects to undertake the detection task predicated on the you only look once (YOLO) algorithm. In this study, we propose a novel approach for surface flaw identification based on the YOLOv5 algorithm, called YOLOv5-KBS. This method integrates attention mechanism and weighted Bidirectional Feature Pyramid Network (BiFPN) into YOLOv5 architecture. Our method addresses issues of background interference and defect size variability in images. Experimental results show that the YOLOv5-KBS model achieves a notable 4.2% increase in mean Average Precision (mAP) and reaches a detection speed of 70 Frames Per Second (FPS), outperforming the baseline model. These findings underscore the effectiveness and potential applications of our proposed method in industrial settings.

SSMA-YOLO: A Lightweight YOLO Model with Enhanced Feature Extraction and Fusion Capabilities for Drone-Aerial Ship Image Detection

Due to the unique distance and angles involved in satellite remote sensing, ships appear with a small pixel area in images, leading to insufficient feature representation. This results in suboptimal performance in ship detection, including potential misses and false detections. Moreover, the complexity of backgrounds in remote sensing images of ships and the clustering of vessels also adversely affect the accuracy of ship detection. Therefore, this paper proposes an optimized model named SSMA-YOLO, based on YOLOv8n. First, this paper introduces a newly designed SSC2f structure that incorporates spatial and channel convolution (SCConv) and spatial group-wise enhancement (SGE) attention mechanisms. This design reduces spatial and channel redundancies within the neural network, enhancing detection accuracy while simultaneously reducing the model’s parameter count. Second, the newly designed MC2f structure employs the multidimensional collaborative attention (MCA) mechanism to efficiently model spatial and channel features, enhancing recognition efficiency in complex backgrounds. Additionally, the asymptotic feature pyramid network (AFPN) structure was designed for progressively fusing multi-level features from the backbone layers, overcoming challenges posed by multi-scale variations. Experiments of the ships dataset show that the proposed model achieved a 4.4% increase in mAP compared to the state-of-the-art single-stage target detection YOLOv8n model while also reducing the number of parameters by 23%.

GBC-BCD: an improved bridge crack detection method based on bidirectional Laplacian pyramid structure with lightweight attention mechanism convolution

ABSTRACT As roads continue to expand globally, the construction of a larger number of bridges has significantly increased the burden on bridge maintenance. Traditional methods for bridge crack detection often suffer from low efficiency and accuracy. To address these issues, we propose an efficient one-stage bridge crack detection method called GBC-BCD based on the YOLOv8n framework. To improve the model’s performance, we introduce the Coordinated Attention mechanism to enhance spatial context awareness and the Bidirectional Feature Pyramid Network to strengthen the ability of feature fusion and multi-scale feature learning. At the same time, in order to make the model more efficient while improving its performance, we introduce the Ghost Module, which significantly reduces the size of the model and raises the speed of object detection. Moreover, we employ transfer learning to improve training efficiency and conserve computational resources for small datasets. Experiments show that our model has the characteristics of lightweight and high speed. Compared with famous lightweight object detection algorithms, the model size is at most reduced by 86.6%, and realises real-time (625 FPS) processing of images ( 640 × 640 ), an 8.3 times increment in speed.

Oriented feature pyramid network for small and dense wheat heads detection and counting

Wheat head detection and counting using deep learning techniques has gained considerable attention in precision agriculture applications such as wheat growth monitoring, yield estimation, and resource allocation. However, the accurate detection of small and dense wheat heads remains challenging due to the inherent variations in their size, orientation, appearance, aspect ratios, density, and the complexity of imaging conditions. To address these challenges, we propose a novel approach called the Oriented Feature Pyramid Network (OFPN) that focuses on detecting rotated wheat heads by utilizing oriented bounding boxes. In order to facilitate the development and evaluation of our proposed method, we introduce a novel dataset named the Rotated Global Wheat Head Dataset (RGWHD). This dataset is constructed by manually annotating images from the Global Wheat Head Detection (GWHD) dataset with oriented bounding boxes. Furthermore, we incorporate a Path-aggregation and Balanced Feature Pyramid Network into our architecture to effectively extract both semantic and positional information from the input images. This is achieved by leveraging feature fusion techniques at multiple scales, enhancing the detection capabilities for small wheat heads. To improve the localization and detection accuracy of dense and overlapping wheat heads, we employ the Soft-NMS algorithm to filter the proposed bounding boxes. Experimental results indicate the superior performance of the OFPN model, achieving a remarkable mean average precision of 85.77% in oriented wheat head detection, surpassing six other state-of-the-art models. Moreover, we observe a substantial improvement in the accuracy of wheat head counting, with an accuracy of 93.97%. This represents an increase of 3.12% compared to the Faster R-CNN method. Both qualitative and quantitative results demonstrate the effectiveness of the proposed OFPN model in accurately localizing and counting wheat heads within various challenging scenarios.

A lightweight improved YOLOv5s model and its deployment for detecting pitaya fruits in daytime and nighttime light-supplement environments

Precise detection and low-cost deployment are the technological basis of intelligent fruit picking. This study proposes a lightweight improved YOLOv5s model to detect pitaya fruits in daytime and nighttime light-supplement environments, and make it successfully deploy in an Android device. This model first uses the module of shufflenetv2 to reconstruct the YOLOv5s backbone network. Then, the study proposes a Concentrated-Comprehensive Convolution Receptive Field Enhancement (C3RFE) module to improve the detection precision of pitaya fruits. Furthermore, a Bidirectional Feature Pyramid Network(BiFPN) feature fusion method is used to enhance the multi-scale feature fusion. Moreover, three optimized Squeeze-and-Excitation (SE) attention modules are added to make full use of the image feature information. Finally, a dynamic label allocation strategy simple Optimal Transport Assignment (simOTA) is used to optimize the YOLOv5s model original label allocation strategy. The experimental results show that the improved model achieves an average precision rate of 97.80 %, with frames per second (FPS) of 139 FPS in a GPU run environment. The model size is only 2.5 MB. Compared to the state-of-the-art SSD, Faster RCNN, YOLOv4, YOLOv4 tiny, YOLOv5s, YOLOv5Lite-s, YOLOXs, YOLOv7, YOLOv7-tiny, YOLOv8n and YOLOv8s, the improved YOLOv5s achieve preferred comprehensive performance in average precision rate, FPS and model size. When deploying this model on the Realme GT Android mobile phone by developing an application based on an NCNN framework, such an application accomplishes real-time pitaya fruit detection with an FPS exceeding 30 FPS. This study can provide technological support for a precise and effective pitaya fruit intelligent picking.

3D Face Reconstruction with Feature Enhancement using Bi-FPN for Forensic Analysis

The representation of facial features in three-dimensional space plays a pivotal role in various applications such as facial recognition, virtual reality, and digital entertainment. However, achieving high-fidelity reconstructions from two-dimensional facial images remains a challenging task, particularly in preserving fine texture details. This research addresses this problem by proposing a novel approach that leverages a combination of advanced techniques, including Resnet, Flame model, Bi-FPN, and a differential render architecture. The primary objective of this study is to enhance texture details in reconstructed 3D facial images. The integration of Bi-FPN (Bi-directional Feature Pyramid Network) enhances feature extraction and fusion across multiple scales, facilitating the preservation of texture details across different regions of the face. The objective is to accurately represent facial features from 2D images in three-dimensional space. By combining these methods, the proposed framework achieves significant improvements in preserving fine texture details and overall facial structure. Experimental results demonstrate the effectiveness of the approach, suggesting its potential for various applications such as virtual try-on and facial animation.

The application prospects of robot pose estimation technology: exploring new directions based on YOLOv8-ApexNet.

Service robot technology is increasingly gaining prominence in the field of artificial intelligence. However, persistent limitations continue to impede its widespread implementation. In this regard, human motion pose estimation emerges as a crucial challenge necessary for enhancing the perceptual and decision-making capacities of service robots. This paper introduces a groundbreaking model, YOLOv8-ApexNet, which integrates advanced technologies, including Bidirectional Routing Attention (BRA) and Generalized Feature Pyramid Network (GFPN). BRA facilitates the capture of inter-keypoint correlations within dynamic environments by introducing a bidirectional information propagation mechanism. Furthermore, GFPN adeptly extracts and integrates feature information across different scales, enabling the model to make more precise predictions for targets of various sizes and shapes. Empirical research findings reveal significant performance enhancements of the YOLOv8-ApexNet model across the COCO and MPII datasets. Compared to existing methodologies, the model demonstrates pronounced advantages in keypoint localization accuracy and robustness. The significance of this research lies in providing an efficient and accurate solution tailored for the realm of service robotics, effectively mitigating the deficiencies inherent in current approaches. By bolstering the accuracy of perception and decision-making, our endeavors unequivocally endorse the widespread integration of service robots within practical applications.

Fine-grained ship image classification and detection based on a vision transformer and multi-grain feature vector FPN model

ABSTRACT In naval and civilian domains, meticulous ship classification and detection are paramount. Nevertheless, predominant research has gravitated toward leveraging Convolutional Neural Network (CNN)-centered methodologies, often overlooking the diverse granularity inherent in ship samples. In our pursuit to holistically extract features from ship images across varying granularities, we present a transformative architecture: the Vision Transformer and Multi-Grain Feature Vector Feature Pyramid Network (ViT-MGFV-FPN). This model synergistically melds the merits of MGFV-FPN with an augmented Vision Transformer (ViT) for a comprehensive image feature extraction. To cater to the extraction of broader image features whilst sidestepping the innate quadratic complexity of traditional ViT, we unveil an enhanced version christened the Global Swin Transformer. Concurrently, the MGFV-FPN is orchestrated to harness the prowess of CNNs in distilling intricate ship attributes. Rigorous empirical evaluations underscore our model’s superiority in juxtaposition with extant CNN and transformer-based paradigms for nuanced ship categorization.

A few-shot target detection method for wildfires: Research example of Fujian Province, China

Unmanned aerial vehicle (UAV) remote-sensing images have a wide range of applications in wildfire monitoring, providing invaluable data for early detection and effective management. This paper proposes an improved few-shot target detection algorithm tailored specifically for wildfire detection. The quality of UAV remote-sensing images is significantly improved by utilizing image enhancement techniques such as Gamma change and Wiener filter, thereby enhancing the accuracy of the detection model. Additionally, ConvNeXt-ECA is used to focus on valid information within the images, which is an improvement of ConvNeXt with the addition of the ECANet attention mechanism. Furthermore, multi-scale feature fusion is performed by adding a feature pyramid network (FPN) to optimize the extracted small target features. The experimental results demonstrate that the improved algorithm achieves a detection accuracy of 93.2%, surpassing Faster R-CNN by 6.6%. Moreover, the improved algorithm outperforms other target detection algorithms YOLOv8, RT-DETR, YoloX, and SSD by 3.4%, 6.4%, 7.6% and 21.1% respectively. This highlights its superior recognition accuracy and robustness in wildfire detection tasks.

LW-YOLO: Lightweight Deep Learning Model for Fast and Precise Defect Detection in Printed Circuit Boards

Printed circuit board (PCB) manufacturing processes are becoming increasingly complex, where even minor defects can impair product performance and yield rates. Precisely identifying PCB defects is critical but remains challenging. Traditional PCB defect detection methods, such as visual inspection and automated technologies, have limitations. While defects can be readily identified based on symmetry, the operational aspect proves to be quite challenging. Deep learning has shown promise in defect detection; however, current deep learning models for PCB defect detection still face issues like large model size, slow detection speed, and suboptimal accuracy. This paper proposes a lightweight YOLOv8 (You Only Look Once version 8)-based model called LW-YOLO (Lightweight You Only Look Once) to address these limitations. Specifically, LW-YOLO incorporates a bidirectional feature pyramid network for multiscale feature fusion, a Partial Convolution module to reduce redundant calculations, and a Minimum Point Distance Intersection over Union loss function to simplify optimization and improve accuracy. Based on the experimental data, LW-YOLO achieved an mAP0.5 of 96.4%, which is 2.2 percentage points higher than YOLOv8; the precision reached 97.1%, surpassing YOLOv8 by 1.7 percentage points; and at the same time, LW-YOLO achieved an FPS of 141.5. The proposed strategies effectively enhance efficiency and accuracy for deep-learning-based PCB defect detection.

High-Efficiency and High-Precision Ship Detection Algorithm Based on Improved YOLOv8n

With the development of the intelligent vision industry, ship detection and identification technology has gradually become a research hotspot in the field of marine insurance and port logistics. However, due to the interference of rain, haze, waves, light, and other bad weather, the robustness and effectiveness of existing detection algorithms remain a continuous challenge. For this reason, an improved YOLOv8n algorithm is proposed for the detection of ship targets under unforeseen environmental conditions. In the proposed method, the efficient multi-scale attention module (C2f_EMAM) is introduced to integrate the context information of different scales so that the convolutional neural network can generate better pixel-level attention to high-level feature maps. In addition, a fully-concatenate bi-directional feature pyramid network (Concatenate_FBiFPN) is adopted to replace the simple superposition/addition of feature map, which can better solve the problem of feature propagation and information flow in target detection. An improved spatial pyramid pooling fast structure (SPPF2+1) is also designed to emphasize low-level pooling features and reduce the pooling depth to accommodate the information characteristics of the ship. A comparison experiment was conducted between other mainstream methods and our proposed algorithm. Results showed that our proposed algorithm outperformed other models by achieving 99.4% of accuracy, 98.2% of precision, 98.5% of recall, 99.1% of mAP@.5, and 85.4% of mAP@.5:.95 on the SeaShips dataset.