Feature Pyramid Network Structure Research Articles

Students’ Classroom Behavior Detection System Incorporating Deformable DETR with Swin Transformer and Light-Weight Feature Pyramid Network

Artificial intelligence (AI) and computer vision technologies have gained significant prominence in the field of education. These technologies enable the detection and analysis of students’ classroom behaviors, providing valuable insights for assessing individual concentration levels. However, the accuracy of target detection methods based on Convolutional Neural Networks (CNNs) can be compromised in classrooms with multiple targets and varying scales, as convolutional operations may result in the loss of location information. In contrast, transformers, which leverage attention mechanisms, have the capability to learn global features and mitigate the information loss caused by convolutional operations. In this paper, we propose a students’ classroom behavior detection system that combines deformable DETR with a Swin Transformer and light-weight Feature Pyramid Network (FPN). By employing a feature pyramid structure, the system can effectively process multi-scale feature maps extracted by the Swin Transformer, thereby improving the detection accuracy for targets of different sizes and scales. Moreover, the integration of the CARAFE lightweight operator into the FPN structure enhances the network’s detection accuracy. To validate the effectiveness of our approach, extensive experiments are conducted on a real dataset of students’ classroom behavior. The experimental results demonstrate a significant 6.1% improvement in detection accuracy compared to state-of-the-art methods. These findings highlight the superiority of our proposed network in accurately detecting and analyzing students’ classroom behaviors. Overall, this research contributes to the field of education by addressing the limitations of CNN-based target detection methods and leveraging the capabilities of transformers to improve accuracy. The proposed system showcases the benefits of integrating deformable DETR, Swin Transformer, and the lightweight FPN in the context of students’ classroom behavior detection. The experimental results provide compelling evidence of the system’s effectiveness and its potential to enhance classroom monitoring and assessment practices.

In-Field Tobacco Leaf Maturity Detection with an Enhanced MobileNetV1: Incorporating a Feature Pyramid Network and Attention Mechanism

The maturity of tobacco leaves plays a decisive role in tobacco production, affecting the quality of the leaves and production control. Traditional recognition of tobacco leaf maturity primarily relies on manual observation and judgment, which is not only inefficient but also susceptible to subjective interference. Particularly in complex field environments, there is limited research on in situ field maturity recognition of tobacco leaves, making maturity recognition a significant challenge. In response to this problem, this study proposed a MobileNetV1 model combined with a Feature Pyramid Network (FPN) and attention mechanism for in situ field maturity recognition of tobacco leaves. By introducing the FPN structure, the model fully exploits multi-scale features and, in combination with Spatial Attention and SE attention mechanisms, further enhances the expression ability of feature map channel features. The experimental results show that this model, with a size of 13.7 M and FPS of 128.12, performed outstandingly well on the task of field maturity recognition of tobacco leaves, achieving an accuracy of 96.3%, superior to classical models such as VGG16, VGG19, ResNet50, and EfficientNetB0, while maintaining excellent computational efficiency and small memory footprint. Experiments were conducted involving noise perturbations, changes in environmental brightness, and occlusions to validate the model’s robustness in dealing with the complex environments that may be encountered in actual applications. Finally, the Score-CAM algorithm was used for result visualization. Heatmaps showed that the vein and color variations of the leaves provide key feature information for maturity recognition. This indirectly validates the importance of leaf texture and color features in maturity recognition and, to some extent, enhances the credibility of the model. The model proposed in this study maintains high performance while having low storage requirements and computational complexity, making it significant for in situ field maturity recognition of tobacco leaves.

IEMask R-CNN: Information-Enhanced Mask R-CNN

The instance segmentation task is relatively difficult in computer vision, which requires not only high-quality masks but also high-accuracy instance category classification. Mask R-CNN has been proven to be a feasible method. However, due to the Feature Pyramid Network (FPN) structure lack useful channel information, global information and low-level texture information, and mask branch cannot obtain useful local-global information, Mask R-CNN is prevented from obtaining high-quality masks and high-accuracy instance category classification. Therefore, we proposed the Information-enhanced Mask R-CNN, called IEMask R-CNN. In the FPN structure of IEMask R-CNN, the information-enhanced FPN will enhance the useful channel information and the global information of the feature maps to solve the issues that the high-level feature map loses useful channel information and inaccurate of instance category classification, meanwhile the bottom-up path enhancement with adaptive feature fusion will ultilize the precise positioning signal in the lower layer to enhance the feature pyramid. In the mask branch of IEMask R-CNN, an encoding-decoding mask head will strength local-global information to gain a high-quality mask. Without bells and whistles, IEMask R-CNN gains significant gains of about 2.60%, 4.00%, 3.17% over Mask R-CNN on MS COCO2017, Cityscapes and LVIS1.0 benchmarks respectively.

Lightweight object detection algorithm based on YOLOv5 for unmanned surface vehicles

Visual detection technology is essential for an unmanned surface vehicle (USV) to perceive the surrounding environment; it can determine the spatial position and category of the object, which provides important environmental information for path planning and collision prevention of the USV. During a close-in reconnaissance mission, it is necessary for a USV to swiftly navigate in a complex maritime environment. Therefore, an object detection algorithm used in USVs should have high detection s peed and accuracy. In this paper, a YOLOv5 lightweight object detection algorithm using a Ghost module and Transformer is proposed for USVs. Firstly, in the backbone network, the original convolution operation in YOLOv5 is upgraded by convolution stacking with depth-wise convolution in the Ghost module. Secondly, to exalt feature extraction without deepening the network depth, we propose integrating the Transformer at the end of the backbone network and Feature Pyramid Network structure in the YOLOv5, which can improve the ability of feature expression. Lastly, the proposed algorithm and six other deep learning algorithms were tested on ship datasets. The results show that the average accuracy of the proposed algorithm is higher than that of the other six algorithms. In particular, in comparison with the original YOLOv5 model, the model size of the proposed algorithm is reduced to 12.24 M, the frames per second reached 138, the detection accuracy was improved by 1.3%, and the mean of average precision (0.5) reached 96.6% (from 95.3%). In the verification experiment, the proposed algorithm was tested on the ship video collected by the “JiuHang 750” USV under different marine environments. The test results show that the proposed algorithm has a significantly improved detection accuracy compared with other lightweight detection algorithms.

Improved Traffic Sign Recognition Algorithm Based on YOLOV4-Tiny

We propose an improved YOLOV4-tiny traffic sign recognition algorithm for easy deployment on mobile or embedded devices to address the problems of a large number of parameters, low recognition accuracy in complex scenarios. The model uses the YOLOV4-tiny network as the basic framework. First, Octave Convolution is introduced into the backbone network to reduce the redundancy of low-frequency features. Second, the convolutional block attention module is used to strengthen the weights of traffic sign regions and reduce the weights of invalid features. Finally, the Feature Pyramid Network structure is replaced by the Simplified Path Aggregation Network structure in the feature fusion stage to enhance the feature information and further reduce the miss detection rate. The experiment proves that our method outperforms YOLOV4-tiny in recognition accuracy and detection speed on the TT100[Formula: see text]K dataset, and can easily meet the requirements of traffic sign recognition.

Benchmarking Deep Learning Models for Instance Segmentation

Instance segmentation has gained attention in various computer vision fields, such as autonomous driving, drone control, and sports analysis. Recently, many successful models have been developed, which can be classified into two categories: accuracy- and speed-focused. Accuracy and inference time are important for real-time applications of this task. However, these models just present inference time measured on different hardware, which makes their comparison difficult. This study is the first to evaluate and compare the performances of state-of-the-art instance segmentation models by focusing on their inference time in a fixed experimental environment. For precise comparison, the test hardware and environment should be identical; hence, we present the accuracy and speed of the models in a fixed hardware environment for quantitative and qualitative analyses. Although speed-focused models run in real-time on high-end GPUs, there is a trade-off between speed and accuracy when the computing power is insufficient. The experimental results show that a feature pyramid network structure may be considered when designing a real-time model, and a balance between the speed and accuracy must be achieved for real-time application.

Water surface object detection using panoramic vision based on improved single-shot multibox detector

In view of the deficiencies in traditional visual water surface object detection, such as the existence of non-detection zones, failure to acquire global information, and deficiencies in a single-shot multibox detector (SSD) object detection algorithm such as remote detection and low detection precision of small objects, this study proposes a water surface object detection algorithm from panoramic vision based on an improved SSD. We reconstruct the backbone network for the SSD algorithm, replace VVG16 with a ResNet-50 network, and add five layers of feature extraction. More abundant semantic information of the shallow feature graph is obtained through a feature pyramid network structure with deconvolution. An experiment is conducted by building a water surface object dataset. Results showed the mean Average Precision (mAP) of the improved algorithm are increased by 4.03%, compared with the existing SSD detecting Algorithm. Improved algorithm can effectively improve the overall detection precision of water surface objects and enhance the detection effect of remote objects.

Monitoring and Prognostic Analysis of Severe Cerebrovascular Diseases Based on Multi-Scale Dynamic Brain Imaging.

Severe cerebrovascular disease is an acute cerebrovascular event that causes severe neurological damage in patients, and is often accompanied by severe dysfunction of multiple systems such as breathing and circulation. Patients with severe cerebrovascular disease are in critical condition, have many complications, and are prone to deterioration of neurological function. Therefore, they need closer monitoring and treatment. The treatment strategy in the acute phase directly determines the prognosis of the patient. The case of this article selected 90 patients with severe cerebrovascular disease who were hospitalized in four wards of the Department of Neurology and the Department of Critical Care Medicine in a university hospital. The included cases were in accordance with the guidelines for the prevention and treatment of cerebrovascular diseases. Patients with cerebral infarction are given routine treatments such as improving cerebral circulation, protecting nutrient brain cells, dehydration, and anti-platelet; patients with cerebral hemorrhage are treated within the corresponding safe time window. We use Statistical Product and Service Solutions (SPSS) Statistics21 software to perform statistical analysis on the results. Based on the study of the feature extraction process of convolutional neural network, according to the hierarchical principle of convolutional neural network, a backbone neural network MF (Multi-Features)—Dense Net that can realize the fusion, and extraction of multi-scale features is designed. The network combines the characteristics of densely connected network and feature pyramid network structure, and combines strong feature extraction ability, high robustness and relatively small parameter amount. An end-to-end monitoring algorithm for severe cerebrovascular diseases based on MF-Dense Net is proposed. In the experiment, the algorithm showed high monitoring accuracy, and at the same time reached the speed of real-time monitoring on the experimental platform. An improved spatial pyramid pooling structure is designed to strengthen the network’s ability to merge and extract local features at the same level and at multiple scales, which can further improve the accuracy of algorithm monitoring by paying a small amount of additional computational cost. At the same time, a method is designed to strengthen the use of low-level features by improving the network structure, which improves the algorithm’s monitoring performance on small-scale severe cerebrovascular diseases. For patients with severe cerebrovascular disease in general, APACHEII1, APACHEII2, APACHEII3 and the trend of APACHEII score change are divided into high-risk group and low-risk group. The overall severe cerebrovascular disease, severe cerebral hemorrhage and severe cerebral infarction are analyzed, respectively. The differences are statistically significant.

Research on a Product Quality Monitoring Method Based on Multi Scale PP-YOLO

To monitor product quality in the production process in real time, this thesis proposes a quality monitoring model based on PaddlePaddle You Only Look Once (PP-YOLO). First, in the preprocessing stage, the data enhancement method and the K-means++ method are used to improve the robustness of the algorithm, and the generated anchor box can screen more refined features earlier. Second, ResNet50-vd with the deformable convolution idea is selected as the backbone of the detection model, the feature pyramid network structure and the composition of the loss function are improved, and the feature learning ability of the model is enhanced to enable it to detect multiple scales of defects. Finally, pruning is performed on the basis of the trained model to reduce the number of model parameters so that it can be deployed in industrial scenarios with limited hardware conditions. Experimental results show that the proposed quality monitoring model can meet the requirements for detection speed and accuracy in actual production, providing a new concept for the deployment of deep learning models in the industrial field.

Learning region-guided scale-aware feature selection for object detection

Scale variation is one of the major challenges in object detection task. Modern region-based object detection architectures often adopt Feature Pyramid Network (FPN) as feature extraction neck to achieve multi-scale feature representation in solving scale variation problem. However, due to the rough feature selection strategy in Region of Interest (RoI) feature extraction step, these methods might not perform well on object detection under strong scale variation. In this work, we are motivated by the limitations of current FPN-based two-stage object detectors and then present a novel module, namely scale-aware feature selective (SAFS) module, that flexibly and adaptively selects feature levels in two-stage object detectors. Specifically, we firstly build the RoI Pyramid in standard FPN structure to extract RoI features from various scale levels. Next, in order to achieve scale-aware mechanism for solving scale variation issue, we develop a novel weighting gate function containing one set of trainable parameters to automatically learn the fusion weight for each RoI feature level, which relieves the limitation of hard feature selection strategy guided by online instance size. Outputs from the RoI features with the learned weights are fused for classification and bounding box regression. Furthermore, we design a multi-level SAFS architecture to obtain different types of RoI feature combinations that ensures our method is more robust to various instance scales. Experimental results show that our SAFS module is very compatible with most of two-stage object detectors and could achieve state-of-the-art results with Average Precision of 48.3 on COCO test-dev and other popular object detection benchmarks. Our code will be made publicly available.

Arbitrary-shaped text detection with adaptive convolution and path enhancement pyramid network

Recently, scene text detection has become an active research field, which is an essential component of scene text reading. Especially, segmentation-based methods are commonly used, since the segmentation results can describe text of arbitrary shape. However, curve texts have a diversity of shapes, scales and orientations, which are difficult to locate, so the detector requires to adjust the local receptive fields size adaptively, which can aggregate multi-scale spatial information to accurately locate the curve text instance. Moreover, the low-level features are critical for localizing large text instances. When using Feature Pyramid Network (FPN) for multi-scale feature fusion, it will prevent the flow of accurate localization signals due to the long path from low-level to top-level. In order to solve these two problems, this paper proposes an Adaptive Convolution and Path Enhancement Pyramid Network (ACPEPNet), which can more accurately locate the text instances with arbitrary shapes. Firstly, an Adaptive Convolution Unit is introduced to improve the ability of backbone to aggregate multi-scale spatial information at the same stage. Specially, this unit is a lightweight component and without the cost of computations, based on this component we present a backbone network for text features extraction. Secondly, the original FPN structure is redesigned to build a short path from the low-level to top-level, in this way, we modify the path from one-way flow to two-way flow and add original features to the final stage of information fusion. Experiments on CTW1500, Total-Text, ICDAR 2015 and MSRA-TD500 validate the robustness of the proposed method. When there is no bells and whistles, this method achieves an F-measure of 80.8% without external training data on CTW1500.

Design of lightweight pedestrian detection network in railway scenes

The deployment of many advanced pedestrian detection applications is largely hindered by the high computational cost of deep convolutional neural networks (CNNs). In this paper, we propose a two-step pruning method to design a lightweight pedestrian detection network in railway scenes. The first step is feature pyramid network (FPN) pruning, which utilizes the characteristic of pedestrian in railway scenes and the FPN structure in YOLOv3. The second step is regular channel pruning, which utilizes network slimming knowledge and is an accelerator-friendly pruning strategy. Our two-step pruning method gives about 88% reduction in parameters and about 74% reduction in computing complexity with comparable detection accuracy.

Spatial Attention Based Real-Time Object Detection Network for Internet of Things Devices

Target detection algorithms for Internet of things (IoT) devices often require both high real-time performance and low computational complexity. Real-time object detection network: You Only Look Once Version 3 (YOLOv3) makes full use of multi-scale features to detect objects by using feature pyramid network structure, and achieves good performance on the premise of guaranteeing fast detection speed. The feature pyramid network of YOLOv3 includes bottom-up feature extraction, top-down sampling and lateral connection of low-level detail features and high-level semantic features. But not all features are useful for object detection. In this article, a novel object detection network Spatial Attention based YOLOv3 (SA-YOLOv3) is proposed. The proposed method adds spatial attention network to the top-down sampling process. The spatial attention network calculates the feature weight matrix based on the up-sampling feature map. SA-YOLOv3 uses the feature weight matrix to filter low-level features and retain more valuable features. Finally, the selected low-level feature map and high-level feature map are concatenated together and feature maps with both spatial information and rich semantic information are obtained. The experimental results on PASCAL VOC2012 datasets and RSOD datasets show that SA-YOLOv3 outperforms YOLOv3.

Small Object Detection Algorithm Based on Feature Pyramid-Enhanced Fusion SSD

In order to improve the detection rate of the traditional single-shot multibox detection algorithm in small object detection, a feature-enhanced fusion SSD object detection algorithm based on the pyramid network is proposed. Firstly, the selected multiscale feature layer is merged with the scale-invariant convolutional layer through the feature pyramid network structure; at the same time, the multiscale feature map is separately converted into the channel number using the scale-invariant convolution kernel. Then, the obtained two sets of pyramid-shaped feature layers are further feature fused to generate a set of enhanced multiscale feature maps, and the scale-invariant convolution is performed again on these layers. Finally, the obtained layer is used for detection and localization. The final location coordinates and confidence are output after nonmaximum suppression. Experimental results on the Pascal VOC 2007 and 2012 datasets confirm that there is a 8.2% improvement in mAP compared to the original SSD and some existing algorithms.