Spatial Pyramid Research Articles

An improved smoking behavior detection algorithm via incorporating an interference information filtering network

Efficient and accurate identification of smoking behavior in public places is crucial for ensuring public health and safety. However, due to various factors like small target size, complex image background, varying cigarette angles and numerous similar objects, current methods still grapple with challenges such as missed detection and false detection when identifying cigarette targets in smoking behavior. This paper presents an enhanced version of the You Only Look Once version5-small algorithm to address these issues effectively. Firstly, to bolster the model's ability for feature extraction of cigarette targets, the Swin Transformer Block structure is integrated into the backbone network to capture long-range dependencies. Secondly, a novel Hybrid Spatial Pyramid Pooling-Fast with Cross Stage Partial Connection module is built based on the foundation of the Spatial Pyramid Pooling-Fast module by integrating both maximum pooling and average pooling to enhance the fusion ability of the multi-scale feature maps. Thirdly, a novel interference information filtering network is introduced to effectively reduce the impact of noise and confusion caused by similar objects, thus enhancing the performance of the model. According to the experimental results, the accuracy of the improved algorithm on the self-made cigarette target image data set reaches 93.5 %, and the recall reaches 89.1 %.

Enhancing deep reinforcement learning for scale flexibility in real-time strategy games

Real-time strategy (RTS) games present a unique challenge for AI agents due to the combination of several fundamental AI problems. While Deep Reinforcement Learning (DRL) has shown promise in the development of autonomous agents for the genre, existing architectures often struggle with games featuring maps of varying dimensions. This limitation hinders the agent’s ability to generalize its learned strategies across different scenarios. This paper proposes a novel approach that overcomes this problem by incorporating Spatial Pyramid Pooling (SPP) within a DRL framework. We leverage the GridNet architecture’s encoder–decoder structure and integrate an SPP layer into the critic network of the Proximal Policy Optimization (PPO) algorithm. This SPP layer dynamically generates a standardized representation of the game state, regardless of the initial observation size. This allows the agent to effectively adapt its decision-making process to any map configuration. Our evaluations demonstrate that the proposed method significantly enhances the model’s flexibility and efficiency in training agents for various RTS game scenarios, albeit with some discernible limitations when applied to very small maps. This approach paves the way for more robust and adaptable AI agents capable of excelling in sequential decision problems with variable-size observations.

Accurate UAV Small Object Detection Based on HRFPN and EfficentVMamba.

(1) Background: Small objects in Unmanned Aerial Vehicle (UAV) images are often scattered throughout various regions of the image, such as the corners, and may be blocked by larger objects, as well as susceptible to image noise. Moreover, due to their small size, these objects occupy a limited area in the image, resulting in a scarcity of effective features for detection. (2) Methods: To address the detection of small objects in UAV imagery, we introduce a novel algorithm called High-Resolution Feature Pyramid Network Mamba-Based YOLO (HRMamba-YOLO). This algorithm leverages the strengths of a High-Resolution Network (HRNet), EfficientVMamba, and YOLOv8, integrating a Double Spatial Pyramid Pooling (Double SPP) module, an Efficient Mamba Module (EMM), and a Fusion Mamba Module (FMM) to enhance feature extraction and capture contextual information. Additionally, a new Multi-Scale Feature Fusion Network, High-Resolution Feature Pyramid Network (HRFPN), and FMM improved feature interactions and enhanced the performance of small object detection. (3) Results: For the VisDroneDET dataset, the proposed algorithm achieved a 4.4% higher Mean Average Precision (mAP) compared to YOLOv8-m. The experimental results showed that HRMamba achieved a mAP of 37.1%, surpassing YOLOv8-m by 3.8% (Dota1.5 dataset). For the UCAS_AOD dataset and the DIOR dataset, our model had a mAP 1.5% and 0.3% higher than the YOLOv8-m model, respectively. To be fair, all the models were trained without a pre-trained model. (4) Conclusions: This study not only highlights the exceptional performance and efficiency of HRMamba-YOLO in small object detection tasks but also provides innovative solutions and valuable insights for future research.

An Automated Diagnosis Method for Lung Cancer Target Detection and Subtype Classification-Based CT Scans.

When dealing with small targets in lung cancer detection, the YOLO V8 algorithm may encounter false positives and misses. To address this issue, this study proposes an enhanced YOLO V8 detection model. The model integrates a large separable kernel attention mechanism into the C2f module to expand the information retrieval range, strengthens the extraction of lung cancer features in the Backbone section, and achieves effective interaction between multi-scale features in the Neck section, thereby enhancing feature representation and robustness. Additionally, depth-wise convolution and Coordinate Attention mechanisms are embedded in the Fast Spatial Pyramid Pooling module to reduce feature loss and improve detection accuracy. This study introduces a Minimum Point Distance-based IOU loss to enhance correlation between predicted and ground truth bounding boxes, improving adaptability and accuracy in small target detection. Experimental validation demonstrates that the improved network outperforms other mainstream detection networks in terms of average precision values and surpasses other classification networks in terms of accuracy. These findings validate the outstanding performance of the enhanced model in the localization and recognition aspects of lung cancer auxiliary diagnosis.

An improved YOLOv8 safety helmet wearing detection network

In the field of industrial safety, wearing helmets plays a vital role in ensuring workers’ health. Aiming at addressing the complex background in the industrial environment, caused by differences in distance, the helmet small target wearing detection methods for misdetection and omission detection problems are needed. An improved YOLOv8 safety helmet wearing detection network is proposed to enhance the capture of details, improve multiscale feature processing and improve the accuracy of small target detection by introducing Dilation-wise residual attention module, atrous spatial pyramid pooling and normalized Wasserstein distance loss function. Experiments were conducted on the SHWD dataset, and the results showed that the mAP of the improved network improved to 92.0%, which exceeded that of the traditional target detection network in terms of accuracy, recall, and other key metrics. These findings further improved the detection of helmet wearing in complex environments and greatly enhanced the accuracy of detection.

Radiological segmentation of knee meniscus ultrasound images based on boundary constraints and multi-scale fusion network

Knee osteoarthritis (KOA) is one of the most popular joint diseases endangering human health because of its high incidence, disability, and younger onset. However, there is no suitable method for early diagnosis, evaluation, and treatment of KOA. In recent years, some clinical studies have found that ultrasound can detect early changes in KOA in advance, and the automatic segmentation of ultrasound images can achieve rapid and effective quantitative research on KOA. However, in ultrasound images, the soft tissue boundaries of the lesion are blurred, making lesion segmentation difficult. Although the U-Net family is one of the best networks for image segmentation, they still have defects such as blurred segmentation boundaries, distorted morphology, and insufficient accuracy when segmenting ultrasound images. To address this issue, we added attention, atrous spatial pyramid pooling (ASSP), and edge loss function terms into the Unet3+ network, which improved the contour clarity and accuracy of output images (the improved Unet3+ Dice acc = 78.74%. Then, we extract the key features of the improved Unet3+ for outputting meniscus images: calculating meniscus area and distance, where the average accuracy of the area is: area_ avg_ acc = 91.12%, with an average distance accuracy of distance_ avg_ acc = 91.14%. This thesis creates a new dataset collection from West China Hospital, Sichuan University, and automated measurement of knee meniscus protrusion area has been achieved for the first time. This article is the first to apply deep learning to ultrasound image segmentation of the knee meniscus, helping doctors conduct qualitative and quantitative analysis of the diagnosis and treatment of early KOA. The results indicate that the improved Unet3+ can assist doctors in automatically diagnosing and evaluating KOA based on meniscus ultrasound images, which is beneficial for guiding early clinical intervention.

Doublem-net: multi-scale spatial pyramid pooling-fast and multi-path adaptive feature pyramid network for UAV detection

Doublem-net: multi-scale spatial pyramid pooling-fast and multi-path adaptive feature pyramid network for UAV detection

SE-CBAM-YOLOv7: An Improved Lightweight Attention Mechanism-Based YOLOv7 for Real-Time Detection of Small Aircraft Targets in Microsatellite Remote Sensing Imaging

Addressing real-time aircraft target detection in microsatellite-based visible light remote sensing video imaging requires considering the limitations of imaging payload resolution, complex ground backgrounds, and the relative positional changes between the platform and aircraft. These factors lead to multi-scale variations in aircraft targets, making high-precision real-time detection of small targets in complex backgrounds a significant challenge for detection algorithms. Hence, this paper introduces a real-time aircraft target detection algorithm for remote sensing imaging using an improved lightweight attention mechanism that relies on the You Only Look Once version 7 (YOLOv7) framework (SE-CBAM-YOLOv7). The proposed algorithm replaces the standard convolution (Conv) with a lightweight convolutional squeeze-and-excitation convolution (SEConv) to reduce the computational parameters and accelerate the detection process of small aircraft targets, thus enhancing real-time onboard processing capabilities. In addition, the SEConv-based spatial pyramid pooling and connected spatial pyramid convolution (SPPCSPC) module extracts image features. It improves detection accuracy while the feature fusion section integrates the convolutional block attention module (CBAM) hybrid attention network, forming the convolutional block attention module Concat (CBAMCAT) module. Furthermore, it optimizes small aircraft target features in channel and spatial dimensions, improving the model’s feature fusion capabilities. Experiments on public remote sensing datasets reveal that the proposed SE-CBAM-YOLOv7 improves detection accuracy by 0.5% and the mAP value by 1.7% compared to YOLOv7, significantly enhancing the detection capability for small-sized aircraft targets in satellite remote sensing imaging.

Study on a Landslide Segmentation Algorithm Based on Improved High-Resolution Networks

Landslides are a kind of geological hazard with great destructive potential. When a landslide event occurs, a reliable landslide segmentation method is important for assessing the extent of the disaster and preventing secondary disasters. Although deep learning methods have been applied to improve the efficiency of landslide segmentation, there are still some problems that need to be solved, such as the poor segmentation due to the similarity between old landslide areas and the background features and missed detections of small-scale landslides. To tackle these challenges, a proposed high-resolution semantic segmentation algorithm for landslide scenes enhances the accuracy of landslide segmentation and addresses the challenge of missed detections in small-scale landslides. The network is based on the high-resolution network (HR-Net), which effectively integrates the efficient channel attention mechanism (efficient channel attention, ECA) into the network to enhance the representation quality of the feature maps. Moreover, the primary backbone of the high-resolution network is further enhanced to extract more profound semantic information. To improve the network’s ability to perceive small-scale landslides, atrous spatial pyramid pooling (ASPP) with ECA modules is introduced. Furthermore, to address the issues arising from inadequate training and reduced accuracy due to the unequal distribution of positive and negative samples, the network employs a combined loss function. This combined loss function effectively supervises the training of the network. Finally, the paper enhances the Loess Plateau landslide dataset using a fractional-order-based image enhancement approach and conducts experimental comparisons on this enriched dataset to evaluate the enhanced network’s performance. The experimental findings show that the proposed methodology achieves higher accuracy in segmentation performance compared to other networks.

Automatic crack segmentation model based on multi-branch aggregation transformer

Crack detection plays a crucial role in evaluating the safety and durability of civil infrastructure. However, detecting cracks of uneven intensity in complex backgrounds is challenging. To overcome this problem, we propose a dual decoder network (CSMT) based on a multi-branch aggregation Transformer, which uses residual atrous spatial pyramid pooling (RASPP) and Transformer dual decoding branches to extract local and global features of different structures. To enhance global feature extraction, we designed a multi-branch aggregation Transformer (MAT) that adaptively weights the features of two attention heads from spatial and channel dimensions to achieve intra block feature aggregation between dimensions. Meanwhile, to obtain multi-scale semantic information, we constructed a new decoding branch, RASPP, which embeds a squeeze-and-excitation (SE) module and residual structures into standard ASPP. Finally, we propose a feature adaptive fusion module (FAM) to enhance feature fusion between adjacent layers and codec layers. Many experiments on three benchmark datasets have shown that the proposed CSMT segmentation network provides excellent performance in a variety of complex scenarios.

Enhancing Apple Disease Detection with Pixel Pyramid Net through Pixel-Level Segmentation and Spatial Pyramid Pooling Techniques

Background: Apple fruit is prone to various diseases that can significantly affect fruit quality. Methods: This paper proposes a novel approach for apple disease detection utilizing PPN-Pixel Pyramid Net, a neural network architecture designed for pixel-level segmentation tasks. The methodology incorporates spatial pyramid pooling techniques to effectively address input images of diverse sizes and aspect ratios. Result: Experimental results showcase the model’s performance metrics across different classes, providing an in-depth analysis of Accuracy, Intersection over unionand Mean BF Score.

FP-Deeplab: a segmentation model for fabric defect detection

Abstract Deep learning for defect detection has become prevalent to improve fabric production and efficiency and quality. However, fabric defect detection faces challenges such as low recognition ratio, suboptimal classification performance, poor detection inefficiency, and high model complexity. To address these issues, an end-to-end semantic segmentation network is proposed employing an efficient encoder-decoder structure, denoted as Feature Pyramid-Deeplab (FP-Deeplab). The improvements involves enhancing the backbone network by improving the MobileNetV3 network for superior performance, a novel Atrous Spatial Pyramid Pooling with Dilated Strip Pooling (ASPP-DSP) module which combines strip pooling, dilated convolution and ASPP, to ensure an expanded receptive field and the capability to gather distant contextual information. Additionally, a Feature Pyramid module (FP module) is proposed to integrate multiscale features at various stages more efficiently. The incorporating of depth-wise separable convolution in FP-Deeplab enables significant parameter and computational cost reduction, catering to online detection requirements. Experimental results certify the superiority of FP-Deeplab over classical and recent segmentation models. Comparative analysis demonstrates higher segmentation accuracy and reduced parameter quantity. Specifically, compared to the benchmark Deeplabv3+ model with MobileV2 as the backbone, FP-Deeplab achieves a notable increase in segmentation accuracy (F1 score and MIoU) by 4.26% and 5.81%, respectively. Moreover, the model parameters (params) are only one-fifth of the original model, indicating the efficiency and effectiveness of our proposed approach.

Improved Chinese Giant Salamander Parental Care Behavior Detection Based on YOLOv8.

Optimizing the breeding techniques and increasing the hatching rate of Andrias davidianus offspring necessitates a thorough understanding of its parental care behaviors. However, A. davidianus' nocturnal and cave-dwelling tendencies pose significant challenges for direct observation. To address this problem, this study constructed a dataset for the parental care behavior of A. davidianus, applied the target detection method to this behavior for the first time, and proposed a detection model for A. davidianus' parental care behavior based on the YOLOv8s algorithm. Firstly, a multi-scale feature fusion convolution (MSConv) is proposed and combined with a C2f module, which significantly enhances the feature extraction capability of the model. Secondly, the large separable kernel attention is introduced into the spatial pyramid pooling fast (SPPF) layer to effectively reduce the interference factors in the complex environment. Thirdly, to address the problem of low quality of captured images, Wise-IoU (WIoU) is used to replace CIoU in the original YOLOv8 to optimize the loss function and improve the model's robustness. The experimental results show that the model achieves 85.7% in the mAP50-95, surpassing the YOLOv8s model by 2.1%. Compared with other mainstream models, the overall performance of our model is much better and can effectively detect the parental care behavior of A. davidianus. Our research method not only offers a reference for the behavior recognition of A. davidianus and other amphibians but also provides a new strategy for the smart breeding of A. davidianus.

Spindle Detection Based on Elastic Time Window and Spatial Pyramid Pooling.

Sleep spindles have emerged as valuable biomarkers for assessing cognitive abilities and related disorders, underscoring the importance of their detection in clinical research. However, template matching-based algorithms using fixed templates may not be able to fully adapt to spindles of different durations. Moreover, inspired by the multiscale feature extraction of images, the use of multiscale feature extraction methods can be used to better adapt to spindles of different frequencies and durations. Therefore, this study proposes a novel automatic spindle detection algorithm based on elastic time windows and spatial pyramid pooling (SPP) for extracting multiscale features. The algorithm utilizes elastic time windows to segment electroencephalogram (EEG) signals, enabling the extraction of features across multiple scales. This approach accommodates significant variations in spindle duration and polarization positioning during different EEG epochs. Additionally, spatial pyramid pooling is integrated into a depthwise separable convolutional (DSC) network to perform multiscale pooling on the segmented spindle signal features at different scales. Compared with existing template matching algorithms, this algorithm's spindle wave polarization positioning is more consistent with the real situation. Experimental results conducted on the public dataset DREAMS show that the average accuracy of this algorithm reaches 95.75%, with an average negative predictive value (NPV) of 96.55%, indicating its advanced performance. The effectiveness of each module was verified through thorough ablation experiments. More importantly, the algorithm shows strong robustness when faced with changes in different experimental subjects. This feature makes the algorithm more accurate at identifying sleep spindles and is expected to help experts automatically detect spindles in sleep EEG signals, reduce the workload and time of manual detection, and improve efficiency.

Vehicle-infrastructure cooperative 3D target detection based on Feature Prediction Atrous Spatial Pyramid Pooling Net

Vehicle-infrastructure cooperative 3D target detection based on Feature Prediction Atrous Spatial Pyramid Pooling Net

Intelligent inversion of mesoscale eddy temperature anomaly profiles based on multi-source remote sensing data

Inversion of the underwater temperature anomaly of mesoscale eddies from sea surface remote sensing data plays an important role in understanding the three-dimensional structure of eddies. In this article, a neural network structure, named the eddy temperature anomaly inversion network (EddyTAINet), is proposed and established to invert the vertical temperature anomalies of eddies using 20-year (2002–2022) Argo profiles and multi-source remote sensing data in the northwestern Pacific Ocean. Vertical temperature anomaly profiles with 45 depth levels ranging from 10 to 1000 m are derived as the outputs of the inversion networks. The input feature sequences are extracted from the mesoscale eddy product based on sea level anomaly (SLA) data, and sea surface temperature (SST) data are also introduced into the proposed network structure to improve its accuracy. For the problem of inconsistent sizes of the 2D matrix SST data corresponding to mesoscale eddies, the spatial pyramid pooling (SPP) network module is introduced, which allows different sizes of SST data to be simply convolved and then subjected to hierarchical feature extraction to obtain fixed-size feature vectors. Validation and comparison of the network structure are performed and show the improved performance of EddyTAINet. The average root mean square error (RMSE) and mean absolute error (MAE) are 0.63 °C/0.66 °C and 0.48 °C/0.51 °C for cyclonic and anticyclonic eddies, respectively, which are more than 20% lower than the results of the network using the SLA-based data.

An Identification Method for Mixed Coal Vitrinite Components Based on An Improved DeepLabv3+ Network

To address the high complexity and low accuracy issues of traditional methods in mixed coal vitrinite identification, this paper proposes a method based on an improved DeepLabv3+ network. First, MobileNetV2 is used as the backbone network to reduce the number of parameters. Second, an atrous convolution layer with a dilation rate of 24 is added to the ASPP (atrous spatial pyramid pooling) module to further increase the receptive field. Meanwhile, a CBAM (convolutional block attention module) attention mechanism with a channel multiplier of 8 is introduced at the output part of the ASPP module to better filter out important semantic features. Then, a corrective convolution module is added to the network’s output to ensure the consistency of each channel’s output feature map for each type of vitrinite. Finally, images of 14 single vitrinite components are used as training samples for network training, and a validation set is used for identification testing. The results show that the improved DeepLabv3+ achieves 6.14% and 3.68% improvements in MIOU (mean intersection over union) and MPA (mean pixel accuracy), respectively, compared to the original DeepLabv3+; 12% and 5.3% improvements compared to U-Net; 9.26% and 4.73% improvements compared to PSPNet with ResNet as the backbone; 5.4% and 9.34% improvements compared to PSPNet with MobileNetV2 as the backbone; and 6.46% and 9.05% improvements compared to HRNet. Additionally, the improved ASPP module increases MIOU and MPA by 3.23% and 1.93%, respectively, compared to the original module. The CBAM attention mechanism with a channel multiplier of 8 improves MIOU and MPA by 1.97% and 1.72%, respectively, compared to the original channel multiplier of 16. The data indicate that the proposed identification method significantly improves recognition accuracy and can be effectively applied to mixed coal vitrinite identification.

A deep transfer learning-based algorithm for concrete surface defect detection

The integrity and safety of concrete structures are crucial in modern civil engineering. Traditional image-defect detection methods are significantly affected by light and noise. Methods based on deep learning, particularly convolutional neural networks (CNNs), offer new solutions for automated defect detection. This paper introduces a concrete surface defect detection algorithm based on deep transfer learning. Our model utilizes the ResNet50 architecture as its backbone and incorporates Atrous Spatial Pyramid Pooling (ASPP) to adapt to the diversity of defects and enhance recognition accuracy. In addition, the use of a self-attention mechanism effectively enhances the model’s focus on and analysis of key defect areas, improving its ability to perceive and select features for defects of various sizes, aiding precise localization and classification under complex conditions. Image preprocessing techniques such as resizing, conversion to grayscale, histogram equalization, and noise addition further enhance the robustness of the model in diverse real-world scenarios. On the publicly available COncrete DEfect BRidge IMage Dataset, our model achieved a mean average precision (mAP@0.5) of 0.90, showing a 3.4% improvement over the current best baseline method. Through transfer learning, it reached 0.92 on our custom dataset, significantly outperforming existing methods. These results demonstrate the effectiveness and superiority of the proposed model for detecting concrete defects. Additionally, we evaluated the impact of different architectures on model performance. Compared to traditional models using the VGG16 as the backbone, the ResNet50 architecture used in this study reduces parameter count by 5.41 times while improving performance by 6.1%.

CFNet: Cross-scale fusion network for medical image segmentation

Learning multi-scale feature representations is essential for medical image segmentation. Most existing frameworks are based on U-shape architecture in which the high-resolution representation is recovered progressively by connecting different levels of the decoder with the low-resolution representation from the encoder. However, intrinsic defects in complementary feature fusion inhibit the U-shape from aggregating efficient global and discriminative features along object boundaries. While Transformer can help model the global features, their computation complexity limits the application in real-time medical scenarios. To address these issues, we propose a Cross-scale Fusion Network (CFNet), combining a cross-scale attention module and pyramidal module to fuse multi-stage/global context information. Specifically, we first utilize large kernel convolution to design the basic building block capable of extracting global and local information. Then, we propose a Bidirectional Atrous Spatial Pyramid Pooling (BiASPP), which employs atrous convolution in the bidirectional paths to capture various shapes and sizes of brain tumors. Furthermore, we introduce a cross-stage attention mechanism to reduce redundant information when merging features from two stages with different semantics. Extensive evaluation was performed on five medical image segmentation datasets: a 3D volumetric dataset, namely Brats benchmarks. CFNet-L achieves 85.74% and 90.98% dice score for Enhanced Tumor and Whole Tumor on Brats2018, respectively. Furthermore, our largest model CFNet-L outperformed other methods on 2D medical image. It achieved 71.95%, 82.79%, and 80.79% SE for STARE, DRIVE, and CHASEDB1, respectively. The code will be available at https://github.com/aminabenabid/CFNet

EC-YOLO: Improved YOLOv7 Model for PCB Electronic Component Detection.

Electronic components are the main components of PCBs (printed circuit boards), so the detection and classification of ECs (electronic components) is an important aspect of recycling used PCBs. However, due to the variety and quantity of ECs, traditional target detection methods for EC classification still have problems such as slow detection speed and low performance, and the accuracy of the detection needs to be improved. To overcome these limitations, this study proposes an enhanced YOLO (you only look once) network (EC-YOLOv7) for detecting EC targets. The network uses ACmix (a mixed model that enjoys the benefits of both self-attention and convolution) as a substitute for the 3 × 3 convolutional modules in the E-ELAN (Extended ELAN) architecture and implements branch links and 1 × 1 convolutional arrays between the ACmix modules to improve the speed of feature retrieval and network inference. Furthermore, the ResNet-ACmix module is engineered to prevent the leakage of function data and to minimise calculation time. Subsequently, the SPPCSPS (spatial pyramid pooling connected spatial pyramid convolution) block has been improved by replacing the serial channels with concurrent channels, which improves the fusion speed of the image features. To effectively capture spatial information and improve detection accuracy, the DyHead (the dynamic head) is utilised to enhance the model's size, mission, and sense of space, which effectively captures spatial information and improves the detection accuracy. A new bounding-box loss regression method, the WIoU-Soft-NMS method, is finally suggested to facilitate prediction regression and improve the localisation accuracy. The experimental results demonstrate that the enhanced YOLOv7 net surpasses the initial YOLOv7 model and other common EC detection methods. The proposed EC-YOLOv7 network reaches a mean accuracy (mAP@0.5) of 94.4% on the PCB dataset and exhibits higher FPS compared to the original YOLOv7 model. In conclusion, it can significantly enhance high-density EC target recognition.