Multi-scale Pooling Research Articles

Automatic Water Body Extraction from SAR Images Based on MADF-Net

Water extraction from synthetic aperture radar (SAR) images has an important application value in wetland monitoring, flood monitoring, etc. However, it still faces the problems of low generalization, weak extraction ability of detailed information, and weak suppression of background noises. Therefore, a new framework, Multi-scale Attention Detailed Feature fusion Network (MADF-Net), is proposed in this paper. It comprises an encoder and a decoder. In the encoder, ResNet101 is used as a solid backbone network to capture four feature levels at different depths, and then the proposed Deep Pyramid Pool (DAPP) module is used to perform multi-scale pooling operations, which ensure that key water features can be captured even in complex backgrounds. In the decoder, a Channel Spatial Attention Module (CSAM) is proposed, which focuses on feature areas that are critical for the identification of water edges by fusing attention weights in channel and spatial dimensions. Finally, the high-level semantic information is effectively fused with the low-level edge features to achieve the final water detection results. In the experiment, Sentinel-1 SAR images of three scenes with different characteristics and scales of water body are used. The PA and IoU of water extraction by MADF-Net can reach 92.77% and 89.03%, respectively, which obviously outperform several other networks. MADF-Net carries out water extraction with high precision from SAR images with different backgrounds, which could also be used for the segmentation and classification of other tasks from SAR images.

Multi-Scale Spatial-Temporal Attention Networks for Functional Connectome Classification.

Many neuropsychiatric disorders are considered to be associated with abnormalities in the functional connectivity networks of the brain. The research on the classification of functional connectivity can therefore provide new perspectives for understanding the pathology of disorders and contribute to early diagnosis and treatment. Functional connectivity exhibits a nature of dynamically changing over time, however, the majority of existing methods are unable to collectively reveal the spatial topology and time-varying characteristics. Furthermore, despite the efforts of limited spatial-temporal studies to capture rich information across different spatial scales, they have not delved into the temporal characteristics among different scales. To address above issues, we propose a novel Multi-Scale Spatial-Temporal Attention Networks (MSSTAN) to exploit the multi-scale spatial-temporal information provided by functional connectome for classification. To fully extract spatial features of brain regions, we propose a Topology Enhanced Graph Transformer module to guide the attention calculations in the learning of spatial features by incorporating topology priors. A Multi-Scale Pooling Strategy is introduced to obtain representations of brain connectome at various scales. Considering the temporal dynamic characteristics between dynamic functional connectome, we employ Locality Sensitive Hashing attention to further capture long-term dependencies in time dynamics across multiple scales and reduce the computational complexity of the original attention mechanism. Experiments on three brain fMRI datasets of MDD and ASD demonstrate the superiority of our proposed approach. In addition, benefiting from the attention mechanism in Transformer, our results are interpretable, which can contribute to the discovery of biomarkers. The code is available at https://github.com/LIST-KONG/MSSTAN.

Rolling bearing fault diagnosis method based on multi-scale pooling residual convolutional neural network under noisy environment

To address the issues of unstable performance and poor generalization ability of bearing fault diagnosis model caused by strong noise and variable operating conditions in actual production，this paper proposes a rolling bearing fault diagnosis method based on MSPRCNN model. By converting vibration signals to frequency domain with FTand utilizing wide convolution kernels for feature extraction, the approach aims to enhance fault detection. A MSPFE module captures information at different scales to simplify complexity, while an UPA module establishes correlations between frequency domain positions. To reduce the impact of noise and address the vanishing gradient problem, the MSPRCNN model employs GC instead of standard convolution, and utilizes the IReLU activation function to improve model feature representation. Experimental results on two datasets show that the fault recognition accuracy is 98.71% under variable loads and 98.2% under variable speeds. The MSPRCNN model outperforms other methods in fault recognition and generalization in noisy environments.

Bilaterally Normalized Scale-Consistent Sinkhorn Distance for Few-Shot Image Classification.

Few-shot image classification aims at exploring transferable features from base classes to recognize images of the unseen novel classes with only a few labeled images. Existing methods usually compare the support features and query features, which are implemented by either matching the global feature vectors or matching the local feature maps at the same position. However, few labeled images fail to capture all the diverse context and intraclass variations, leading to mismatch issues for existing methods. On one hand, due to the misaligned position and cluttered background, existing methods suffer from the object mismatch issue. On the other hand, due to the scale inconsistency between images, existing methods suffer from the scale mismatch issue. In this article, we propose the bilaterally normalized scale-consistent Sinkhorn distance (BSSD) to solve these issues. First, instead of same-position matching, we use the Sinkhorn distance to find an optimal matching between images, mitigating the object mismatch caused by misaligned position. Meanwhile, we propose the intraimage and interimage attentions as the bilateral normalization on the Sinkhorn distance to suppress the object mismatch caused by background clutter. Second, local feature maps are enhanced with the multiscale pooling strategy, making the Sinkhorn distance possible to find a consistent matching scale between images. Experimental results show the effectiveness of the proposed approach, and we achieve the state-of-the-art on three few-shot benchmarks.

Spindle Detection Based on Elastic Time Window and Spatial Pyramid Pooling.

Sleep spindles have emerged as valuable biomarkers for assessing cognitive abilities and related disorders, underscoring the importance of their detection in clinical research. However, template matching-based algorithms using fixed templates may not be able to fully adapt to spindles of different durations. Moreover, inspired by the multiscale feature extraction of images, the use of multiscale feature extraction methods can be used to better adapt to spindles of different frequencies and durations. Therefore, this study proposes a novel automatic spindle detection algorithm based on elastic time windows and spatial pyramid pooling (SPP) for extracting multiscale features. The algorithm utilizes elastic time windows to segment electroencephalogram (EEG) signals, enabling the extraction of features across multiple scales. This approach accommodates significant variations in spindle duration and polarization positioning during different EEG epochs. Additionally, spatial pyramid pooling is integrated into a depthwise separable convolutional (DSC) network to perform multiscale pooling on the segmented spindle signal features at different scales. Compared with existing template matching algorithms, this algorithm's spindle wave polarization positioning is more consistent with the real situation. Experimental results conducted on the public dataset DREAMS show that the average accuracy of this algorithm reaches 95.75%, with an average negative predictive value (NPV) of 96.55%, indicating its advanced performance. The effectiveness of each module was verified through thorough ablation experiments. More importantly, the algorithm shows strong robustness when faced with changes in different experimental subjects. This feature makes the algorithm more accurate at identifying sleep spindles and is expected to help experts automatically detect spindles in sleep EEG signals, reduce the workload and time of manual detection, and improve efficiency.

RVS-FDSC: A retinal vessel segmentation method with four-directional strip convolution to enhance feature extraction

Retinal vessel segmentation is important to assist ophthalmologists in the diagnosis and treatment of ophthalmic diseases. However, traditional convolutional neural networks (CNNs) are constrained by using square convolution kernels for retinal vessel feature extraction, thereby overlooking the structural characteristics of retinal vessels, which to some extent limits model performance. Consequently, we propose a retinal vessel segmentation method with four-directional strip convolution to enhance feature extraction (RVS-FDSC). Firstly, due to the shape structure of retinal vessels which have branching paths and appear locally as strips, we have constructed a residual strip convolution (RSC) module by using strip convolutions in four different directions: horizontal, vertical, antidiagonal, and main diagonal. These four directions can roughly simulate the branching directions of vessels, making it possible to better extract vessel features. Furthermore, to more comprehensively understand and utilize both local details and global context, RVS-FDSC introduces a multi-scale pooling feature fusion (MSPF2) module. Finally, to better focus on important features and reduce the attention to unimportant or redundant information, RVS-FDSC introduces a residual parallel dual attention (RPDA) module. We conduct experiments on three retinal vessel segmentation datasets: DRIVE, CHASE-DB1, and STARE. RVS-FDSC achieves excellent results on these three public datasets, with AUC of 0.9856, 0.9867, and 0.9808, Acc of 0.9692, 0.9743, and 0.9718, and Spe of 0.9878, 0.9856 and 0.9882, respectively. These results indicate the superior performance of RVS-FDSC compared to other state-of-the-art methods.

Multi-scale pooling learning for camouflaged instance segmentation

Multi-scale pooling learning for camouflaged instance segmentation

A Reconfigurable Framework for Neural Network Based Video In-Loop Filtering

This article proposes a reconfigurable framework for neural network based video in-loop filtering to guide large-scale models for content-aware processing. Specifically, the backbone neural model is decomposed into several convolutional groups and the encoder systematically traverses all candidate configurations combined by these groups to find the best one. The selected configuration index is then encapsulated as side information and passed to the decoder, enabling dynamic model reconfiguration during the decoding stage. The preceding reconfiguration process is only deployed in the inference stage on top of a pre-trained backbone model. Furthermore, we devise WMSPFormer , a wavelet multi-scale Poolformer, as the backbone network structure. WMSPFormer utilizes a wavelet-based multi-scale structure to losslessly decompose the input into multiple scales for spatial-spectral features aggregation. Moreover, it uses multi-scale pooling operations ( MSPoolformer ) instead of complicated matrix calculations to substitute the attention process. We also extend MSPoolformer to a large-scale version using more parameters, referred to as MSPoolformerExt . Extensive experiments demonstrate that the proposed WMSPFormer+Reconfig. and WMSPFormerExt+Reconfig. achieve a remarkable 7.13% and 7.92% BD-Rate reduction over the anchor H.266/VVC, outperforming most existing methods evaluated under the same training and testing conditions. In addition, the low-complexity nature of the WMSPFormer series makes it attractive for practical applications.

Classification of Typical Static Objects in Road Scenes Based on LO-Net

Mobile LiDAR technology is a powerful tool that accurately captures spatial information about typical static objects in road scenes. However, the precise extraction and classification of these objects pose persistent technical challenges. In this paper, we employ a deep learning approach to tackle the point cloud classification problem. Despite the popularity of the PointNet++ network for direct point cloud processing, it encounters issues related to insufficient feature learning and low accuracy. To address these limitations, we introduce a novel layer-wise optimization network, LO-Net. Initially, LO-Net utilizes the set abstraction module from PointNet++ to extract initial local features. It further enhances these features through the edge convolution capabilities of GraphConv and optimizes them using the “Unite_module” for semantic enhancement. Finally, it employs a point cloud spatial pyramid joint pooling module, developed by the authors, for the multiscale pooling of final low-level local features. Combining three layers of local features, LO-Net sends them to the fully connected layer for accurate point cloud classification. Considering real-world scenarios, road scene data often consist of incomplete point cloud data due to factors such as occlusion. In contrast, models in public datasets are typically more complete but may not accurately reflect real-world conditions. To bridge this gap, we transformed road point cloud data collected by mobile LiDAR into a dataset suitable for network training. This dataset encompasses nine common road scene features; hence, we named it the Road9 dataset and conducted classification research based on this dataset. The experimental analysis demonstrates that the proposed algorithm model yielded favorable results on the public datasets ModelNet40, ModelNet10, and the Sydney Urban Objects Dataset, achieving accuracies of 91.2%, 94.2%, and 79.5%, respectively. On the custom road scene dataset, Road9, the algorithm model proposed in this paper demonstrated outstanding classification performance, achieving a classification accuracy of 98.5%.

Design of a novel panoptic segmentation using multi-scale pooling model for tooth segmentation

Design of a novel panoptic segmentation using multi-scale pooling model for tooth segmentation

X-ray weld defect detection method based on dense connection and multi-scale pooling

X-ray weld defect detection method based on dense connection and multi-scale pooling

A CNN-Based Multi-Scale Pooling Strategy for Acoustic Scene Classification

A CNN-Based Multi-Scale Pooling Strategy for Acoustic Scene Classification

MSFANet: Multi-Scale Strip Feature Attention Network for Cloud and Cloud Shadow Segmentation

Cloud and cloud shadow segmentation is one of the most critical challenges in remote sensing image processing. Because of susceptibility to factors such as disturbance from terrain features and noise, as well as a poor capacity to generalize, conventional deep learning networks, when directly used to cloud and cloud shade detection and division, have a tendency to lose fine features and spatial data, leading to coarse segmentation of cloud and cloud shadow borders, false detections, and omissions of targets. To address the aforementioned issues, a multi-scale strip feature attention network (MSFANet) is proposed. This approach uses Resnet18 as the backbone for obtaining semantic data at multiple levels. It incorporates a particular attention module that we name the deep-layer multi-scale pooling attention module (DMPA), aimed at extracting multi-scale contextual semantic data, deep channel feature information, and deep spatial feature information. Furthermore, a skip connection module named the boundary detail feature perception module (BDFP) is introduced to promote information interaction and fusion between adjacent layers of the backbone network. This module performs feature exploration on both the height and width dimensions of the characteristic pattern to enhance the recovery of boundary detail intelligence of the detection targets. Finally, during the decoding phase, a self-attention module named the cross-layer self-attention feature fusion module (CSFF) is employed to direct the aggregation of deeplayer semantic feature and shallow detail feature. This approach facilitates the extraction of feature information to the maximum extent while conducting image restoration. The experimental outcomes unequivocally prove the efficacy of our network in effectively addressing complex cloud-covered scenes, showcasing good performance across the cloud and cloud shadow datasets, the HRC_WHU dataset, and the SPARCS dataset. Our model outperforms existing methods in terms of segmentation accuracy, underscoring its paramount importance in the field of cloud recognition research.

A Decoupled Semantic–Detail Learning Network for Remote Sensing Object Detection in Complex Backgrounds

Detecting multi-scale objects in complex backgrounds is a crucial challenge in remote sensing. The main challenge is that the localization and identification of objects in complex backgrounds can be inaccurate. To address this issue, a decoupled semantic–detail learning network (DSDL-Net) was proposed. Our proposed approach comprises two components. Firstly, we introduce a multi-receptive field feature fusion and detail mining (MRF-DM) module, which learns higher semantic-level representations by fusing multi-scale receptive fields. Subsequently, it uses multi-scale pooling to preserve detail texture information at different scales. Secondly, we present an adaptive cross-level semantic–detail fusion (CSDF) network that leverages a feature pyramid with fusion between detailed features extracted from the backbone network and high-level semantic features obtained from the topmost layer of the pyramid. The fusion is accomplished through two rounds of parallel global–local contextual feature extraction, with shared learning for global context information between the two rounds. Furthermore, to effectively enhance fine-grained texture features conducive to object localization and features conducive to object semantic recognition, we adopt and improve two enhancement modules with attention mechanisms, making them simpler and more lightweight. Our experimental results demonstrate that our approach outperforms 12 benchmark models on three publicly available remote sensing datasets (DIOR, HRRSD, and RSOD) regarding average precision (AP) at small, medium, and large scales. On the DIOR dataset, our model achieved a 2.19% improvement in mAP@0.5 compared to the baseline model, with a parameter reduction of 14.07%.

GMTS: GNN-based multi-scale transformer siamese network for remote sensing building change detection

ABSTRACT With the remarkable success of change detection (CD) in remote sensing images in the context of deep learning, many convolutional neural network (CNN) based methods have been proposed. In the current research, to obtain a better context modeling method for remote sensing images and to capture more spatiotemporal characteristics, several attention-based methods and transformer (TR)-based methods have been proposed. Recent research has also continued to innovate on TR-based methods, and many new methods have been proposed. Most of them require a huge number of calculation to achieve good results. Therefore, using the TR-based mehtod while maintaining the overhead low is a problem to be solved. Here, we propose a GNN-based multi-scale transformer siamese network for remote sensing image change detection (GMTS) that maintains a low network overhead while effectively modeling context in the spatiotemporal domain. We also design a novel hybrid backbone to extract features. Compared with the current CNN backbone, our backbone network has a lower overhead and achieves better results. Further, we use high/low frequency (HiLo) attention to extract more detailed local features and the multi-scale pooling pyramid transformer (MPPT) module to focus on more global features respectively. Finally, we leverage the context modeling capabilities of TR in the spatiotemporal domain to optimize the extracted features. We have a relatively low number of parameters compared to that required by current TR-based methods and achieve a good effect improvement, which provides a good balance between efficiency and performance.

GL-Segnet: Global-Local representation learning net for medical image segmentation.

Medical image segmentation has long been a compelling and fundamental problem in the realm of neuroscience. This is an extremely challenging task due to the intensely interfering irrelevant background information to segment the target. State-of-the-art methods fail to consider simultaneously addressing both long-range and short-range dependencies, and commonly emphasize the semantic information characterization capability while ignoring the geometric detail information implied in the shallow feature maps resulting in the dropping of crucial features. To tackle the above problem, we propose a Global-Local representation learning net for medical image segmentation, namely GL-Segnet. In the Feature encoder, we utilize the Multi-Scale Convolution (MSC) and Multi-Scale Pooling (MSP) modules to encode the global semantic representation information at the shallow level of the network, and multi-scale feature fusion operations are applied to enrich local geometric detail information in a cross-level manner. Beyond that, we adopt a global semantic feature extraction module to perform filtering of irrelevant background information. In Attention-enhancing Decoder, we use the Attention-based feature decoding module to refine the multi-scale fused feature information, which provides effective cues for attention decoding. We exploit the structural similarity between images and the edge gradient information to propose a hybrid loss to improve the segmentation accuracy of the model. Extensive experiments on medical image segmentation from Glas, ISIC, Brain Tumors and SIIM-ACR demonstrated that our GL-Segnet is superior to existing state-of-art methods in subjective visual performance and objective evaluation.

Trimap-guided feature mining and fusion network for natural image matting

Mining interesting objects with trimap guidance and fusing multi-level features are two important issues for trimap-based matting. For feature mining, existing methods simply feed trimaps to an encoder network and mine features evenly in different regions with a decoding module, which lacks an efficient and effective way to mine interesting objects pointed out by trimaps. For emerging content-based feature fusion, most existing matting methods only focus on local features which lack the guidance of a global feature with strong semantic information related to the interesting object. In this paper, we propose a trimap-guided feature mining and fusion network consisting of our trimap-guided non-background multi-scale pooling (TMP) module and global-local context-aware fusion (GLF) modules. Considering that trimap provides strong semantic guidance, our TMP module focuses effective feature mining on interesting objects under the guidance of trimap without extra parameters. Furthermore, our GLF modules use global semantic information of interesting objects mined by our TMP module to guide an effective global-local context-aware multi-level feature fusion. In addition, we build a common interesting object matting (CIOM) dataset to advance high-quality image matting. Experimental results on the Composition-1k test set, Alphamatting benchmark, and our CIOM test set demonstrate that our method outperforms state-of-the-art approaches. Codes and models will be publicly available at https://github.com/Serge-weihao/TMF-Matting.

Research on abnormal data detection of gas boiler supply based on deep learning network

On the basis of YOLO deep network detection method, a new abnormal data detection method is proposed to meet the needs of gas boiler abnormal data detection. In the feature extraction layer, the SENet structure is embedded between DBL and Pooling. Through compression, excitation and recalibration, the feature extraction of data information is more accurate. After feature extraction layer, multi-scale pooling processing mechanism is introduced to improve the learning efficiency of YOLO network. The first group and the second group of experiments respectively proved that the introduction of SENet structure and multi-scale pooling mechanism improved the feature extraction accuracy of YOLO network and the convergence speed of iteration process. The third group of experimental results show that the detection accuracy of the detection method proposed in this paper is significantly higher than CNN method, RNN method and YOLO method, and it is more suitable for the detection of abnormal data of gas boilers.

Automatic detection of photovoltaic facilities from Sentinel-2 observations by the enhanced U-Net method

With the enactment of supportive government policies and the increasing maturity of solar photovoltaic (PV) technologies, solar PV energy has become the most cost-effective new energy resource worldwide. Geospatial information on existing solar PV power systems is necessary to manage and optimize the deployment of new PV facilities. In this study, we propose a new deep-learning network, named the enhanced U-Net (E-UNET), to detect PV facilities from Sentinel-2 multi-spectral remote sensing data. Our E-UNET features an enhanced encoder–decoder structure that can efficiently extract spectral and spatial features simultaneously by combining a multi-spectral three-dimensional convolution path and a multi-scale pooling block. We compare the performance of the E-UNET with other semantic segmentation deep-learning networks and a pixel-based random forest classifier. The experimental results show that the E-UNET performs better than the other methods. It achieves an overall accuracy, Matthews correlation coefficient, F1, kappa coefficient, and recall of 0.989, 0.862, 0.869, 0.934, and 0.875, respectively. The experimental results also indicate that the E-UNET accurately detects PV facilities from various complex environments with high accuracy in terms of PV integrity and details.

Weakly-Supervised Semantic Feature Refinement Network for MMW Concealed Object Detection

The concealed object detection in millimeter-wave human body images is a challenging task due to the noise and dim-small objects. Exploiting the spatial dependencies to mine the difference between the object and the noise is vital for the discrimination of objects. However, most approaches ignore the context around the object. In this paper, a concealed object detection framework based on structural context is proposed to suppress noise interference and refine localizable semantic features. The framework consists of two subnetworks, structural region-based multi-scale weakly supervised feature refinement and local context-based concealed object detection. The multi-scale weakly supervised feature refinement is constructed to learn position-aware semantics of objects of various sizes while suppressing background noises in structural regions. Specifically, a multi-scale pooling method is proposed to better localize objects of different sizes, and an object-activated region enhancement module is designed to strengthen object semantic representations and suppress the background interference. Moreover, an adaptive local context aggregation module is designed to integrate the local context around the bounding box in the concealed object detection, which improves the discrimination of the model for the dim-small objects. Experimental results on the AMMW and the PMMW datasets demonstrate that the proposed approach improves detection performance with lower false alarm rates.